Evaluation of the Comprehensive Primary Care Initiative: First Annual Report

Mathematica Policy Research, January 2015

In October 2012, the Center for Medicare and Medicaid Innovation of the Centers for Medicare and Medicaid Services (CMS), in a unique collaboration between public and private health care payers, launched the Comprehensive Primary Care (CPC) initiative to improve primary care delivery in seven regions across the United States. …The substantial transformation … is expected to achieve better health care, better health outcomes, and lower costs.

http://www.mathematica-mpr.com/our-publications-and-findings/publications/evaluation-of-the-comprehensive-primary-care-initiative-first-annual-report  p. xiii

Some managed care proposals lack evidence but can be subjected to testing to see if they work. Other managed care proposals lack evidence and cannot be tested because they are too vaguely described. The “patient-centered medical home” (PCMH) is an example of the latter. The Comprehensive Primary Care Initiative, a test of the PCMH “model” being conducted by the Center for Medicare and Medicaid Innovation (CMMI), illustrates the problem. It tests too many vaguely defined variables at once.

First, a word about labels. The CMMI chose the label “comprehensive primary care” (CPC) rather than “medical home” for an insubstantial reason: It wanted to involve multiple insurers in the experiment, and in their view previous “home” experiments used only single insurers. But “CPC” is clearly just the PCMH concept with a different label. CMMI tells us the CPC Initiative “builds on the ‘medical home’ concept,” and that it is authorized by Section 3021 of the Affordable Care Act, which mentions the “patient-centered medical home” but not “comprehensive primary care.”  Not surprisingly, the CPC Initiative is widely seen as a test of the PCMH concept.

Here is how the CMMI defines the CPC experiment:

Over the next four years, the Comprehensive Primary Care Initiative will evaluate whether a core set of comprehensive primary care functions, coupled with payment reform, enhanced data to guide practice improvement, and the meaningful use of health information technology, can achieve better health, better health care, and lower costs through continuous improvement.  (p. 1)

Notice first that four variables are being tested at once:

• “a core set of … care functions” (i.e. the usual “features” attributed to “homes”)
• “payment reform,”
• “enhanced data,” and
• “meaningful use of HIT.”

Multiple components within the thing being tested – the intervention, the drug, etc. – is not necessarily an impediment to testing. Scientists often test entities or methods that contain multiple variables; the “Mediterranean diet” and the Diabetes Prevention Program are examples. What makes the CPC “model” untestable is the vagueness of all four of the variables. Any one of these variables alone would be impossible to test. Bundling all four into a single “model” creates a conceptual mess.

To grasp how messy this “model” is, please click here and view the circular diagram.  Viewer discretion is advised. At first glance you think you’re looking at a mandala from the cover of a Grateful Dead album. Upon closer examination you realize you’re looking at an attempt by your government to explain the “logic” of the CPC Initiative. This mandala bears the title, “Comprehensive Primary Care Initiative Logic Diagram.” (You can find this title if you click here and scroll down to “additional information.”)

The mandala attempts to explain each of the four elements of the CPC “model,” but further explanation only deepens the viewer’s sense of being drawn down a rabbit hole where you hear words coming out of people’s mouths but they’re not making sense. For example, we discover that the first variable – “comprehensive primary care functions” – consists of five “functions.” These functions have labels we have heard before from “home” advocates such as “patient and caregiver engagement” and “coordination of care across the medical neighborhood.” How do we operationalize (reduce to measurable components) “engagement”? How would we know that patient X is “engaged” while patient Y is “unengaged”? How do we operationalize “coordination” and “medical neighborhood,” much less coordination “across” said medical neighborhood?

CMMI’s mandala is no better at explaining the other three elements of the CPC “model.” To take one example for each of those elements:

• “enhanced accountable payment” somehow consists of “strategic use of practice revenue”;
• “continuous improvement driven by data” somehow consists of “culture of improvement”; and
• “use of HIT” includes “continuous improvement of HIT.”

“Accountable payment”? “Strategic use?” “Culture of improvement?” “Improvement of HIT,” continuous or otherwise? How would we recognize these things (sorry, “thing” is the best descriptor I can come up with) if we happened to stumble over one?

CMMI attempted to operationalize the four variables by listing a series of tasks each “practice” must complete. (In the “medical home” rabbit hole, there are no “clinics” anymore. They are all “practices.”) CMMI calls these tasks “milestones.” The “milestones” become more numerous and specific over the course of the four-year demonstration (click here to see all the “milestones”). Here are the nine “milestones” for the first year (2013) as reported by Mathematica in their report on the first-year of the CPC experiment:

• Estimate CPC revenues and develop a plan for their reinvestment in the practice.
• Stratify patients by risk status and provide care management to high-risk patients.
• Ensure 24/7 access to the medical record for the practice’s providers.
• Assess and improve patient experience with care by conducting a patient survey or forming a patient and family advisory council … that meets quarterly.
• Use data to guide care improvement by selecting one quality and one utilization measure on which to focus.
• Improve care coordination in the medical neighborhood by selecting one area for focus.
• Improve patient shared decision-making capacity by selecting one decision aid.
• Participate in the regional learning community.
• Attest to Stage 1 meaningful use. (Table 6.1, p. 82)

Mathematica assures us these tasks not mere busywork. “While the Milestones themselves are not evidence based,” writes Mathematica, “they are rooted in strong conceptual thinking about what activities a practice needs to pursue to achieve comprehensive primary care.” (p. 81) This is classic managed care gibberish. The key phrases are so abstract they could mean anything, and the logic is circular. (The “milestone” requiring use of a decision aid is in fact evidence-based, but it is the only one.)

These milestones are more specific than the mushy labels in the CPC “logic” mandala, but they are still so vague they leave enormous latitude for clinics to react with diverse activities – activities so diverse that the uniformity of behavior necessary to test any milestone is impossible to achieve. Is the “patient and family council” (PFAC) a bona fide council if it consists of one patient and one family member? How would a clinic know when it has “focused” on its “one quality measure” or on “coordination in the medical neighborhood”?

Here, from the Mathematica report, is an example of how the amorphous “milestones” encourage diverse behaviors:

Among the 100 practices that choose to form a PFAC to help assess and improve patient experience with care, all reported having more than one patient on the PFAC. … The most common area of focus – communication – was chosen by 59 percent of practices. (p. 85)

So some PFACs will have more than two patients, some won’t. Some will “focus” on “communication,” whatever that means, and others will “focus” on something else. (Incidentally, do two-patient advisory committees reveal “strong conceptual thinking?”)

The CPC Initiative, like all tests of the “medical home,” attempts to assess the impact of too many vaguely defined variables at once. It is time to terminate experiments testing the fuzzy “home model” and shift the funds to experiments that test more plausible and more testable hypotheses. If the goal of the PCMH movement is simply to strengthen primary care, then let’s funnel more resources into primary care. If the goal is to take better care of chronically ill patients, then let’s conduct research on specific services for specific types of patients and, for those services that are shown to work, let’s invent a billing code for them and pay for them.

Kip Sullivan, J.D., is a member of the board of Minnesota Physicians for a National Health Program. His articles have appeared in The New York Times, The Nation, The New England Journal of Medicine, Health Affairs, the Journal of Health Politics, Policy and Law, and the Los Angeles Times.