If You Can’t Measure Performance, Can You Improve It?
By Robert A. Berenson, MD
JAMA Forum, January 13, 2016
“If you can’t measure it, you can’t manage it” is an often-quoted admonition commonly attributed to the late W. Edwards Deming, a leader in the field of quality improvement. Some well-respected health policy experts have adopted as a truism a popular variation of the Deming quote — “if something cannot be measured, it cannot be improved” — and point to the recent enactment of the Medicare Access and CHIP Reauthorization Act of 2015 (MACRA) as a confirmation of “the broadening societal embrace” of this concept.
The problem is that Deming actually wrote, “It is wrong to suppose that if you can’t measure it, you can’t manage it — a costly myth” — the exact opposite. Deming consistently cautioned against requiring measurement to guide management decisions, observing that the most important data needed to manage often are unknown and unknowable.
Many Routes to Improvement
The requirement for measurement as essential to management and improvement is a fallacy, not a self-evident truth and not supported by Deming, other management experts, or common sense. There are many routes to improvement, such as doing things better based on experience, example, as well as evidence from research studies.
Comparative public performance using meaningful and accurate measures has led to quality improvements, as clinicians and hospitals reflect on their own comparative performance and seek to improve their public standing. Examples include improved hospital care for patients experiencing heart attacks and improved renal dialysis. In most clinical areas, however, we lack readily available measures to use as valid benchmarks to assess performance.
Not deterred, however, last year a rarely bipartisan Congress passed the MACRA legislation. Its core element was repealing the unsustainable sustainable growth rate mechanism threatening huge payment cuts to physicians caring for Medicare patients. The law called for development of “value based” payment approaches that would pay for quality and cost outcomes, rather than just for the myriad services physicians provide or order, whether or not the services are needed or well performed. “Paying for value, not volume” has become the slogan du jour, itself assuming a mostly unchallenged position in health policy circles.
Now comes the hard part: actually achieving greater value, rather than fashioning an increasingly complex, intrusive, and likely doomed attempt to measure value.
After the MACRA’s Merit-Based Incentive Payment System (MIPS) is fully phased in early in the next decade, a physician caring for Medicare patients under MIPS stands to lose up to 9% of their Medicare payments or conceivably gain 27%, based on their performance on measures of quality, their use of health care resources, the extent to which they have implemented electronic health records, and their participation in quality improvement activities.
But performance on a few, random and often unreliable measures of performance can provide a highly misleading snapshot of any physician’s value.
A Bad Idea?
Practical challenges aside, pay for performance for health professionals may simply be a bad idea. Behavioral economists find that tangible rewards can undermine motivation for tasks that are intrinsically interesting or rewarding. Furthermore, such rewards have their strongest negative impact when they are perceived as being large, controlling, contingent on very specific task performance, or associated with surveillance, deadlines, or threats, as with MIPS.
Another major problem with the current preoccupation with measurement as the central route to improvement is the assumption that if a quality problem isn’t being measured, it basically doesn’t exist. A prime example is diagnosis errors. Recently, an Institute of Medicine (IOM) committee, on which I was a member, issued Improving Diagnosis in Health Care, documenting serious errors of diagnosis in 5% to 15% of interactions with the health care system.
As the report emphasizes, we cannot now measure the accuracy of diagnoses, which means MIPS scores will not include performance on this core physician competency. Still, the IOM committee proposed numerous improvement strategies. These include development of immediate feedback programs to erring clinicians from patients and other health professionals when a serious misdiagnosis occurs (making errors memorable if not measureable), greater attention in medical education to the cognitive bias that commonly clouds clinicians’ judgment, improved systems to ensure that abnormal test results are promptly communicated to patients and diagnostic team members, and giving patients direct access to their medical records so they can introduce relevant, missing information and correct the misinformation that is common in clinical records.
These and other IOM recommendations represent better practices that might dramatically improve diagnostic accuracy, relying not on performance measures but on adopting better work processes and focused education. Measures would help, but substantial progress can be made regardless.
The overarching concern is that under MIPS and similar programs, physicians will focus on the money while their intrinsic motivation to make accurate, timely diagnoses as a core responsibility will be crowded out. If so, the worthwhile recommendations in the IOM report will likely sit on the shelf, gathering dust, thanks to the misguided supposition that “if you can’t measure it, you can’t manage it.”
CMS chief vows to replace meaningful use with better policy
AMA Wire, January 13, 2016
Centers for Medicare & Medicaid Services (CMS) Acting Administrator Andy Slavitt on Monday said that the agency is changing its culture to focus more on listening to physician needs and giving them the freedom they need to keep patients at the center of the practice of medicine.
Referring to execution of the electronic health record (EHR) meaningful use program, Slavitt noted that the agency’s previous regulatory approach created difficulties. “When in doubt, I think, do less and figure it out.”
“The meaningful use program as it has existed will now be effectively over and replaced with something better,” Slavitt said.
In its place will be the new Merit-Based Incentive Payment System (MIPS), called for in the Medicare Access and CHIP Reauthorization Act of 2015, which is intended to sunset the three existing reporting programs and streamline them into a single program.
“The stakes are high for this program,” Slavitt said. “As any physician will tell you, physician burden and frustration levels are real. Programs designed to improve often distract. Done poorly, measures are divorced from how physicians practice and add to the cynicism that the people who build these programs just don’t get it.”
As several prior Quote of the Day messages warned, MACRA’s Merit-Based Incentive Payment System (MIPS) is a horrendous trade-off for getting rid of the flawed SGR payment system. At least for the next decade, we are going to have to live with a system which supposedly will reward or penalize physicians based on measured performance when “the most important data needed to manage often are unknown and unknowable.”
Robert Berenson writes, “a few, random and often unreliable measures of performance can provide a highly misleading snapshot of any physician’s value.” Further, under MIPS, “physicians will focus on the money while their intrinsic motivation to make accurate, timely diagnoses as a core responsibility will be crowded out.”
CMS is responding to the great dissatisfaction with the administrative burden of the “meaningful use” program for electronic health records, which would have been the source of many of these performance measurements. But what is their response? Acting CMS Administrator Andy Slavitt says, “The meaningful use program as it has existed will now be effectively over and replaced with something better” – MIPS! He says, “Done poorly, measures are divorced from how physicians practice and add to the cynicism that the people who build these programs just don’t get it.”
Well, yes. They just don’t get it.