Kip Sullivan comments on the assessment of quality in the review of HMO plan performance by Robert Miller and Harold Luft:
Back in March of this year, Uwe Reinhardt and I debated whether managed care damaged the quality of medical care. Uwe cited an unpublished literature review by Robert Miller and Harold Luft that Uwe described as "inconclusive," that is, it found managed care and FFS care to be roughly equivalent in quality. I wrote Uwe back and asked, "Do you know if Luft and Miller's new review excluded studies that didn't control for coverage differences? If it did not, why do you give its conclusions any credibility?" Uwe did not answer me.
Now we know the answer to the first of my two questions. The latest Health Affairs (July-August 2002) contains the Miller-Luft literature review Uwe was no doubt talking about. As was the case in Miller-Luft's 1994 and 1997 literature reviews, this one (which covers studies published between 1997 and June 2001) includes studies that failed to control for coverage differences, a failing which will often bias results in favor of managed care plans and rarely bias results against fee-for-service doctors (see the PNHP web site for more details on this argument). This was no oversight; at the end of their article, Miller and Luft make a point of arguing with an article I published in the American Journal of Public Health in 1999 criticizing their methodology. Their reasoning is very strange.
I find this latest review to be deceptive for two other reasons. (1) In places, it leads the reader to think that the studies reviewed compared HMOs to FFS plans, when in fact few studies looked at what most people think of when we hear the phrase "FFS plan" -- indemnity plans that don't use utilization review or other managed care tactics. (2) In many places, the review leads the reader to believe the studies compared HMOs to "non-HMOs," when in fact some of the studies mixed HMOs and PPOs together.
My conclusion: None of Miller and Luft's three reviews are reliable. My review in the Am J Pub Health uses the soundest methodology, and that review concludes HMO care is inferior to FFS care.
Kip Sullivan _______________________
Following is the abstract of the article by Miller and Luft to which Sullivan refers. Also is the comment from that article referring to the prior critique by Kip Sullivan. Please note that this was written prior to Kip Sullivan's comments which appear above and, therefore, could not be responsive to this latest critique.
Health Affairs July/August 2002 HMO Plan Performance Update: An Analysis Of The Literature, 1997-2001 By Robert H. Miller , Harold S. Luft
Abstract
This paper synthesizes results from peer-reviewed literature published from 1997 to mid-2001, on various dimensions of health maintenance organization (HMO)plan performance. Results from seventy-nine studies suggest that both types of plans provide roughly comparable quality of care, while HMOs lower use of hospital and other expensive resources somewhat. At the same time, HMO enrollees report worse results on many measures of access to care and lower levels of satisfaction, compared with non-HMO enrollees. Quality-of-care results in particular are heterogeneous, which suggests that quality is not uniform - that it varies widely among providers, plans (HMO and non-HMO), and geographic areas.
Responding to an earlier critique.
After our last literature review, Kip Sullivan argued that our past findings were biased in favor of HMOs because we did not adjust for the fact that HMOs tend to provide more comprehensive coverage than non-HMO plans do. Adjust for those differences, the argument goes, and HMO findings would be much less favorable. This argument is flawed because it ignores the fact that plans are selected on the basis of a "bundle" of characteristics or attributes, not just one. In general, HMOs tend to have more-comprehensive coverage (that is, fewer price constraints on demand), combined with more nonprice constraints - narrower networks and other nonprice attempts (gatekeepers and prior authorizations) to limit specialist visits and expensive services. Non-HMO plans tend to have less comprehensive coverage (that is, more price constraints on demand) but fewer nonprice constraints. Sullivan's proposal holds constant only one part of the "bundle" coverage, ignoring the other. FFS plans with comprehensive coverage and few nonprice constraints on demand have high utilization, high premiums, and few insured persons. Although the Canadian-style single-payer FFS system has few price restrictions, it too must constrain demand-through overall budgets, long-range resource planning, and queuing.
Also flawed is an additional argument that the apparent comparability of quality is because Medicare, as a result of inadequate risk-adjustment methods, over-pays HMOs, which can therefore offer better coverage (such as outpatient prescription drugs), leading to better outcomes than would have been the case without the overpayment. Implicitly, HMO quality is as good as it appears only because HMOs use more resources than FFS plans use.
There is evidence that Medicare overpaid Medicare HMOs by approximately 6-7 percent, according to two estimates. However, the implication with respect to quality is flawed, since it ignores the fact that many FFS Medicare beneficiaries, employers, or Medicaid pay extra for supplemental insurance coverage and that beneficiaries also incur copayments, deductibles, and other out-of-pocket expenses. These payments add substantially more resources for FFS than for HMO enrollees. As a result, while FFS Medicare beneficiaries may use fewer Medicare-financed dollars (risk-adjusted) than their HMO counterparts use, they almost certainly use more real resources in toto.
In fact, if quality is comparable for HMO and non-HMO plans, then many HMO enrollees who have a choice among plans seem willing to accept lower satisfaction for lower out-of-pocket payments. Based on the findings from the articles reviewed here and in the past, the trade-off overall does not appear to be one of lower quality of care for lower out-of-pocket payments. We note, however, that not all enrollees have a choice of plans, and this may be a source of some of the backlash against managed care.