By Danny McCormick, David Bor, Stephanie Woolhandler, and David Himmelstein
Health Affairs Blog, March 12th, 2012
Our recent Health Affairs article linking increased test ordering to electronic access to results has elicited heated responses, including a blog post by Farzad Mostashari, National Coordinator for Health IT. Some of the assertions in his blog post are mistaken. Some take us to task for claims we never made, or for studying only some of the myriad issues relevant to medical computing. And many reflect wishful thinking regarding health IT; an acceptance of deeply flawed evidence of its benefit, and skepticism about solid data that leads to unwelcome conclusions.
Dr. Mostashariâs use of a government website to critique our paper, will, we hope, open a fruitful dialogue. We trust that in the interest of fairness he will also post our response on his agencyâs site.
Our study analyzed government survey data on a nationally representative sample of 28,741 patient visits to 1187 office-based physicians. We found that electronic access to computerized imaging results (either the report or the actual image) was associated with a 40% -70% increase in imaging tests, including sharp increases in expensive tests like MRIs and CT scans; the findings for blood tests were similar. Although the survey did not collect data on payments for the tests, itâs hard to imagine how a 40% to 70% increase in testing could fail to increase imaging costs.
Dr. Mostashariâs statement that âreducing test orders is not the way that health IT is meant to reduce costsâ is surprising, and contradicts statements by his predecessor as National Coordinator that electronic access to a previous CT scan helped him to avoid ordering a duplicate and âsaved a whole bunch of money.â A Rand study, widely cited by health IT advocates including President Obama, estimated that health IT would save $6.6 billion annually on outpatient imaging and lab testing. Another frequently quoted estimate of HIT-based savings projected annual cost reductions of $8.3 billion on imaging and $8.1 billion on lab testing.
We focused on electronic access to results because the common understanding of how health IT might decrease test ordering is that it would facilitate retrieval of previous results, avoiding duplicate tests. Indeed, itâs clear from the extensive press coverage that our study was seen as contravening this âconventional wisdomâ.
Nonetheless, Dr. Mostashari criticizes us for analyzing the impact of physiciansâ electronic access to imaging and test results, but not other aspects of electronic health record (EHR) use. We did, however, analyze the relationship of EHRs to test ordering in a subsidiary analysis. While physicians use of a full EHR was associated with a 19% increase in image ordering, as we noted in the paper this finding was not statistically significant. While we cautiously (and properly) interpreted this as a ânullâ finding, these data are inconsistent with Mostashariâs optimistic view that use of a full EHR reduces costs.
He asserts that our 2008 data are passe, and that health IT meeting todayâs âmeaningful useâ criteria definitely saves money. The data we analyzed were the latest available data when we initiated the study. While the proportion of outpatient physicians utilizing health IT has grown since 2008, we are unaware of any âgame changingâ health IT developments in the past four years that are would produce substantially different results if the study were repeated today. The EHR vendors that dominated the market in 2008 remain, by and large, todayâs market leaders, and their products have undergone mostly modest tweaks. Mostashariâs contention that 2012 EHRs â incorporating decision support and electronic information exchange â save money in ways not possible in 2008 should be tested through additional research but remains merely a hypothesis. We hope that some day his predicted savings can be achieved.
Dr. Mostashari offers his own explanation for our findings, suggesting that doctors who are inclined to order more tests are also inclined to purchase health IT for viewing test results electronically rather than on paper. He offers no evidence for this assertion and ignores the fact that we explored (and rejected) this explanation by analyzing subgroups of doctors who are unlikely to be the decision maker for IT purchases â e.g. employed physicians, those working in an HMO setting etc. In other words, electronic access to results predicted more test ordering whether or not the ordering physician was responsible for health IT purchases.
He incorrectly states that our analysis did not take into account patientsâ severity of illness, physiciansâ level of training, and the nature of physiciansâ financial arrangements. In fact, we reported subsidiary multivariate analyses that included several serious diagnoses; all of our models included physician specialty (which we specified in several different ways); and all models included adjustment for an extensive list of indicators of financial arrangements (e.g. whether the physician owned the practice or was an employee; the type of office; whether the practice was owned by a hospital; whether the physician was a solo practitioner; whether the physicianâs compensation was based, in part or whole on âprofilingâ; and whether the practice was predominantly prepaid). We also performed a series of subsidiary analyses that explored whether physicians with a proclivity to âself referâ patients for imaging tests accounted for our finding; they didnât.
Dr. Mostashari criticizes us for failing to assess whether health IT improved the quality or appropriateness of care. Of course, these were not the topic of our research. Those are different studies for a different time. However, we would note that other large-scale studies have found no, or trivial quality improvements associated with HIT outside of a few flagship institutions4-6.
Dr. Mostashariâs strongest claim is that observational studies like ours (and most other health policy studies, including some by Dr. Mostashari himself) cannot prove causation. This is surely true. As long time teachers of evidence based medicine we took care to couch our conclusions in cautious terms, stating only that âComputerization, whatever its other benefits, remains unproven as a cost control strategy.â
But Dr. Motashari is less cautious, asserting that the case for HIT is closed. The paper he cites to buttress this claim (authored by members of his own agency) culled studies reporting any impact of HIT on virtually any aspect of care, and accepted authorsâ claims of benefit without regard to study quality or statistical niceties. Thus, a focus groupâs impressions of benefit are accorded the same weight as nationwide studies of Medicare data showing virtually no impact of computerization on quality measures. Reports of a reduction from 70% to 38 % in âmissed bi
lling opportunitiesâ or a $7,000 reduction in office supply costs are among the 92% of studies judged âpositiveâ. While the literature review he cites is interesting, nothing in it contradicts our findings.
Dr. Mostashari is also correct in reiterating that randomized trials are the best way to assess health IT. In fact, no randomized trial has ever been published that examines patientsâ outcomes or costs associated with off-the-shelf health IT systems that dominate the U.S. market. No drug or new medical device could pass FDA review based on such thin evidence as we have on health IT. Yet his agency is disbursing $19 billion in federal funds to stimulate the adoption of this inadequately evaluated technology. Dr. Mostashari is perhaps the only person in our nation who commands the resources needed to mount a well done randomized controlled trial to fairly assess the impact of health IT, and the comparative efficacy of the various EHR options.
Finally, Dr. Mostashariâs unbridled faith in technology is mirrored by his belief that ACOs are the next panacea for health costs and quality. That health policy flavor-of-the-month also remains wholly unproven.