Effective Clinical Practice
Effective Clinical Practice, March/April 2002
Christian M. Birkmeyer, MS, Joshua Lee, MD, (Department of Medicine, Dartmouth-Hitchcock Medical Center, Lebanon, New Hampshire), David W. Bates, MD, (Division of General Internal Medicine, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts), John D. Birkmeyer, MD, (Department of Surgery, Dartmouth-Hitchcock Medical Center, Lebanon, New Hampshire), (VA Outcomes Group, VA Medical Center, White River Junction, Vermont)
For author affiliations, current addresses, and contributions, see end of text.
Since the release of the Institute of Medicine's report "To err is human," (1) there has been growing interest in electronic order entry as a tool for reducing medication errors in hospitalized patients. With electronic order entry, clinicians place orders through computer workstations linked to databases containing patient-specific clinical information and error-prevention software. A number of studies have demonstrated the potential of these systems to substantially reduce medication errors. (2-4) As a result, the Leapfrog Group, a large consortium of private and public purchasers, is advocating implementation of electronic order entry at hospitals caring for its more than 30 million employees and beneficiaries. (5) According to their estimates, electronic order entry would prevent over 500,000 serious medication errors in the United States if fully implemented nationwide. (6)
While few dispute the potential for electronic order entry to reduce medication errors, many question whether hospitals can afford it. Electronic order entry requires that hospitals make substantial investments in information technology; implementation also requires a considerable commitment of clinician time and its related costs. Although many point to savings likely to be achieved from reduced errors and increased efficiency, the economic benefits of electronic order entry have not been carefully considered.
In this article, we categorize the costs and savings associated with electronic order entry from the hospital's perspective. Although the data are insufficient to calculate the financial "bottom line" for hospitals, we quantify costs and savings as much as possible.
In this section, we consider both the initial (implementation) and ongoing costs associated with electronic order entry (Table 1). While much of these costs are directly related to the technology itself and computer staff in hospitals' information departments, another important cost to consider is the clinician time required to implement electronic order entry.
A clinical information system (CIS) with reporting capabilities is a prerequisite for electronic order entry, so hospitals without an adequate CIS will need to purchase one. With a CIS in place, implementation of electronic order entry implies several additional system costs, including upgrading existing hardware platforms and networks. The electronic order entry software can be licensed from a commercial vendor or developed locally. Either way, substantial programming is required to link the order entry software to the existing hospital CIS and databases. Hospitals must purchase and maintain sufficient computer workstations for clinician users in the hospital. Finally, hospital information departments must increase the size of their computer support staffs ("help desks") as electronic order entry is implemented.
Implementation of electronic order entry also requires clinicians to invest a substantial amount of time. Clinician leaders must be involved in championing electronic order entry and selecting the system. A larger number of clinicians is required to standardize clinical documentation (what goes in and how) and "rules" for clinical alerts. Clinicians must help develop an ordering catalogue tailored to their hospital, along with diagnosis-specific order sets and protocols. Finally, all clinician users must invest the time required to learn and use the system.
The overall costs of implementing electronic order entry will depend on the size of the hospital and the status of its CIS. The relevance of hospital size is obvious--bigger hospitals will have higher software licensing fees, require more workstations, and have more clinician users who require training and support. In addition, larger hospitals tend to have more specialty-specific programs, which require customized programming. The status of the hospital's existing CIS is also extremely important. At hospitals with sophisticated information systems already capable of electronic order entry, simply "turning on" electronic order entry will be relatively inexpensive. In contrast, costs will be substantially higher at hospitals needing to purchase a new CIS or at those with advanced but homegrown information systems (into which electronic order entry software must be integrated).
In collaboration with industry consultants at Cerner Corporation (a major vendor of health care information management systems), we estimated the costs associated with implementing electronic order entry in hospitals that have a CIS with electronic order entry capabilities, those that have a CIS without electronic order entry capabilities, and for those that require a new CIS (Table 2). To reflect the full range of hospital size, we estimated implementation costs at a small hospital (200 beds) and a very large hospital (1000 beds). (Very small hospitals [<200 beds] may lack economies of scale to justify traditional, on-site electronic order entry systems and may do better with remote access systems, which are a viable option.) Estimates were revised after input from clinicians with direct experience in the implementation of electronic order entry. Reflecting the multiple assumptions involved, we provide lower and upper bound estimates for initial and ongoing costs (one for each scenario).
As summarized in Table 3, costs associated with implementing electronic order entry could vary 10-fold across hospitals. At the low end (lowest-cost estimate for small hospitals with CIS with electronic order entry capabilities), implementation would cost $496,000 initially and $174,000 annually thereafter. At the high end (highest-cost estimate for large hospitals requiring a new CIS), costs could be almost $15 million initially and almost $1.5 million annually. Appendix Table 1 provides a more detailed breakdown for scenario 1. Similar data for scenario 2 and scenario 3 follow. Still finer details about assumptions and subcomponent costs for all three scenarios are available at www.leapfroggroup.org.
Savings from Fewer Medication Errors and Adverse Drug Events
Medication errors--errors in the process of ordering, dispensing, or administering a medication--are common in hospitalized patients. Based on the rate from one study by Bates and colleagues (3) (142 errors per 1000 patient days), approximately half of all inpatients are subject to a medication error. Fortunately, most medication errors are recognized before incorrect treatments are received by patients and, of those that are not, most errors pose little threat to patient safety. Thus, less than 5% of all medication errors result in actual adverse drug events. (7)
Costs of Adverse Drug Events
Medication errors resulting in adverse drug events have obvious costs. Three studies have assessed the costs associated with adverse drug events. (8-10) In one large case-control study at LDS Hospital in Salt Lake City, Classen and coworkers compared 1580 patients with adverse drug events to over 20,000 controls on the basis of diagnosis-related group, age, sex, illness acuity, and year of admission. They found patients with adverse drug events averaged nearly $5000 more in hospital costs, $2013 of which was attributable to the adverse drug event. (8) Evans and colleagues, using similar methods, estimated the attributable cost of an adverse drug event to be $1939. (9) In the third study, at the Brigham and Women's Hospital in Boston, the average attributable cost for adverse drug events was $2595. (10) In two studies, adverse drug events decreased with the introduction of electronic order entry (although the reduction was not statistically significant in either study). In the first study, error-related adverse drug events decreased by only 17% after implementation of electronic order entry (P = 0.3). (2) In the second study (based on a smaller sample but a more advanced electronic order entry system), these rates fell by 62%. (3) However, both studies lacked statistical power because adverse events related to drug errors are relatively infrequent. Evans and coworkers (11) reported a marked, statistically significant reduction in (overall) adverse drug event rates after implementation of electronic order entry, but this study was constrained to a relatively narrow clinical context (antibiotics and antiinfective therapy in intensive care unit patients).
Costs of Medication Errors
Even medication errors that do not harm patients have cost implications, mainly related to personnel time and inefficiency. For example, the ward nurse recognizes that his patient is allergic to penicillin just before administering a dose of intravenous ampicillin. The antibiotic must be transported back to the inpatient pharmacy for reshelving (or discarded if premixed). The nurse must also track down the responsible physician, who must order another antibiotic from the pharmacy. According to one estimate, such errors result in about 20 minutes of lost work on average per error, largely by nurses and pharmacists (personal communication: William Tierney, MD).
Figure 1 shows that the number of serious medication errors declined significantly after implementation of electronic order entry. In the one study that examined rates of all medication errors (data not shown), the total number of medication errors fell from 142 to 26.6 per 1000 patient-days after implementation of electronic order entry--an 81% reduction. (3)
Synthesizing the existing literature and incorporating judgments from experts in the field where necessary, we estimated savings that might be accrued from fewer medication errors and adverse drug events. As shown in Table 4, overall savings are directly proportional to hospital size (i.e., number of admissions), ranging from $184,000 annually for a 200-bed hospital to $919,000 for a 1000-bed hospital. Potential savings from fewer adverse drug events outweigh those from reduction of less serious medication errors.
Other Sources of Potential Savings
While reductions in errors and adverse drug events may be most the important aspect of an electronic order entry system with regard to patient safety, other sources of savings may be more significant financially. Some of these savings include 1) medication substitution (ordering of less expensive but equally efficacious drugs), 2) reduced laboratory testing and imaging, 3) the ability of electronic order entry to increase the proportion of patients on cost-effective, disease-specific critical pathways, and 4) potential gains in clinical efficiency. In one randomized, controlled trial that evaluated a system combining these features, Tierney and colleagues (12) found that length of stay was 0.9 days shorter and charges were 12.7% lower than those of the control group. Although we could not calculate savings because of limited data, we summarize what is known about these other sources of potential savings.
Savings from Medication Substitution
Electronic order entry systems offer more cost-effective alternatives to physicians at the time of ordering. When physicians order a specific medication, for example, they may be prompted to consider a less expensive but equally efficacious alternative within the same therapeutic class. At the Brigham and Women's Hospital, physicians ordering H2 blockers were prompted to order either oral nizatidine or intravenous ranitidine, which the hospital obtains at considerably lower costs than other H2 blockers. (4) After the computerized guideline was implemented, use of the recommended drug increased from 15.6% to 81.3% for all H2 blocker orders. It is likely that substantial savings would also be realized for expensive antibiotics, cardiovascular medications, and numerous other medication classes. In addition to encouraging medication substitution, electronic order entry systems help ensure proper dosing, such as decreased dosing (or longer dosing intervals) for patients with impaired renal function.
Savings from Fewer Unnecessary Tests
Electronic order entry may reduce the costs of unnecessary laboratory and imaging tests by showing physicians charges associated with individual tests at the time they are ordered. In a randomized, controlled trial in the outpatient setting, charge display reduced testing by 14%. (13) In addition, electronic order entry helps eliminate redundant tests. For example, a physician ordering a complete blood count would be reminded that the patient had one 4 hours ago. In one study, Bates and colleagues found that electronic order entry identified 437 redundant laboratory tests among 5700 admissions (8%). Physicians cancelled 69% of these tests after their redundancy was flagged by the electronic order entry system. (14) Additional savings can be achieved for unnecessary radiologic and other studies (e.g., averting the routine preoperative chest x-ray and electrocardiogram in a patient who had these tests the month before).
Greater Use of Clinical Pathways
Electronic order entry may produce substantial savings through increased use of clinical pathways. Electronic order entry applications facilitate the creation of disease-specific (or procedure-specific) care protocols. By reducing unwanted variation in processes of care and aligning clinical practice with evidence-based guidelines, such pathways may improve patient outcomes while reducing resource use and lengths of stay. For example, in one study of hospitalized patients with infectious diseases, use of electronic order entry was associated with significantly lower total hospital costs (average reduction of $9000) and hospital stays (reduction of 2.9 days). (11)
Potential Gains in Clinician Efficiency
Although difficult to quantify, electronic order entry, as part of a well-integrated CIS, produces clinical efficiency gains on several levels. Electronic order entry reduces time spent by nurses, clerical staff, and pharmacists in processing individual orders. By its ability to create templates for entire order sets (e.g., admission or postoperative orders), electronic order entry also improves physician efficiency.
Although based on the best information currently available, our cost estimates have several limitations. Our estimates required numerous assumptions and, as reflected by the large gaps between our lowest-cost and highest-cost estimates, are very imprecise. In addition, we relied heavily on input from industry experts, not direct measurements at hospitals that have implemented electronic order entry. Although clinical experts reviewed our estimates for face validity, industry and electronic order entry enthusiasts may tend to underestimate the true costs of system implementation.
Next, our estimated savings from fewer errors and adverse drug events relies on effectiveness data from a single institution (Brigham and Women's Hospital). Further studies are being done; however, it is uncertain whether they will show similar reductions in error. Although other sources of potential savings may be larger in magnitude, there is little empirical information about the savings likely to be achieved through lower medication costs, fewer tests, greater use of clinical pathways, and general gains in clinician efficiency.
We do not consider quantitatively how costs and savings associated with electronic order entry are distributed between hospitals and payers. We assume that electronic order entry costs are initially borne by hospitals but are ultimately passed on (indirectly) to payers and patients. The distribution of savings is more difficult to predict. Some types of savings (e.g., clinical efficiency gains) may accrue primarily to the hospital. Other types of savings (e.g., reduced errors, use of less expensive medications, fewer tests), however, will be divided between hospitals and payers. Initially, this will primarily occur according to hospital payer mix; over time, however, savings for this or any intervention that allows health services to be delivered more efficiently will likely be passed on to payers and beneficiaries.
Finally, we did not assess the indirect economic benefits associated with better quality care (e.g., reduced disability or lives saved). Under broad assumptions about the number of deaths occurring as a result of medication errors, the Leapfrog Group estimated that electronic order entry could save $549 million annually if fully implemented in the United States. (15) However, large studies that directly assess the effectiveness of electronic order entry in reducing morbidity and mortality are needed.
Although we have attempted to categorize the costs and savings likely to be associated with electronic order entry, few studies have measured the financial effects of electronic order entry in practice. The strongest study of economic benefit is from Wishard Memorial Hospital where investigators found a 12.7% reduction in charges. (12) Also, informatics experts have estimated net savings to be $5 to 10 million annually at the Brigham and Women's Hospital and $8 million annually at Sarasota Memorial Hospital after implementation of electronic order entry. (2, 16) However, limited information was provided about assumptions used to estimate the net savings in the latter two studies, making it is hard to critique the validity of their estimates.
Although costs will likely vary across hospitals, there is no doubt that implementing electronic order entry will be very expensive at many centers. Savings, although even more difficult to quantify, may also be very large. More real-world studies that describe the financial effects of electronic order entry will be required to predict the "bottom line" for hospitals considering whether to implement these systems. In the meantime, the primary motivation for electronic order entry should be improving quality of care, not reducing health care costs.
1. Kohn LT, Corrigan J, Donaldson MS, Institute of Medicine (U.S.) Committee on Quality of Health Care in America. To Err Is Human: Building a Safer Health System. Washington, DC: National Academy Pr; 1999.
2. Bates DW, Leape LL, Cullen DJ, et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA. 1998;280:1311-6.
3. Bates DW, Teich JM, Lee J, et al. The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc. 1999;6:313-21.
4. Teich JM, Merchia PR, Schmiz JL, Kuperman GJ, Spurr CD, Bates DW. Effects of computerized physician order entry on prescribing practices. Arch Intern Med. 2000;160:2741-7.
5. Milstein A, Galvin RS, Delbanco SF, Salber P, Buck CR Jr. Improving the safety of health care: The Leapfrog Initiative. Eff Clin Pract. 2000;3:313-6.
6. Birkmeyer JD, Birkmeyer CM, Wennberg DE, Young M. Leapfrog patient safety standards: the potential benefits of universal adoption. Washington, DC: The Leapfrog Group; 2000.
7. Bates DW, Boyle DL, Vander Vliet MB, Schneider J, Leape L. Relationship between medication errors and adverse drug events. J Gen Intern Med. 1995;10:199-205.
8. Classen DC, Pestotnik SL, Evans RS, Lloyd JF, Burke JP. Adverse drug events in hospitalized patients. Excess length of stay, extra costs, and attributable mortality. JAMA. 1997;277:301-6.
9. Evans RS, Classen DC, Stevens LE, et al. Using a hospital information system to assess the effects of adverse drug events. Proc AMIA Annu Fall Symp. 1993;161-5.
10. Bates DW, Spell N, Cullen DJ, et al. The costs of adverse drug events in hospitalized patients. Adverse Drug Events Prevention Study Group. JAMA. 1997;277:307-11.
11. Evans RS, Pestotnik SL, Classen DC, et al. A computer-assisted management program for antibiotics and other anti-infective agents. N Engl J Med. 1998;338:232-8.
12. Tierney WM, Miller ME, Overhage JM, McDonald CJ. Physician inpatient order writing on microcomputer workstations. Effects on resource utilization. JAMA. 1993;269:379-83.
13. Tierney WM, Miller ME, McDonald CJ. The effect on test ordering of informing physicians of the charges for outpatient diagnostic tests. N Engl J Med. 1990;322:1499-1504.
14. Bates DW, Kuperman GJ, Rittenberg E, et al. A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests. Am J Med. 1999;106:144-50.
15. Birkmeyer JD, Birkmeyer CM, Skinner JS. Economic Implications of the Leapfrog Safety Standards. Washington, DC: The Leapfrog Group; 2001.
16. http://www.eclipsnet.com/External/Company/99-10-11_SMHcost_savings.htm [NOTE: This link has expired.]
The views expressed herein do not necessarily represent the views of the Department of Veterans Affairs or the United States Government.
This analysis was supported in part by the Business Roundtable, sponsor of the Leapfrog Group. Dr. John D. Birkmeyer is supported by a Career Development Award from the VA Health Services Research & Development program.
Christian M. Birkmeyer, MS, Department of Surgery, Dartmouth-Hitchcock Medical Center, 1 Medical Center Drive, Lebanon, NH; telephone: 603-650-7400; fax: 603-650-6061; e-mail: firstname.lastname@example.org.