Effective Clinical Practice
Effective Clinical Practice, March/April 2001
Donna L. Washington, Paul G. Shekelle, Carl D. Stevens
For author affiliations, current addresses, and contributions, see end of text.
Context. Many ambulatory care facilities do not have resources to provide same-day care for all walk-in patients. Yet, there are few guidelines that identify patients for whom care can be safely deferred.
Objective. To describe the development and implementation of deferred-care guidelines for adults with musculoskeletal complaints.
Design. Consensus process and field test.
Guideline Development. After an eight-member multidisciplinary physician panel identified critical factors that necessitate same-day care, we created 34 clinical scenarios to consider for deferred care. In 22 scenarios, the panel members agreed that deferred care was safe. These were formatted into screening guidelines for back, neck, isolated extremity, and generalized muscle pain.
Implementation. In reliability testing between two nurses reading 40 patient scenarios, interrater agreement for deferred care was nearly perfect (k=0.95). The guidelines were then applied to 448 patients presenting with musculoskeletal complaints to a Veterans Administration ambulatory care triage station. One hundred seven (24%) patients met guidelines for deferred care. Seventy-six patients agreed to have their care deferred, of which 66 kept their return appointment.
Conclusions. Our guidelines suggest that a substantial proportion of patients with musculoskeletal complaints can have their care deferred. Most patients were willing to do so and kept their follow-up appointment. Use of these guidelines could help decompress ambulatory settings with limited resources to provide nonemergency same-day care.
Emergency departments are often overwhelmed by a patient volume that exceeds their capacity to provide same-day care. (1, 2) In a survey of directors of public and teaching hospital emergency departments, 65% reported that overcrowding had a negative impact on the quality of care they provided. (3) When demand for services exceeds the resources available for providing same-day care, clinicians must determine who could safely have their care deferred. However, triage practices in current use are not standardized and have been shown to have poor reproducibility and predictive validity in identifying patients who require emergent care and/or hospitalization. (4-12) Use of explicit clinical guidelines for decision making about same-day versus deferred care could reduce waiting times and improve the quality of care.
Deferred care seems to be acceptable to some patients. For example, Baker and colleagues (13) found that 45% of patients awaiting care at one hospital emergency department would be willing to return the next day for an appointment rather than wait in the emergency department for care. In a study of deferred-care practice, (14) only 1.3% of patients complained about being deferred. Another study found no difference in satisfaction between referred patients and those receiving care in the emergency department. (15) Although most dissatisfied patients cited being deferred as their reason for dissatisfaction, this was balanced by dissatisfaction with long waiting times and the type of treatment received among those cared for in the emergency department.
Musculoskeletal symptoms are a useful area for which to develop ambulatory care screening guidelines, because they represent the most common class of complaints among patients seeking emergency department care. In 1995, musculoskeletal symptoms were the principal reason for 12.3% of emergency department visits nationwide. (16) The medical conditions of patients with such complaints range from mild self-limited conditions, such as muscle strain, which may be cared for in the offices of primary care physicians, to potentially life-threatening ones, such as epidural abscess, in which immediate intervention could prevent serious medical consequences. Our objective was to develop and operationalize explicit clinical guidelines to select adults with musculoskeletal complaints who could be referred for an appointment at a later date. We also determined the proportion of patients to whom the deferred-care guidelines applied in our emergency department.
We used the RAND-UCLA Method (17) to develop the guidelines because it has been shown to have reproducibility and predictive validity in other applications. (18-20) First, we identified from the medical literature common primary musculoskeletal complaints in adults, (16) including low-back pain, neck pain, isolated extremity complaints, and generalized musculoskeletal pain. For each symptom complex, clinical factors that may affect the safety of deferring care were identified after a comprehensive literature review and discussion with internist and emergency physician consultants. We selected for inclusion only clinical factors that are available to a nurse conducting a screening evaluation, such as medical history and limited physical findings.
We convened an eight-member multidisciplinary physician panel to rate the need for same-day care. Participants included three emergency physicians, four internists, and one family practitioner, all board certified in their respective specialties, and representing academic, community, and public hospital settings. Individuals were chosen because of their clinical expertise and included the methodologist for the Agency for Health Care Policy and Research (AHCPR) Low Back Guidelines Panel and Director of the AHCPR Southern California Evidence-Based Practice Center.
Identifying Important Clinical Factors
A flow chart illustrating the guideline development methods is presented in Figure 1. Participants first reviewed the list of clinical factors used in making an urgency assessment. Next, participants identified a subset of critical clinical factors that, if present, would be sufficient by themselves to mandate same-day care (such as new incontinence in low-back pain). The critical factors are evident in the triage form as a series of yes/no questions (Appendix 1, Nontraumatic Low-Back Pain). Any "yes" response was considered a contraindication to deferred care. A full list of the critical factors is available.
The extent and quality of the evidence regarding the predictive value of the critical factors varied widely. For example, the AHCPR low-back pain guidelines and supporting literature were available for the low-back pain ratings; however, only indirect evidence was available for critical factors associated with neck pain. For each chief complaint, clinical factors that were judged to be noncritical—that is, not by themselves mandating same-day care—were combined to create 34 clinical scenarios.
Using a nine-point scale in a two-round process, panelists independently rated the safety of deferring care to the next day in a nonemergency setting for each scenario. The first round of ratings preceded, and the second round followed a day-long meeting in which areas of uncertainty and disagreement were discussed. We defined medical safety to mean that in waiting a day for care, the patient would not suffer serious preventable morbidity. A rating of 9 denoted a scenario in which it was very safe to delay care by 1 day and a rating of 1 identified a scenario for which it was very unsafe to delay care. Uncertain scenarios, in which it was neither clearly safe nor clearly unsafe to delay care by 1 day were rated 5. Panelists could use any of the nine points on the scale.
We specified disagreement among the panelists as being two or more panelists rating a scenario 7 or higher and two or more rating the scenario 3 or lower. (15-16) As shown in Figure 1, only scenarios receiving a median second-round panel rating of 7 or higher, without disagreement, were regarded safe for deferred care (n = 22 scenarios). We then converted the safety classifications and lists of critical clinical factors into symptom-specific standardized data forms for use by a nurse to make decisions on a real-time basis for people who walk into a clinic or an emergency department requesting care. The complete set of all four data forms is available in this web version of ECP.
To measure the interrater agreement in guideline implementation, we created 40 patient scenarios and had 2 nurses independently screen each scenario using the guidelines. We computed agreement for deferred-care decisions using the k statistic. (21)
We also conducted a prospective observational study to determine the proportion of patients who meet deferred-care criteria using these guidelines in the Veterans Administration (VA) Greater Los Angeles Healthcare System, a tertiary care medical center where approximately 30,000 patients request unscheduled care each year. These patients are directed to a common triage area for the emergency department, urgent care clinic, and deferred-care appointments. The emergency department screening and referral procedures allow for deferral of care by up to 1 week for ambulatory patients requesting emergency department or other ambulatory care services who do not have an emergency condition. All patients requesting unscheduled ambulatory care services are assessed by a registered nurse (RN) as part of the screening process, then assigned to either same-day care in the emergency department or urgent care clinic or are given a deferred-care appointment at the medical center's Primary and Acute Care Center. Determination of which patients do not require same-day care is left to the implicit judgment of an experienced triage nurse and a supervising physician.
We implemented our system of explicit guidelines as an aid to the nurse's implicit judgment. We trained all nurses who perform triage in guideline use before implementation. Training consisted of four 2-hour sessions over the course of 2 weeks. All participating nurses were RNs with at least 1 year of triage experience. After training, nurses used the explicit guidelines to screen all patients who had musculoskeletal symptoms addressed by the guidelines. Nurses were instructed that their judgment should supersede the guidelines if they disagreed with guideline assessment and, in their opinion, the patient should be seen.
Patients eligible for deferred care were told that they did not appear to have an emergency condition and had the option of receiving an appointment for a later date in the Primary and Acute Care Center with the patient's primary care provider or a member of his/her clinic. Deferred patients were also given the medical center's 24-hour toll-free telephone care number to call if their symptoms changed or worsened or if they had new health questions or concerns. Patient's who declined deferred-care appointments were assigned to same-day care in the urgent-care clinic.
The proportion of ambulatory patients meeting the deferred-care guidelines and the timing of their appointments were assessed for a 4-month period by reviewing the nurse triage data forms. Information on no-show rates for clinic appointments was obtained from the electronic hospital information system's appointment display menu. Our institutional review board approved the study.
Development and Reliability
The panelists agreed on the degree of safety associated with deferred care for 27 of the 34 (79%) musculoskeletal symptom scenarios and agreed that deferred care was, in fact, safe in 22. One-page data forms for collection and interpretation of clinical factors during the nurse screening evaluation were created for each musculoskeletal primary complaint (Appendix 1.) View the complete set of forms. To apply the deferred-care guidelines, a screening nurse first completed the data forms. Next, if no critical clinical factors were present, the algorithm was applied to the noncritical clinical factors to reach a screening recommendation. In reliability testing, nurses' deferred care decisions agreed for 39 of 40 standardized patient scenarios using the guidelines (overall interrater agreement 97.5%; k = 0.95). (21)
Ambulatory care triage nurses screened 448 patients presenting with musculoskeletal symptoms. The mean age of the patients screened was 52 years; 98% were male (36% black, 28% white, 4% Hispanic, 1% Asian, and 32% specified another race or did not specify a race). Twenty-one percent of patients screened had service-connected disabilities, and of the remainder, 95% qualified for VA services on the basis of VA low-income guidelines (equivalent to an annual income of less than $21,610).
The number of patients with each musculoskeletal chief complaint and the impact of the guidelines on deferred-care decisions and follow-up is presented in Table 1. Overall, 107 of 448 (24%) patients screened with the guidelines met the criteria for deferred care. No patient who met the deferred-care criteria was judged by the nurse to require same-day care. Seventy-six of the 107 (71%) patients eligible for deferred care agreed to have their care deferred. Many patients cited a long drive and/or transportation difficulties as a reason for declining. The most common follow-up interval for patients receiving deferred-care appointments was 1 day (occurring in 28 of 76 patients), and the median follow-up interval was 3 days. Sixty-six of the 76 referred patients (87%) kept their appointments.
We developed explicit clinical guidelines to assist decision making for same-day versus deferred care for adults with musculoskeletal complaints. The guidelines qualify about one quarter of walk-in VA patients with musculoskeletal complaints for deferred care.
In our guideline-development process, we used a method that combines a review of the best available evidence with the clinical judgment of a multidisciplinary group of experts. An alternate approach to deferred-care guideline development is to prospectively collect information on all clinical factors that may affect the safety of deferring care of ambulatory patients, then correlate them with measured clinical outcomes. However, such an approach would require collection of large, detailed databases and might require years of effort to provide needed guidelines. Until such data are available, methods that combine the available evidence and expert judgment, such as we have used, offer the best way to develop guidelines for triage.
Although our methods produced guidelines that seem to have content validity, it would be desirable to evaluate them further in a study of their ability to prospectively identify patients with medical conditions for whom urgent care prevents morbidity or mortality. There are several potential approaches to such a study, each of which are subject to methodologic or feasibility limitations. The highest quality evidence is obtained from prospective, double-blind, randomized, controlled trials (RCTs). Applied to the validation of our deferred-care guidelines, an RCT might involve selecting patients who meet criteria for deferred care, randomly assigning them to either deferred care or standard same-day care, and then determining clinical outcomes. Randomization enhances the comparability of the two groups and provides a valid basis for measuring efficacy. (22) Although it may be possible to blind clinicians to treatment assignment, it would not be possible to blind patients, and this may bias their reporting of subjective outcomes. In addition, it would take a large clinical trial to identify guideline flaws that result in poor outcomes for low-prevalence conditions.
A prospective cohort study is an alternate approach to assessing the guidelines' predictive validity. Such a study might involve prospectively determining which patients meet and which do not meet deferred-care guidelines, allowing (in a natural experiment) patients to receive care assignments by triage nurses or clinicians who are blinded to the guideline determination, following patients to determine their clinical diagnosis or development of adverse outcomes, then correlating the occurrence of adverse outcomes with the guideline determination (eligible or ineligible for deferred care) and the time interval for evaluation (same-day vs. deferred care). This design is logistically less complex than an RCT and has the same advantage of prospective collection of data; however, it substitutes selection bias of patients to same-day versus deferred care for the attribution bias of patients in an RCT who are not blinded to their study group assignment. A retrospective medical record review of triage assignments and outcomes, while perhaps being the logistically easiest study design to implement, is subject to the additional disadvantage introduced by missing data.
The economic implications of a deferred-care policy is another area that warrants further evaluation. Although there are currently no published cost-effectiveness analyses of emergency department deferred-care policies, there are several studies on costs of care in alternate sites that indirectly address this issue. Williams (23) found that although the costs of nonurgent care in the emergency department are low compared with costs of semiurgent and urgent visits, they are still greater than the median charges for a new-patient office visit. Since actual costs are generally lower than charges, the potential cost savings of delivering care in nonemergency settings may be even greater than those cited by that study. In addition, referral of emergency patients into primary care may reduce future costs; Hansagi and colleagues (24) found less emergency department use in the 1 year following the index evaluation for patients referred to primary care than in those receiving their index visit in the emergency department. In another evaluation of an emergency department deferred-care policy in a pediatric population, Rivera and colleagues (25) found that the costs of deferred care (including triage, emergency department, and clinic costs) were 57% lower than if all patients had been treated in the emergency department. Collectively, these studies suggest that deferred care may be cost-effective.
Our results have two important limitations. First, our guidelines, similar to the vast majority of emergency department and ambulatory care screening guidelines in current use, rest solely on content validity. Past applications of the method we used in developing our guidelines have been shown to produce guidelines with face, construct, and predictive validity for procedures for which there were no RCT data at the time. (18-20) Regardless, rigorous assessment of the guidelines would be enhanced by additional research directed toward assessing the guidelines' predictive validity and using approaches to minimize potential limitations of the study design effect.
A second limitation of our study is that our finding regarding the potential impact of the guidelines for deferring care may differ in other settings. Results from the VA may not necessarily be generalizable to other patient populations because VA patients have many comorbid conditions, and given the way that the critical clinical factors were defined, this may result in identifying fewer patients who could be referred for next-day care than might occur in a community hospital emergency department. In addition, the gender distribution of VA patients is skewed compared with most community hospital emergency department and ambulatory care settings.
Despite these limitations, our deferred-care guidelines address several important challenges in the implementation of guideline-driven clinical decision making. Standardization of the screening process, as demonstrated by our reliability test results, is addressed by presenting the guidelines in an explicit format with precise definitions of potentially ambiguous terms. The guidelines have been operationalized for validation testing or use in settings that wish to implement a deferred-care policy.
We developed and operationalized screening guidelines to inform decision making about the timing of care for patients with common musculoskeletal complaints. Use of these guidelines requires only the limited clinical data available to an ambulatory clinic or emergency department triage nurse. This approach may be useful in ambulatory settings that currently rely on implicit judgments of unknown reproducibility to make determinations about which patients do and which do not require same-day care.
|Take Home Points
1. Nadel V. Emergency departments: unevenly affected by growth and change in patient use. Washington, DC: US General Accounting Office, Human Resources Division; 1993; Publication GAO/HRD 93-4.
2. Friedman E, Hagland MM, Hudson T, McNamara P. The sagging safety net. Emergency departments on the brink of crisis. Hospitals. 1992;66:26-40.
3. Lynn SG, Kellermann AL. Critical decision-making: managing the emergency department in an overcrowded hospital. Ann Emerg Med. 1991;20:287-92.
4. Gill JM, Reese CL 4th, Diamond JJ. Disagreement among health care professionals about the urgent care needs of emergency department patients. Ann Emerg Med. 1996;28:474-9.
5. Mitchell TA. Nonurgent emergency department visits—whose definition? [Editorial] Ann Emerg Med. 1994;24:961-3.
6. Stratmann W, Ullman R. A study of consumer attitudes about health care: the role of the emergency room. Med Care. 1975;13:1033-43.
7. Haddy R, Schmaler M, Epting R. Nonemergency emergency room use in patients with and without primary care physicians. J Fam Pract. 1987;4:389-92.
8. Wuerz R, Fernandes CMB, Alarcon J, for the Emergency Department Operations Research Working Group. Inconsistency of emergency department triage. Ann Emerg Med. 1998;32:431-5.
9. Young GP, Wagner MB, Kellermann AL, Ellis J, Bouley D. Ambulatory visits to hospital emergency departments. JAMA. 1996;276:460-5.
10. Brillman JC, Doezema D, Tandberg D, et al. Triage: limitations in predicting need for emergent care and hospital admission. Ann Emerg Med. 1996;27:493-500.
11. Lowe RA, Bindman AB, Ulrich SK, et al. Refusing care to emergency department patients: evaluation of published triage guidelines. Ann Emerg Med. 1994;23:286-93.
12. Birnbaum A, Gallagher EJ, Utkewicz M, Gennis P, Carter W. Failure to validate a predictive model for refusal of care to emergency-department patients. Acad Emerg Med. 1994;1:213-7.
13. Baker DW, Stevens CD, Brook RH. Patients who leave a public hospital emergency department without being seen by a physician. Causes and consequences. JAMA. 1991;266:1085-90.
14. Derlet RW, Nishio DA. Refusing care to patients who present to an emergency department. Ann Emerg Med. 1990;19:262-7.
15. Hansagi H. Referral of non-urgent cases from an emergency department: patient compliance, satisfaction and attitudes. Scand J Soc Med. 1990;18:249-55.
16. Stussman BJ. National Hospital Ambulatory Medical Care Survey: 1995 emergency department summary. Adv Data. 1997;1-19.
17. Brook RH, Chassin MR, Fink A, Kosecoff J, Park RE. A method for the detailed assessment of the appropriateness of medical technologies. Int J Technol Assess Health Care. 1986;2:53-63.
18. Brook RH. The RAND/UCLA appropriateness method. In: McCormick KA, Moore SR, Siegel RA, eds. Methodology Perspectives. Rockville, MD: Public Health Service, U.S. Department of Health and Human Services; 1994. AHCPR Pub. No. 95-0009:59-70.
19. Merrick NJ, Fink A, Park RE, et al. Derivation of clinical indications for carotid endarterectomy by an expert panel. Santa Monica, CA: Rand Corporation; 1991.
20. Shekelle PG, Chassin MR, Park RE. Assessing the predictive validity of the RAND/UCLA appropriateness method criteria for performing carotid endarterectomy. Int J Technol Assess Health Care. 1998;14:707-27.
21. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159-74.
22. U.S. Preventive Services Task Force. Guide to clinical preventive services: an assessment of the effectiveness of 169 interventions. Baltimore: Williams & Wilkins; 1989.
23. Williams RM. The costs of visits to emergency departments. N Engl J Med. 1996;334:642-6.
24. Hansagi H, Allebeck P, Edhag O. Health care utilization after referral from a hospital emergency department. Scand J Soc Med 1989;17:291-9.
25. Rivara FP, Wall HP, Worley P, James KD. Pediatric nurse triage. Am J Dis Child. 1986;140:205-10.