ACP American College of Physicians - Internal Medicine - Doctors for Adults

Effective Clinical Practice

A Randomized Trial of the Use of Patient Self-Assessment Data To Improve Community Practices

Effective Clinical Practice, January/February 1999

John H. Wasson, Therese A. Stukel, Julia E. Weiss, Ron D. Hays, Anne M. Jette, Eugene C. Nelson

For author affiliations, current addresses, and contributions, see end of text.

See related editorial

Objective. Because of time constraints in the office environment, problems of concern to elderly patients may not be raised during clinic visits. To facilitate communication about geriatric health problems, we examined the impact of a strategy that used patient self-assessment data to improve community practices.

Design. Twenty-two primary care practices were randomized to participate in the intervention strategy (intervention practices) or to provide usual care (usual care practices).

Setting. Primary care practices in 16 towns in New Hampshire (total, 45 physicians).

Patients. 1651 patients 70 years of age or older.

Intervention. All patients received a mailed survey that asked about their health problems and about how well these problems were being addressed by their physicians. In the intervention practices, these data were used to generate a customized letter that directed the patient to specific sections in an 80-page modified version of the National Institute on Aging's Age Pages and were summarized and communicated to the patient's physician.

Main Outcome Measure. Change from baseline in patientsí overall assessment of health care.

Results. In 8 of 11 intervention practices, patients felt that their care had improved over the 2-year study period. This improvement occurred in only 1 of 11 usual care practices (P = 0.003). Patients in intervention practices reported receiving significantly more help with physical function, fall prevention, and assistance for memory problems. Self-assessed health status did not differ in the two groups.

Conclusion. A standard, easy-to-implement strategy to improve the quality of provider-patient interactions can improve the satisfaction of older patients cared for in community practices.

Take Home Points

It is difficult to successfully manage elderly patients in a community practice because these patients usually have several problems and a clinician's time is often limited. (1) However, if the clinician is prepared to assess and manage the broad range of issues that matter to the patient and if the patient is well informed and educated about these issues, quality of care is improved. (2) The gap between successful and typical geriatric management has stimulated research into new strategies for care delivery.

One effective way to improve geriatric care is to deploy a special clinical team to direct patient care and follow-up. (2-5) Another is to implement a group education and management program. (6, 7) Unfortunately, both approaches are expensive and, consequently, are often targeted only at patients who are at greatest risk. Team care and group education efforts, therefore, may have limited applicability in the "real world" of community practice.

An alternative to targeting is to bring the benefits of a strategy based on standardized patient self-assessment, feedback, and education into everyday clinical work. Preliminary efforts in this direction were successful when they focused on retirees' risk behaviors. (8) Whether such a strategy would be successful for patients with a wide range of functional and clinical concerns has not been addressed. For this reason, we did a controlled trial of the use of patient self-assessment data to improve the quality of provider-patient interaction in busy primary care practices.

Methods

Overview

We designed and implemented a randomized trial of a strategy intended to improve the care of elderly patients. The unit of randomization was the practice. Clinicians in practices that were assigned to participate in the intervention (intervention practices) were engaged in discussions and had a prominent role in designing the intervention described below. Patients in both the intervention practices and the usual care practices received questionnaires at the beginning and end of the study. The duration of the study was 2 years.

Practice and Patient Recruitment

We mailed a letter describing the study to 116 general internists and family practitioners in New Hampshire. To enhance the generalizability of our results, we recruited practices from three regions: the New Hampshire seacoast, the urban corridor around the Merrimack Valley, and the area north of the White Mountains. Twenty-two practices were enrolled and randomized to receive the intervention or provide usual care.

In participating practices, all physicians who cared for patients 70 years of age and older were included in the study. A total of 45 physicians (27 family practitioners and 18 internists) participated. These physicians had a mean age of 43 years; 39 were men, and 6 were women.

Each practice provided a complete list of patients 70 years of age or older who had been seen by a participating physician in the previous year; patients in nursing homes were excluded. Project staff mailed each of these patients a description of the project; an informed consent form (approved by our institutional review board); and a baseline survey about the patient's health, previous medical care, and demographic characteristics. The survey was completed, and consent was obtained from 3051 patients.

Randomization

Randomization of patients would have been inappropriate for our study. Our interest was to engage study clinicians in the plan and design of the intervention. If these same clinicians were to care for "control patients," some of these patients would inevitably have been exposed to the intervention (i.e., they would have been "contaminated"). Consequently, the unit of randomization was the same level at which the intervention occurred--the practice.

Randomization occurred with the following practice strata: three regions, two size categories (solo or partnership practices), and two specialty categories (family practice or internal medicine).

Intervention

Within 6 weeks of randomization, our project staff met with the providers and office staff of each intervention practice. We used a standard approach to discuss how elderly patients were managed in the office and how the office staff could try new methods to improve care. (9) Figure 1 shows the strategy that the intervention practices designed and adopted to measure and improve geriatric care. (10) This strategy involved three major components.

Patient Self-Assessment of Health Care and Needs

Within 3 months of randomization, the project staff mailed a 30-item geriatric health assessment survey to patients in their homes. The survey was repeated 15 months later. Eighty percent of patients returned both surveys.

The survey inquired about instrumental activities of daily living, (11 )the number of medications taken by the patient and the out-of-pocket costs of those medications, and the degree of bother from common geriatric symptoms. (12, 13) Assessments of health in the survey of patients in the intervention practices were measured with the Dartmouth COOP Chart System. (14, 15) When patients had substantial difficulties with physical function, emotional status, pain, daily activities, social support, or social activities, they were asked to report whether their physician was aware of the problem, the quality of the explanation they had received about the problem, and the impact of any previous treatments given for the problem. Several additional questions identified risks to patients' health; these questions were about the use of cigarettes and alcohol, difficulties with driving a car, awareness of advanced care plans, past receipt of immunizations, understanding of hazards in the home, methods for keeping track of medications, and inadequate financial resources.

Tailored Patient Education

Patients in intervention practices who returned a completed geriatric health assessment questionnaire were sent an 80-page geriatric health manual to promote better understanding and self-management of common geriatric problems. The information in the manual was modified from the National Institute on Aging Age Pages. (16) A cover letter, generated by the computer algorithm and signed by each patient's physician, provided a personalized summary of the patient's clinically significant responses and recommended sections in the manual that would address the patient's needs (Appendix Figure 1).

Feedback to the Physician

Each patient's survey was summarized, printed on a flow sheet, and mailed to the patient's physician. The flow sheets highlighted substantial functional problems and patients' perceptions of previous attention to these problems. The physicians planned to incorporate the patient flow sheets into the medical record and to discuss the results with each patient (Appendix Figure 2). In addition, physicians were given summary reports based on the data from all of their patients.

Patients in the usual care practices received a self-assessment survey but no tailored patient education. No feedback data were given to physicians in the usual care practices.

Outcome Measures and Analysis

All patients received a questionnaire at the beginning and the end of the study. The primary outcome was the change in patients' overall assessment of health care during the study. At study entry, patients were asked to agree or disagree (on a 5-point Likert scale) with the statement "there are some things about the medical care I receive that could be better." At the end of the study, patients indicated whether the medical care from their regular physician was "a lot better, somewhat better, a little better, unchanged or worse."

These data were summarized at the practice level and were used to produce Figure 2. The mean assessment of health care at study entry for all of the patients in a single practice was compared with the grand mean for all 22 practices. Similarly, the mean assessment of health care at study closure within a single practice was compared with the grand mean for all practices. This analysis shows the relative standing of each practice (from the patient's perspective) at the beginning and end of the study. The proportion of usual care practices with improved relative standing (i.e., a positive slope) was compared with the proportion of intervention practices that had improved relative standing (test for two-sample proportion).

Data on the secondary end points as depicted in Figure 3 were obtained from the final questionnaire. Patients were asked a series of questions that had the following general form: "Since the beginning of this project two years ago, how much has your doctor, the office staff or any written information they gave you, helped you with ____ ?" Specific questions addressed clinical problems ("your urinating or wetting"), functional limitations ("your needs for support"), and the patient's understanding of preventive approaches ("your understanding of the advantages of a written advance care plan [living will]"). Responses were collapsed into dichotomous outcomes (e.g., helped/not helped). Patients were also asked whether they had received pneumococcal vaccine, had exercised regularly, and had documented an advance care plan.

Differences between patients in the usual care and intervention practices are expressed by using odds ratios with 95% CIs. These were developed by using an analysis for clustered binary data with overdispersed logistic regression model (weighting by the number of patients in each physician's practice responding to each question or reporting specific clinical or functional problems). (17) Each model adjusted for practice area and size, physician specialty, percentage of patients 85 years of age and older, percentage of female patients, percentage of patients with at least a high school education, presence of any limits on instrumental activities of daily living, and the percentage of patients in poor or fair health at baseline.

To assess patients' change in health during the study, we compared their instrumental activities of daily living (10) and SF-36 scores (summarized by using physical and mental health component scores) (18) on the baseline and final questionnaires. Hospital cost data were obtained from Medicare, and patients were asked about their medication costs. These continuous secondary outcomes were analyzed by using a linear mixed-effects model that adjusted for patient age, sex, education, limits on instrumental activities of daily living, and fair or poor health and included practice area, size, and physician specialty as random effects. (19)

At the end of the study, practices were asked to indicate the status of patients who did not return a final survey. Patients were categorized as dead, in nursing homes, or lost to follow-up for specified reasons. We also used the New Hampshire death index to identify patients who had died during the study period.

Results

Study Execution

Of the 3051 patients who entered the study, 459 died during the study period (251 [16%] of those in usual care practices and 208 [14%] of those in intervention practices) and 60 entered nursing homes (27 [2%] of those in usual care practices and 33 [4%] of those in intervention practices). In addition, 318 patients had moved, had transferred care, or had not been seen by their physicians during the study period. If the remaining 2214 patients are considered the denominator, the final survey response rate is 75% (1651 of 2214): 73% of patients in usual care practices and 77% of patients in intervention practices.

Compared with patients who remained in the study, the 1400 patients who withdrew were older (80 versus 78 years of age) and were more likely to be in fair or poor health at baseline (34% versus 19%), to have reported hospitalization in the past 6 months (16% versus 9%), to have limits on instrumental activities of daily living (40% versus 22%), and to believe that their clinical care could be better (33% versus 29%). However, patients who withdrew from the intervention group did not differ from those who withdrew from the usual care group.

Table 1 shows the baseline characteristics of the 22 practices and the 1651 patients who completed the baseline and final questionnaires.

Primary Outcome

Figure 2 shows the change in patient-assessed health care quality in the intervention and usual care practices. Over the 2-year study period, 8 of the 11 intervention practices improved their relative standing with regard to how their patients judged them. Only 1 of the 11 usual care practices showed this improvement (P= 0.003 for the comparison of the intervention with usual care). Findings were similar when a physician-level analysis was used: Nineteen of the 24 physicians in intervention practices but only 5 of the 21 physicians in usual care practices improved (P= 0.007).

Secondary Outcomes

Figure 3 shows the odds ratios and CIs for 22 patient assessments of health care. Compared with patients in usual care practices, patients in intervention practices had a more favorable impression of their health care in 18 of 22 assessments (P< 0.01 by sign test). In particular, more patients in intervention practices reported receiving help with limitations in physical and daily activities (82% versus 70%), emotional problems (71% versus 59%), social needs (63% versus 50%), and problems with thinking or memory (64% versus 50%). The intervention also enhanced patients' self-reported understanding of methods to prevent falls (39% versus 23%) and of advanced care plans (73% versus 55%).

Unfortunately, these favorable assessments did not translate into obvious improvements in health (Table 2). As expected, the self-reported health status of patients in both groups declined, and no difference was seen between the groups. A trend in the instrumental activities of daily living favored the intervention (P= 0.06). Mortality and estimated costs were similar in both groups.

Subgroup Analysis: Patients Who Recall Specific Discussion

Ninety-three percent of the patients in intervention practices recalled receiving the tailored patient education materials, and 74% reported reading 3 or more of the 19 sections in the geriatric health manual. Only 23% (n = 187), however, reported that their physician or someone in the physician's office had specifically discussed the results of their health assessment or educational materials with them.

Figure 4 shows the assessments of this subgroup. Compared with other patients in intervention practices (and patients in usual care practices), members of this subgroup who recalled specific discussions of their self-assessment were much more likely to have favorable impressions of their health care.

Discussion

We conducted a controlled trial of the use of patient self-assessment data to improve provider-patient interaction in busy primary care practices. Two years after randomization, patients in intervention practices were more likely than patients in usual care practices to report receiving improved medical care, gaining a better understanding of several important threats to their health, and receiving greater assistance with some functional and clinical problems. The intervention rests on the well-documented observation that a synergistic interaction of standard health assessment, customized patient education, and productive physician-patient communication is necessary to improve care for old and chronically ill persons. (2, 20) The subgroup analysis suggests that increasing provider participation could greatly increase the benefit of the intervention. Unfortunately, we did not give physicians aggressive feedback about patient recall of the intervention.

Several additional limitations of our trial deserve comment.

Loss to Follow-up

Many patients who entered our study did not finish it. Some of the attrition was due to death; some to nursing home admission; some to changes in residence; some to increasing cognitive impairment; and some, undoubtedly, to patient choice. Such losses are one of the challenges of studying a community-dwelling elderly population (mean age, 78 years). We are reassured, however, that attrition was similar in both groups and did not bias our comparison.

Generalizability

The attrition seen in our study does raise concerns about the generalizability of our results. Clearly, the intervention is not for all patients. Participants must be cognitively intact to complete their self-assessment data. We did, however, observe both very old and sick patients completing these data.

The participation of volunteer physicians may also raise concern about generalizability. At the end of the study, patients in both groups reported that more than 90% of their care came from one provider: This is a marker of high-quality ambulatory care and a potential source of improved outcomes. (21) Our results show that a strategy of standardized assessment, feedback, and education may make presumably "good" care even better. Whether this would apply to physicians who do not volunteer to participate is open to question.

Self-Assessment

Finally, in the diverse and independent practices included in our study, we were unable to obtain direct measures of clinician actions or assessments (e.g., blood pressure measurements or diagnoses). We relied on patient reports about many processes and outcomes of care that matter to them. We believe that our reliance on patient perceptions of care is justified not only because these perceptions are easy to reliably obtain from diverse practice settings but also because positive perceptions of care result in better adherence to management recommendations. (22)

However, patients in the intervention practices had no improvement in physical and emotional function according to the SF-36, even though they claimed to have received greater assistance with physical and emotional problems. Was the SF-36 insensitive to change, or was improvement in care unrelated to improvement in function? The possibility that the SF-36 might have been unresponsive to change is supported by the observation that the instrumental activities of daily living tended to worsen more over time in the usual care practices than in the intervention practices.

Implications

Clinicians and health care administrators can now choose from numerous options to improve care for elderly and chronically ill outpatients. Past studies have suggested that $10, 000 might buy team care for approximately 5 to 10 patients a year (3) and group education for about 75 to 150 patients for a year. (6, 7) For the same cost, we estimate that the strategy studied here can be made available to more than 500 elderly patients.

Our results show that a strategy based on patient self-assessment data, patient and provider feedback, and patient education can be implemented in busy physicians' practices to the benefit of many older patients. However, for this strategy to succeed, the clinical team must actively and continuously reinforce it.

Take Home Points
  • "Real-world approaches" are needed to improve care for elderly persons.
  • Previous work suggests that facilitating patient-provider interaction would be one step toward that goal.
  • We designed, implemented, and tested a simple intervention that uses patient self-assessment data in 22 primary care practices in New Hampshire.
  • Patients in intervention practices were more likely than patients in usual care practices to report that their care had improved.
  • The intervention had the greatest impact when it was reinforced by the clinicians.

 

References

1. Miller DK, Brunworth D, Brunworth DS, Hagan R, Morley JE. Efficiency of geriatric case-finding in a private practitioner's office. J Am Geriatr Soc. 1995;43:533-7.

2. Von Korff M, Gruman J, Schaefer J, Curry SJ, Wagner EH. Collaborative management of chronic illness. Ann Intern Med. 1997;127:1097-102.

3. Boult C, Boult L, Morishita L, et al. Outpatient geriatric evaluation and management. J Am Geriatr Soc. 1998;46:296-302.

4. Stuck AE, Siu AL, Wieland GD, Adams J, Rubenstein LZ. Comprehensive geriatric assessment: a meta-analysis of controlled trials. Lancet. 1993;342:1032-6.

5. Essential components of geriatric care provided through health maintenance organizations. The HMO Workgroup on Care Management. J Am Geriatr Soc. 1998;46:303-8.

6. Lorig KR, Mazonson PD, Holman HR. Evidence suggesting that health education for self-management in patients with chronic arthritis has sustained health benefits while reducing health care costs. Arthritis Rheum. 1993;36:439-46.

7. Beck A, Scott J, Williams P, Robertson B, Jackson D, Gade G, et al. A randomised trial of group outpatient visits for chronically ill older HMO members. J Am Geriatr Soc. 1997;45:543-9.

8. Fries JF, Bloch DA, Harrington H, Richardson N, Beck R. Two-year results of a randomized controlled trial of a health promotion program in a retiree population: the Bank of America Study. Am J Med. 1993;94:455-62.

9. Nelson EC, Batalden PB, Plume SK, Mohr JJ. Improving health care, Part 2: A clinical improvement worksheet and users' manual. Jt Comm J Qual Improv. 1996;22:531-48.

10. Wasson JH, Jette AM, Johnson DJ, Mohr JJ, Nelson EC. A replicable and customizable approach to improve ambulatory care and research. J Ambulatory Care Manage. 1997;20:17-27.

11. Fillenbaum GG. Screening the elderly. A brief instrumental activities of daily living measure. J Am Geriatr Soc. 1985;33:698-706.

12. Hale WE, Perkins LL, May FE, Marks RG, Stewart RB. Symptom prevalence in the elderly. An evaluation of age, sex, disease, and medication use. J Am Geriatr Soc. 1986;34:333-40.

13. Wasson J, Nierenberg D, Landgraf J, et al. The effect of a patient questionnaire on drug-related symptoms in elderly outpatients. In: Rowe JW, Ahronheim JC, Lawton MP, eds. Annual Review of Gerontology & Geriatrics. New York: Springer; 1992;12:109-25.

14. Nelson EC, Landgraf JM, Hays RD, Wasson JH, Kirk JW. The functional status of patients. How can it be measured in physicians' offices? Med Care. 1990;28:1111-26.

15. Nelson EC, Wasson JH, Johnson DJ, Hays RD. Dartmouth COOP Functional Health Assessment Charts: brief measures for clinical practice. In: Cramer JA, Spilker B, eds. Quality of Life and Pharmacoeconomics: An Introduction. Philadelphia: Lippincott-Raven; 1996:161-8.

16. U.S. Department of Health and Human Services, Public Health Service, National Institutes of Health. Age Page. Gaithersburg, MD: National Institute on Aging; 1993.

17. McCullagh P, Nelder JA. Generalized Linear Models. 2d ed. New York: Chapman and Hall; 1989.

18. Ware JE, Kosinski M, Keller SD. SF-36. Physical and mental health summary scales: a user's manual. Lincoln: RI Quality Metric; 1997.

19. Koepsell TD, Martin DC, Diehr PH, Psaty BM, Wagner EH, Perrin EB, et al. Data analysis and sample size issues in evaluations of community-based health promotion and disease prevention programs: a mixed-model analysis of variance approach. J Clin Epidemiol. 1991;44:701-13.

20. Reuben DB, Hirsch SH, Frank JC, Maly RC, Schlesinger MS, Weintraub N, et al. The Prevention for Elderly Persons (PEP) Program: a model of municipal and academic partnerships to meet the needs of older persons for preventive services. J Am Geriatr Soc. 1996;44:1394-8.

21. Wasson JH, Sauvigne AE, Mogielnicki RP, Frey WG, Sox CH, Gaudette C, et al. Continuity of outpatient medical care in elderly men. A randomized trial. JAMA. 1984;252: 2413-7.

22. Devor M, Wang A, Renvall M, Feigal D, Ramsdell J. Compliance with social and safety recommendations in an outpatient comprehensive geriatric assessment program. J Gerontol. 1994;49:168-73.

Acknowledgments

The authors thank the following New Hampshire physicians for their participation in the project: Drs. James G. Kennedy, Keith Lammers, and Lana Wlodyka (Amherst); Paul Tung and William Dudley (Dover); L. Mark Reiner (Exeter); Charles Wolcott (Franconia); John Kaminski (Hampton); Richard Douglass (Hillsboro); Gregory Neilley and Francis Fuselier (Keene/Monadnock); Pancras van der Laan, Brian Beattie, Wendy Muello, Jorge Devillafane, Elwin Falkenham, and John Jackson (Lancaster); David Bishop, Robert Peraino, and Kathleen Smith (Littleton); Edward Hall, James McGreehan, and Bernard Rosen (Merrimack); Anita Yap, Albee Budnitz, Alexis-Ann Bundschuh, Kenneth Dolkart, Steven Levine, Todd Spencer, Jesse Wagner, Harry Ward, Peter Hacker, and David Gilmour (Nashua); John J. Haley (New Ipswich); Ted Benoit, Karen Brainard, Paul Friedrichs, Monique LaRouche, Peter Degnan, and Andrew Weeks (Newmarket/Raymond); Michael Stein (Salem); Peter Richards (Somersworth); and Alan Stein and Gerard Hevern (Suncook).

The authors also thank Drs. Gilbert Welch and Joan Teno for their critical review of this paper and Deborah Johnson, Judith Knapp, Gail Wasson, and Priscilla White for database management, quality control of data collection, and liaison with physicians and office staff.

Grant Support

In part by the John A. Hartford Foundation, New York, New York.

Correspondence

John H. Wasson, MD, 7265 Butler Building, Dartmouth Medical School, Hanover, NH 03755-3862; telephone: 603-650-1823; fax: 603-650-1331; e-mail: john.h.wasson@dartmouth.edu.