ACP American College of Physicians - Internal Medicine - Doctors for Adults

Effective Clinical Practice


Linking Experimental Research Methods and Health Services Research: Everything Is Possible

Effective Clinical Practice, May/June 2000.

John R. Feussner, MD

For author affiliations, current addresses, and contributions, see end of text.

In the late 1980s, the move in the United States to a primary care model of health care had not yet emerged from the health policy debate. Nonetheless, Wasson and colleagues (1) at the Veterans Affairs (VA) Medical Center in White River Junction, Vermont, were already considering ways to improve access to and quality of health care while reducing costs in primary care. Their early work studied how to improve systems for following patients in primary care. They evaluated telephone communication as a way of reducing medical care utilization (substituting telephone contacts for routine clinic visits) without adversely affecting patient health status. In today's parlance, Wasson and colleagues studied the possibility that costs and services utilization could be reduced without compromising quality of care. This randomized trial represented an early health services research effort to apply systems solutions to improve the efficiency of services delivery. And it worked: The trial was positive. In addition, they documented the feasibility of applying rigorous experimental research methods to health services research questions. Their work, and that of like-minded researchers, continues to expand the knowledge base for effective clinical practice.

The current issue of ecp includes a VA Cooperative Study that again examines the effect of telephone care in a randomized trial. (2) The research addresses a common strategy designed to improve communication with patients and to assure timely access to health care. The investigators used end points that represent important patient and systems outcomes: self-reported health status and health services utilization. They focused on answering the research question definitively for primary care providers. So what did they find? That telephone care had little effect, even though more than 2000 calls to 257 patients were made over the 2-year study period. The investigators note correctly that their trial was negative because they failed to improve the study outcomes. Despite the lack of improvement in study outcomes, however, the information obtained from this trial is useful to primary care providers.

So What Went Wrong?

The investigators seem to be at a loss to explain why their results differed from those of the original single-site trial. In the current study, the telephone intervention became a supplemental feature of care delivery rather than a substitute for clinic visits, thereby diminishing any effect of the intervention on health services use. Perhaps the initial study was biased--the principal investigator was on site and was a study participant. In the jargon of clinical trials, this type of bias is called co-intervention, and it implies that the principal investigator was an important part of the intervention. When the co-intervention is eliminated, one can ascertain the true effect of the intended intervention. This seems like a plausible explanation for the failure of the current intervention to have as great an impact as the original intervention.

Other factors may also have diminished the effect of the intervention. First, the patients may have been too sick to benefit. Veterans who use VA health care are sicker than veterans who do not use VA care and are sicker than the general population. For example, consider self-perceived health, as measured by the SF-36. In the general U.S. population, the mean SF-36 score is about 50 (the instrument score is 0 to 100, with 100 representing perfect health). In the VA population, the mean score is approximately 35. (3, 4) In the current study, nearly one third of the patients described their health as fair to poor and only 23% rated their health as excellent. The high burden of illness among veterans may also explain the findings of another VA cooperative study in which intensive primary care failed to reduce hospital readmissions (another example of an intervention to enhance primary care that failed to reduce utilization). (5) One could postulate that in a sicker patient population, telephone care cannot readily substitute for a clinic visit. However, because the burden of illness among veterans in the original study of telephone care was similar to that in the current study and the hospital readmission study, "sickness" alone cannot explain the negative study result.

Second, primary care in general may have improved. The health care system is not static. Since both studies from White River Junction were done in the VA system, secular changes may have led to improvements in usual care, diminishing differences between the intervention and the usual care arms. Since 1995, the Department of Veterans Affairs has made a concerted effort to enhance primary care and has made the delivery of comprehensive, coordinated primary care a high priority. The VA has even changed their emphasis in physician training from specialty care to primary care. The increased priority given to primary care and improved health care delivery systems may have improved the usual primary care delivered in the current trial. Improvements in primary care delivery would make it less likely that the current trial would observe the same benefits noted by Wasson and colleagues, whose study was initiated nearly a decade earlier. (1)

Third, the intervention may have been too weak. In the current study, almost one third of the telephone contacts were less than 5 minutes and 80% were less than 10 minutes. The intervention may have included too little contact and too little content to significantly affect patient or systems outcomes. The content of the telephone contact is at least as important as the telephone contact itself. Newer automated telephone disease management interventions may prove to be more effective than brief contacts by primary care providers. The potential usefulness of the telephone should not be discounted simply because the current study is negative.

So much for explanations. Negative trials are valuable in and of themselves. All care delivered to patients is costly, and it is worth knowing what benefit might be realized from any intervention. There is also value in rigorously studying such "systems approaches" as telephone care. More studies focusing on systems approaches to services delivery need to be done. Often, such studies are so pragmatic that they have difficulty attracting the enthusiasm of research funding agencies. The current trial demonstrates a special benefit of a publicly funded national health system such as the VA, in which health services research projects receive a high priority. The VA includes national leaders who are systems thinkers and who are committed to solving health care delivery problems by evaluating and implementing effective systems solutions.

At the same time, one of the limitations of the VA system is that its enrolled patient population is predominately male and older than the general U.S. population. In the future, cooperation and collaboration between the VA, the largest integrated health system in the United States, and private health care systems could address systems questions in ways that enhance the generalizability of such health services research.

What Next?

From a systems perspective, there are many issues to pursue. One obvious priority is how to best organize and integrate the spectrum of health services, from routine outpatient care to acute hospitalization, restorative or rehabilitation care, and long-term care. The physician workforce continues to be a major concern, including the role of nonphysician providers in primary care and the relation between primary care and specialty providers. Access to health care remains an important concern for health systems and their patients, particularly those in rural areas (where telemedicine may play an important role). Some systems research is being greatly facilitated by improvements in information systems and the realization of an electronic medical record--particularly research on practice variation, the process of guideline implementation, and the tracking of health services use and quality in many chronic diseases. Issues of chronic disease management (especially development of optimal strategies for patient communication and improvement of treatment compliance) and ongoing assessment of quality of care and prevention of medical errors are also high-priority areas for research. Finally, such concerns as the financing of health care, care for the uninsured, and health care among women and ethnic minority groups are ongoing research priorities.

The importance of the current trial should not be lost because the result was negative. The investigators viewed health care delivery issues from a systems perspective; they used rigorous experimental research methods; they required improvements in outcomes relevant to patients and a health system; and they pursued publication of their work, even though the result was negative. Other investigators should follow their lead and conduct studies like this. At least in the VA health system, we are committed to facilitating and funding such systems approaches to health care delivery issues.


1. Wasson J, Gaudette C, Whaley F, Sauvigne A, Baribeau P, Welch HG. Telephone care as a substitute for routine clinical follow-up. JAMA. 1992;267:1788-93.

2. Welch HG, Johnson DJ, Edson R. Telephone care as an adjunct to routine medical follow-up: a negative randomized trial. Eff Clin Pract. 2000;3:123-30.

3. Kazis LE, Ren XS, Lee A, et al. Health status in VA patients: results from the veterans health study. Am J Med Qual. 1999;

4. Kazis LE, Miller DR, Clark J, et al. Health-related quality of life in patients served by the Department of Veterans Affairs: results form the Veterans Health Study. Arch Intern Med. 1998;158:626-32.

5. Weinberger M, Oddone EZ, Henderson WG. Does increased access to primary care reduce hospital readmissions? Veterans Affairs Cooperative Study Group on Primary Care and Hospital Readmission. N Engl J Med. 1996;334:1441-7.

John R. Feussner, MD, Department of Veterans Affairs, Washington, D.C.


The opinions expressed by Dr. Feussner are his own and do not necessarily represent those of the Department of Veterans Affairs.


John R. Feussner, MD, Chief Research and Development (12), Department of Veterans Affairs, 810 Vermont Avenue NW, Washington, DC, 20420; telephone: 202-273-8284; fax: 202-273-6526; e-mail: