ACP American College of Physicians - Internal Medicine - Doctors for Adults

Effective Clinical Practice

Performance Measures for Ethics Quality

Effective Clinical Practice, November/December 1999.

Recent health plan advertisements suggest a belief that patients care about ethics. These ads display soft-focus portraits of trusting patient­physician relationships, statements of physicians unhindered in advocating for their patients, and claims of patient-centered and relationship-based decision making. Two recent billboards near Chicago's O'Hare airport proclaimed, "We're your health plan, not your doctor" and "Excellent health care begins with listening." These ads reflect a marketing perception that the public wants health plans that provide care in a setting of trust and on a foundation of high ethical standards.

In this Ethics Matters, I propose that adherence to ethical standards is an important aspect of health care quality and that performance measures for ethics should be a component of quality assessment and reporting in health care. Measures of ethics quality are likely to be more complementary to traditional performance measures, and they may improve overall quality measurement compared with using only disease-based measures. A framework for developing and testing potential performance measures for ethics is presented through the collaborative Ethical Force (E-Force) Program, recently initiated by the American Medical Association's (AMA) Institute for Ethics.

Measurable Aspects of Health Care Quality

There are at least three distinct and measurable aspects of health care quality: technical quality, service quality, and ethics quality. These three aspects differ from the three ways to measure quality using structure, process, and outcome measures (Table 1) as described by Donabedian more than 30 years ago.1 Although there should be valid, reliable, and feasible performance measures for technical, service, and ethics quality, most measures today focus on the technical aspect—partly because these measures are meaningful to the clinicians and researchers who create them.2

Why Measure Ethics Quality?

The most important reason to measure ethics as part of health care quality is that adherence to high ethical standards is meaningful to patients. It may be hard for patients to identify with performance measures that focus on diseases they don't have or expect to have. However, every patient is likely to identify to some degree with ethical issues, such as the privacy of sensitive information, trust in the advocacy of caregivers, and participation in medical decision making. In addition, over the past 2 years, surveys by Kaiser Foundation/Harvard have shown that the public is increasingly concerned that health plans have little regard for communal ethical norms, for example, in providing health care for vulnerable persons.3

Some health care quality experts believe that the ethical norms of health organizations are more important to patients than are satisfaction measures, such as the demeanor of a receptionist or the condition of the waiting area.4 Some have called for development of standards that address ethical issues,5 suggesting that explicit ethical standards might become an important source of patient trust in health plans and physician groups.6 The President's Advisory Commission on Consumer Protection and Quality in the Health Care Industry recommended development of better measures of "interpersonal aspects of care," which would be addressed by a comprehensive set of performance measures for ethics.7

Performance measures for ethics might also measure overall quality well. The President's commission pointed out the need for "better summary measures that address quality across multiple dimensions."7 This is because clinical measures of technical quality typically measure quality of care for only one disease or type of disease; that is, they are disease-specific measures. Of course, the hope of those using disease-specific measures is that performance on these measures will reflect quality throughout the organization, or at least in one important area.

For example, one might believe that the surgical mortality rate during coronary artery bypass grafting predicts the quality of all surgical care, or that mammography rates measure the quality of preventive care. But quality measures like these are analogous to indicator species in ecology, where the health of an entire ecosystem is estimated by the status of a single animal or plant. If the indicator species is in trouble, the rest of the forest may be too, but protecting the indicator species may or may not protect other important species. The quality of cardiac surgery may say nothing about the quality of other types of surgery. Similarly, high mammography and vaccination rates may not confer excellence in all of preventive medicine. An organization's interest in maintaining a high level of ethical standards, however, should both reflect and foster a similar level of concern with providing high-quality care overall.

Is Ethics Quality Measurable?

Ethics quality is, of course, a loaded term; there is no gold standard for ethical behavior despite the vigorous argument on the topic that has been going on for centuries. Is concern for justice most important, or concern for autonomy? Is it ethically more laudable to fulfill a duty to community or to special relations? In health care, these issues are made more complex by occasionally competing ethical standards based on business, public health, and personal and professional ethics.8 The impossibility of reaching a consensus on these issues could be paralyzing; indeed, some balk at the entire idea of performance measures for ethics. They may ask, "In the absence of gold standards for ethical behavior, how can we possibly determine whether one organization has better ethics quality than another?"

But let us ask in return, "Does the absence of gold standards distinguish ethics from technical or service quality?" The answer, for a great many quality measures, is no. Even many technical quality measures have no gold standard.9 For example, the right number of angioplasties for a population depends on many factors, such as available resources, values of the population, and alternate spending priorities.10 Aiming for zero complications in cardiac surgery sounds good, but if achieved, it might mean that higher-risk patients are being avoided.11 How much time, energy, and money should go into, for example, delivering vaccines or preventing complications? There is no single right answer.10

When standard-setting organizations, such as accreditation groups, government regulators, and "report card" producers, want to measure performance but do not have a gold standard, they often do three things: delineate baseline norms, suggest goals, and require procedures that ensure that complex issues are given appropriate attention. Adherence to community norms, progress in reaching aspirational goals, and use of acceptable mechanisms for handling complex and important problems are all, fortunately, measurable.

This approach can also work for ethical domains. In fact, this is the approach that has been taken when ethical issues (such as patient involvement in biomedical research) have been included by standard-setting organizations. First, certain baseline standards are set (e.g., research that involves more than minimal risk may be done only with signed consent). Second, some aspirational goals are given (e.g., a goal for all research participants is a complete understanding of the potential risks, benefits, and alternatives to the research to which they are consenting). Third, mechanisms are established to ensure that proper attention is paid to these complex issues (e.g., the establishment of investigational review boards at research institutions that receive federal funds).12

In this context, performance measures for ethics are not idle speculation; current accreditation standards and legislation already address several important ethics quality domains. For example, some current standards protect the confidentiality of health information, establish and charge clinical ethics committees, and provide due process for grievances.13, 14 However, performance measures for these standards have not all been well validated, and other important domains of ethics have yet to be addressed. Furthermore, these ethics topics have not generally been regarded as a related set of quality domains, complete with interactions that sometimes present conflicting demands.13 Viewing ethics quality as a complex set of performance characteristics allows comparisons and a clearer view of important trade-offs.8, 15

Developing New Performance Measures for Ethics Quality

New performance measures for ethics should be incorporated into existing self-assessment programs, report cards, accreditation standards, and other quality measurement tasks and would be applicable in all health care delivery environments. However, developing these measures will require a collaborative effort among many health care participants. One ongoing process for developing new measures of ethics quality is described in the following discussion.

Table 2 provides examples of potential measures for a few suggested domains of ethics quality. The domains of ethics listed are derived from previous work16 and from an ongoing, structured review of ethics documents from health plans and health-related organizations.17 The suggested performance measures for ethics in Table 2 are controversial, perhaps even more so than for other performance measures. After all, whose values and norms should be used to develop and score such measures? Some organizations may decide to ignore performance measures for ethical standards with which they do not agree. Others, however, will change their self-expectations after learning that an ethical norm is more widely accepted than was previously believed. Thus, as with other quality measures, the development and implementation process can provoke both resistance and beneficial change. Beneficial changes are more likely to occur when measures are developed in a legitimate process and are carefully validated before being used for public reporting.

Careful development and validation of ethics performance measures is the purpose of the E-Force Program (Appendix), which uses a standardized process to develop and test new ethics quality performance measures. The E-Force Oversight Body selects domains of ethics for measure development and monitors the work of expert advisory panels, which are charged with developing and testing measures in one domain. The following staged process is being used by E-Force to develop and validate new ethics quality performance measures.

First, a multidisciplinary conference panel of experts assesses the new measures proposed by each expert advisory panel for their reasonableness (face validity) and ensures that the measures address the relevant contents of the ethical domain in question (content validity). For example, expert reviewers are now assessing measures in the domain of privacy and confidentiality. They are asked whether proposed measures are related to the ethical issue at hand and whether each measure addresses a salient part of the issue for relevant parties, such as practitioners, plans, or employers. The experts also consider proposed measures to assess whether the standards can realistically be achieved.

Measures surviving these challenges are subsequently tested in a selection of health care organizations. Several reviewers will be involved in these trials to test for interrater reliability, and an organization's performance on the new measures will be compared with its performance on some existing and related quality measures to assess the measures' convergent validity.

Finally, for some measures, it may be possible to assess criterion validity by comparing results on the new measures with relevant outcomes. However, ethics-related outcomes are not yet well developed. Outcomes of interest might include trust in the physician, health plan, hospital, or other care delivery organizations; agreement with the validity of adopted decision-making processes; and willingness to divulge sensitive information to organizational representatives. To date, few tools to measure such ethics-related outcomes have been developed,18 and further research by the E-Force Program is indicated in this area.

Conclusions

Each participant in health care delivery has an important stake in the ethical norms and values that are espoused by the other participants in health care. These reciprocal stakes necessitate the creation of valid and reliable methods to assess each party's fundamental ethical obligations. Although a few domains of ethics quality are already incorporated in quality measurement, ethics performance is not assessed in an organized way. In a collaborative and rigorous process, the E-Force Program is now attempting to discern and develop sets of new ethics performance measures.

References

1. Donabedian A. Explorations in Quality Assessment and Monitoring. Volume 1. Ann Arbor, MI: Health Administration Pr; 1980.

2. McGlynn EA. Choosing and evaluating clinical performance measures. Jt Comm J Qual Improv. 1998;24:470-9.

3. Kaiser/Harvard 1998 Survey of Americans' Views on the Consumer Protection Debate. Available at http://www.
kff.org/archive/health_policy.

4. Cleary PD, Edgman-Levitan S. Health care quality. Incorporating consumer perspectives. JAMA. 1997;278:1608-12.

5. Shortell SM, Waters TM, Clarke KW, Budetti PP. Physicians as double agents: maintaining trust in an era of multiple accountabilities. JAMA. 1998;280:1102-8.

6. Gray BH. Trust and trustworthy care in the managed care era. Health Aff. 1997;16:34-49.

7. Presidential Advisory Commission on Consumer Protection and Quality in the Health Care Industry. Final Report, Establishing Health Care Quality as a National Priority. Advancing Quality Measurement and Reporting. Washington, DC: US Gov Pr Office; 1998. Also available at http://www.
hcqualitycommission.gov/final/chap04.html.

8. Woodstock Theological Center. Ethical Issues in Managed Health Care Organizations. Seminar in Business Ethics. Washington DC: Georgetown Univ Pr; 1999:42.

9. McGlynn EA. Six challenges in measuring the quality of health care. Health Aff. 1997;16:7-21.

10. Tu JV, Naylor CD, Kumar D, DeBuono BA, McNeil BJ, Hannan EL. Coronary artery bypass graft surgery in Ontario and New York State: which rate is right? Steering Committee of the Cardiac Care Network of Ontario. Ann Intern Med. 1997;126:13-9.

11. Schneider EC, Epstein AM. Influence of cardiac-surgery performance reports on referral practices and access to care. A survey of cardiovascular specialists. N Engl J Med. 1996;
335:251-6.

12. Code of Federal Regulations (CFR) Title 46, Part 45, Subpart A. See also, Medical records privacy: access needed for research, but oversight of privacy protections is limited. GAO/HEHS-99-55; February 1999.

13. Joint Commission on Accreditation of Healthcare Organizations. Ethical Issues and Patient Rights: Across the Continuum of Care. Oakbrook Terrace, IL: Joint Commission on Accreditation of Healthcare Organizations; 1998.

14. National Committee for Quality Assurance. Accreditation 2000: Draft Standards for Accreditation of Managed Care Organizations and Managed Behavioral Healthcare Organizations. Washington, DC: National Committee for Quality Assurance; 1999.

15. Schyve P. Patient rights and organization ethics: the Joint Commission perspective. Bioethics Forum. 1996;12:13-20.

16.Emanuel LL. A professional response to demands for accountability: practical recommendations regarding ethical aspects of patient care. Working Group on Accountability. Ann Intern Med. 1996;124:240-9.

17. A pilot study analysis of codes of professional/medical ethics. Research Triangle Institute Project Number 7154-000. July 31, 1998.

18. Kao AC, Green DC, Zaslavsky AM, Koplan JP, Cleary PD. The relationship between method of physician payment and patient trust. JAMA. 1998;280:1708-14.

Disclaimer

The views expressed in this article are those of the author and may not reflect the official policies of the American Medical Association.

Correspondence

Matthew K. Wynia, MD, MPH, Assistant Vice President, Institute for Ethics, American Medical Association, 515 North State Street, Chicago, IL 60610; telephone: 312-464-4980; fax: 312-464-4613; e-mail: matthew_wynia@ama-assn.org.