Effective Clinical Practice
UNANSWERED QUESTIONS: RESEARCH UNDER CONSTRUCTION
Effective Clinical Practice, May/June 2001
James D. Sargent
For author affiliations, current addresses, and contributions, see end of text.
Context. The clinical management of children with attention deficit/hyperactivity disorder (ADHD) requires measurement of behavioral outcomes at home and at school. This information is difficult to obtain.
General Question. Could a Web site that facilitates exchange of information between parents, teachers, and health care professionals improve behavioral outcomes for children with ADHD?
Specific Research Challenge. Designing a pragmatic intervention and evaluation strategy.
Proposed Approach. Establish a secure Web site to share observations on a child with ADHD. Access to a child's electronic record will be limited to team members defined by the parents. The record will contain documentation of consent for team members to communicate both diagnostic and treatment information. The record will feature two methods of communication: 1) the ability to input and graphically view structured behavioral questionnaire data that measure levels of attention and hyperactivity/impulsivity, and 2) an electronic bulletin board that allows users to post and respond to text messages that specifically describe behavior. The Web site would be tested in a randomized trial.
Unanswered Questions. Given its complexity, which part of the intervention should be isolated for study? What is the appropriate unit of analysis (individual, practice, or school)? Can an unbiased assessment of behavior be obtained when parents and teachers are not blinded to intervention status?
The management of school problems in children and adolescents is a relatively new area of clinical practice that bedevils most practitioners. But with the explosion in the number of children who are being diagnosed with attention deficit/hyperactivity disorder (ADHD), it is becoming increasingly difficult to ignore. Educators are prompting parents to consider the diagnosis, and parents are increasingly seeking ways to improve their children's school performance. Both look to their local health provider for assistance. The typical scenario is a parent bringing their first grader to the doctor after the teacher has suggested that the child might have ADHD. In a 15-minute visit the practitioner is supposed to answer the question, "Does he?" Equally vexing are the multiple follow-up visits, during which the practitioner is supposed to answer the question, "Is the medication working?"
Answering these questions requires data that are not available in the office. Experts argue that behavioral data are needed both from observers in the home and at school. Setting up the systems to obtain these data (developing a form, sending it out, making sure it is returned, coding it, and synthesizing it), however, would be a nightmare for an individual practitioner. Moreover, the activity is not paid for (beyond the diagnostic visit). As a result, most primary care providers manage the condition by responding to parent reports. Unfortunately, what parents say about school behavior often differs from what teachers observe in the classroom.
In this article, I will offer one possible solution: a Web site for parents, teachers, and physicians to share and respond to observations on a child's behavior. The site will allow two categories of observation: 1) quantitative measurement on the level of inattention and hyperactivity (using standard psychometric instruments) and 2) qualitative description of specific behavior events (using free-text fields). I begin by briefly summarizing ADHD and move on to describe the proposed Web site.
What is ADHD?
ADHD is a term we apply to children (or adults) who exhibit a group of behaviors characterized by inattention, hyperactivity, and/or impulsivity. Inattention is when a person is not able to concentrate on an important sensory input (e.g., when the teacher is giving out the homework assignment). Hyperactivity is a term applied to excessive motor activity (e.g., when a child is frequently jumping up out of his seat). Impulsivity is when a person acts without appropriate forethought or planning (e.g., when a child speaks out in class without raising his hand). To satisfy Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV) criteria for ADHD, at least six symptoms of inattention or at least six symptoms of hyperactivity/impulsivity shown in Table 1 must be present. (1, 2) In addition, these symptoms must have been present before age 7 and must cause significant impairment in at least two settings (e.g., academic, social, or occupational functioning). (3) Finally, these symptoms should not occur during the course of other mental disorders (e.g., children with clinical depression will often manifest many inattentive symptoms). (3)
ADHD is a term applied to a constellation of behaviors--not surprisingly, there is controversy over its cause. The mainstream medical and psychiatric communities consider ADHD as a disorder of brain function, arguing that it should be treated with medication the same as other mental disorders, such as psychosis or depression. (4-8) However, ADHD symptoms tend to improve as children mature and gain self-regulation skills, suggesting that with most children ADHD is not a chronic disease process. Moreover, the high prevalence of these behaviors in preadolescent boys suggests that it is probably part of the normative behavior spectrum. Indeed, some argue that ADHD is not a disorder but a behavioral style that causes dysfunction in certain settings like the school classroom. (9) Others feel that ADHD is not a disorder but a common set of symptoms in children with other learning disorders. (10)
Who Has ADHD?
ADHD is common, occurring in about 5% to 8% of school-age children. (11) Boys are around five times more likely than girls to be diagnosed with the disorder. (12) The number of children being treated for ADHD is rising. Whether this is due to an actual increase in the prevalence of the disorder or greater recognition of it is unclear. (13) From 1971 to 1987, the percentage of public elementary school students in Baltimore receiving medication increased 5-fold, to nearly 6%. (14) From 1990 to 1993 the number of outpatient visits for ADHD rose nearly 25-fold to 4,195,000. (13) In general, the symptoms of ADHD diminish as children age, by about 50% every 5 years between the ages 10 and 25. (15) In some children, however, the symptoms of ADHD persist through adolescence and into adulthood. There are no good criteria to guide practitioners in determining when to withdraw medication once treatment has begun.
A diagnosis of ADHD is associated with a high level of uncertainty because of its complexity and because the behavior criteria are environment- and observer-dependent. Children change their behavior in response to their environment. If a child who tends to show ADHD symptoms is placed in a structured and orderly home and school environment, chances are good that many of the symptoms will subside (or at least not develop into "problem behavior"). In addition, different observers have different levels of tolerance, especially for hyperactive/impulsive behavior. Teachers who foster dynamic, interactive classrooms generally have a higher tolerance for active students and are less likely to report symptoms. The same is true for the children's parents and the households in which the children live. Such observer and environment variations can result in different evaluations of the same child, and there are no clear guidelines for how to proceed when, for example, a parent thinks her child has ADHD but the teachers do not. (16)
Treatment of ADHD
There are two general approaches to ADHD: behavioral and medical management. For children, behavioral management involves the development and implementation of behavior strategies at home and at school. This plan typically includes regular communication between teachers and parents (e.g., a daily report card) and develop consistent responses to behavior (e.g., a "time-out" for inappropriate actions). (15, 17) In addition, there are many techniques for controlling classroom behavior, some as simple as briefly placing a hand on the shoulder of a child who is getting restless to remind him to regulate his activity level.
Medical management, usually with stimulant medication, is more efficacious than behavioral management alone. (18) The benefits of stimulant medication are well documented in randomized clinical trials. (4, 15, 19-21) Most medication trials have limited follow-up, generally less than 6 months after initiation of therapy. The two studies that have evaluated efficacy beyond 6 months suggest that effects can be seen for 1 to 2 years. (18, 22) It is not clear at this time whether treatment with stimulants improves long-term educational outcomes or occupational attainment, or whether treatment alters the chances of developing conditions that are seen more frequently with ADHD, such as conduct disorders (when children engage in antisocial behavior).
Challenges in Management
One challenge is that ADHD needs to be managed by a team of persons (typically parents, teachers, school nurses, psychologists, and physicians). This ideal, however, is rarely realized. First, team members are often not identified. When the team is not defined, there is no expectation that communication will take place. Even if team members were clearly identified, it is difficult to get them together at the same time and place. Typically, members communicate in pairs over the phone or, say, at a parent-teacher meeting. In addition, many members of the team (e.g., teachers, nurses, and doctors) may care for several children with ADHD and other behavior disorders. Without a uniform template for how teams should function, the dynamic of each team is left to chance. This results in marked variation in the level of communication and effectiveness from child to child.
A second challenge is that while medications are managed from the clinic, the behavior being treated is observed in school or at home. This is in stark contrast to other chronic diseases, like diabetes and hyperlipidemia, that can be managed through observations made in the clinic (e.g., HbA1C and fasting cholesterol). Thus, the clinician is making management decisions based on an outcome that is exhibited in another time and context. Without measures of school and home behavior, it is impossible to determine treatment efficacy. Although it is possible to ask parents questions about the child's behavior at home, it is not possible to ask questions about school behavior without establishing a line of communication with the child's teacher. Many physicians manage their ADHD patients on the basis of parental reports of school behavior; unfortunately, parent and teacher reports of school behavior often conflict. (16)
Proposed Intervention: PDT Connection
To address both of these challenges, we propose a Web-based communications system PDT(parent-doctor-teacher) Connection--to assist in the management of children with ADHD. The key components of PDT Connection are illustrated in the proposed first page of the child's electronic record (Figure 1). This page allows the user to access other areas: parental permission (consent form), clinical record (problem list, medications, and behavior plan), behavioral data (allows user to enter or view behavior data), and information exchange (bulletin board allowing text communication).
Access and Team Membership
The parents and the primary care physician determine the ADHD management team, and the names of these individuals are posted in an electronic consent form, which allows them to exchange information. User access to the Web site would be password protected and limited to team members. Establishing a management team sets the expectation that team members will communicate with each other. The goal of the rest of the Web site is to facilitate communication.
Structured Behavioral Data: Input
We will design the Web site to gather structured data on behavior from ADHD team members who observe the child in school and at home, assessing the level of inattention and hyperactivity/impulsivity by using a psychometric questionnaire in the public domain. One example of this type of instrument is SNAP-IV (Swanson, Nelson, and Pelham), a questionnaire that assesses inattention and hyperactivity/impulsivity using the DSM items listed in Table 1. Users will be guided through the questionnaire, and these data will be logged into a behavior database for access by the data presentation section.
Structured Behavioral Data: Output
The structured behavioral data can aid diagnosis and treatment. The diagnosis itself might be brought into question if observers disagree about whether the child has six symptoms of inattention or hyperactivity/impulsivity. To evaluate treatment, it is important to know how the severity of the symptoms changes over time. Figure 2 shows how regularly collected hyperactivity/impulsivity symptoms from the SNAP-IV might appear when graphed over time. The process might again highlight important differences among observers.
Standardized questionnaires provide one important form of summary information but cannot convey the richness of the child's experiences at school or home. One solution is to allow team members to post messages on a secure electronic bulletin board. The bulletin board allows teachers and parents to communicate specific behavioral events and/or to elicit a response from one or more members of the ADHD management team. The communication is asynchronous; the messages can be read and responded to by team members at their convenience. Text messages with a certain purpose could also include a preferred structure. For example, a behavioral mention (Figure 3) might be used by teachers and parents to report a notable behavior. Because messages on an electronic bulletin board remain accessible over time, this area should evolve into a chronological description of the child's behavior.
Reminders and Prompts
Users need to be told when it is time to use the Web site. Teachers and parents need to be reminded to enter behavioral data periodically. Other team members need to be prompted to look at the bulletin board after an entry has been made. We will design an e-mail message system for this task. To keep information secure within the site, the e-mail message will be generic, directing the user to his or her area within PDT Connection where tasks are listed (and removed from the list after completion). When registered users enter the Web site, they are given access to their work list page, which lists the patient records that need to be updated or read.
We envision that the reminder protocols for the structured behavior data would be established by the treating clinician. The frequency for completing the behavior questionnaire would probably vary. For example, during a medication trial, the clinician might require weekly follow-up behavior data for a short period and use PDT Connection to evaluate response for different doses of stimulant medication. We imagine two basic options for follow-up reminders: One will be a short-term, weekly follow-up option that is time-limited; the other will be a longer term (i.e., monthly) option for regular behavior surveillance.
Challenges in Evaluation
PDT Connection will be developed using an iterative process in which a team of potential users responds to various preliminary versions of the Web site. Next, we would conduct a pilot test in 10 to 20 children during a school year to see if people find it useful and to work out the bugs. In this concluding section, we assume that these stages of development have been successfully completed and consider the major issues in designing a randomized trial (Table 2).
Which Part of the Intervention Should Be Isolated?
The first issue arises because the Web site is a multimodal intervention. There are several communication features of the site that could affect behavior--the determination of an ADHD management team (setting an expectation of communication), posting pertinent diagnostic and treatment information in a place where everyone can see it, gathering and viewing behavioral data, and exchanging information through the free-text bulletin board. The last two factors are of greatest interest to us. Facilitating the gathering and viewing of structured behavioral data tests whether regular evaluation and feedback of quantitative data to ADHD team members improve outcomes. The information exchange tests whether increasing interactive communication between team members improves outcomes. This activity could become burdensome for a team member because of the amount of time that could become involved in reading and responding to multiple messages. Ideally, one would like to know whether exchanging either structured or free-text data is useful and whether adding both together results in an additive or an interactive effect. This could be tested through a factorial design, as depicted in Figure 4. Randomization would be blocked on these two factors, yielding four groups: control (in which the outcomes are assessed regularly for the purposes of evaluation, but clinical care is not intervened with), the Web site with behavioral data communication only, the Web site with free-text bulletin board communication only, and the Web site with both.
What Is the Unit of Analysis?
The next question involves the level of randomization. This trial is complicated by the fact that randomization could take place on several levels: the individual or group, and if the group level is chosen it could be randomized at the level of the practice or the school. Each has advantages and disadvantages. Randomization at the individual level maximizes power to detect an effect of the management system on the child. However, for this level of randomization, multiple users could be assigned to multiple arms of the intervention, increasing the possibility of contamination. For example, communicating through the bulletin board about the behavior of one student could affect the teacher's approach to another student assigned to the control group. To the extent that experience with the intervention causes a team member to act differently toward a patient in another treatment category, the intervention effect is biased toward the null. On the other hand, even if teachers are sensitized to the care of all of their ADHD children, they will have access to PDT Connection for only some. To the extent that use of the system allows for effective communication, resulting in better care, this should result in measurable improvement in behavior despite some contamination.
Concern about contamination makes us favor randomization at the group level over the individual level. Group randomization does, however, require a larger sample size to detect a given treatment effect (see Primer on Group Randomized Trials January/February ecp (23)). The groups being randomized here could be either practices or schools.
Randomization by practice has at least two advantages. First, we are familiar with the challenge children with ADHD pose to health providers and feel that it would be easier to recruit clinical practices than schools. There are research networks of pediatric practices (Pediatric Research in Office Settings [PROS]) that could collaborate with this project. In addition, as health professionals, we are more interested in developing efficiencies in the medical system of care and randomization at the practice level that would allow us to look at the implications of this intervention for a practice. In contrast, schools would have to be recruited individually because to our knowledge there is no similar research consortium currently in place in the education system. One advantage of school-level randomization is in the economy of hiring an unbiased observer in the school to assess behavior outcomes for multiple children.
What Are the Outcomes?
Finally, we have to determine outcomes and find an unbiased way to measure them. For children with ADHD, we would expect the management system to effect short- and medium-term outcomes. Short-term outcomes include levels of hyperactivity and impulsivity in the classroom and ability to complete such tasks as homework and class assignments. Medium-term educational outcomes include achievement test scores and grades. School function has a large impact on a child's sense of worth, so another medium-term outcome would be self-esteem and motivation to do well in school. In addition to functional status of the child, there are several important process outcomes. The system must meet the approval of parents, teachers, and health professionals. It is not clear at this time how much additional time would be required of PDT Connection users or whether the possible benefits in patient care are worth the time investment. In addition, will parents be comfortable with posting confidential information on an Internet Web site? These aspects of user satisfaction are key elements of ultimate success.
Observation bias in assessing behavior would be another major issue for this randomized trial. Teachers and parents typically evaluate behavior of children with ADHD; this type of assessment can work well in a medication trial when everyone is blinded to intervention status. In this trial, however, parents and teachers are an integral part of the intervention and cannot be blinded. Could participation in the study bias their behavioral and educational assessments? Absolutely. Thus, we could not be certain that the intervention has resulted in a true behavior or educational benefit for the child. To ensure unbiased assessment of school behavior, someone not involved in the ADHD management team will have to determine behavior outcomes periodically. Although observer bias is a problem in assessing efficacy, it may be an advantage in the dissemination of PDT Connection. Communication among satisfied users about a new technology motivates others to adopt it and is thus a key element in marketing.
Web-based information management systems for children with ADHD are a potentially powerful and convenient way to address communication barriers between teachers, parents, and physicians. The advantages of a Web-based communications system include easy access from the home and work environment, the ability to communicate behavior through structured questionnaires and text messages without having to schedule mutually convenient times, and the ability to compute and present structured behavior data over time. Despite these advantages, practitioners will have to address concerns about privacy and find ways to show that these systems do indeed improve care before they will be adopted widely.
|Take Home Points
1. Gomez R, Harvey J, Quick C, Scharer I, Harris G. DSM-IV AD/HD: confirmatory factor models, prevalence, and gender and age differences based on parent and teacher ratings of Australian primary school children. J Child Psychol Psychiatry. 1999;40:265-74.
2. Lahey B, Applegate B, McBurnett K, et al. DSM-IV: field trials for attention deficit hyperactivity disorder in children and adolescents. Am J Psychiatry. 1993;151:1673-85.
3. Frances A, First M, Pincus H. DSM-IV Guidebook. Washington, DC: American Psychiatric Pr;1995.
4. Hinshaw S, Klein R, Abikoff H. Childhood attention-deficit hyperactivity disorder: nonpharmacologic and combination treatments. In: Nathan PE, Gorman JM, eds. A Guide to Treatments That Work. New York: Oxford University Pr; 1998:26-41.
5. Jadad AR, Boyle M, Cunningham C, Kim M, Schachar R. Treatment of Attention-Deficit/Hyperactivity Disorder. Rockville: Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services;1999.
6. Richters JE, Arnold LE, Jensen PS, et al. NIMH collaborative multisite multimodal treatment study of children with ADHD: I. Background and rationale. J Am Acad Child Adolesc Psychiatry. 1995;34:987-1000.
7. Committee on Quality Improvement. Subcommittee on Attention-Deficit/Hyperactivity Disorder. Clinical practice guideline: diagnosis and evaluation of the child with attention-deficit/hyperactivity disorder. Pediatrics. 2000;105:1158-70.
8. Barkley RA. Attention-Deficit Hyperactivity Disorder: A Handbook for Diagnosis and Treatment, ed 2. New York: Gilford Pr;1998.
9. Carey WB. Problems in diagnosing attention and activity. Pediatrics. 1999;103:664-7.
10. Levine M. Educational Care. Cambridge, MA: Educators Publishing Service;1994:13.
11. Green M, Wong M, Atkins D, et al. Diagnosis of attention deficit/hyperactivity disorder. Rockville, MD: Agency for Health Care Policy and Research;1999.
12. Arnold LE. Sex differences in ADHD: conference summary. J Abnorm Child Psychol. 1996;24:555-69.
13. Swanson JM, Lerner M, Williams L. More frequent diagnosis of attention deficit-hyperactivity disorder [Letter]. N Engl J Med. 1995;333:944.
14. Safer DJ, Zito JM, Fine EM. Increased methylphenidate usage for attention deficit disorder in the 1990s. Pediatrics. 1996; 98:1084-8.
15. Goldman LS, Genel M, Bezman RJ, Slanetz PJ. Diagnosis and treatment of attention-deficit/hyperactivity disorder in children and adolescents. JAMA. 1998;279:1100-7.
16. Swanson J, Lerner M, March J, Gresham FM. Assessment and intervention for attention-deficit/hyperactivity disorder in the schools: Lessons from the MTA study. Pediatr Clin North Am. 1999;46:993-1009.
17. Pelham W, Wheeler T, Chronis A. Empirically supported psychosocial treatments for attention deficit hyperactivity disorder. J Clin Child Psychol. 1998;27:190-205.
18. Group TMC. A 14-month randomized clinical trial of treatment strategies for attention-deficit/hyperactivity disorder. Arch Gen Psychiatry. 1999;56:1073-86.
19. Pelham W, Murphy H. Behavioral and pharmacological treatment of hyper-activity and attention-deficit disorders. In: Herson M, Breuning S, eds. Pharmacological and Behavioral Treatment: An Integrative Approach. New York: Wiley;1986: 108-47.
20. Spencer T, Biederman J, Wilens T, Harding M, O'Donnell D, Griffin S. Pharmacotherapy of attention-deficit hyperactivity disorder across the life cycle. J Am Acad Child Adolesc Psychiatry. 1996;35:409-32.
21. Swanson J, McBurnett K, Wigal T, et al. Effect of stimulant medication on hyperactive children: a review of reviews. Exceptional Children. 1993;60:154-62.
22. Gillberg C, Melander H, von Knorring AL, et al. Long-term stimulant treatment of children with attention-deficit hyperactivity disorder symptoms: a randomized, double-blind, placebo-controlled trial. Arch Gen Psychiatry. 1997;54:857-64.
23. Primer on group randomized trials. Eff Clin Pract. 2001;4:42-3.