xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam 22 Ka 4500
controlfield tag 007 cr-bnu---uuuuu
008 s2009 flu s 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0003300
An exploratory analysis of the ecological validity of a performance-based assessment of attention
h [electronic resource] /
by Eun-Yeop Lee.
[Tampa, Fla] :
b University of South Florida,
Title from PDF of title page.
Document formatted into pages; contains X pages.
Dissertation (Ph.D.)--University of South Florida, 2009.
Includes bibliographical references.
Text (Electronic dissertation) in PDF format.
Mode of access: World Wide Web.
System requirements: World Wide Web browser and PDF reader.
ABSTRACT: Executive functions impact everyday functioning. An individual's ability to adapt to and navigate their physical and social environments is largely determined by the ability to organize oneself, to plan and to coordinate activities. Despite the wide variety of cognitive tests that assess various aspects of executive function, there has been little work to validate the use of these measures in predicting real world functioning (Sbordone, Seyranian, & Ruff, 2000), particularly in children where characterization of executive function is less specified. Evaluating the ecological validity of neuropsychological tests has become an increasingly important topic over the past decade (Chaytor & Schmitter-Edgecombe, 2003). Ecologically valid assessments of executive function and attentional deficits provide insight into deficits related to the child's everyday adaptive functioning, which can assist in identifying targets for interventions. Although many performance based measures and caregiver behavior checklists exist for assessing a wide range of behaviors and adaptive functioning skills in children, comprehensive measures of executive functions are relatively new and largely unexplored. The purpose of this study was to investigate and to define better the relationship between attention and corresponding behaviors that represent executive functions and social/adaptive functioning. More specifically, this study sought to explore the correlation between ratings of varying subcomponents of attention (e.g., selective attention, sustained attention, and attentional control/switching), executive function behaviors, and ratings of social/adaptive functioning. Additionally, gender considerations were examined with aims to determine how this factor may affect the degree of relationship between the proposed variables. Results of multiple regression and correlational analyses revealed the ability of child attentional performance to predict executive function and social/adaptive functioning behaviors. As parent/caregiver and teacher ratings of executive function behaviors increased thus noting adept skills in these areas of functioning child performance on measures of selective attention, sustained attention, and attentional control/shifting were also reported to improve. Future research should continue to explore the construct validity, positive predictive power, negative predictive power, diagnostic sensitivity and specificity of the Test of Everyday Attention for Children (TEA-Ch).
Advisor: Harold Keller, Ph.D.
School age children
x Psychological and Social Foundations
t USF Electronic Theses and Dissertations.
An Exploratory Analysis of the Ecological Validity of a Performance-Based Assessment of Attention by Eun-Yeop Lee A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy Department of Psychological and Social Foundations College of Education University of South Florida Major Professor: Harold Keller, Ph.D. Richard Marshall, Ph.D. Kathy Bradley Klug, Ph.D. Kathleen Armstrong, Ph.D. Date of Approval: September 29, 2009 Keywords: executive functions, social/adaptive func tioning, school age children, neuropsychological tests Copyright 2009, Eun-Yeop Lee
i Table of Contents List of Tables iv Abstract vi Chapter 1 Â– Introduction 1 Statement of the Problem 1 Definition of Executive Functions 4 Neuroanatomical Substrate of Executive Functions 6 Developmental Trajectory of Executive Functions 7 Validity of Executive Functions in Children 12 Construct Validity of the Assessment of Executive Functions 13 The Need for Measures 14 Conceptualization of Attention 15 Anatomical Structures Involved in the Control of A ttention 17 Subcomponents of Attention 17 Differential Assessment of Attention 19 Research Questions 21 Significance of the Study 23 Chapter 2 Â– Review of the Literature 26 Introduction 26 Executive Function Deficits and Academic Outcomes 28 Executive Functions and Behavioral Implications 30 Relationship between Executive Functions and Daily Functioning 33 Models of Attention 35 Relationship of Attentional Deficits to Future Out comes 37 Relationship of Attention to Behavior and Academic Achievement 40 Executive Function in Attention-Impaired Groups 43 Guidelines for the Assessment of Attention and Exe cutive Functions 44 Developmental Changes in the Assessment of Executi ve Functions and Attention 49 Gender Considerations in the Assessment of Attenti on and Executive Functions 54 Ecological Validity of Performance-Based Tests 56 Performance Measures and Rating Scales 58
ii Parent and Teacher Reports 60 Ecological Validity of the BRIEF and Assessment of Attention 63 TEA-Ch and the Assessment of Attention 65 Purpose of the Study 69 Chapter 3 Â– Method 71 Participants 71 Inclusion Criteria 73 Exclusion Criteria 73 Research Design 75 Instrumentation 76 Behavior Rating Inventory of Executive Function ( BRIEF) 76 Adaptive Behavior Assessment System2nd Edition (ABAS-II) 81 The Test of Everyday Attention for Children (TEACh) 86 Procedures 93 Ethical Considerations 93 Training Activities 94 Training of Test Administration 94 Recruitment of Schools for Participation 96 Parent Data Collection 97 Teacher Data Collection 98 Student Data Collection 98 Integrity Checklist 99 Scoring of Protocols 100 Data Analyses 100 Internal Reliability 101 Univariate and Bivariate Analyses 102 Multiple Regression Analysis 104 Research Question 1 104 Research Question 2 105 Research Question 3 106 Research Question 4 107 Chapter 4 Â– Results 108 Data Screening 110 Research Questions 111 Research Question 1 111 Research Question 2 115 Research Question 3 120 Research Question 4 121 Data Analysis 121 Descriptive Statistics 122 ANOVA Results 122 Summary 124 Chapter 5 Â– Discussion 126
iii Overview of Study Objectives 126 Attentional Performance and Executive Function Beh aviors 127 Attentional Performance and Social Adaptive Functi on Behaviors 130 Executive Functions and Social/Adaptive Behavior 1 31 Gender Effects within TEA-Ch Measures 133 Implications 135 Early Intervention and Prevention 139 Limitations 141 Future Directions for Research 144 References 147 Appendices 163 Appendix A: TEA-Ch Subtest Descriptions 164 Appendix B: TEA-Ch Administration/Scoring Guideline 166 Appendix C: Integrity Checklist 168 Appendix D: Consent and Assent Forms 169 About the Author End Page
iv List of Tables Table 1 Distribution of Demographic Variables Acros s Child Participants 73 Table 2 Protocol of Power Analyses 75 Table 3 Overview of TEA-Ch Training Activities and Data Collection 95 Table 4 CronbachÂ’s Alpha Scores for Each Questionna ire 102 Table 5 Sample Sizes, Means, and Standard Deviation s of Variables 103 Table 6 Intercorrelations Between TEA-Ch Subcompone nts and BRIEF-Parent and Teacher (GEC) Forms 110 Table 7 Multiple Regression Examining the Relations hip of Each TEA-Ch Variable to BRIEF-Parent GEC Scores While Holding All Other Variables Constant 112 Table 8 Forward Stepwise Multiple Regression Examin ing the Relationship of Creature Counting and Sky Search DT to BRIEF-Parent GEC Scores 112 Table 9 Multiple Regression Examining the Relations hip of Each TEA-Ch Variable to BRIEF-Teacher GEC Scores Whil e Holding All Other Variables Constant 114 Table 10 Forward Stepwise Multiple Regression Exami ning the Relationship of Creature Counting and Score! to BRIEF-Teacher GEC Scores 114 Table 11 Intercorrelations Between TEA-Ch Subcompon ents and ABAS-II-Parent/Caregiver and Teacher (GAC) Versio ns 116 Table 12 Multiple Regression Examining the Relation ship of Each TEA-Ch Variable to ABAS-II Parent/Caregiver GAC S cores While Holding All Other Variables Constant 116
v Table 13 Forward Stepwise Multiple Regression Exami ning the Relationship of Creature Counting to ABAS-II-Pare nt/ Caregiver GAC Scores 117 Table 14 Multiple Regression Examining the Relation ship of Each TEA-Ch Variable to ABAS-II Teacher scores GAC Sc ores While Holding All Other Variables Constant 11 9 Table 15 Forward Stepwise Multiple Regression Exami ning the Relationship of Score! and Creature Counting to ABAS-II-Teacher GAC Scores 119 Table 16 Intercorrelations Between ABAS-II-Parent/C aregiver and Teacher (GAC) Versions and BRIEF-MI and BRI Parent/Teacher Forms 121 Table 17 Means and Standard Deviations for Dependen t Variables by Gender 122 Table 18 Results of ANOVA for Each Dependent Variab le 123
vi An Exploratory Analysis of the Ecological Validity of a Performance-Based Assessment of Attention Eun-Yeop Lee ABSTRACT Executive functions impact everyday functioning. An individualÂ’s ability to adapt to and navigate their physical and social environme nts is largely determined by the ability to organize oneself, to plan and to coordinate acti vities. Despite the wide variety of cognitive tests that assess various aspects of exec utive function, there has been little work to validate the use of these measures in predicting real world functioning (Sbordone, Seyranian, & Ruff, 2000), particularly in children where characterization of executive function is less specified. Evaluating the ecologic al validity of neuropsychological tests has become an increasingly important topic over the past decade (Chaytor & SchmitterEdgecombe, 2003). Ecologically valid assessments of executive function and attentional deficits provide insight into deficits related to t he childÂ’s everyday adaptive functioning, which can assist in identifying targets for interve ntions. Although many performance based measures and caregiver behavior checklists ex ist for assessing a wide range of behaviors and adaptive functioning skills in childr en, comprehensive measures of executive functions are relatively new and largely unexplored. The purpose of this study was to investigate and to define better the relationship between attention and corresponding behaviors that represent executive functions and
vii social/adaptive functioning. More specifically, thi s study sought to explore the correlation between ratings of varying subcomponents of attenti on (e.g., selective attention, sustained attention, and attentional control/switching), exec utive function behaviors, and ratings of social/adaptive functioning. Additionally, gender c onsiderations were examined with aims to determine how this factor may affect the de gree of relationship between the proposed variables. Results of multiple regression and correlational an alyses revealed the ability of child attentional performance to predict executive function and social/adaptive functioning behaviors. As parent/caregiver and teac her ratings of executive function behaviors increased thus noting adept skills in the se areas of functioning child performance on measures of selective attention, sus tained attention, and attentional control/shifting were also reported to improve. Fut ure research should continue to explore the construct validity, positive predictive power, negative predictive power, diagnostic sensitivity and specificity of the Test of Everyday Attention for Children (TEA-Ch).
1 Chapter 1 Introduction Statement of the Problem Current literature supports that various attentiona l capacities provide the basis for many of the cognitive and neuropsychological functi ons that are required for everyday operations (Cooley & Morris, 1990; Heaton et al., 2 001; Price, Joschko & Kerns, 2003, Stavro, Ettenhofer & Nigg, 2007). The ability to at tend plays a critical role in the individual expression of cognitive and behavioral f unctioning, thus exerting considerable influence on academic and social development. Despi te the importance of good attention skills, Â“poor concentrationÂ” is a relatively common problem in childhood (WarnerRogers, Taylor, Taylor & Sandberg, 2000). Estimates indicate that 10-15% of the general population report to experience clinically signific ant levels of attention problems (Heaton et al., 2001; Mirksy, Anthony, Duncun, Ahearn & Kel lam, 1991). Impairment of attention is also characteristic of m any other disorders including numerous psychiatric and neurological disorders. Th ese conditions may include transient and reversible manifestations of neurologic conditi ons including traumatic brain injury, response to medication, and withdrawal states or pr ogressive impairments including ParkinsonÂ’s disease and neurodegenerative dementias (e.g, diffuse Lewy body disease). Lastly, attention deficits are also present in more static conditions including major affective disorders, anxiety disorders, sleep disor ders, and various developmental
2 disorders including AspergerÂ’s Syndrome, TouretteÂ’s Syndrome, and TurnerÂ’s Syndrome (Coffey, McAllister & Silver, 2006; Cohen et al., 2 001; Friedman et al., 2007; Heaton et al., 2001; Manly et al., 2001). However, no other c hildhood psychiatric disorder manifests the degree of impact on attention than th at experienced by individuals diagnosed with Attention-Deficit/Hyperactivity Diso rder (ADHD) (Heaton et al., 2001). ADHD is one of the most prevalent neurodevelopmenta l disorders, occurring in approximately 7% of school age children, and 5% of adolescents and adults (Pasini, Paloscia, Alessandrelli, Porfirio & Curatolo, 2007; Sciutto & Eisenberg, 2007). Other studies have reported estimates ranging as high as 16% of the general population as meeting diagnostic criteria (Shafritz, Marchione, G ore, Shaywitz & Shaywitz, 2004). On average, estimates place at least one child diagnos ed with ADHD in every classroom in America (Fabiano & Pelham, 2003). In addition, clin ic referrals for ADHD reportedly consume 30-40% of resources in child psychopatholog y (Stavro et al., 2007). This disorder has become a significant public health iss ue affecting education, employment, social interactions, adaptive functioning, and over all quality of life (Heaton et al., 2001; Stavro et al., 2007). Long-term consequences have d ocumented lower educational, behavioral, and occupational achievement as well as increased risk and vulnerability for the development of additional psychiatric disorders (Barkley, 1997; Shafritz, et al., 2004; Stavro et al., 2007). The impact of this disorder c reates an intense need for support from children, families, schools, and mental health serv ices (Lorys, Hynd & Lahey, 1990). Given the prevalence of ADHD, it is not surprising that considerable research efforts have been devoted to the etiology, diagnosi s, and clinical manifestations of this disorder (Biederman & Faraone, 2005). Recently, the dynamic, multidimensional nature
3 of executive functions has been hypothesized to be characteristically impaired in individuals meeting criteria for ADHD (Barkley, 199 7; Stavro et al., 2007). To date, the construct of executive functions has been challengi ng to define and assess in clinical settings (Barkley, 1997; Stuff & Alexander, 2000). In the past, the symptoms of ADHD have been described in behavioral terms incorporati ng inattention, hyperactivity, and impulsivity tendencies. Despite the vast research, consensus fails to be reached concerning a specific neurocognitive mechanism attr ibutable to the behavioral problems of ADHD (Wu, Anderson & Castiello, 2002). Currently prevailing theories conceptualize ADHD as a neurologically based disord er characterized by deficits in executive function as well as weaknesses in sustain ed and divided attention (Biederman & Faraone, 2005; Mullane & Corkum, 2007; Pasini et al., 2007). Although the emphasis on studying executive functions has been greatly hi ghlighted due to an interest in further understanding and redefining ADHD, there is an incr easing normative population of children who struggle behaviorally and academically (Barkley, 1997; Stavro et al., 2007). Therefore, difficulties with attention are not nece ssarily confined to clinical populations (Mirsky et al., 1991). The purpose of this study was to investigate and be tter define the relationship between attention and corresponding behaviors that have been designed to represent executive functions and social/adaptive functioning Research is provided to support the integration of the neurosciences with the field of education by defining the relevant role of executive functions and attention as it relates to learning and functioning across home, school, and community settings (Meltzer, 2007). Sci entific understanding of the mind and brain is advancing quickly and societyÂ’s need to im prove the quality of education is a
4 reoccurring concern. Naturally, there is a great in terest in applying findings from brain research to guide educational practice (Meltzer, 20 07). The following sections provide an overview of executive functions and attention as well as the study and assessment of these constructs as it applies to child populations Definition of Executive Functions Researchers have proposed multiple models of execu tive functions with varying degrees of overlap however; a specific definition o f the construct remains elusive. The term Â“executive functionÂ” was initially described w ithin the context of cognitive theory and in the past twenty years has become the focus o f widespread research interest, particularly in children (Denckla, 1996; Espy, Kaum ann, Glisky & McDiarmid, 2001; Hughes & Graham, 2002). Difficulties with documenti ng the role of executive function, beyond heterogeneity of individual profiles, are pa rtially attributed to the breadth of functions and developmental dynamics of what consti tutes executive function (Meltzer, 2007). Executive function is best understood as a b road umbrella term governing a collection of separate but inter-related processes that are necessary for completing purposeful, goal-directed behaviors (Anderson, 2002 ; Hughes & Graham, 2002; Weyandt, 2005). The construct of executive function includes all s upervisory or self-regulatory functions that organize and direct cognitive, emoti onal, and behavioral functions towards attaining future goals (Anderson, 1998; Anderson, 2 002; Brocki & Bohlin, 2006; Gioia, Isquith, Guy & Kenworthy, 2001; Hughes, 2002; Pasin i et al., 2007; Robbins, 1992). Among these functions, four discrete but inter-rela ted executive domains can be defined: attentional control, information processing, cognit ive flexibility, and goal setting
5 (Anderson, 2002). Furthermore, within these compone nts are a variety of proposed underlying processes, subcomponents, or subdomains (Gioia et al., 2001) including a range of theoretical constructs such as, anticipati on, goal selection, planning, set-shifting, reasoning, initiation, self-regulation, inhibition, attention, and utilization of feedback (Anderson, 2002; Barkley, 1997; Biederman et al., 2 004; Gioia et al., 2000; Robbins, 1992). Specifically, these functions are proposed t o direct and modulate attentional processes such as sustaining optimal levels of arou sal and vigilance (Barkley, 1997). Gioia and Isquith (2004) also include the role of cognition in their interpretation of executive function. They suggest that emotional control and regulation of oneÂ’s affective state is reciprocally related to efficien t problem solving (Gioia & Isquith, 2004). Similarly, executive functions have also been assoc iated with involvement in guiding socially useful, personally enhancing, constructive and creative activities (Anderson, Anderson, Northam, Jacobs & Mikiewicz, 2002). Thus, Â“executive dysfunctionÂ” may be reflected in test performance as evidenced by poor planning/organization, perseveration, inability to correct errors or utilize feedback, an d rigid or concrete thought processes (Anderson, 1998). Apart from the vastly inclusive subdomains and rel ated tasks of executive functions, various models tend to incorporate sets of common attributes. Upon reviewing the terminology and definitions related to executiv e function that exist in current research a general set of beliefs appear to be universally a ccepted. These core features state that: (1) executive functions are primarily localized and supported by the prefrontal cortex, (2) executive functions follow a developmental traj ectory that is mediated by the environment, and (3) there is no unitary condition of executive dysfunction, but rather
6 distinct executive function profiles that may prese nt in different clinical conditions (Gioia, 2000). These features will be revisited and explored in detail upon review of the neuroanatomical structures involved in specific exe cutive functions. Neuroanatomical Substrate of Executive Functions One common view of the neuroanatomic organization of executive functions is that they are located solely within the frontal lob es and specifically in the prefrontal region (Anderson et al., 2002; Barkley, 1997; Gioia et al., 2000; Stavro et al., 2007). Neuropsychological evidence obtained from patients with frontal lobe lesions in conjunction with functional neuroimaging support th e hypothesis that the prefrontal cortex plays a major and specific role in response selection processes (Robbins, 1996). However, as functional anatomy research expands it is becoming apparent that constricting executive functions to the frontal lob e may be an oversimplification of the organizational execution of the brain. Thus, it is understood that although the frontal regions play a vital role in the mediation of perfo rmance intact executive function rests upon the integrity of the entire brain (Anderson, 1 998). The prefrontal region is an association region. An association area is a multimodal area that receives information from sensory areas a nd is involved in Â“higher orderÂ” functions such as perception, abstract thoughts, de cision-making, etc. The frontal association area lies in the frontal lobe and is in volved in creating general plans for actions that are activated through connections to t he primary motor cortex and basal ganglia (Gazzaniga, Ivry & Mangun, 2002). The prefr ontal region has extensive connections that span all areas of the neocortex vi a cortico-cortical projections as well as with limbic and subcortical structures including th e cingulate gyrus, hippocampus,
7 reticular activating system, basal ganglia, thalamu s, and motor system of the frontal lobes (Anderson et al., 2002; Dawson & Guare, 2004; Gioia et al., 2000; Weyandt, 2005). The prefrontal cortex is not a functionally homogenous region, which has implications for the nature and organization of executive functions (Rob bins, 1996; Weyandt, 2005). Importantly, damage or disorder involving any compo nent of the frontal system may interfere with the bidirectional connections of the prefrontal cortex, and in turn, influence performance of executive function tasks (Anderson e t al., 2002; Gioia et al., 2000). Developmental Trajectory of Executive Functions The assessment of executive functions in children stem from controversial roots. Historically, many researchers considered executive functions to be Â“functionally silentÂ” in children under the age of 12 years (Anderson, 19 98; Espy et al., 2001). Such beliefs were aligned with the popular perception that child ren lack inhibitory control, are easily distractible, and have difficulty shifting from one cognitive task to another (Anderson, 1998). However, as current research strongly suppor ts, executive functions although not present in their fully developed form can be measur ed across the early life span. Similar to the assessment of other cognitive skills such as language, developmentally appropriate tasks must be used that take into account the more limited behavioral repertoire of young children (Espy et al., 2001). Particularly for chil dren, if executive abilities and attention can be reliably assessed prior to school entry or d uring early school years, early intervention can be accessed to reduce the adverse impact on future outcomes (Espy et al., 2001). As previously discussed, frontal lobe functioning appears to play a central and pervasive role in human cognition as it serves to o rganize and modulate higher brain
8 functioning (e.g., reasoning, abstraction, emotions behavior) (Anderson, 1998; Espy et al., 2001; Gazzaniga, Ivry & Mangun, 2002). One of its main functions is to assist individuals with goal-directed and self-regulatory behaviors (Romine & Reynolds, 2005). The acquisition of abilities thought to be mediated by the frontal lobes emerge in childhood and continue to develop through late adol escence and into early adulthood, contrary to prior belief and in contrast to the ear lier maturation of other cortical regions (Romine & Reynolds, 2005). Neuroanatomical, neuroph ysiological, and neurochemical changes occur in the continued development of front al lobes throughout the lifespan (Romine & Reynolds, 2005). It is proposed that development of the frontal lob es follow a hierarchical pattern, consistent with processes such as dendritic arboriz ation, myelination, and synaptogenesis which progress through stages from primary, sensory association areas and lastly to frontal regions (Anderson, 1998; Chugani, 1999; Rom ine & Reynolds, 2005). Secondary and tertiary systems that involve language, learnin g, memory, emotion, cognition, and attention continue to develop beyond birth (Romine & Reynolds, 2005). These changes have been reported to parallel the development of c ognitive and social abilities observed during childhood and adolescence. The functional de velopments that are mediated by the frontal lobes have also been perceived to exist as a multistage process with different functions maturing at different rates (Chugani, 199 9; Romine & Reynolds, 2005). Researchers have attempted to define the differenti al components of executive functions and align them with their unique developmental traj ectories. A growing body of research describes the sequential improvement of performance of executive tasks through childhood that coincides with growth spurts observe d in frontal lobe development
9 (Anderson, 1998; Romine & Reynolds, 2005; Welsh & P ennington, 1988). For example, children develop attentional control initially from birth to five years, followed by the development of inhibitory control from three to fou r years. Working memory is proposed to develop by four or five years, cognitive flexibi lity emerges by seven to nine years, and more complex problem solving develops from 11 to 13 years with later proficiency and refinement of skills continuing to emerge through a dolescence and adulthood (Anderson, 2002; Espy et al., 2001). In a study of 100 pediatric participants, ages 3-1 2 years, Welsh et al. (1991) proposed that executive functions develop in three prominent stages of skill integration and maturation. Organized strategic planning behavi ors were detected by six years of age while adult-like performance on increasingly comple x measures of organized search ability, and utilization of hypothesis testing was evident by 10 years of age. Tasks of verbal fluency, motor sequence, and use of complex planning skills were proposed to be in continual development at the age of 12 years. De spite findings, a serious limitation of this study involved the use of measures that were o riginally designed for the assessment of executive functions in adults. Thus, they are un likely to have maintained adequate validity when task complexity was simplified for us e with children and adolescents (Welsh et al., 1991). Likewise, in their meta-analysis of frontal lobe f unctioning, Romine and Reynolds (2005) found that the greatest period of overall de velopment occurred between the ages of six and eight years. The capacity to shift betwe en response sets first emerged around four years of age and became more fluent by the age of six years (Espy et al., 2001). Moderate increases in skill level were proposed to be evident between the ages of 9-12
10 years and performance approximating adult levels wa s projected to occur between adolescence and the early 20s. Similar to previous findings, researchers reported that between the ages of five and eight years, basic cog nitive abilities were present and evident through performance on recognition memory, concept formation, set-shifting, and rudimentary planning skills (Romine & Reynolds, 200 5). Welsh et al. (1999) also documented rapid advances in systematic problem solving during this period. Thus, evidence supports the Â“5-7 year shiftÂ” that was first coined by White (1965), to refer to a transition pe riod characterized by childrenÂ’s increased ability to think autonomously and the eme rgence of strategic and controlled self-regulation, skills of inhibition, and the abil ity to maintain attention on complex problems, planfulness, and reflection (Welsh & Penn ington, 1988). By the age of 10 years, the ability to inhibit attention to distract ible stimuli and perseveratory responses were thought to be proficient with mastery achieved by 12 years of age (Romine & Reynolds, 2005; Stuss, 1992). Additional skills suc h as planning, visual working memory, coordination of working memory and inhibiti on, verbal fluency, and motor sequencing are skills mediated by the frontal lobes and require development beyond adolescence (Anderson, 1998; Romine & Reynolds, 200 5; Stuss, 1992). Processing speed was also proposed to increase during this period, a llowing for faster response rates and solution times, greater output, and commission of f ewer errors (Stuss, 1992). In sum, the developmental emergence and growth of executive functions have several important implications. First, executive fu nctions have demonstrated close associations with the prefrontal cortex, an area of the brain that was long thought to be functionally inactive until very late in developmen t (Hughes, 2002). Second, impairments
11 in executive functions are now perceived to play a key role in a range of developmental disorders. In addition, interest in the normative d evelopment of executive functions has also heightened which has provided an opportunity t o identify distinct executive functions. The research collaboratively lends suppo rt to the fact that there is no singular core disorder of executive function (Gioia et al., 2001) and rather clinical as well as normative groups may reflect unique profiles of exe cutive function deficits. Finally, the emergence of executive functions is understood to v ary across age specific groups of individuals and parallels the subsequent stages of development. Research has begun to examine and further delineate a time-related course for the development of specific executive subdomains (e.g., inhibitory control, att ention, shifting, cognitive flexibility, planning, and organizational skills). (Anderson, 19 98; Stuss, 1992; Romine & Reynolds, 2005). Likewise, cognitive models also support a hierarch ical view of development. Specifically, PiagetÂ’s theory of cognitive developm ent (Piaget, 1963) is highly compatible with current understanding of cerebral d evelopment, although it fails to provide reference to relevant neural substrates (An derson, 1998). PiagetÂ’s model consists of four sequential cognitive stages described as se nsorimotor (birth2 years), preoperational (two to seven years), concrete opera tional (seven to nine years), and formal operational (adolescence). It is worthy to n ote the close associations of timing between transitions of proposed cognitive stages an d growth spurts identified within the framework of executive function development. In par ticular, this research lends support to the importance of recognizing and examining exec utive functions within a
12 developmentally appropriate context allowing for as sessment of specific skills in varying aged populations of children (Anderson, 1998; Weyan dt, 2005). Validity of Executive Functions in Children Executive functions play a vital role in the develo pment of intellectual, academic achievement, adaptive/social functioning, and commu nication aspects of a childÂ’s life. Therefore, given the importance of developing skill s in childhood, measures that are suitable for use with children are essential (Ander son et al., 2002). As previously indicated, executive dysfunction is not represented by a homogenous pattern of behavior, but instead may be reflected in a diverse array of deficits that are associated with the severity and location of impairment as related to b rain structure and functional anatomy. Typically, during formal assessment executive impai rments are examined through tasks understood to elicit impulsivity, disinhibition, di fficulties monitoring and regulating performance, poor planning/problem solving, perseve ration, and cognitive inflexibility. Aside from cognitive deficits, specific behavioral and personality traits may also be indicative of executive dysfunction including dimin ished affective response, apathy, reduced social judgment, inadequate self-control, a nd poor interpersonal skills (Gioia & Isquith, 2004; Stuss, 1992). Although executive functions are measurable in chi ldren, accurate identification of the cognitive aspects of executive dysfunction r emains elusive. Oftentimes, researchers and clinicians depend on performance obtained on st andardized neuropsychological measures, (e.g. problem solving tasks), which may l ack sufficient construct validity (Anderson et al., 2002; Brocki & Bohlin, 2006). The complexity of many executive function tasks that are presented in standardized n europsychological measures are likely
13 to result in measures that are indicative of pooled outcomes of several distinct underlying processes (Hughes & Graham, 2002). Moreover, tradit ional measures of executive functions are dependent on lower-level cognitive sk ills such as language and memory making it difficult to determine the influence of t he targeted executive components. Given the relatively limited processing capacity of children, it is not necessary to elicit several processes simultaneously in order to tap in to targeted executive functions (Hughes & Graham, 2002). As critics of neuropsychol ogical measures of executive functions note, there has been little attempt to is olate and identify the specific impairments that researchers seek to study in clini cal and normative populations (Anderson, 1998). Construct Validity of the Assessment of Executive F unctions In addition, inconsistencies between performance o n traditional executive function measures and real life behavior often surf ace (Anderson, 2002). Neuropsychological tests are commonly administered in well-structured, quiet, clinic settings with minimal distractions where the examin er plans and initiates the majority of the evaluation, thereby contributing to a lack of e cological validity (Anderson, 1998; Anderson et al., 2002). Performances on such tests are unlikely to be representative of behaviors exhibited in the home, classroom, or soci al environments. Thus, this information is likely to be limited in use when con sidering the development of interventions in the school and home settings. Lastly, the vast majority of tasks have been desig ned and validated for use with adult populations (Anderson, 2002). Simply utilizin g downward extensions of adult tasks are expected to be of little interest or relevance to children. In addition, these tasks
14 frequently lack sufficient normative data for use w ith a younger population and lack the data necessary to differentiate between normative a nd clinical populations within a developmental context (Anderson, 1998; Anderson et al., 2002). For example, fluent literacy emerges relatively late in development, ho wever many adult executive function tasks depend upon routine written language and read ing skills which are developmentally inappropriate for use with children (e.g., Stroop t ests, Trail-making) (Hughes & Graham, 2002). Furthermore, assumptions that such tests sim ilarly detect localized dysfunction in groups of adults and children alike remain question able (Anderson, 1998; Meltzer, 2007). The Need for Measures There is clearly a need for valid, sensitive, and efficient assessment tools that evaluate specific executive function impairments th at are appropriate for use with children. Frequently, clinicians rely on observatio n and informed judgment in collaboration with reports from family and social c ontexts (Anderson, 1998). In order to establish valid measures of executive function, it is essential to use measures that detect the primary skills of interest through novelty, com plexity, and the need to integrate information to elicit executive skills (Anderson, 1 998). An accurate understanding of normal cognitive development is critical for school and health professionals working with children and adolescents. This knowledge will enabl e earlier identification of developmental deviations, improve diagnostic capabi lities, and emphasize use of age appropriate tools in the assessment stage (Anderson 2002). Presently, the Behavior Rating Inventory of Execut ive Function (BRIEF) is the only behavior rating scale that has been designed t o explore childhood executive functions relative to the home and school settings (Mares, McLuckie, Schwartz & Saini,
15 2007). The BRIEF utilizes parent and teacher rating s to sample childrenÂ’s everyday executive skills. It has been designed for use with a broad range of childhood disorders to enhance traditional clinic-based assessments and to provide an increased level of ecological validity for clinical assessments (Mahon e et al., 2002). In response to concerns regarding the sensitivity and validity of individua lly administered cognitive measures of executive function, this rating scale seeks to serv e as an important indicator of everyday, rather than test-driven, executive function (Gioia et al., 2000). This tool is envisioned to serve as a supplement to other methods of gathering data on executive functions. Conceptualization of Attention At present, there is no unified operational defini tion of attention. Researchers however, do generally agree that attention is a mul tidimensional construct that requires a multi method approach to assessment at varying poin ts of development (Manly et al., 2001). Similar to the difficulties faced with opera tionalizing and measuring executive functions, the fundamental problem in measuring att ention is the difficulty with accurately capturing this construct while taking in to consideration the developmental aspects and stages of progression (Manly et al, 200 1; Palfrey et al., 1985). Manly et al. (2001) indicates that attention cannot be measured unless an individual is asked to do something Subsequently, when the individual performs a task, additional perceptual, cognitive, and output systems inevitably influence performance on the task even more so than attention itself (DeGangi & Proges, 1990; Manl y et al., 2001; Warner-Rogers et al., 2001). Attention is a multi dimensional construct consist ing of a number of components (Mirsky et al., 1991). Broadly, attention has been defined as a cognitive brain mechanism
16 that enables one to process relevant inputs, though ts, or actions while ignoring irrelevant or distracting stimuli (Gazzaniga, Ivry & Mangun, 2 002). Behaviorally, attention is studied by identifying specific, overt actions or r esponses and examining the variables and functional relationships that operate to contro l such behaviors (Warner-Rogers et al., 2001). Â“Attentive behaviorÂ” is typically assessed b y directly observing an individualÂ’s interaction with the environment, or indirectly ass essed by asking others familiar with the individual to rate the occurrence of behaviors (War ner-Rogers et al., 2001). On the other hand, Â“attention problemsÂ” are broadl y used to describe a collection of behavioral difficulties that may include inatten tiveness, distractibility, poor concentration, impulsivity, and hyperactivity, or a lack of appropriate response to the ongoing environment (Friedman et al., 2007; WarnerRogers et al., 2001). Importantly, individual differences in attention problems can be perceived as a continuum where at a specific level, deficits may be considered a signif icant impairment, not only in clinical populations, but also in normative groups of childr en (Friedman et al., 2007). Neuropsychological models of attention attempt to explain the frequency of attentional difficulties that are present in a rang e of acquired and developmental neurologic disorders. These models generally propos e that the brainÂ’s attentional system depends on the efficient functioning of a broad net work of distinct neuronal structures rather than the control of a unitary neural system (Anderson, Fenwick, Manly & Robertson, 1998; Castellanos, 1997). Damage or dysf unction in any of the involved areas may result in deficiencies affecting attention gene rally, or differentially on specific aspects of attentional processing (Anderson et al., 1998). Although the nature of the neuropsychological impairment varies as a function of the specific frontal region
17 affected, attentional and executive functioning imp airments appear to be common in most forms of frontal lobe disorders including ADHD (Bar kley, 1997). Anatomical Structures Involved in the Control of At tention Major anatomical structures that have been implica ted in the control of attention include the cortex, basal ganglia, and thalamus (Ca stellanos, 1997). The cortical-striatalpallidal-thalamic-cortical circuit is a neuroanatom ical loop that provides feedback to other cortical regions and serves as a pathway for many executive functions, including attention. Briefly, neuronal signals travel from th e prefrontal cortex to the subsequent structures of the pathway where the final result is feedback that is sent back to the original cortical output regions and to additional cortical areas. Dysfunction within these pathways has been implicated in attentional deficit s (Castellanos, 1997). Given the involvement of numerous structures, attention may b e viewed as a complex system that is subserved by multiple attentional networks and mani fested through different types of attention rather than a unitary construct (Wang & F an, 2007), thus, leading into a discussion of the different subcomponents of attent ion. Subcomponents of Attention A number of contemporary theoretical models of att ention have been developed, which typically divide the construct of attention i nto several different component processes (DeGangi & Proges, 1990; Heaton et al., 2 001). Traditional neuropsychological assessment of attention, particularly in the assess ment of children and ADHD has typically assessed multiple frontal lobe abilities, including response inhibition, ability to shift set (flexibility), and planning/organization. Similar tasks have been conducted with adult patients with acquired brain lesions and more recently, through functional imaging
18 studies (Heaton et al., 2001; Manly et al., 2001). Converging research has allowed enhanced understanding of the neural basis of atten tion and separations between different attentional systems (Manly et al., 2001). In their review, Posner and Petersen (1990) argued for three characteristics of attention functions within the brain. Initially, th e researchers presented the notion that specific attention systems exist. Furthermore, thes e attention systems were noted to be separable from more Â“basicÂ” perceptual, cognitive, and output systems. Lastly, authors stated that within the attention system, specific b rain regions and neural networks performed different types of operations. On the bas is of their theoretical understanding, distinct systems were proposed and characterized as : (a) a capacity to move attention within space (spatial attention); (b) a capacity to enhance the processing of targets regardless of spatial location (selective attention ); and (c) a capacity to maintain a particular processing set over time (sustained atte ntion) (Manly et al., 2001). These conclusions have important clinical implications. T he functional/anatomical separation of attention systems from other cognitive and basic pe rception indicates that whether it evolves through acquired brain damage or developmen tal anomaly, it is altogether possible to present with deficits that are exclusiv ely or predominantly attentional in nature (Manly et al., 2001). In addition, the separ ation of attentional systems, depending on the locus of the damage, could allow individuals to present with distinct profiles of attentional deficits, each with different implicati ons for problems in everyday life (Manly et al., 2001).
19 Differential Assessment of Attention Although the interpretation of ADHD has become inc reasingly cognitive in its emphasis from a primarily behavioral understanding, the diagnosis continues to rest exclusively on reports of behavior and inferences a bout underlying processes. Although parents or teachers are often asked to indicate deg ree of difficulties with sustaining adequate attention, there are few reliable measures of such capacities (Manly et al., 2001). The assessment of attentional disorders and difficulties have traditionally relied on information obtained from clinical interviews and b ehavioral observations with supplemental data acquired from parent and teacher reports on behavior rating scales (American Academy of Pediatrics, 2000; Barkley, 199 8). Although multi-informant rating scales can provide clinicians with useful in formation regarding childrenÂ’s attentional impairments in everyday settings, objec tive measures of attention can provide additional information from a more controlled, stan dardized, first-hand assessment that may be less susceptible to reporting bias (Heaton e t al., 2001). Recently, there has been a call for alternative strategies in the assessment o f ADHD, including an emphasis on objective laboratory and clinic-based instruments t hat can provide both research and clinical utility (Frick, 2000). Likewise, the Natio nal Institute of Health (NIH) has emphasized the need for studies to address the natu re of cognitive processing in the diagnosis of attentional disorders by considering m ultidimensional aspects (NIH Consensus Development Panel, 2000). The Test of Everyday Attention for Children (TEA-C h; Manly, Robertson, Anderson & Nimmo-Smith, 1999) is a measure, that is gaining greater attention in the United States and has been noted to have considerab le potential for use in assessing
20 different subcomponents of attention (sustained, s elective, attentional control/switching). The TEA-Ch presents several potential advantages wh en compared to other existing objective measures that purport to assess attention A distinct advantage of the TEA-Ch is its inclusion of multiple components of attention w hereas the majority of other commonly used neuropsychological tests typically examine onl y one component (e.g., continuous performance test, Wisconsin Card Sorting Test, Trai l Making Test, Stroop Color and Word Task). The TEA-Ch also utilizes various sensor y modalities throughout its administration, including visual, auditory, and mot or modalities (Heaton et al., 2001; Manly et al., 2001). This is important to consider since most neuropsychological tests of sustained and selective attention have focused sole ly on visual presentation of stimuli (Cooley & Morris, 1990). Finally, the TEA-Ch was de signed for the purposes of addressing the lack of ecological validity between real world functioning and neuropsychological tests as it seeks to more closel y simulate real world attentional demands (Heaton et al., 2001). The use of the BRIEF in conjunction with the TEA-Ch may provide valuable data in description of cogniti ve and behavioral impairments across a variety of settings (Gioia et al., 2000). In summary, the existing research literature highl ights the significant impact of executive function deficits on everyday functioning (Warner-Rogers et al., 2000). An individualÂ’s success in adapting to and navigating through daily routines and the environment is determined by their ability to utili ze organizational and coordination skills. These skills include expectations for selfmonitoring and self-regulatory skills of behavior and the ability to inhibit and adapt respo nses according to the changing conditions of the environment. Despite the wide var iety of cognitive tests that are
21 purported to assess various aspects of executive fu nctions, there is a paucity of research validating the use of such measures in predicting r eal world functioning. This is particularly true in children where characterizatio n of executive functions has proven to be much more difficult (Manly et al., 2001). A caveat in neuropsychological assessment is the l imited ecological validity that is found with the use of many neuropsychological me asures. Current research suggests that although neuropsychological tests can be helpf ul in identifying differences among clinical and control groups, they often lack utilit y in predicting behavior outside of the clinic or laboratory settings (Sbordone, 1996). Thu s, more research and test development to improve the ecological validity of neuropsycholo gical assessment measures is needed (Gioia & Isquith, 2004). In addition, although many performance-based measures and caregiver behavior checklists exist for assessing a wide range of behaviors, specific measures dedicated to examining multiple components of a single executive function construct, namely attention, warrants further resea rch and exploration as to the utility and developmental appropriateness with child population s. Research Questions This research study developed specific aims. The f irst was to investigate the predictive validity of a specific measure of attent ion (i.e., TEA-Ch) and the three different subtypes of attention as proposed by Manl y and colleagues (1991). This relationship was determined by relating a performan ce-based measure of attention and behavior rating scale, in differentiating between c hildren presenting with varying degrees of executive function skills. The second aim was to explore the ecological validity of a performance-based measure of attention by examining the relationship with a behavior
22 rating scale of executive functions, and a social/a daptive measure, as reported by parents/caregivers and teachers. Lastly, gender eff ects were explored. The purpose of this study was to examine the relationship between atten tion, executive function behaviors, and social/adaptive functioning through exploratory analysis. The following research questions are addressed: 1. What is the relationship between attention and exec utive function behaviors as determined by the correlation between subcomponent( s) of attention (sustained, selective, shifting/attentional control ) and executive function behaviors? a. What is the relationship between attention and pare nt ratings of executive function behaviors as determined by the correlation between subcomponent(s) of attention (sustained, selective, shifting/attentional control) and executive function behaviors? b. What is the relationship between attention and teac her ratings of executive function behaviors as determined by the correlation between subcomponent(s) of attention (sustained, selective, shifting/attentional control) and executive function behaviors? 2. What is the relationship between attention and soci al/adaptive functioning as determined by the correlation between subcomponent( s) of attention (sustained, selective, shifting/attentional control ) and social/adaptive functioning? a. What is the relationship between attention and soci al/adaptive functioning as determined by the correlation between subcompone nt(s) of attention
23 (sustained, selective, shifting/attentional control ) and social/adaptive functioning? b. What is the relationship between attention and teac her ratings of social/adaptive functioning as determined by the co rrelation between subcomponent(s) of attention (sustained, selective, shifting/attentional control) and social/adaptive functioning? 3. What is the relationship between executive function behaviors and social/adaptive functioning? 4. How does the relationship between attention, execut ive function behaviors, and social/adaptive functioning differ (if any) by gender? a. How does the relationship between parent ratings of attention, executive function behaviors, and social/adaptive functioning differ (if any) by gender? b. How does the relationship between teacher ratings o f attention, executive function behaviors, and social/adaptive functioning differ (if any) by gender? Significance of the study Neuropsychological research on normal, age-related changes has most often focused on the two extremes of the lifespan: infanc y and aging populations. Although some normative studies have provided data related t o school age children, there is a relative lack of theoretical interest in developmen tal changes occurring during school age years (Korkman, 2001). This study examined the ecol ogical utility of a performancebased measure of executive function, specifically i n the area of attention, in a normative
24 sample of school age children. The use of the TEA-C h in this study attempted to contribute to the definition and understanding of s eparable attention subcomponents. The TEA-Ch is a unique test instrument that offers spec ific evaluation of attention and its subcomponents. Although research to date supports t he ability of the TEA-Ch to assess specific attentional deficits in various clinical p opulations, only a limited number of studies exist, and even fewer of these studies have occurred within the United States. In addition, this study examined how attention influen ces various aspects of a childÂ’s daily functioning. In addition, as attentional difficulti es are inherent in most school age children the information derived from a normative s ample in regards to common weaknesses and strengths of attention is likely to be useful in determining the level necessary to warrant clinical significance. Lastly, information gathered from this study is li kely to facilitate a common language between parents, teachers, and psychologis ts in utilizing neuropsychological measures to supplement current assessments of execu tive functions and attention in the school and home settings. Primary caregivers and ed ucators can collaborate in developing targeted interventions to address common attentiona l and related behavioral difficulties by analyzing executive functions with familiar term s such as planning, organization, study skills, and self-monitoring which are underst ood to be relevant to education and learning. Academic and behavioral success is increa singly dependent on studentsÂ’ ability to plan their time, organize, and prioritize inform ation. Weaknesses in these core executive function processes are not easily identif ied, and modifications are clearly needed in diagnostic and teaching methods. The over arching goal of this study is to narrow the lingering gap between research and educa tional practice and to improve
25 methods of identifying and teaching students who pr esent with weaknesses in executive functions and, specifically attention. Thus, knowle dge regarding specific subcomponents of attention will lend further support for generati ng relevant interventions for success in the classroom and in everyday functioning.
26 Chapter 2 Review of the Literature Introduction The developmental trajectories in childrenÂ’s socia l, emotional, and behavioral spheres are embedded in ecological models that cons ist of important factors at the individual, family, school, and community levels (R iggs, Blair & Greensberg, 2003). Specifically, at the individual level there has bee n a remarkable increase of interest in the early development of executive functions and their associations with and influences by multiple other factors (Korkman, 2001). This trend may be due to the increased understanding of impairments in executive functions that are now thought to play a central role in a variety of developmental disorder s (Clark, Prior & Kinsella, 2002; Hughes & Graham, 2002). Clinical evidence provides support that individuals with damage to the prefrontal cortex and associated regi ons of the brain experience problems with a range of executive tasks involving planning, flexibility, organization, and working memory (Beveridge, Jarrold & Pettit, 2002; Robbins, 1996). Several lines of evidence also provide illustratio n of ongoing development of executive functions throughout childhood. Physiolog ical research describes substantial development of the central nervous system to contin ue through adolescence and early adulthood with anterior regions of the cerebral cor tex maturing later on in life (Robbins, 1996). Neuropsychological studies have confirmed si milar types of growth spurts as
27 evidenced by distinct improvements in performance o n tests purported to measure targeted executive functions. Furthermore, convergi ng research suggests that physiological growth spurts may coincide with trans itions in cognitive development, which reflect ongoing cerebral development (Korkman 2001). The lack of ability to plan, reason, utilize abstract and flexible thinking is l ikely to impinge on a childÂ’s capacity to learn and benefit from the environment and the clas sroom setting. During the preschool and school age years in particular, the impact of e nvironmental stimuli and formal instruction is perhaps greater than in any other pe riod of life (Korkman, 2001). At present, there are few but growing numbers of v alid and appropriate tests of executive functions available for childhood populat ions. Of those that are currently available, many were originally designed for adult populations and lack adequate child norms precluding accurate interpretation of develop mentally appropriate levels of performance. In addition, many of these instruments lack standardized administration and scoring procedures (Anderson, 1998; Riggs, Blair & Greenberg, 2003). Establishing valid, reliable assessments of executive function i n children is likely to provide additional insight into the pattern of development of specific executive skills present in childhood. This literature review will examine the widespread impact of executive function with a specific emphasis on attention in school age children. Issues that will be considered include differences in profiles of execu tive functions as they manifest across the developmental age span as well as differences i n presentation based on gender. In addition, child outcomes associated with deficits o f executive functions and attention will be discussed and reviewed. Topics will cover areas examining the ecological validity of neuropsychological assessments with focus on a spec ific performance-based measure of
28 attention and the importance of obtaining ratings f rom teachers and parents in multi method, multi modal assessments of executive functi ons and attention. Limitations of the studies reviewed will be discussed in an attempt to direct future research needs. Executive Function Deficits and Academic Outcomes Studies of psychiatric, neurologic and other devel opmental disorders have repeatedly demonstrated significant impairments in functional outcomes, and thus strongly support the critical role of executive fun ctions for complex human behavior (Biederman et al., 2004). Substantial evidence indi cates that executive functions play an important role in learning during childhood (St. Cl air-Thompson & Gathercole, 2006). St. Clair-Thompson and Gathercole (2006) demonstrated that the specific executive functions of shifting, updating, and inhibition wer e related to achievement in the areas of English, Mathematics, and Science. In addition, ach ievement in these academic areas was further influenced by verbal and visuo-spatial work ing memory tasks. Researchers imply the need for a greater understanding for the import ance of structured learning activities to prevent working memory overload and reduce processi ng and storage requirements by providing manipulatives and external memory aids (S t. Clair-Thompson & Gathercole, 2006). Biederman and colleagues (2004) also support the a ssociation between executive function deficits, academic, and psychosocial impai rments in groups of children diagnosed with ADHD. Assessments of psychosocial, c ognitive, and neuropsychological functioning indicated a correlation with deficits i n these areas, and an increase in the risk for grade retention, learning disabilities, and low er academic achievement. Control participants who met criteria for executive functio n deficits (EFD) were also diminished
29 in their level of academic outcomes as compared to control participants. This provides support for the importance of assessing executive f unctioning in normative groups as well as vulnerable groups. Additionally, the analysis of age as a modifying factor did not provide evidence that the developmental trajectorie s of neuropsychological functioning influenced academic or psychiatric outcomes (Bieder man et al., 2004). Furthermore, Waber, Gerber, Turcios, Wagner and Fo rbes (2006), demonstrated a clear and systematic relationship with behaviors in dicative of executive functioning as obtained from the BRIEF and childrenÂ’s performance on high stakes achievement testing. Teacher reports of executive functions, as manifest ed by everyday behavior were highly correlated with achievement test scores. Neuropsych ological measures accounted for 3040% of the variance in test scores. However, childr en also performed at or above normative expectations on laboratory measures of wo rking memory, processing speed, planning, and motor coordination. Externalizing and internalizing behavioral measures were also well within normal limits. Overall, these findings highlight the potential dissociations between traditionally administered la boratory measures from ecological measures of neuropsychological functioning. Specifi cally, the BRIEF appeared to be more sensitive to childrenÂ’s everyday classroom fun ctioning that was particularly relevant to childrenÂ’s ability to obtain higher sco res on a high stakes test. In sum, although the participants of this study were in no means identified as presenting with diminished psychosocial adjustment or executive fun ctions, they appeared to experience a marked decrease in rate of performance on an achi evement test despite their competence on neuropsychological measures, which ar e often purported to indicate risk for learning problems. These findings have potentia l implications for educational
30 approaches. Executive functions, particularly metac ognitive skills, appear to be susceptible to both environmental influences and ta rgeted interventions. Executive Functions and Behavioral Implications According to prominent neuropsychological theories of executive function, deficits not only interfere with social and academi c functioning but are also related to the successful control of oneÂ’s behavior through self-i nitiation, strategic planning, cognitive planning, and impulse control (Barkley, 1997; Brock i & Bohlin, 2006; Mullane & Corkum, 2007). These findings suggest that executiv e dysfunction and in particular, deficits with inhibition are consistent with the ex ecutive dysfunction theory of ADHD as most prominently outlined by Barkley (1997). Barkle y proposes that the core deficit in ADHD is behavioral inhibition, which in turn affect s the development of executive functions that are necessary for self-regulation of behavior, cognition, and emotions. This hierarchical model hypothesizes that behavioral inh ibition consists of the ability to inhibit a prepotent response, to interrupt an ongoing respo nse, and resist interference by extraneous stimuli during the intervening interval (Barkley, 1997; Brocki & Bohlin, 2006; Mullane & Corkum, 2007). According to Barkley (1997), adequate inhibitory control must initially develop and is essential for the development and function of the other identified subtypes of executive functions. A nother component of BarkleyÂ’s theory addresses the developmental dimension, which sugges ts that rudiments of inhibition are present in children as young as five years of age. Furthermore, fully matured inhibitory control has been suggested to develop between the a ges of 8 and 12 years (Barkley, 1997; Brock & Bohlin, 2006; Hughes & Graham, 2002; Mullan e & Corkum, 2007).
31 In order to further examine the impact of executi ve function deficits on behavioral outcomes, Brocki and Bohlin (2006) condu cted a study investigating the normal developmental change in the relation between executive functions and the core behavioral symptoms most closely associated with di agnostic criteria for ADHD (hyperactivity, impulsivity, and inattention). In a ddition, symptoms that most often cooccur with childhood hyperactivity (externalizing a nd internalizing problems) were also incorporated into this study. Sample participants c onsisted of 92 children aged 6-13 years. Executive functions were assessed by adminis tering various cognitive measures examining disinhibition, speed/arousal, verbal work ing memory, non-verbal working memory, and fluency. Results indicated that althoug h disinhibition was positively related to hyperactivity/impulsivity and inattention mainly for the youngest age group, there were no significant age effects. However, age effec ts were demonstrated between speed/arousal and inattention as well as between ve rbal working memory/fluency and inattention. For the oldest age group poor performa nce on cognitive measures was associated with high ratings of inattention (Brocki & Bohlin, 2006). In summary, the results from this study highlight the importance of developmental analysis of normal change in cognitive processes and behavioral profil es in understanding the nature of childhood disorders. Although findings suggest that symptoms change with maturation in the manifestation of symptoms related to ADHD, it a ppears that the key to understanding this disorder as either a developmental or categori cal disorder lies in comparing the development in clinical and normative samples. With similar goals of studying executive function and behavior, Riggs, Blair, and Greenburg (2003) aimed to investigate the link betw een inhibitory control, sequencing
32 ability, and the behavioral development of early sc hool age children. Concurrent and 2year longitudinal relationships were examined betwe en two aspects of executive function and both parent and teacher reports of externalizin g and internalizing symptoms of behavior. Participants included 60 regular educatio n classroom students aged 6-years 9months to 9-years, 2-months (32 males and 28 female s). Assessment measures included the Stroop test, portions of the WISC-R, Trail Maki ng, and parent/teacher ratings on respective versions of the Child Behavior Checklist (CBCL; Achenbach, 1991). Results provided evidence that childrenÂ’s ability to perfor m on tasks of executive ability during the 1st and 2nd grade predicted change in level of behavioral prob lems over a 2-year period. These findings are indicative of a possible developmental lag between childrenÂ’s acquisition of neurocognitive capacities and the be havioral patterns associated with them. Lastly, this study indicated that when compared to children with executive function deficits at the time of initial assessment, childre n with proficient executive skills appeared to demonstrate fewer behavior problem symp toms over a 2year period. One implication of such findings is that executive function deficits place young children Â“at riskÂ” for developing behavior problems later on. Therefore, it may be beneficial to intervene with early school age child ren who demonstrate with weaknesses in executive functioning to enhance behavioral deve lopment and prevent the potential future onset of behavioral difficulties. Lastly, re searchers suggest the consideration of placing children with poor executive function skill s in environments that promote the development of such skills. For example, schools ma y utilize small classroom environments and decrease distractions in the class room to enhance childrenÂ’s ability to
33 focus their attention and to successfully inhibit a nd sequence behavior (Riggs, Blair & Greenberg, 2003). Relationship between Executive Functions and Daily Functioning The linkage of laboratory and clinical measures to real world functioning has been a reoccurring topic of investigation (Burgess, Alde rman, Evans, Emslie & Wilson, 1998; Stavro, Ettenhover & Nigg, 2007). It has been large ly assumed that individuals experiencing difficulties in everyday functioning w ere also likely to reflect a similar degree of difficulty to that observed in a testing situation (Burgess et al., 1998). There is now emerging evidence to suggest that executive abi lities as assessed through neuropsychological testing has implications for beh avior in various contexts outside of clinical settings (Stavro, Ettenhofer & Nigg, 2007) However, it is unknown as to what extent the executive function deficits detected on neuropsychological testing are related to performance in real-world activities, primarily because little is known about executive function in outside settings (Lawrence et al., 2004 ). Since current understanding of executive function deficits are typically derived f rom neuropsychological testing conducted in clinical settings researchers support further examination of the generalizability of current neuropsychological theo ries of childhood disorders to performance on tasks in outside environments (Lawre nce et al., 2004). Lawrence et al. (2004) set out to determine whethe r children diagnosed with ADHD exhibited cognitive deficits as evidenced by d ifficulties with tasks of executive functions and processing speed as measured by neuro psychological tests and real world activities. Overall aims were to examine the relati onship between cognitive deficits observed during neuropsychological testing and real -world activities. Assessment
34 measures included the completion of two neuropsycho logical and two real life tasks: the Stroop test, WCST, videogames, tasks at the zoo, an d four subtests administered from the WISC-III. Consistent with stated hypotheses, the cl inical group exhibited executive function deficits on both neuropsychological tasks and real world activities. Children diagnosed with ADHD exhibited similar problems whil e playing a highly motivating adventure videogame, visiting the zoo, and during t he administration of a standard neuropsychological test. This finding mitigates the argument for the lack of motivation leading to deflated performance rather than executi ve function deficits. Furthermore, results of this study support the hypotheses that e xecutive functions and speed of processing are impaired in ADHD and evidenced acros s a wide variety of activities and contexts in addition to testing situations. Clark, Prior, and Kinsella (2002) also investigate d the extent to which executive function capacities were linked to everyday adaptiv e outcomes. Significant relationships were found between all test performances on executi ve measures, adaptive behavior, and reading ability in adolescents. Multiple regression analyses indicated that verbal ability predicted communication and reading scores while ex ecutive function abilities contributed significant variance to the prediction in the adaptive behavior, communication, and socialization domains. Researche rs propose that the associations demonstrated between adaptive and neurocognitive im pairments add to the insights necessary to understand the bases of various disord ers. Future studies are needed to study the generalizability of these results with other sa mples including community and clinical groups as well as female participants, for addressi ng potential gender differences. In
35 addition, studies with younger population are neces sary in order to determine the onset of relationships between executive measures and social /adaptive behavior. Models of Attention One of the most pervasive yet obscure behavioral d eficits encountered in educational and clinical settings is the symptom of impaired attention (Mirsky, 1991; Posner & Peterson, 1990). Thus far, the construct o f attention has received much less behavioral, theoretical, and statistical attention than that of research examining other neuropsychological constructs such as memory, learn ing, and language, for example. However, it has been estimated that approximately 5 -20% of children suffer from some form of impairment in attention (Mirksy, 1991). Oth er estimates report figures reaching as high as 30% of all school age children. In addit ion, impairment of attention is commonly characteristic of many psychiatric as well as neurologic and metabolic disorders. Thus, it is then plausible that clinical populations in conjunction with classroom identified problems with concentration an d learning may indeed contribute to a notable population of children suffering from impai red attention at one time or another (Mirsky, 1991; Posner & Peterson, 1990). Although varying models of attention exist accordi ng to interpretation and differing emphasis on various components of the mul tidimensional construct, attention is oftentimes conceptualized using a 4-factor model. M irsky and colleagues (1991) developed this neuropsychological model of attentio n to consist of separate elements of attention including selective or focused attention, attentional shift, sustained attention, and divided attention (Heaton et al., 2001; Mirsky et al., 1991; Wu, Anderson & Castiello, 2002). These factors are assumed to exis t as separable factors that can be
36 measured individually but operate cohesively within the attentional system (Posner & Peterson, 1990). Sustained attention involves maint aining attention over extended periods of time. Selective or focused attention refers to a n individualÂ’s ability to select target information and attend to one relevant component wh ile ignoring other distracters. Attentional shift is described as the ability to ch ange attentional focus flexibly and adaptively. Lastly, divided attention refers to an individualÂ’s ability to focus on all simultaneously occurring stimuli (Heaton et al., 20 01; Mirksy et al., 1991). In sum, this model of attention attempts to organize a rather di ffuse and global concept as a more manageable group of processes or elements. Extended research elsewhere has attempted to link these elements to organization of cerebral structures and systems (Mirsky et al., 1991; Wilding, 2005). The statistical development of MirskyÂ’s model of a ttention consisted of obtaining neuropsychological test scores from adult and child sample populations. The initial effort of the researchers in deriving their assessment bat tery of attention was designed from collaborating targeted neuropsychological tests use d routinely in clinical settings. The results of the data analyses yielded four factors t hat identified distinct components of attention ultimately conforming to the multi-elemen t model. Results of the principal component analysis revealed similar patterns of com ponent skills identified in both the adult and child samples to support the elements of focus-execute, sustain, encode, and shift to reflect pertinent and distinct components of attention (Mirsky et al., 1991). In conclusion, and in regards to the findings of t his study, researchers assert that attention can be viewed as a process involving the four above mentioned independent elements and serve the purpose for testing neuropsy chological hypotheses related to
37 disorders of attention. Future research is encourag ed to compare this model of attention with other existing models to confirm the proposed elements of this model and/or to revise current definitions of attention. Particular ly, for childhood populations the goal of developing a model for conceptualizing the componen ts or elements of attention are in understanding the relation of attention to aspects of academic performance and behavior difficulties. These concepts will be further discus sed throughout this review. Relationship of Attentional Deficits to Future Outc omes Attentional capacity is a critical dimension of man y psychological, social, and cognitive problems. Individual differences in child hood and adolescent attention problems vary along a continuum. Attention problems are oftentimes associated with learning disabilities, psychosocial outcomes such a s deficits with social skills, academic and occupational performance, and decreased global adaptive functioning (Friedman et al., 2007; Palfrey, Levine, Walker & Sullivan, 1985 ). Anderson, Jacobs and Harvey (2000) investigated the effects of prefrontal lesio ns with respect to attentional abilities. Selective subtests were administered from the Test of Everyday Attention for Children (TEA-Ch) as well as parent ratings from the Behavio ral Rating Inventory of Executive Function (BRIEF) to assess everyday executive and a ttentional function. Overall, results indicated lower performance across all of the compo nents of cognitive and behavioral measures of attention that were investigated. These results are consistent with expectations from adult based studies, which sugges t that the anterior regions of the brain are responsible for shifting and divided attention. Aspects of processing speed reflect higher levels of attentional resources, which are r equired to perform effectively on such tasks.
38 The implications of these deficits for both cognit ive and everyday functioning are vast as they suggest that children with frontal lob e pathology are likely to have difficulties coping with a range of everyday activi ties, and more so with those requiring greater cognitive resources such as self-monitoring and cognitive flexibility. The early onset of prefrontal dysfunction as evidenced by def icits in performance on attentional tasks may underlie at least some cases of psychopat hology for both clinical and normative groups (Anderson et al., 2000). If specif ic deficits are identified early on, there is a greater chance that further decline and signif icant impact on various other areas of functioning may be avoided or mitigated through tar geted strategies and interventions. In order to study the emergence of attention defic its in early childhood, Palfrey and colleagues (1985) documented the occurrence of poor concentration, distractibility, behavioral disorganization, self-monitoring, and ov eractivity in a sample population consisting of 174 children enrolled in an early edu cation program. The children were followed prospectively from birth to school entry. Children who presented with early onset and persistence of attention problems were re ported to have consumed the greatest amount of special education services in school incl uding various therapies, and resource programs. This study indicates that in a large numb er of children, precursors of attention problems are present and identifiable during early childhood years and as a result, vigilance for early signs of attention deficits may be justified as a component of preventive pediatric care. Much of the evidence for a link between everyday a ttention problems and executive function comes from clinical studies of i ndividuals diagnosed with ADHD. This population may be considered an extreme on the continuum of individual
39 differences in attentional control and behavioral s elf-regulation (Friedman et al., 2007). The few studies that have explored the impact of at tentional deficits in normative samples have found similar impairments on tasks of executiv e functions. Studies thus far have failed to address how developmental stability and c hange in attention problems over time relate to later levels of executive functions and a ttention (Warner-Rogers et al., 2000). Inattentive behavior is a predominant feature of ma ny psychiatric disorders, however, little is known about the relative developmental ri sk associated with attentional deficits and if and when it occurs in isolation from other m aladaptive patterns of behavior. In some respects, this lack of understanding and resea rch may be linked to the difficulties in conceptualizing and accurately measuring attention (Warner-Rogers et al., 2000). In order to address some of the gaps related to th e consequences of attentional deficits in normative populations, Friedman et al. (2007) investigated the impact of attentional difficulties in relation to three separ able executive functions (inhibiting, updating, shifting) across an eight year time span. Researchers also examined the predictability of later impairment in overall funct ioning as influenced by developmental stability and change in attention problems. Partici pants were 866 individual twins (422 male, 444 female) recruited from the Colorado Longi tudinal Twin Study at the Institute for Behavioral Genetics. Attention problem ratings were obtained from the 20-item Attention Problems scale included on the Teacher Re port Form (Achenbach, 1991). In addition, subjects were administered the Wechsler A dult Intelligence Scale (WAIS-III; Wechsler, 1997), and tasks of executive functions a t the ages of 16 and 17 years. Overall, results indicated average attention probl ems scores to be low as were expected given the unselected sample of participant s. However, correlations among
40 attention problem scores at different ages were mod erate to high despite different teachers rating each individual across the 8-year t ime span. These data suggest that attention problems exhibited during school age year s are quite stable. In addition, correlations computed between attention problems at age seven years with later executive functioning, IQ, and the consistency of the correla tions between these factors across the years were low to moderate. Researchers raise the p ossibility that it may be oneÂ’s initial level of attention problem rather than the change a cross time that is related to future executive function skills and IQ. Furthermore, resu lts indicate that individual differences in executive function abilities are important in un derstanding normal variations in attention. Lastly, these findings illustrate the ex ternal validity of current cognitive models of executive control and attention that although ar e developed through laboratory based research are applicable towards understanding every day problems. Relationship of Attention to Behavior and Academic Achievement Although performance on tasks designed to measure various attentional components are frequently compared to ratings of AD HD it is less frequently compared to global measures of maladjustment despite its rel ationship to attention difficulties. The presence of deficits in attention and executive fun ctions in the development of psychopathology and overall impact on future outcom e has been well-documented (Pennington & Ozonoff, 1996). For young children, a ttention deficits have frequently been investigated by evaluating the role of early h yperactive and inattentive behaviors with difficulties related to behavior and academic performance (Barkley, 1997). In a longitudinal study conducted by Palfrey et al. (198 5) researchers examined the occurrence of concentration, distractibility, behav ioral disorganization, self-monitoring,
41 and hyperactivity in a sample of 174 children follo wed from birth to the time of school entry. Participants were assessed by multiple behavior ratings and obse rvations. Overall, findings indicated a greater presence of socio-emot ional difficulties, lower academic achievement, and greater need for special education services for children who were rated with greater attentional difficulties as compared t o children who presented with less persistent or normal capacities of attention. Significant levels of attention problems were identified in 5% of all children assessed. Over the period from birth to kindergarten, 40% of the preschool children were rated to display som e difficulties with attention, however, the majority of children were not rated to reach le vels that would warrant further concern. The findings of this study highlight the importance of the association of persistent attentional concerns and the potential long-term co nsequences of early-detected attention problems. With similar aims, Warner-Rogers and colleagues (2 000) elicited a large community-based sample to compare the developmental functioning, social, and environmental backgrounds of children presenting wi th purely inattentive behaviors as compared to children presenting with overactive beh aviors and combined problems of inattention and hyperactivity. Parents and teachers were interviewed in order to obtain information in the areas of learning, behavior, sel f-esteem, following directions, and teacher/peer relations relevant to the home and sch ool settings. General psychometric measures and measures of cognitive functioning were administered including the WISCR, CPT, a central-incidental learning task (CIL), p aired associates learning task (PAL), and the 20-item Matching Familiar Figures Task (MFF T; Cairns & Cammock, 1978). Results indicated that elevated rates of inattentiv e behaviors as reported by teachers and
42 parents had unique implications for behavior and ac ademic functioning. In brief, inattentive behaviors were closely associated with adjustment problems in the classroom, specifically lowered self-esteem and the need for r epeated instructions. Intellectual functioning overall, particularly in the area of re ading was evidenced by lower performance as compared to other sample groups. In addition, children who exhibited greater problems with attention were more likely to have received speech therapy and have had language delays in their early development al histories. In regards to behavioral differences, inattention was more closely linked to problems with social interaction whereas hyperactive groups were more likely to have had problems with conduct. This study has important clinical and research imp lications. Although children with attentional deficits were noted to exhibit gen eral cognitive impairments, reading problems and poor adjustment in the classroom, it i s hypothesized that these children will be less likely to receive support through appropria te interventions in the classroom since they do not typically exhibit externalizing problem s. Thus, authors report the need for formal evaluations of cognitive, academic, and pare nt/teacher reports in addressing the needs for children who display predominantly inatte ntive behaviors so that the neglect of children who exhibit attentional deficits may be pr evented. A discussion of the limitations of this study is warranted as the gener alizability of the findings is not likely. The inclusion of other age ranges as well as enroll ing female participants is necessary in the study of attentional deficits. Overall, the pre sence of deficits with attention in early childhood may be viewed as a developmental risk fac tor considering the potential impact on later academic functioning and behavior. More re search is needed to identify what, if any are the long-term implications of inattentive b ehavior. Lastly, the previous studies,
43 which have examined the effects of attentional defi cits on academic achievement and behavior, have consisted mainly of observations, in terviews, rating scales, and standardized measures of cognitive functioning. Alt hough problems with attention appear to impact functioning in these specific areas it is also important to determine the effect on a childÂ’s everyday functioning outside of the clini cal setting. Executive Function in Attention-Impaired Groups The shift in the emphasis on the frontal lobe dysf unction theory in relation to attention difficulties has resulted in the use of v arious neuropsychological tests to evaluate for specific deficits in children diagnose d with ADHD (Barkley, 1997, Heaton et al., 2001). Neuropsychological testing of children diagnosed with ADHD seeks to assess multiple frontal lobe abilities including response inhibition, ability to set shift (cognitive flexibility), and planning/organization (Heaton et al., 2001). Specifically, ADHD has been associated with executive functioning and spec ific deficits with sustained and divided attention. Although the reliance on executi ve functions in a theory of ADHD assists in unifying attentional and inhibitory defi cits that are commonly highlighted in the diagnosis of ADHD, there is also growing dissatisfa ction with the allencompassing characteristics of this concept (Wu, Anderson & Cas tiello, 2007). Thus, it is necessary to establish conceptual and theoretical clarity in the study of executive functions and to develop measures that are sensitive and specific fo r measuring them. Existing studies assessing executive functions and attention in ADHD vary widely as a result of many variables including sample size, age, comorbidity, gender, IQ, and selection criteria (Wu, Anderson & Castiello, 2002; Pasini et al., 2007). I n many cases, these additional factors fail to be addressed appropriately. Furthermore, pr evious studies of attention and
44 executive functions have yet to study various domai ns of executive function and attention in the same sample population. Therefore, up to thi s point it has been difficult to generate conclusions about the executive function and attent ion profile in ADHD based on the current literature (Pasini et al., 2007). In an attempt to address the above-mentioned chall enges and methodological difficulties, Pasini et al. (2007) conducted a stud y to assess executive functions in relation to control variables such as IQ and basic neuropsyc hological performance. Overall, results comparing clinical and normative samples di d not generate differences on tasks for age. However, clinical and normative groups per formed with significant differences in the areas of divided attention, inhibition of re sponse, variability of reaction times, phonological, and visual working memory. With simil ar goals, Wu, Anderson and Castiello (2002) also conducted a study investigati ng multiple aspects of executive functioning in children diagnosed with ADHD. A batt ery of neuropsychological tests that allowed analysis of specific cognitive processing m echanisms including attentional components, impulsivity, planning, and problem solv ing were administered. Overall, findings indicated that children diagnosed with ADH D had slower verbal responses and deficits in sustained attention. These results indi cate that the various measures used were successful in measuring different but related varia bles and are consistent with the notion of the multifaceted construct of executive function s. Guidelines for the Assessment of Attention and Exec utive Functions Although historically ADHD has been categorized as a psychiatric disorder, educational systems are being required to become mo re cognizant of its impact on school functioning and the challenges with identifying and providing accommodations for
45 children diagnosed with disorders of attention (Ang ello et al., 2003; Koonce, 2007). However, the assessment, diagnosis, and treatment o f children diagnosed with ADHD continues to be perceived as a complex task due to issues of comorbidity, developmental changes, gender sensitivity, and the multidimension al nature of attention (Koonce, 2007). Thus, the multi faceted nature of ADHD has served a s a catalyst for generating a wealth of curiosity and research towards increasing unders tanding of this disorder. Various methods of assessment include the use of clinical i nterviews, behavior rating scales, and psychological tests (Mandal et al., 1999; Simonsen & Bullis, 2007). Many endorse the use of a multi-method assessment protocol involving a clinical interview with caregivers, behavioral observation of the child, behavior ratin g scales to be completed by multiple informants, and administration of clinic-based meas ures (Barkley, 1997; Frazier, 2004; Koonce, 2007; Mandal et al., 1999; Simonson & Bulli s, 2007). In summary, there is currently not an endorsed gold standard battery of approved instruments for assessing attention disorders. Consequently, there remains gr eat uncertainty and a number of questions regarding the instruments that provide th e best utility and performance for assisting in the identification of children with si gnificant deficits in attention (Frazier et al., 2004). Angello and colleagues (2003) recommend the use of a multi-method assessment approach in capturing a childÂ’s behavior and pervas iveness of impairment across settings. Instruments and strategies that have commonly been employed throughout this assessment include behavior ratings, direct observa tions, review of school records, and assessment of academic skills (DuPaul & Stoner, 199 4; Mandal et al., 1999). Combining various assessment strategies assist in minimizing the limitations that are associated with
46 the use of any one method or instrument. Researcher s caution that rating scales do not provide exhaustive information about the child, env ironmental variables, or information relevant to response function. Thus, behavior ratin g scales have limited utility in rendering a formal diagnosis and are inappropriate for use in conducting functional behavioral assessments specifically related to hypo thesis testing. Lastly, clinicians are cautioned from relying on personal preference for a particular scale in determining its selection for assessment. Further research is warra nted to link assessment information to specific intervention strategies as well as determi ning guidelines for appropriateness and effectiveness of various rating scales in treatment monitoring activities. Finally, authors suggest that research is necessary in identifying t he best method for aggregating data from multiple informants and multiple sources in ma king diagnostic decisions (Angello et al., 2003; Frazier et al., 2004). With respect to efficiency among all of the differ ent assessment tools available there is great variability in regards to the format amount of information collected, time required for administration, and degree to which th ey are practical and efficient in a school setting (Frazier et al., 2004; Simonsen & Bu llis, 2007). According to past studies, only behavioral rating scales were considered to be highly useful and efficient tools in the assessment of attention deficits in the school sett ing (Simonsen & Bullis, 2007). Although relevant within a comprehensive assessment in conjunction with interviews, observations, etc., behavior rating scales alone ar e insufficient for making an informed decision about diagnosis. Due to the large and incr easing numbers of referrals for attention problems, it is necessary to develop and evaluate a system of assessment that balances effectiveness and efficiency (Simonsen & B ullis, 2007).
47 Simonsen and Bullis (2007) advocate for the use of a multiple gating system as an ideal solution for addressing this need. Multiple g ating is defined as a way to identify a set of screening measures with established predicti ve validity with each measure adding uniquely to identify children with specific disorde rs and diagnoses. The measures are administered sequentially so that the least intensi ve measures are given first (e.g., checklists), and more intensive measures are admini stered later (e.g., interviews, observations). This approach is proposed to balance the costs and benefit of an in depth assessment by reserving the most comprehensive asse ssment for children who are most likely to be identified with ADHD or other disorder s of attention. Preliminary results from this study indicated that children with ADHD c ould be classified with the appropriate subtype on the basis of parent/observer ratings of student behavior with 88% accuracy. Future studies are needed to validate thi s system with a larger sample size and to establish cut scores indicating the need to prog ress to the next gate of assessment and when further assessment is unnecessary. Despite lim itations, this study proposes to take an initial step in addressing the need for a standa rd assessment protocol in the assessment and intervention of attention and ADHD. Similarly lacking in research is the recommended us e of specific assessment tools and measures within an assessment protocol in exami ning attention. Given the wide variability in preference with respect to adherence and application of diagnostic methods Koonce (2007) incorporated a case scenario that spe cified the age range and gender of a fictitious child and attempted to identify school p sychologistsÂ’ assessment practices with children presenting with attention symptoms. Variab les such as time spent on performing specific activities, frequency of test use, percent age of attention referrals, and types of
48 test batteries were among factors explored. School psychologists selected several systems of direct behavior observations thus reflecting the importance of gathering information across multiple settings to provide a better unders tanding of the childÂ’s behavioral strengths and areas of concerns. The majority of re spondents (92%) noted that traditional psychological assessments were an important part of the assessment battery; however, there is limited evidence supporting their usefulne ss in diagnosing attention disorders. It is hypothesized that since there is a relatively hi gh incidence of learning problems among children with deficits in attention the considerati on of intelligence and academic achievement testing may be warranted. Finally, resp ondents did not rate the endorsement of neuropsychological tests and use of CPTs with hi gh frequency. However, it appears that school psychologists are becoming more sensiti ve to the emerging literature regarding the importance of neuropsychological test ing in the evaluation of attention and executive functions. Certainly, current research highlights the relevan t role of cognitive deficits and particularly impairments in attention and executive functions to be considered a core part of ADHD (Barkley, 1997; Koonce, 2007). The results of this investigation emphasize the importance of identifying the current assessment an d decision-making methods that school psychologists are employing in their practic e. In addition, it may also be useful for researchers to develop strategies to introduce neur opsychological tests in a way that would be conducive for use in school settings, whic h are typically the primary work settings of school psychologists. The inclusion of such assessment tools will enable school psychologists to utilize current research fi ndings in the assessment of attention and
49 executive functions for which the literature has co ntinued to highlight as a central role in ADHD and disorders of attention (Barkley, 1997; Koo nce, 2007; Angello et al., 2003). Developmental Changes in the Assessment of Executiv e Functions and Attention As previously mentioned prevalence figures of indi viduals exhibiting general Â“attentional difficulties,Â” are substantial. In a s tudy conducted by Kellam et al. (1975) between 15% and 25% of an epidemiological sample of low socioeconomic status (SES), urban African American 1st grade children were reported to exhibit moderate t o severe attentional problems as indicated by teacher report s. Furthermore, Rutter, Tizard and Whitmore (1970) report approximately 30% of an epid emiological population of school age children as presenting with attentional difficu lties (Rebok et al., 1997). The limited data in regards to the developmental changes in att ention of normative samples indicate that the ability to sustain attention and inhibit e xtraneous distractions appear to increase with age particularly between the ages of eight and 10 years, with skills reaching adult levels by adolescence. Additional research supports that sustained attention reaches rates of stabilization between the ages of eight and 10 y ears but increases significantly between the ages of 11 years through adulthood (Rebok et al ., 1997). Given such variability in results, it is important to demonstrate the extent of normal change and continuity of various subtypes of attention over time. In an attempt to examine the developmental traject ory of attentional performance by subtype and the possible influence of gender on the development of attention, Rebok and colleagues (1997) followed a cohort of 435 urba n children, who had previously participated in an epidemiological study of attenti on, into early adolescence. Assessment measures that were utilized include the NIMH Labora tory of Psychology and
50 Psychopathology (LPP) battery which consisted of 12 standard tests that were designed to measure four different aspects of attention (focus, execute, shift, encoding) according to the Mirsky model of attention (1991). Correlation a nalyses were conducted to assess the degree of stability of attentional performance acro ss time. Results indicated significant reductions in omissions errors and improvement in r eaction times from ages 8-13 years on different administrations of the Continuous Perf ormance Task (CPT), a measure of sustained attention, with effects varying by task d ifficulty level and gender. In addition, there were significant improvements across age on m easures of attentional focus and response execution. Overall, the most rapid changes in attention occurred between the ages of eight and 10 years with more subtle changes occurring between the ages of 10 and 13 years. In examination of the effects of task gender, and interaction across age groups, gender was purported to make a significant contribution to change in reaction time from age 10-13 years with females outperformin g males. Researchers indicate that results highlight the importance of developmental e pidemiological approaches for assessing and predicting the normal development of attentional function in school age children (Rebok et al., 1997). Similarly, Klenberg, Korkman, Lahti-Nuuttila (2001 ) conducted a study in an attempt to provide more insight into the developmen tal progression of attention and executive functions in preschool and school age chi ldren. Researchers sought to replicate the results of previous developmental studies by us ing a new set of neuropsychological measures proposed to tap into the functions of both attention and executive functions. The participants consisted of 400 Finnish children, aged 3-12 years who had previously participated in the standardization of the Finnish version of the NEPSY (Korkman et al.,
51 1997, 1998). Each age group consisted of 38 to 41 c hildren and was composed of approximately 50% boys and 50% girls. Relative matu rity was noted by leveling off in performance across the age groups which first occur red at the age of six years on the Statute subtest, which is a subtest assessing the i nhibition of movements or vocalizations. Additionally, relative maturity was observed at the age of seven years on tasks of shifting attention, and at the age of eight years on the Tow er subtest. Lastly, at the age of 10 years researchers indicated that relative maturity was ac hieved in the subtests of focused attention as well as visual and auditory attention tasks. These results are in accordance with Rebok et al. (1997) who also found rapid chang es in several components of attention to occur between ages eight and 10 years of age and only gradual changes to occur beyond this age. Overall, the present results parallel beliefs prop osed by BarkleyÂ’s (1997) model of inhibition, sustained attention, and executive func tions. According to this theory, inhibitory functions are thought to serve as basic functions for more complex executive functions. The observed developmental stages provid e further support for the multidimensional nature of attention and executive functions. In collaboration with previously presented literature and as an overall r eview of the executive function and attention literature, there is general implication that attention is an evolving cognitive process (Cooley & Morris, 1990). A critical trend i n the early development of attention is a shift from external to voluntary control. There i s considerable evidence to indicate that age influences executive functions and specifically attentional performance. For example, the capacity to sustain attention for longer durati ons of time, inhibit inappropriate responses, and shift attention are skills that have been shown to become more efficient
52 throughout childhood and adolescence (Pascualvaca e t al., 1997). Furthermore, as children mature they increase in their ability to m aintain their responsiveness longer, and are more flexible and discriminating in their proce ssing of information towards more systematic and logical methods of exploration (Warn er-Rogers et al., 2001). Many studies examine children from a wide age range and thus inconsistencies in the literature are hypothesized to be heavily influenced by age-re lated variables (Warner-Rogers et al., 2001). Anderson and colleagues (2001) plotted the develop ment of executive skills through late childhood and early adolescence to add ress gaps in the literature. Participants were divided into six groups based on age (11-11.11 years, 12-12.11 years, 13-13.11 years, 14-14.11 years, 15-15.11 years, 1617.11 years). Utilized measures included tests of intellectual ability, and measure s designed to assess executive function skills in the areas of attentional control, attenti onal shifting, memory, and goal setting. Overall, results indicated a relatively flat develo pmental trajectory for executive functions during late childhood and early adolescen ce, in comparison to the rapid maturation that has been documented in early and mi ddle childhood. Gender differences offered some suggestions for a crossover effect occ urring around ages 12 and 13 years when females appeared to become more efficient than males on a range of tasks. The results of this study support the multidimensional nature of executive functions and the importance of assessing the range of skills across the developmental time span with sensitivity to differences across age ranges. Despi te the difficulties with operationalizing the measures used to study different components of attention, the review of these studies provide information concerning the developmental pr ogression of attention and executive
53 functions. Comprehension of the progression of deve lopment allows researchers to appropriately assess specific attention and executi ve functioning abilities with relevant age groups. In attempting to target developmental changes occur ring within a subcomponent of attention, Betts, McCay, Maruff and Anderson (20 06) focused on the development of sustained attention in children. The capacity to su stain attention plays a key role in childrenÂ’s school performance, influencing the chil dÂ’s ability to maintain concentration over long periods of time in order to integrate lar ge amounts of information. Therefore, impairments in sustained attention are likely to in fluence the childÂ’s capability to acquire and integrate new skills and knowledge (Betts et al ., 2006). Participants were divided into three age groups: 5-6 years, 8-9 years, and 11-12 y ears. All participants completed a computer-administered battery of nine neuropsycholo gical subtests that were designed to tap into aspects of attention and information proce ssing reported to be sensitive to the subtle changes in performance. Rapid growth was obs erved to occur from five to six years and eight to nine years of age. A development al plateau was evident from eight to nine and 11-12 years with only minor improvements o ccurring during the latter school age years. Overall, increasing age was associated w ith improved performance with five to six year old children presenting with the greatest degree of variability in performance. These differential findings were interpreted to su ggest that the skills underpinning performance on measures of sustained attention disp lay varying developmental trajectories. Performance decreased on the higher l oaded tasks regardless of age. These findings are consistent with the adult literature, which has established a trend featuring a decrease in correct responses and an increase in re action time as task load increases.
54 These data demonstrate that when task demands becom e too great, participants are unable to cope thus leading to deterioration in performanc e. The current research has applications in educational settings where this kno wledge can be employed to design schedules that most effectively use the limited hou rs in a school day. It is suggested that children best attend when small amounts of informat ion are presented through effective presentation, such as computers and classroom games (Betts et al., 2006). Gender Considerations in the Assessment of Attentio n and Executive Functions Despite the pervasiveness of attention deficits an d their detrimental effects on child functioning, little is known about the factor s that influence attentional performance in typically developing children (Pascualvaca et al ., 1997). There have been few studies addressing the impact of certain characteristics su ch as age, gender, and environmental factors including socioeconomic status, and family background on attentional performance. The majority of the studies on attenti onal performance in children have failed to adequately address gender and the few ava ilable studies have utilized single measures. Since these tests measure specific compon ents of attention, results cannot be generalized to other attention processes. Furthermo re, evidence from various lines of research suggest that boys and girls may present wi th differing attentional profiles and in disorders that are characterized by attention probl ems. ADHD is not only more commonly diagnosed in boys but the disorder is now implied to be expressed differently between genders (Pascualvaca et al., 1997). Underst anding the role of gender on attentional performance may help researchers in dif ferentiating normal gender differences from variations in the manifestation of disorders c haracterized by impaired attention.
55 Pascualvaca et al. (1997) conducted a study with 4 35 first and second-grade participants (214 boys, 221 girls) between the ages of seven and eight years selected from a larger sample who had participated in a collabora tive study by the Prevention Research Center of the Johns Hopkins University and Baltimor e City Public Schools. The objectives of the study focused on assessing the di fferences in attentional capacities in boys and girls in a nonclinical, unselected sample. Measures that were selected for use in this study were based on the theoretical model prop osed by Mirsky and colleagues (1991) suggesting that four separate processes or elements of attention include the ability to focus maintain or sustain focus over time, change or shift attention, and encode information. Specifically, measures that were admin istered include selected subtests on the WISC-R (Digit Cancellation, Coding, Arithmetic, Digit Span), the Continuous Performance Test (CPT), WCST, and the Peabody Pictu re Vocabulary Test-Revised as an estimate of verbal intelligence. A series of ANC OVAs controlling for age were computed to compare boys and girls on the various a ttention measures. Overall results indicated that gender did indeed h ave an impact on attentional performance. Findings suggested that girls were mor e skillful at focusing their attention on a particular target, ignoring distractions, and executing a rapid response. Authors purport that some of the gender differences, partic ularly characteristics reflecting impulsivity or disinhibition may reflect difference s in maturation rate. Other gender differences in attentional performance were alleged to reflect differences in brain organization since some of the brain regions involv ed in the support of attentional functions are not fully myelinated until adolescenc e (Robbins, 1996). Authors offer the argument that since girls appear to perform better on tasks designed to assess attentional
56 performance, only girls who present with the most s evere degree of difficulty may be identified or diagnosed with attentional disorders and thus, more girls may benefit from treatment than simply those who are identified as r eaching clinical levels. Future studies exploring differential norms for boys and girls pre senting with attentional problems are encouraged since according to the findings of this study girls may not be identified as often as boys since they tend to perform better on measures of attention. Ecological Validity of Performance-based Tests The evaluation of an individualÂ’s ability to funct ion adaptively in the real world is reported to be directly assessed by less than half of clinicians who conduct comprehensive neuropsychological assessments (Price Joschko & Kerns, 2003). Rather, tests of various cognitive domains are used in an a ttempt to acquire insight about daily functioning abilities. Â“Ecological validityÂ” is def ined as the predictive and functional relationship between an individualÂ’s performance on a set of neuropsychological measures with the individualÂ’s behavior across comm on settings including home, work, school, and community (Price, Joschko & Kerns, 2003 ). In addition, the term Â“veridicalityÂ” refers to the extent to which tests can predict functioning in real world settings. According to researchers, in order to est ablish such ecological validity various relationships should be established including: (1) the relationship between individual cognitive functions and the specific targeted behav iors to be predicted, (2) the relationship between cognitive functions and psycho logical test scores, and lastly (3) the relationship between test scores and the specific t argeted behaviors (Price, Joschko & Kerns, 2003). In an attempt to improve the ability to assess and capture an individualÂ’s functional abilities beyond that accounted for by s cores on tests of intelligence, adaptive
57 behavior measures have been formulated to provide o bjective assessments of everyday functioning. Adaptive functioning refers to the dai ly activities that are required for personal and social self-sufficiency and is typical ly measured through checklists or interview format (Price, Joschko & Kerns, 2003). Ov erall, although a number of studies have attempted to determine the ability of neuropsy chological tests to predict functioning in real world settings they have primarily been con ducted with adult populations. In order to assess the ecological validity of test s of executive functions, Burgess et al. (1998) conducted a study that aimed to compare the ecological validity of 11 measures of executive function taken from six different test s, and relate findings to a set of behavioral characteristics indicative of daily func tioning. Research participants consisted of adult patients with varying neurological disorde rs (e.g., head injuries, dementia, cerebrovascular accidents, etc.). Overall findings indicated lower performance by patients across all measures of executive function as compar ed to the control group. In addition, performance on neuropsychological executive functio n test measures reflected impairments in everyday life as evidenced by signif icant correlations with observersÂ’ ratings of patient problems in daily living. Furthe rmore, such correlations were higher as compared to values obtained between tests of memory reading, and naming. As per findings, authors suggest utilizing clinical interv iews, questionnaires, ratings scales, or other measures of behavior change to describe daily functioning not readily measurable by commonly used neurological tests of executive fu nctions. Although adult studies are important in providing guidance and informing child studies, ecological validity of neuropsychological tests must also extend to younge r populations.
58 Neuropsychological assessments are frequently used by clinicians and assumed to relate to real world or adaptive functioning, howev er, there is limited data to support such predictions, particularly with child populations. I t is also notable that among this existing database, the literature examining the relationship between adaptive functioning and attention tests are far scarcer. For this purpose, Price, Joschko and Kerns (2003) sought to determine the association between several types of attention and adaptive functioning in a heterogeneous clinical sample. Four components of a ttention, namely focused attention, sustained attention, verbal span, and complex worki ng memory were assessed as separate components. Measures of attention were selected bas ed on commonly administered tests purported to measure each of the four proposed subc omponents of attention. The Scales of Independent BehaviorRevised (SIB-R; Bruininks, Woodcock, Weatherman & Hill, 1996) was used to collect data related to adaptive functioning. The overall results of this study provide evidence suggesting that correlations between measures of attention and adaptive functioning are beyond the relationship be tween attention and intelligence, and between intelligence and adaptive functioning. Auth ors indicate the important implications of findings for neuropsychologists sin ce conclusions and recommendations depend heavily on ecological validity of tests used Future studies are called upon to provide further information on the meaning of commo nly used neuropsychological tests as well as to consider the developmental difference s in the assessment of various components of attention. Performance Measures and Rating Scales Thus far, the discussions of assessing attention a nd executive functions have focused solely on the administration of performance tasks. As such, they are brief
59 samplings of abilities designed to assess attention al capacities and are typically administered in laboratory settings utilizing manip ulatives, paper and pencil, visual presentation via computer, or audiotape. Rating sca les on the other hand, are used to record behaviors that reflect deficits in naturalis tic settings (e.g., home, school, community) across a specified period of time (e.g., months, years). Behavior ratings of attention are pervasive in the child clinical liter ature and are frequently used in accordance with measures of attentional performance (Mandal et al., 1999). However, past studies examining attention and executive func tions have oftentimes failed to use performance measures and rating scales collaborativ ely, presumably because they reflect different theoretical backgrounds and analyses (i.e ., cognitive and behavioral) (Cooley & Morris, 1990). Despite their utility in assessing t he development of attentional performance, there are limitations associated with performance measures. Specifically, there are concerns in regards to confounding proces ses and task impurity since attention can only be measured relative to another activity, derived or observed. Many of the available performance tasks involve other perceptua l (verbal, spatial), cognitive (memory, semantic concept formation), and output sy stems (motor) (Cooley & Morris, 1990; Manly et al., 2001). Furthermore, the potenti al for confound is more relevant when assessing children who typically show a greater var iability than adults along these overlapping dimensions. Thus, it is challenging to discriminate changes in attention from other general maturational processes or to determin e normative progression through reliance on results obtained solely from performanc e measures (Manly et al., 2001). Parent and teacher rating scales are most commonly used in the evaluation of attentional deficits. Some measures are designed sp ecifically for the measure of behaviors
60 reflecting attention whereas others combine multipl e dimensions of child psychopathology. Current research emphasizes the im portance of including multiinformant, multimodal assessments and collaboration of data in describing a childÂ’s school, home, and community functioning particularl y in the assessment of ADHD and impairments of executive functions (Mandal et al., 1999; Mares et al., 2007). However, it is cautioned that Â“moreÂ” does not necessarily equat e with Â“betterÂ” assessments as multiple informants do not always result in adequat e interrater reliabilities and are oftentimes subject to bias (Tripp, Schaughency & Cl arke, 2006). Agreement between parents and teachers are typically modest at best w hen assessing symptomatology and oftentimes depend on the scales that are used (Ache nbach, McConaughy & Howell, 1987; Mares et al., 2007). Discrepancies are also l ikely to occur from behavioral variability in different situations however, the qu estion remains as to how best to integrate information for the purposes of decisionmaking. Despite the discrepancy in information obtained from multi-informants, collabo rating teacher and parent reports are likely to enhance the likelihood of the early ident ification or in the very least recognition of executive function problems not otherwise recogn ized by clinicians. This provides the opportunity for teachers and parents to implement b ehavioral and academic programming prior to the onset of any learning, social, or beha vioral problems (Mares et al., 2007). Parent and Teacher Reports Formal neuropsychological testing and clinical obse rvations continue to indicate support for the growing consensus that executive fu nction deficits are central in the impairments that are observed in individuals diagno sed with ADHD. This theory suggests that tasks and expectations for performance often d iffer across the home and school
61 environments due to the differing demands of execut ive functions (Mares et al., 2007). Across these different settings, children presentin g with various deficits of executive functions tend to display observable behavioral dif ferences relative to their peer group. To date, the BRIEF is the only behavioral rating sc ale that has been developed to explore childhood executive functions in home and school en vironments. In addition, the majority of the studies utilizing the BRIEF have pr imarily limited their scope to parent ratings while excluding perceptions of teachers. Mares and colleagues (2007) found teachers overall to report greater levels of executive functioning impairment on all scales of t he BRIEF as compared to parent ratings. Authors interpreted these results to sugge st that either teachers may be better able than parents to identify executive function deficit s in children diagnosed with ADHD or that children diagnosed with ADHD may experience mo re difficulties in a structured school setting than at home. Overall, parents and t eachers agreed that impairments in planning, organizing, and inhibition were main indi cators for a positive diagnosis. Tripp et al. (2006) found teacher ratings to be more sens itive, specific, and accurate in the overall classification of diagnostic groups (ADHD). Combining parent and teacher measures determined consistency overall with teache r ratings. However, results support the importance of including parent and teacher rati ng scales in the assessment of attention and ADHD. The use of rating scales with teachers an d parents are encouraged for their cost efficient and least intrusive characteristics (Tripp et al., 2006). Although specific measures are not recommended ove r others, information from both teachers and parents are strongly encouraged a nd are particularly important in determining strengths and weaknesses in child profi les (Dewey, Crawford & Kaplan,
62 2003). Previous research has suggested that when ca refully elicited and interpreted parental concerns in particular can be just as accu rate as developmental-behavioral screening tests in identifying children with disabi lities (Glascoe, 2000). For clinicians, parents and teachers are valuable reporters since t hey are familiar with the childÂ’s history as well as current levels of functioning across mul tiple settings, which in many cases are necessary for the recognition and description of ps ychopathology and learning difficulties (Dewey et al., 2003). Most importantly, they can al so identify strengths as well as weaknesses that are not always captured through sta ndardized testing (Dewey et al., 2000; Koonce, 2007). In order to examine the usefulness of parent repor t measures of childrenÂ’s cognitive functioning and academic abilities, Dewey and colleagues (2003) sought to determine whether parental reports were able to con tribute information beyond that of data obtained from a standardized psychometric asse ssment. Researchers examined the role of parental reports of everyday cognitive func tioning in the ability to distinguish between children with reading disabilities (RD), AD HD, and combined ADHD + RD. Parent reports resulted in a significant increase i n the number of children correctly classified as compared to the use of psychometric m easures alone. These findings are consistent with previous research indicating that p sychometric assessments are not particularly useful in differential diagnosis of ch ildren with attentional deficits (Barkley, 1997). Moreover, laboratory measures of attention c urrently available may not be sufficiently sensitive to the heterogeneity of etio logical factors contributing to attentional deficits (Warner-Rogers et al., 2001).
63 Ecological Validity of the BRIEF and Assessment of Attention As previously highlighted in this review, a caveat in neuropsychological assessment has been the limited ecological validity among many neuropsychological assessment measures. Ecological validity has become an increasingly important focus in neuropsychological assessment and particularly rele vant in the study of executive functions, which coordinate oneÂ’s cognitive and beh avioral capacities with real world demand situations (Gioia & Isquith, 2004). To date, support for the ecological validity of the TEA-Ch as a standardized measure in the assessm ent of attention has been presented. However, given the problems involved in the assessm ent of executive functions and attention with performance tasks, Gioia and colleag ues (2000) sought to develop a structured behavior rating system, which successful ly assesses psychological and neuropsychological functions while maintaining a hi gh level of ecological validity. In developing the BRIEF, emphasis was placed on th e construction of an ecologically valid instrument (Gioia & Isquith, 200 4). Researchers believed that teachers and parents possessed a wealth of information regar ding a childÂ’s executive function behaviors across different settings. They report th at the impetus for the development of the BRIEF originated from the clinical need to be m ore efficient and systematic in the collection of information on the childÂ’s everyday m anifestation of executive function behavior across relevant settings (e.g., home, scho ol, community) (Gioia et al., 2000). The behavior rating system of the BRIEF has several unique properties that contribute to its ecological validity, including: (1) the ability to collect information about a childÂ’s executive functions from observation of behavior in natural settings, (2) the ability of information across a variety of different executive subdomains to be gathered in a
64 relatively short period of time, (3) the ability to collapse observations of a childÂ’s executive functions over an extended time interval, and (4) the ability to readily translate scores to peer-reference normative data (Gioia et a l., 2000). A review of the existing empirical research examin ing the BRIEF generally supports the ecological validity of this instrument (Mangeot, Armstrong, Colvin, Yeates & Taylor, 2002; Gilotty et al., 2002; Gioia & Isqui th, 2004) and has also predicted social adaptive behavior in different clinical populations (Mangeot et al., 2002; Gioia & Isquith, 2004). Giolotty et al., 2002 found significant rela tionships between measures of the Vineland Adaptive Behavior Scales (VABS) and BRIEF. Specifically, the subdomains of Initiate and Working Memory on the BRIEF were signi ficantly correlated with almost all aspects of adaptive behavior as assessed by the VAB S. Limitations of this study include the small sample size as well as the restricted dat a from only parents that are subject to interpretational bias. Although the limitations pre clude the drawing of conclusions about causality, the findings emphasize the importance of further investigating the role of adaptive functioning and its relationship with beha vior of executive functions. Mangeot et al. (2002) conducted a study designed t o examine the long-term executive dysfunction following a specialized popul ation of childhood traumatic brain injury (TBI) patients using the BRIEF. Neuropsychol ogical measures of executive functions demonstrated modest associations with par ent ratings on the BRIEF. Parent ratings were however strongly related to measures o f emotional and behavioral adjustment as well as adaptive behavior for childre n in all groups. Thus, findings indicate that deficits in the behavioral manifestations of e xecutive functions are related to general measures of psychosocial and adaptive functioning l ending further support to the
65 ecological validity of the BRIEF. Overall, the BRIE F appears to provide a more comprehensive understanding of daily executive func tions than any one of the commonly used neuropsychological tests. Future studies are c alled to determine whether parent ratings of executive functions on measures such as the BRIEF are also related to results of neuroimaging and for various clinical population s. Implications of all these studies provide an important bridge toward understanding th e impact of test-based deficits on a childÂ’s everyday adaptive functioning (Gioia & Isqu ith, 2004). TEA-Ch and the Assessment of Attention The assessment of ADHD has traditionally relied on information obtained from clinical interviews, behavioral observations, and d ata from parent/teacher behavior rating scales (Heaton et al., 2001). Although multi-inform ant questionnaires provide clinicians with useful information regarding childrenÂ’s attent ional impairments in everyday settings that would otherwise be uninformed, objective test measures of attention can also provide clinicians with a more controlled, standardized fir st-hand assessment that is less susceptible to reporting bias (Heaton et al., 2001) Although neuropsychological testing of children diagnosed with ADHD has increased, test s that focus on attentional abilities typically continue to assess only one component of attention while neglecting other subcomponents proposed by various other models of a ttention. According to authors, although most researchers agree that attention exis ts as a multidimensional construct, the majority fail to incorporate this understanding int o their assessment strategies (Cooley & Morris, 1990). A relatively recent measure that has been alleged to have considerable potential for use in assessing various components of attentio n is the Test of Everyday for Children
66 (TEA-Ch; Manly, Robertson, Anderson & Nimmo-Smith, 1999). In developing the TEACh, the aim of the authors was to adapt the measure s that had previously been proven to be effective in assessing adult attention for appro priate use and application to children (Manly et al., 2001). Initially, researchers attemp ted to minimize the demands on memory, reasoning, task comprehension, motor speed, verbal ability, and perceptual acuity to avoid the interference of confounding fac tors while still maintaining the demands on the targeted attentional system (Manly e t al., 2001). This measure presents several advantages when compared to current existin g objective measures designed to assess attention. One distinct advantage of the TEA -Ch includes the assessment of multiple components of attention. Furthermore, the reliability of the TEA-Ch is enhanced by multiple subtests that assess each factor of att ention. Most importantly, the dimensions of attention on this measure are model and theory d riven. Neuropsychological tests that simultaneously assess multiple constructs and abili ties such as memory, motor speed, and response inhibition in addition to attention run in to the issue of potential confounds when attempting to assess the single construct of attent ion (Heaton et al., 2001). Thus, the TEA-Ch is hypothesized to be a more ecologically va lid measure of attention as it uses tasks that attempt to simulate real world attention al demands. (Heaton et al., 2001; Manly et al., 2001). Authors are careful to note that the subtests of the TEA-Ch are not measures of attention; rather they are measures of auditory and visual detection, of counting, of response speed, etc. Separable attention processes are inferred constructs that are believed to contribute significantly to differences in performance on these tasks. In attempts to minimize complexity of instructions, in corporating practice sessions, and
67 reducing demands of perception, memory, and reasoni ng, the aim in developing this instrument was to minimize variability due to non-a ttentional factors. In an attempt to provide support for the value of differentially assessing attentional functions, Manly and colleagues (2001) administered subtests of the TEA-Ch and WISC-III to a sample population of 24 boys all meeting diagnostic criteria for ADHD. Participants were administered the Score!, Sc ore DT, Walk DonÂ’t Walk, Sky Search, Sky Search DT, and Opposite Worlds subtests of the TEA-Ch. In addition, all boys were administered the Vocabulary and Block Des ign subtests from the WISC-III. Results indicated deficits on performance across al l subtests of the TEA-Ch that were designed to assess sustained attention and attentio nal control however, notably no deficits in performance on measures of speeded-visual search tasks. There are no hypotheses provided as to the differences in performance on tw o of three components of attention. However, it is noted that these results are in line with previous findings that have emphasized both sustained attention deficits as wel l as difficulties with the suppression of prepotent responses resulting from abnormalities in the right frontal systems for populations diagnosed with ADHD (Barkley, 1997). Overall, there have been few published studies uti lizing the TEA-Ch and although studies have been conducted with both Australian pa rticipants and sample populations from the United Kingdom, researchers have failed to examine the performance of children from the United States on this measure. He aton et al. (2001) conducted a study to address this lack in research as well as to prov ide a more detailed examination of the utility of this measure in a clinical population of children diagnosed with ADHD. Participants in this study were divided into an ADH D group (n = 63) and a non-ADHD
68 clinical control group (n = 23) all between the age s of 6-15 years. Fifty-one subjects were male which reflects the greater prevalence of males as compared to females diagnosed with ADHD in clinical settings. Parents and teacher s completed the Revised ConnorsÂ’ Parent and Teacher Rating Scales, and all children were administered the TEA-Ch, reporting nine of the 13 scaled scores which is sup ported for producing an ideal model as indicated in the manual. Overall, results of the study indicated that the c linical ADHD group performed significantly worse than the clinical control group on subtests assessing sustained and attentional control/shifting while the groups perfo rmed comparably on the tasks assessing divided attention. These findings indicate that chi ldren diagnosed with ADHD show distinct deficits in attention rather than global d eficits and note the importance of considering specific subcomponents of attention. Li mitations of this study include the relatively low sample size as well as the use of st imulant medication by nearly half of the children included in the clinical ADHD group. Futur e studies examining the correlation between TEA-Ch performance and results obtained on various other tests of attention, executive function, and ratings on parent and teach er reports of behavior are warranted in further exploration of this instrument. In an attempt to examine the validity of the TEA-C h, Sutcliffe and colleagues (2006) compared children diagnosed with ADHD when o n and off stimulant medication. This study examined sustained and attentional contr ol by administering four subtests of the TEA-Ch and a reading assessment. Parent ratings were obtained at two different points in time via the SNAP-IV rating scale (Swanso n, 1992) to indicate their childÂ’s attentional state on and off stimulant medication. Participants from the clinical group
69 performed significantly lower than controls on the TEA-Ch subtests when off stimulant medication. In contrast, when children were taking stimulant medication no significant differences were found on three of the four TEA-Ch subtests. There were no differences on measures of intelligence or reading between clin ical and control groups and between clinical groups when on and off stimulant medicatio n. Overall, results indicated some support for the sensitivity of specific subtests of the TEA-Ch in assessment of attention, however not all measures produced significant diffe rences when medication status was change. Limitations of this study include the small sample size and therefore lack generalizability of these findings. Purpose of the Study Despite the wide variety of cognitive tests availab le that are purported to assess various executive functions, there has been little work to validate the use of these measures in predicting real world functioning, part icularly in children where characterization of executive function deficits are specified to attentional difficulties. Although many performance-based measures and parent /teacher rating scales exist for assessing a wide range of behaviors and adaptive fu nctioning in children, specific measures of attention and subcomponents of attentio n have not been fully explored. As previously highlighted, a current limitation in neu ropsychological assessment is the concern with questionable ecological validity among many of commonly endorsed measures. More research and test development to imp rove the ecological validity of neuropsychological assessment measures is needed. This study seeks to assess and further establish th e ecological validity of the Behavior Rating Inventory of Executive Function (BR IEF) in conjunction with the Test
70 of Everyday Attention for Children (TEA-Ch) in a sa mple of school age children. Although research to date supports the ecological v alidity of the BRIEF and the TEA-Ch in isolation and in conjunction with other measures no current studies exist that correspond to the study of the TEA-Ch, BRIEF and as sessment of adaptive functioning. Furthermore, this study will attempt to better defi ne the relationship between behaviors indicative of executive functioning, attention, and adaptive functioning in a normative group of school age children. Currently, there is n o commonly used measure of attention that can fully capture all proposed subcomponents o f attention. Clinicians have previously relied on a wide variety of formal measu res including clinical interviews, behavioral observations, and rating scales to gain a more thorough understanding of the child. It is likely that this study will bring to l ight the utility and appropriateness for the use of neuropsychological measures not only in clin ic settings but also in the schools. Lastly, the inclusion of parents, teachers, and sch ool psychologists in the assessment of attention and deficits of executive function defici ts allow for a common language towards collaboratively generating interventions and strate gies that will be applicable across the home, school, and community settings.
71 Chapter 3 Method The purpose of this study was to investigate and to define better the relationship between attention and corresponding behaviors that have been designed to represent executive functions and social/adaptive functioning in a normative sample of school age children. More specifically, this study sought to e xplore the correlation between ratings of varying subcomponents of attention (e.g., select ive attention, sustained attention, and attentional control/switching), executive function behaviors, and ratings of social/adaptive functioning that were obtained sepa rately from reports by parents/caregivers and elementary school teachers. Additionally, gender considerations were examined with aims to determine how this facto r may affect the degree of relationship between the proposed variables. This c hapter presents information regarding participants who were involved in this study, the m ethod through which data were collected, and the analyses conducted. Participants The participants in this study were 48 school age c hildren ranging in age from 8years, 0-months to 10-years, 11-months, a parent/ca regiver per child, and his/her elementary school teacher. All students who took pa rt in this study were enrolled in elementary schools housed within a large school dis trict located in and around West Central Florida during the 2008-2009 school year. T he established range for this study
72 included students who were enrolled in school durin g critical transition times in regards to the school curriculum during these specific ages and concurrent stages of development. Specifically in school, these transitions correspon d with increased expectations for organizational demands and the introduction of task s such as complex writing assignments, book reports, and multiple-choice test s that require coordination and integration of various skills and strategies (Meltz er, 2007). Researchers have also documented rapid advances in systematic problem sol ving, increased ability to think autonomously and the emergence of strategic and con trolled self-regulation, skills of inhibition, and the ability to maintain attention o n complex problems, planfulness, and reflection during this developmental period (White, 1965; Welsh & Pennington, 1988; Welsh et al., 1999). The school district from which child participants w ere recruited is the eighth largest district in Florida and has 160 school site s and centers including 65 elementary schools. The most recently available student demogr aphics for this particular school district were reported for the 2006-2007 school yea r. The district reported 90,284 students enrolled in school with 19,559 (22%) stude nts noted to be eligible for free and reduced lunch. Statistics describe the population o f students enrolled in this school district to be composed of 57.4% White, 19.3% Hisp anic, 21.8% Black, 1.3% Asian/Pacific Islander, and 0.2% American Indian/Al aska Native. Comparatively, the sample population for this study was composed of 87 .5% (42) White, 8.3% (4) Black, and 4.2 % (2) Hispanic. Similar to the district pop ulation, students who were eligible for free and reduced lunch composed 20.8 % (10) of the sample population. Of the 48
73 participants included in this study, 52% (25) were females and 48% (23) were males with a mean age of 9.04 years (Table 1). Table 1 Distribution of Demographic Variables Across Child Participants 8-0 to 8-11 (N = 15) 9-0 to 9-11 (N = 16) 10-0 to 10-11 (N = 17) Gender Female 9 (19%) 8(17%) 8(17%) Male 6 (13%) 8(17%) 9(19%) Note N= 48 Inclusion Criteria Students between the ages of 8-years, 0 months to 10-years, 11 months of age were eligible to take part in this st udy. Furthermore, students who were only proficient in English and enrolled in the gene ral education classroom setting for the majority of their academic studies (>75%) were incl uded in this study. Elementary school teachers involved in this study were employed full time in the classroom allowing significant time for student interaction and observ ation. The purpose of this criterion ensured that teachers had adequate knowledge of the studentsÂ’ skills and abilities considered representative of the student. A primary caregiver was defined as an adult with whom a child lives and the adult assumes respo nsibility for the child. Teachers and caregivers taking part in this study had continuous opportunities to observe the child across activities involved with daily functioning. Exclusion Criteria Students who were excluded from participating in this study were enrolled in their classrooms for less than six months; outside of the established 8years, 0-months to 10-years, 11-months age range, a nd/or native speakers of a language
74 other than English. In addition, students who prese nted with evidence of deafness, blindness, specific learning disorders, and/or psyc hiatric disorders were also unable to participate in this study. Elementary school teache rs who were excluded from this study were individuals who were not employed full time in the classroom or those who had been employed for less than six months in their cur rent classrooms. Primary caregivers who were excluded from this study included individu als who did not live with the child or assume responsibility for the child. Furthermore primary caregivers who were not fluent English speakers were also excluded from thi s study. The number of child, teacher, and caregiver partic ipants were selected based on the sample participants accessible to the researche r and through statistical determination. In an a priori power analyses (Cohen, 1988), sample size is computed as a function of the required power level (1-), the pre identified significance level (), and the population effect size to be detected with probability 1-. A power analysis was conducted a priori in order to determine the sample size necessary to obt ain power of .80 for various effect sizes. A power analysis was generated for multiple regression analysis with three predictors (sustained, selective, shifting/attentio nal control attention) for two separate regression models (parent/caregiver and teacher). T he alpha level was adjusted according to the Bonferroni adjustment in order to control fo r Type I error (Glass & Hopkins, 1996). The G*Power 3 power analysis program (G*Powe r 3; Faul, Erdfelder, Lang & Buchner, 2007) was used for this analysis. This is a computer program that was designed as a standalone application to consider several typ es of statistical tests commonly used in social and behavioral research (Faul et al., 2007). Cohen (1988) provides guidelines from which to interpret practical use for standardized e ffect sizes; a small effect size is defined
75 as f2 < .02, a medium effect size is defined f2 = .15, and a large effect size is defined as f2 < .35 (Cohen, 1988; Faul et al., 2007). For example, the final sample size for this study included 48 parents/caregivers, 48 children, and th eir teachers, which is calculated to provide a statistically significant effect if an ef fect size of f2 < .35 (large) is established (Table 2). The power analysis indicated a minimum s ample size of 42 for a large effect size, a minimum sample size of 91 for a medium effe ct size, and a large effect size requiring a minimum of 651 participants. Based on t he literature and current studies, a large effect size was expected thus the sample size was hypothesized to be sufficient according to the power analysis. Table 2 Protocol of Power Analyses N for Small, Medium, and Large Effect Size at Power = .8 for = .05 Test = .05 Large effect f2 < .35 = .05 Medium effect f2 = .15 = .05 Small effect f2 < .02 N N N Significant r 42 91 651 Research Design The design of this exploratory analysis study is co rrelational. The independent variables were entered into a regression model in a n attempt to account for variance in the dependent variable, and to identify the variabl es contributing most to the prediction of the dependent variable. Attention scores as measure d by the TEA-Ch, posed as independent variables (predictor variables) while e xecutive function behaviors as measured by the BRIEF-Teacher, BRIEF-Parent/Caregiv er, and social/adaptive
76 functioning as measured by the ABAS-II Parent and T eacher represented the dependent variables (outcome variables). Instrumentation The measure and rating scales that were selected to assess the primary constructs of interest in this study included executive functi on behaviors, subcomponents of attention, and social/adaptive behavior. Three diff erent measures were utilized for this study: a) two behavior rating scales (information t o be provided by teachers and parents/caregivers); b) standardized neuropsycholog ical measure of attention. Behavior Rating Inventory of Executive Function (BR IEF). The Behavior Rating Inventory of Executive Function (BRIEF; Gioia et al ., 2000) was derived from theoretical and empirically based definitions of executive func tion and from items submitted by practicing clinical neuropsychologists (Malloy & Gr ace, 2005). The parent and teacher versions of the BRIEF were designed to assess execu tive functions in the home and school setting, respectively (Gioia et al., 2001). Each BRIEF questionnaire contains 86 items and requires approximately 15-20 minutes for completion. Children are evaluated on a 3-point Likert scale ( never, sometimes, often ). The BRIEF contains three general indices: Behavioral Regulation Index (consisting of three scales: Inhibit, Shift, and Emotional Control), Metacognition Index (consisting of the remaining five scales: Initiation, Task Organization/Planning, Environment al Organization, Self-Monitoring, Working Memory), and a Global Executive Composite ( GEC) which combines the sum of all eight scales. The Behavioral Regulation Inde x (BRI) represents a childÂ’s ability to shift cognitive set and modulate their emotions and behaviors appropriately via inhibitory control. Behavioral regulation enables a child to s uccessfully engage in active, systematic
77 problem solving, and supports appropriate self-regu lation. The Metacognition Index (MI) represents the childÂ’s ability to initiate, plan, o rganize, and sustain future-oriented problem solving in working memory. Specifically, th is index is interpreted as the ability to cognitively self-manage tasks, and reflects the childÂ’s ability to monitor his or her own performance (Gioia et al., 2000; McCandless & OÂ’Lau ghlin, 2007). The MI is designed to relate directly to a childÂ’s ability to actively problem solve in a variety of contexts and situations. The GEC is a summary score that incorpo rates all eight clinical scales of the BRIEF. This standard score will be used as the main measure for representing executive function behaviors for this study and to which atte ntion will be compared and related. All raw scale scores are transformed into t-scores for interpretation. Scale scores that are greater than t = 65 are considered clinically signi ficant (Gioia et al., 2000; McCandless & OÂ’Laughlin, 2007). Specific items for the BRIEF scales were generated from actual descriptions of behavioral executive function difficulties obtained during clinical interviews with parents and teachers, ensuring good face and content validi ty. Item-category membership was determined by the sorting decisions of 12 clinical neuropsychologists, as well as statistical analyses (item-total correlation analys es, principal factor analyses, and interrater agreement; Gioia et al., 2000), to suppo rt each scale and index structure of the BRIEF. The normative data samples were obtained thr ough public and private school settings across rural, urban, and suburban areas th roughout the state of Maryland. Both parents and teachers for 296 children, permitting e xamination of agreement between raters, completed rating forms. The BRIEF Parent Fo rm was normed on 1,419 children aged 5-18 years. The Teacher Form was completed by a sample consisting of 720
78 teachers. A specific breakdown of the Parent and Te acher Form normative samples by age and gender are provided in the manual (Gioia et al., 2000). Factor analytic studies of the normative sample wer e conducted and provided support for two underlying factors, which were used to develop the MI and BRI indices (Gioia et al., 2000). The BRIEF scales have demonst rated strong psychometric properties with the majority of correlations falling in the mo derate to high range. Internal consistency reflects the degree to which items in a single scale are measuring the same underlying construct (Glass & Hopkins, 1996). The t ypical internal consistency statistic that is reported is CronbachÂ’s alpha (), which is derived as the mean correlation of all possible sets of scales within a scale (Cohen, 1992 ). For both the Parent and Teacher Forms of the BRIEF reported internal consistency is high, ranging from .8 to .98. (Gioia et al., 2000; Malloy & Grace, 2005). Inter-rater re liability assesses the degree to which two independent observers rate a child in a similar manner (Glass & Hopkins, 1996). This measure provides an indication of scale stability a cross multiple informants as well as across multiple settings. The correlation (reliabil ity coefficient) between parent and teacher ratings of the same child is typically lowe r (i.e., .3 to .5) than parent-parent or teacher-teacher inter-rater reliabilities for ratin g scales (Achenbach et al., 1987). The correlations derived between like scales of the BRI EF for parent and teacher raters were moderate (overall .32) for the normative group (Gio ia et al., 2000; Malloy & Grace, 2005). Notably, correlations between parent and tea cher ratings for two of the scales were significantly lower for Initiate (r = .18) and Orga nization of Materials (r = .15). Such findings are stated to be a result of the differenc es in environmental structure that exists between the home and school settings (Gioia et al., 2000).
79 Test-retest reliability indicates the stability of a measure over time for behaviors that are presumed to remain relatively constant (Gl ass & Hopkins, 1996). Test-retest reliability was examined in both clinical and norma tive subsamples for the Parent Form and in a subsample of the normative sample for the Teacher Form. The mean test-retest correlation across the clinical scales ranged from .76 to .85. Results were collected over a two-week period for the normative subsample rated b y parents/caregivers. For the parent clinical subsample, the mean test-retest correlatio n for the clinical scales ranged from .72 to .84 over an average of three weeks. The test-ret est correlations for the Teacher Form were the strongest and ranged from .83 to .92 over a threeto fiveweek interval (Gioia et al., 2000). Validity refers to the accuracy with which an instr ument measures the intended construct. Content validity is defined as the degre e to which an instrumentÂ’s item content reflects the constructs that it was proposed to mea sure (Glass, & Hopkins, 1996). The construct validity has been examined by correlating the BRIEF scales with a variety of other measures with which it theoretically should c orrelate. To explore the convergent and divergent validity of the BRIEF, the individual scales and summary indexes have been correlated in a variety of clinical samples wi th other rating scale measures of attentional and behavioral functioning. McCandless and OÂ’Laughlin (2007) evaluated the validity and clinical usefulness of the BRIEF in id entifying children diagnosed with ADHD, and to determine if select BRIEF scales could accurately differentiate the inattentive subtype from the combined subtypes of A DHD. In addition, the study also examined the interrater reliability and convergent validity between parent and teacher reports on the BRIEF by considering the relationshi p between teacher and parent reports
80 on the BRIEF with a broad-range behavior rating sca le (BASC; Reynolds & Kamphaus, 1992). Correlations ranged from .24 (Shift) to .70 (MI) for the BASC Attention scale and from .26 (Shift) to .83 (Inhibit) for the BASC Hype ractivity Scale. Both the Inhibit and Emotional Control scales were more strongly correla ted with the BASC Hyperactivity scale as compared to the Inattention scale. For tea cher ratings, six of the eight BRIEF scales were significantly correlated with the BASC Inattention scale, and all of the BRIEF scales were significantly correlated with the BASC Hyperactivity scale. Significant correlations ranged from .26 (Shift) to .67 (Initiate) for the BASC Inattention scale and from .24 (Working Memory) to .59 (Inhibit ) for the BASC Hyperactivity scale. Finally, the overall agreement between parents and teachers on the BRIEF, as indicated by the GEC, was minimal (r = .13). Parent and teach er ratings were significantly correlated for three of the eight BRIEF scales, inc luding Inhibit, Plan/Organize, and the Monitor. Parent-teacher agreement was lowest for th e Emotional Control, Initiate, and Organization of Materials. This study provides support for convergent validity and clinical utility of the BRIEF as parent and teacher ratings on BRIEF scales were found to be significantly associated with both reports of inattention and hyp eractivity as indicated on the BASC. In support of the clinical utility, the Working Memory scale was effective in distinguishing the ADHD groups from the nonclinical groups, wherea s the Inhibit scale was able to distinguish between subtypes. With similar aims in the exploration of construct validity of the BRIEF, Sullivan and Riccio (2006) conducted a study examining the characteristics of the Frontal Lobe/Executive Contr ol (FLEC) scale of the BASC Parent Rating Scales and its relationship with the BRIEF-P arent Form. Scores on the FLEC were
81 correlated with scores on the BRIEF-Parent Form, an d the ConnersÂ’ Parent Rating Scales Revised-Short Form (CPRS-Short Form) to determine t he extent to which they were related. Scores on the FLEC were significantly corr elated with all of the scales on the BRIEF-Parent Form and the CPRS-Short Form with corr elations ranging from .45 (Organization of materials) to .83 (GEC) and .63 (A DHD Index) to .77 (Oppositional), respectively. Overall, the highest correlations bet ween the BASC-FLEC scale and the BRIEF were obtained on the global scales of the BRI EF (i.e., BRI, MI, and GEC). Thus, authors concluded that the BRIEF and BASC-FLEC appe ar to measure the same dimensions including both the cognitive and behavio ral dimensions of executive dysfunction and providing further support for the c onstruct validity of the BRIEF. Adaptive Behavior Assessment System-2nd edition (ABAS-II). The Adaptive Behavior Assessment System-2nd edition (ABAS-II; Ha rrison & Oakland, 2003a) is a comprehensive, multidimensional, norm-referenced be havior rating scale that is designed to assess the practical, everyday skills that are r equired by individuals to function and meet environmental demands, including those needed to effectively and independently care for oneself and to interact with others. Five ABAS-II forms are available for different age groups to be rated by different rater s. The age range for the instrument is birth to 89 years. Ratings are determined by observ ations across various settings by multiple raters. This assessment instrument aids in the classification and diagnosis of disabilities as well as providing a profile of indi vidual strengths and limitations in adaptive behavior and can function as an ongoing to ol of progress monitoring of adaptive skills (Harrison, 1999). Furthermore, researchers s upport the use of the ABAS-II as a contribution of an ecologically valid instrument fo r assessing various adaptive
82 functioning skills of individuals. The ABAS-II may be used to evaluate individuals with learning difficulties, ADHD, and other impairments related to motor, speech and language, hearing, and neuropsychological disorders to determine how the individual is responding to daily demands from the environment. I ncluded items are stated to be developmentally sensitive and appropriate for use w ith young children. The ABAS-II is estimated to take approximately 15-2 0 minutes for the completion and scoring of the long version, and 510 minutes for the short version. Since some adaptive skills are required in specific setti ngs apart from others, or are more observable in particular settings, separate forms f or teachers (daycare providers), parents (primary caregivers), and adults were designed. Thi s allows for the assessment of the adaptive skills that are most suitable for the spec ific setting and type of informant (Harrison & Oakland, 2000). Additionally, two age-s pecific versions are available for the parent (contains 232 items) and teacher forms, one for children age 0-5 years, and the other for children and adolescents age 5-21 years. The ABAS-II is currently the only instrument that i ncorporates the current American Association of Intellectual Disabilities ( AAID) guidelines for evaluating the four general areas of adaptive behavior (Conceptual Social, Practical, and General Adaptive Composite or GAC. The ABAS-II consists of the Conceptual Domain, which includes the skill areas of Communication, Function al Academics, Self Direction, and Health and Safety. The Social Domain includes the S ocial and Leisure skill areas. The Practical Domain includes the skill areas of Self-C are, Home Living, Community Use, Health and Safety, and Work. Motor skill area score s are available on the two forms appropriate for children up to age 5 years. The GAC compares an individualÂ’s global
83 adaptive skills to the adaptive skills of others in the same age group from the standardization sample. Composite scores are derive d for each of the areas. For both forms, respondents have a four-choice option format ( Always or Almost Always When Needed, Sometimes When Needed, Never or Almost Neve r When Needed, or Is Not Able ) to rate the frequency that a behavior is correctly performed by the child when needed. All scores are based on age-related norms. The General Adaptive Composite and domain composite scores have a mean of 100 and a standard deviation of 15. Skill area standard scores have a mean of 10 and a standard deviation o f 3. Age-based percentile ranks and test-age equivalents are included up to the test ag e of 22 years. Procedures taken for test development and standardi zation are thoroughly described in the ABAS-II manual. The standardizatio n sample was based on the United States census data provided in 1990 (School and Adu lt forms) and 2000 (Infant and Preschool forms). Thirty-one separate age groups wi th at least 100 participants in each group were assessed using the Infant-Preschool, Sch ool Age, and Adult forms. Further specification of groups, maps, and tables are provi ded in the manual. The ABAS-II was standardized between December 1998 and October 2002 however, there is no mention of the use of normative information from the original ABAS. In addition to typically developing participants, the standardization sample included 20 clinical samples as well (e.g., ADHD, autistic disorder, and visual impairme nt). Reliability studies were conducted as part of the s tandardization process and provided evidence of a high degree of internal cons istency. Furthermore, the majority of skill areas reported internal consistency coefficie nts of .90 or greater. The average internal consistency coefficient for the sampleÂ’s G AC ranged from .97 to .99. Three
84 separate studies evaluating test-retest reliability from teachers and parents were conducted and support very high reliabilities. The test-retest interval was approximately 2 weeks. Tables by age group for each of the ABAS-II forms as well as for the various scores are included. Sample sizes were composed of 30 to 207 participants. The majority of the GAC correlations were above or near .9 for t eacher, parent, and adult forms. Confirmatory factor analytic data indicate that uni dimensional and multidimensional models most accurately describe the ABAS-II standar dization results. All five forms were examined using a host of factor analytic techniques Although the single-factor, GAC model proposed the closest fit with these data, the re was additional evidence to also support the three-factor model (Conceptual, Social, Practical). The one-factor model is consistent with the construct of overall adaptive f unctioning therefore, the GAC summary score will be the score representing a childÂ’s soci al adaptive functioning for the purposes of this study. According to the developers, the ABAS-II was founde d upon the theoretical basis that each of its skill areas should be minimally re lated to each other, highly related to their respective adaptive domains, and strongly cor related with the GAC. Thus, the construct validity of the ABAS-II is supported by i ntercorrelational data across and among the skill areas, domains, and the GAC. Across all forms, the intercorrelations between the skill areas were in the moderate range (.4Â’s to .7Â’s). The correlations between the skill areas and their adaptive domains range be tween .55 and .78, whereas skill areas and the GAC are .64 to .82. Lastly, correlations be tween the adaptive domains and the GAC fall between .78 and .93. These results suggest that the ABAS-II fits the theoretical basis for which it was designed.
85 A number of specific validation studies are reporte d in the manual and support concurrent validity of this instrument with other f requently used measures. For the adaptive behavior-related scales, small samples (<6 0 children in each sample) were used to compare scores on the ABAS-II with the Vineland Adaptive Behavior Scale (VABSCE), VABS Interview Edition (VABS-IE), Scales of In dependent Behavior-Revised (SIB-R), and the BASC. Correlations reported betwee n the Adaptive Behavior Composite on the VABS-CE and the GAC was .75 for the Teacher/ Daycare Provider form and .84 for the Teacher Form. The correlation between the G AC and the VABS-IE Adaptive Behavior Composite was reported to be .7. The lowes t correlation of .57 was reported between the GAC and the SIB-R Broad Independence st andard score. Additionally, the correlation between the GAC and the BASC Adaptive S kills Composite was .8 for a sample of 37 preschool aged children. The ABAS-II manual lists three negative correlation s between the personality dimensions assessed by the BASC and the ABAS-II sco res. However, it is expected that as behavior problems increase, adaptive behavior sc ores decrease. The correlation between the GAC and the BASC scales for Externalizi ng Problems was -.49, -.39 for Internalizing Problems, and -.66 for the Behavior S ymptoms Index. Additional studies including the relationship betwe en the ABAS-II and various measures of intelligence (e.g., WPPSI-II, WAIS-II, WISC-IV) and achievement (e.g., WASI, WIAT) measures briefly noted moderate correla tions ranging in the .4Â’s to .5Â’s and .6Â’s respectively. This finding is consistent w ith previous research suggesting that adaptive behavior and cognitive functioning are sep arate but related constructs (Rust & Wallace, 2004).
86 As indicated by the current literature, by the inst ruments developers, and by test reviewers, the ABAS-II appears to be an important a ddition to the field of assessment of social adaptive behavior. In particular, a great de al of effort has been put forth to update a new test in order to expand the applicable age rang e and consider the revisions proposed according to AAID guidelines. There is an abundance of reliability and validity data included in the manual, which supports its high int ernal consistency and reliability. The ABAS-II is proposed to be an appropriate tool for s creening, placement, diagnostics, and research purposes. The Test of Everyday Attention for Children (TEA-Ch ). The Test of Everyday Attention for Children (TEA-Ch; Manly et al., 1999) is a standardized clinical battery that was developed as a modification of the 8-subte st Test of Everyday Attention (TEA; Robertson, Ward, Ridgeway & Nimmo-Smith, 1996). Thi s measure was designed to assess various components of attention in adults. A lthough this instrument was developed according to a model of attention that was original ly proposed for adults, it has been modified accordingly for use with school age childr en as a measure of how well students can control their attention to achieve goals (e.g., paying attention to the teacher, focusing on a boring task, etc). The measures from the TEA t hat best represented these three factors of attention were considered in the design of the TEA-Ch and new tasks were added based on the research and current literature on childrenÂ’s attention (Manly et al., 1999). The TEA-Ch is particularly relevant for identifying the patterns of attentional problems as well as to facilitate development of tr eatment and management programs in children diagnosed with or suspected of having atte ntion difficulties. Thus, the distinct
87 advantage of the TEA-Ch relates to its inclusion of multiple components of attention which sets it apart from most other measures of att ention (i.e., Continuous Performance Tasks, Speeded Classification Tasks, and the Wiscon sin Card Sorting Tests), which generally examine only one component (Heaton et al. 2002). Differing patterns of attentional problems may be apparent, requiring sep arate assessment in order for a comprehensive description of an individualÂ’s diffic ulties and strengths. The TEA-Ch allows the assessment of the pattern of attentional difficulties and strengths by including a variety of activities that emphasize distinct typ es of attentional skill. The TEA-Ch is normed for children ages 6-16 years a nd is composed of nine game-like subtests reported to assess three main do mains of attention: focused (selective) attention, sustained attention, and attentional con trol/switching. The test employs various sensory modalities during test administration, incl uding visual, auditory, and motor modalities. To further accommodate its use with chi ldren, practice items are included within subtests and standardized instructions requi re the child to paraphrase directions in order to ensure their comprehension. The reliabilit y of the TEA-Ch is enhanced by devoting multiple subtests to each factor of attent ion. Five subtests assess sustained attention while two subtests assess selective atten tion and attentional control/switching. The TEA-Ch is comprised of a screener version and a full nine-subtest administration. The brief screener allows for an estimate of perfor mance on each of the three factors of attention and dual task performance. The full admin istration is reported to take approximately one hour to complete and the subtest screener is composed of the first four subtests, which takes approximately 20-25 minutes. There are two parallel forms (A and B) included in the TEA-Ch that allow for retesting purposes. However, the developers
88 caution that attention skills measured by the TEA-C h develop rapidly in childhood thus complicating the interpretation of a childÂ’s second performance unless it is reasonably close in time to the first assessment. Therefore, i n assessing the reliability, all of the retests were conducted within 20 days of the first TEA-Ch administration. Test-retest reliability was assessed on a random subgroup of 55 children from across the age range. They were re-administered the TEA-Ch between 6 and 15 days following the first administration. PearsonÂ’s correlations were compute d between raw performance scores at test 1 and test 2. The reported correlations ranged from .64 (Score!) to .92 (Opposite Worlds) and computed percentage agreement from 57-7 6.2% (Manly et al., 1999). In addition, correlations were computed while controll ing for age due to the wide age range of the TEA-Ch sample and reports correlation coeffi cients falling between .65, and .87, with percentage agreements from 71-76.2%. These dat a do not provide an estimate of the long-term effects of prior completion of the TEA-Ch on subsequent performance over long periods of time, nor of more than one retest o ver any period of time. Therefore, it is cautioned that particular care should be used in ma king comparisons with the normative data under these circumstances. In developing the TEA-Ch, the researchers aimed to adapt measures that had proven effective in adult attention to be applicabl e for use with children. Although it is difficult for neuropsychological tests to completel y eliminate confounds, the developers of the TEA-Ch have attempted to minimize the demand s on memory, reasoning, task comprehension, written expression, motor speed, ver bal ability/comprehension (reading), and perceptual acuity, while maintaining the demand s on the targeted attentional system, and thus providing a more objective measure of chil drenÂ’s abilities. By comparison,
89 many other current neuropsychological tests assess multiple constructs and combined abilities including memory, response inhibition, mo tor speed, and various executive functioning skills, in addition to attention. To date, there are very few published studies util izing the TEA-Ch apart from the normative studies conducted by the test developers. Few studies have evaluated the ability of the TEA-Ch to assess subcomponents of at tention in normative as well as clinical samples of children (Heaton et al., 2001; Manly et al., 1999). However, both construct and concurrent validity have been explore d. As described in the manual the normative sample was composed of 293 children betwe en the ages of 6 and 16 years recruited from state schools located in Melbourne, Australia. Equal numbers of boys and girls were tested and stratified into six age-bands (6-7 years, 7-9 years, 9-11 years, 11-13 years, 13-15 years, and 15-16 years), which takes i nto account the rapid development of attention skills in children. Exclusion criteria in cluded previous head injury or neurological illness, developmental delay or sensor y loss, and/or referral for attentional or learning problems, and the need for special educ ation services. Additional information describing specific gender and age distribution of this sample is included in the manual. An important question regarding the validity of the TEA-Ch is the extent to which the separate factors of selective attention, sustai ned attention, and attentional control/shifting identifies distinct patterns of pe rformance. The relationship between the observed scores in the TEA-Ch and the three latent constructs defined were examined in a Structural Equation Model. This technique provide s a number of measures that provide the best fit of the hypothetical model to the obser ved data. Three explanatory factors (latent variables) were entered as the a priori mod el (sustained, selective, and
90 shifting/attentional control). These were linked to variables designed to be representative of each subtest included in the TEA-Ch. The three-f actor model, with the variables from the TEA-Ch linked to only one factor thus giving a close fit to the data and indicating a representative pattern of performance observed in a large group of children. Overall, it was reported that a unitary model of attention form ed a poor fit to the observed variance, while a three-factor model of sustained attention, selective attention, and attentional control/shifting formed a significant and parsimoni ous fit. In summary, the dimensions of attention proposed and measured on the TEA-Ch are s hown to have been model-based, and theory-driven. The scaled scores generated for each of the subtests representing sustained, selective, and attention control/shiftin g of attention will be used for the purposes of this study. Although the TEA-Ch consists of nine subtests, 13 t otal scores are generated. The manual, however, reports that the results of the fa ctor analysis yielded an ideal model using 9 of the 13 scores. Furthermore, according to the developers of the TEA-Ch, four scores comprising the screener may also be utilized to provide an estimate of the three attention factors and dual task performance. The cu rrent study will follow this model and utilize the following four scores: Sky Search Atten tion Score, Creature Counting Timing Score, Score!, and Sky Search DT. This screener als o offers a plausible measure to be administered in the school setting where time restr ictions are applicable. Each subtest represents a different subcomponent of attention an d represent sustained, selective, attention control/shifting of attention, and dual t asks of attention. According to the TEA-Ch manual, 96 children from th e normative sample were administered the Stroop task (Trenerry, Crosson, De boe & Leber, 1989), Trails Test
91 (Spreen & Strauss, 1991), and Matching Familiar Fig ures Test (Arizmendi, Paulsen & Domino, 1981). Comparisons between measures of the TEA-Ch with these other tests of attention were made using partial correlations (con trolled for age) on raw scores. Subtests of the TEA-Ch would be expected to correlate highly with instruments that are designed to measure similar components of attention while lo w correlations would be expected between these tests and other subtests of the TEA-C h proposed to measure separate aspects of attention. The Stroop task, Trails A and B are designed to assess components of selective attention similar to Sky Search and Ma p Mission on the TEA-Ch. When the relationships between these tests were examined, co rrelations ranged from .31 to .69 and reported to show statistically significant relation ships on this capacity of attention. In addition, non-significant relationships were observ ed with the other subtests of the TEACh emphasizing the separable nature of the attentio n factors. Additionally, the MFFT, which places demands on a childÂ’s ability to resist impulsive responding, shows significant relationships with a number of TEA-Ch m easures requiring similar capacities designed to measure attentional control and aspects of sustained attention (coefficients ranging from .2 to .4). In addition, non-significan t correlations were derived for the remaining subtests of the TEA-Ch when compared to t hese measures. Additional assessments of validity were undertaken to determine whether TEACh subtests reflect general ability and academic ac hievement as opposed to specific areas of attentional functions. If relationships between tests of intellectual ability and subtests of the TEA-Ch are obtained it may be interpreted th at an additional assessment of attention apart from general intellectual ability m ay be redundant. However, given that IQ measures average performance across cognitive domai ns, attention would be expected to
92 contribute additionally to the variance of scores. Pearson correlations were derived between the scaled scores for the TEA-Ch variables and four WISC-III scaled scores (Vocabulary, Similarities, Block Design, Object Ass embly) for 160 children. Overall, IQ accounted for little of the variance across TEA-Ch subtests for children with average IQ scores, and thus IQ scores were not reported to acc urately predict how a child may perform on the TEA-Ch (Manly et al., 1999). Coeffic ients ranged from -.002 to .31. The highest correlation obtained was between Creature C ounting accuracy, where children are instructed to switch repeatedly between counting up ward and counting downward according to the printed arrows, and overall IQ (.3 1). These results suggest that overall the TEA-Ch is assessing abilities that are not othe rwise accounted for by measures of general ability. Similar to the comparison between subtests of the T EA-Ch and the WISC-III, researchers compared measures of academic achieveme nt to subtests of attention by administering the Reading, Spelling, and Arithmetic scales on the Wide Ranging Achievement Test-Revised (WRAT; Justak & Wilkinson, 1984). Overall, subtests of the TEA-Ch that were designed to assess selective atten tion did not show strong relationships with tests of academic achievement (.09 and .13). H owever, the subtests from the TEACh that were designed to measure sustained attentio n did indicate correlations across each of the scores of academic achievement. This finding suggests that a relationship exists only between the ability to focus and sustain atten tion with achievement in the areas of Reading, Writing, and Spelling, but not with select ive and shifting attention. In summary, the previously discussed studies indica te the usefulness of the TEACh for assessing important subcomponents of childre nÂ’s attention. The TEA-Ch has
93 demonstrated its relation to other measures of atte ntion, intellectual ability, and academic achievement in addition to construct validity. An e xploration of the technical properties of the two previously presented rating scales of ex ecutive function behaviors (BRIEF) and social/adaptive functioning (ABAS-II) indicate that scores obtained on these instruments are adequately reliable and valid. The technical properties of these instruments, along with their specificity, lack of intrusiveness, and appropriateness for the characteristics of this sample make them viable ins truments for use in this study. Procedures Ethical considerations. Several steps were taken to protect all research participants. Approval was obtained from the Univer sity of South Florida Institutional Review Board (IRB), Polk County Schools Office of A ssessment, Accountability, and Evaluation, and from the principals of each of the participating elementary schools before data collection and contact with classroom teachers parents/caregivers, and students was made. Parents were provided an informed consent for m to sign describing the purpose of the study, rights of the participants, nature of th eir involvement, measures to ensure participant anonymity, methods in which data were t o be collected, and a description of how data will be stored during and after research c ompletion. The explanation of participantsÂ’ rights included information regarding confidentiality, ability of participants to withdraw, refusal to answer any question, and em phasis on voluntary participation. Teachers were informed of the purpose of the study, confidentiality procedures, and voluntary participation in a similar manner. Rating forms completed by teacher/parent pairs and test protocols were assigned a number. Id entifying information were removed
94 from these documents. All of the forms and letters that were distributed to parents were thereafter organized according to student number an d entered into a computer database. Training Activities Training of test administration. The author and co-authors of the TEA-Ch were contacted by the primary investigator in regards to best practices and recommended methods for becoming trained in, and training other professionals on the administration of the TEA-Ch. The primary investigator was trained in the administration of the TEACh by referring to and adhering to the standardized conditions and administration rules as indicated in the manual. Furthermore, the primary i nvestigator observed the administration of the TEA-Ch by licensed clinical p sychologists and administered the measure under supervision across numerous occasions In addition, practice administrations were completed with fellow graduate students and school age children between the ages of 8-0 years and 10-11 years. In o rder to ensure integrity of scoring procedures, all protocols were scored, and reviewed by a licensed clinical psychologist. An overview of the TEA-Ch and data collection proce dures to the administration and educators employed in the targeted school district specific to the current study are provided below in Table 1. Training activities rela ted to administration and utility of the TEA-Ch were developed by referring to and consideri ng the guidelines set forth by the American Educational Research Association and the A merican Psychological Association. The study was introduced to the district Psychologi cal Services, Senior Manager and to the school psychology faculty in the summer of 2008 to familiarize potential TEACh administrators and data collectors with the purp ose and methods of this study.
95 Following this brief overview, a detailed workshop was conducted for the purposes of providing training on the TEA-Ch including administ ration, scoring, and interpretation. Lastly, an Integrity Checklist was composed, review ed, and distributed to school psychologists for the purposes of collating the dat a and employing consistent information and record gathering procedures across all research ers. Table 3 Overview of TEA-Ch Training Activities and Data Col lection Training Activity Purpose Data Collected Study Introduction (Summer 2008) Power Point Presentation for School Psychologists (school district) Purpose of Study/Rationale 1. Literature Review 2. Data Collection 3. Test Measures School District Letter of Support (Senior Manager of Psychological Services) TEA-Ch Training and Workshop (In collaboration with District Research Committee and Standardized Test Review Committee) (Fall 2008) 1. TEA-Ch Overview Development and Standardization Clinical and Research Strengths Validity and Reliability 2. TEA-Ch Administration Description of Subtests Materials/Administration Model and Role Play 3. Protocol Scoring Review Scoring Sheet Scoring Rules 4. Interpretation Limitations Confidentiality 5. Discussion and Questions Data Collection (District Research Committee) (Winter 2008-Spring 2009) Integrity Checklist 1. Overview 2. Data Collection Procedures Teacher Consent Form Parent Consent Form Child Assent Form BRIEFParent/Teacher ABAS-II Parent/Teacher TEA-Ch protocol
96 Recruitment of schools for participation The district currently houses 64 elementary schools. Each of the principals from the se schools were contacted by the primary investigator to participate either via emai l or phone. However, only seven (10.9%) principals agreed to allow their teachers t o participate. All teachers designated to 3rd through 5th grade classes were contacted by the primary invest igator or school psychologist either in person, via telephone, or th rough email and invited to participate after permission from the principals of each school was obtained. From 79 eligible 3rd through 5th grade teachers, 17 agreed to participate (21.5%). Teachers were selected based on voluntary participation and also based on whether students enrolled in their classrooms met inclusion/exclusion criteria accordi ng to age, duration of enrollment, presence of any handicapping conditions, and native language spoken by the parents/caregivers and students. Teachers were requ ested to select eligible students according to inclusion and exclusion criteria as pr edetermined for this study to send home consent forms. Only those students returning parent consent forms were included in the study. Only one parent declined to participate in t he study (2%). Teachers were consulted regarding the best approach es for collecting information from parents/caregivers, and the times most conveni ent for the researcher to collect rating forms, and consent forms from the teachers. After t he investigator consulted with teachers, standard procedures were developed and fo llowed throughout the data collection process and across all sites. Modes of c ommunication consisted of sending letters home with students via teachers in the morn ing and afternoons during drop off and pick up times. When letters, consent forms and rati ng packets were completed and returned, teachers from each of the sites were requ ested to contact the investigator via
97 email or through direct communication. Preparatory to receiving data forms, the investigator then contacted the chairperson of the Research Committee via intra-district courier who was designated to receive all consent f orms and behavior rating forms from teachers via intra-district courier. Parent data collection Parental consent forms were provided to consentin g classroom teachers and distributed to parents/careg ivers who were nominated by their teachers according to the caregiver and student inc lusion criteria for participation in this study. The consent forms reiterated confidentiality of all responses and further stated that the purpose of data collection was strictly for res earch use. Parents/caregivers were then asked to return consent forms to their childÂ’s clas sroom teacher. All rating scales were requested for completion by one parent, legal guard ian, or primary caregiver per child participant. Consenting parents/caregivers were sen t a packet through means of the classroom teacher, which included the ABAS-II Paren t and the BRIEF Parent version forms. All rating scales were counterbalanced to de crease possible bias resulting from order effects. Parents of students who were assigne d an even number in the database were instructed to fill out the ABAS-II first while pare nts and teachers of students assigned an odd number were instructed to first complete the re spective BRIEF version. The completion time was predicted to range from 20-30 m inutes for the completion of both rating scales. All sets of child measures were requ ested for completion within a two-week span and the investigator arranged for the complete d forms to be sent to the chairperson of the Research Committee via intra-district courie r. If parents/caregivers giving consent to participate in the study did not return rating s cales by the due date indicated in the cover letter, an additional letter was sent out as a reminder. All parents/caregivers
98 returning completed behavior rating forms were give n a $10 Wal-Mart gift card as a token of appreciation for their time and input. Teacher data collection Teachers received the ABAS-II Teacher, and the BR IEF teacher versions to complete for children for whom parental consent was obtained. Rating scales were counterbalanced in a similar fashion th rough procedures described above for parents. The completion time for teachers was predi cted to range from 20-30 minutes. All sets of child measures were requested for completio n within a two-week span and the investigator arranged for the completed forms to be sent to the chairperson of the Research Committee via intra-district courier. Teac hers were given a $10 Wal-Mart gift card for each set of behavior rating forms complete d (one per student). Student data collection Informed written assent was obtained the day of t he TEACh administration. During this time, the investigat or, who completed all test administrations, picked the child up from his/her c lassroom and walked them down to the predetermined testing area (e.g., office, library, conference room, etc.). The research project was introduced to the student and they were asked whether they were interested in participating. If interest was expressed, the inves tigator proceeded to obtain assent, which consisted of reviewing the entire protocol of the s tudy, obtaining signatures authorizing assent, explaining confidentiality, and answering a ny questions that arose. Students received a copy of the assent form. The total time estimated for the administration of the nine subtests of the TEA-Ch (Form A) was proposed t o be approximately 20 minutes. Students were given the opportunity to ask question s throughout the testing as well as to take breaks. In addition, students were provided th e opportunity to terminate testing if
99 they wished to do so without experiencing any reper cussions. None of the students opted to terminate testing and all components of the TEACh were administered in one session. Integrity Checklist. Data collection steps and integrity were measured b y the Integrity Checklist (see Appendix C). This checklis t described and outlined the individual steps required in the distribution and collection o f protocols, rating scales, and consent/assent forms. This reference form was poste d on the front of each packet including teacher and parent/caregiver rating forms In composing the Integrity Checklist, each step of the data collection process was broken down and individual steps were analyzed and defined. Individual steps were defined in terms of adherence (were the correct data collected from the appropriate individ ual) and quality (were the data collected as planned in the order planned). The pri mary investigator designated that all steps would be adhered to and implemented in the or der designated for acceptable treatment and test administration integrity. All st udy materials were collected by the district Research Committee and once each step of t he process was completed and checked off on the Integrity Checklist, completed s cales were sent via courier to the designated individual. The Research Committee docum ented and tracked all forms received through means of an electronic spreadsheet which was continually updated. This dataset was sent to the primary investigator t o ensure accuracy of tracking. Furthermore, each TEA-Ch administration was complet ed by the primary investigator. Overall, a high level of consistency and integrity was maintained across the data collection phase as evidenced by electronic trackin g, consistent test administration, and reference to the Integrity Checklist.
100 Scoring of Protocols Teachers were requested to briefly review parent and teacher completed rating forms to ensure that all q uestions were completed. On two occasions parent completed rating forms were return ed home due to skipped items. Thus, this initial checking method ensured that none of t he data had to be thrown out due to missing data. All scores obtained on the protocols were entered into a database according to an assigned student number. All of the rating sc ales and TEA-Ch assessments were originally scored and re-scored by the primary inve stigator. However, to ensure the highest level of integrity of scoring procedures an d to obtain inter-rater reliability, 25% of the protocols were also randomly selected and re-sc ored by a licensed school psychologist. Inter-rating reliability was generate d through calculation of Cohen's kappa coefficient (Agreements/Agreements + Disagreements) (Cohen, 1960). Only scores producing 100% agreement according to the inter-rat er reliability formula were included in this study. In cases where 100% agreement failed to be attained protocol noted that the data would be re-scored by both raters until this l evel of agreement was achieved. All data that were collected for the purpose of this st udy reached 100% agreement as demonstrated by equivalent scores acquired from bot h raters. Data Analyses The primary objective of this current study was to determine if an individually administered measure of attention and report forms of executive function and social/adaptive functioning are significantly relat ed. In addition, this study examined the ability of the TEA-Ch and its separate factors to p redict outcomes representative of executive function behaviors and social/adaptive fu nctioning. The analyses were conducted separately for teacher and parent measure s to examine the relationship
101 between the sets of variables of the BRIEF and the corresponding version of the ABASII. Separate analyses were conducted for all relati onships between the BRIEF-Parent Form and the ABAS-II Parent/Caregiver version and b etween the BRIEF-Teacher Form and ABAS-II Teacher version. For all analyses in th is study, the significance level was preset to p < .05. Internal Reliability In order to assess the reliability of scores on the BRIEF, ABAS-II and the TEACh the reliability estimates of internal consistenc y were calculated. In situations where multiple raters are requested, an approach to compu ting a consistency estimate of interrater reliability is to compute CronbachÂ’s alp ha coefficient (Stemler, 2004) which was used for this study. CronbachÂ’s alpha coefficie nt is a measure of internal consistency reliability and is useful for understanding the ext ent to which the ratings from multiple raters measure a common dimension. If CronbachÂ’s al pha estimate among raters is low, then this implies that the majority of the variance in the total composite score is due to error variance, and not true score variance (Stemle r, 2004). The reliability coefficient will describe the degree to which the BRIEF, ABAS-II, an d TEA-Ch will represent something other than measurement error. In essence, if two se ts of parallel measures agree perfectly then the computed coefficient should be 1. The reli ability coefficients that are provided for each of the measures estimates the correlation between the obtained scores from parents and teachers and the score on a parallel fo rm of the measure (Glass & Hopkins, 1996). Landis and Koch (1977) suggest that coeffici ent values from .41Â–.6 are moderate, and that values above .6 are substantial. If the sy stems demonstrate poor reliability, then the information that is produced from the scales wi ll not be meaningful. However, if the
102 scales produce strong reliabilities, the informatio n is suggested to be much more meaningful (Glass & Hopkins, 1996). Based upon the data acquired from this study, CronbachÂ’s alpha values ranged from .661 to .94. Th ese scores suggest an adequate level of reliability (Field, 2009). Specific data for eac h assessment and rating scale are presented (Table 4). Table 4 CronbachÂ’s Alpha Scores for Each Questionnaire Measure Subscore Alpha Scores Behavior Rating Inventory of Executive Function (BR IEF) Global Executive Composite (GEC)-Parent Form .922* Global Executive Composite (GEC)-Teacher Form .878* Adaptive Behavior Assessment System-2 nd edition (ABAS-II) General Adaptive Composite (GAC)Parent/Caregiver .940* General Adaptive Composite (GAC)Teacher .878* The Test of Everyday Attention for Children (TEA-Ch ) Sky Search (selective attention) .661* Score! (sustained attention) .731* Creature Counting (attentional control/switching) 881* Sky Search DT (dual task) .721* indicates an adequate level of internal reliabili ty (Aron & Aron, 1997) Univariate and Bivariate Analyses For each measure, descriptive data were collected and provided for the entire sample population. The mean, median, standard devia tion, skewness and kurtosis values of scores on the three measures are reported in tab ular form. Procedures to screen for outliers and linear relationships were instituted t hrough preliminary data checking methods. Bivariate (PearsonÂ’s coefficient) zero-ord er correlation coefficients between all
103 variables included in the study were examined. In a ddition, data were run through a statistical program with and without the inclusion of all possible outliers. This was to ensure that outlier values were not due to coding e rrors. Descriptive data for each measure are provided (Tab le 5). In order to ascertain that the distribution of scores was approximately n ormal, values of skewness and kurtosis were examined. As indicated below the majority of k urtosis values were computed to fall near zero. Typically, positive values of skewness i ndicate an abundance of low scores in the distribution whereas negative values indicate a buildup of high scores. Similarly, positive values of kurtosis indicate a pointy and h eavy-tailed distribution, whereas negative values indicate a flat and light-tailed di stribution (Field, 2009). The further the value is from zero, the more likely that the data a re not normally distributed. After converting scores to z-scores (dividing by standard error), the majority of values were determined to fall below the critical value z +/1 .96, with the exception of the BRIEFParent (GEC) which was slightly leptokurtic. Overal l, kurtosis values in the residuals were not indicated to be significant thus, results inform a relatively normal distribution. Table 5 Sample Sizes, Means, and Standard Deviations of Var iables Mean SD Skewness Kurtosis Min Max Score! 9.38 3.68 -.141 -.845 2 15 Sky Search 10.33 2.68 -.026 -.124 5 16 Creature Counting 9.83 3.30 -.490 .074 1 15 Sky Search DT 6.67 3.62 -.133 -1.180 1 13 BRIEF-Parent (GEC) 56.58 11.69 .035 -1.258 35 77 BRIEF-Teacher (GEC) 58.56 11.48 .302 -.495 41 86 ABAS-II-Parent (GAC) 97.71 12.89 .145 -.803 74 120 ABAS-II-Teacher (GAC) 105.48 11.50 -.678 -.485 80 120 Note N=48
104 Multiple Regression Analysis Multiple regression and correlation techniques were applied to answer the proposed research questions. Interaction effects an d main effects were examined in order to determine whether specific independent variables had an effect on the dependent variables of attention, behaviors of executive func tion, and social/adaptive behavior. A complete description of the statistical procedures for each research question is presented below. Research Question 1 : What is the relationship between attention and ex ecutive function behaviors as determined by the correlation between subcomponent(s) of attention (sustained, selective, shifting/attention al control) and executive function behaviors? Separate multiple regression analyses we re conducted to determine if there was a relation between each subcomponent of attenti on with parent/caregiver and teacher ratings of behaviors informing executive functions. Before conducting the analyses, assumptions of multiple regression analysis were co nsidered including independence and collinearity. Data were graphed to determine linear ity. The general purpose of multiple regression is to examine the relationship between s everal independent or predictor variables and a dependent or criterion variable (Gl ass & Hopkins, 1996). Customarily, the degree to which two or more predictors (indepen dent or X variables) are related to the dependent ( Y ) variable is expressed in the correlation coeffici ent R In multiple regressions, R can assume values between 0 and 1 (Glass & Hopkins 1996). Thus, according to the research questions proposed, data analyses evaluated whether the TEACh scores were able to significantly predict varian ce in scores representative of executive function behaviors as rated by parents/caregivers a nd teachers.
105 Research Question 1a: A correlational design and separate multiple regre ssion analyses were conducted for each subcomponent of at tention (selective, sustained, shifting/attentional control). The predictor variab les were scaled subtest scores from the TEA-Ch (selective, sustained, shifting/attentional control subcomponents) derived from child performance. The outcome variable for each of the analyses was the Global Executive Composite (GEC), BRIEF-Parent form. Research Question 1b : A correlational design and separate multiple regr ession analyses were conducted for each subcomponent of at tention (divided, shifting/attentional control, sustained). The predictor variables were s caled subtest scores from the TEA-Ch (devoted to selective, sustained, shifting/attentio nal control subcomponents) derived from child performance. The outcome variable for each of the analyses was the Global Executive Composite (GEC), BRIEF-Teacher form. Research Question 2: What is the relationship between attention and social/adaptive functioning as determined by the co rrelation between subcomponent(s) of attention (sustained, selective, shifting/attention al control) and social/adaptive functioning? The ecological validity of the TEA-Ch scores was determined by assessing its ability to predict social/adaptive functioning in a sample of school age children. As such, the greater the predictive power of the TEA-C h, the greater the ecological validity as defined in this study. Research Question 2a: A correlational design and separate multiple regre ssion analyses was conducted for each subcomponent of att ention (devoted to selective, sustained, shifting/attentional control). The predi ctor variables were scaled subtest scores from the TEA-Ch (selective, sustained, shifting/att entional control subcomponents)
106 derived from child performance. The outcome variabl e for each of the analyses was the Global Adaptive Composite score (GAC) obtained on t he ABAS-II Parent/Caregiver version. Research Question 2b: A correlational design and separate multiple regres sion analyses were conducted for each subcomponent of at tention (selective, sustained, shifting/attentional control). The predictor variab les were scaled subtest scores from the TEA-Ch (devoted to selective, sustained, shifting/a ttentional control subcomponents) derived from child performance. The outcome variabl e for each of the analyses was the Global Adaptive Composite score (GAC) obtained on t he ABAS-II Teacher version. Research Question 3 : What is the relationship between executive functi on behaviors, and social/adaptive functioning? In orde r to determine the extent to which ratings on the BRIEF-Parent and Teacher Forms were related with the ABAS-II Parent/Caregiver and Teacher versions a correlation al matrix was constructed. Values from this analysis determined the relationship betw een index scores obtained from the BRIEF composed of the Metacognition Index (MI), Beh avioral Regulation Index (BRI), and the GAC of the ABAS-II. This particular analysi s examined the overall significance of the relationship between specific executive func tion behaviors and social/adaptive functioning and examined whether the overlap betwee n the instruments were greater than that expected by chance. Research Questions 4: How does the relationship between attention, execu tive function behaviors, and social/adaptive functioning differ (if any) by gender? The question of the statistical significance of main ef fect and interaction for gender and each of the dependent variables was examined by means of an F -test. Interaction effects are
107 often referred to as moderator effects because the interacting third variable, which changes the relation between two original variables is a moderator variable, which can alter the original relationship (Field, 2009). This analysis was conducted to examine whether the performance of males to females on chil d performance of attention and teacher and parent/caregiver measures differed. Research Question 4a : The influence of gender on dependent measures was individually tested by examining the interaction ef fect between attentional performance as measured by the scaled subtest scores derived on the TEA-Ch and parent/caregiver ratings on the GEC, GAC, respectively. Research Question 4b : The influence of gender on dependent measures was individually tested by examining the interaction ef fect between attentional performance as measured by the scaled subtest scores derived on the TEA-Ch and teacher ratings on the GEC, GAC, respectively.
108 Chapter 4 Results The purpose of this chapter is to provide the resul ts of data analyses that were conducted to answer the research questions. Data an alysis, data screening, considerations of assumptions and results are presented in this ch apter. The relationship between attention and executive function behaviors as deter mined by the correlation between subcomponents of attention (sustained, selective, s hifting/attentional control), and executive function behaviors are presented. The var iables used to assess this relationship included the subtest scores from the TEA-Ch, BRIEFParent (GEC), and BRIEF-Teacher (GEC). The dimensions of attention on the TEA-Ch are model -based and theory-driven. Specific subtests from the TEA-Ch battery were ascr ibed to different attentional factors determined by a structural equation model to provid e support for its validity with the three-factor model giving a close fit to the data ( Heaton et al., 2001). The Score! subtest was identified as a measure of Â“sustained attention ,Â” which describes attention that requires the active maintenance of a particular res ponse set under conditions of low environment support (e.g., when there are few trigg ers to the relevant behavior, when the task lacks interest or reward) (Manly et al., 2001) The Sky Search subtest was associated with Â“selective attention,Â” which is designed to as sess the childÂ’s ability to attend to target stimuli in the presence of distracters (Heat on et al., 2001). The Creature Counting
109 subtest was purported to measure Â“attentional contr ol/shifting attention,Â” and is associated with switching from one task or mental s et to another (Manly et al., 2001). Lastly, the Sky Search DT task was presented as a m easure of dual Â“sustained attention,Â” and Â“attentional control/shifting attention.Â” Accor ding to past research, performance decrements under dual task conditions tend to form sensitive measures of neurological impairment and thus the TEA-Ch combines two of its subtests for such purposes (Manly et al., 2001). Pearson Product-Moment Correlation coefficients (PP MCC) were calculated to measure the relationships between the dependent var iables and child performance on the TEA-Ch. The variables found to be significantly cor related with the GEC score from the BRIEF-Parent were the subtest scores from Sky Searc h (r=-.331, p=.05), Score! (r=-.289, p=.05), and Creature Counting (r=-.424, p<.01). Fur thermore, the variables that were found to be significantly correlated with the GEC s core from the BRIEF-Teacher were the subtest scores from Sky Search (r=-.302, p=.05) Score! (r=-.482, p<.01), and Creature Counting (r=-.537, p<.01) (Table 6). Teach er ratings of executive functions were most highly correlated with child performance on tasks assessing components of sustained attention and shifting/attentional contro l.
110 Table 6 Intercorrelations Between TEA-Ch Subcomponents and BRIEF-Parent and Teacher (GEC) Forms Variable 1 2 3 4 5 6 TEA-Ch Subcomponents 1 Sky Search .361* .387** .093 -.331* -.302* 2 Score .361* .37** .232 -.289* -.482** 3 Creature Counting .387* .37* .607** -.424** -.53 7** 4 Sky Search DT .093 .232 .607** .039 -.135 BRIEF(GEC) 5 BRIEF-Parent (GEC) -.331* -.289* -.424** .039 -. 572** 6 BRIEF-Teacher (GEC) -.302* -.482** -.537** -.135 .572** Note N=48 *Correlation is significant at the 0.05 level (2-ta iled) **Correlation is significant at the 0.01 level (2-t ailed) Data screening. Next, a multiple regression analysis was conducted to assess the relative contribution of each variable to the total GEC scor es from the BRIEF-Parent and BRIEF-Teacher while holding all other variables con stant. Assumptions of regression analysis were reviewed prior to running the analyse s. Regression is relatively robust to the assumption that the predictor variables are fix ed. The scales that were used to measure the predictor variables were determined to have adequate reliability levels as previously indicated. Furthermore, Variance Inflati on Factors (VIF), which indicates whether a predictor has a strong linear relationshi p with the other predictors, were examined in the assessment of multicollinearity and values were not noted to approach or exceed ten providing support that collinearity was valid for this model (Myers, 1990). In addition, examination of the correlation matrix did not indicate substantial correlations between predictors (r >.9) which is also indicative of violation of multicollinearity.
111 CookÂ’s distance ranged from .000 to .121, and .000 to .276 suggesting that outliers did not significantly influence the results of either d ata analysis. Finally, the Durbin-Watson statistic was generated which informs whether the a ssumption of independent errors is tenable. According to the acquired data, the genera ted Durbin-Watson statistics were 1.47 and 1.863, which falls in between the recommended g uidelines of 1 and 3 (Field, 2009). Results implied that all the variables included in the model were properly measured. In addition, each of the predictor variables and the a ssociated residuals were understood to be independent in the population and both of the es timates of regression coefficients and the significance tests were unbiased. The normality assumption was investigated by examining histograms and corresponding p-p plots of standardized residuals for linearity and homoscedacity (constant variance of residual te rms at each level of predictor variables) (Field, 2009). Each histogram and the ac companying p-p plot demonstrated that the normality assumption for each predictor va riable was not violated. Research Questions Research Question 1a All of the TEA-Ch subtest scores were included in the regression model in order to determine which subcom ponents of attention contributed substantially to the modelÂ’s ability to predict BRI EF-Parent GEC scores. Results indicated that TEA-Ch subtest scores accounted for 35% of the variance in the total GEC score of the BRIEF-Parent (R2=.346, R -adj. =.285, F (4, 43) =5.675, p<.001). Th is analysis revealed that Creature Counting and Sky Se arch DT contributed significantly to the variability of BRIEF-Parent GEC scores (Table 7 ). Thus, parent ratings of executive functions were most predicted by measures assessing attentional control/shifting attention and a dual task combining sustained attention and s elective attention.
112 Consequently, a forward stepwise multiple regressi on analysis was conducted using these two predictors to define the subsequent regression model. Creature Counting accounted for 18% of the variation in BRIEF-Parent GEC scores. When Sky Search DT scores were included this value was increased to 31 .8% indicating that the dual task of sustained and selective attention accounted for an additional 13% of the variation in parent ratings of executive function behaviors (Tab le 8). Table 7 Multiple Regression Examining the Relationship of E ach TEA-Ch Variable to BRIEFParent GEC Scores While Holding All Other Variables Constant Unstandardized Coefficients Standardized Coefficie nts Variable Regression Coefficient ( ) Standard Error Regression Coefficient ( ) t -value Sig. (Constant) 76.188 6.37 11.956 .000 Sky Search -.376 .617 -.086 -.609 .546 Score! -.431 .438 -.136 -.986 .33 Creature Counting -2.175 .615 -.614 -3.539 .001Â† Sky Search DT 1.457 .512 .451 2.846 .007Â† Note N=48 Â† indicates significance at the =.05 level Table 8 Forward Stepwise Multiple Regression Examining the Relationship of Creature Counting and Sky Search DT to BRIEF-Parent GEC Scores Outcome Variable Predictor Variable(s) R 2 Adjusted R 2 F -statistic, Probability BRIEF-Parent GEC Creature Counting R 2 = .18, Adj. R2= .162 F (1, 46) = 10.071, p = .003 Sky Search DT R 2 = 0.318, Adj. R2= .288 R2 change = .139 F (1,45) = 9.169, p = .004 Note N=48
113 Research Question 1b. In order to examine the ability of TEA-Ch subtest scores to predict teacher ratings of executive function be haviors all of the childrenÂ’s performance scores were included in the regression model. Performance on all subtests of attention accounted for 44% of the variance in the total teacher ratings of executive function behaviors (R2=.441, R -adj. =.389, F (4, 43) =8.487, p<.001). Th e results of the multiple regression analysis revealed that three in dependent variables (Score!, Creature Counting, and Sky Search DT) contributed significan tly to the variability of BRIEFTeacher GEC scores (Table 9). Overall, tasks of sus tained attention, attentional control/shifting attention, and a dual task combini ng these tasks of attention accounted for the greatest degree of predictability in teache r ratings. A forward stepwise multiple regression analysis wa s conducted using these three predictors to define the subsequent regression mode l. Creature Counting, which assesses attentional control/shifting attention, accounted f or 28.8% of the variation in BRIEFTeacher GEC scores. Furthermore, when the Score! su btest scores associated with sustained attention were included, this value incre ased to 38.1% accounting for an additional 9.3% of the variability in teacher rati ngs of executive function behaviors. Lastly, when the Sky Search DT subtest scores, repr esenting a measure of dual attention, were included results accounted for an additional 5 .9% of the variability in teacher ratings (Table 10).
114 Table 9 Multiple Regression Examining the Relationship of E ach TEA-Ch Variable to BRIEFTeacher GEC Scores While Holding All Other Variable s Constant Unstandardized Coefficients Standardized Coefficie nts Variable Regression Coefficient () Standard Error Regression Coefficient () t -value Sig. (Constant) 81.672 5.782 14.125 .000 Sky Search .118 .56 .028 .211 .834 Score! -1.057 .397 -.338 -2.662 .009Â† Creature Counting -2.128 .558 -6.12 -3.816 .000Â† Sky Search DT .991 .464 .313 2.134 .0363Â† Note N=48 Â† indicates significance at the =.05 level Table 10 Forward Stepwise Multiple Regression Examining the Relationship of Creature Counting and Score! to BRIEF-Teacher GEC Scores Outcome Variable Predictor Variable(s) R2, Adjusted R2 F -statistic, Probability BRIEF-Teacher GEC Creature Counting R 2 = .288, Adj. R2= .273 F (1, 46) = 18.606, p <.001 Score! R 2 = 0.381, Adj. R2= .354 R2 change = .093 F (1,45) = 6.788, p =.012 Sky Search DT R 2 = 0.441, Adj. R2= .402 R2 change = .059 F (1,44) = 4.663, p =.036 Note N=48
115 Research Question 2a. The relationship between attention and social/adapt ive functioning was examined by computing a correlation matrix between subcomponent(s) of attention (sustained, selective, shifting/attent ional control) and social/adaptive functioning. The variables used to explore this rel ationship included four subtest scores from the TEA-Ch and ABAS II-Parent/Caregiver (GAC). Pearson Product-Moment Correlation coefficients (PPMCC) were calculated to measure the relationships between the dependent variables and child performance on th e TEA-Ch. The variables found to be significantly correlated with the GAC score from th e ABAS-II-Parent/Caregiver were the subtest scores from Sky Search (r=.356, p=.05), and Creature Counting (r=.396, p<.01). Parent ratings of social/adaptive functioning were most highly associated with tasks representing selective attention and attentional co ntrol/shifting attention. Initially, all of the TEA-Ch subtest scores were i ncluded in the regression model. However, only scores obtained from the Creature Cou nting subtest significantly contributed to the variability of parent ratings of social/adaptive functioning behaviors. This finding notes a high degree of predictability from child performance on tasks assessing attentional control/shifting attention. A linear regression analysis revealed that Creature Counting was a significant predictor of pa rent social/adaptive functioning scores ( = .387, p = .048), accounting for 16% of the varia nce in the total ABAS-IIParent/Caregiver GAC scores (R2=.156, F (4, 43) =2.925, p<.001) (Table 13). Theref ore, child performance on a measure of attentional contr ol/shifting attention was identified to have the greatest ability to predict measures of pa rent ratings of social/adaptive functioning and skills related to daily living.
116 Table 11 Intercorrelations Between TEA-Ch Subcomponents and ABAS-II-Parent/Caregiver and Teacher (GAC) Versions Variable 1 2 3 4 5 6 TEA-Ch Subcomponents 1 Sky Search .361* .387** .093 .356* .204 2 Score .361* .77** .232 .188 .631** 3 Creature Counting .397** .37** .607** .396** .52 2** 4 Sky Search DT .093 .232 .396** .132 .189 ABAS-II (GAC) 5 ABAS-II-Parent/Caregiver .356* .188 .396** .132 .189 6 ABAS-II-Teacher .204 .631** .522** .278 .189 Note N=48 *Correlation is significant at the 0.05 level (2-ta iled) **Correlation is significant at the 0.01 level (2-t ailed) Table 12 Multiple Regression Examining the Relationship of E ach TEA-Ch Variable to ABAS-II Parent/Caregiver GAC Scores While Holding All Other Variables Constant Unstandardized Coefficients Standardized Coefficie nts Variable Regression Coefficient () Standard Error Regression Coefficient () t -value Sig. (Constant) 75.028 7.7 9.744 .000 Sky Search 1.057 7.46 .219 1.415 .164 Score! -2.24E-02 .529 -.006 -.042 .966 Creature Counting 1.512 .743 .387 2.035 .048Â† Sky Search DT -.435 .619 -.122 -.703 .486 Note N=48 Â† indicates significance at the =.05 level
117 Table 13 Forward Stepwise Multiple Regression Examining the Relationship of Creature Counting to ABAS-II-Parent/Caregiver GAC Scores Outcome Variable Predictor Variable(s) R 2 Adjusted R 2 F -statistic, Probability ABAS-II-Teacher GAC Creature Counting R 2 = .156 Adj. R2= .138 F (4, 43) = 2.925, p <.001 Note N=48 Research Question 2b. The relationship between attention and social/adapt ive functioning was examined by computing a correlation matrix between subcomponent(s) of attention (sustained, selective, shifting/attent ional control) and social/adaptive functioning. The variables used to explore this rel ationship included four subtest scores from the TEA-Ch and ABAS II-Teacher (GAC). Pearson Product-Moment Correlation coefficients (PPMCC) were calculated to measure the relationships between the dependent variables and child performance on the TE A-Ch. The variables found to be significantly correlated with the GAC score from th e ABAS-II-Teacher were the subtest scores from Score! (r=.631, p<.01), and Creature Co unting (r=.522, p<.01). Teacher ratings of social/adaptive functioning were most hi ghly associated with tasks representing sustained attention and attentional control/shiftin g attention (Table 11). Once again, a multiple regression analysis was con ducted to assess the relative contribution of each variable to the total GAC scor e from the ABAS-II-Teacher and the four subtest scores from the TEA-Ch while holding a ll other variables constant. Assumptions of regression analysis were again revie wed prior to running the analyses. VIF values were examined and collinearity was deter mined to be tenable (Myers, 1990). In addition, the correlation matrix did not indicat e substantial values between predictors (r >.9). CookÂ’s distance values ranged from .000 to .28, and from .000 to .211 suggesting
118 that outliers did not significantly influence the r esults of either data analysis. Finally, the Durbin-Watson statistics generated values of 2.299 and 1.07 which falls in between the recommended guidelines (Field, 2009) indicating tha t this assumption was met for both analyses. Histograms and normal probability plots w ere also examined for linearity and homoscedacity. Data indicated favorable conditions allowing the analyses to proceed without violation of assumptions. All of the TEA-Ch subtest scores were simultaneous ly included in the regression model and accounted for 52% of the variance in the total GAC score of the ABAS-IITeacher (R2=.515, R -adj. =.47, F (4, 43) =11.424, p<.001). Th e results of the multiple regression revealed that the Score!, (r=.631, p<.01 ) and Creature Counting (r=.522, p<.01) subtests significantly contributed to the va riability of teacher ratings of social/adaptive functioning behaviors (Table 14). T hese components of attention are associated with sustained attention and attentional control/shifting attention. The stepwise forward multiple regression analysis using these tw o predictors indicated that Creature Counting accounted for 28.8% of the variation of te acher ratings of social/adaptive functioning indicating the importance of attentiona l control/shifting attention in accounting for the variability of teacher ratings. When Score! subtests scores were included, this value increased to 35.4% thus noting components of sustained attention to account for an additional 9.3% of the variation in teacher ratings of social/adaptive functioning (Table 15). However, it should be noted that in the review of histograms for the residuals as well as normal probability plots d ata indicated non normal distribution of the residual values. Although most tests (specifica lly the F-test) are quite robust with
119 regard to violations of this assumption, conclusion s should be interpreted with caution and limitations to generalization of data are appli cable. Table 14 Multiple Regression Examining the Relationship of E ach TEA-Ch Variable to ABAS-II Teacher scores GAC Scores While Holding All Other Variables Consta nt Unstandardized Coefficients Standardized Coefficie nts Variable Regression Coefficient () Standard Error Regression Coefficient () t -value Sig. (Constant) 83.384 5.391 15.468 .000 Sky Search -.666 .533 -.155 -1.276 .209 Score! 1.71 .37 .547 4.62 .000Â† Creature Counting 1.536 .52 .441 2.954 .005Â† Sky Search DT -.323 .433 -.102 -.747 .459 Note N=48 Â† indicates significance at the =.05 level Table 15 Forward Stepwise Multiple Regression Examining the Relationship of Score! and Creature Counting to ABAS-II-Teacher GAC Scores Outcome Variable Predictor Variable(s) R2, Adjusted R2 F -statistic, Probability ABAS-II-Teacher GAC Creature Counting R 2 = .288, Adj. R2= .273 F (1, 46) = 18.606, p <.001 Score! R 2 = 0.381, Adj. R2= .354 R2 change = .093 F (1,45) = 6.788, p = .012 Note N=48
120 Research Question 3 The relationship between executive function behav iors, and social/adaptive functioning was also explored. The variables used to examine the strength of this relationship included the index scores from the BRIEF-Parent, BRIEF-Teacher (MI, BRI), ABAS II-Parent/Caregiver, and ABAS-II Te acher (GAC) scores. Pearson Product-Moment Correlation coefficients (PPMCC) wer e calculated to measure the relationships between the dependent variables and i ndexes of the BRIEF. Both index scores from the BRIEF-Parent forms were found to be significantly correlated with the GAC scores from the ABAS-II-Parent/Caregiver. Resul ts indicated correlations with the MI (r=-.374, p<.01), and BRI (r=-.286, p=.05) (Tabl e 16) scores. Furthermore, all variables from the BRIEF-Teacher were also found to be significantly correlated with the GAC scores from the ABAS-II-Teacher. Results from t his correlational analysis also indicated MI (r=-.527, p<.01), and BRI (r=-.525, p< .01) parent rating scores to correlate highly with ABAS-II Teacher ratings. Although ABASParent/Caregiver GAC scores were not significantly correlated with either of th e teacher ratings from the BRIEF, the ABAS-II Teacher scores were significantly correlate d at the p<.01 level for the BRIEFMI Teacher (r=-.673), and BRIEF-BRI Teacher (r=-.51 3) (Table 16).
121 Table 16 Intercorrelations Between ABAS-II-Parent/Caregiver and Teacher (GAC) Versions and BRIEF-MI and BRI Parent/ Teacher Forms BRIEF-MI Parent BRIEF-BRI Parent BRIEF-MI Teacher BRIEF-BRI Teacher ABAS-II (GAC) ABAS-II-Parent/Caregiver -.374** -.286* -.168 -.18 1 ABAS-II-Teacher -.527** -.525** -.673** -.513** Note N=48 *Correlation is significant at the 0.05 level (2-ta iled) **Correlation is significant at the 0.01 level (2-t ailed) Research Questions 4. The effect of gender on the performance of assessm ents purported to measure components of attention was ex plored. The statistical significance of main effect and interaction for gender and each of the dependent variables were examined by means of a multivariate analysis of var iance (MANOVA) to compare the subtests scores from the TEA-Ch as achieved by male s to the performance of females. Data Analysis Multivariate analysis of variance (MANOVA) was sele cted for data analysis to determine if the groups differed significantly on t he set of dependent variables. MANOVA incorporates information about several outco me measures and therefore informs of whether groups of participants can be di stinguished by a combination of scores earned on several dependent measures (Field, 2009). Furthermore, an advantage of MANOVA analysis is that it allows the researcher to gain power, which may detect differences that univariate analyses alone may not detect. The null hypothesis was posed to communicate no significant differences existing between performances according to gender. The significance level was preset to p < .05.
122 Descriptive statistics. The dependent variables that were examined include child performance on the subtests of the TEA-Ch. Differen ces across these subtests were examined across gender. Means and standard deviatio ns of each dependent variable by group and of the whole sample ( N =48) are presented (Table 17). The results of the MANOVA were significant at the =.05 level ( =.728, F (4, 43) =4.01, p<.05), indicating a significant effect of gender on performance of subtests on the TEA-Ch. Furthermore, based upon results from th e BoxÂ’s M Test of Equality of Covariance Matrices, the equality of variances assu mption was considered to be tenable (BoxÂ’s M=15.503, F(10,9945.813)=1.403, p=.172), not ing that the F value calculated by the MANOVA is considered robust thus not likely con tributing to Type I error. Furthermore, LeveneÂ’s Test of Equality of Error Var iances was not significant for any of the dependent variables thus the assumption of equa lity of covariance matrices was met. Table 17 Means and Standard Deviations for Dependent Variabl es by Gender Dependent Variable Female (n =25 ) Male (n =23 ) Total (n=48) TEA-Ch Sky Search 9.56(2.53) 11.17(2.62) 10.33(2.68) Score! 8.56(4.09) 10.26(3) 9.37(3.768) Creature Counting 10.28(3.52) 9.35(3.05) 9.83(3.3) Sky Search DT 6.52(3.34) 6.83(3.97) 6.67(3.62) Note N=48 *Note: Standard Deviations in parentheses ANOVA Results Given the significant results of the MANOVA, post hoc analyses were conducted in the form of separate univariate a nalysis of covariance (ANOVA) with the outcome variables. Post hoc t tests were conducted to assess for indiv idual differences
123 between groups on each measure while accounting for the difference in-group variances. A modified Bonferroni was utilized while examining the significance of the t scores. The significance value for these analyses was set to p < .025 to maintain a conservative estimate of statistical significance with two group s. However, results noted that none of the ANOVA analyses were found to be statistically s ignificant at the adjusted level (Table 18). Overall, results informed that the null hypothesis could not be rejected and significant differences between scores based on gen der could not be obtained on the TEA-Ch subtests. Table 18 Results of ANOVA for Each Dependent Variable Source Sum of Squares Mean Square F -value p Sky Search Model 31.202 31.202 4.699 0.035 Error 305.464 6.641 Corrected Total 635.25 Score! Model 34.655 34.655 2.654 0.11 Error 600.595 13.056 Corrected Total 635.25 Creature Counting Model 10.409 10.409 .953 0.334 Error 502.257 10.919 Corrected Total 512.667 Sky Search DT Model 1.122 1.122 .084 0.773 Error 615.544 13.381 Corrected Total 616.667 Note N=48
124 Summary This chapter described the data analysis, screenin g, and results in the exploration of the ability of specific components of attention to predict executive function behaviors and social/adaptive functioning. In brief, results of multiple regression and correlational analyses revealed child performance on specific mea sures of attention predict executive function and social/adaptive functioning behaviors. As parent/caregiver and teacher ratings of executive function behaviors increased, child performance on measures of selective attention, sustained attention, and atten tional control/shifting were reported to improve. Results indicate that children with highly rated acquisition and implementation of executive function behaviors possess a greater n umber of skills related to shifting cognitive set, modulating their emotions and exhibi ting inhibitory control, and systematic problem solving. Furthermore, children were also rated to have highe r tendencies to initiate, plan, organize, and sustain future-oriented problem solvi ng in working memory, behaviors associated with the MI index of the BRIEF. Measures purported to assess attentional control/shifting attention (e.g., Creature Counting ), and simultaneous assessments of sustained attention and selective attention (e.g., Sky Search DT) were able to account for a significant amount of the variability of parent/c aregiver ratings of executive functions. Similarly, sustained attention and simultaneous sus tained and selective attention measures were also noted to predict higher teacher ratings of executive functions. Finally, a sole measure of sustained attention (e.g., Score! ) was identified as a critical predictor in teacher ratings.
125 Based on parent/caregiver ratings of social/adaptiv e functioning, children who were noted to perform a greater number of independe nt living skills earned increasingly adept scores on measures of selective attention and attentional control/shifting attention. Teacher ratings of social/adaptive functioning indi cated proficient abilities for children earning higher scores on measures of sustained atte ntion and attentional control/shifting attention. In the use of attention measures to pred ict social/adaptive functioning, measures of attentional control/shifting, sustained attention, and simultaneous sustained attention and selective attention were identified a s significant predictors of parent/caregiver ratings. Furthermore, parent/caregiver and teacher ratings o f executive function and social/adaptive functioning behaviors were signific antly related across measures. That is, informants agreed that higher levels of executive f unction behaviors were related to greater acquisition and performance of daily living skills and social/adaptive functioning abilities. Finally, gender differences did not diff erentiate between child performance on measures of attention.
126 Chapter 5 Discussion Overview of Study Objectives Attention is a commonly used term in education, psy chiatry, and psychology. It is often defined as an internal cognitive process by w hich one actively selects environmental information (i.e., sensation) or acti vely processes information from internal sources (i.e., visceral cues or other thou ght processes). In more general terms, attention can be defined as an ability to focus and maintain interest on a given task or idea, and that, which includes managing distraction s (DeGangi & Proges, 1990). According to some researchers, (DeGangi & Proges, 1 990; Manly et al., 2001) one model of attention identifies three subcategories includi ng selective attention, attentional control/shifting attention, and sustained attention The TEA-Ch was designed to assess these domains of attention, which are commonly appl ied by children in their daily activities. The purpose of this study was to investigate and to define better the relationship between attention and corresponding behaviors that represent executive functions and social/adaptive functioning in a normative sample o f school age children. This research was initiated in hopes of providing evidence to est ablish the TEA-Ch as an ecologically valid instrument for use with children. The impetus for this research was the dearth of evidence-based research supporting the ecological v alidity of pediatric
127 neuropsychological assessment tools (Gioia & Isquit h, 2004). Ecological validity was assessed by determining the degree to which the TEA -Ch subtests could predict important aspects of a childÂ’s everyday functioning by assessing the general strength of association to outcome domains (Gioia & Isquith, 20 04). Critical to ecological validity is the concept of veridicality, the degree to which a measure predicts a particular aspect of a childÂ’s everyday functioning (Burgess et al., 1998) Thus, social/adaptive behavior and behaviors of executive function were selected as im portant aspects of daily functioning. Data were collected and analyzed from a sample comp osed of 48 school age children ranging in age from 8-years, 0-months to 1 0-years, 11-months, a parent/caregiver, and his/her elementary school tea cher. Rapid changes in different components of attention are known to occur in child ren between ages 8 to 10 years (Rebox, 1997) which define the age range of this st udyÂ’s sample population. This chapter provides a description of the results, interpretati on, and the implications of these results as they pertain to the assessment of attention and executive functions in children. The discussion concludes with limitations of this study and recommendations for future research. Attentional Performance and Executive Function Beha viors (Research Questions 1a and 1b). The results of this study suggest that attentional control/shifting attention, or the ability to switch attentional focus is the most sig nificant predictor of executive function behavior ratings accounting for 18% and 28.8% of th e variability, respectively. These findings are supported by previous researchers who report that attentional control is not only an important component of executive functions; the ability to shift often serves as a
128 critical underlying ability for the emergence of Â“ higher order Â” skills such as cognitive flexibility, working memory, and self-monitoring sk ills (Anderson et al., 2001). For example, Blair (2002) reported that children who co nsistently exhibited negative emotionality were more likely to experience difficu lty in the application of these Â“ higherorder Â” cognitive processes due to their inability to shi ft attention away from a negative source, and practice planning and reflective proble m solving in social situations. That is, emotional control may also stem, in part, for a chi ldÂ’s ability to shift attention away from a negative feeling or thought, in order to retain a nd implement skills from a set of ethical codes or principles (Feifer & Rattan, 2007). In add ition, students with poor executive functioning skills have difficulties adapting their behavior to the constant changing of social circumstances, particularly where frustratio n and anger must be tempered for the pursuit of attaining a goal (Feifer & Rattan, 2007) Overall, studies consistently indicate that executive function skills (and specifically th e ability to switch attention in a flexible manner in order to adjust oneÂ’s behavior and respon se) are of critical importance for establishing and maintaining socially appropriate i nteractions within a classroom setting. These skills allow students to self-monitor emotion al impulses and to regulate motor related processes for successful adaptation to thei r learning environment (Feifer & Rattan, 2007). Furthermore, in a study describing the developmenta l sequence of attention and executive functions in a child population ranging i n age from 3 to 12 years, inhibitory functions followed by maturation of auditory and vi sual attention functions were noted to have developed by the age of 10 years. However, the development of fluency and shifting of attention continue into adolescence. This resear ch provides support for the acquisition
129 and development of attentional control/shifting att ention by teachers and parents in the selected age group. According to the research, the other components of attention including inhibition and sustained attention appear to serve as prerequisites for more complex forms of attention such as attentional cont rol/shifting attention (Klenberg, Korman, Lahti-Nuuttila, 2001). The variability of parent/caregiver and teacher ra tings of executive function behaviors was also accounted for by subtests assess ing sustained attention and a dual task combining sustained and selective attention. Previo us studies have noted that sustained attention is stable between the ages of 8 and 10 ye ars but increases significantly from age 11 through adulthood which may explain the lower de gree of accountability although parents/caregivers and teachers continued to endors e this component of attention to predict levels of executive functions (Rebok et al, 1997). The amount of variance accounted for by subtests of the TEA-Ch utilizing t he BRIEF as an outcome measure provides a degree of support for ecological validit y. Studies conducted using the BRIEF have provided compelling evidence with its use with various clinical populations (e.g., ADHD, mild and severe TBI, Autistic Spectrum Disord ers, and other medical and developmental conditions) (Gilotty et al., 2002; Gi oia et al., 2002). An approach to establishing the ecological of neuropsychological m easures is to relate test scores in a given cognitive domain to scores on measures of eve ryday tasks within that same domain across situations. Measures assessing components of executive functions would be related to measures of everyday executive functioni ng across situations (Chaytor & Schmitter-Edgecombe, 2003). Thus, the high degree o f association between the BRIEF
130 and TEA-Ch subtests indicate support for ecological validity using this type of approach in support of use of the TEA-Ch in examining specif ic components of attention. Attentional Performance and Social Adaptive Functio n Behaviors (Research Questions 2a and 2b) Attention is a multifaceted construct that influen ces the efficiency of many other cognitive processes including adaptive functioning. Attention problems are likely to pose as a challenge in performing multi-step or complex adaptive tasks as a result of difficulties with processing information efficientl y or inattention to details (e.g., social cues). Children with attention problems may become discouraged more easily and seek assistance from an adult. In general, neuropsycholo gical tests have been purported to account for a significant proportion of the varianc e in measures of adaptive functioning and particularly tasks that involve complex cogniti ve processing (Price et al., 2003). Specifically, research conducted on various clinica l and nonclinical child populations provide support for an association between attentio n deficits and adaptive functioning. Similar to previous research (Price et al., 2003; R ebok et al., 1997), this study demonstrated a measure of attentional control/shift ing to be most predictive of parent/caregiver and teacher ratings of social/adap tive functioning. In addition, a measure of sustained attention was also predictive of teach er ratings. The current study extends existing research by providing associations between adaptive behavior and specific components of attention with sustained attention an d attentional control/shifting attention providing the greatest degree of predictability acr oss teacher and parent ratings. Although previous studies have employed a number of cognitiv e domains in the assessment of neuropsychological functions and adaptive behavior, few if any have sampled a domain
131 using more than one measure or task (Price et al., 2003). Therefore, the use of the TEACh in the assessment of separable components of att ention and in relation to social/adaptive functioning provides further suppor t to the literature in highlighting the importance of comprehensive assessments of one spec ific domain of neuropsychological functioning. There are important implications of the association between skills of social/adaptive functioning and attention. Parent r eport measures have been shown to be advantageous as they offer a higher degree of ecolo gical validity than can be easily attained with direct assessment in testing conditio ns (Papazoglou et al., 2009). Additionally, scores on the Attention Problems subs cale of the Child Behavior Checklist (CBCL) have been shown to be significantly associat ed with deficits of attention on neuropsychological testing in child populations (Pa pazoglou et al., 2009). Thus, parent report measures of behavior might serve as an effec tive screen to identify children at risk for later delays in adaptive functioning so that mo re comprehensive evaluations may be conducted. Executive Functions and Social/Adaptive Behavior (R esearch Question 3) Significant correlations between the BRIEF MI and B RI indexes and composites of social/adaptive behavior were found across paren t/caregiver and teacher ratings. ABAS-II Parent/Caregiver scores were related to par ent ratings of metacognitive abilities as well as behavioral regulation but were not highl y associated with teacher ratings of these same measures. Many of the individual items f rom the parent ABAS-II are contextually specific to the home setting (i.e., co oks his or her own food, uses a washing machine to wash clothes) and are behaviors that are characteristically far removed from
132 the school setting. Thus, there is little overlap a cross many of the behaviors assessed in the parent ABAS-II and those that a teacher may typ ically observe at school. Furthermore, parents/caregivers are likely to obser ve children engaging in many social/adaptive functioning behaviors that exceed e xpectations and behaviors exhibited in the school environment. Teacher and parent ratings of social/adaptive funct ioning were significantly correlated with the BRI and MI executive function i ndexes. The strongest correlations were between the MI indexes from both teacher and p arent/caregiver ratings and the social/adaptive composite scores (GAC) as completed by the teachers. As neuropsychological measurement of executive functio ning are providing information predictive of daily functioning these data may lead to more effective interventions and recommendations because clinicians will be better a dept to predict the types of difficulties a child may present considering his or her own unique cluster of cognitive performance. Early neuropsychological assessment ma y lead to the detection and thereby prevention of emotional and behavioral consequences of executive function deficits. Suggested findings from this study may yield inform ation that is critical in promoting specific classroom wide strategies in support of st rengthening these types of skills rather than assuming natural acquisition. In practice, neu ropsychologists are asked to identify functional strengths and weaknesses to translate fi ndings into implications and predictions for the child in his or her everyday mi lieu. Furthermore, pediatric neuropsychologists are often requested to assess a childÂ’s cognitive profile to inform referral questions rega rding academic placement, necessary interventions and accommodations, Individualized Ed ucation Plan (IEP) goals,
133 implications for school and community functioning, and future behavioral and emotional developments that may be expected in the course of a childÂ’s development (Gioia & Isquith, 2004). Results from the present study impl y that both indexes and scales of the BRIEF provide important data regarding the interpla y between executive functions and social/adaptive functioning behaviors thus providin g support from the predictive ability of executive functions as they relate to the everyd ay environment. Gender Effects within TEA-Ch Measures (Research Que stion 4a and 4b) Overall, no significant differences between the pe rformance of male and female participants were identified on any task of sustain ed, selective, attentional control/shifting or dual task of attention. These results support pr evious research conducted using the TEA-Ch (Chan et al., 2008; Heaton et al., 2001). St udies of attentional capacities have produced inconsistent data although the similaritie s between males and females have tended to be more notable than the differences part icularly in the younger and school age populations (Korkman, 2001). The few studies that h ave explored the component of gender are limited merely because many report resul ts based on only one measure of attention. Although many studies have found results similar to the findings of the current study indicating a lack of gender differences, Pasc ualvaca et al. (1997) noted that both gender and intelligence had an impact on performanc e on various tasks of attention. Their findings indicated higher level of performance by f emale participants on the Continuous Performance Test, Digit Cancellation task, and the Coding subtest of the WISC-R. Conclusions of this study noted that females were f ound to outperform males on tasks requiring focus of attention on a particular stimul us, ignoring irrelevant stimuli, and
134 making a rapid response. However, differences were not found in participantsÂ’ capacity to shift attention as evidenced by performance on t he WCST. These results should also consider the limitations of the measures used in as sessing Â‘pureÂ’ attention as most tasks of attention and executive functions inadvertently measure more than one aspect of behavior and cognition (Manly et al., 2001). The TE A-Ch is purported to present tasks that have been designed for specifically minimizing demands on memory, reasoning, task comprehension, motor speed, verbal ability, and per ceptual acuity while maintaining the demands on the targeted attentional system (Manly e t al., 2001). In addition, females are thought to mature earlier than males (Tanner, 1962) and variations in physical maturation have been associa ted with changes in behavior and cognitive performance. Klenberg and colleagues (200 1) found significant effects of gender across all subtests of attention with female s outperforming males from ages 8, 9, and 12 years of age. Gender differences in activity level and impulsivity also appear to be mediated by maturation (Pascualvaca et al., 1997). Gale and Lynn (1972) found that females made fewer errors on a vigilance task at 7, 8, and 12 years of age but did not differ from males between 9 and 11 years of age. Th e majority of participants that were included in the current study fall within this age range thus supporting a lack of disparity in performance on tasks related to vigilance includ ing shifting attention and selective attention. Furthermore, the ages at which females tend to outp erform males coincide with Â“spurtsÂ” in brain development (Epstein, 1974; Hudsp eth & Pribram, 1992) with females tending to progress faster along the developmental pathway. Other gender differences in attentional performance have reported to reflect di fferences in brain structure and
135 organization. According to this formulation, differ ent attentional functions are supported by distinct cerebral regions. Research addressing t he differences between males and females associated with cognitive abilities indicat e that males have typically outperformed females on tasks of mathematical reaso ning and complex visual-spatial skills whereas females consistently excelled on tas ks of verbal fluency, manual dexterity, and visual scanning (Gouchie & Kimura, 1991). Howev er, some of the brain regions involved in the support of attentional functions ar e not fully myelinated until adolescence, suggesting that gender differences bec ome more pronounced at puberty. This evidence supports current findings, inasmuch a s participant ages from this sample fall largely in the prepubertal stages of developme nt (Hudspeth & Pribram, 1992; Pascualvaca et al., 1997). Implications Results from the current study highlight the impor tance of assessing executive function behaviors and separable components of atte ntion. In regards to application of findings in the schools, interventions that enhance attention have been identified in the literature, and implementing them has led to some s uccess in improving specific skills. Developmental psychologists are keenly aware of the importance of adapting learning environments for children with varying cognitive pr ofiles. A fundamental premise of most neuropsychological assessment and intervention is that children present with specific strengths and weaknesses in their profiles of learning and social/adaptive functioning. In consideration of individual perform ance, appropriate educational programs should acknowledge and support these profi les (Waber et al., 2006). The main objective of a school psychologist working with a c hild and his/her family is to identify
136 the problem, and then develop and implement interve ntions with the highest likelihood of success. The results of this study may assist in de veloping interventions by identifying specific target variables. In some cases a detailed clinical interview or structured questionnaire may provide sufficient information to adequately answer the referral question. However, data derived from behavioral rat ing scales and questionnaire data only pertains to the informantÂ’s knowledge of the c hildÂ’s ability to cope with existing cognitive demands. Furthermore, there are many noncognitive reasons that a child may have difficulty functioning in everyday situations including psychiatric or medical illness. Neuropsychological assessments such as the TEA-Ch c an assist with uncovering whether a childÂ’s functional difficulties are a result of c ognitive deficits (Chaytor & SchmitterEdgecombe, 2003). Evaluating the ecological validity of neuropsychol ogical assessments has become an increasingly important research topic of researc h over the past decade (Chaytor & Schmitter-Edgecombe, 2003; Rabin, Burton & Barr, 20 07). Ecological validity has become an especially important focus in neuropsycho logical assessment with particular relevance for executive functions that coordinate a n individualÂ’s cognitive and behavioral capacities with real world demand situations (Gioia & Isquith, 2004). The advent of brain imaging techniques has also shifted the role of neu ropsychological testing data from diagnosis of brain pathology and lesion localizatio n to the assessment of functional capacities at home, work, school thereby elevating the importance and emphasis on generating ecologically valid measures of neuropsyc hological constructs (Chaytor & Schmitter-Edgecombe, 2003; Rabin, Burton & Barr, 20 07).
137 However, the neuropsychological tests that are most commonly utilized have not changed concurrently with the referral questions an d thus the same tests that were previously developed to answer diagnostic questions are now utilized to answer questions regarding real world functioning with very little e mpirical evidence to support this practice (Chaytor & Schmitter-Edgecombe, 2003). Res earch indicates that the everyday manifestation of executive functioning impairment m ay differ in important ways from executive functioning deficits captured by neuropsy chological tests in the laboratory (Chaytor & Schmitter-Edgecombe, 2007). Because the recommendations that clinicians generate are to address everyday functioning and ca n have far-reaching consequences for patientsÂ’ lives, it is important to demonstrate tha t neuropsychological measures have ecological validity. In their recent review, Chaytor and Schmitter-Edgec ombe (2003) defined ecological validity as the degree to which task per formance corresponds to real world performance and argued that ecological validity doe s not necessarily describe a task; rather it describes the inferences that are drawn f rom task performance. Similarly, Burgess et al., (2006) defined ecological validity as a measure of the Â“representativenessÂ” of the task or the correspondence between the task and real-life situations and the Â“generalizabilityÂ” of the task or the degree to whi ch task performance predicts problems in real-life settings. Franzen and Wilhem (1996) re fer to the Â“verisimilitudeÂ” of tasks, or their resemblance to demands in the everyday enviro nment as measured by the degree to which the cognitive demands of a task theoretically resembles the cognitive demands in everyday functioning. The verisimilitude approach t o increasing the ecological validity of neuropsychological assessment has led to the develo pment of several standardized
138 clinical tests including the TEA-Ch (Chaytor & Schm itter-Edgecombe, 2003). This test attempts to simulate everyday tasks that require sp ecific components of attention including searching a map or a telephone directory, listening to lists of spelling words, organizing several activities, following rules, and planning problem solutions. Veridicality is another approach used to assess the degree of ecological validity of neuropsychological assessment measures. This term r efers to the degree to which existing tests are related to measures of everyday functioni ng (Franzen & Wilhem, 1996). Typically, this type of research involves the use o f statistical techniques to relate to performance on traditional neuropsychological tests to measures of real world functioning including employment status, questionna ires, or clinician ratings. More recent approaches to improving the ecological valid ity in the assessment of executive functions has been to examine the veridicality of s tandard executive function tests and to favor those measures which show a positive relation ship with important everyday outcome variables (Chaytor & Schmitter-Edgecombe, 2 003). A measureÂ’s veridicality is also influenced by the everyday outcome variable selected. Although in the past, the majority of stu dies examining veridicality have used general measures including, job performance or adap tive behavior (Gilotty et al., 2002), while others have incorporated measures of behavior more specifically related to executive control (Burgess et al., 2006). The prese nt study incorporated both a measure of adaptive behavior as well as a measure specifica lly related to executive function in order to provide further support for its utility an d application to real world settings. Results from this study indicate significant associ ations of the TEA-Ch with specific measures of executive functions as well as adaptive functioning thus providing support
139 for ecological validity. Neuropsychologists are oft en called to assist in predicting the impact of behavioral and cognitive deficits on func tioning across environments(Rabin, Burton & Barr, 2007). This study provides prelimina ry data in support of its use in identifying strengths and weaknesses in attentional profiles and potential for use in predicting specific skills of attention in everyday settings. Early Intervention and Prevention Findings from this study also have implications for early identification and prevention techniques for children at risk for diff iculties with executive function and attention. The concept and definitions of executive functions and their association with various disorders are important knowledge areas for individuals working in education, health, and mental health fields. It is particularl y important for providers to have an understanding of the basic issues related to assess ment and remediation of executive function deficits and areas of weakness (Calhoun, 2 006). Delays in executive functions appear to present as symptoms of many disorders (e. g., Autistic spectrum disorders, ADHD, conduct disorder, phenylketonuria, TouretteÂ’s syndrome, brain injury) (Calhoun, 2006). Furthermore, Gioia and colleagues (2001) dis cuss the association between deficits of executive functions and language disabilities. The integration of school psychology and neuropsych ology is particularly relevant to the early childhood population. During this critical period of development, timely identification of neurologically based risk indicators and special needs, followed by the reliable implementation of evidence-based in terventions, can ameliorate learning and behavioral difficulties that may otherwise comp romise a child's successful attainment of critical skills. For example, while children wit h early problems with decoding are
140 readily identified for reading interventions, stude nts with deficits in executive functions may not be considered at risk for difficulties with academics or behavior until their deficits become frankly apparent in the later eleme ntary grades or beyond. Early monitoring and training in the acquisition and impl ementation of executive function behaviors and components of attention may prevent s tudents in the normative population from experiencing more entrenched academic and beha vioral difficulties. For example, rudimentary forms of working memory and inhibitory control are present relatively early in life and show a rapid development throughout pre school and early school age years (Anderson et al., 2001; Korkman, 2001; Rebok et al. 1997). A recent study conducted by Thorell and colleagues (2009) explored the possibility of implementing interventions intended to improve abilities of working memory and inhibition. This study reported that bri ef visuo-spatial working memory training had significant effects on both previously trained and non-trained working memory tasks in both the verbal and spatial domains However, interventions that targeted inhibition only affected performance on pr eviously trained tasks with no evidence of generalization. Such findings suggest t hat attentional functions appear to differ in terms of how amenable they are to trainin g. Similarly, Barkley (1996) also discusses a techniqu e that attempts to explicitly teach the executive functions related to delayed re sponding (e.g., inhibition, planning, etc.). This method involves delaying a response to a situation to increase the time allotted in objective goal setting, systematic screening for appropriate responses, and response selection and enactment. Delayed responding is a sk ill that must be overtly taught through discrete instruction, modeling and reinforc ed in natural settings. It is consistent
141 with many interventions that focus on decreasing im pulsive actions by reinforcing a brief Â“think timeÂ” before proceeding with a response. In regards to the application of findings to the pr actice of educators and school psychologists, it has been well established in the literature that difficulties with attention and behavioral control represent the most common re ason for school referrals (Angello et al., 2003; Koonce, 2007). Therefore, it is importan t for school psychologists to acquire the knowledge and skills that are necessary to cond uct a comprehensive assessment of attention that should not be limited to behavioral rating scales, observational data, and interviews (Angello et al., 2003; Koonce, 2007). Th e utility of rating scales in guiding the treatment development process are limited in provid ing a means for assessment information to be linked to specific intervention s trategies. Angello and colleagues (2003) reported that although the most commonly used ratin g scales in the assessment of attention symptoms offer a brief and convenient met hod for obtaining information about symptomatology, only a few provided sufficient evid ence to justify their use for inclusion in a school based assessment (Angello et al., 2003) Findings from this study provide additional support for potential use and benefit of a performance-based assessment specific to attention as well as executive function behaviors based on growing evidence in support of the ecological validity of these rece ntly developed measures. Limitations The degree to which the conclusions of a study can be generalized to individuals outside of the study is dependent upon the limitati ons inherent in the study. Overall, the number of participants recruited and the instrument s used are limitations of this study. Results should be interpreted with these two limita tions in mind. The first limitation of
142 the current study is that analyses were conducted o n a relatively small sample (i.e., <100 participants). Ideally, the current model should be tested on a larger sample that includes racial/ethnic, and socioeconomic diversity to deter mine the reliability of results and to increase the power to allow analyses to be conducte d by age bands. In addition, potential for bias was introduced into the study through the sampling methods employed. Participants were voluntary, off ered a gift card, and representative of a limited geographical area. This convenience sampl e does not ensure that the sample is representative of the population at large. Therefor e, the results obtained, though characteristic of the sample, may not generalize to the larger group from which the sample was accessed. Finally, additional demographic data were not colle cted for the participants that may have better explained the results from the data analyses. For example, cognitive abilities, reading level, parent education, nor tea cher experience were assessed, even though each could significant implications for inte rpretation of items included on rating scales as well as child performance on tasks of att ention. Recent findings suggest that neuropsychological functions, especially executive functions are by no means Â“hardwired,Â” particularly for children reared in communi ties of low socio-economic status. For these children, environmental factors may play a fa r greater role in outcome than they are for children from backgrounds of greater social and economic advantage (Waber et al., 2006). Similarly, although information in regards t o age was collected the impact of this factor was not investigated due to limited sample s ize. Therefore, these additional factors and the impact on executive functions and attention should be further explored on larger and more diverse child populations.
143 Last, one problem related to the use of a normativ e population to study development is related to the inherent limitations of psychological assessments. Floor and ceiling effects may weaken the conclusiveness of th e findings. Furthermore, it may not be possible to establish with certainty the degree to which age-related changes in the performance of specific tests indicate a true devel opmental trajectory or whether it is a reflection of the testÂ’s ability to calculate the n ormal variance across age bands (Korkman, 2001). The second limitation is in regards to the tools th at were used for this study. The BRIEF and ABAS-II are vulnerable to several limitat ions ubiquitous to behavior rating inventories (Pelham, Fabiano & Massetti, 2005). Suc h limitations include susceptibility to inter-rater variance as well as parent/caregiver and teacher bias (Meyer et al., 2001). For example, one specific study indicated that mate rnal depression influenced ratings by endorsing a greater number of symptoms associated w ith ADHD in a child who otherwise failed to meet diagnostic criteria (Chi & Hinshaw, 2002). Further, rater bias is a pervasive question when utilizing self-report mea sures. Often, caregivers completing questionnaires sometimes provide ratings that make their children appear more socially acceptable. Future studies would also benefit from measuring executive function and attention skills of interest using a variety of met hods including behavioral testing, additional parent and teacher report, and classroom observation to create more robust estimates of these skills. In addition, the psychometric properties of the TEA -Ch may limit the reliability of the findings from this study. Although it has be en normed for use with children recruited from an Australian population, the ongoin g normative study for use in the
144 United States has not yet been completed. Furthermo re, in spite of the support for the validity of measures, it is not possible to validat e fully measures of attention and executive functions, because the components of thes e functions are still unknown. In addition, the literature also suggests that the tes tÂ’s validity may also be dependent on the age of the participants, which is a factor that was not explored in the current study. At a certain age, one type of performance may separate s trong and weak performance more sensitively as compared to other ages (Klenberg, Ko rkman, Lahti-Nuuttila, 2001). Although some continuity appears to exist in cognit ive capacity from one age to another, particularly as cognitive expectations inc rease, longitudinal tests across all ages for more specific types of performance are not yet well established. It is therefore important to evaluate neurocognitive development vi a comprehensive methods by utilizing a wide set of tasks. Accordingly, when us ing data acquired from several age groups, the factors obtained in this study may also reflect the influence of age. Most likely, correlations among attention and executive functions task performance change during development and the associations between fac tors are likely to be different for various age groups. This significantly impacts the generalizability and interpretability of the data collected for this study. Further research is required before the TEA-Ch can be fully utilized as a reliable and valid measure for the assessment of attentional performance in typically developing children and sp ecific clinical cases. Future Directions for Research Several questions need to be explored in future re search. Evidence of continual change and maturation of executive function skills from early childhood into early and late adolescence emphasizes the need for neuropsych ologists and educators to gain an
145 understanding of the normal development of attentio nal capacity (Anderson et al., 2001). Significant improvements across ages have been foun d including the ability to focus attention, to execute a response, to shift attentio nal focus, and to encode information in memory. Researchers are called upon to examine low scores on measures of attention as antecedents of a broader set of maladaptive develop mental consequences aside from adaptive behavior as identified in this study to in clude psychiatric symptomotology, drug use, school dropout, suspension, etc. (Rebok et al. 1997). In addition, although current results suggest some evidence of gender differences that differ in younger as compared to adolescent populations further exploration is requi red to replicate and delineate such findings. The study of attentional training is a relatively n ew area of research and future studies should identify which attentional functions can be trained. Furthermore, attentional training should be examined across the developmental age span in order to determine essential time periods in which this type of training should be considered and fostered. This exploration would extend current kno wledge in regards to the effects of cognitive training and determine how it can be gene ralized to other cognitive, behavioral, attentional, and executive function behaviors (Thor ell et al., 2009). Cognitive functions appear to differ in terms of how easily they can be trained. Thus, differences might be explained by modifications in the anatomical basis and time course of the underlying psychological and neural processes of working memor y and inhibition. In summary, the TEA-Ch has demonstrated its utility as a test of attention that may be used across a wide range of clinical populat ions (Baron, 2001). Questions about the construct validity, positive predictive power, negative predictive power, diagnostic
146 sensitivity, specificity, and neuroanatomical corre lation need to be further investigated. In this study, the four-subtest screener was utilized which also spurs additional research to assess the difference in accuracy with which it can differentiate between various clinical populations in a manner that is similar to or disti nct from results obtained on the full battery. In addition, it remains to be seen whether the TEA-Ch dual tasks of attention correlate highly with other dual task performance t ests. Additionally, as interest in developing ecologicall y valid measures of executive functions grows future researchers may consider adm inistering the TEA-Ch under normal testing conditions as well as administering the alt ernate form under conditions with distractions (e.g., music, classroom noise, etc.). For example, though listening to and repeating a list of words bears theoretical similar ities to learning that occurs in school, the controlled rate of presentation, isolated and steri le assessment setting, guided practice over trials, and cues to organize the information m ay not approximate classroom demands. The more likely environment where listenin g and remembering is required is one in which there are distractions, the presentati on rate is varied, there is limited opportunity for rehearsal, and there may be additio nal demands including note taking. While these modifications would provide only qualit ative data and may not be reliable, considering such variables could increase the clini cianÂ’s ability to provide support for issues related to ecological validity (Gioia & Isqu ith, 2004). Although much remains to be investigated, developers of the TEA-Ch appear to have produced a clinical instrument that continues to accumulate support for its use in various child populations.
147 References Achenbach, T., McConaughy, S., & Howell, C. (1987). Child/adolescent behavioral and emotional problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin, 101 213-232. American Academy of Pediatrics. (2000). Clinical pr actice guideline: Diagnosis and evaluation of the child with attention-deficit/hype ractivity disorder. Pediatrics, 105 1158-1170. American Educational Research Association, The Amer ican Psychological Association. (1999). The standards for educational and psychological tes ting Washington, DC: American Educational Research Association. Anderson, P. (2002). Assessment development of exec utive function (EF) during childhood. Child Neuropsychology, 8 71-82. Anderson, V. (1998). Assessing executive functions in children: Biological, psychological, and developmental considerations. Neuropsychological Rehabilitation, 8 319-349. Anderson, V., Anderson, P., Northam, E., Jacobs, R. & Catroppa, C. (2001). Development of executive functions through late chi ldhood and adolescence in an Australian sample. Developmental Neuropsychology, 20 385-406. Anderson, V., Fenwick, T., Manley, T., & Robertson, I. (1998). Attentional skills following traumatic brain injury in childhood: A co mponential analysis. Brain Injury, 12 937-949.
148 Anderson, V., Jacobs, R., & Harvey, A.S. (2005). Pr efrontal lesions and attentional skills in childhood. Journal of the International Neuropsychological Soc iety, 11 817831. Angello, L.M., Volpe, R.J., DiPerna, J.C., GueaskoMoore, S.P., Gureasko-Moore, D.P., Nebrig, M.R., & Ota, K. (2003). Assessment of atten tion-deficit/hyperactivity disorder: An evaluation of six published rating sca les. School Psychology Review, 32 241-262. American Academy of Child and Adolescent Psychiatry (1997) Practice parameters for the assessment and treatment of children, adolescen ts, and adults with attentiondeficit/hyperactivity disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 36, 85s-121s. Aron, A. & Aron, E.N. (1997). Statistics for the behavioral and social sciences: A brief course New Jersey: Prentice-Hall, Inc. Arterberry, M. E., Midgett, C., Putnick, D. L., & B ornstein, M. H. (2007). Early attention and literary experiences predict adaptive communica tion. First Language, 27 175Â–189 Band, G.P.H., & Scheres, A. (2005). Is inhibition i mpaired in ADHD? British Journal of Developmental Psychology, 23 517-521. Barkley, R.A. (1997). Behavioral inhibition, sustai ned attention, and executive functions: Constructing a unifying theory of ADHD. Psychological Bulletin, 121 65-94. Barkley, R.A. (1998). Attention deficit hyperactivity disorder: A handboo k for diagnosis and treatment (2nd ed.). New York: The Guilford Press.
149 Barkley, R.A. (1999). Response inhibition in attent ion-deficit/hyperactivity disorder. Mental Retardation and Developmental Disabilities, 5 177-184. Beveridge, M., Jarold, C., & Pettit, E. (2002). An experimental approach to executive fingerprinting in young children. Infant and Child Development, 11 107-123. Biederman, J., Monuteaux, M.C., Doyle, A.E., Seidma n, L.J., Wilens, T.E., Ferrero, F., Morgan, C.L., & Faraone, S. (2004). Impact of execu tive function deficits and Attention-Deficit/Hyperactivity Disorder (ADHD) on academic outcomes in children. Journal of Consulting and Clinical Psychology, 72 757-766. Brocki, K.C., & Bohlin, G. (2006). Developmental ch ange in the relation between executive functions and symptoms of ADHD and co-occ urring behaviour problems. Infant and Child Development, 15 19-40. Burgess, P.W., Alderman, N., Evans, J., Emslie, H., Wilson, B.A. (1998). The ecological validity of tests of executive function. Journal of the International Neuropsychological Society, 4 547-558. Burgess, P.W., Alderman, N., Forbes, C., Costello, A., Coates, L.M.A., Dawson, D.R., Anderson, N.D., Gilbert, S.J., Dumontheil, I., & Ch annon, S. (2006). The case for the development and use of Â“ecologically validÂ” mea sures of executive function in experimental and clinical neuropsychology. Journal of the International Neuropsychological Society, 12 194-209. Calhoun, J. (2006). Executive functions: A discussi on of the issues facing children with Autism Spectrum disorders and related disorders Seminars in Speech and Language, 27 60-72.
150 Castellanos, F.X. (1997). Toward a pathophysiology of attention-deficit/hyperactivity disorder. Clinical Pediatrics, 36 381-393. Ceci, S.J., & Tishman, J. (1984), Hyperactivity and incidental memory: Evidence for attentional diffusion. Child Development, 55 2192-2203. Chan, R.C.K., Wang, L., Ye, J., Leung, W.W.Y., Mok, M.Y.K. (2008). A psychometric study of the Test of Everyday Attention for Childre n in the Chinese setting. Archives of Clinical Neuropsychology, 23 455-466. Chaytor, N., & Schmitter-Edgecombe, M. (2003). The ecological validity of neuropsychological tests: A review of the literatur e on everyday cognitive skills. Neuropsychology Review, 13 181-199. Chaytor, N., & Schmitter-Edgecombe, M. (2007). Frac tionation of the dysexecutive syndrome in a heterogeneous neurological sample: Co mparing the Dysexecutive Questionnaire and the Brock Adaptive Functioning Qu estionnaire. Brain Injury, 21 615-621. Chi, T.C., & Hinshaw, S.P. (2002). Mother child rel ationships of children with ADHD: The role of maternal depressive symptoms and depres sion-related distortions. Journal of Abnormal Child Psychology, 30 387-400. Chugani, H.T. (1999). PET scanning studies of human brain development and plasticity. Developmental Neuropsychology, 16 379-381. Clark, C., Prior, M., & Kinsella, G. (2002). The re lationship between executive function abilities, adaptive behavior, and academic achievem ent in children with externalizing behavior problems. Journal of Child Psychology and Psychiatry, 43 785-796.
151 Coffey, C.E., McAllister, T.W., Silver, J.M. (2006) Guide to neuropsychiatric therapeutics New York: Lippincott Williams & Wilkins. Cohen, J. (1960). A coefficient of agreement for nominal scales Educational and Psychological Measurement, 20, 37Â–46 Cohen, J. (1988). Statistical power analysis for the behavioral scien ces (2nd ed.) Hillsdale, NJ: Erlbaum. Cohen, J. (1992). Quantitative methods in psycholog y a power primer. Psychological Bulletin, 112 155-159. Cohen, R., Lohr, I., Paul, R., & Boland, R. (2001) Impairments of attention and effort among patients with major affective disorders. The Journal of Neuropsychiatry and Clinical Neurosciences 13 385-395. Cooley, E.L., & Morris, R.D. (1990). Attention in c hildren: A neuropsychologically based model for assessment. Developmental Neuropsychology, 6 239-274. Crawford, J.R. (1998). Introduction to the assessme nt of attention and executive functioning. Neuropsychological Rehabilitation, 8 209-211. Cruddace, S.A., & Riddell, P.M. (2006). Attention p rocesses in children with movement difficulties, reading difficulties or both. Journal of Abnormal Child Psychology, 34 675-683. Dawson, P., & Guare, R. (2004). Executive skills in children and adolescents New York: The Guilford Press. DeGangi, G., & Proges, S. (1990). Neuroscience foundations of human performance Rockville, MD: American Occupation Therapy Associat ion.
152 Denckla, M.R. (1996). Research on executive functio n in a neurodevelopmental context: Application of clinical measures. Developmental Neuropsychology, 12 5-15. Dewey, D., Crawford, S.G., Kaplan, B.J. (2003). Cli nical importance of parent ratings of everyday cognitive abilities in children with learn ing and attention problems. Journal of Learning Disabilities, 36 87-95. Dickstein, S.G., Bannon, K., Castellanos, F.X., & M ilham, M.P. (2006). The neural correlates of attention deficit hyperactivity disor der: An ALE meta-analysis. Journal of Child Psychology and Psychiatry, 47 1051-1062. Epstein, H.T. (1974). Phrenoblysis: Special brain a nd neural growth periods: Human mental development. Developmental Psychobiology, 7, 217-224. Espy, K.A., Kaufmann, P.M., Glisky, M.L., & McDiarm id, M.D. (2001). New procedures to assess executive functions in prescho ol children. The Clinical Neuropsychologist, 15 46-58. Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39 175-191. Feifer, S.G., & Rattan, G. (2007). Executive functi oning skills in male students with social-emotional disorders. International Journal of Neuroscience, 117 15651577. Field, A.P. (2009). Discovering statistics using SPSS (3rd ed.). London: Sage. Frazier, T.W., Demaree, H.A., & Youngstrom, E.A. (2 004). Meta-analysis of intellectual and neuropsychological test performance in attentio n-deficit/hyperactivity disorder. Neuropsychology, 18 543-555.
153 Frick, P.J. (2000). Laboratory and performance-base d measures of childhood disorders: Introduction to the special section. Journal of Clinical Child Psychology, 29 475478. Frick, P.J. (2000). Laboratory and performance-base d measures of childhood disorders: Introduction to the special section. Journal of Clinical Child Psychology, 29 475478. Friedman, N.P., Haberstick, B.C., Willcutt, E.G., M iyake, A., Young, S.E., Corley, R.P., & Hewitt, J.K. (2007). Greater attention problems d uring childhood predict poorer executive functioning in late adolescence. Psychological Science, 18 893-900. Gale, A., & Lynn, R. (1972). A developmental study of attention. British Journal of Educational Psychology, 42 260-266. Gazzaniga, M.S., Ivry, R.B., & Mangun, G.R. (2002). Cognitive neuroscience: The biology of the mind (2nd ed.). New York: W.W. Norton & Company, Inc. Glascoe, F.P. (2000). Evidence-based approach to de velopmental and behavioural surveillance using parentsÂ’ concerns. Child: Care, Health and Development, 26 137-149. Glass, G. V., & Hopkins, K. H. (1996). Statistical methods in education and psychology Boston: Allyn and Bacon. Gilotty, L., Kenworthy, L., Sirian, L., Black, D.O. & Wagner, A.E. (2002). Adaptive skills and executive function in autism spectrum di sorders. Child Neuropsychology, 8 241-248. Gioia, G.A., Isquith, P.K. (2004). Ecological asses sment of executive function in traumatic brain injury. Developmental Neuropsychology, 25 135-158.
154 Gioia, G.A., Isquith, P.K., Guy, S.C., & Kenworthy, L. (2000). Behavior Rating Inventory of Executive Function Odessa, FL: Psychological Assessment Resources. Gouchie, C., & Kimura, D. (1991). The relationship between testosterone levels and cognitive ability patterns. Psychoneuroendocrinology, 16 323-334. Haile Mariam, A., Bradley-Johnson, S., & Johnson, C .M. (2002). PediatriciansÂ’ preferences for ADHD information from schools. School Psychology Review, 31 94-105. Harrison, P.L., & Oakland, T. (2003). Adaptive Behavior Assessment System-Second Edition San Antonio, TX: The Psychological Corporation. Heaton, S.C., Reader, S.K., Preston, A.S., Fennell, E.B., Puyana, O.E., Gill, N., & Johnson, J.H. (2001). Child Neuropsychology (Neuropsychology, Development and Cognition: Section C) 7 251-264. Hoyt, W.T., Leierer, S., & Millington, M.J. (2006). Analysis and interpretation of findings using multiple regression techniques. Rehabilitation Counseling Bulletin, 49 223-233. Hudspeth, W.J., & Pribram, K.H. (1990). Stages of b rain and cognitive maturation. Journal of Educational Psychology, 82 881-884. Hudspeth, W.J., & Pribram, K.H. (1992). Psychophysi ological indices of cerebral maturation. International Journal of Psychophysiology, 12 19-29. Hughes, C. (2002). Executive functions and developm ent: Why the interest? Infant and Child Development, 11 69-71.
155 Hughes, C., & Graham, A. (2002). Measuring executiv e functions in childhood: Problems and solutions? Child and Adolescent Mental Health, 7 131-142. Isquith, P.K., Gioia, G.A., & Espy, K.A. (2004). Ex ecutive function in preschool children: Examination through everyday behavior. Developmental Neuropsychology, 26 403-422. Kenworthy, L., Yerys, B.E., Gutermuth, L., & Wallac e, G.L. (2008). Understanding the executive control in Autism Spectrum Disorders in t he lab and in the real world. Neuropsychology Review, 18 320-338. Klenberg, L., Korkman, M., & Lahti-Nuuttila, P. (20 01). Differential development of attention and executive functions in 3to 12year old Finnish children. Developmental Neuropsychology, 20 407-428. Koonce, D.A. (2007). Attention-deficit/hyperactivit y disorder assessment practices by practicing school psychologists: A national survey. Journal of Psycheducational Assessment, 25 319-333. Korkman, M., Kirk, U., & Kemp, S. (1997). NEPSY New York: Psychological Corporation. Landis, J. R., & Koch, G. G. (1977). The measuremen t of observer agreement for categorical data. Biometrics, 33 159-174. Lawrence, V., Houghton, S., Douglas, G., Durkin, K. Whiting, K., & Tannock, R. (2004). Executive function and ADHD: A comparison o f childrenÂ’s performance during neuropsychological testing and real-world ac tivities. Journal of Attention Disorders, 7 137-148.
156 Lezak, M.D., Howieson, D.B., & Loring, D.W. (Eds.). (2004). Neuropsychological Assessment (4th ed). New York: Oxford University Press. Lorys, A.R., Hynd, G.W., & Lahey, B.B. (1990). Do n eurocognitive measures differentiate attention deficit disorder (ADD) with and without hyperactivity? Archives of Clinical Neuropsychology, 5 119-135. Lyon, G.R., & Krasnegor, N.A., (1996). Attention, Memory, and Executive Functions Baltimore: Brookes. Mandal, R.L., Olmi, D.J., Wilczynski, S.M. (1999). Behavior rating scales: Concordance between multiple informants in the diagnosis of att ention-deficit/hyperactivity disorder. Journal of Attention Disorders, 3 97-103. Mangeot, S., Armstrong, K., Colvin, A.N., Yeates, K .O., & Taylor, H.G. (2002). Longterm executive function deficits in children with t raumatic brain injuries: Assessment using the Behavior Rating Inventory of E xecutive Function (BRIEF). Child Neuropsychology, 8 271-284. Manley, T., Anderson, V., Nimmo-Smith, I., Turner, A., Watson, P., & Robertson, I.H. (2001). The differential assessment of childrenÂ’s a ttention: The Test of Everyday Attention for Children (TEA-Ch), normative sample a nd ADHD performance. Journal of Child Psychology and Psychiatry, 42 1065-1081. Manley, T., Robertson, I., Anderson, V., & Nimmo-Sm ith, I. (1999). The test of everyday attention for children. Cambridge, UK: Thames Valle y Test Company. Malloy, P., & Grace, J. (2005). A review of rating scales for measuring behavior change due to frontal systems damage. Cognitive Behavior Neurology, 18, 18-27.
157 Mares, D., McLuckie, A., Schwartz, M., & Saini, M. (2007). Executive function impairments in children with attention-deficit/hype ractivity disorder: Do they differ between school and home environments? The Canadian Journal of Psychiatry, 52 527-533. Meltzer, L. (2007). Executive function in education New York: The Guilford Press. Meyer, G.J., Finn, S.E., Eyde, L.D., Kay, G.G., Mor eland, K.L., Dies, R.R., Eisman, E.J., Kubiszyn, T.W., & Reed, R.G. (2001). Psychological testing and psychological assessment: A review of evidence and issues. American Psychologist, 56 128165. Mirsky, A.F., Anthony, B.J., Duncan, C.C., Ahearn, M.B., & Kellam, S.G. (1991). Analysis of the elements of attention; A neuropsych ological approach. Neuropsychology Review, 2 109-145. Myers, R. (1990). Classical and modern regression with applications (2nd ed.)Bosotn, MA: Duxbury. Mullane, J.C., & Corkum, P.V. (2007). The relations hip between working memory, inhibition, and performance on the Wisconsin Card S orting Test in children with and without ADHD. Journal of Psychoeducational Assessment, 25 211-221. NIH Consensus Development Panel (2000). National In stitutes of Health Consensus Development Conference statement: Diagnosis and tre atment of attentiondeficit/hyperactivity disorder (ADHD). Journal of the American Academy of Child and Adolescent Psychiatry, 39 182-193.
158 Ozonoff & Jensen (1999). Brief Report: Specific exe cutive function profiles in three neurodevelopmental disorders. Journal of Autism and Developmental Disorders, 29, 171-177. Palfrey, J.S., Levine, M.D., Walker, D.K., & Sulliv an, M. (1985). The emergence of attention deficits in early childhood: A prospectiv e study. Developmental and Behavioral Pediatrics, 6 339-347. Papazoglou, A., King, T.Z., Morris, R.D., Krawiecki N. (2009). Parent report of attention problems predicts later adaptive function ing in children with brain tumors. Child Neuropsychology, 15 40-52. Pascualvaca, D.M., Anthony, B.J., Arnold, E., Rebok G.W., Ahearn, M., Kellam, S.G., & Mirsky, A.F. (1997). Attention performance in an epidemiological sample of urban children: The role of gender and verbal intel ligence. Child Neuropsychology, 3 13-27. Pasini, A., Paloscia, C., Alessandrelli, R., Porfir io, M.C., & Curatolo, P. (2007). Attention and executive functions profile in drug n ave ADHD subtypes. Brain & Development, 29 400-408. Pedhazur, E.J. (1997). Multiple regression in behavioral research: Explana tion and prediction (3rd ed.) Australia: Wadsworth Thomson Learning Inc. Pelham, W.E., Fabiano, G.A., Massetti, G.M (2005). Evidence-based assessment of Attention-Deficit/Hyperactivity Disorder in childre n and adolescents. Journal of Clinical and Child Adolescent Psychology, 34 3, 449-476.
159 Perugini, E.M., Harvey, E.A., Lovejoy, D.W., Sandst rom, K., & Webb, A.H. (2000). The predictive power of combined neuropsychological mea sures for AttentionDeficit/Hyperactivity Disorder in children. Child Neuropsychology, 6, 101-114. Price, K.J., Joschko, M., & Kern, K. (2003). The ec ological validity of pediatric neuropsychological tests of attention. The Clinical Neuropsychologist, 17 170181. Posner, M.I., & Peterson, S.E. (1990). The attentio n system of the human brain. Annual Review of Neuroscience, 13 25-42. Rapport, M.D., Chung, K-M., Shore, G., Denney, C.B. & Isaacs, P. (2000). Upgrading the science and technology of assessment and diagno sis: Laboratory and clinicbased assessment of children with ADHD. Journal of Clinical Child Psychology, 29 555-568. Rebok, G.W., Smith, C. B., Pascualvaca, D.M., Mirsk y, A.F., Anthony, B. J., & Kellam, S.G. (1997). Developmental changes in attentional p erformance in urban children from eight to thirteen years. Child Neuropsychology, 3 28-46. Riggs, N.R., Blair, C.B., & Greenberg, M.T. (2003). Concurrent and 2-year longitudinal relations between executive function and the behavi or of 1st and 2nd grade children Child Neuropsychology, 9 267-276. Robbins, T.W. (1996). Dissociating executive functi ons of the prefrontal cortex. Phil. Trans.R. Soc. Lond. 351 1463-1471. Romine, C.B., Reynolds, C.R. (2005). A model of the development of frontal lobe functioning: Findings from a meta-analysis. Applied Neuropsychology, 12 190201.
160 Roth, R.M., & Saykin, A.J. (2004). Executive dysfun ction in attention deficit/hyperactivity disorder: Cognitive and neuro imaging findings. Psychiatric Clinics of North America, 27 83-96. Rust, J.O., & Wallace, M.A. (2004). Test review. Journal of Psychoeducational Assessment, 22 367-373. Sciutto, M.J., & Eisenberg, M. (2007). Evaluating t he evidence for and against the overdiagnosis of ADHD. Journal of Attention Disorders, 11 106-113. Shafritz, K.M., Marchione, K.E., Gore, J.C., Shaywi tz, S.E., & Shaywitz, B.A. (2004). The effects of methylphenidate on neural systems of attention in Attention Deficit hyperactivity disorder. American Journal of Psychiatry, 161 1990-1997. Shallice, T., Marzocchi, G.M., Coser, S., Del Savio M., Meuter, R.F., & Rumiati, R.I. (2002). Executive function profile of children with attention deficit hyperactivity disorder. Developmental Neuropsychology, 21 43-71. Simonsen, B.M., & Bullis, M.D. (2007). The effectiv eness of using a multiple gating approach to discriminate among ADHD subtypes. Journal of Emotional and Behavioral Disorders, 15 223-236. St. Clair-Thompson, H.L., & Gathercole, S.E. (2006) Executive functions and achievements in school: Shifting, updating, inhibit ion, and working memory. The Quarterly Journal of Experimental Psychology, 59 745-759. Stavro, G.M., Ettenhofer, M.L., & Nigg, J.T. (2007) Executive functions and adaptive functioning in young adult attention-deficit/hypera ctivity disorder. Journal of the International Neuropsychological Society, 13 324-334.
161 Stein, M. A., Szumowski, E., Blondis, T. A., and Ro izen, N. J. (1995). Adaptive skills dysfunction in ADD and ADHD children. Journal of Child Psychology and Psychiatry, 36 663Â–670. Stemler, Steven E. (2004). A comparison of consensu s, consistency, and measurement approaches to estimating interrater reliability. Practical Assessment, Research & Evaluation, 9 110-115. Stuss, D.T. (1992). Biological and psychological de velopment of executive functions. Brain and Cognition, 20 8-23. Stuss, D.T., & Alexander, M.P. (2000). Executive fu nctions and the frontal lobes: A conceptual view. Psychological Research, 63 289-298. Tanner, J.M. (1962). Growth at adolescence Oxford: Blackwell Scientific Publications. Tripp, G., Schaughency, E.A., & Clarke, B. (2006). Parent and teacher rating scales in the evaluation of attention-deficit/hyperactivity disor der: Contribution to diagnosis and differential diagnosis in clinically referred c hildren. Developmental and Behavioral Pediatrics, 27 209-217. Waber, D.P., Gerber, E.B., Turcios, V.Y., Wagner, E .R., & Forbes, P.W. (2006). Executive functions and performance on high-stakes testing in children from urban schools. Developmental Neuropsychology, 29 459-477. Warner-Rogers, J., Taylor, A., Taylor, E., & Sandbe rg, S. (2000). Inattentive behavior in childhood: Epidemiology and implications for develo pment. Journal of Learning Disabilities, 33 520-536.
162 Welsh, M.C., & Pennington, B.F. (1988). Assessing f rontal lobe functioning in children: Views from developmental psychology. Developmental Neuropsychology 4 199230. Welsh, M.C., Pennington, B.F. (1991). A normative-d evelopmental study of executive function: A window on prefrontal function in childr en. Developmental Neuropsychology, 7 131-149. Weyandt, L.L. (2005). Executive functions in childr en, adolescents, and adults with attention deficit hyperactivity disorder: Introduct ion to the special issue. Developmental Neuropsychology, 27 1-10. Wilding, T. (2005). Is there any value in the diffe rential assessment of attention? British Journal of Developmental Psychology, 23 507-512. Wu, K.K., Anderson, V., & Castiello, U. (2002). Neu ropsychological evaluation of deficits in executive functioning for ADHD children with or without learning disabilities. Developmental Neuropsychology, 22 501-531.
164 Appendix A: TEA-Ch Subtest Descriptions Sustained Attention Subtests Score! Children are required to silently count the number of target tones that are presented on a 15 minute audiotape. There are 10 total trials, wit h varying interval tones ranging in number from 9 to 15. This is a good test of the chi ldÂ’s ability to self-sustain his or her own attention due to the long gaps in between tones and the redundancy of the task. Sky Search DT This is a Â“dual taskÂ” that requires children to sim ultaneously perform tasks from the Sky Search and Score! Subtests. As such, this subtest i nvolves simultaneously identifying visual targets present among distracters, and count ing tones on an audiotape. Score DT This is a Â“dual taskÂ” which combines a task of coun ting the number of tones presented with another listening task. The child is required to listen for an animal name during an audiotaped news report as they count the number of tones presented. After each of the 10 trials the child is asked to report the number of t ones counted and the name of the animal. Walk, DonÂ’t Walk Children are asked to mark steps along a paper pat h with a pen each time they hear a tone on the tape, but refrain from marking a step i f the tone is immediately followed by a second tone. The rate of the tones increase as the child progresses through the 20 trials. Code Transmission Children are asked to listen to a long, monotonous series of spoken numbers, listening for two Â‘5sÂ’ to be presented in a row. When this patter n is noted, the child is asked to state the number presented prior to the target Â‘5s.Â’ Selective Attention Subtests Sky Search In this brief, timed subtest children are instructe d to find target spaceships present among similar distracter spaceships. The second part of t his subtest involves no distracter stimuli and serves as a control for motor function. Subtrac ting the score obtained on part 1 from part 2 gives a measure of the childÂ’s ability to ma ke this selection that is relatively free from the influence of motor function. Map Mission Children are given one minute to quickly locate sma ll targets in an array of distracters.
165 Attentional Control/Switching Creature Counting Children are required to switch between counting fo rward and backward in response to visual stimuli (creatures in a tunnel). Speed and a ccuracy are factored into the scoring. Opposite Worlds In the Â“Same WorldÂ” scenario, children are asked to name digits 1 and 2 scattered along a path. In the subsequent Â“Opposite WorldÂ” task, chil dren must say Â‘1Â’ when they see Â‘2Â’ and Â‘2Â’ when they see Â‘1.Â’
166 Appendix B: TEA-Ch Administration and Scoring by Su btest Subtest 1: Sky Search In this brief, timed task children are instructed t o find and circle targets present among similar distracter targets. The second part of this subtest involves no distracter stimuli and serves as a control for motor function. Subtracting the score obtained on Part 1 from Part 2 provides a measure of the childÂ’s ability to indi cate a response free from the influence of motor function. MATERIALS : Stopwatch, non permanent marker, small Sky Search practice sheet, large Sky Search test sheet (version A or B), Sky S earch small Motor Control sheet ___________________________________________________ _____________________ Subtest 2: Score! Children are required to count silently the number of target tones that are presented on a CD. There are 10 total trials, with varying interva l tones ranging in number from 9 to 15. MATERIALS : CD player, CD ___________________________________________________ _____________________ Subtest 3: Creature Counting Children are required to switch between counting fo rward and backward in response to visual stimuli. Speed and accuracy are factored int o the scoring. MATERIALS : Stimulus book, stopwatch ___________________________________________________ _____________________ Subtest 4: Sky Search DT This is a Â“dual taskÂ” that requires children to sim ultaneously perform tasks from the Sky Search and Score! subtests. This task involves simu ltaneously identifying visual targets present among distracters, and counting tones that are presented on a CD. MATERIALS : CD player, CD, stopwatch, non-permanent marker ___________________________________________________ _____________________ Subtest 5: Map Mission Children are given one minute to quickly locate sma ll targets in an array of distracters. MATERIALS : Large map (version A or B), non-permanent marker, stopwatch ___________________________________________________ _____________________
167 Subtest 6: Score DT This is a Â“dual taskÂ” of attention. The child is in structed to listen for an animal name during a news report as they count the number of to nes presented. After each trial the child is asked to report the number of tones counte d and the name of the animal. MATERIALS : CD player, CD. ___________________________________________________ _____________________ Subtest 7: Walk, DonÂ’t Walk Children are asked to mark steps along a paper pat h with a pen each time they hear a tone on the tape, but refrain from marking a step i f the tone is immediately followed by a second tone. The rate of the tones increase as the child progresses through the 20 trials. MATERIALS : CD player, CD, non-permanent marker, Walk DonÂ’t W alk sheet ___________________________________________________ _____________________ Subtest 8: Opposite Worlds In the Â“Same WorldÂ” scenario, children are asked to name digits 1 and 2 scattered along a path. In the subsequent Â“Opposite WorldÂ” task, chil dren must say Â‘1Â’ when they see Â‘2Â’ and Â‘2Â’ when they see Â‘1.Â’ MATERIALS : Stimulus book, stopwatch ___________________________________________________ _____________________ Subtest 9: Code Transmission Children are asked to listen to a long, monotonous series of spoken numbers, listening for two Â‘5sÂ’ to be presented in a row. When this patter n is noted, the child is asked to state the number presented prior to the target Â‘5s.Â’ MATERIALS : CD player, CD ___________________________________________________ _____________________
168 Appendix C: Integrity Checklist Please be sure to review and complete all steps in the outlined order before turning in protocols and consent forms. This Integrity Checkli st is provided for the purposes of ensuring consistent methods for data collection across all p articipating school psychologists. ___ Make contact with elementary school teachers an d distribute the Teacher Informed Consent to Participate in Research form to those who indicate possible interest. ___ Obtain signed USF Teacher Informed Consent to Participate in Rese arch form from elementary school classroom teacher. ___ Distribute USF Parental Informed Consent to Participate in Researc h form to teachers to pass out to parents. ___ Collect Teacher Informed Consent to Participate in Research and Parental Informed Consent to Participate in Research within one week from date of distribution. ___ Distribute Parent and Teacher packets of ratin g forms (pre-made) to elementary school classroom teachers (includes BRIEF, ABAS-II) for distribution to parents. ___ Consult with classroom teacher regarding best t ime/place for TEA-Ch administration (allot 30-35 minutes per testing ses sion). ___ Review and obtain signed childÂ’s Assent to Participate in Research form on date of testing. ___ Administer the TEA-Ch according to guidelines a nd rules provided in the packet. ___ Collect Parent and Teacher packets of rating fo rms (BRIEF and ABAS-II) within the two weeks as indicated on the consent forms. ___ Send all protocols, ratings forms, and consent forms as one packet as provided in the manila envelope to designated Research Committe e member via district courier. Thank you for your participation and hard work! Ple ase feel free to contact me via email ( firstname.lastname@example.org ) or phone (716-908-1921) for any questions, commen ts, or concerns.
169 Appendix D: Consent and Assent Forms
179 About the Author Eun-Yeop Lee received a BachelorÂ’s Degree in Psych obiology from Binghamton University in 2003 and a M.A. in School Psychology from the University of South Florida in 2005. She earned a graduate certificate in Aging and Neuroscience from the University of South Florida in 2007. Following her doctoral internship through the Polk County Public Schools she completed a clinical inte rnship in Pediatric Neuropsychology. Ms. Lee has also worked in a variety of settings in cluding public schools, outpatient clinics, and university affiliated hospitals. Her research interests include neuropsychological assessment as well as behavioral and cognitive implications of Autism Spectrum disorders, genetic disorders, and m edical disorders.