USF Libraries
USF Digital Collections

The effectiveness of prevent-teach-reinforce :

MISSING IMAGE

Material Information

Title:
The effectiveness of prevent-teach-reinforce : does the presence of comorbid internalizing behavior problems moderate outcomes for children with externalizing behavior problems?
Physical Description:
Book
Language:
English
Creator:
Saari, Bonnie
Publisher:
University of South Florida
Place of Publication:
Tampa, Fla
Publication Date:

Subjects

Subjects / Keywords:
School-Based Treatment
Positive Behavior Support
Elementary
Intervention
Functional Behavior Assessment
Dissertations, Academic -- Psychological & Social Foundations -- Masters -- USF   ( lcsh )
Genre:
non-fiction   ( marcgt )

Notes

Abstract:
ABSTRACT: This study examined the effectiveness of a school-based intervention process known as Prevent-Teach-Reinforce for children with a combination of externalizing and internalizing behaviors compared to children with only externalizing behaviors. The dependent variables examined were social skills, problem behaviors, and academic engaged time. Data for the current study were taken from archival data collected by the Florida Mental Health Institute that included students in kindergarten through 8th grade. A series of repeated-measures analysis of variance (ANOVA) was used to identify differences in improvement on the dependent variables for the two groups of students. Research questions focused on the main effects as well as interaction effects between the type(s) of behavioral problems displayed (i.e., externalizing only, combination of externalizing and internalizing). Behavior problem classification was determined by calculating students' individual subscale scores on the Social Skills Rating Scale. The current study found support for the use of the Prevent-Teach-Reinforce intervention for children with varying behavioral profiles. Significant improvements were found in social skills, behavioral problems, and academic engaged time for students. Additionally, results of this study indicate that internalizing behaviors did not serve as a moderator to treatment effectiveness for students with externalizing behavior problems who received the PTR intervention. That is, improvements were similar for both groups, demonstrating that PTR is a process that can be used in an equally-effective way for both populations.
Thesis:
Thesis (Ed.S.)--University of South Florida, 2010.
Bibliography:
Includes bibliographical references.
System Details:
Mode of access: World Wide Web.
System Details:
System requirements: World Wide Web browser and PDF reader.
Statement of Responsibility:
by Bonnie Saari.
General Note:
Title from PDF of title page.
General Note:
Document formatted into pages; contains X pages.

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
usfldc doi - E14-SFE0003308
usfldc handle - e14.3308
System ID:
SFS0027624:00001


This item is only available as the following downloads:


Full Text
xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam 22 Ka 4500
controlfield tag 007 cr-bnu---uuuuu
008 s2010 flu s 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0003308
035
(OCoLC)
040
FHM
c FHM
049
FHMM
090
XX9999 (Online)
1 100
Saari, Bonnie.
0 245
The effectiveness of prevent-teach-reinforce :
b does the presence of comorbid internalizing behavior problems moderate outcomes for children with externalizing behavior problems?
h [electronic resource] /
by Bonnie Saari.
260
[Tampa, Fla] :
University of South Florida,
2010.
500
Title from PDF of title page.
Document formatted into pages; contains X pages.
502
Thesis (Ed.S.)--University of South Florida, 2010.
504
Includes bibliographical references.
516
Text (Electronic thesis) in PDF format.
538
Mode of access: World Wide Web.
System requirements: World Wide Web browser and PDF reader.
3 520
ABSTRACT: This study examined the effectiveness of a school-based intervention process known as Prevent-Teach-Reinforce for children with a combination of externalizing and internalizing behaviors compared to children with only externalizing behaviors. The dependent variables examined were social skills, problem behaviors, and academic engaged time. Data for the current study were taken from archival data collected by the Florida Mental Health Institute that included students in kindergarten through 8th grade. A series of repeated-measures analysis of variance (ANOVA) was used to identify differences in improvement on the dependent variables for the two groups of students. Research questions focused on the main effects as well as interaction effects between the type(s) of behavioral problems displayed (i.e., externalizing only, combination of externalizing and internalizing). Behavior problem classification was determined by calculating students' individual subscale scores on the Social Skills Rating Scale. The current study found support for the use of the Prevent-Teach-Reinforce intervention for children with varying behavioral profiles. Significant improvements were found in social skills, behavioral problems, and academic engaged time for students. Additionally, results of this study indicate that internalizing behaviors did not serve as a moderator to treatment effectiveness for students with externalizing behavior problems who received the PTR intervention. That is, improvements were similar for both groups, demonstrating that PTR is a process that can be used in an equally-effective way for both populations.
590
Advisor: Linda Raffaele Mendez, Ph.D.
653
School-Based Treatment
Positive Behavior Support
Elementary
Intervention
Functional Behavior Assessment
690
Dissertations, Academic
z USF
x Psychological & Social Foundations
Masters.
773
t USF Electronic Theses and Dissertations.
4 856
u http://digital.lib.usf.edu/?e14.3308



PAGE 1

The Effectiveness of Prevent Teach Reinforce: Does the Presence of Comorbid Internalizing Behavior Problems Moderat e Outcomes for Children with Externalizing Behavior Problems? b y Bonnie Saari A thesis submitted in partial fulfillment of the requirements for the degree of Education Specialist Department of Psychological and Social Foundations College of Education University of South Florida Major Professor: Linda Raffaele Mendez, Ph.D. Shannon Suldo, Ph.D. Rose Iovannone, Ph.D. John Ferron, Ph.D. Date of Approval: December 14, 2009 Keywords: School Based Treatment, Positive Behavior Support, Elementary, Intervention, Functional Behavior Assessment Copyright 2010 Bonnie Saari

PAGE 2

Acknowledgments I would like to thank my committee members for all the time and effort they spent helping me with this research project. I would first like to thank Dr. Raffaele Mendez for her patience, encouragement, and feedback during this entire process. I want to thank D r. Suldo for teaching me to maintain high standards for myself and my work I would like to thank Dr. Ferron for his willingness to answer all my statistic s questions and his patience throughout this Finally, I want to thank Dr. Iovannone for allowing me to work with her and providing me with her expertise. The help I have received from each of you has been invaluable.

PAGE 3

i Table of Contents List of Tables iii Li st of Figures iv Abstract v Chapter 1 Introduction 1 Statement of the Problem 1 Rationale for the Study 3 Purpose of the Study 3 Significance of the Study 4 Cha pter 2 Literature Review 6 Overview 6 Prevalence of External izing Behaviors in Youth 6 Impact of Externalizing B ehaviors on Development 7 Academics 8 Peer Relationships 9 Long term Outcomes 10 Students with Both Externalizing and Internalizing Behaviors 1 0 Treat ment and Interventi on 12 The Importance of Moderators in Treatment 13 Combined Medication and Behavioral S tudies 13 Paren t Training 15 School Based Treatment 18 Positive Behavi or Support 18 Prevent T each Reinforce 22 Conclusion 2 7 Chapter 3 Meth ods 29 Study Design 29 Curr ent Study 3 1 Par ticipants 32 Instrumentation 32 Type of Behavio ral Problem 32 Soci al Skills 34 Behavior al Problem 35 Academic Engaged Time 35 P rocedure 36 Data Analyse s 3 7

PAGE 4

ii Chapter 4 Results 38 Demographic Characteristics of Sample 39 Mean Pre and Post Scores 40 Outcome Variable 1: Academic Engaged Time 41 Outcome Variable 2: Social Skills 4 4 Outcome Variable 3: Behavior Problems 47 Chapter 5 Discussion 5 0 Major Findings of the Study 51 Contributions to the Literature 5 2 Limitations 53 Directions for Future Research 56 Implications for the Field of School Psychology 57 Conclusion 58 References 61

PAGE 5

iii List of Tables Table 1 Demographic Characteristics of Sample 39 Table 2 Mean Pre, Post and Follow up Sc ores for Students with o nly Externalizing Problems 40 Table 3 Mean Pre, Post, and Follow up Score s for Students with Comorbid Problems 40 Table 4 AET ANOVA for Students with Externalizing Behaviors b y Time 41 Table 5 AET ANOVA for Students with Both Externalizing and Internalizing Behaviors by Time 42 Table 6 AET ANOVA with Time and Group Membership and their Interaction 43 Table 7 Social Skills ANOVA for Students with Externalizing Behaviors by Time 44 Table 8 Social Skills ANOVA for Students with Comorbid Behaviors by Time 45 Table 9 Social Skills Scores ANOVA with Time and Group Membership and their Interaction 46 Table 10 Behavior Problems ANOVA for Students with Externalizing Behaviors by Time 47 Table 1 1 Behavior Problems ANOVA for Students with Comorbid Behaviors by Time 48 Table 1 2 Behavior Problems ANOVA wi th Time and Group Membership and their Interaction 49

PAGE 6

iv List of Figures Figure 1 Mean Pre, Post, and Follow up Academic Engaged Time Scores 43 Figure 2 Mean Pre, Post and Follow up Social Skills Scores 46 Figure 3 Mean Pre, Post, and Follow up Behavior Problem Scores 49

PAGE 7

v The Effectiveness of Prevent Teach Reinforce: Does the Presence of Comorbid Internalizing Behavior Problems Moderate Outcomes for Children with Externalizing Behavior Problems? Bonnie Saari ABSTRACT This study examined the effectiveness of a school based intervention process known as Prevent Teach Reinforce for children with a combination of externalizing and internalizing behaviors compared to children with only externalizing behaviors. Th e dependent variables examined were social skills, problem behaviors, and academic engaged time. Data for the current study were taken from archival data collected by the Florida Mental Health Institute that included students in kindergarten through 8 th grade. A series of repeated measures analysis of variance (ANOVA) was used to identify differences in improvement on the dependent variables for the two groups of students. Research questions focused on the main effects as well as interaction effects bet ween the type(s) of behavioral problems displayed (i.e., externalizing only, combination of externalizing and internalizing). Behavior problem classification was determined by Sc ale.

PAGE 8

vi The current study found support for the use of the Prevent Teach Reinforce intervention for children with varying behavioral profiles. Significant improvements were found in social skills, behavioral problems, and academic engaged time for students. Additionally, results of this study indicate that internalizing behaviors did not serve as a moderator to treatment effectiveness for students with externalizing behavior problems who received the PTR intervention. That is, improvements were similar for both groups, demonstrating that PTR is a process that can be used in an equally effective way for both populations.

PAGE 9

1 Chapter I Introduction Statem ent of the Problem Reports from parents, teachers, national statistics, and school data attest to the fact that a substanti al number of children display problematic behavior in schools ( Carpenter & Nangle, 2002 ; Federal Interagency Forum on Child and Family Statistics 2008 ; Mash & Barkley, 2003 ). Likewise, a wealth of research has documented the negative outcomes associated with these behaviors. Specifically, externalizing behavioral problems (e.g., defiance, physical aggression, and verbal aggression) are negatively correlated with academic achievement ( Brunnekreef, Sonneville, Althaus, Minderaa, Oldehinkel, Verhulst, & Ormel, 2007 ) positive relationships with teachers ( Henricsson & Rydell, 2004 ) positive relationshi ps with peers ( Coie & Kupersmidt, 1983 ) and self esteem ( Krettenauer, Ullrich, Hofmann, and Edelstein, 2003 ). Additionally, externalizing behaviors are positively correlated with school suspension ( Skiba, Peterson & Williams, 1997 ) dropout rates ( Bock, Tapscott, & Savner, 1998 ) and association with deviant peers groups ( Dishion, French, & Patterson, 1995 ; Reid, 1993 ) Adding to these concerns is the knowledge that many children with externalizing behaviors also display internalizing behavior problems ( e.g., withdrawal from peers, flat affect, and excessive crying ; Mash & Barkley, 2003 ). The co occurrence of internalizing behaviors with externalizing behaviors poses additional concerns in that these students

PAGE 10

2 often respond differently to treatment than those children who have only one type of behavioral concern ( Jensen, et al., 2001; Kazdin & Wassell, 1999 ; March et al., 2000 ). Within the school setting, the treatment of behavior problems is increasingly delivered through a three tiered model know n as Positive Behavior Support (PBS) ( Dunlap Sailor, Horner, & Sugai, 2009 ) Within this system, as the intensity of a problem increases so does the intervention used to treat the problem. R esearch regarding PBS and students with both externalizing and internalizing behaviors has demonstrated that these students are resistant to less intensive interventions (known as Tier 1 and Tier II ; Lane, Wehby, Roberston, & Rogers, 2007 ). Because of the ir resistance to these interventions, a more individualized process, known as a functional behavior analysis, may be used (i.e., a Tier III intervention). A functional behavior analysis (FBA) is a process in which the antecedents, behaviors, and consequen ces to behaviors are identified (Scott, Anderson, Mancil, & Alter, 2009) The goal of this process is to find ways to alter the relationship between these factors in order to decrease problematic behavior and increase desirable behavior. Prevent Teach R einforce (PTR) is a specific approach to the Tier III process that seeks to address concerns regarding the feasibility of such a process in schools by standardizing the procedures ( Iovannone, Greenbaum, Wang, Kincaid, & Strain, 2009; Kern, Hilt, & Gresham, 2004 ) In this way, the process can be implemented by a teacher in a typical classroom with the guidance of a behavior consultant as opposed to being conducted by an expert. Preliminary data support the use of PTR in reducing problematic behavior and in creasing prosocial behavior in children with severe externalizing behaviors ( Iovannone, Greenbaum, Wang, Kincaid, & Strain, 2009 ) However,

PAGE 11

3 additional research is needed to assess additional popula tions that may benefit from PTR including children with comorbid internalizing and externalizing behavior problems Rationale for the Study To date no research has explor ed the effectiveness of the PTR process for those students with both externalizing and internalizing behavioral concerns. Some research purp orts that functional behavioral assessment may not be effective for dealing with behaviors related to internalizing concerns because this process only identifies proximal variables without addressing distal and personal variables such as family dysfunction and emotional regulation ( ). Additionally, research that addresses intervention effectiveness often demo nstrates that interventions that have been shown to be effective with one group of students do not necessarily generalize to all students since many children have co occurring concerns that need special consideration ( Jensen, et al., 2001; Kazdin & Wassell, 1999; March et al., 2000 ). Purpose of the Study The current study has been designed to address two areas in which additional research is needed First, this study will add to the research regar ding treatment effectiveness for children with externalizing behavior problems that co occur with internalizi ng psychopathology Specifically, information will be gathered to assess whether the school based, Tier III intervention known as Prevent Teach Reinforce is effective for students displaying both externalizing and internalizing b ehavioral problems. Secon d this study will provide additional empirical data on the use of FBAs with internalizing

PAGE 12

4 behavior problems. The following specific research questions will be investigated in the study: 1. Do children with only externalizing behaviors show improvement in social skills, problem behaviors, and academic engaged time as a result of the PTR intervention? 2. Do children with a combination of externalizing and internalizing behavior problems show improvement in social skills, problem behav iors, and academic engaged time as a result of the PTR intervention? 3. Is there a difference in levels of improvement in social skills, problem behaviors, and academic engaged time as a result of the PTR intervention for children with externalizing behaviors only vs. those with a comb ination of internalizing and externalizing behavior problems? Significance of the Study A wealth of information exists regarding the negative impact of behavioral problems on academic, social, and emotional outcomes for children (Brunnekreef, Sonneville, A lthaus, Minderaa, Oldehinkel, Verhulst, and Ormel, 2007; Coie & Kupersmidt, 1983; Henricsson & Rydell, 2004; Krettenauer, Ullrich, Hofmann, and Edelstein, 2003 ; Nelson, Benner, Lane & Smith, 2004 ) Th e knowledge that behavior problems are a concern that w arrants action must be considered within the context of recent initiatives from many professional organizations to promote increased use of evidence guide treatment decisions ( Kratochwill, 2007) However, current research that investigates treatment effectiveness often demonstrates the tendency of children with

PAGE 13

5 both externalizing and internalizing behaviors to respond differently to treatment than those children with only one t ype of behavioral concern (Jensen, et al., 2001; Kazdin & Wassell, 1999; March et al., 2000 ). Additionally, most of our current knowledge of what is effective when working with this population is limited to studies using randomized clinical trials. While such studies may demonstrate interventions that are effective when working with this population, the resources needed to carry them out properly may not be available in typical schools (Kratochwill, 2007) In light of these lines of research, studies are needed that can contribute to our understanding of what may be effective in addressing the needs of these children with co occurring internalizing and externalizing behavioral problems in schools Therefore, s tudying the effectiveness of the PTR intervent ion will contribute to our understanding of how to work with this population in schools That is, a school based study demonstrating that a standardized, manual based approach to an FBA is effective for this population would demonstrate that behavioral principles can be used to address the needs of students with both externalizing and internalizing behavioral concerns.

PAGE 14

6 Chapter II Literature Review Overview This chapter begins with an overview of the literature on behavior problems in youth including their prevalence and relationships with academic achievement, social development and long term outcomes. Subsequently research regarding externalizing behaviors in combination with in ternalizin g behaviors is discussed This is followed by a brief overview of the different approaches to investigating internalizing behavior and a review of behavioral intervention research in the mental health field including school based treatment of ch ildren with behavioral concern s. The chapter concludes with a discussion of positive behavior support and a three tiered service delivery model in schools in which the severity of problems guides decisions about how to best serve the child with an emphasis on a Tier III intervention known as Prevent Teach Reinforce (PTR ; Iovannone, Greenbaum, Wang, Kincaid, & Strain, 2009 ). The potential of PTR to address the needs of children displaying externalizing behaviors with internalizing behaviors serving as a moderator is discussed at the conclusion of the chapter Prevalence of Externalizing Behaviors in Youth Externalizing behaviors in youth are those behaviors that present themselves outwardly and are characterized by the child acting negativel y towards the external environment ( Campbell, Shaw, & Gilliom, 2000 ; Eisenberg, et al., 2001 ). Examples of these types of behavior include disruptive, hyperactive, and aggressive behaviors

PAGE 15

7 ( Hinshaw, 1987 ). These behaviors are different from internalizing behaviors in that while internalizing behaviors are also presented outwardly, they affect the internal, psychological environment of the child more so than the external environment. Examples of internalizing behaviors may include withdrawal and inhibitio n. In 2006, five percent of children were reported by their parents as displaying ( Federal Interagency Forum on Child and Family Statistics, 2008 ). If the prevalen ce of these behaviors is assessed in terms of the Diagnostic and Statistical Manual of Mental Disorders IV TR (DSM IV TR) criteria for behavioral disorders, 9% to 13% of children from ages 9 to 17 meet criteria for diagnosable emotional or behavioral disor ders (American Psychiatric Association, 2000). These maladaptive behaviors manifest in multiple settings, including school. For instance, both ADHD and Conduct Disorder are classified as DSM IV externalizing behavior problems in that they are aggressive, anti social, and hyperactive in nature ( American Psychiatric Association, 2000 ). Attention Deficit Hyperactive Disorder (ADHD) occurs in about 3% 7 % of the school population, while Conduct Disorder (CD) is diagnosed in about 6% 16% of males and 2 9% of f emales in the school population ( American Psychiatric Association, 2000 ). Impact of Externalizing Behaviors on Development The negative impact of externalizing behaviors in schools is extensive. To demonstrate this point, t he following section will briefly discuss the specific negative impact that externalizing behavior problems can have on academics and school experiences, teacher and peer relationships, and long term outcomes for children Academics

PAGE 16

8 Research shows that students with externalizing beh aviors show deficits across multiple academic areas (e.g ., mathematics, reading, and written languag e; Nelson, Benner, Lane & Smith, 2004 ). One factor contributing to this finding is that children with externalizing behaviors ha ve been shown to demonst rate deficits in information processing as well as language deficits. For instance, Brunnekreef, Sonneville, Althaus, Minderaa, Oldehinkel, Verhulst, and Ormel ( 2007 ) compared children identified through the parent report Child Behavior Checklist as displ aying externalizing behavior problems to children with internalizing problems only as well as to children with no behavior problems When comparing the groups on speed and accuracy measures form the Amsterdam Neuropsychological Tasks program, children wit h externalizing behaviors showed significantly worse skills than children with internalizing behaviors only, who did not differ from children with no behavioral issues. Similarly Seguin, Parent, Treblay, and Zelazo (2009) assessed longitudinal data for c hildren involved in the Quebec Longitudinal Study of Child Development. Mothers Preschool Behavior Questionnaire. When the researchers analyzed this behavioral data and data collected from the Peabody Picture Vocabulary Test Revised, they found that physical aggression was related to receptive language deficits even after controlling for other cognitive abilities and environmental factors. In addition to skill def icits in students with externalizing behavior problems is the presence of undesirable behaviors. These students display excessive behaviors such as verbal and physical aggression towards peers, defiance, and in class disruptions All of these behaviors are associated with suspension from school ( Skiba, Peterson, & Williams,

PAGE 17

9 1997 ). When students with frequent conduct problems are removed from the classroom environment, they are placed at an even greater disadvantage as their exposure to the ma terial becomes even more limited. This removal from the class then leads to students falling further behind academically ( Bock, Tapscott, & Savner, 199 8). In addition to the immediate negative effects associated with suspension, long term negative effec ts also can occur. Instead of deterring students from making bad choices, suspension is one of the to p school related reasons for dropping out of school frequent feedb ack from the school environment that they are perceived as unable or unworthy of graduation and are therefore encouraged, indirectly, to dropout ( DeRidder, 1991 ). Therefore, the presence of undesirable external behaviors can be thought of as a catalyst fo r other negative events. The defiant and aggressive nature of students with externalizing behavior also leads to more conflict with and negative attitudes toward teachers than is typical for peers without externalizing behaviors ( Henricsson & Rydell, 2004 ) In fact, teachers cite behavioral issues as one of the major obstacles to teaching ( Carpenter & Nangle, 2002 ; Gould, 2002 ). Teachers who continue to have negative interactions with this type of student may find it difficult to deal with the stress asso ciated with interacting with these students and may respond in a more negative manner, thus making academ ic success more difficult for the student ( Strain, Lamber, Kerr, Stragg, & Lenker, 1983) Peer relationships Peer relationships are yet another area th at can be adversely affected by maladaptive behaviors. Students with externalizing problem behaviors can have difficulty forming and

PAGE 18

10 maintaining peer relationships because they can be obtrusive, hyperactive, aggressive, excessive, and intense ( Coie & Kuper smidt, 1983; Mash & Barkley, 2003 ). Other students may be overwhelmed or unnerved by these actions, and they may avoid interactions with these children, leading them to turn to other deviant peers for friendships. Such friendships can lead to other problems (e.g., juvenile delinquency ; Dishion, French, & Patterson, 1995; Reid, 1993) Furthermore this lack of positive interaction with typical students can inhibit the development of important interpersonal skills ( Jimerson, Egeland, & Teo, 1999 ). Long term outcomes Long term consequences accompany these short term effects of problematic behavior. Behavior problems in preschool are the single best predictor of delinquency in adolescence, gang membership, and adult incarceratio n (Dishion, French, & Patterson, 1995; Reid, 1993) Likewise, these behavior problems in childhood are associated with violence, su bstance abuse, and anxiety in adulthood. These children are more likely to experience divorce, unemployment, and psychiatric illness in adulthood than their same age peers without behavior problems ( Coie & Dodge, 1998; Kazdin, 1985 ) Students with Both Externalizing and Internalizing Behaviors As demonstrated in the research literature, the future prospects for children with externalizing behavio rs are quite troublesome and thus warrant attention. Within this population, however, is a subgroup of students who display both externalizing and internalizing behaviors and warrant additional attention McConaughy and Achenbach (199 4 ) found that 40 to 44% of children who had elevated scores on the Aggressive Behavior syndrome subscale of the Child Behavior Checklist (CBCL) also had elevated

PAGE 19

11 scores on the Anxious/Depr essed syndrome subscale. Delinquent Behavior and Anxious / Depressed subscale scores also were simultaneous ly elevated in 25 to 31% of their sample. When information on these same children was collected through the Teacher Report Form (TRF), co occurring externalizing and internalizing behaviors were present in 23 to 30% o f the sample. Similarly, when using the same general population sample, McConaughy, Skiba, and Russell ( 1993 ) found that of the sample who met the borderline clinical cutpoint scores for externalizing behaviors on the CBCL, 51% also met cutpoint scores fo r internalizing behaviors. For the TRF, 42 to 44% of t he sample had simultaneous borderline clinical cutpoint scores on both the externalizing and internalizing scales. If comorbidity is examined in terms of specific disorders, Oppositional Defiant Disor der has been found to be comorbid with depressive symptoms in 17.2% of children and comorbid with anxiety symptoms in 8.9% of children ( Boylan, Vaillancourt, Boyle, & Szatmari, 2007 ). ADHD has been found to be comorbid with anxiety problems i n 25 33 % of c hildren and comorbid with depressive problems in approximately 23% of children ( Bauermeister, et al. 2007; Jarrett & Ollendick, 2008 ). Research has shown that students with co occurring internalizing and externalizing behavior problems demonstrate worse outcomes than those children with only one type of behavioral concern ( Harrington, Fudge, Rutter, & Pickles, 1991). For instance, Brunnekreef et al. (2007 ) found that children with only externalizing problem behaviors showed significantly poorer performa nce on speed and accuracy tasks on the Amsterdam Neuropsychological Tasks program while children with internalizing problems alone did not differ from a comparison group in their task However, students who had both internalizing and externa lizing problem s showed the lowest proficiency on

PAGE 20

12 the tasks Likewise, Wright (2001 ) found that Separation Anxiety D isorder worsens externalizing disruptive behavior s Other research, however, has shown that internalizing behaviors serve to protect externalizing behavi ors from becoming problematic ( Walker, Lahey, Russo, Christ, McBurnett, Loeber, Stouthamer Loeber, & Green, 1991 ). Sp ecifically, students with both C onduc t D isorder and anxiety have been shown to experience fewer soci al problems than children with Conduct D isorder alone Researchers have concluded that internalizing behaviors may serve as either a protective or exacerbating force lives (Jarret t & Ollendick 2008 ). Furthermore, it has been noted that more research should examine internalizin g and e xternalizing behaviors as inter related entities that influence one another and thus warrant special consideration ( Chase & Eyberg, 2008 ). Treatment and Intervention A majority of research regarding comorbid behavior problems focuses on homotypic comorbidity (i.e., behaviors that are contained to either exclusively internalizing behaviors or exclusively externalizing behaviors) with limited attention focused on treatment outcomes for children with a combination of internalizing and externalizing b ehavior problems ( Chase & Eyberg, 2008 ). The following sections will review the limited research available regarding treatment of children with co occurring internalizing and externalizing behaviors Specifically, research regarding psychopharmacological treatment, family based treatment, and school based treatment will be reviewed

PAGE 21

13 The Importance of Moderators in Treatment When investigating both externalizing and internalizing behaviors, the manner in which researchers choose to describe their samp le varies. Often, researchers examining both types of behavioral issues define their sample as displaying behaviors occur more often than would be expected by chance a Mash & Barkley, 2003, p. 37 ). An additional line of research investigates internalizing behaviors or comorbidity as a can help identify which patients will be most responsive to a specific treatment ( Kraemer, Wilson, Fairburn, Mphil, & Agras 2002). Another way to think of a moderator is as a variable that changes the relationship between the risk factor and the outcome. Additionally, some researchers have chosen to discuss internalizing and externalizing behavior influences as an interaction The term internalizing behaviors can yield an interaction effect, which would indicate that one variable (i.e., internalizing behavior) is serving as a moderator. An equivalent way to interpret this interaction would be to say that comorbidity led to a different pattern of results than would be fo und if only one type of behavior was present. In all, w hile different terms are used throughout the research literature, it is important to note that the se terms represent the same sample conditions Combined M edi cation and Behavioral Studies Several studies have examined how the presence of both internalizing and externalizing disorders impacts treatment with psychotropic medication behavioral

PAGE 22

14 treatments or both T he Multimodal Treatment Study of Children with ADHD (MTA) investigate d the efficacy of medication and behavioral treatments for children diagnosed with ADHD (MTA Cooperative Group, 1999) Information was collected on 579 children ranging in age from 7 to 9 years (80% male, 61% Caucasian, 20% African American, and 19% other). During treatm ent, children were randomly assigned to one of four conditions (i.e., medication management, behavioral treatment, a combination of the two, or a community comparison). Medication management was conducted by titrating medication and adjusting the timing an d dosage based on parent and teacher ratings over the course of treatment. Behavioral treatment consisted of 14 months of individual and group parent training, 4 months of classroom management training for the teacher, and an 8 week summer program for the child. The community comparison was referred to community care resources after an assessment was completed using the Diagnostic Interview Schedule for Children parent report ( DISC P) parent and teacher completed Swanson, Nolan, and Pelhan scale (SNAP) the Social Skills Rating Scale (SSRS) and the Multidimensional Anxiety Scale for Children (MASC). Preliminary results showed that treatment which involved medication management (i.e., combination treatment and medication management alone) w as more effe ctive than those treatments without medication management (i.e., behavior treatment only and community referral) in treating ADHD behaviors. However, when examining improvements in internalizing behaviors, combined treatment (i.e., both behavioral and med ical treatment) was more effective than unimodal treatment (i.e., behavioral treatment only or medical treatment only) Behavioral treatment and medical treatment did not differ in their effectiveness at improving internalizing symptoms.

PAGE 23

15 To further invest igate the role of these internalizing behaviors in treatment for children with ADHD Jens e n et al. (2001) grouped the same children from the MTA study into one of four categories based on data collected from parent reports (i.e., ADHD only, ADHD and anxie ty, ADHD and ODD/CD, and ADHD, ODD/CD, and anxiety). The findings of their study revealed that the presence of certain combinations of behavioral problems was related to differences in effective treatment. More specifically, children with ADHD only as we ll as children with ADHD plus other externalizing behaviors responded best to the medication only treatment; the addition of the behavioral component did not influence treatment positively or negatively Children with ADHD in combination with anxiety responded in a similar, positive direction to either medication or behavioral treatments. Finally, children with all these disorders (i.e., ADHD, anxiety, and ODD/CD) made the most improvements with the combined treatment (i.e., both medication and behavioral treatment). These findings suggest that it is important for practitioners to determine the specific types of behavioral issues that a child is presenting in order to provide the most effective treatment. Parent Training Few studies are available that investigate parent training programs and outcomes for children with both internalizing and externalizing behaviors. However, Kazdin and Wassell (1999) conducted a study examining the effectiveness of cogniti ve proble m solving skills training (PSST) and parent management training (PMT) for famil ies with children referred for Conduct D isorder. Two hundred children ranging in age from 3 to 13 years participated in the study with their parents. A majority of the childre n were Caucasian males. While most of the participants met diagnostic criteria for either

PAGE 24

16 Conduct Disorder or Oppositional Defiant Disorder (71%), others met criteria for dif ferent DSM IV disorders (e.g., Attention Deficit Hyperactivity Disorder and Major Depressive D isorder). Seventy nine percent of the children met comorbidity criteria (i.e., criteria for CD or ODD as well as either another externalizing disorder or an internalizing disorder) Kazdin and Wassell (1991) trained p articipants in cognitiv e problem solving thr ough a structured treatment manual which outlined each session. Cognitive problem solving training consisted of 20 to 25 sessions with the child in which modeling, role playing, corrective feedback, and reinforcement were used to teac h skills. Parents were taught management strategies during 16 individual sessions through practice, feedback, and shaping. Parents and children were brought together to review and practice strategies they had learned. At the conclusion of treatment, th e investigators measured therapeutic change by symptoms. Analyses revealed that comorbidity was associated with less therapeutic change which the researchers attr ibuted to previous research knowledge stating that more severe behavioral problems and greater numbers of symptoms are associated with less therapeutic improvement over time. W hile th e investigators did not specify if certain combinations of behavioral pr oblems were associated with less therapeutic change (i.e., externalizing behaviors combined with internalizing behaviors versus exclusively externalizing behaviors), these results provide support regarding the importance of addressing whether a child ha s c omorbid issues when selecting a parent training program

PAGE 25

17 A parent training study that provides more insight into the role of internalizing behaviors in treatment outcomes is seen in Chase and ) study Sixty four children with Oppositional De fiant Disorder were included in treatment using Parent Child Interaction Therapy. Because 15 of the chil dren in the study had comorbid Separation Anxiety D isorder and an additional 26 children had clinically significant internalizing scores on the Child Behavior Checklist, the researchers were able to investigate the role of internalizing behaviors on treatment outcomes. Therapy lasted for 8 12 sessions. During the first phase of therapy, known as child directed interaction, parents learned skills to pla y with their child and were coached through the use of a one way mirror and a bug in the ear microphone. This phase of therapy lasted until parents had mastered the skills (an average of 5 sessions). Next, parents learned discipline strategies during the parent directed interaction phase of therapy. This phase also continued until parents had mastered the skills taught. Overall, therapy lasted an average of 14 sessions. At the end of therapy, the researchers found that children with comorbid ODD and SAD did not differ in their response to treatment than those children with ODD only. Specifically, treatment significantly reduced ODD symptoms in both groups Additionally, SAD symptoms decreased, with 73% of children no longer meeting diagnostic criteria for SAD at the end of therapy. The results of this study demonstrated that parent training with Parent Child Interaction Therapy is effective in reducing both internalizing and externalizing behaviors problems.

PAGE 26

18 School B ased Treatment While the studies described above demonstrate the importance of identifying co occurring problems (i.e., the presence of both externalizing and internalizing behaviors) and examining how internalizing or externalizing symptoms may serve as a moderator of treatment outcomes it is also important to consider the setting in which treatment occurs Jacob and Coustasse (2008) listed the following factors as reasons why schools are the optimal setting for treatment of students with behavioral and emotional issues : schools are fa miliar to students, so they may not experience the same uneasiness that may arise from visiting other settings; transportation barriers are eliminated by delivering treatment in schools; data can be collected on the student in various ways, from different people, and in varying settings; and cost of care is less expensive in schools than in private and community based settings. In fact, schools are considered the de facto provider of services to children with behavioral and emotional needs, with 70 80% of children with such needs receiving services from school personnel (Burns, Costello, Angold, Tweed, Stangl, Farmer, et al., 1995) Based on this i nformation many interventions that are created for these children are formatted for delivery in the school. T he following section will review school based intervention plans for those children displaying behavior problems. Positive behavior support. One increasingly popular school based intervention known as Positive Behavior S upport (PBS) uses an interconnected system of prevention and intervention strategies throughout the entire school to reduce problematic behaviors. PBS can be used to develop social skills and reduce problematic behavior for students in general education a s well as for students with developmental disabilities, autism and

PAGE 27

19 emotional and behavioral disorders ( Sugai & Horner, 2007; Todd, Horner, Sugai, & Colvin 1999 ). PBS has been shown to promote prosocial behavior s increa se aca demic engaged time, improve academic performance, and decrease office discipline referrals ( Lassen, Steel, & Sailor, 2006 ; Sugai & Horner, 2007 ). The 2006 Florida demonstrated th e following positive outcomes for schools implementing PBS school wide : Average number of office discipline referrals (ODR) decreased 28% after 1 year of implementation and continued to decline for the following two years Out of school suspension (OSS) was 41% lower at schools with high levels of PBS implementation than at schools with lower levels of implementation 20% reduction in in school suspension after 1 year of implementation of PBS PBS schools had higher rates of students achieving L evel 3 on FCAT reading than other schools for the 2004 2005 and 2006 2007 school years. PBS uses three levels of eviden ce based interventions to address behavioral problems, with increasing levels of support at each level. T hat is, as the severity and intensity of the problem increases, so does the intensity of the intervention. Because a ll interventions used in PBS schools are empirically supported, consistent, high standards are created across classrooms ( McIntosh, Horner, & Sugai, 2009 ). At the school wide level, or Tier I, procedures are applicable to all students in all settings. The use of these

PAGE 28

20 standardized processes with a culture of clear expectations is intended to be effective for 100 % of the stu dents Researchers have investigated school wide PBS to assess its effectiveness for all students. Lane, Wehby, Roberston, and Rogers (2007 ) investigated the degree to which varying student profiles impact response to PBS. A total of 178 high school st udents (grades 10 12) were nominated by their English teachers using a modified Systematic Screening for Behavior Disorders scale (SSBD). Teachers nominated one student for each of the following categories: externalizing, internalizing, comorbid, and typi cal. Teachers received training in SW PBS for one year prior to implementing the program at their school. After training, teachers implemented the Tier 1 components of PBS throughout their school. Students selected for the study were monitored through d iscipline referrals, tardiness, GPA, and referrals for additional supports. Effect size data suggested that students responded differently to the program, with students with comorbid problems showing the least responsiveness. Students in the comorbid grou p showed slight decreases in GPA ( ES = 0.12) worsening tardiness ( ES = 0.36) and slight improvement in suspension s ( ES = 0.05) In comparison, students with externalizing only and internalizing only behaviors respectively showed improvements in GPA ( ES = 0 .22, 0.39), tardiness ( ES = 0.17, 0.60), and suspensions ( ES = 0.04, 0.27). This study provides a good example of how a well implemented, low intensity approach may not be effective for students with both externalizing and internalizing behaviors. For these students who do not respond to well implemented programs at Tier I

PAGE 29

21 behavior s upport team s to inform decisions to move to Tier II interventions. At Tier II, groups of students are identified as displaying similar behavior problems and patterns and as needing specific skill development. These group interventions are flexible but sy stematic and include the following core features: continuous availability, rapid access, low effort by teachers, consistency with school wide expectations, implementation by all staff in the school, flexibility based on assessment, and continuous monitorin g ( Hawken, Adolphson, MacLeod, & Schumann, 2009) Examples of common Tier II interventions that have been shown to be effective include Check in Check out (CICO) ( Filter, McKenna, Benedict, Ho rn er, Todd, & Watson, 2007 ) and the Behavior Educat ion Program (BEP ; Hawken & Horner, 2003 ; Hawken, MacLeod, & Rawlings, 2007 ; March & Horner, 2002 ). While all interventions used within the PBS system are empirically based, these interventions may not be effective for those students with unique characteris tics (e.g., students with both externalizing and in ternalizing behaviors). A review of the literature revealed no studies investigating the effectiveness of Tier II interventions for children with both internalizing and externalizing behaviors. When it is decided by educational and human service agencies that a more personalized, problem solving approach (i.e., Tier III interventions) will be used, it is typical for a functional behavior assessment (FBA) to be conducted and an individualized supp ort plan to be developed. This process involves the following essential steps: 1. identify goals of intervention 2. gather relevant information through records reviews, interviews and observations

PAGE 30

22 3. develop summary statements (i.e., statements that describe the relationship 4. generate behavior support plan, 5. implement and monitor outcomes ( Scott, Anderson, Mancil, & Alter, 2009 ). These behavior support plans are identifiable in that they are positive, proactive, educative, and functional. Furthermore, these plans use principles of applied behavior analysis. That is, the interventions used meet the following criteria: the environment is altered to remove the triggering event; new skills are taught to replace probl ematic behavior; and rewards for negative behavior are minimized while clear rewards for appropriate behavior are maximized Finally, strategies are used that enhance the up activities ; Dunlap, Sailor, Horner, & Sugai, 2009 ) Research supports the use of an FBA in developing effective interventions (Blakeslee, Sugai, & Gruba, 1994; Kern, Hilt, & Gresham, 2004 ). Furthermore, the use of FBA in schools is endorsed by the Nat ional Association of School Psychologists, the National Association of State Directors of Education, and the National Institute of Health. Prevent Teach R einforce Despite research supporting the use of FBAs in schools for children with severe behavioral problems, this practice is not a standard process used effectively in most schools (Blood & Neel, 2007). S ome researchers have questioned whether such a process is feasible in a school setting ( Kern, Hilt, & Gresham, 2004 ). Specifically, issues regardin g whether teachers in schools a re able to implement such a process accurately have been posed. Iovannone, et al. (2009) identified three limitations preventing the wide scale application of FBAs in schools: 1) previous research on FBAs

PAGE 31

23 has involved study settings with intense involvement by researchers; therefore these research results do not transfer to the typical setting provided in schools; 2) most research has been limited to single subject studies, without much attention to conduct ing randomized controlled trials; and 3) previous training efforts in FBA implementation for school personnel have not been successful. To address such issues, a tertiary level intervention known as Prevent Teach Reinforce (PTR) was developed ( Iovannone, et al. 2009 ) PTR uses a collaborative and systematic approach to completing an FBA, thus increasing teacher acceptance and standardization of this process. Teachers guide the FBA process through the use of a reader friendly manual as well as from assis tance from a PTR consultant. The teacher manual provides personnel with the steps to be completed as well as background information on each step of the process, directions and activities for each meeting, and homework assignments to be completed by each t eam member outside of the meetings. The PTR consultant serves to gather homework assignments by team decided dates and synthesize the data. This data is then presented to the team for discussion, refinement, and consensus The 5 step process outlined in the manual is as follows: team building, goal setting, PTR assessment, PTR intervention, and PTR evaluation. 1. Team b uilding In the first step, s pecific team members are selected ( with as few members as the teacher and PTR consultant). The team also dec ides at this point how consensus will be reached in future steps as well as the responsibilities of each team member. 2. Goal s etting. The second step of goal setting is comprised of three components. First each team member identifies goals for the student to achieve within 3 domain (i.e., social,

PAGE 32

24 academic, behavioral ). At this poi nt, team members identify both the behaviors they would like to see decrease as well as behaviors they would like to see increase. Next, a consensus on beha viors to be targeted and their operational definitions is reached Finally, a strategy fo r measuring behaviors is developed and data is collected daily throughout baseline and the intervention 3. PTR a ssessment. The third step involved is the fu nctional behavior assessment which involves the assessment of preventative, teaching, and reinforcement variables. Each team member independently answers questions related to these three areas and the PTR consultant synthesizes the information to develop a draft hypothesis based on the data received The purpose of this step is for team members to come to a consensus on hypotheses regarding the antecedents to the behavior, the function of the behavior, and the events that follow the behavior. The specif ic areas addressed are as follows: Prevention The context in whic h the problem behavior occurs is identified In other words, events or circumstances that serve as triggers to the problematic behavior are identified by the team members. Teach At this st age, the goal is to identify an acceptable behavior to replace the maladaptive behavior. The replacement behavior can be functionally equivalent to the problem behavior (i.e., escape attention) or it can be a pro social desired behavio r that is incompati ble with the problematic behavior Reinforce The final stage of the assessment involves identifying ways to change the consequences so that the acceptable behavior is more likely to occur and the maladaptive behavior is less likely to occur. To accomplish this, the reinforcements identified during the functional assessment cannot

PAGE 33

25 follow the problematic behavior. Ins tead, the reinforcement provided is matched to the purpose or function of the problem behavior. That is, if the behavior was to receive attention, the reinforcement for the appropriate behavior must have some way for the child to continue to receive atten tion. 4. PTR Intervention The fourth step involves using the data gathered during the functional behavior assessment to select interventions from a menu provided in the manual. To ensure that the selected interventions align with the hypotheses developed in Step 3, descriptions of each intervention, as well as implementation examples are provided. Information is also provided regarded implementation issues, such as the time required, to ensure that feasible interventions are chosen by the team. In order to reach consensus, members are asked to rank order two to four strategies within each category (i.e., an intervention strategy that prevents problem behavior from occurrin g by addressing the antecedents; an intervention that teaches the student one new s kill or replacement behavior ; and a reinforcement intervention to increase the likelihood that the new appropriate behavior will be repeated ) After the interventions are selected, the PTR consultant assists the team in developing the behavior interventio n plan with specific descriptions of the intervention strategies as well as a task analysis of each intervention. Once the behavior intervention plan is written, a plan is developed to provide training and support for the teacher to ensure fidelity of th e intervention This training is provided by the PTR consultant and involves strategies such as role playing, discussion, and question and answers. Teachers are scored at prior to implementing the intervention using a checklist with all elements of the i ntervention that should be present. Teachers receive a

PAGE 34

26 score below 80% means that th e teacher receives additional training or, if the teacher continues to receive a score below 80%, a decision is made as to whether the plan should be modified or continued. Additional support is provided with up to 12 hours of consultation with the PTR co nsultant and 3 direct observations to ensure fidelity of the intervention. 5. PTR Evaluation The final step involves measuring and evaluating the outcome data through the tool decided in Step 2. At this point, the team determines next steps for the inter vention (i.e., expand, fade, change) To assess the effectiveness of the PTR intervention in a typical school setting for students with severe behavior problems, Iovannone, et al. (2009) recruited 245 students across 65 schools, grades K through 8. Stude nts were randomly assigned to either receive the PTR intervention or to receive the services that would usually be delivered to them at their school. Data were collected on students behavior problem through the Social Skills Rating System and academic engaged time through direct observation When pre and post data were analyzed, it was found that students who had received the PTR intervention had significantly higher social skills scores and academic engaged times than their peers w ho had not received the PTR intervention. Additionally, problem behavior scores for students in the PTR intervention group were significantly lower than those students in the comparison group. Despite the positive results found using the PTR process, ques tions regarding the effectiveness of the intervention for children with both internalizing and externalizing behaviors have not been addressed. Researchers have noted that the lack of research

PAGE 35

27 regarding functional behavior assessment for children with inte rnalizing problems is a shortcoming of the field ( Kern, Hilt, & Gresham, 2004 ). Fu rthermore, s ome researchers behavior is complex and cannot be simplified to proximal antecedents and consequences ( Cone, 1997 ; Evans, 1999 ; Horner, 1995 ; ; Miller Williams, & McCoy, 2004) Therefore, the effectiveness of the PTR process for those children who display both externalizing and internalizing behaviors warrants further attention. Conclusio n Research demonstrating the negative effects of externalizing behaviors is well documented. Furthermore, those children with both externalizing and internalizing behaviors appear to represent a population with specific treatment needs Research within t he menta l health field shows that children with both externalizing and internalizing behaviors may require treatment that differs from treatment for children with only one type of behavioral problem ( Jensen et al, 2001 ; Kazdin & Wassell, 1999 ; MTA Cooperative Group, 1999 ) Furthermore, school based research documenting the effectiveness of interventions for children with comorbid internalizing and externalizing behavior problems is limited. However, the limited research that does exist has demonst rated the need to identify and consider the unique characteristics of the student ( Lane, Wehby, Roberston, & Rogers, 2007 ). Within the school setting, services are often delivered in a three tiered model in which increasing problem severity results in more intensive services. For those students who have severe behavioral problems, f unctional behavior assessment is often used as a T ier 3 intervention ( Scott, Anderson, Mancil, & Alter, 2009) Prevent Teach Reinforce

PAGE 36

28 wa s developed to address concerns regardi ng the feasibility of conducting functional behavior assessments within schools (Kern, Hilt, & Gresham, 2004; Kincaid & Iovannone, 2008) However, the effectiveness of PTR for those children with both externalizing and internalizing behaviors has yet to b e addressed in the research differing treatment needs as well as research that states that functional behavior assessments with this type of population may require more consideration than ju st proximal variables, further study of this topic is needed ( Cone, 1997; Evans, 1999 ; The present study will therefore investigate the effectiveness of the Prevent Teach Reinforce intervention for children displaying both externalizing and internal izing behaviors.

PAGE 37

29 CHAPTER III Methods Study Design This study used archival data from a study investigating the effectiveness of the Prevent Teach Reinforce intervention for students with challenging behaviors in grades K through 8 The original study was conducted by researchers at the Florida Mental Health Institute (FMHI) to investigate the effectiveness of PTR for children with externalizing behavioral problems. After receiving approval from the university Institutional Review Board to conduct the original study, three school districts in Central Florida and two in Colorado agreed to participate in the study. District personnel recommended potential schools to be contacted. Project staff contacted the principals of the recommended schools, described the study, and scheduled a time to present to the faculty if principals indicated an interest in participating. After providing overviews of the project to faculty, teachers indicating interest in volunteering received further explanation of the research and signed informed consent. Participants were selected from 65 schools across fiv e public school districts. Three school districts were located in Central Florida, and two were located in Colorado. The number of students served by each school ranged from 20,500 to 194,000.

PAGE 38

30 Teachers who volunteered to participate in the study were aske d to nominate students in their classrooms who engaged in severe behavior problems that were disruptive to the school environment and/or dangerous to themselves and others through the use of the Systematic Screening for Behavior Disorders (SSBD). Unrespon siveness to tier 1 and tier 2 interventions was not a requirement for the students who were nominated. The Systematic Screening for Behavior Disorders ( SSBD ) is a multiple gating tool used to identify students with behavioral problems (Walker & Severson, 1991 ) The first gate requires teachers to rank order students with internalizing and externalizing behaviors Students who were rank ordered in the top three positions on Gate 1 move d on to Gate 2 in which teachers rate d behavioral problems through the Critical Events Inventory (CEI). Possible scores on the Critical Events Inventory range from 0 (i.e., no observable problematic behaviors) to 35 (i.e., 35 types of observable problematic behaviors). The caregivers of each student who was rank ordered number one on Gate 1 and who had a minimum of five critical events on Gate 2 was contacted by the teacher to ascertain whether the family would be interested in the project and would allow the project staff to contact them to provide further explanation. Each family agreeing to be contacted received a visit from a PTR consultant who described the study and attempted to obtain informed consent. If the parent gave consent, the student was randomly assigned to the intervention or wait list comparison group. If the parent did not give sought.

PAGE 39

31 Recruitment consisted of 2 waves. During the 2005 2006 school year, 100 students were recruited for the study, with 50 being ran domly assigned to the treatment condition and 50 students serving as the control group. The following school year (2006 2007), the control group from the previous year received the treatment, and an additional 100 students were recruited for the second wa ve (50 students for the treatment group and 50 students for the control group). During the 2007 2008 school year, the second control group received the PTR intervention. A cademic engaged time, social skills, and problem behaviors data were collected by trained graduate students at three points in time: pre intervention, post intervention, and at follow up On average, 71 days passed between baseline assessment and posttest assessment, and follow up assessment occurred 6 to 8 m onths after posttest asses sment, which typically was the following school year with a different teacher than the teacher involved in the original PTR process. Current Study The current study assessed the effectiveness of the Prevent Teach Reinforce intervention for children with bo th externalizing and internalizing behavior problems compared to those students who have only have externalizing behavior problems. The following research questions were investigated: 1. Do children with externalizing behavior problems only show improvement in social skills, problem behaviors, and academic engaged time as a result of the PTR intervention?

PAGE 40

32 2. Do children with a combination of externalizing and internalizing behavior problems show improvement in social skills, problem behav iors, and academic engag ed time as a result of the PTR intervention? 3. Is there a difference in levels of improvement in social skills, problem behaviors, and academic engaged time as a result of the PTR intervention for children with externalizing behavior problems only vs. those with a combination of internalizing and externalizing behavior problems? Participants Participants for the current study included 74 students from kindergarten through eighth grade who received the PTR intervention All participants were described by their teacher s as engaging in severe behavior problems that were disruptive to the school environment and/or dangerous to themselves and others as measured by the SSBD Furthermore, these behaviors occurred with a frequency of at least one incident per week a nd were sustained for at least six (6) months. Instrumentation Type of Behavioral Problem at three points in time (i.e., pre intervention, post intervention, and follow up) using the Social Skills Rating System (SSRS; Gresham & Elliot, 1990). The SSRS uses three separate rating forms for teachers, parents, and students to measure three domains: social skills, problem behaviors, and academic competence. Additionally, there are three forms for different ages: preschool (3 5 years), elementary (kindergarten grade 6), and secondary (grades 7 12). The specific number of items varies from form to form. Each

PAGE 41

33 form takes up to 20 minutes to complete, with the rater stating their perception of the frequency of behaviors occurring using a 3 For the current study, the Problem Behaviors Subscale was used to define the using the teacher fo rm elementary grade level form which has 57 items Within the Problem Behaviors Subscale, three domains are assessed: externalizing problems, internalizing problems, and hyperactivity problems. Examples As outlined in the SSRS manual, i ndividual items on the Problem Behaviors Scale are classified as internalizing behaviors or externalizing behaviors. Individual items representing internalizing behaviors are added together to get a raw score for internalizing behaviors. The same procedure is followed for externalizing items. Appendices in the manual are then used to convert raw scores for both the internalizing items and the externalizing items into des criptive behavior levels, based on the specific fewer fewer problematic behaviors than the standardization s ample (i.e., one standard deviation or more below the mean of the standardization sample) while a score of displays more problematic behaviors than the average of the standardization sample (i.e., one standard deviation or mo re above the mean of the standardization sample). For the purposes of this study, at the time of the pre intervention assessment, participants with a on both the externalizing and internalizing scales were classified as

PAGE 42

34 having both externalizing and internalizing behavior problems. Participants with a score on only the externalizing scale were classified as having externalizing behavior problems only. For the Problem Behaviors Subscale, the elementary level, teacher endor sed form has consistency reliabilities ranging from .78 to .88 (internalizing and externalizing, respectively). The validity of the Problem Behaviors Subscale score has been established by correlating the externalizing and internalizing scales with the co rresponding externalizing and internalizing scale of the Child Behavior Checklist Teacher From. Validity scores are .75 for the externalizing scale and .59 for the internalizing scale. S ocial Skills teachers at three points in time (i.e., pre intervention, post intervention, and follow up). S was defined in terms of scores obtained on the Social Skills subscale of the SSRS. Example statements on this subscale inc introduces herself or himself to Items endorsed by teachers on this subscale are added to obtain a total raw score. Appendices in the manual are then used to convert the total raw score into a The standard scores have a mean of 100 and a standard deviation of 15. Children with social skills scores below 85 are classified as having fewer so cial skills than the standardization sample while those with a score above 115 are classified as having more social skills than the standardization sample The teacher form, Social Skills subscale of the SSRS has an internal consistency of .94. Negative correlations between the Social Skills subscale and the Problem subscale

PAGE 43

35 of the CBCL teacher forms demonstrate the validity of the scale (i.e., total scale scores correlation of .64.). Behavioral P roblem intervention, post intervention, and follow up). Behavior problems was defined in terms of scores obtained on the Problem Beha viors subscale of the SSRS. was assessed using this scale, the independent variable look ed specifically at whether the behavior was externalizing or internalizing (i.e., to classify participan ts into either the externalizing only or both the externalizing and internalizing group). The examined the overall standard score on the Behavioral Problems scale to assess whether problematic behaviors increased, decr eased or stayed the same following the PTR intervention. S tandard s cores on this subscale are calculated and reported the same as those on the Social Skills subscale (i.e., converting raw scores; scores with a mean of 100 and a standard deviation of 15). Academic Engaged Time was assessed using a modified version of the academic engaged time ( AET ) measure from the SSBD ( Walker & Severson, 1991 ; Iovannone, et al., 2009) The AET measures the amount of time a stu dent is actively engaged during independent instruction. To calculate academic engaged time, an observer uses a stopwatch to record the amount of time the student is actively engaged during two separate 15 minute intervals and then divides this time by

PAGE 44

36 th e total length of the observation. Data for academic engaged time was collected pre intervention, post intervention, and at follow up. The validity and reliability of the AET are dependent on the individuals performing the observations. Data collector s were trained by the project director and data coordinator and instructed on definitions of examples and non examples of academic engagement. After receiving instruction, data collectors practiced with examples on a DVD and compared and discussed their responses to one another as well as the answer key to the DVD. Once inter rater agreement was established, the data collectors were permitted to conduct observations for the purpose of the study. Inter r ater reliability was periodically checked throughout data collection, with 20% of observations being checked. I nter rater reliability for these observations ranged from .93 to .99. Procedure To conduct the study, the following steps were followed: 1. Approva l was obtained from the Institutional Review Board (IRB). 2. T he data set was obtained from Dr. Rose Iovan none at FMHI. 3. P articipants were type of behavioral That is, participants with on both the externalizing and internalizing scales of the SSRS were classified as having both externalizing and internalizing behavior problems. Participants with on the externalizing scale of the SSRS only were classified as having e xternalizing behavior problems only.

PAGE 45

37 Data Analys e s Question 1 : Do children with only externalizing behavior problems show improvement in social skills, problem behaviors, and academic engaged time as a result of the PTR intervention? A series of repeated measures ANOVA s was run for each dependent variable to determine if there were differences in the dependent variable means (i.e., social skills scores, teacher rated behavior problem scores, and percentage of time academically engaged) from pre te st to post test for students with only externalizing behavior problems The significance level for these analyses was set at .0 1 Question 2 : Do children with a combination of externalizing and internalizing behavior problems show improvement in social sk ills, problem behaviors, and academic engaged time as a result of the PTR intervention? A series of repeated measures ANOVA s was run for each dependent variable to determine if there were differences in the dependent variable means (i.e., social skills sc ores, problem behavior scores, and percentage of time academically engaged) from pre test to post test for students with both externalizing and internalizing behaviors Again, t he significance level w as set at .0 1 Question 3 : Is there a difference in lev els of improvement in social skills, problem behaviors, and academic engaged time as a result of the PTR intervention for children with externalizing behaviors only vs. those with a combination of internalizing and externalizing behavior problems? A serie s of repeated measures ANOVA s was run for each dependent variable to determine whether the level of change between groups was significant (i.e., whether there was a significant interaction effect). The significance level was set at .05.

PAGE 46

38 Chapter IV Results The research questions posed in this study were answered through a series of ANOVAs that examin ed change s in social skills, academic engaged time, and behavior problems following the PTR intervention for students with externalizing behavior problem s only and for students with both externalizing and internalizing behavior problems. Th is chapter describes the results of these analyses. Before running ANOVAs, descriptive statistics were run in order to obtain a more complete understanding of the sampl e. Percentages were calculated in order to better understand the distribution of ethnicity, gender, socio economic status, and education placement in the sample. Additionally, the average age of students in each group was calculated. Table 1 provides an overview of these demographic characteristics, sorted by their behavioral profile.

PAGE 47

39 Table 1 Demogr aphic Characteristics of Sample _____________________________________________________ __________________ Demographic Overall Ext Only Int Only Comorbid Neither Variable Sample ( n=74 ) ( n = 39 ) ( n =2 ) ( n = 17 ) ( n =16 ) Age* 8.04 7.83 11.00 8.44 7.75 Male 76.62 73.17 100.0 77.78 81.25 White 57.14 56.10 100.0 50.00 62.50 Black 15.58 17.07 0.0 22.22 6.25 Hispanic 23.38 24.39 0.0 22.22 25.00 Other 3.90 2.44 0.0 5.56 6.25 Free/ Reduced 48.05 43.90 50.0 50.00 56.25 Lunch Special Education 54.55 46.34 100.0 50.00 75.00 Placement Note: N umbers reported as percentages *Numbers reported as means Overall, a review of these data suggest s several points. First, when compared to the overall distribution of the sample, black students appear to be over represented in the co morbid group and under lizing These results, however, are likely due to the small number of students in the t is important to note that students with only internali zing behavior problems were not included in additional analyses because the sample size was too small to analyze once students with incomplete data were removed from the original, archival dataset

PAGE 48

40 Mean scores across pre post and follow up assessment for both students with only externalizing behaviors and students with both externalizing and internalizing behaviors are reported in Table s 2 and 3 respectively Review of the data for academic engaged time shows higher AET rates for both groups of students at post assessment, but both groups show a return to AET rates similar to baseline when assessed at follow up. When looking at social skills and behavior problems, both groups show gains at post assessment, with these improvements sustaining over time until follow up assessment These results suggest an improvement in academic engaged time, social skills, and behavior problems for both groups of students. However, these improvements appear to only sustain over time for social skills and behavior problems. Table 2 Mean Pre, Post and Follow up Scores for Students with Only Externalizing Problems _____________________________________________________ __________________ Outcome Variable Pre SD Post SD FU SD AET 0.46 0.22 0.62 0.18 0.45 0.22 Social Skills 77.13 12.49 86.41 11.89 85.95 11.60 Behavior Problems 125.49 7.42 119.02 10.94 118.31 12.41 Table 3 Mean Pre, Post and Follow up Scores for Students with Comorbid Problems _____________________________________________________ __________________ Outcome Variable Pre SD Post SD FU SD AET 0.50 0.24 0.59 0.19 0.50 0.24 Social Skills 75.35 9.97 86.18 15.49 84.59 8.69 Behavior Problems 132.59 7.60 126.06 11.37 121.94 9.51 _______________________________________________________________________

PAGE 49

41 Outcome Variable 1: Academic Engaged Time To determine if there was a significant difference in academic engaged time across time, data were subjected to a one way repeated measures analysis of variance (ANOVA). The results of this analysis for students with externalizing behaviors only are reported in Table 4 As is shown, a significant effect was observed from over time for students with externalizing behavior problems, F ( 2, 76 ) = 11.62 p <.0001. Post hoc Geisser correction for df(error ) revealed that baseline AET scores w ere significantly lower than the post treatment AET scores, F (1, 38) = 25.09, p <.001. Follow up AET scores were significantly lower than post treatment scores, F (1, 38)= 15.05, p <.001 and similar to baseline scores, F (1, 38)= 0.01, p = 0.91. These results indicate that academic engaged time improved for students after receiving the PTR intervention, but after 6 months, academic engaged time returned to rates similar to baseline. Table 4 AET ANOVA f or Students with Externalizing Behaviors by Time _____________________________________________________ ___________________ Source df SS MS F Treatment 2 0.71 0.36 11.62 Residual 76 2.34 0.03 Total 77 Results of a one way ANOVA for students with both externalizing and internalizing behaviors are reported in Table 5 A significant effect was not found over

PAGE 50

42 time F ( 2, 32 ) = 2.33 p > .05. The se results indicate that students did not improve their academic engaged time after receiving the PTR intervention. Table 5 AET ANOVA for Students with Both Externalizing and Internalizing Behaviors by Time ________________________ _____________________________ ___________________ Source df SS MS F Treatment 2 0. 10 0. 05 2.33 Residual 3 2 0.69 0. 02 Total 33 Table 6 displays ANOVA results for the interaction between time and group membership. Results of these analyses show that while there was a significant change in academic engaged time over time, F ( 2, 108 )= 9.52 p <.001, the mean scores for each group were not significantly different, F (1, 54 )= 0.16 p =0.6 9 Additionally, these analyses demonstrate a non significant difference in improvement over time between the two groups of students, F ( 2, 108 )= 0.49 p = 0. 48 This means that both grou ps of students displayed similar rates of change in academic engaged time M eans for this analysis are shown in Figure 1.

PAGE 51

43 Table 6 AET ANOVA with Time and Group Membership and their I nteraction ________________________________________________________________________ Source df SS MS F Group 1 0.01 0.01 0.16 Error (Group) 54 4.32 0.08 Time 2 0.53 0.27 9.52 Error (Time) 108 3.03 0.03 Time*Group 2 0.04 0.02 0.72 Total 55 Figure 1 Mean Pre, Post, and Follow up Academic Engaged Time Scores 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 Pre Intervention Post Intervention Follow up Externalizing Only Comorbid

PAGE 52

44 Outcome Variable 2: Social Skills To determine if there was a significant difference in social skills scores across time, data were subjected to a one way repeated measures analysis of variance (ANOVA). The results of this analysis for students with externalizing behaviors only are reported in Table 7 As is shown, a significant effect was observed from over t ime for students with externalizing behavior problems, F (2, 76) = 1 0.96 p <.0001. Post hoc Geisser correction for df(error) revealed that baseline social skills scores were significan tly lower than the post treatment social skills scores, F (1, 38) = 16.22, p <.001, and significantly lower than follow up social skills scores, F (1, 38) = 12.91, p <.001. Post treatment social skills scores were similar to follow up scores, F (1, 38) = 0.06, p = 0.81. These results indicate social skills scores improved for students after recei ving the PTR intervention, and these improvements were sustained six months later. Table 7 Social Skills ANOVA for Students with Externalizing Behaviors by Time _____ ________________________________________________ ___________________ Source df SS MS F Treatment 2 2134.22 1067.11 10.96* Residual 76 7401.11 97.38 Total 38 Results of a one way ANOVA for students with both externalizing and internalizing behaviors are reported in Table 8 As is shown, a significant effect was observed from over time for students with comorbid behavior problems, F (2, 32) = 7.25,

PAGE 53

45 p <.01 Post Greenhouse Geisser correction for df(error) revealed that there was a significant difference in mean social skills scores between pre intervention and post intervention F (1, 16) = 12.79, p <.001, as well as between pre intervention and follow up F (1, 16)= 14.96, p <.001 Mean scores between post intervention and follow up were not significant F (1, 16) = 0.19, p = 0.67 indicating that students made significant improvements after receiving the intervention and maintained similar levels of improvement in social skills from post intervention and follow up. These results mirror the results of students with externalizing behavior problems only. Table 8 Social Skills ANOVA for Students with Com orbid Behaviors by Time _____________________________________________________ ___________________ Source df SS MS F Treatment 2 1161.45 580.73 7.25* Residual 32 2563.88 80.12 Total 16 Table 9 displays ANOVA results for the interaction between time and group membership. Results of these analyses show that while there was a significant change in scores over time for each group, F ( 2, 108 )= 15.71 p <.001, the mean scores for each group were not s ignificantly different, F ( 1, 54 )= 0.18 p =0. 67 These results also demonstrate a non significant difference in improvement over time between the two groups of students, F ( 2, 108 )= 0.08 p = 0.92 Similar to the results found with academic engaged time both groups of students significantly improved their social skills scores

PAGE 54

46 but the amount of improvement between the two groups was not significantly different. The means for this analysis are shown in Figure 2. Table 9 Social Skills Scores ANOVA with Time and Group Membership and their Interaction ________________________________________________________________________ Source df SS MS F Group 1 44.81 44.81 0.18 Error (Group) 54 1093.17 242.47 Time 2 2898.45 1449.22 15.71 Error (Time) 108 9964.99 92.27 Time*Group 2 15.07 0.08 0.92 Total 55 Figure 2. Mean Pre, Post, and Follow up Social Skills Scores 68 70 72 74 76 78 80 82 84 86 88 Pre Intervention Post Intervention Follow up Externalizing Only Comorbid

PAGE 55

47 Outcome Variable 3: Behavior Problems To determine if there was a significant difference in behavior problem scores across time, data were subjected to a one way repeated measures analysis of variance (ANOVA). The results of this analysis for students with externalizing behaviors only are rep orted in Table 10 As is shown, a significant effect was observed from pre to post assessment for students with externalizing behavior problems, F (2, 76) = 10.19 p <.001. Post Ge isser correction for df(error) revealed that baseline behavior problem scores were significantly higher than the post treatment behavior problem scores, F (1, 38) = 15.59, p <.001, and significantly higher than follow up behavior problem scores, F (1, 38) = 15.07, p <.00 1. Post treatment behavior problem scores were similar to follow up scores, F (1, 38) = 0.17, p = 0.69. These results indicate behavior problem scores improved for students after rece iving the PTR intervention, and these improvements were sustained six months later. Table 10 Behavior Problems ANOVA for Students with Externalizing Behaviors by Time _____________________________________________________ ___________________ Source df SS MS F Treatment 2 1219.56 609.78 10.19* Residual 76 4545.78 59.81 Total 38

PAGE 56

48 Results of a one way ANOVA for students with both externalizing and internalizing behaviors are reported in Table 1 1 A significant effect was observed over time, F (2, 32) = 6.39 p <.001. Post test with the Greenhouse Geisser correction for df(error) revealed that baseline behavior problem scores were similar to post treatment behavior problem scores, F (1, 16) = 4.99, p = 0.04, and significa ntly higher than follow up behavior problem scores, F (1, 16) = 16.09, p <.00 1. Post treatment behavior problem scores were similar to f ollow up scores, F (1, 16) = 1.48 p = 0. 24 These results indicate that significant gains regarding behavior problems were not made until 6 months after the intervention had been implemented Table 11 Behavior Problems ANOVA for Students with Comorbid Behaviors by Time _____________________________________________________ ___________________ Source df SS MS F Tre atment 2 980.04 490.02 6.3 9 Residual 32 2453.96 76.69 Total 16 Table 1 2 displays ANOVA results for the interaction between time and group membership. Results of these analyses showed that there was a significant change in scores o ver time for each group, F (2, 108)= 15.52 p <.001 as well as a significant difference between the mean sco res for each group F (1, 54)= 6.78 p <.001. However, these analyses demonstrate a non significant difference in improvement over time between t he two grou ps of students, F (2, 108)= 0.72, p = 0.48 Therefore, while the students with comorbid behavior problems had significantly higher mean scores to begin

PAGE 57

49 with, the rate of improvement over time was equally significant for both groups. The means of this analysis are shown in Figure 3. Table 1 2 Behavior Problems ANOVA with Time and Group Membership and their Interaction ________________________________________________________________________ Source df SS MS F Group 1 1245.86 1245.86 6.78 Er ror (Group) 54 9927.29 183.84 Time 2 2012.42 1006.21 15.52 Error (Time) 108 6999.74 64.81 Time*Group 2 93.08 46.54 0.49 Total 55 Figure 3. Mean Pre, Post, and Follow up Behavior Problem Scores 110 115 120 125 130 135 Pre Intervention Post Intervention Follow up Externalizing Only Comorbid

PAGE 58

50 Chapter V Discussion The purpose of this study was to investigate the effectiveness of a Tier III intervention involving functional behavior assessment called Prevent Teach Reinforce (PTR) for children with varying behavioral concerns. Specifically, this study examined whethe r the presence of internalizing behaviors served as a moderator to improvements in social skills, behavior problems, and academic engaged time for children with externalizing behavior problems who received the intervention Such a study was pursued first to contribute to the research literature regarding effective treatments for children displaying comorbid externalizing and internalizing behavioral problems within schools Additionally, the study aimed to contribute to understanding the effectiveness of functional behavior assessment for students with internalizing behaviors as limited research has investigated this topic. The current study used archival data gathered usi ng the Social Skills Rating System (SSRS; Gresham & Elliot, 1990) both to define and categorize the behavioral characteristics of the children as well as to measure improvement in social skills, behavior problems, and academic engaged time. The following research questions were addressed :

PAGE 59

51 Question 1 : Do children with only externalizing behavior problems show improvement in social skills, problem behaviors, and academic engaged time as a result of the PTR intervention? Question 2 : Do children with a combi nation of externalizing and internalizing behavior problems show improvement in social skills, problem behaviors, and academic engaged time as a result of the PTR intervention? Question 3 : Is there a difference in levels of improvement in social skills, problem behaviors, and academic engaged time as a result of the PTR intervention for children with externalizing behaviors only vs. those with a combination of internalizing and externalizing behavior problems? Major Findings of the Study Results of the current study indicate that internalizing behaviors did not serve as a moderator in treatment effectiveness. Specifically, both groups of students made similar significant improvements on all three outcome measures (i.e., social skills, aca demic engaged time, and behavior problems). These results add to the mixed findings that exist within this field of study. Specifically, some researchers have found results similar to those of the current study (i.e., similar clinically significant impr ovement over time for children with only externalizing problems compared to those with comorbid externalizing and internalizing behavior problems; Chase & Eyberg, 2008) Other researchers, however, have shown that internalizing behaviors d o moderate the effectiveness of treatment for children with externalizing behavior problems ( Jensen et al., 2001; K azdin & Wassell, 1999). A possible conclusion to draw from these mixed findings may be that the presence of internalizing behaviors does not u niversally serve as a moderator to

PAGE 60

52 treatment effectiveness. In other words, one cannot generalize findings from one study and say that because internalizing behaviors affected treatment outcomes in one intervention that this will be the same outcome in an other intervention. Contributions to the Literature This study has contributed to the literature on the effectiveness of a specific Tier III intervention for students with comorbid internalizing and externalizing behavior problems in several ways First investigation of the characteristics of the original study sample showed that 26 % of children identified by their teachers as having externalizing concerns also manifested significant levels of internalizing concerns When this study was first proposed there was some concern about whether any students would emerge as having clinically significant comorbid behaviors or as having clinically significant internalizing behaviors only. Analysis of the characteristics of the sample revealed that a substantial students with only internalizing behavior problems was not large enough to allow statistical analyse s to be conducted, students did emerge demonstrating this profile. This is particularly interesting given that teachers were asked only to identify students with significant externalizing problems. While the specific behavioral profiles of the students di d not impact the effectiveness of this particular intervention, conducting this study did show that unique profiles emerge d among students and therefore, future researchers and practitioners should be mindful of whether the profiles of their population mat ch the intervention being used.

PAGE 61

53 Secondly, the questions addressed in this study align with current initiatives aimed at increasing the use of evidence based interventions with student populations ( Kratochwill, 2007 ) The results of the current study, along with previous research which (Iovannone, et al., 2009) Along these same lines, the findings of this study are important for understanding what is effective when working with children with unique behavioral profiles (i.e., comorbid externalizing and internalizing behavior problems), as previous studies have shown that children with co morbid problems sometimes require interventions that differ from children with only one type of behavioral problem (Jensen et al., 2001; Kazdin & Wassell, 1999) Finally, the finding that significant improvements were shown after implementing the PTR int ervention for students with comorbid externalizing and internalizing behavior problems provides support for the use of FBA with children with internalizing problems, an area of research that has received limited attention and some have speculated would be ineffective because of the lack of consideration of distal variables (e.g., family dysfunction ; Cone, 1997; Evans, 1999 ; ). These results, h owever, should be interpret ed with caution as there was no analysis done in the current study that looked at students with internalizing behavior problems only. Limitations Some limitations should be noted for this study. One important limitation to this study is the lack of a control group. Without a control group for comparison, it cannot be

PAGE 62

54 stated that improvements after receiving the PTR intervention are due to the intervention itself as opposed to other factors Howe ver, Iovannone, et al (2009) found significantly better outcomes for students receiving the PTR intervention over those students who d id not receive the intervention when using data from the sample used for the current study Also the use of archival d ata meant that the sample size was limited. After students were divided by their specific behavior problems, some groups had samples sizes that were too small to allow for statistical analyses. Specifically, there was no group of students available for a nalysis which represented internalizing behavior problems only. The addition of such a group would further clarify what types of behavioral concerns can be addre ssed using the PTR intervention. That is, f rom the results of the current study, it cannot be stated that PTR would be effective for students with only internalizing behavior problems Previous research has shown that interventions that are effective for externalizing problems and for comorbid externalizing and internalizing problems may not be t he most effective intervention for students with internalizing problems only ( Jensen et al., 2001 ). Also, limited sample sizes may have contributed to a lack of significant results when examining the academic engaged time scores for students wi th comorbid behavior problems. Another limitation that arose as a result of using archival data is related to the tools used to classify behavior problems. Because the original study used the SSRS as a collection tool, this was the de facto tool used to classify st udents as displaying clinically significant externalizing and/ or internalizing behaviors. However, there are other measures of internalizing and externalizing behaviors (e.g., the CBCL) that are used more

PAGE 63

55 frequently in the literature to identify various t ypes of behavior problems in children. Use of another measure may have classified children in the current study differently. Next, it is important to note that, although significant improvements were made in all three measures, neither group of students m oved into ranges of behavior considered would still have significant difficulties with these students, despite effectively implementing the PTR interventions. Also, b ecause data were only collected from schools where administrators agreed to participate it is possible that these schools differed from those where administrators chose not to participate. Specifically, schools that had more buy in from the administratio n (as indicated by their agreement to participate in the study) may have also had more support from administration and buy in from the staff responsible for implementing the PTR process. Factors such as these (i.e., buy in and support) are known to influe nce the degree to which a new initiative (i.e., the PTR intervention) is successful (Hall & Hord, 2006). Therefore, it is uncertain whether the same results would have been found at other schools. Next it is important to note the sample characteristics of the current study. A majority of the students included in this study were male, so based on this study, it is still unclear how beneficial this intervention would be for females. Also, most students in the current study were white. Therefore, the same results may not emerge when using this process with more ethnically diverse populations. Another limitation that should be noted relates to those students who did not meet criteria for inclusion in the current study. Students were originally identified by teachers

PAGE 64

56 as displaying severe behavior problems and were therefore included in the original PTR study. However, when SSRS cutoff scores were assessed for the current study many students did not have clinically significant behavior problems. Therefore, data for these students were not analyzed. It would be important, however, to understand if the PTR intervention resulted in different levels of improvement for these students with less severe behavior problems. Finally, the current study only assessed three specific constructs (i.e., social skills, academic engaged time, and behavior problems). As such, it is unclear from this stud6y what type of affect PTR would have on other constructs worthy of inv estigation (e.g., suspension, grades, scores on achievement tests). This is especially noteworthy as the MTA study (1999) found that certain interventions improved some areas but not others. Directions for Future Research Based on the results of the curr ent study as well as the aforementioned limitations, some recommendations for future research should be noted Many of these recommendations for future research are directly related to sample characteristics. First, in the current study future research assessing PTR effectiveness should address this by purposefully identifying and including such a group in the research design. This is important as previous research has shown that inter ventions that are effective for externalizing problems and for comorbid externalizing and internalizing problems may not be the best intervention for students with internalizing problems only ( Jensen et al., 2001) This research would not only contribute to knowledge of PTR and its effectiveness, it also would add research to the understudied area of FBAs and their use with internalizing

PAGE 65

57 behavior issues. Next, future research should examine the effectiveness of the PTR intervention with groups with varying demographic characteristics. Specifically, research well as its effectiveness for females. This is an especially important line of researc h as school personnel are increasingly being expected to use approaches that have been supported empirically ( Kratockwill, & Shernoff, 2004). Additionally, it is important that future research attempt to replicate the results of the current study while mak ing some adjustments to the research design. First, future research should use other measures of internalizing and externalizing behaviors to classify students. This is important as it is unclear whether the tool used for the current study was the most a ccurate measure of externalizing and internalizing behavior problems. Also, f uture research sh ould investigate the impact of PTR on other variables (e.g., suspension and grade retention) Also it would be beneficial to know whether PTR only improves ext ernalizing type behavioral issues (e.g., decrease in fighting), or if it also positively affects outcomes more closely associated with internalizing problems (e.g., decrease in social withdrawal). Implications for the Field of School Psychology The results of the current study have important implications for the field of school psychology. First, b ecause of the applied nature of this study, these results can be transferred easily into school based practice. That is, the results of this study hel p inform practitioners of an intervention which is supported by research that they can use with children with severe behavioral concerns This is especially important, given that school

PAGE 66

58 personnel are increasingly being held to standards which include the u se of evidence based practices. Furthermore the current study identifies, more specifically than previous research, the types of students who may benefit from an FBA based process. As noted previously, using FBAs routinely and effectively in schools for students with severe behavior problems is not a common practice (Blood & Neel, 2007). The findings from this study not only reiterate the effectiveness of FBAs with this population o f students and therefore the importance of using such an approach, but expand on previous research by identifying specific groups of students who can benefit from this process. For instance, t he finding that internalizing behaviors did not serve as a mode rator of treatment outcomes suggests that practitioners can use such a program with children with severe comorbid behavioral concerns Also in reviewing the demographic characteristics of the sample, it was noted that roughly half of the students in each group received free or reduced price lunch. This information is important when it is considered within the context of the current study results. That is, the results of the current study further helps practitioners understand what strategies are effecti ve in working with economically diverse populations. Conclusion The negative impact of behavioral issues is well documented in the research literature. As such, research investigating effective strategies for working with students with behavioral issues is important, especially for those students who demonstrate comorbid issues which may prevent these students from responding to certain strategies. The current study found support for the use of the Prevent Teach Reinforce intervention

PAGE 67

59 for children with v arying behavioral profiles. Significant improvements were found in social skills, behavioral problems, and academic engaged time for students with only externalizing behavior problems as well as significant improvements in social skills and behavior proble ms for students with both externalizing and internalizing behavior problems. Additionally, these improvements were similar for both groups, demonstrating that PTR is a process that can be used in an equally effective way for both populations in a typical school setting

PAGE 68

60 References Achenbach, T.M. (1991). Burlington, VT: University of Vermont Department of Psychiatry. American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4 th ed.). Washington, DC. Author. Bauermeister, J.J., Shrout, P.E., Ramirez, R., Bravo, M., Alegria, M., Martinez Taboas, A., Chavez, L., Rubio Stipec, M., Garcia, P., Ribera, J.C., and Canino, G. (2007). ADHD correlates, comorbidity, and impairment in community and treated samples of children and adolescents. Journal of Abnormal Child Psychology, 35 (6), 883 898. Blakeslee, T., Sugai, G., & Gruba, J. (1994). A review of functional asse ssment use in data based intervention studies. Journal of Behavioral Education, 4 (4), 397 413. Blood, E, & Neel, R.S. (2007). From FBA to implementation: A look at what is actually delivered. Education and Treatment of Children, 30 67 80. Bock, S.J., Ta pscott, K.E., & Savner, J.L. (1998). Suspension and expulsion: effective management for students? Intervention in School and Clinic, 34 (1), 50 52. Boylan, K., Vaillancourt, T., Boyle, M., & Szatm ari, P. ( 2007). Comorbidity of internalizing disorders in c hildren with oppositional defiant disorder. European Child and Adolescent Psychiatry, 16 (8), 484 494.

PAGE 69

61 Brunnekreef, J.A., DeSonneville, L.M., Althaus, M., Minderaa, R.B., Oldehinkel, A.J., Verhulst, F.C., & Ormel, J. (2007). Inf ormation processing profiles of internalizing and externalizing behavior problems: evidence from a population based sample of preadolescents. Journal of Child Psychology and Psychiatry, 48 (2), 185 193. Burns, B.J., Costello, E.J., Angold A., Tweed, D., Stangl, D., Farmer, E.M. et al. (1995). Health Affairs, 14 (3): 147 159. Campbell, S.B., Shaw, D.S., & Gilliom, M. (2000). Early externalizing behavior problems: Toddlers and p reschoolers at risk for later maladjustment. Development and Psychopathology, 12 467 488. Carpenter, E.M., & Nangle, D.W. (2002). Effects of brief verbal instructions on aggression: a replication in a Head Start setting. Child & Family Behavior Therapy, 24 (4), 21 38. Chase, R.M. & Eyberg, S.M. (2008). Clinical presentation and treatment outcome for children with comorbid externalizing and internalizing symptoms. Anxiety Disorders, 22 273 282. Coie, J.K., & Dodge, K.A. (1998). Aggression and antisocial behavior. In W. Damon & N. Eisenberg (Eds.), Handbook of child psychology, 5 th edition, Volume 3, Social, emotional and personality development NY: John Wiley & Sons. Coie J.D. & Kupersmidt, J.B. (1983). A behavioral analysis of emerging social status in Child Development, 54 (6), 1400 1416.

PAGE 70

62 and school wide discipline in general education. Education and Treatment of Children, 16 361 381. Cone, J.D. (1997). Issues in functional analysis in behavioral assessment. Behavior Research and Therapy, 35 (3), 259 275. DeRidder, L.M. (1991). How suspension and expulsion contribute to dropping out. E ducation Digest, 56 44 47. Dishion, T.J., French, D.C., & Patterson, G.R. (1995). The development and ecology of antisocial behavior. In D. Cicchetti & D.J. Cohen (Eds.), Developmental psychopathology, Vol.2: Risk, disorder, and adaptation (pp.421 471). New York: John Wiley & Sons. Dunlap, G., Sailor, W., Horner, R.H., & Sugai, G. (2009). Overview and history of positive behavior support. In W. Sailor, G. Dunlap, G. Sugai, & R. Horner (Eds.), Handbook of Positive Behavior Support (pp. 3 16). New York, NY: Springer. Eisenberg, N., Cumberland, A., Spinard, T.L., Fabes, R.A., Shepard, S.A., Reiser, M., et externalizing and internalizing problem behavior. Child Development, 72 1112 1134. Evans, I.M. (1999). Child focused behavioral assessment and modification. Journal of Clinical Child Psychology, 28 493 501. Federal Interagency Forum on Child and Family Statistics. (2008). brief: key national indicators of well being, 2008 Retrieved August 17, 2008, from http://childstats.gov/pdf/ac2008/ac_08.pdf

PAGE 71

63 Filter, K.J., McKenna, M.K., Benedict, E.A., Horner, R.A., Todd, A.W., & Watson, J. (2007). Check in / check out: a post hoc evaluation of an efficient, secondary level targeted intervention for reducing problem behaviors in schools. Education and Treatment of Children, 30 (1), 69 84. bility in functional assessment and individual program development. Education and Treatment of Children, 18 (4), 499 509. Gresham, F.M. & Elliot, S.N. (1990). Social Skills Rating System: Manua l. Circle Pines, MN: American Guidance Service. Hall, G.E. & Hord, S.M. (2001). Implementing Change: Patterns, Principles, and Potholes (2 nd ed .) Boston, MA: Pearson Education, Inc. Harrington, R., Fudge, H., Rutter, M., & Pickles, M. (1991). Adult outcomes of childhood and adolescent depression: II. Links with an tisocial disorders. Journal of the American Academy of Child and Adolescent Psychiatry, 30 (3), 434 439. Hawken, L.S., Adolphson, S.L., MacLeod, K.S., & Schumann, J. (2009). Secondary tier interventions and supports. In W. Sailor, G. Dunlap, G. Sugai, & R Horner (Eds.), Handbook of Positive Behavior Support (pp. 395 420), New York, NY: Springer. Hawken, L.S. & Horner, R.H. (2003). Implementing a targeted intervention within a school wide system of behavior support. Journal of Behavioral Education, 12 225 240.

PAGE 72

64 Hawken, L.S., MacLeod, K.S., & Rawlings, L. (2007). Effects of the behavior education program (BEP) on office discipline referrals of elementary school students. Journal of Positive Behavior Interventions, 9 (2), 94 101. Haynes, S.N., & Clinical Psychology Review, 10 649 668. Henricsson, L. & Rydell, A.M. (2004). Elementary school children with behavior problems: teacher child relations and self perception. A prospective study. Merrill Palmer Quarterly, 50 (2), 111 138. Hinshaw, S.P. (1987). On the distinction between attentional deficits/hyperactivity and conduct problems/aggression in child psychopathology. Psychological Bulletin, 101 443 463. Iovannone, R., Greenbau m, P.E., Wang, Wei, Kincaid, D., Dunlap, G., & Strain, P. (2009). Randomized controlled trial of the Prevent Teach Reinforce (PTR) tertiary intervention for students with problem behaviors: Preliminary outcomes. Journal of Emotional and Behavioral Disorde rs, 17 (4), 213 225. Jacob, S. & Coustasse, A. (2008). School based mental health: A de facto mental health system for children. Journal of Hospital Marketing & Public Relations, 18 (2), 197 211. Jarrett, M.A., & Ollendick, T.H. (2008). A conceptual review of the comorbidity of attention deficit/hyperactivity disorder and anxiety: Implications for future research and practice. Clinical Psychology Review, 28 (7), 1266 1280.

PAGE 73

65 Jensen, P.S., Hinshaw, S.P., Kraemer, H.C., et al (2001). ADHD comorbidity findings from the MTA study: Comparing comorbid subgroups. Journal of the American Academy of Child and Adolescent Psychiatry, 40 (2), 147 158. Jimerson, S., Egeland, B., & Teo, A. (1999). A longitudinal study of achievement trajectories: Factors associ ated with change. Journal of Educational Psychology, 91 (1), 116 126. Kazdin, A. (1985). Treatment of antisocial in children and adolescents Homewood, IL: Dorsey. Kazdin A.E., & Wassell, G. (1999). Barriers to treatment participation and therapeutic change among children referred for conduct disorder Journal of Clinical Child Psychology, 28 (2), 160 172. Kraemer, H.C., Wilson, G.T., Fairburn, C.G., MPhil, F., & Agras, W.S. (2002). Mediators and moderators of treatment effects in randomized clinical trials. Archives of General Psychiatry, 59 (10), 877 883. Kern, L., Hilt, A.M., & Gresham, F. (2004). An evaluation of the functional behavioral assessment process used with students with or at risk for emotional and behavioral disorders. Education and T reatment of Children, 27 (4), 439 452. Kratockwill, T.R., & Shernoff, E.S. (2004). Evidence based practice: Promoting evidence based interventions in school psychology. School Psychology Review, 33 (1), 34 48. Krettenauer, T., Ullrich, M., Hofmann, V., & E delstein, W. (2003). Behavioral problems in childhood and adolescence as predictors of ego level attainment in early adulthood. Merrill Palmer Quarterly, 49 (2), 125 153.

PAGE 74

66 Lane, K.L., Wehby, J.H., Robertson, E.J., & Rogers, L.A. (2007). How do different types of high school students respond to schoolwide positive behavior support programs? Characteristics and responsiveness of teacher identified students. Journal of Emoti onal and Behavioral Disorders, 15 (1), 3 20. Lassen, S.R., Steele, M.M., & Sailor, W. (2006). The relationship of school wide positive behavior support to academic achievement in an urban middle school. Psychology in the Schools, 43 701 712. Liu, Jianghong. (2004). Childhood externalizing behavior: Theory and i mplications. Journal of Child and Adolescent Psychiatric Nursing, 17 (3), 93 103. March, J.E., Swanson, J.M., Arnold, L.E., Hoza, B., Conners, K., Hinshaw, S.P., Hechtman, L., Kraemer, H.C., Greenhill, L.L., Abikoff, H.B., Elliot, L.G., Jenson, P.S., New corn, J.H., Vitiello, B., Severe, J., Wells, K.C., & Pelham, W.E. (2000). Anxiety as a predictor and outcome variable in the multimodal treatment study of children with ADHD (MTA). Journal of Abnormal Child Psychology, 28 (6), 527 541. March, R.E. & Horne r, R.H. (2002). Feasibility and contributions of functional behavioral assessment in schools. Journal of Emotional and Behavior Disorders, 10 158 170. Mash, E.J., & Barkley, R.A. (Eds.), Child Psychopathology (2 nd ed.). New York: The Guildford Press. McConaughy, S.H., & Achenbach, T. M. (1994). Comorbidity of empirically based syndromes in matched general population and clinical samples. Journal of Child Psychology and Psychiatry, 35 (6), 1141 1157.

PAGE 75

67 McConaughy, S.H. & Skiba, R.J. (1993). Comorbidity o f externalizing and internalizing problems. School Psychology Review, 22 (3), 421 436. McIntosh, K., Horner, R.H., & Sugai, G. (2009). Sustainability of systems level evidence based practices in schools: current knowledge and future directions. In W. Sailor, G. Dulap, G. Sugai, & R. Horner (Eds.), Handbook of Positive Behavior Support (pp. 327 352). New York, NY: Springer. Miller, J.A., Williams, S.J., & McCoy, E.L.B. (2004). Using multimodal functional behavioral assessment to inform treatment se lection for children with either emotional disturbance or social maladjustment. Psychology in the Schools, 41 (8), 867 877. MTA Cooperative Group (1999). A 14 month randomized clinical trail of treatment strategies for attention deficit/hyperactivity diso rder. Multimodal treatment study of children with ADHD. Archives of General Psychiatry, 56 (12), 1073 1086. Nelson, J.R., Benner, G.J., Lane, K., & Smith, B.W. (2004). Academic achievement of K 12 students with emotional behavioral disorders. Exceptional Children, 71 (1), 59 73. Reid, J.B. (1993). Prevention of conduct disorder before and after school entry: Relating interventions to developmental findings. Development and Psychopathology 5, 243 262. Scott, T.M., Anderson, C., Mancil, R., & Alter, P. ( 2009). Function based supports for individual students in school settings. In W. Sailor, G. Dunlap, G. Sugai, & R. Horner (Eds.), Handbook of Positive Behavior Support (pp. 421 441). New York, NY: Springer.

PAGE 76

68 Seguin, J.R., Parent, S., Tremblay, R.E., & Ze lazo, P.D. (2009). Different neurocognitive functions regulating physical aggression and hyperactivity in early childhood. Journal of Child Psychology and Psychiatry, 50 679 687. Skiba, R.J., Peterson, R.L, & Williams, T. (1997). Office referrals and suspension: Disciplinary intervention in middle schools. Education and Treatment of Children 20, 295 315. Strain, P.S., Lambert, D., Kerr, M.M., Stragg, V., & Lenker, D. (1983). Naturalistic compliance. Journal of Applied Behavior Analysis 16, 243 249. Sugai, R., & Horner, R. (2007). SW PBS and RtI: Lessons being learned. OSEP Center on Positive Behavior Interventions and Supports. Retrieved August 23 2008, from http://www.pbis.org/main.htm. Todd, A.W., Horner, R.H., Sugai, G., & Colvin, G. (1999). Individualizing school wide discipline for students with chronic problem behaviors: A team approach. Effective School Practices, 17 72 82. Walker, J.L., Lahey, B.B., Russo, M.F., Frick, P.J., Christ, M.J., McBurnett, K., Loeber, R., Stouthamer Loeber, M., & Green, S.M. (1991). Anxiety, inhibition, and conduct disorder in children: Relations to social impairment. Journal of the American Ac ademy of Child Adolescent Psychiatry, 30 (2), 187 191. Wright, D.J. (2001). The relationship of anxiety and externalizing disorders. Dissertation Abstracts International: The Sciences and Engineering, 62 (4 B), 2084.