|USFDC Home | USF Electronic Theses and Dissertations||| RSS|
This item is only available as the following downloads:
xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam Ka
controlfield tag 001 001945946
007 cr mnu|||uuuuu
008 080715s2007 flu sbm 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0002200
Williams, Jillian Leigh.
Caregivers perceptions of the effectiveness of the Helping Our Toddlers, Developing Our Children's Skills parent training program :
b a pilot study
h [electronic resource] /
by Jillian Leigh Williams.
[Tampa, Fla] :
University of South Florida,
ABSTRACT: This study was designed to evaluate a parent training curriculum: Helping Our Toddlers Developing Our Children's Skills (HOT DOCS¨ ) using archival data collected between August 2006 and April 2007. The evaluation studied the impact of specific components of the parent training program on both participants' knowledge and attitudes and their perceptions of targeted children's behavior. One-hundred-forty-six caregivers of children between the ages of 14 months and ten years of age participated in the parent training program and were included in the analyses. Measures included a pre/post knowledge test, rating scales of child problem behavior, weekly progress monitoring forms for caregiver behavior at home, and a program evaluation survey. Results indicated significant increases in caregiver knowledge following participation in the program. Prior to participation, caregivers' perceptions of the severity of child problem behaviors and deficits in adaptive behaviors were significantly different from a normative sample. Following participation in the program, results showed significant decreases in caregiver perceptions of the severity of child problem behaviors, but no significant differences in child adaptive behaviors. Caregiver feedback indicated high levels of satisfaction with the program.
Thesis (Ed.S.)--University of South Florida, 2007.
Includes bibliographical references.
Text (Electronic thesis) in PDF format.
System requirements: World Wide Web browser and PDF reader.
Mode of access: World Wide Web.
Title from PDF of title page.
Document formatted into pages; contains 151 pages.
Co-adviser: Kathy Bradley-Klug, Ph.D.
Co-adviser: Kathleen Armstrong, Ph.D.
Positive behavior support.
x School Psychology
t USF Electronic Theses and Dissertations.
Caregivers Perceptions of the Effectiveness of the Helping Our Toddlers, Developing Our Childrens Skills Parent Training Program: A Pilot Study by Jillian Leigh Williams A thesis submitted in partial fulfillment of the requirements for the degree of Educational Specialist Department of Psychological and Social Foundations College of Education University of South Florida Co-Major Professor: Kathy Bradley-Klug, Ph.D. Co-Major Professor: Kathleen Armstrong, Ph.D. Constance Hines, Ph.D. Date of Approval: November 2, 2007 Keywords: parent training, positive be havior support, early in tervention, challenging behavior, problem solving Copyright 2007, Jill ian L. Williams
i Table of Contents List of Tables iv List of Figures vii Abstract viii Chapter 1 Statement of the Problem 1 Theoretical Framework 3 Overview of the HOT DOCS Parent Training Program 4 Purpose of the Study 6 Research Questions 6 Significance of the Study 7 Definition of Terms 7 Chapter 2 Overview 9 Prevalence of Young Children with Challenging Behavior 9 Outcomes Associated with Early Emerging Behavior Problems 12 Role of Parenting in Child Behavior Problems 13 Outcomes Associ ated with Early Intervention 20 Parent Training as an Intervention 21 Applying Principles of Po sitive Behavior Support to Parent Training 32 Summary 37 Chapter 3 Introduction 38 Participants 38 Description of Caregivers 39 Description of Target Children 41 Caregivers Not Completing Training (Drop Outs) 43 Description of Caregivers Not Completing Training 43 Description of Target Children of Caregivers Not Completing Training 45 Setting 46 HOT DOCS Parent Training Program 46 Session 1 47 Session 2 47 Session 3 48 Session 4 48 Session 5 49
ii Session 6 49 Measures 50 HOT DOCS Demographics Form 50 HOT DOCS Knowledge Test 50 HOT DOCS Tip Tracker Sheets 51 Child Behavior Checklist 51 Adaptive Behavior Assessment System, 2nd Edition 53 HOT DOCS Program Evaluation Survey 56 Data Collection 56 Data Analysis 58 Caregiver Knowledge 58 Caregiver Perceptions of Severity of Child Behavior 58 Changes in Child Problem & Adaptive Behavior 60 Caregiver Skills at Home 62 Caregivers Overall Perceptions of the HOT DOCS Program 62 Chapter 4 Overview 64 Caregiver Knowledge 64 Research Question #1 64 Caregiver Perceptions of Severity of Child Behavior 65 Research Question #2 65 Research Question #3 68 Changes in Child Problem & Adaptive Behavior 75 Research Question #4 75 Research Question #5 78 Caregiver Skills at Home 82 Research Question #6a 82 Research Question #6b 84 Caregivers Overall Perceptions of the HOT DOCS Program 84 Research Question #7 84 Descriptive Analysis of Quantitative Data 85 Thematic Analysis of Free-Response Data 88 Chapter 5 Overview 100 Demographic Characteristics 100 Rates and Patterns of Caregiver Attendance 100 Comparison of Caregiver Demographics with Hillsborough County Demographics 102 Comparison of Child and Caregiver Demographics with Previous Studies 104 Caregiver Knowledge 107 Caregiver Perceptions of Severity of Child Behavior 108 Changes in Child Problem & Adaptive Behavior 111 Caregiver Skills at Home 114
iii Caregivers Overall Perceptions of the HOT DOCS Program 116 Implications for Practitioners 118 Limitations 120 Directions for Future Research 121 Conclusion 122 References 123 Appendices 141 Appendix A: The HOT DOCS Parent Training Program 142 Appendix B: Relation betw een Research Questions and Variables 143 Appendix C: HOT DOCS Demographic Form (English) 144 Appendix D: HOT DOCS Demographic Form (Spanish Version) 145 Appendix E: HOT DOCS Knowledge Test (English) 146 Appendix F: HOT DOCS Demographic Form (Spanish Version) 147 Appendix G: HOT DOCS Tip Tracker #1 (English) 148 Appendix H: HOT DOCS Tip Tracker #1 (Spanish Version) 149 Appendix I: HOT DOCS Program Evaluation Survey (English) 150 Appendix J: HOT DOCS Program Evaluation Survey (Spanish Version) 151
iv List of Tables Table 1 Attendance Record of Initial Caregiver Sample 39 Table 2 Breakdown of Participant Sample by Gender, Race/Ethnicity, and Education Level 40 Table 3 Breakdown of Participant Sample by SES Indicator 41 Table 4 Relation of Caregiver to Target Child 41 Table 5 Number and Percent of Target Children by Preexisting Diagnosis 42 Table 6 Breakdown of Program Non-Completers by Gender, Race/Ethnicity, and Education Level 44 Table 7 Breakdown of Program NonCompleters by SES Indicator 44 Table 8 Relation of Caregiver to Target Child for Program NonCompleters 45 Table 9 Breakdown of Target Children of Program Non-Completers by Preexisting Diagnosis 45 Table 10 Data Matrix for Two-Fact or Repeated Measures Design for Problem Behavior 61 Table 11 Data Matrix for Two-Fact or Repeated Measures Design for Adaptive Behavior 61 Table 12 Means and Standard Deviati ons for Participant Scores on the Knowledge Test 64 Table 13 Observed and Expected Fr equencies for CBCL Internalizing Subscale T-Scores 66 Table 14 Observed and Expected Fr equencies for CBCL Externalizing Subscale T-Scores 67 Table 15 Observed and Expected Fr equencies for ABAS-II Conceptual Scale Scores 72
v Table 16 Observed and Expected Fre quencies for ABAS-II Social Scale Scores 72 Table 17 Chi Square Goodness of Fit Test for ABAS-II Practical Scale Scores 73 Table 18 Means and Standard Deviat ions of Preand Posttest CBCL Scores by Scale 75 Table 19 Analysis of Variance of CBCL Preand Posttest Scores 76 Table 20 Means and Standard Deviati ons of Preand Posttest ABAS-II Scores by Scale 78 Table 21 Analysis of Variance of AB AS-II Preand Posttest Scores 79 Table 22 Average Daily Parent Ratings of Ease or Difficulty of Skill Use at Home 83 Table 23 Descriptive Statistics and Co rrelations for Weekly Skill Use and Ratings of Difficulty 84 Table 24 Ratings of Particip ant Satisfaction with the HOT DOCS Training Program 87 Table 25 How are you using the information you learned in HOT DOCS? 88 Table 26 Sample Verbatim Responses for Item #1: How Are You Using Information You Learned in HOT DOCS? 89 Table 27 Have you shared the information from HOT DOCS with? Check all that apply. 90 Table 28 If you have shared information from HOT DOCS with others, please describe how they have benefited from this information? 91 Table 29 Sample Verbatim Responses for Item #3: Describe How Others Have Benefited from HOT DOCS Information You Shared. 92 Table 30 What can we do to improve HOT DOCS? 93 Table 31 Sample Verbatim Responses for Item #4: What Can We Do To Improve HOT DOCS? 94 Table 32 What did you value most? 95
vi Table 33 Sample Verbatim Responses to Question #5: What Did You Value Most? 96 Table 34 What suggestions do you have for future HOT DOCS trainings?97 Table 35 Sample Verbatim Responses to Question #6: What Suggestions Do You Have for Future HOT DOCS Trainings? 99
vii List of Figures Figure 1. Number of expected and observed CBCL T-scores by descriptive category 71 Figure 2. Number of observed and e xpected ABAS-II standard scores by descriptive category 74 Figure 3. Preand posttest mean scores for CBCL scales 77 Figure 4. Preand posttest mean scores for ABAS-II scales 81 Figure 5. Average Daily Parent Ratings of Ease or Difficulty of Skill Use at Home 83
viii Caregivers Perceptions of the Effectiveness of the Helping Our Toddlers, Developing Our Childrens Skills Parent Training Program: A Pilot Study Jillian Leigh Williams ABSTRACT This study was designed to evalua te a parent training curriculum: Helping Our Toddlers Developing Our Childrens Skills (HOT DOCS) using archival data collected between August 2006 and April 2007. The eval uation studied the impact of specific components of the parent training program on both participants knowledge and attitudes and their perceptions of targeted childrens behavior. One-hundred-for ty-six caregivers of children between the ages of 14 months and te n years of age particip ated in the parent training program and were included in th e analyses. Measures included a pre/post knowledge test, rating scales of child probl em behavior, weekly progress monitoring forms for caregiver behavior at home, and a program evaluation survey. Results indicated significant increases in caregiver knowledge following participation in the program. Prior to participation, caregivers pe rceptions of the severity of child problem behaviors and deficits in adaptive behaviors were signi ficantly different from a normative sample. Following participation in the program, results showed significant decreases in caregiver perceptions of the severity of child problem behaviors, but no signi ficant differences in child adaptive behaviors. Caregiver feedback indicated high levels of satisfaction with the program.
1 CHAPTER 1 Introduction Statement of the Problem After nearly three decades of cross-disc iplinary research, professionals in the fields of psychology, education, and medicine are no longer su rprised that their client lists, student rosters, and appointment schedul es are filled with young children displaying challenging behaviors. The most comm only cited challenging behaviors in young children (between the ages of 2 and 7 years old) include sleeping difficulties, mealtime and feeding issues, toilet training, temper tantrums, a ggression, sibling rivalry and noncompliance. Recent research has shown that approximately 15%-25% of all typically developing preschool children have chronic levels of behavior problems that fall within the mild to moderate range (Campbell, 1995; Keenan & Wakschlag, 2000; Knapp, Ammen, Arstein-Kerslake, Poulson, & Ma stergeorge, 2007; La vigne et al., 1996). Prevalence rates of chronic behavior problem s for minority children and/or children in low-income families have been identified as ranging between 25% and 35% of typically developing children (Gross, Sambro ok, & Fogg, 1999; Webster-Stratton, 1998). The long-term outcomes associated with early onset challeng ing behavior in young children have been well-documented (Coie & Dodge, 1998; Dishion, French, & Patterson, 1995; Kazdin, 1995; Moffitt, 1993; Re id, 1993; Tremblay 2000). In general, the earlier the problem behavior develops th e more stable and intense the associated negative outcomes are over time. Dishion, Fr ench, and Patterson (1995) found that early
2 appearing behavior problems in a childs pres chool career are the single best predictor of delinquency in adolescence, gang membership, and adult incarceration. Other researchers have identified similarly poor long-term outcomes related to academic and school performance. Kazdin (1993) and Tremblay (2000) concluded from their research that preschoolers with challenging behaviors are at a greater risk of experiencing school failure than typically developing children. Several studies have investigated the poor social and interpersonal outcomes associated wi th developing challengi ng behaviors at an early age. Coie and Dodge (1998) found that preschoolers with challenging behavior were more likely to experience early and persistent peer reje ction. Strain and his colleagues (1983) reported that preschoolers wi th challenging behaviors also were more likely to experience more punitive interac tions with teachers than their typically developing peers. Reid (1993) found that earl y appearing aggressive behavior is the single best predictor of juvenile gang membership and violence. In response to research demonstrating the rapid and enduring increase in the prevalence rates of young children with challeng ing behavior and the associated negative long-term outcomes, professionals across di sciplines have developed a variety of treatments to help prevent and treat th ese behaviors. For example, psychotropic medications (Barkley, 1997), i ndividual clinical therapy or counseling with the child (Barkley et al., 2000; Foreha nd & Long, 1988), individual cons ultation with the family (Anastopoulos, Shelton, DuPaul, & Guevrem ont, 1993; Feinfield & Baker, 2004), play therapy (Blackwell, 2005; McNeil, Ca page, Bahl, & Blanc, 1999; Nixon, Sweeny, Erickson, Touyz, 2003), and behavioral parent training (Kazdin, 1997; Sanders, Mazzucchelli, & Studman, 2004; Webster-Stratton, 1998) have all been evaluated for
3 their efficacy in reducing challenging behavi or in young children. A more recent advance in this body of research is the downward exte nsion of the principles of positive behavior support (PBS) as an intervention techni que for young children and their families (Buschbacher, Fox, & Clarke 2004; Dunlap & Fox, 1996; Frea, 2004; Hieneman, Childs, & Sergay, 2006). Of these interventions a nd treatments, behavioral parent training delivered in a group format has been shown to be an effective treatment for challenging behavior in young children, while utilizing th e least amount of resources and empowering parents to prevent the deve lopment of future problem behaviors (Lundahl, Risser, & Lovejoy, 2006; Maughan, Christiansen, Je nson, Olympia, & Clark, 2005; Nelson, 1995; Sandall & Ostrosky, 1999; Smith & Fox, 2003). Theoretical Framework Historically, one of the major theories guiding the inquiry into chronic behavior problems in young children is Skinners (1953) theory of behavioris m. At its foundation, behaviorism postulates that all behavior is ob servable and functional. Behaviorism relies on the manipulation of antecedents and conse quences and the effects of reinforcement and punishment as a means of changing and shaping behavior. In addition to approaching the study of challenging beha vior in young children from a behavioral theoretical framework, it is necessary to view the pr oblem through an ecological model of child development (Bronfenbrenner, 1979). An ecol ogical model takes into account biological, sociological, and psychological domains of child development and functioning (Sontag, 1996). From an ecological perspective, manipul ation of a childs environment, including the behavior of caretakers, will directly impact the childs behavior (Bronfenbrenner, 1979).
4 A newer lens through which researchers a nd practitioners have begun to approach challenging child behaviors is through the prin ciples of positive behavior support (PBS). Positive behavior support has emerged from the study of applied behavior analysis (ABA) and is an approach to studying child behavior problems by viewing problems as a lack of behavioral adaptation (Dunlap, 2006). ABA was established as a science in the 1960s in which learning principles were sy stematically applied to produce socially important changes in behavior (Cooper, Heron, & Heward, 1987). PBS emerged in the late 1980s as a strategy of intervention and support, employing concepts and techniques from ABA and other disciplines, with the inte nt of enhancing an i ndividuals quality of life and reducing problem beha viors (Carr et al., 2002). Overview of the HOT DOCS Parent Training Program HOT DOCS, or Helping Our Toddlers Developing Our Childrens Skills (Armstrong, Lilly & Curtiss, 2006) is a behavioral parent training curriculum based on the principles of positive behavior support. HOT DOCS meets criteria fo r a behavioral intervention, such as 1) centering around an operant model of behavior, 2) providing parents with detailed information on effectiv e parenting strategies, 3) focusing on control of antecedents instead of punitive conse quences, and 4) programming specifically designed to enhance generalization from th e training setting to the home setting. The original Helping Our Toddlers ( H.O.T .) curriculum (Armstrong & Hornbeck, 2005) was developed through a U.S. Department of E ducation grant, with funds matched by the Childrens Board of Hillsborough County, Florida (Fox, Dunlap, & Powell 2002). The grant was provided to fund research to investigate the effectiveness of positive behavior support (PBS) applied to toddlers with challengi ng behavior and was referred to as the
5 Early Intervention Positive Behavi or Support (EIPBS) project. The H.O.T. curriculum was developed by the EIPBS project director and a parent of a young child diagnosed with Autism. The H.O.T curriculum was based on the principles of PBS (i.e., understanding the function of behavior, an tecedents and consequences, and teaching replacement behaviors). The parent training program cons isted of six weeks of group training conducted in community settings such as churches and the YMCA. The original H.O.T curriculum was delivered to four cohorts of parents and caregivers of young children with challenging behaviors, av eraging 8-12 individuals per group between 2005 and 2006. Data collected du ring initial parent training groups included demographic information, parent sa tisfaction with the program, knowledge of basic behavioral principles, and use of parenting skills taught in class. These data were used to generate outcome reports requir ed by the funding agency. Focus groups and follow-up surveys conducted upon completion of the fourth cohort of participants reported that 100% of parents who participat ed in the program noticed improvements in their own parenting skills and their childs behavior (Armstrong, Hornbeck, Beam, Mack, & Popkave, 2006). Following the first four cohorts of H.O.T. parent training, several substantial revisions to materials, procedur es, and data collecti on were made to the curriculum. Subsequently, the original H.O.T curriculum has evolved into a manualized training program called Helping Our Toddlers, Developing Our Childrens Skills ( HOT DOCS; Armstrong, Lilly, & Curtiss, 2006). Although in itial qualitative reports of parent satisfaction and improvements in child behavi or suggest success of the program, a more rigorous and standardized evaluation of the HOT DOCS parent training curriculum is needed.
6 Purpose of the Study The current study was designed to serve as a preliminary investigation of caregivers perceptions of the effectiveness of the Helping Our Toddlers Developing Our Childrens Skills (HOT DOCS) parent training program. Th e study evaluated the impact of specific components of the parent trai ning program on caregivers knowledge and attitudes and their perceptions of targeted childrens behavior. In addition, data from this study will be used to inves tigate the extent to which the intervention was efficacious. Research Questions 1. What is the impact of participation in the 6-week HOT DOCS parent training program on caregiver knowledge as measur ed by preand posttest scores on the HOT DOCS Knowledge Test ? 2. Do caregivers perceive their child as having more problem behavior than a normative sample prior to participation in the 6-week parent training program? 3. Do caregivers perceive their child as having less adaptive behavior than a normative sample prior to participation in the 6-week parent training program? 4. To what extent do caregivers perceive a decrease in child problem behavior following caregiver participation in th e 6-week parent-t raining program? 5. To what extent do caregivers perceive an increase in child adaptive behavior following caregiver participation in the 6-week parent-training program? 6. a. What is the frequency and ease of use of the weekly parent ing tips as reported by caregivers? b. Is there a relation between frequency of use and ease of use as measured by the HOT DOCS Tip Tracker sheets?
7 7. What are caregivers overa ll perceptions of the HOT DOCS parent training program as measured by the HOT DOCS Program Evaluation Survey ? Significance of the Study Results of this pilot study will be used to help the researcher modify and improve the instruments and procedures used to evaluate outcomes of the HOT DOCS parent training program. The results will also be shar ed with the authors of the parent training program in order to help improve and refine the contents and delivery of the program. This study will also investigate whether or not the HOT DOCS program is an effective intervention for increasing caregiver knowledge and improving child behavior. Definition of Terms Young children will be defined for the purposes of this study as children between the ages of 2 and 7 years of age. Behavioral parent training is defined as an intervention technique in which professionals provide training in specific parenting skill s and techniques, which are derived from a behavioral perspective, to parents of young children. Behavioral parent training programs generally have four comm on elements: 1) centering around an operant model of behavior, 2) providing parents w ith detailed information on appropriate and effective parenting strategies, 3) focusing on control of antecedents instead of punitive consequences, and 4) programming specifica lly intended to enhan ce generalization from the training setting to the home and commun ity settings (Fienfield & Baker, 2004). Challenging behavior is defined as a pattern of re peated behaviors, or perception of behavior, that interferes w ith or is at risk of interf ering with optimal learning or
8 engagement in pro-social inte ractions with peers and adults (Dunst, Trivett, & Cutspec, 2002). Challenging behavior is therefore de fined on the basis of its effects.
9 CHAPTER 2 Review of Related Literature Overview This chapter provides a review of the lit erature relevant to this study. Challenging behavior in young children is discussed, including prevalence rates, negative outcomes associated with early emerging behavior proble ms, and the role of parenting skills in the development of challenging behavior. Res earch supporting the impor tance of prevention and early intervention is reviewed, as well as the effectiveness of parent training as an intervention. Finally, the application of a positive behavior interventions (PBS) framework in interventions for young children with challenging behaviors is examined. This chapter concludes with a discussion of the importance of providing effective behavioral parent training through a positiv e behavior support framework to enable parents and caregivers to prev ent and correct cha llenging behavior in young children as early as possible. Prevalence of Young Children with Challenging Behavior Numerous studies conducted over the pa st 30 years have shown a dramatic increase in the number of young children w ho are referred to professionals due to challenging behaviors (Campbell, 1995; Kn app, et al., 2007; Lavigne et al., 1996). Studies report that up to 75% of all psychol ogical referrals for children are related to disruptive and noncompliant behavior (Feinfie ld & Baker, 2004). Researchers also have found that the proportion of ch ildren meeting the criteria fo r a clinical diagnosis of
10 oppositional defiant disorder (ODD) ranges be tween 7% and 25%, depending on the age of the population surveyed (W ebster-Stratton, 2000). Overal l, the prevalence rate for challenging behaviors in young children varies between 10% and 16% for the general population (Campbell, 1995; Schuhmann, Foot e, Eyberg, Boggs, & Algina; 1998; Webster-Stratton, 2000) and be tween 25% and 30% for child ren living in poverty (Gross et al., 1999; Keenan & Waksch lag, 2000; Qi & Kaiser, 2003). Gross and colleagues (1999) conducted a cro ss-sectional study of 2and 3-yearold children from low-income families to describe the prevalence rates and correlates of challenging behaviors in preschool childre n. The study included parents of 133 young children from 10 daycare centers in an urban city. Most of the parents included in this study were African American (64%) or Latino (25%) and were categorized as being low in socio-economic status based on income level (e.g., 50% of participants earned less than the states median income). Parents co mpleted measures of type and intensity of child behavior problems, parenting self-e fficacy, parental discip line strategies, and parental stress. Findings from the study showed that 32% of the young children had clinically significant levels of problem behaviors in the home setting. These results should be interpreted with caution given that the sample was composed of two minority ethnic groups of low SES. Results from thes e findings should only be generalized to similar populations. In 2003, Qi and Kaiser conducted a review of research pertaini ng specifically to challenging behaviors in young children from low-income families. These researchers reviewed and summarized research on this topic published between 1991 and 2002 with the goal of synthesizing prevalence rates of behavior problems and identifying risk
11 factors for behavior problems. Results of this study showed that children whose families are poor are significantly more likely than middleor upper-class families to develop behavior problems. Findings from this review were similar to previous reports (Gross et al., 1999; DelHomme, Sinclair, & Kasari, 1994; Feil, Walker, Severson, & Ball, 2000) in estimating that prevalence rate of challe nging behavior for children from low-SES families is approximately 30%. Keenan and Wakschlag (2000) conducted a study to examine the severity of challenging behaviors exhibi ted by preschool-aged childre n. The authors completed comprehensive psychological evaluations with 79 clinic-referred preschoolers from a primarily low-SES, urban setting. The co mprehensive evaluations included semistructured diagnostic parent interview (Schedule for Affec tive Disorders for School-Age Children-epidemiological 5th version; Orvaschel & Puig -Antich, 1995), child behavior rating scales (Child Behavior Checklist; Ache nbach, 1991), direct obser vations of parentchild interactions, developmental assessment (Differential Abilities Scales; Elliot, 1983), and overall clinical impairment ratings (Child Global Assessm ent Scale; Setterber, Bird, Guld, Shaffer, & Fisher, 1992). Results indi cated that nearly 80% of the preschool children met Diagnostic a nd Statistical Manual-4th Edition (DSM-IV, American Psychiatric Association, 1994) criteria for a disruptive beha vior or Attention-Deficit Disorder. Specifically, 60% of the children met criteria for Oppositional Defiant Disorder (ODD) and 42% met criteria for Conduct Di sorder (CD). These findings support the growing body of research identifying incr easing prevalence and severity rates of disruptive behaviors in young children.
12 Outcomes Associated with Early Emerging Behavior Problems The problem of increasing prevalence ra tes of challenging behavior in young children becomes more significant when th e long-term outcomes of early-emerging behavior problems are taken into account. Ch ildren who are identified as hard to manage at ages 3 and 4 years old are twice as lik ely as their typically-developing peers to continue to display problem behavior into adolescence (Campbell & Ewing, 1990; Egeland, Kalkoske, Gottesman, & Erikson, 1990; Fischer, Rolf, Hasazi & Cummings, 1984). Egeland and colleagues (1990) conducte d a longitudinal study in which they assessed the stability of beha vior problems in children beginning in preschool and following-up through 3rd grade. Parents of 118 children between the ages of 4 and 5 years completed child behavior rating scales and measures of parental stress and mental health. Assessments also included direct observations of child behavior and semistructured parent interviews. Ninety-six ch ildren met criteria for behaviors including acting out, withdrawal, or atte ntion problems. Twenty-two ch ildren did not meet criteria and served as the control group. Results indi cated a high degree of stability in the presence of child problem behaviors. A lim itation of this study was that the children included in the study were all at least 4 years old, which excl uded a critical portion of the young children at-risk for developing behavior problems who are between the ages of 2 and 3 years. A similar study conducted by Campbell a nd Ewing (1990) also tracked the stability of behavior problems first identifie d in the preschool year s; however, in this study, follow-up assessments were conducted at age 6 years and again at 9 years and focused specifically on the children who we re excluded from the age range in the
13 previous study. Parents of 51, 3-year-old children completed behavior rating scales, parenting stress indices, semi-str uctured interviews and particip ated in direct observations of behavior. Twenty-nine of the children we re classified as hard-to-manage and 22 children served as developmentally appropriate control group peers. Results of this study showed that children who exhibi ted clinically significant probl em behavior at 3 years of age were more likely to continue to demonstr ate problem behaviors at ages 6 and 9 years. Results also showed that the majority (67 %) of children who had clinically significant behavior problems at 6 years of age met Diagnostic and Statistical Manual-3rd Edition (DSM-III; American Psychiatric Association, 19 87) criteria for extern alizing disorders at age 9. Young children who demonstrat e challenging behavior in the preschool years are more likely to experience school failure (Kazdin, 1993; Tremblay, 2000), peer rejection (Coie & Dodge, 1998), punitive teacher interact ions (Strain, Lambert, Kerr, Stragg, & Lenker, 1983), and unpleasant family inte ractions (Patterson & Fleischman, 1979). Preschoolers with early-emerging challenging be havior are also more likely to have adult lives characterized by violence, abuse, loneliness, psychiatric illness, injury, unemployment, divorce, and early death (Coie & Dodge, 1998; Kazdin, 1995; Lipsey & Derzon, 1998; Olweus, 1991; Walker, Colvin, & Ramsey, 1995). Role of Parenting in Child Behavior Problems Much of the recent research conducted in the fields of psychology and education has focused on the etiology of challenging be havior in young children. A major theme to emerge in this body of research is that pare nting style and parent-c hild relationships are significant determinants of child mental he alth problems, including challenging behavior
14 (Loeber & Dishion, 1983; Patterson, DeBaryshe, & Ramsey, 1989; Rutter, 1991; Stormshak, Bierman, McMahon, Lengua, 2000). Studies have shown that a common factor in the etiology of most childh ood behavior problems and social-emotional disorders is difficulty in the parent-ch ild relationship (Kendziora & OLeary, 1993; Mrazek, Mrazek, & Klinnert, 1995; Patterson et al., 1989; Ruttner, 1991; Shaw, Emery, & Turner, 1993). Negative parent -child interact ion styles are more frequently observed in families with young children with behavior problems and are predictive of more persistence in disruptive beha viors (Buss, 1981; Feinfield, 19 95; Pettit, Bates, & Dodge, 1993; Webster-Stratton, 1985). A classic mode l in the field of child psychology is Pattersons (1982) coercion model, which explains how negative parent-child interactions lacking warmth and negotiation serve to ex acerbate a childs problem behaviors, especially aggression. Parenting difficultie s produce combinations of oppositional and avoidant behaviors in children, which in turn increase parental negativ ity (Bradley et al., 2003; Cummings & Davies, 1994). If this coer cive cycle is prolonge d the result is a strained parent-child relati onship and persistent challengi ng child behavior (Patterson, 1982). Denham, Workman, Cole, Weissbrod, Ke ndziora, and Zahn-Waxler (2000) conducted a study to examine the contribution of parental emotions and behaviors to the emergence of disruptive and noncompliant be haviors in preschool children. The study included 79 mothers and fathers and their childr en, who met criteria for being at-risk for development of disruptive beha vior disorders. Children invo lved in this study ranged in age from 2 years to 5 years, with a mean age of 4 years. Participants in this study were predominantly Caucasian (96%) and from a mi ddleor upper-class socio-economic status
15 (96%). Families were evaluated at four times during the 4-year longitudinal study, including a pretest, two pr ogress monitoring evaluations, and a posttest. Researchers assessed childrens externalizing behavior through parent and teacher reports using Achenbachs (1991) Child Behavior Checklist (CBCL) and Teacher Report Form (TRF), as well as Youth Self-Reports (YSF). Parenting skills were assessed at the first and fourth assessment through direct observation of parent s interactions with their children in naturalistic play activities. Parenting patte rns were coded for patterns of behavior, including supportive presence, limit setting, allowance of autonomy, negative affect, quality of instruction, and confidence. Parent ing patterns also were coded for emotional expression, including anger and happiness. Resu lts of the study indi cated that children with externalizing problems evident during th e pre-test continued to have behavior problems at the 2-year and 4-year follow-up ev aluations. Results also demonstrated that proactive parenting techniques (e.g., being supportive, giving clear directions, setting limits) predicted decreased behavior problems overtime, especially for children with clinically significant levels of problem behaviors at pretest. Conversely, children of parents who frequently expressed anger were mo re likely to have continued or worsening externalizing behaviors at the follow-up eval uations. The results of this study should be interpreted with caution, given the limite d diversity in ethnicity and SES of the participants included and the small sample size. Other studies have shown that parent s of young children with externalizing behaviors use more frequent verbal and corporal punishment than parents of young children without challenging behaviors (Nicholson, Fox, & Johnson, 2005). Nicholson, Fox, and Johnson (2005) conducted a study inve stigating the difficulties of parenting
16 children with challenging behavi or as well as the protective f actors that may exist in these families. Preschool teachers identified 30 ch ildren (ages 2 to 5 years) who displayed challenging behaviors and a matched gr oup of 30 children who did not display challenging behaviors to serve as the comp arison group. Teacher classification of child behavior problems was confirmed using the Sutter-Eyberg Student Behavior Inventory (Eyberg & Pincus, 1999). The final sample consisted of 60 children and their mothers who were mostly Caucasian (93%), married (78%), and had a minimum of a high school diploma (72%). Each mother was asked to co mplete a self-report measure of parenting behavior (Parent Behavior Checklist; Fox, 1994), and two ratings scales of child behavior (Child Behavior Scale; Fox & Nicholson, 2003; Eyberg Child Behavior Inventory; Eyberg & Pincus, 1999) during a home interv iew. With regard to parent behavior, significant results were found (p <.05) in the differences betw een the parenting practices of mothers of children with challenging be havior and mothers of children with typical behaviors. Specifically, mothers of children with challenging behavior reported more frequent use of verbal and corporal punish ment than mothers in the control group. No differences were found between the mothers us e of nurturing behavi ors or expectations. With regard to child behavior, mothers of children with challenging behavior rated their childrens behavior at home to be significantly more probl ematic than mothers in the control group on both the ECBI and CBS. Results of this study indicated that mothers of children with teacher-identified challenging behavior interact with their children differently than mothers of children without challenging behaviors. This study provided evidence of differences in parenting practices in families of children with typical and challenging behavior, however, generalization of these results are limited due to a small
17 sample size and homogenous participant demogra phics. The conclusions of this study are also limited by the use of only self-report m easures and no direct obs ervations of parent or child behavior. A similar study by Stormshak and colleagues (2000) also investig ated differences in parent-child interactions in families with children with challenging behavior, but avoided the problem of limite d generalizability in the prev ious study by selecting a more diverse sample. This study was conducted with a large population-based sample of at-risk and diverse 1st grade students from four locatio ns across the United States (North Carolina, Tennessee, Washington, and Pe nnsylvania). The sample included 631 kindergartners (mean age 6.45 years) with chal lenging behavior from various ethnic and racial groups (49% minority-predominantly Af rican American, 51% European American) and socio-economic status levels as well as a matched comparison sample of 387 children without challenging behaviors. Measures us ed in this study incl uded parent (Child Behavior Checklist; Achenbach, 1991) and teacher reports (Teach er Observation of Classroom Adaptation-Revised; Kellem, 1989) of child behavior and several self-report measures of parenting pract ices (Conflict Ta ctics Scale; Straus, 1989; Parent Questionnaire; Strayhorn & Weidman, 1988; Pa renting Practices Inventory; CPPRG, 1996). Results indicated that parents who re ported that their children had challenging behaviors also reported significantly more frequent use of punitive discipline strategies and aggressive parenting styl es (e.g., yelling, spanking, thr eatening) than parents who reported their childrens behavior to be w ithin normal limits. Punitive discipline and inconsistent parenting were significantly a ssociated with child oppositional, aggressive, and hyperactive behaviors. With the excepti on of a stronger relationship between punitive
18 discipline strategies and child problem beha viors for African American than European American parents, there were no signifi cant differences between ethnic groups across parenting practices or child behavior found in this study. Th is lack of significant group differences suggests a high degree of consistenc y in the influence of parenting practices on child behavior across ethnic groups in Amer ica. Similar to previous studies, the absence of direct assessment of child beha vior, parenting practi ces, and parent-child interactions presents a limitation to the results of the study. While negative parenting practices can pr oduce or exacerbate problem behavior in children, child problem behaviors can also lead to increased le vels of parent stress, and marital conflict (Forehand & Long, 1988; Pa tterson, Reid, & Dishion, 1992; WebsterStratton & Hammond, 1997). Following the cyclic model, elevated levels of chronic parental stress are associated with the maintenance of extern alizing behavior problems in children (Campbell, 1997; Heller, Baker, He nker, & Hinshaw, 1996). Recent research also has shown that nurturing, authoritative, responsive parenting th at utilizes positive behavioral interventions can improve child be havior, enhance child development, reduce the need for professional services in the futu re and reduce parent st ress (Hebbler, Spiker, Mallik, Scrborough, Simeonsson, & Collier, 200 1; Nicholson et al., 2005; Ramey & Ramey, 1998; Shonokoff & Phillips 2000). Pettit, Bates, and Dodge (1993) conducte d a longitudinal stud y investigating the family interaction variables that were predictive of childrens externalizing problems during the transition from kindergarten to 1st grade. Specifically, the researchers investigated the hypothesis that positive-proactive and negati ve-coercive parenting styles would make independent, nonoverlapping contributions to the prediction of conduct
19 problems in children. The sample included 165 families who were recruited from a larger, ongoing study (see Dodge, Bates, & Peti t, 1990). The sample consisted of a range of social classes (high, middle, and low inco me families) and equal numbers of boys (n = 82) and girls (n = 83). The sample was predominantly White (84%) and represented twoparent families (70%). The children were stra tified into groups of high, medium, and low aggression based upon mothers ratings of child aggression on the Child Behavior Checklist (Achenbach & Edelbr ock, 1983). All children were observed in their homes during the summer prior to beginning kindergarten using a focused-narrative observational system to code various family interactions. Observations were conducted on two separate occasions for each family, lasting approximately two hours each, and were typically conducted during or near dinner time. Families were instructed to proceed with their normal routines and behaviors and attempt to ignore the observers as much as possible. In addition to the di rect observations, parents comp leted child behavior rating scales. All three data collection methods (hom e observations, parent rating scale, and teacher rating scale) were completed again a year later, in the su mmer prior to children beginning 1st grade. Results indicated a strong correlation (p<.05) between negativecoercive parenting by mothers and child ex ternalizing behavior problems in and 1st grade (behaviors rated by both parents and teacher s). Correlations between negative-coercive parenting by fathers and child externalizing behavior problems were not significant at the kindergarten or first grade levels. This study also found that early, positive parent-child and family interactions predicted lower leve ls of externalizing behavior problems in kindergarten and first grade. These results pr ovide support for the significant influence of parenting styles and parent-child interac tion patterns on child behavior problems.
20 Outcomes Associated with Early Intervention Despite the projections of negative shor tand long-term outcomes for children who develop challenging behaviors at an earl y age, research has shown that the use of evidence-based intervention techniques can pr event and alleviate many of the associated negative outcomes (Marchant, Young, & West, 2004; Walker, Kavanaugh, Stiller, Golly, Severson, & Feil, 1998; Webster-Stratton, 1998). Marchant a nd colleagues (2004) recently demonstrated that pr evention strategies implemente d as early as the preschool years helped children av oid more severe problems later in life. In this study, four 4-yearold children who were consider ed to be at-risk for develo ping antisocial behavior and their parents participated in an interven tion training program. Du ring the training phase, the parent coach (first author) developed a co llaborative relationship with parents, trained parents to use specific parenting skills, and provided parents with immediate feedback on their use of the skills. Specific skills in cluded a direct teaching sequence aimed at increasing child compliance with multi-step di rections and a corrective teaching sequence used when the child was non-compliant w ith adult direction. The direct teaching sequence included describing the skill (complia nce) and the steps the child should follow, giving reasons that show the benefit of co mpliance, showing or modeling the steps of compliance for the child, and giving the chil d feedback in the form of praise or correction. The corrective teaching sequence included being positive (praise), describing the incorrect behavior, promp ting the correct behavior (rol e play if necessary), and praising the child for listening and trying again. The study used a multiple baseline design across the four parent-child dyads to investigate parent and child behaviors in baseline, training, coaching, and follow-up pha ses. Results of the study showed that
21 children as young as 4 years old were able to show improvements in their behavior following a brief parent-child intervention. Li mitations of this study included the small sample size and a homogenous sample in term s of ethnicity (all four families were Caucasian). Despite its limited generalizability, the results of this study suggest that early intervention for challenging behaviors in young ch ildren can be effective with children as young as 4 years of age. When parents use responsive parenti ng practices and positive behavioral interventions in the early years, behavior prob lems are less entrenched, easier to treat, and the potential impact upon future developmenta l trajectories is greater (Dunlap & Fox, 1996; Lutzker & Campbell, 1994; WebsterStratton, 1998). In other studies, early intervention has been associat ed with a decreased risk of withdrawal, aggression, noncompliance, teen pregnancy, juvenile deli nquency, and special education placement (Strain & Timm, 2001). The application of evid ence-based treatment approaches has also been associated with increas ed self-control, self-monitori ng, self-correction, and socialemotional health (Webster-Stratton, 1990); mo re positive peer relationships and social skills (Denham & Burton, 1996); and improved academic success (Walker et al., 1998). Parent Training as an Intervention Despite the available evidence supporting the effectiveness of early intervention, there is a lack of services, resources, and empirically-supported interv entions available to parents of young children disp laying challenging behavior (Kazdin & Kendall, 1998; Walker et al., 1998). Recent estimates have shown that fewer than 10% of young children who show early signs of problem behavior r eceive services for their difficulties (Kazdin & Kendall, 1998). For those children who do receive services, the outcomes may still be
22 bleak, considering research findings that the developmen tal course of challenging behavior is predictably nega tive for children who are not tr eated or who receive poor treatment (Lipsey & Derzon, 1998; Patterson & Fleishman, 1979; Wahler & Dumas, 1986). Kumpfer and Alvarado (2003) also sugges ted that a lack of professional training in evidence-based intervention approaches may be contributing to small effect sizes in prevention and intervention research. The lack of available services is even more dismaying in the light of research findings s howing that if challeng ing behaviors are not altered by the time a child reaches the age of nine years, the behavior problems are considered chronic and will require con tinuing and costly intervention (Dodge, 1993). In order to maximize available resources and maintain a cost-effective method of service delivery, intervention techniques re aching the most children using the fewest resources have recently drawn attention. The mo st promising and effective of these costreducing interventions is parent training (Kazdin, 1995) Parent training involves professionals teaching parents and other caregive rs the basics in behavioral principles and behavior management techniques, which the pa rents can then apply with their children. Parent training programs have been shown to be effective when delivered to individual parents or to groups of parents (Feinfie ld & Baker, 2004). Many researchers have provided evidence supporting the use of beha vioral parent training programs to reduce the development and persistence of problem be havior and improve th e quality of parentchild interactions (Gro ss et al., 2003; Maughan et al., 2005; McMahon & Forehand, 2003; Nixon et al., 2003). The majority of empirically-supported parent training programs have four common components: a) based on an operant model; b) provide detailed information on the effective and appr opriate use of time-out procedures; c) focus
23 on antecedent control instead of punitiv e consequences; and d) program for generalization from the training setting to natural settings, including home and community contexts (Feinfield & Baker, 2004) Research has also shown that programs that focus on changing parenting behavior have a stronger effect on child behavior outcomes than do programs that focus on cha nging parents attitudes (Sanders, 1996). In an analysis of parent trai ning research conducted by Webste r-Stratton and Taylor (2001), available evidence suggested that parent tr aining produced the grea test effects with children between the ages of 3 and 10 year s; created clinic-based changes that generalized to the home setting (but not ofte n to the school setting); created clinically significant and meaningful improve ments in two thirds of targeted children; and resulted in changes in childrens behavi or lasting up to four years. In 2005, Maughan and colleagues conducte d a meta-analysis to collect and quantitatively analyze the existing body of literature and research available regarding behavioral parent training as a treatment for externalizing behavior problems in children. The meta-analysis provided a description of studies, summarized the effects of the treatment studied, described vari ables that affected the treat ment effects, and calculated an effect size to indicate the significance of each treatments effects. Studies which were included in the meta-analysis were: a) c onducted between 1966 and 2001; b) targeted least one externalizing behavior; c) targ eted children who did not have autism or developmental delays; d) included treatment procedures such as training parents or caregivers in the use of reinforcement and/ or time-out and one additional parenting procedure; e) targeted childre n between the ages of 3 and 16 years old; f) used at least one outcome measure on childs behavior; g) used either between-subjects group design,
24 within-subjects group design, or single-subject design; h) in corporated graphs displaying raw data representing baseline data with at l east 5 data points if si ngle subject design was used. To find research studies, the authors searched using intern et tools and journal databases looking for all studies on behavior al parent training conducted within the specified time period. The search resulted in 294 studies, of which, 79 (26%) met the remaining inclusion criteria. Each study was coded for specific in formation related to participant demographics, research design and methods, training program com ponents, and outcome assessment. Effect sizes were calcu lated using statistics such as t, F or p values when means and standard deviations were not avai lable. For between-subjects designs, effect sizes were calculated based upon differences between pretest and posttest scores between the control and treatment group participants. For within-sub jects designs, effect sizes were calculated based upon difference between pretest and posttest scores for a single sample, divided by the pretest standard de viation (producing a standardized mean change). For single-subject designs, effect sizes were calculated using the ITSACORR computer program. After an effect size was computed for each individual study, a composite effect size with a 95% confidence inte rval was calculated for each of the three research design types (between-subjects, wi thin-subjects, and single-subjects designs). Potential bias for studies not included in th e meta-analysis, which may not have been available due to null results, no effect or lack of publishing, was corrected for by calculating a Fail Safe N which represented the number of studies that would have had to be included in the meta-analysis if all the possible studies were included.
25 For the 79 studies included, 108 separate effect sizes were calculated. Most of the studies used a group training format (n=32), some used individual consultation (n=20), some used controlled learning techniques (n =10), and the remaining studies used mixed methodology (n=17). There were 2,083 partic ipants in the between-subjects groups; 1,088 participants in the with in-subjects groups. There were 15 single-subject studies, which yielded 1,482 data points. The unweighted mean effect size for between-subjects studies was d = .58 (each study contributes equally to ove rall mean) and the weighted mean effect size was d = .30 (95% CI .21 to .39). There were no signifi cant outliers in the between-subjects group. Because the confidence interval did not incl ude zero, it was assumed that behavioral parent training conducted in a group format had a significant eff ect on the criterion variable. Differences in effect size were f ound when studies were analyzed separately based on the coded criteria variab les. Studies with parents of children between the ages of 3 and 5 years had an effect size of .40 while studies with parents of children between the ages of 6 and 8 years had an average effect size of .19 and children between the ages of 9 and 11 years had an average effect size of 1.36. Studies with training programs using 1 to 5 sessions had a mean effect size of .96; t hose using 6 to 10 sessions had a mean effect size of .50; those using 11 to 15 sessions ha d a mean effect size of .45; and those using more than 15 sessions had a mean effect size of .08; indicating that larger effects were found when fewer sessions were used, although no further explanation or interpretation of these differences were provided. In su mmary, variables significantly impacting the effect size of between-subjects studies incl uded method of outcome assessment, child
26 age, method of program delivery, number of sessions, method of assignment to conditions, and use of reliability assessments. The unweighted mean effect size for with in-subjects studies was d = .74 and the weighted mean effect size was .68 (95% CI .59 to .77). The confidence interval for the within-subjects groups did not include zero, indicating that the studies had a significant impact on outcome measures. There was one outlier present in this group, which was removed from further statistical analyses. St udies delivering traini ng in an individual consultation format had an average effect size of .43, while studies using a group format had an average effect size of .70. This findi ng supported related studi es in finding larger effects when training was delivered in a gr oup format, which has been explained by the positive effects of peer support and mode ling (Lundahl, et al., 2005). In summary, variables significantly impacting the effect size of within-subjects studies included method of outcome assessment and method of program delivery. The unweighted mean effect size for single-subjects studies was d = .59 and the weighted mean effect size was d = .54 (95% CI .43 to .65). There were no significant outliers in the single-subj ects group. The confidence inte rval did not include zero, implying the treatment had a significant eff ect on the criterion variable. In summary, variables significantly impacting the effect si ze of single-subjects st udies included child age and method of program delivery. Results of the meta-analysis suggest that be havioral parent trai ning is an effective intervention for reducing externalizing probl em behaviors in children; however, the effectiveness of this intervention is not as la rge as it was hypothesized to be prior to the meta-analysis. The overall mean weighted effect sizes for between-subjects, within-
27 subjects, and single-subject rese arch designs were all within the small to moderate range and were considered potentia lly significant (between-subj ects and single-subjects) and compelling (within-subjects). The authors cauti oned over-interpretation of the superior average effect size for with in-subjects design ove r between-subjects a nd single-subjects designs, citing previous research showing that this type of res earch design causes inflated effect sizes, regardless of actual treatment ef fects on outcomes. The authors also caution against over-interpretation of differences in effect size based on method of outcome assessment, citing a potential for parent biases in self-reported outcome measures versus direct observation. Suggestions for future rese arch included coding studies for treatment integrity and social validity measures. Limitations of the meta-analysis included variability in the methodological quality of studies reviewed and methodological limitations in calculating effect sizes for outcomes in single-subject designs. Over the past 20 years, research ers have conducted numerous studies investigating the effectiveness of various parent training programs, including the Incredible Years (Webster-Stratton, 2001), Parent Child Interaction Therapy (Eyberg, 1988), and Triple P-Positive Parenting Practices (Sanders, 1999). Despite differences in training components, duration, and research methodology, several meta-analyses have shown that much of the outcome research available reported si milar findings supporting the effectiveness of behavioral parent trai ning programs in impr oving behavior in young children (Conroy, Dunlap, Clarke, & Alter, 2005; Lundahl et al., 2006; Maughan et al., 2005). In one examination of the Incredible Years parent training series, Scott (2005) tested the effects of this program in a clinical practice setting. Participants were 59
28 parents of children ages three to eight year s residing in London and Southern England. All children were referred for antisocial beha vior to their local community mental health agency. The Parent Account of Child Symptoms was used as a semi-structured interview to gather parents reports of childrens an tisocial behavior preand post-intervention. Parents also completed the Strengths and Difficulties Questionnaire (SDQ) as a selfreport of their childs conduct problems, hyperactivity, peer relationships, and prosocial behavior. Parents received the 12-week BASIC parent training program of the Incredible Years series, which was administered accordi ng to the manual. A control (waiting list) group was used for comparison purposes. Faci litators of all sessions were trained therapists from each local health agency. Immediately following the end of intervention, parent reports of child behavior as meas ured by the interview showed significant decreases in antisocial behavior; similar findings were shown for negative behavior reports on the SDQ, but with smaller effect sizes. Similar or even greater decreases in antisocial behavior and hypera ctivity were found at the oneyear follow-up as compared to controls. Peer relationships did no t show significant improvement following intervention. The researchers also found that risk factors such as ethnic minority, single parent families, and low SES did not redu ce treatment effectiveness. Demographic information did not include the percentages of participants who were mothers versus fathers. This would be valuable informa tion to report regardi ng whether or not the program was effective for both parents. It is necessary to evaluate research conducted with American children and families and dive rse ethnic populations to determine whether this training series will be as effective with American children and families as it was for English participants.
29 The Incredible Years parenting program was also evaluated among 634 ethnically diverse mothers of childre n enrolled in Head Start (Reid, Webster-Stratton, & Beauchaine, 2001). The CBCL was used to as sess externalizing be haviors including aggression and antisocial behaviors from pare nt reports. Parents of all ethnic groups receiving intervention were observed to be more positive, less inconsistent, and use less harsh discipline in their pare nting (as measured via the Dyad ic Parent-Child Interactive Coding System Revised (DPICS-R) compared to parents in the control group, who were exposed to only the regular Head Start program. Additionally, children of parents receiving the intervention were observed vi a the DPICS-R to exhibit fewer behavior problems at one-year follow-up; however, CBCL reports were not significantly improved for the intervention group. Importantly, fe w differences were reported across ethnic groups and significant differences were onl y found among the use of positive parenting and use of critical statements to children as measured by the DPICS-R. These results indicate the applicabil ity of this program for ethnically divers e populations. The large sample size and randomized, controlled design add statistical streng th to the positive findings of this study. Schuhmann and colleagues (1998) conducted a randomized, controlled trial of Parent Child Interaction Therapy (PCIT) with 64 clinic-referred families. Participants were assigned to a PCIT treatment conditi on (n=37) or a waitlist control group (n=27). Criteria for inclusion specified that all families referred had a child who was of preschool age (3 to 5 years) with a DSM-IV diagnosis of conduct disorder. Fam ilies in the treatment condition participated in PCIT sessions wh ile control group families were evaluated using the outcome measures, but had no other c ontact with the therapists or researchers.
30 Outcome measures included direct observation of the quality of parent-child interactions using the Dyadic Parent-Child Interac tion Coding System-II (DPCICS-II; Eyberg, Bessmer, Newcomb, Edwards, & Robinson, 1994) the Parental Locus of Control Scale (PLOC; Campis, Lyman, & Prentice-Dunn, 1986) and the Dyadic Adjustment Scale (DAS; Spanier, 1976). Assessments were re-administered every 4 months during treatment and at a follow-up assessment 4 m onths after the final PCIT session. Results showed that parents participating in PCIT sessions had more positive interactions with their children, and children demonstrated more frequent compliance with parent direction as compared to the parents in the waitlist control group. Parents in the PCIT group also reported lower levels of parental stress and gr eater internal locus of control in parenting practices compared to the waitlist control group. Finally, parents in the PCIT group reported greater improvements in their children s behavior following the therapy sessions than did the control group parents. Diffe rentially positive outcomes for the PCIT group were maintained at the 4-month follow-up a ssessment. A limitation of this study was the relatively brief follow-up period, as researcher s determined maintenance of outcomes at four months post-treatment. Further research assessing trea tment maintenance at longer intervals following treatment termination woul d strengthen the efficacy reports for PCIT. A more recent study provided support for the long-term maintenance of treatment outcomes for PCIT (Eyberg, Funderburk, Hembree-Kigin, McNeil, Querido & Hood, 2001). Eyberg and colleagues (2001) studied th e maintenance of treatment outcomes for 13 families with preschoolers diagnosed with conduct disorder at oneand two-years post-treatment. Treatment effectiveness was measured by the DPCICS-II (Eyberg et al., 1994), the Parenting Stress Index (PSI; Abid in, 1995), the PLOC (Campis et al., 1986),
31 and the DAS (Spainier, 1976). Significant diffe rences (p < .05) we re found between the PCIT families and the control group families on all measures. Eight of the 13 families maintained positive treatment effects at the oneand two-year follow-up assessments. Sanders, Markie-Dadds, Tully, and Bor ( 2000) conducted a controlled trial of Triple P-Positive Parenting Practices (TPP) in which three variants of the program ranging in levels of intensity were compar ed on 305 preschool-aged children (mean age 3 years) at risk for developing conduct problem s. Families were randomly assigned to one of four conditions: (a) enhanced level, (b) st andard level, (c) self-directed, and (d) waitlist control. The various conditi ons varied from practitioner-a ssisted to self-directed using booklets and videos at the familys home The standard program involved teaching parents 17 core child management strategies Ten of the strategies were designed to increase childrens competence and devel opment (e.g., talking with children; physical affection; praise; at tention; engaging activ ities; setting a good example). The remaining seven strategies were designed to help parents manage challenging behaviors by engaging in positive parenting practices (e.g., se tting rules; directed discussion; planned ignoring; clear, direct inst ructions; logical c onsequences; and time-out). Parents were taught a six-step planned activ ities routine to enhance the generalization and maintenance of parenting skills (e.g., plan ahead; decide on rule s; select engaging activities; decide on rewards and consequences; and hold follow-up di scussions with the child). Parents were taught to apply parenting skill s to a broad range of target behaviors in both home and community settings with the target child and their sibli ngs. Short-term and long-term follow-up data were collected on the effectiv eness of the intervention. Various measures were utilized to collect frequency and intensity of behavior information for each child in
32 order to ascertain the level of behavior ch ange pre and post-intervention. Specifically, the Parent Daily Report (PDR ; Chamberlain & Reid, 1987), Parenting Scale (PS; Arnold, O'Leary, Wolff, & Acker, 1993), and the Pa rent Problem Checklist (PPC; Dadds & Powell, 1991) were utilized. The results show ed that all levels of the TPP produced significant results for the children and families taking part in the study, however, the enhanced (most intensive) version produced the greatest results. Applying Principles of Positive Be havior Support to Parent Training Positive behavior support (PBS) refers to a process designed to address problem behaviors by helping caregiver s understand the function of th eir childs behavior, then teaching their children the needed replacement skills through implementation of positive behavioral strategies (Dunlap et al., 2003). Identifying the purpose served by the childs problem behavior, or the func tion of the behavior, is a cen tral tenet of PBS. Parent training programs which teach parents to so lve the problem of challenging behavior by identifying the behavioral f unction (e.g., obtain, escape, contro l), help caregivers develop new strategies to support their children, whic h are practiced within the familys daily routines (Armstrong, Hornbeck et al., 2006; Dunlap & Fox, 1996). Conroy, Dunlap, Clarke and Alter (2005) conducte d a meta-analysis of the ava ilable research on the use of PBS interventions with young children. The me ta-analysis included research conducted between 1984 and 2003, which was published in 23 peer-reviewed journals. Articles that met inclusion criteria were evaluate d based on the following demographic and methodological variables: a) di sability type; b) age and gender; c) availability of demographic data; d) setting; e) dependent m easures; f) intervention type; g) intervention agents; h) study design; i) re porting of generalization trea tment fidelity and social
33 validation data. Results indicated that the majority of the interventions targeted children between the ages of 3 and 6 years (80%), who most often had developmental delays, including autism and pervasive developmental delays (59%) or intellectual disabilities (29%). The majority of the studies used t eachers (42%), researcher s (37%), and family members (26%) as intervention agents a nd were conducted in school (62%) or home (26%) settings. Most studies used destruc tive (74%) or disruptiv e (53%) behaviors as outcome measures as indices of decrease s in problem behavior; however, a large percentage of studies also incl uded indices of increases in ad aptive or prosocial skills as outcome measures, such as sk ill performance (45%), engagement (30%), or social interaction (23%). The incl usion of positive or prosocia l outcomes is not surprising, considering the focus on positive behaviors and learning new skills is one of the central tenets of PBS. Results also indicated that a large percentage of the studies used multicomponent intervention plans (45%), making it difficult to identify what specific intervention strategies led to the behavioral improvements. Additionally, only one of the studies used a between-group experimental design, while 85% of the studies used a within-group, single-subject de sign. Finally, the meta-analysis found that very few of the studies incorporated measures of generalizabil ity (15%), treatment fidelity (8%), or social validity (26%) in their research. Overall re sults of the meta-analysis highlighted an increasing trend for professionals in ear ly education to use positive behavior interventions with children w ho display challenging behaviors. A single-subject study co nducted by Buschbacher and colleagues (2004) demonstrated the effectiveness of teaching a parent to implement positive behavior support interventions in the home setting w ith a child who had severe medical and
34 behavioral problems, specifically, autism spectrum behaviors and Landau-Kleffner syndrome. The research design used in this study was a concurrent multiple-baseline across behaviors design, which allowed the rese archers to conduct statistical analyses and to draw meaningful interpretations from these results. Results of this study indicated that individually administered PBS interventions reduced the childs challenging behavior, increased the childs engagement, increased positive parent-child interactions, decreased negative parent-child interac tions, and increased the number of days the child slept throughout the night. In addition to child and parent outcome measures, the researchers asked four independent adults to view videotapes of the inte rvention sessions and to rate whether or not the intervention was socially acceptable and whether or not the childs behavior changed in visible, meaningful wa ys. All four reviewers indicated that the intervention was acceptable and the childs be havior meaningfully improved. Despite the limitations in generalizability due to single-s ubject design and very specific medical and psychosocial characteristics of the subject, this study provi des support for the usefulness and social acceptability of PBS interventions. Duda, Dunlap, Fox, Lentini and Clarke (2004) conducted a study, which demonstrated the effectiveness of using PBS in terventions in a preschool setting with two young children. Duda and her colleagues used a single-subject, ABAB design to evaluate the interventions effectiveness at managi ng two 3-year-old girls behavior in a community preschool. The researchers conducte d extensive consultation with school staff and provided training in the principles of teaming and PBS. Following this extensive preparatory period, the research ers facilitated a team-based functional assessment of the two girls problem behaviors. Once the team determined the function of the girls
35 challenging behaviors, the rese archers assisted the team in developing PBS intervention strategies and trained the teachers to imple ment the interventions using modeling and feedback procedures. Outcome data consiste d of direct observation of the childrens engagement in specific classroom activities, frequency of challenging behavior, treatment integrity as measured by teacher compliance w ith the intervention components, and social validity data as repor ted by the teachers and a nave obs erver. Results indicated that during the intervention phase following the initi al baseline phase, both girls level of engagement increased and frequency of probl em behaviors decreased. During the return to baseline condition following the first inte rvention phase, both girls showed rapid returns to initial baseline levels of low engagement and high frequency of challenging behavior. In the final intervention phase both girls again showed increases in engagement and decreases in challenging behavior. Although social validity data indicated that teachers felt c onfident that the intervention components were acceptable and that they were able to perform all compone nts, treatment integrity data indicated that both teachers left out critical components of the interven tion during multiple fidelity observations. Despite its rigorous res earch methodology, this study had several weaknesses, which should be considered when interpreting the results. These limitations included a restricted sample size compos ed of two girls who both had significant developmental delays and provision of intens ive training and support to the teachers by the researchers prior to, during, and after completion of the study. While these limitations do not lessen the significance of the effectiveness of the intervention for the specific participants in this study, th ey suggest that demographic and participant variables may have been responsible for changes in ch ild and teacher behavior. These results
36 contributed additional support to the feasibil ity, acceptability, and effectiveness of PBS interventions to prevent and correct problem beha viors in young children. While PBS has been documented in recent years to assist individuals of all ages and developmental levels in the home, school, and community settings (Bushcbacher, Fox, & Clarke, 2004; Conroy et al., 2005; Duda et al., 2004; Fox, Dunlap, & Cushing, 2002; Fox, Dunlap, & Powell, 2002; Fox & Little, 2001; Vaughn, White, Johnston, & Dunlap, 2005), its availability has been limited due to costs associated with providing individually administered, in tensive services (Armstrong, Hornbeck et al., 2006). To increase the availability of PBS to more families of young children, a parent training curriculum organized around the six core princi ples of PBS and the literature on early childhood development and infant me ntal health was developed. Preliminary reports indicate the effectiveness of the Helping Our Toddlers, Developing Our Childrens Skills ( HOT DOCS) parent training program as a means of reducing challenging behavior in yo ung children and improving parent-child relationships based on the results of a study completed by Armstrong, Hornbeck and colleagues (2006). Although these preliminary fi ndings are encouraging in light of their results indicating high levels of parent satisf action with the program and parent reports of improvements in child behavior (Armstrong et al., 2006), recent federal mandates, such as the Individuals with Disabilities E ducation Improvement Act (IDEIA, U.S. Department of Education, 2004) and the No Child Left Behind Act (NCLB, 2001), emphasize the importance of using only thos e interventions that are empiricallysupported through rigorous and co mpetent research. If practitioners are go ing to continue to use HOT DOCS the program must be formally evaluated.
37 Summary The past three decades of research have indicated an alarming and ever-growing need for interventions that address challeng ing behavior in young children. Studies have consistently demonstrated prevalence rates of challenging behavior upwards of 25% in the 3to 5-year age group. Longitudinal research also ha s clearly demonstrated the profuse, long-term negative outcomes associat ed with early emerging behavior problems. Following the research on increasing prevalen ce rates and long-term negative outcomes, researchers and practitioners ha ve developed a multitude of strategies for preventing and treating behavior problems in children and fa milies. Of these interventions, behavioral parent training has been suppor ted by numerous, repeated, we ll-designed studies and is generally considered the best-practices approach to preventing and remediating challenging behavior in young children. Finally, recent research in the field of positive behavior support has demonstrated the principles incorporated in PBS interventions to be effective and socially accep table as interventions for young children with challenging behaviors. The past three decades of research has clearly indicated a need for empiricallysupported, evidence-based parent training in terventions, and more recent research has indicated that approaching prevention thr ough a PBS framework will enable parents and caregivers to prevent and corre ct challenging behavior in young children as early as possible.
38 CHAPTER 3 Methods Introduction The purpose of this study was a prelim inary investigation of caregivers perceptions of the effectiveness of the Helping Our Toddlers Developing Our Childrens Skills (HOT DOCS) parent training program. This ch apter presents information about participants, setting, the HOT DOCS parent training program, tools for measurement, methods of data collection, and methods of data analysis. Research methods used in this study were dictated by the archival nature of the data. The design used by the developers was a one-group, pretest/posttest design. Participants Parents and/or caregivers were referred by their pediatrician, psychologist, or therapist, or were recru ited through community advertisement with brochures and posters, to participate in a university-based parent training program for families with children displaying challenging or disruptive behavior. As referrals were made or caregivers responded to public advertisements caregivers names were added to a waitlist for future parent training sessions. Twohundred-sixty caregivers were scheduled to participate in the parent trai ning program. As is shown in Table 1, of the expected 260 caregivers, 71 caregivers did not return re minder phone calls and did not participate in the program. Of the 189 caregivers who were present for the first session, 30 (11.5%) attended fewer than three of the remaini ng sessions and were considered drop-outs.
39 Thirteen (5%) of the caregivers attended three or more sessions but elected not to sign the Internal Review Board (IRB) release form and therefore were not included in data collection, although th ey did complete the course. The fi nal participant sample consisted of 146 caregivers who attended three or more of the sessions in one of fifteen classes conducted between August 2006 and April 2007 (11 delivered in English and four delivered in Spanish). Table 1 Attendance Record of Initial Caregiver Sample Attendance record # Caregivers Percent Scheduled to attend 260 100 Never attended 71 27.3 Attended first session 189 72.7 Attended fewer than 3 sessions total 30 11.5 Attended 3+ sessions but did not sign IRB 13 5.0 Signed IRB and attended 3+ sessions 146 56.2 Note : Percent reported is percen t of caregivers expected to attend the first session. Description of Caregivers A breakdown of the final participant samp le by gender, race/ethnicity, education level, and type of insurance is shown in Tabl es 2 and 3. Participants were 32.2% male (n = 47) and 67.8% female (n = 99). Particip ants ranged in age from 23 to 69 years (M = 38.5, SD = 9.19). The sample consisted of caregiv ers reporting their ra ce or ethnicity as White (43.8%), Hispanic (34.9%), African Am erican or Black (5.5%), Other (3.4%), Native American (2.7%), or Asian (0.7%). Care givers reported level of education varied from less than a high school diploma to a graduate level degree, with the largest percentage of participants (26.7%) receiving a degree from a 4-year college (n = 39). Approximately 19% of the participants repo rted having earned a high school diploma or
40 less, 20% reported having t echnical training or a two-y ear college degree, and 52% reported having a four-year college degree or graduate level degree. Table 2 Breakdown of Participant Sample by Gende r, Race/Ethnicity, and Education Level Variable Number Percent (%) Gender Female 99 67.8 Male 47 32.2 Race/Ethnicity African American/Black 8 5.5 White 64 43.8 Hispanic 51 34.9 Asian 1 0.7 Native American 4 2.7 Other 5 3.4 Not Reported 13 8.9 Caregiver Education Level Less than HS 4 2.7 HS Diploma 23 15.8 Technical Training 9 6.2 2-Year College Degree 21 14.4 4-Year College Degree 39 26.7 Graduate Degree 38 26.0 Not Reported 12 8.2 Note n = 146 (Only participants who completed 3 or more sessions of parent training and consented to participate in the study by signi ng the IRB consent form were included in data analysis). Within the context of this study, type of insurance was used as a general indicator of socio-economic status, w ith private insurance repres enting higher socio-economic status and Medicaid or no insurance repres enting lower socio-economic status. As is shown in Table 3, approximately 56% of partic ipants reported havi ng private insurance, 26% of participants reported having Medicaid insurance, and 5% of participants reported having no insurance. Some participants (12%) did not respon d to this item.
41 Table 3 Breakdown of Participant Sample by SES Indicator Type of Insurance Number Percent (%) Private 82 56.2 Medicaid 38 26.0 No Insurance 8 5.4 Not Reported 18 12.3 Note n = 14 As shown in Table 4, of the female pa rticipants, 79 reporte d being the childs mother or adoptive/foster mother, 12 reported being child service providers, six reported being the childs grandmother, one reporte d being the childs aunt, and one reported being the childs sister. Of the male particip ants, 43 reported being th e childs father or adoptive/foster father and four repor ted being the childs grandfather. Table 4 Relation of Caregiver to Target Child Relation Number Percent (%) Females (n = 99) Mother & Adoptive/Foster Mother 79 79.8 Grandmother 6 6.1 Other Female Relative 2 2.0 Child Service Provider 12 12.1 Males (n = 47) Father & Adoptive/Foster Father 43 91.5 Grandfather 4 8.5 Note n = 146 Description of Target Children Target children ranged in age from 14 months to ten years (M = 47.0 months, SD = 23.89). Approximately 34% of the targeted children had existing medical and/or psychological diagnoses. Many of the remaining children had recently been evaluated by pediatricians or psychologists due to parent or teacher c oncerns with development and
42 behavior, but did not meet criteria for a diagnosis according to the Diagnostic and Statistical Manual-4th Edition, Text Revision (DSM-IV -TR, American Psychiatric Association, 2000). As is shown in Table 5, of the children in the samp le with preexisting diagnoses, 20 (13.7%) were children with a diagnosis on the autism spectrum including Pervasive Developmental Disorder (ASD/P DD), seven (4.8%) were children with developmental delays, six (4.1%) were childr en with speech or language impairments, five (3.4%) were children with Attention-De ficit/Hyperactivity Di sorder (ADHD), five (3.4%) were children with epilepsy, two (1.4%) were children with Prader Willi syndrome, two (1.4%) were children with schizencephaly (diagnosis reported by caregiver on demographic questionnaire), one (<1%) was a child with a hearing impairment, and one (<1%) was a child with cerebral palsy. Table 5 Number and Percent of Target Children by Preexisting Diagnosis Childs Preexisting Diagnosis Number Percent (%) None 97 66.4 ASD/PDD 20 13.7 Developmental Delay 7 4.8 Speech-Language Impairment 6 4.1 ADHD 5 3.4 Epilepsy 5 3.4 Prader Willi Syndrome 2 1.4 Schizencephaly 2 1.4 Hearing Impairment 1 0.7 Cerebral Palsy 1 0.7 Note n = 146
43 Caregivers Not Completing Training (Drop Outs) Description of Caregivers Not Completing Training A breakdown of the participants who di d not complete the training by gender, race/ethnicity, education level, and type of insurance is shown in Tables 6 and 7. Demographic information available from the 30 participants who did not complete at least three sessions was analyzed with descri ptive statistics. Participants who did not complete the training were 43.3% (n = 13) males and 56.7% (n = 17) females. The participants who dropped out before comple ting the program ranged in age from 24-50 years (M = 32.95, SD = 4.43). The particip ants who dropped-out reported their race/ethnicity as Hispanic (33.3%), Cau casian (26.7%), Other (10.0%), or African American (6.7%). Caregivers reported level of education varied from less than a high school diploma to a graduate level degree, w ith the largest percentage of participants (20.0%) completing graduate level training (n = 6). Approximately 17% of the caregivers reported having earned a high school diploma or less, 30% reported having technical training or a two-year colle ge degree, and 37% reported having a four-year college degree or graduate level degree. Forty-three percent of the partic ipants who dropped-out reported having private insurance, 30% re ported having Medicaid insurance, and 27% reported having no insurance.
44 Table 6 Breakdown of Program Non-Completers by Ge nder, Race/Ethnicity, and Education Level Variable Number Percent (%) Gender Female 17 56.7 Male 13 43.3 Ethnicity African American/Black 2 6.7 Caucasian 8 26.7 Hispanic 10 33.3 Other 3 10.0 Not Reported 7 23.3 Parent Education Level Less than HS 3 10.0 HS Diploma 2 6.7 Technical Training 2 6.7 2-Year College Degree 5 16.7 4-Year College Degree 5 16.7 Graduate Degree 6 20.0 Not Reported 7 23.3 Note n = 30 (only participants who signed th e IRB but completed fewer than 3 sessions of parent training were used in data analysis). Table 7 Breakdown of Program Non-Completers by SES Indicator Type of Insurance Number Percent (%) Private 13 43.3 Medicaid 9 30.0 Not Reported 8 26.7 Note n = 30 As shown in Table 8, of the female car egivers who did not complete training, 15 reported being the childs mother, one reporte d being the childs aunt, and one reported being a child services provi der. All of the male caregivers not completing training reported being the childs father.
45 Table 8 Relation of Caregiver to Target Child for Program Non-Completers Relation Number Percent (%) Females (n = 17) Mother 15 88.2 Aunt 1 5.9 Child Service Provider 1 5.9 Males (n = 13) Father 13 100 Note n = 30 Description of Target Children of Caregivers Not Completing Training Targeted children of the caregivers who dropped out of the program ranged in age from 24 months to seven years (M = 48.23 months, SD = 21.74). As shown in Table 9, the majority of these children were id entified with preexis ting medical and/or psychological conditions; including five ch ildren (16.7%) identif ied on the autism spectrum, one child (3.3%) with ADHD, one ch ild (3.3%) with a hearing impairment, and one child (3.3%) identified w ith developmental delays. Table 9 Breakdown of Target Children of Program Non-Completers by Preexisting Diagnosis Childs Preexisting Diagnosis Number Percent (%) None 22 73.3 ASD/PDD 5 16.7 ADHD 1 3.3 Hearing Impairment 1 3.3 Developmental Delay 1 3.3 Note n = 30 Differences between the demographic char acteristics of program completers and program non-completers or drop-outs were compared through vi sual inspection of percentages. Overall, the demographic charac teristics of caregive rs who completed the training program and those who dropped out be fore completing the program appeared to
46 be very similar. Program completers a nd non-completers differed slightly in the percentage of caregivers reporting their ethnicity as Caucasian. Setting This study was conducted at a large Univer sity in West Central Florida. The parent training program was delivered in the Childrens Medical Services clinic, which is run by the Department of Pediatrics at the Univ ersity. Parent training groups were held in conference rooms within a campus clinic. HOT DOCS Parent Training Program The HOT DOCS parent training program was deliv ered in six sessions. Each of the six sessions lasted approximately two hours. The first session included thirty minutes of socialization, including a light dinner provided by trainers a nd brief introductions; twenty minutes during which parents complete d the demographics form and pretest (see description of measures below for details); and one hour of behavior al parent training. The second, third, fourth, and fifth sessions in cluded 30 minutes of socialization, peer support, and review followed by training. The sixth session included 30 minutes of socialization, peer support, and review fo llowed by training, and then finished with twenty minutes during which parents comple ted the posttest and a program evaluation survey (see description of m easures below for details). The training for each session included lecture, practice exercises, role pl aying, and video vignettes. Each session also included a Parenting Tip and a Special Play Activity. The weekly Parenting Tips were specific skills parents were asked to pr actice using throughout the following week. Parents were asked to use the HOT DOCS Tip Tracker sheets to keep a record of the number of days they used the skill, to rate how difficult or easy the skill was to use each
47 day, and to provide specific examples of how they used the skill with their children each week. The Special Play Activities were specif ic play activities parents were asked to engage in with their child for five minutes each day of the following week. Parents were provided with the small toys necessary to enga ge their child in the play activity and were given instructions, examples, and a worksheet with guidelines describing how to use the five minutes of special play to teach th eir child motor, communication, and socialemotional skills. A more detailed description of each training session follows. Session One. The first session provided partic ipants with an overview of the HOT DOCS program and an introduction to early childhood development. Parents were instructed in brain development, typical ages for achievement of developmental milestones and warning signs for delays in de velopment, school read iness skills, and an overview of the problem-solving process. The Parenting Tip for the first session was Use Positive Words, which was explained to parents as telling children what to do instead of what not to do. For example, pare nts should say, Feet on the floor, instead of Stop jumping on the couch. A class act ivity was conducted in which parents brainstormed positive ways to rephrase twenty of the most common behaviors parents usually respond to with No! or Stop! The Special Play activity for session one was Bubbles. Each participant was given a cont ainer of bubbles to use for this activity. A detailed breakdown of the session contents, tips, and activities is provided in Appendix A. Session Two The second session focused on teaching parents about the importance of healthy routines and ritual s in promoting positive development and adaptive behavior in young children. Sleep routines, or the activit ies surrounding
48 bedtime, were highlighted, si nce this is the most common problematic routine for most parents and children. The Pare nting Tip for this session wa s Catch Them Being Good, which prompted parents to focus on the positiv e behaviors or skills their children exhibit each day and to respond with specific, labele d praise for these behaviors. The Special Play activity for this session was readi ng, for which parents were again provided instruction, examples, and a detailed worksheet of activities. Each participant was given a developmentally appropriate storybook. Session Three The third session introduced parent s to the basics of behavior development in young children, including the co ncepts of social learning, modeling, antecedents and consequences, reinforcement and the function of behavior. In this session, parents were introduced to the problem-solving chart, which includes triggers, behaviors, consequences, preventions, new skills, and new responses. In this session, parents learned to complete the first three s ections. The Parenting Ti p for this session was Use Calm Voice, which reminded parents to use a calm, quiet voice in response to their childs behavior, especially in response to challenging or noncompliant behavior. The Special Play Activity was coloring, for whic h each participant was given a coloring book and a box of crayons. Session Four The fourth session provided parent s with training in the use of various preventative strategi es, including using timers, providing prompts, clarifying expectations, visual schedules or prompts, and personalized st ories. The Parenting Tip for this session was Use Preventions, which pr omoted parents use of the preventative techniques taught in the session. The Speci al Play Activity was fun dough, for which
49 each parent was provided with one colo r or tub of dough and a cartoon character placemat. Session Five The fifth session provided parents w ith training in how to teach their children new skills and replacement skills for challenging behaviors. In this session parents began to complete the second half of the problem solving chart, including the preventions and new skills sections. Parent s were also provided instruction in the appropriate uses and steps for Time-Out from Positive Reinforcement and what to do when children misbehave or are non-compliant. The Parenting Tip for this session was Follow Through, which provided parents with a brief script to use whenever their children did not comply with a direction or task. The Speci al Play Activity was playing with a ball, which each parent was provided before leaving the session. Session Six. The sixth and final session focuse d on helping parents understand and manage their own stress as well as providing a summary and review of the content of the previous sessions. Parents completed the fina l categories of the problem solving behavior chart by listing the variety of new responses pa rents can have to thei r childs appropriate behaviors. These new responses include specific praise, prompting, validation and redirection, and follow through. The Parenti ng Tip for this session was Take 5 for Yourself, which reminded parents to focus on their own health and stress levels each day. There was not a new Special Play Activity for this week, but parents were prompted to use one of the five previously lear ned Special Play Activities each day. All of the materials, curricula, presen tations, and handouts were translated to Spanish (Armstrong, Lilly, Curtiss, Salinas, Chiraboga, & Ortiz, 2006) by a team of USF university students and staff including a fellow in internal medicine and pediatrics who
50 was originally from Ecuador; a master of pub lic health student with medical degree and a Fulbright Scholar, who was originally from Nicaragua; a doctoral intern in school psychology, who spoke Spanish as a second la nguage; and a parent liaison for the HOT DOCS program, who was originally from Columbia. Measures HOT DOCS Demographics Form The Demographics Form was developed by the HOT DOCS authors in order to collect informa tion about the caregiver participants and the children the parents targeted as ha ving challenging behavi or who were involved in the parent training program. This form includes 10 questions which ask the caregivers to indicate their address, gender, age, childs age, age(s) of other children in the home, type and name of health insurance, relations hip to targeted child, ethnicity, and level of education. The demographics form is av ailable in both English and Spanish (see Appendices C and D). HOT DOCS Knowledge Test The Knowledge Test was also developed by the HOT DOCS authors in order to assess caregiver s knowledge of child development, behavioral principles, and pa renting strategies. Although th e test includes items from various areas of knowledge covered in the pare nting program, at this point there are not enough items per area to investigate cluster scores. For the purposes of this study, only total scores were recorded and analyzed. Th e test consists of twenty True/False statements and takes approximately ten minutes to complete. The pre-test was administered during the first session, follo wing the program overview and prior to the first lecture. The posttest was administered during the sixth session, following completion
51 of the final lecture. The knowledge test is available in both English and Spanish (see Appendices E and F). HOT DOCS Tip Tracker Sheets. The Tip Tracker sheets were developed by the HOT DOCS authors to monitor, on a daily basis, caregivers use of the skills learned in the sessions at home. The sh eets contain seven columns (one for each day of the week) with a 5-point Likert-type scale, which asks ca regivers to rate each day their ease of use of the specific parenting skill of the week with their child. The Likert scale ranges from 1 = Very difficult to 4 = Easy In addition, a response option, Did not use skill is provided. Caregivers are asked to circle this option if they did not use the skill that day. The sheet also contains four blank lines on which caregivers are asked to give specific examples of how they used the parenting tip with the ta rget child. These caregiver responses were used to validate the participan ts understanding of th e skill and appropriate implementation. The sheets are available in both English and Spanish (see examples in Appendices G and H). Child Behavior Checklist The Child Behavior Checklist (CBCL; Achenbach, 2001) was developed to assess internalizing and externaliz ing behaviors in children. There are multiple versions of the CBCL depe nding on the childs age and the source of information. The CBCL 1-5 was developed for use with children between the ages of 18 and 71 months of age and can be co mpleted by parents/caregivers and/or teachers/caregivers. The CBCL 6-18 was develo ped for use with children and adolescents between the ages of 6 and 18 years and can be completed by parents/caregivers and teachers. The CBCL problem behavior scores are grouped into two broad-band factors (internalizing and externalizi ng problems), a total broad-ba nd score derived by averaging
52 weighted scores from the broad-band fact ors, and eight narrow-band subscales. The narrow-band subscales include aggressive behavior, anxious/d epressed, attention problems, delinquent behavior, social problem s, somatic complaints thought problems, and withdrawn behavior. All versions of the CBCL are available in English and Spanish. Since this study will evaluate the results of ch ildren between the ages of 2 and 7 years, both the CBCL 1-5 and CBCL 6-18 will be used for data analysis. Both forms of the CBCL are very similar in design, differing only in the type and amount of items asked (CBCL 1-5 has 99 items, CBCL 6-18 has 112 items). Each form is a questionnaire that asks parents to rate th eir childs behavior in the previous 2 months by rating each item on a three-point scale: 0 = not true of the child, 1 = somewhat or sometimes true and 2 = very true or often true Several items include prompts for parents to provide brief descriptions of problems, di sabilities, most significant parent concerns, and to list their childs st rengths. Completing the CBCL takes approximately 20 minutes. Responses are scored using a computerized scoring software program. Scores are expressed as T-scores with a mean of 50 and a standard deviation of 10. A T-score of 64 or below is in the normal range; 65-69 is in th e borderline range; and 70 or above is in the clinical range. Scores in the borderline or clinical range indi cate that a childs behavior problems are more significant than other children the same age and gender. The CBCL 1-5 was normed on a national sample of 700 children The manual reports median internal consistency coefficients for the Internalizing and Externalizing scales that range from .76 to .92. Studies of the CBCL subscales indicated high retest reliability (Withdrawn: r = .82; Somatic Complaints: r = .95; Anxious/Depressed: r = .86; Social Problems: r = .87; Internalizing Problems: r = .89) and adequate interrater
53 reliability (Withdrawn: r = .66; Somatic Complaints: r = .52; Anxious/Depressed : r = .77; Social Problems: r = .77; Internalizing Problems: r = .66; Achenbach, 1991). The CBCL 6-18 was normed on a national sample of 1,753 children The manual reports median internal consistency coefficients for the Internalizing and Externalizing scales that range from .78 to .97. Studies of the CBCL subscales indicated high test-retest reliability ( r = .90) and high content and criterion rela ted validity. For the purposes of this study, the following scores will be used for analysis: Internalizing Problems, Externalizing Problems, and marg inal pretest and posttest means. Adaptive Behavior Assessment System-2nd Edition The Adaptive Behavior Assessment System-Second Edition (ABA S-II; Harrison & Oakland, 2003) was developed to assess adaptive skil ls and levels of adaptive f unctioning for individuals from birth to 89 years of age. There are multiple versions of the ABAS-II depending on the age of the child and the source of information. The ABAS-II Pare nt/Primary Caregiver Form (Ages 0-5) was developed for children ages bi rth to 5 years 11 months and is completed by parents or caregivers. The ABAS-II Paren t/Primary Caregiver Form (Ages 5-21) was developed for children, adoles cents, and young adults ages 5 to 21 years old. There also are ABAS-II forms, which ask teachers, caregivers, and daycare providers to rate childrens adaptive skills in similar domains as the parent forms for children between the ages of 2 to 5 years or 5 to 21 years. Norm-referenced scores include three broad domains of adaptive behavior (Conceptual, So cial, and Practical), a combined General Adaptive Composite (GAC), and 10 sub-domain skill areas. The skill areas measured by the ABAS-II Parent/Primary Caregiver Fo rm are communication, community use, functional pre-academics, home living, health a nd safety, leisure, self care, self-direction,
54 social, and motor skills. The skill areas that make up the Conceptual domain are communication, functional pre-academics, and self-direction. The Social domain is composed of skill areas that measure social sk ills and leisurely skills. The skill areas that make up the Practical domain are self-care, home living, community use, and health and safety. Both the ABAS-II Parent/Primary Ca regiver Form (Ages 0-5) and the ABAS-II Parent Form (Ages 5-21) are available in English and Spanish. Since this study will evaluate the results of children between the ages of 2 and 7 years, the ABAS-II Parent/Primary Caregiver Form (Ages 0-5) a nd the Parent Form (Ages 5-21) will be used for data analysis. For the purposes of this study, the following scores will be used for analysis: Conceptual, Social, and Practical Domains and General Adaptive Composite. The ABAS-II Parent/Primary Caregiver Form (Ages 0-5) and the Parent Form (Ages 5-21) are similar in design, differi ng only in the number and type of items (Parent/Primary Caregiver Form has 241 items, the Parent Form has 232 items). Both forms of the ABAS-II are questionnaires that ask parents or caregivers to rate their childs current performance on adaptive ski lls functioning. Parents or caregivers are asked to rate each item using the following scale: 0 = Is not able to do the skill, 1 = Never or almost never when needed to do the skill, 2 = Sometimes when needed will do the skill, and 3 = Always or almost always when needed will do the skill. The ABAS-II Parent/Primary Caregiver Form and the Pare nt Form take approximately 20 minutes to complete. Responses are scored using a comput erized scoring software program. Specific skill area scaled scores have mean of 10 and a standard deviation of 3. The skill area scores combine to form the three ABAS-II broad domain scores and the GAC score, each with a composite score mean of 100 and a standard deviation of 15. Composite scores
55 falling between 90 and 109 and scaled scores falling between 8 and 12 are classified in the average range. The ABAS-II Parent/Primary Caregive r Form (Ages 0-5) was normed on a national sample of 1,350 children ages birth to 5 years 11 months with demographics similar to the 2000 U.S. census. The manual reports the internal consistency for the skill area scores to range from 0.80-0.92, and 0.91-0. 97 for the composite scores. Studies of the ABAS-II subscales indicated high te st-retest reliability (Communication: r = 0.82; Community Use: r = 0.79; Functional Pre-Academics: r = 0.85; Home Living: r = 0.83; Health and Safety: r = 0.81; Leisure: r = 0.80; Self-Care: r = 0.81; Self-Direction: r = 0.80; Social: r = 0.81; Motor: r = 0.80; Harrison & Oakland, 2003). The ABAS-II Parent Form (Ages 5-21) was normed on a national sample of 1,670 children and adolescents between 5 and 21 years of age with demographics similar to the 2000 U.S. census. The manual reports the internal consistency for the skill area scores to range from 0.86-0.93, and 0.95-0.98 for the composite scores. Studies of the ABAS-II subscales indicated hi gh test-retest reliability (Communication: r = 0.84; Community Use: r = 0.91; Functional Academics: r = 0.92; Home Living: r = 0.87; Health and Safety: r = 0.89; Leisure: r = 0.88; Self-Care: r = 0.90; Self-Direction: r = 0.88; Social: r = 0.91; Harrison & Oakland, 2003). In summary, the CBCL and ABAS-II are ps ychometrically sound instruments, as evidenced by their validity and reliability estimates. Each instrument makes a different contribution toward providing information about a childs overall functioning. The CBCL measures problem behavior and the ABAS-II assesses adaptive behavior and functional skills.
56 HOT DOCS Program Evaluation Survey The Program Evaluation Survey was developed by the HOT DOCS authors to assess caregiver participants perceptions of the effectiveness of the parent training program. The survey consists of eight statements about the benefits of HOT DOCS to parents, the skill of HOT DOCS trainers, HOT DOCS impact on child and family behaviors and relationships, wh ich caregivers are asked to rate on a 4-point Likert-type scale as strongly agree, ag ree, disagree, or strongly disagree. The survey also consists of six open-ended questions, which prompt caregivers to share their perceptions on the usefulness of the program as well as any suggestions for future traini ngs or improvements to the current program. The survey is available in both English and Sp anish (see Appendices I and J). Data Collection The pilot study used archival data, as the researcher analyzed data collected by the HOT DOCS authors prior to the impl ementation of the research program. Results of the pilot study will be used to make modifications to the existing processes, procedures, and assessment instruments prior to developing a full-scale program evaluation study. Data collected for each participant included a de mographics information sheet; a knowledge preand posttest of the basic principles of positive behavior support, behaviorism, and child development; behavior rating scales (CBCL and ABAS-II); weekly progress monitoring forms for caregivers home use of parenting techniques; and a program evaluation survey on caregivers perceptions of the usefulness and effectiveness of the program. Caregivers completed the Demographics Form and the Knowledge Pretest during the first session. Caregivers were also given the appropr iate behavior rating scales according to the age of the targeted child during the first session and were asked to
57 complete and return the forms the next week. Caregivers completed individual Tip Tracker sheets each week for the seven days following the training day. Tip Tracker sheets were completed for each of the firs t through the fifth sessions. Caregivers completed the Knowledge Posttest during the final session of training. To supplement the quantitativ e data collected, qualitativ e data were collected in the form of open-ended ques tions and prompts on the Program Evaluation Survey administered during the final parent training session. Guid ed response questions on the program evaluation survey prompted caregivers to respond to the following: 1) usefulness of information learned in the program; 2) sharing of information learned in the program with others; 3) possible improveme nts to the training program; 4) aspects of the program caregivers valued most; and 5) a ny suggestions for future parent trainings. A packet of behavior rating scales was mailed to each caregiver three months after completion of the parent training progr am to collect posttest data. A postage-paid envelope addressed to the HOT DOCS authors at the Child Development Clinic was included for return of the completed instru ments. Included in the packet was a letter detailing the request for information, a list of procedures for completing the instruments, and a description of how the information would be used as part of the research project. Caregivers also were informed that they would receive a follow-up phone call from the researchers to interpret the results of the behavioral assessments. Reminder postcards were mailed to participants who had not retu rned the behavior rating scales two weeks after the original mailing. Participants who had not returned the posttest behavioral assessments three weeks afte r the postcards were mailed, were called on the telephone and prompted to return the rating scales.
58 Data Analysis Thirty participants attended fewer than three sessions and were considered dropouts and an additional 13 participants comp leted the training program but did not sign the IRB consent form. Data from participan ts who did not complete three or more sessions or did not sign an IRB consent form will not be included in any of the analyses. The final sample consisted of 146 respondents. Measures of effect size were calculated to provide information about the strength of the relationship between the independent variable and the dependent variables (Stevens, 1999). Caregiver Knowledge Research Question #1. What is the impact of pa rticipation in the 6-week HOT DOCS parent training program on caregiver knowl edge as measured by preand posttest scores on the HOT DOCS Knowledge Test ? A dependent means t-test was conducted us ing each subjects pretest score and posttest score on the HOT DOCS Knowledge Test which is composed of 20 True/False items. Scores were reported as total number of items correct. Of the 146 participants analyzed in the demographics section a to tal of 112 participants completed both the pretest and posttest, attended three or more sessions, and signed the IRB consent form. Thirty-four participants completed either the pretest or the posttest, but did not complete both. Caregiver Perceptions of Severity of Child Behavior Research Question #2. Do caregivers perceive their child as having more problem behavior than a normative sample prior to pa rticipation in the 6week parent training program?
59 Descriptive statistics were used to an alyze the severity le vels of caregiver perceptions of child problem behavior prio r to participating in the parent training program. Caregiver ratings on the CBCL were used as indicators of problem behaviors in children. Caregiver ratin gs were analyzed using the de scriptive categories assigned to specific score ranges as desi gnated in the CBCL manual. Number and percent of standard scor es falling within the non-significant, borderline, and clini cally significant categories were ca lculated for the Internalizing, Externalizing, and Total Problem s scales of the CBCL. To analyze caregiver perceptions of child problem behavior, a chi-square goodness of fit analysis was calculated using the observed number of scores in the sample in the Non-Significant (T-scores less than 65), Borderline (T-scores between 65 and 69), a nd Clinically Significant (T-scores greater than or equal to 70) categorie s on the Internalizing and Exte rnalizing scales of the CBCL and the expected number of scores in each of the three descriptive categories as predicted for a normal distribution of scores in a na tional sample. One-hundred-one participants completed the CBCL rating scale at pretest and were included in the calculations used to answer research question #2. Research Question #3. Do caregivers perceive thei r child as havi ng less adaptive behavior than a normative sample prior to pa rticipation in the 6week parent training program? Descriptive statistics were used to an alyze the severity le vels of caregiver perceptions of child adaptive behavior prio r to participating in the parent training program. Caregiver ratings on the ABAS-II were used as indicators of adaptive behavior in children. Caregiver ratings were analyzed using the descri ptive categories assigned to
60 specific score ranges as designated in th e ABAS-II manual. Number and percent of standard scores falling within the non-signi ficant, borderline, and clinically significant categories were calculated for the Conceptu al, Social, Practical, and Global Adaptive Composite scales of the ABAS-II. To analyze caregiver perceptions of adaptive behavior, a chi-square goodness of fit analysis was calcu lated using the observed number of scores in the sample in the Non-Significant (T-scores gr eater than or equal to 80), Borderline (Tscores between 70 and 79), and Clinically Significant (T-scores less than 69) categories on the Conceptual, Social, and Practical scal es of the ABAS-II and the expected number of scores in each of the three descriptive categories based on the normed distribution in the national sample. One-hundred-six particip ants completed the ABAS-II rating scale at pretest and were included in the calculations used to answer research question #3. Changes in Child Problem & Adaptive Behavior Research Question #4. To what extent do caregivers perceive a decrease in child problem behavior following caregiver partic ipation in the 6-week parent-training program? A two-factor repeated measures ANOVA was conducted to analyze the differences between subjects pretest and posttest scores on th e Internalizing and Externalizing scales on the CBCL The two within-subjects (re peated) factors were type of scale (A) (i.e., Internalizi ng and Externalizing) and time (T ) (i.e., pretest and posttest) as shown in the data matrix in Table 10 belo w. Twenty-eight participants completed and returned both pretest and posttest CBCL rating scales.
61 Table 10 Data Matrix for Two-Factor Repeated Measures Design for Problem Behavior Type of Scale (A) Internalizing Externalizing Subject (S) Pretest Posttest Pretest Posttest S1 X111 X112 X121 X122 S2 X211 X212 X221 X222 Sn Xn11 Xn12 Xn21 Xn22 Research Question #5. To what extent do caregivers perceive an increase in child adaptive behavior following caregiver partic ipation in the 6-week parent-training program? A two-factor repeated measures ANOVA was conducted to analyze the differences between subjects pretest and pos ttest scores on the Conceptual, Social, and Practical scales on the ABAS-II. The two within -subjects (repeated) factors were type of scale (A) (i.e., Conceptual, Social, and Practi cal) and time (T) (i.e., pretest and posttest) as shown in the data matrix in Table 11 belo w. Twenty-seven participants completed and returned both pretest and posttest ABAS-II rating scales. Table 11 Data Matrix for Two-Factor Repeated Measures Design for Adaptive Behavior Type of Scale (A) Conceptual Social Practical Subject (S) Pretest Posttest Pretest Posttest Pretest Posttest S1 X111 X112 X121 X122 X131 X132 S2 X211 X212 X221 X222 X231 X232 Sn Xn11 Xn12 Xn21 Xn22 Xn31 Xn32
62 Caregiver Skills at Home Research Question #6a. What is the frequency and ease of use of the weekly parenting tips as reported by caregivers? The frequency of use per week of each parenting skill was computed from the weekly Tip Tracker forms. To determine ease of use, the overall mean caregiver rating of reported ease or difficulty of use of each skill was computed. Mean, maximum, median, and standard deviation of ratings was reported. Research Question 6b. Is there a relation between frequency of use and ease of use as measured by the HOT DOCS Tip Tracker sheets? To determine if there was a relationship between frequency of use and ease of use, zero-order correlations between number of days used and average difficulty rating per week were calculated. An average of 63% of participants returned completed Tip Tracker forms each of the five weeks homework was assigned. Each weeks data were analyzed separately as a different skill was assigned each week. Caregivers Overall Perceptions of the HOT DOCS Program Research Question #7. What are caregivers ove rall perceptions of the HOT DOCS parent training program as measured by the HOT DOCS Program Evaluation Survey ? Caregivers mean ratings of satisfaction with the HOT DOCS program were computed using quantitative data obtained from the HOT DOCS Program Evaluation Survey Thematic analyses of caregiver respon ses to open-ended questions and prompts were conducted for items #1, 3, 4, 5, and 6. Part icipant responses were systematically coded as individual thought units and then th emes were identified in order to identify
63 similarities and differences for each question or prompt. Codes and categories used to analyze the data were derived directly from the available data rather than searching for and coding concepts derived from existi ng sources (Gall, Gall, & Borg, 2007). Onehundred-fourteen participants completed the Program Evaluation Survey Refer to Appendix B for a visual repres entation of data sources for each research question.
64 CHAPTER 4 Results Overview The following chapter presents results of various data analyses used to answer each research question. Results are organized by research question. Caregiver Knowledge Research Question #1. What is the impact of pa rticipation in the 6-week HOT DOCS parent training program on caregiver knowl edge as measured by preand posttest scores on the HOT DOCS Knowledge Test ? A dependent means t-test was calculated between subjects pretest and posttest scores on the HOT DOCS Knowledge Test Means and standard de viations of pretest and posttest scores of caregivers knowledge are reported in Table 12. To determine if there was a significant difference betw een preand posttest scores on the Knowledge Test data were subjected to a dependent means ttest. The results of the t-test show that the participants mean posttest score was significantly higher than the participants mean pretest score, t(1,111) = 8.45, p<.001. The effect size for the t-test was large ( d = 1.13). Table 12 Means and Standard Deviations for Part icipant Scores on the Knowledge Test Measure M SD Minimum Maximum Skewness Kurtosis Pre-Test 16.03 1.92 11 19 -.682 .063 Post-Test 17.34 1.50 13 20 -.436 .077 Note. : n = 112
65 Caregiver Perceptions of Severity of Child Behavior Research Question #2. Do caregivers perceive their child as having more problem behavior than a normative sample prior to pa rticipation in the 6week parent training program? In order to describe and analyze caregiver perceptions of the severity of child problem behaviors before participation in the program, the frequency and percent of caregiver ratings of child beha vior falling within specific descriptive categories on the CBCL administered at pretest were calculated. Frequencies and percents were calculated using the Internalizing and Externalizing Pr oblems T-scores. The fr equencies of scores falling within these ranges were compared to th e number of scores expected to fall within each category according to the perc entages under the normal curve. On the CBCL, scores classified as normal or Non-Significant ranged from 0 to 64; scores classified as Borderline ranged from 65 to 70; and scores classified as Clinically Significant are those reaching 70 and above. The normal curv e predicts that 93.94% of scores will fall within the Non-Significant rang e, 3.79% of scores will fall within the Borderline range, and 2.27% of scores will fall within the Clinically Significant range for the CBCL. Chi-square analyses were cal culated between obser ved and expected frequencies of scores in each descriptive ca tegory for scores in the Internalizing and Externalizing subscales. Refer to Table 13 for observed and expected frequency distributions for Internalizing subscale score comparisons and to Table 14 for Externalizing subscale score compar isons. The alpha-level used was = .01.
66 Table 13 Observed and Expected Frequencies fo r CBCL Internalizing Subscale T-Scores Category Observed f Expected f Non-Significant 57 94.880 Borderline 13 3.828 Clinically Significant 28 2.293 Note n = 101 A chi-square goodness of fit test was conducted ( = .01) using participants scores on the Internalizing subs cale. The resultant overall test was statistically significant, 2 (1, N = 101) = 252.24. A significant differe nce between the expected frequency of scores in each descriptive category and the actu al or obtained frequenc y of scores in each descriptive category for the CBCL Intern alizing subscale was found. Caregivers perceived children in the sample to have highe r frequencies of more severe internalizing problem behavior than would be expect ed for a normative sample. Specifically, significantly more childrens scores fell within the Clinically Significant and Borderline descriptive categories and si gnificantly fewer childrens scores fell within the NonSignificant descriptive category than were expected. Nearly twelve times the number of children expected to have scores in the C linically Significant range were found in the sample. Effect size was calculat ed to describe the strength of the relationship between the expected and obtained values. The effect size for the chi-square calc ulation for scores on the Internalizing subscale was large ( w = 1.508), indicating that the differences between participants perceptions of th e severity of child problem behavior and expectations for a normative sample were not only statistically si gnificant but also clinically meaningful.
67 Table 14 Observed and Expected Frequencies fo r CBCL Externalizing Subscale T-Scores Category Observed f Expected f Non-Significant 52 94.880 Borderline 14 3.828 Clinically Significant 32 2.293 Note. n = 101 A chi-square goodness of fit test was conducted ( = .01) using participants scores on the Externalizing subscale. The resultant overall test was statistically significant, 2 (1, N = 101) = 335.66. A significant difference between the expected frequency of scores in each descriptive categ ory and the actual or obtained frequency of scores in each descriptive category for the CBCL Externalizing subscale was found. Caregivers perceived children in the sample to have higher frequencies of more severe externalizing problem behavior than w ould be expected for a normative sample. Specifically, significantly more childrens scores fell within the Clinically Significant and Borderline descriptive categories and significantly fewer childrens scores fell within the Non-Significant descriptive category than were expected. The observed number of children in the sample whose Externalizing subscale scores fell within the Clinically Significant range was nearly fourteen times the number expected to fall within that range. Effect size was calculated to describe the strength of the relationship between the expected and obtained values. The effect size for the chi-square calc ulation for scores on the Internalizing subscale was large ( w = 1.823), indicating that the differences between participants perceptions of th e severity of child problem behavior and expectations for a normative sample were not only statistically sign ificant but also clinically meaningful.
68 A graphic comparison of observed and exp ected frequencies of T-scores for the Internalizing and Externalizing scales is shown in Figure 1. Research Question #3. Do caregivers perceive thei r child as havi ng less adaptive behavior than a normative sample prior to pa rticipation in the 6week parent training program? In order to describe and analyze parent pe rceptions of the severi ty of deficits in child adaptive behaviors before participation in the program, the frequency and percent of caregiver ratings of child be havior falling within specifi c descriptive categories on the ABAS-II administered at pretest were calculated. Frequencies and percents were calculated using the Conceptual, Social, a nd Practical domain standard scores. The frequencies of scores falling within these rang es were compared to the number of scores expected to fall within each category according to the percentages under the normal curve for a national normative sample. On the ABAS-II, scores classified as normal or NonSignificant ranged from 80 or a bove; scores classified as Borderline ranged from 70 to 79; and scores classified as Clinically Si gnificant ranged from 69 and below. The normal curve predicts that 91.1% of scores will fa ll within the Non-Significant range, 6.7% of scores will fall within the Borderline range and 2.2% of scores will fall within the Clinically Significant range for the ABAS-II. A chi-square goodness of fit test was calculated between observed and expected freq uencies of scores in each descriptive category for scores on the Conceptual, Social, and Practical domains. Refer to Table 15 for observed and expected distributions for Conceptual domain score comparisons, Table 16 for observed and expected distributions for Social domain score comparisons, and Table 17 for Practical doma in score comparisons.
71 95 4 2 57 13 28 52 14 320 10 20 30 40 50 60 70 80 90 100123Number of scores Expected frequency Observed Internalizing scale Observed Externalizing scale Non-Significant Borderline Clinically Significant Figure 1. Number of expected and observed CBCL T-scores by descriptive category. Note n = 101
72 Table 15 Observed and Expected Frequencies fo r ABAS-II Conceptual Scale Scores Category Observed f Expected f Non-Significant 62 96.566 Borderline 16 7.102 Clinically Significant 28 2.332 Note n = 106 Chi-square critical values were obtained from a critical values table according to degrees of freedom (k-2). For the analyses conducted for scores on the Conceptual, Social, and Practical domains of the ABAS-II, the critical 2 (2, N = 106) was 5.99. The statistic obtained from chi-square analysis calc ulations was compared to the critical value from the table. The observed 2 for scores on the Conceptual domain was 306.04, indicating a significant difference between the expected frequency of scores in each descriptive category and the actual or obtained frequency of scores in each descriptive category. Effect size was calculated to describe the strength of the difference between the expected and obtained values. The effect size for the chi-square calc ulation for scores on the Conceptual domain was large ( w = 1.699). Table 16 Observed and Expected Frequencies for ABAS-II Social Scale Scores Category Observed f Expected f Non-Significant 55 96.566 Borderline 22 7.102 Clinically Significant 29 2.332 Note n = 106 The observed 2 for scores on the Social domain was 354.11, indicating a significant difference between the expected frequency of scores in each descriptive category and the actual or obt ained frequency of scores in each descriptive category.
73 Effect size was calculated to describe the stre ngth of the difference between the expected and obtained values. The effect size for the chi-square calculation for scores on the Conceptual domain was large ( w = 1.823). Table 17 Chi Square Goodness of Fit Test fo r ABAS-II Practical Scale Scores Category Observed f Expected f Non-Significant 47 93.833 Borderline 21 6.901 Clinically Significant 35 2.266 Note n = 103 The observed 2 for scores on the Practical domain was 525.04, indicating a significant difference between the expected frequency of scores in each descriptive category and the actual or obt ained frequency of scores in each descriptive category. Effect size was calculated to describe the stre ngth of the difference between the expected and obtained values. The effect size for the chi-square calculation for scores on the Conceptual domain was large ( w = 2.258). A graphic comparison of observed and expected frequencies of standard scores for the Conceptual, Social, and Practical scales is shown in Figure 2.
74 97 7 2 62 16 28 55 22 29 47 21 350 10 20 30 40 50 60 70 80 90 100123Number of scores Expected frequencies Observed Conceptual scale Observed Social scale Observed Practical scale Non-Si g nificant Borderline Clinicall y Si g nificantFigure 2. Number of observed and expected ABAS -II standard scores by descriptive category. Note n = 106
75 Changes in Child Problem & Adaptive Behavior Research Question #4. To what extent do caregivers perceive a decrease in child problem behavior following caregiver partic ipation in the 6-week parent-training program? In order to analyze potentia l changes in the severity of child problem behavior as perceived by caregivers from pretest and pos ttest, a two-factor repeated measures analysis of variance (ANOVA) was computed. M eans and standard deviations of pretest and posttest rating scale scores on the two subscales of the CBCL are reported in Table 18. Table 18 Means and Standard Deviations of Pr eand Posttest CBCL Scores by Scale Pretest Posttest CBCL Scales M SD M SD Internalizing 56.23 11.29 52.77 10.85 Externalizing 59.79 12.46 54.23 11.78 Marginal Means 57.55 53.64 Note n = 28 The two within-subjects fact ors were type of scal e, A (Internalizing and Externalizing) and time, T (pre test and posttest). As shown in Table 19, results revealed a non-significant interaction effect (p>.05), a st atistically significant main effect for time, F(1, 27) = 8.489, p<.01, and a non-significant main effect for scale (p>.05).
76 Table 19 Analysis of Variance of CBCL Preand Posttest Scores Source df SS MS F Scale (A) 1 299.01 299.01 3.530 Time (T) 1 428.22 428.22 8.489* Subject (S) 27 9206.17 340.97 A x T 1 55.72 55.72 3.683 S x A (Scale Error) 27 2287.24 84.71 S x T (Time Error) 27 1362.03 50.45 SAT (Residual) 27 408.53 15.13 Total 111 14046.92 Note *p<.05, ***p<.001 Follow-up of the significant main effect for Time (T), was done by examining the overall CBCL pretest and posttest mean scores (i.e., marginal means). The mean posttest score (M = 53.64) was significantly lower than the mean pretest score (M = 57.55). This finding indicates that caregive rs perceived severity of ch ildrens problem behavior was greater at pretest time as compared to postte st time. On the CBCL, higher scores indicate more severe levels of problem behavior; theref ore, a decrease in sc ores from pretest to posttest indicates caregivers perceived children to have significantly less severe levels of problem behavior following participation in th e program. Refer to Figure 3 for a graphic representation of the pretes t and posttest mean scores for the Internalizing and Externalizing scales of the CBCL.
77 45 50 55 60 65 70 7512Standard Score Clinically Significant Borderline N on-Significant Pretest Marginal Mean Posttest Marginal MeanSignificant difference between p retest and posttest mean scores Figure 3. Preand posttest mean scores for CBCL scales.
78 Research Question #5. To what extent do caregivers perceive an increase in child adaptive behavior following caregiver partic ipation in the 6-week parent-training program? In order to analyze potentia l changes in the severity of deficits in child adaptive behavior as perceived by car egivers from pretest to postt est, a two-factor repeated measures analysis of variance (ANOVA) was computed. Means and standard deviations of pretest and posttest rating scale scores on the three subscales of the ABAS-II are reported in Table 20. Table 20 Means and Standard Deviations of Preand Posttest ABAS-II Scores by Scale Pre Post ABAS-II M SD M SD Conceptual 81.74 19.93 87.12 18.95 Social 79.78 20.40 83.76 20.93 Practical 76.08 17.02 78.00 17.52 GAC 78.58 19.20 81.29 20.45 Note n = 27 The two within-subjects fact ors were type of scale, A (Conceptual, Social, Practical) and time, T (pretest, posttest). As shown in Table 21, results revealed a nonsignificant interaction effect (p>.05) and a non-significant main effect for time (p>.05), indicating there were no signifi cant differences in scores from pretest to posttest. A statistically significant main effect for scale, F(2, 26) = 24.657, p<.001) was observed. Follow-up of the significant scale main effect was conducted using Tukeys posthoc test. Results indicated a significant di fference in the mean scores between the Conceptual and Practical mean scale scor es. No significant differences were found between the Conceptual and Social scales or the Practical and Social scales.
79 Table 21 Analysis of Variance of ABAS -II Preand Posttest Scores Source df SS MS F Scale (A) 1 2760.33 2760.33 24.657* Time (T) 1 777.93 777.93 3.313 Subject (S) 26 50195.00 1930.58 A x T 1 7.26 7.26 0.181 S x A (Scale Error) 26 2910.67 111.95 S x T (Time Error) 26 6104.24 234.78 SAT (Residual) 26 1040.74 40.03 Total 161 63796.17 Note *p <.001 These findings indicate that while care givers perceptions of the level of childrens adaptive behavior di d not significantly change foll owing participation in the program, caregivers perceptions of children s adaptive behavior across each of the three scales of the ABAS-II were significantly differe nt from one another. At both pretest and posttest, the majority of caregivers reported ch ildrens scores on the Conceptual scale to be highest and scores on the Practical scale to be the lowest, with scores on the Social scale falling between the two other scales. On the ABAS-II, higher scores indicate more advanced development of adaptive skills; therefore, caregivers perceived childrens Conceptual skills to be the mo st superior adaptive skill ar ea, followed by Social skills, and Practical skills to be the least develope d skill area. Adaptive skills measured within the Conceptual scale included communicati on, functional academics/pre-academics, and self-direction. Adaptive skills measured within the Social scale included leisure and social interaction. Adaptive skills meas ured within the Prac tical scale included community use, home living, health and safet y, and self-care. Refer to Figure 4 for a graphic representation of the pretest and postt est mean scores for the Conceptual, Social, and Practical scales of the ABAS-II.
81 65 70 75 80 85 9012 St an d ar d S core Pretest Mean Score Posttest Mean Score Conceptual Domain Social Domain Practical Domain N on-Significant Borderline Clinically Significant Significant differences between scale mean scores Figure 4. Preand posttest mean scores for ABAS-II scales.
82 Caregiver Skills at Home Research Question #6a. What is the frequency and ease of use of the weekly parenting tips as reported by caregivers? Each participants responses on the w eekly progress monitoring forms were analyzed. Average daily rating of reported difficulty for each of the five skills was computed. Means and standard deviations of the number of days per week caregivers reported using each of the skills is reporte d in Table 22. A graphic display of these ratings is presented in Figure 5. Descriptive statistics and correlations were computed for data from each week separately. A visual analysis of the graph displaying average daily ratings of ease or difficulty of use indicated differential caregiver ra tings of ease of use across the five skills Caregivers rated Catch Them Being Good as being the easiest skill to implement at home, followed by Use Preventions Use Calm Voice Follow Through and Use Positive Words Each participants responses on the w eekly progress monitoring forms were analyzed. Average daily rating of reported difficulty for each of the five skills was computed. Means and standard deviations of the number of days per week caregivers reported using each of the skills is reporte d in Table 22. A graphic display of these ratings is presented in Figure 5. Descriptive statistics and correlations were computed for data from each week separately. A visual analysis of the graph displaying average daily ratings of ease or difficulty of use indicated differential caregiver ratings of ease of use across the five skills Caregivers rated Catch Them Being Good as being the easiest skill to implement at home, followed by Use Preventions Use Calm Voice Follow Through and Use Positive Words
83 Four of the five skills followed a distin ct pattern of ease or difficulty of use. Specifically, caregivers reported the skill as be ing easy to use on the first two or three days of the week, followed by a mid-week p eak in difficulty of use, and finally a reduction in difficulty for the final two or th ree days of the week. The skills that most clearly followed this pattern included Use Preventions Follow Through and Use Calm Voice Catch Them Being Good also followed the pattern, but with a less dramatic peak in difficulty. Caregiver ratings for Use Positive Words did not follow this pattern. For this skill, caregivers rated the sk ill as initially being more difficult to implement and progressively getting easier through the week. Table 22 Average Daily Parent Ratings of Ease or Difficulty of Skill Use at Home Day Positive Words Catch Them Being Good Calm Voice Use Prevention Follow Through Day 1 2.58 3.22 2.89 2.87 2.79 Day 2 2.69 3.29 2.92 3.13 2.95 Day 3 2.66 3.26 3.03 3.11 2.95 Day 4 2.76 3.23 2.93 2.89 2.81 Day 5 2.82 3.29 3.10 3.06 2.91 Day 6 2.91 3.41 3.10 3.25 3.06 Day 7 2.96 3.50 3.10 3.27 3.08 Number 106 102 93 83 73 Note Response scale: 1-Very difficult, 2-Diffi cult, 3-Neither Difficu lt nor easy, 4-Very easy.
83 2.5 2.6 2.7 2.8 2.9 3 3.1 3.2 3.3 3.4 3.5 Day 1Day 2Day 3Day 4Day 5Day 6Day 7Difficulty RatingMore Easy More Difficult Catch Them Being Good Use Preventions Calm Voice Follow Through Positive WordsFigure 5. Average Daily Parent Ratings of Ease or Difficulty of Skill Use at Home
84 Research Question 6b. Is there a relation between frequency of use and ease of use as measured by the HOT DOCS Tip Tracker sheets? To determine if there was a relationshi p between ease or difficulty of use and frequency of use, zero-order correlations we re computed between the number of days participants reported using the skill and the average difficulty rating participants reported for each weekly parenting skill. Refer to Table 23 for descriptive statistics and correlations. As is shown, the relationship be tween frequency o fuse and ease of use was not statistically signif icant (p>.05) for any of the five skills. Table 23 Descriptive Statistics and Correlations for W eekly Skill Use and Ratings of Difficulty Days Used Correlations Parenting Skill N M SD r p Use Positive Words 106 6.40 1.16 .03 .75 Catch Being Good 102 6.64 0.99 -.14 .17 Use Calm Voice 93 6.67 0.97 .06 .57 Use Preventions 83 6.60 0.81 .17 .13 Follow Through 73 6.42 1.18 .07 .54 Caregivers Overall Perceptions of the HOT DOCS Program Research Question #7. What are caregivers ove rall perceptions of the HOT DOCS parent training program as measured by the HOT DOCS Program Evaluation Survey ? Caregivers ratings of satisfaction with the HOT DOCS program were analyzed using descriptive statistics. In addition, a them atic analysis of caregiver responses to freeresponse questions and prompts was conducted. The codes and categories used to analyze the free-response data were deri ved directly from the availabl e data rather than searching for and coding concepts derived from existing sources (Gall et al., 2007). A total of 114
85 caregivers completed the Program Evaluation Survey Research question #7 was answered through the use of two-stage quant itative-qualitative anal yses (Onwuegbuzie & Teddlie, 2002). In the first stage, caregiv ers responses were analyzed using a phenomenological approach, in which responses were systemati cally coded by the resear cher as individual thought units in order to identify similarities and differences for each question or prompt and then grouped using inductive reasoning to identify themes and generate a conceptual framework to interpret the existing data. In the second stage, each of the derived themes was quantitized using endorsement rates indica ting the percent of participants endorsing a given theme for each item (Minor, Onweugbuzie Witcher, & James, 2002). Endorsement rates were calculated by assigning a value of 1 to each participant whose response represented the theme and a 0 to each part icipant whose response did not include a thought unit representing the theme. For each th eme, the total number of 1s was divided by the total number of particip ants responding to the item. In order to ensure intercoder agreement, a graduate student not involved in the HOT DOCS parent training program was recruited to code the free-response items according to the thematic categories identified by the primary researcher. Ac ross the various free-response items on the program evaluation survey the overall intercoder reliability between the primary researcher and the independent coder was approximately 87%. Descriptive Analysis of Quantitative Data As shown in Table 24, the overall ma jority of partic ipants (97.4%) Agreed or Strongly Agreed that the HOT DOCS program met their expecta tions. More specifically, participants Agreed or Strongly Agreed that the program was beneficial to their families
86 (97.4%), the trainers were knowledgeable and effective instructors (100%), the parenting tips were beneficial (95.6%), the special play strategies promoted positive interactions with children (97.4%), that the program positively impacted parenting attitudes and practices (95.6%), and that the program positiv ely impacted childrens behavior (94.7%). Of the eight statements used to gauge pa rticipants perceptions of the usefulness of the program, only three statements were marked as Strongly Disagree by one participant each. In gene ral, these three statements all rela ted to the caregivers ability to implement parenting strategies presented in class, changes in ch ildrens behavior at home, and the participants overa ll evaluation of the program. Th ese data indicate that for one caregiver, this level of intervention was not matched appropriately to the level of severity of problem behavior the child demons trated in the home. The highest percentage of responses endorsed by car egivers as being in the Disagree or Strongly Disagree categories were on items related to caregivers ability to effectively implement program strategies in the home and the subsequent lack of improvement in child behavior following participation in the program.
87 Table 24 Ratings of Participant Satisfaction with the HOT DOCS Training Program Strongly Agree Agree Disagree Strongly Disagree n % n % n % n % The HOT DOCS program was beneficial to my family 89 78 22 19 3 3 0 0 The presenter(s) were knowledgeable and effective in communicating this topic 106 93 8 7 0 0 0 0 I am able to utilize these strategies with my children 88 77 22 19 3 3 1 1 The Parenting Tips are beneficial to me 92 82 17 15 3 3 0 0 The Special Play Activities promoted interactions with my child 71 63 40 36 1 1 0 0 The information I learned in HOT DOCS has changed my parenting practices 73 64 36 32 4 4 0 0 HOT DOCS strategies have positively impacted my childs behavior 70 62 38 34 4 4 1 1 Overall, the HOT DOCS program met my expectations 81 71 30 26 2 2 1 1 Note. : N = 114
88 Thematic Analysis of Free-Response Data Table 25 contains the themes that em erged from caregivers answers to freeresponse question #1 on the su rvey, How are you using the information you learned in HOT DOCS? Ten participants handed in a surve y, but did not respond to this particular item, resulting in an N of 104 for this item. As is shown, the following themes emerged from caregivers responses: use of a specifi c skill, problem solving behavior, sharing information with others, improved relationshi p with child/others, change in parenting attitude, and improved communication with child/others. Table 25 How are you using the information you learned in HOT DOCS? Response Theme/Category Frequency Endorsement Rate % Use of a specific skill 73 70.2 Problem solving behavior 16 15.4 Share information with others 14 13.5 Improved relationship with child/others 9 8.7 Change in parenting attitude 5 4.8 Improved communication with child/others 2 1.9 Note n = 104 The majority of participan ts (70.2%) endorsed responses falling within the theme of using a specific skill. Verbatim responses of using a specific sk ill include doing my best to apply what I learne d as often as possible, we are using these methods to change/prevent negative behavior, we have used calm voice and positive words and it does work, to teach them how to do routin es and rituals, we mostly use prevention techniques, and implemented a timer at be d time and gave warnings. Refer to Table 26 for sample verbatim responses from other themes identified for this item.
89 Table 26 Sample Verbatim Responses for Item #1: How Are You Using Information You Learned in HOT DOCS? Response Theme/Category Example Parent Response Use of a specific skill Doing my best to apply what I learned as often as possible We are using these methods to change/prevent negative behavior We have used calm voice, positive words and it does work To teach them how to do routines and rituals We mostly use prev ention techniques Implemented a timer at bed time and gave warnings Problem solving behavior I am better ab le to deal with problem behavior by understanding why Using behavior chart to help deal with childs defiant behavior To problem solve my childs behavior We use the charts, look for tr iggers, identify behavior function and consequences Share information with others I am also sharing this information with family, friends, and peers I am the grandfather, do not liv e with child, but have been able to guide my daughter in dealing with him I have talked to a lot of people about this Teaching it to my interns Improved relationship with child/others To have a calmer, happier home Helped me to interact in a positive way with my son To make me better in the way I interact with my children This has really changed the way I parent Change in parenting attitude We have changed our attitudes as parents To feel more in control To help reduce levels of frustration Totally changed the way I see my son Improved communication with child/others Communication with my daughter has improved as we share what we are learning Improving communication betw een myself and my son Note: n = 104
90 Table 27 contains the frequencies and per cents of responses that emerged from caregivers answers to question #2 on the survey, Have you shared information from HOT DOCS? Participants were instructed to check the boxes of all people with whom they had shared information. Approximately 95% of participants (n = 109) indicated that they had shared information from HOT DOCS with their spouse, friends, and/or other family members. Approximately 25% of the participants (n = 29) indicated sharing information with a professional, such as an early interventio nist, therapist, or teacher. Less than 15% of participants (n = 17) reported sharing information with someone other than the options listed. Verbatim examples of other people with whom participants shared information included el ementary school admini strators, clients of child welfare system who are reunited with ch ildren, case worker for foster children, coworkers, youth at church program, and parent s of children at a daycare class. Less than 10% of participants (n = 7) indicated that they had shared information from HOT DOCS with their pediatrician. Table 27 Have you shared the information from HOT DOCS with? Check all that apply. Frequency Percent Spouse or partner 80 70.2 Friends 76 66.7 Other family members 69 60.5 Interventionist, therap ist, or teacher 29 25.4 Other 17 14.9 Pediatrician 7 6.1 Note n = 114 Table 28 contains the themes that emerge d from caregivers responses to prompt #3 on the survey, If you have shared information from HOT DOCS with others, please describe how they have benefited from this information. A total of 86 participants
91 responded to this item. As is shown, the following themes emerged from caregivers responses: increased knowledge use of specific skills, no knowledge of benefits to others, others wanting to enroll in HOT DOCS, support or positive feedback for caregivers use of new parenting skills, others wanting more information. Table 28 If you have shared information from HOT DOCS with others, please describe how they have benefited from this information? Response Theme/Category Frequency Endorsement Rate % Increased knowledge 29 33.7 Use of specific skills 23 26.7 Dont know how others have benefited 13 15.1 Want to enroll in HOT DOCS 12 14.0 Support or positive feedback for new parenting skills 10 11.6 Want more information 7 8.1 Note n = 86 Approximately 33% of participants (n = 29) reported that those with whom they shared information benefited by increasi ng their knowledge. Approximately 25% of participants (n = 23) reported others bene fiting by learning and using specific skills. Verbatim responses of increased knowledge included: the teacher likes to get new information to use with the class, friends ask what we learned each week, my daughter is using some information in the schoo l setting, and it helped my parents with their interaction with my son. Verbatim res ponses of using a specific skill included: my Mom tried to use calm voice and positive words, my husband has learned to control his emotions and stay calm, they love the ti ps, more aware of negative words, using timer for taking turns with siblings. Refe r to Table 29 for sample verbatim responses from other themes identified for this item.
92 Table 29 Sample Verbatim Responses for Item #3: De scribe How Others Have Benefited from HOT DOCS Information You Shared. Response Theme/Category Example parent response Increased knowledge The teacher likes to get new information to use with the class Friends ask what we learned each week My daughter is using some information in the school setting It helped my parents with their interaction with my son Use of specific skills My Mom tried to use calm voice and positive words My husband has learned to control his emotions and stay calm They love the tips More aware of negative words, using timer for taking turns with siblings Dont know how others have benefited Too soon to tell I dont know if anything has been utilized I am unable to determine the benefits Not sure Want to enroll in HOT DOCS They called to sign up for class They would like to come to the program, even those without children They called to sign up for the class My 3 friends are going to come to HOT DOCS Support or positive feedback for new parenting skills They give me positive feedback Has helped them to reinforce what were trying to do at home My spouse and I talk about how things work or dont work Friends who have offered our family support Want more information Will look into taking a class for special needs child They get excited and want information They show interest and curiosity It gave them cause for pause and they have asked more questions about my school Note: n = 86
93 Table 30 contains the themes that emer ged from caregivers answers to freeresponse question #4 on the surve y, What can we do to improve HOT DOCS? A total of 87 participants responded to this item. The following themes emerged from caregivers responses: nothing/fine as it is, more time fo r instruction/training, changes to location or scheduling, train more people, more video vi gnettes/scenarios/exampl es, changes to food, specify training by childs age or disability, involve children/families, and provide additional resources. Table 30 What can we do to improve HOT DOCS? Response Theme/Category Frequency Endorsement Rate % Nothing, fine as is 36 41.4 More time 20 23.0 Changes to location, scheduling 9 10.3 Train more people 8 9.2 More video vignettes, scenarios, examples 7 8.0 Changes to food 5 5.7 Specify training by childs age or disability 4 4.6 Involve children/families 3 3.4 Provide additional resources 3 3.4 Note n = 87 Approximately 40% of the participants who responded (n = 36) indicated that no changes to the HOT DOCS program were necessary. Verbatim responses within this theme included: keep up the good work, dont change a thing, its perfect, not much room for improvement, really cant think of anything at the moment, Im really happy with how the course went, and the program is excellent at the moment. Refer to Table 31 for sample verbatim responses from other themes identified for this item.
94 Table 31 Sample Verbatim Responses for Item #4: What Can We Do To Improve HOT DOCS? Response Theme/Category Example parent response Nothing, fine as is Keep up the good work We really appreciate your time and training More time Make-up classes Add more hours so that we can get more in-depth Group class follow-up 3 to 6 months later Changes to location, scheduling Classes in south Tampa Wanted class during the day so my wife could attend Bring HOT DOCS to a community center or school location Train more people Come to school s and teach EEIP and ASD teachers Train more students so more classes can be offered More video vignettes, scenarios, examples Bring more videos of successful parents Have us videotape a typical da y and use it in class to allow group to analyze behavior Changes to food Keep cookies away! Offer a variety of meals each week Better drinks and softer bread for sandwiches Specify training by childs age or disability Felt the course was geared for younger children Majority of topics seemed to be directed to individuals with the ability to communicate Involve children/families After the course I want parents and children to meet Give the children the opport unity to come to class Provide additional resources Handout any valuable websites, like for healthier snacks or support groups Give more material about other programs like TEACCH Miscellaneous (responses given by only one participant) Offer email to address questions during course Offer way to receive additiona l support if needed after course ends Provide child care Give focus to healthy punishments Note: n = 87
95 Table 32 contains the themes that emer ged from caregivers answers to freeresponse question #5 on the survey, Wha t did you value most ? A total of 102 participants responded to this item. The following themes emerged from caregivers responses: acquiring skills; support and inter action with other careg ivers; instructors knowledge, attitude, and support; provision of materials; problem solving skills; homework, weekly review, and validation of current parenting sk ills and abilities. Table 32 What did you value most? Response Theme/Category Frequency Endorsement Rate % Acquiring specific skills 63 61.8 Support and interaction with other caregivers 20 19.6 Instructors knowledge, attitude, support 14 13.7 Provision of materials 13 12.7 Problem solving skills 13 12.7 Homework, review weekly 5 4.9 Validation of current pare nting skills/abilities 4 3.9 Note n = 102 The majority of the participants who responded to this item (62%) indicated that they valued specific skills they acquired the most. Verbatim responses within this theme included: teaching about calmness and timers, activities each week, teach my son positive words, I learned new techniques that really worked, the preventions that I can put in place to hopefully avoid melt downs and behavior problems. Refer to Table 33 for sample verbatim responses from other themes identified for this item.
96 Table 33 Sample Verbatim Responses to Question #5: What Did You Value Most? Response Theme/Category Example parent response Acquiring specific skills Teaching about calmness and timers I learned new techniques that really worked Support and interaction with other caregivers Listening to other people shar e similar situations they are going through Positive support from others living in the same situation as our family Provision of materials Obtaining the materials for the special play times, a nice surprise Binder with notes The signs as a reminder of the sessions Problem solving skills Knowing that thei r behavior is to get or get out of something Learning about how to identify the function of behavior, triggers, consequences, etc. Validation of current parenting skills/abilities Confirmation of some techniques I was already using Learning that Im not doing a te rrible job, this all takes work Instructors knowledge, attitude, support How understanding the instructors were to the problems we were having Attitude of teachers Having a professi onal intervene Knowledgeable facilitators of the class Homework, weekly review Activities each week Tasks to do every week The review of homework Miscellaneous (responses given by only one participant) It was free Based on adult learning principles, not too much per session Relaxed atmosphere, structure of the class Course was very well linked, one step requires the next and so on Note: n = 102
97 Table 34 contains the themes that emer ged from caregivers answers to freeresponse question #6 on the survey, Wha t suggestions do you have for future HOT DOCS trainings? A total of 71 participants responded to this item. The following themes emerged from caregivers responses : more time for instruction; nothing, the program is fine as it is; specify training by ch ilds age or disability ; involve children and families; train other professionals; more movies, examples, and scenarios; provide additional resources; offer futu re support or contact methods; changes to food; more relaxation training; and change s to scheduling or location. Table 34 What suggestions do you have for future HOT DOCS trainings? Response Theme/Category Frequency Endorsement Rate % More time 21 29.6 Nothing, fine as is 18 25.4 Specify training by childs age or disability 7 9.9 Involve children/families 6 8.5 Train other professionals 6 8.5 More movies, examples, scenarios 4 5.6 Provide additional resources 3 4.2 Offer future support, contact methods 3 4.2 Changes to food 2 2.8 More relaxation training 2 2.8 Changes to scheduling, location 2 2.8 Note n = 71 Nearly 30% of participants (n = 21) indicated that the program would benefit from increasing the time for training and inst ruction. Approximately 25% of participants (n = 18) responded that no improveme nts can or should be made to HOT DOCS. Verbatim responses of more time for instruc tion included: add more hours of classes so that we can get more in depth, make classes day sessions, have more classes, and add another class several months later wh en suggestions are put into practice and
98 evaluate results. Verbatim responses of no recommended changes included: great job, not room for much improvement, and dont change it, doing a good job, I am really happy with how the course went. Refer to Ta ble 35 for sample verbatim responses from other themes identified for this item.
99 Table 35 Sample Verbatim Responses to Question #6: What Suggestions Do You Have for Future HOT DOCS Trainings? Response Theme/Category Example parent response More time Have more classes Add another class several months later Nothing, fine as is Great job, not room for much improvement Dont change it, doing a good job Specify training by childs age or disability More information about young, nonverbal children Break up the class with childre n with different problems Involve children/families Be able to bring children Reunions or classes for parents and children Train other professionals E xpand out to provide trainings to daycares and schools Doctors, teachers and day care workers should attend More movies, examples, scenarios Bring videos from home to share and analyze Bring more videos about successful parents Provide additional resources Have a lit of printed web s ites referenced for handouts Resources to take home Offer future support, contact methods Perhaps a way to continue to communicate with previous attendees, like a website or chat forum Do you have future support? 1-800-# or a website? Changes to food Change food every class If theres going to be cookies, maybe milk More relaxation training Do th e relaxation training nightly 20 minutes of relaxation instead of 10 Changes to scheduling, location Bring training to Brandon area Classes during the day Miscellaneous (responses given by only one participant) Speakers to hear videos playing Provide cue cards for parents to use at home Note n = 71
100 CHAPTER 5 Discussion Overview In response to the increasing number of young children displaying early-emerging challenging behavior, professiona ls have increased their efforts to find evidence-based interventions to address child and caregiver needs. Th e current study served as a preliminary investigation of caregivers perceptions of the e ffectiveness of the Helping Our Toddlers Developing Our Ch ildrens Skills (HOT DOCS) parent training program. The study evaluated the impact of specific co mponents of the parent training program on caregivers knowledge and attit udes and their perceptions of targeted childrens behavior. Results of the study will be used to modify and improve the HOT DOCS program. Demographic Characteristics Rates and Patterns of Caregiver Attendance Rates and patterns of caregiver attend ance and attrition were analyzed and compared with findings from previous studi es of group-delivered behavioral parent training. Overall patterns of attendance and rates of attrition found in this study were similar to those found in previ ous research (Eyberg et al., 200 1; Feinfield & Baker, 2004; Kazdin, 1997; Sanders et al., 2000). Of th e 189 caregivers attending the first of six sessions of HOT DOCS training, 146 completed the program (e.g., attended three or more sessions), resulting in an attrition rate of 23%. Eyberg and colleagues (2001) reported similar rates of attrit ion in an evaluation of the Parent-Child Interaction Therapy
101 (PCIT) intervention. Specifically, of the orig inal twenty participants, 13 completed the training, resulting in a 30% attrition rate. Fienfield and Baker (2004) reported lower levels of attrition in an evaluation of a mu ltimodal, manually guided group treatment for parents of children with challenging behavi or. Of the 56 caregivers enrolled in the program four dropped out of the treatment group and five dropped out of the waitlist control group, resulting in an overall attrition rate of 16%. Several previous studies of group parent training interventions have reported significantly lower attriti on rates than found in this study (B arkley et al., 2000; Reid et al., 2001; Webster-Stratton & Hammond, 1997). Several of the programs reporting low rates of caregiver drop -out have provided pa rticipants with incentives for attendance and completion of the program. For exam ple, in an evaluation of the Incredible Years parent training program, Reid, Webste r-Stratton, and Beauchaine (2 001) reported attrition rates of less than 10%. Parents part icipating in this study were given $50 for participation in each pre-, post-, and follow-up assess ment. Other training programs offered individualized, child-focused in tervention services to program completers (Barkley et al., 2000; Sanders, 1999), which seemed to serve as a non-tangible incentiv e for attendance. Attrition rates reported in st udies of early intervention ut ilizing the principles of positive behavior support (PBS) also have re ported lower rates of attrition than were found in this study (Buschbacher et al., 2004; Vaughn et al., 2005). However, rates of attrition in PBS intervention research should be interpreted with caution when comparing these studies with other inte rvention program research. Acco rding to results of a metaanalysis of PBS research, the majority of PB S interventions (85%) have been delivered in an individual, one-on-one format with a pa rent or caregiver and the interventionist
102 (Conroy et al., 2005). Other studies of interven tions based on PBS principles reporting no participant attrition were conducted using a single-subj ect design with teachers or daycare workers as part of their daily responsib ilities (Duda et al ., 2004; Fox & Little, 2001). Additionally, most PBS interventions are designed for the individual child or family. Consequently, the intervention pr ograms are designed to address the specific needs, concerns, and strengths of individual families and are not intended for delivery to multiple children or families at once. In summ ary, rates of attenda nce and attrition found in this study are comparable to other group-delivered, behavior al parent training programs with the exception of those studies providing incentives for participation. Comparison of Caregiver Demographics with Hillsborough County Demographics Demographic information for the caregiver s serving as particip ants in this study was compared with local demographic inform ation provided by the United States Census Bureau for Hillsborough County. According to the results of this study, the participant sample consisted of 15% fewer caregivers re porting their ethnicity as Caucasian (44% versus 59%), 11% fewer caregivers reporting th eir ethnicity as Black/African American (5.5% versus 16.3%), and 14% more caregiver s reporting their ethnicity as Hispanic (35% versus 21.2%) than adults residing in Hillsborough County in 2005 (United States Census Bureau, http://quickfacts.census.gov/qfd/states/12/12057.html). These results suggest that the HOT DOCS program provided early interven tion services to caregivers from a racial/ethnic group and SES categor y, which have been underserved by previous parenting programs, including Hispanic a nd/or Spanish-speaking caregivers and lowincome caregivers. However, these results also suggest a di sproportionately low percentage of Black/African American caregivers participating in the HOT DOCS
103 program. Preliminary analysis of the caregiv ers signing up to partic ipate in the program but not completing training (e.g., drop-outs) did not indicate differentia l rates of attrition for caregivers reporting their race/ethni city as Black/African American. The underrepresentation of Black/African American caregivers in the HOT DOCS program is likely related to the lack of families from this race/ethnic category who self-refer and/or are referred by professionals to part icipate. The high percentage of sample participants reporting their race/ethnicity as Hi spanic as compared to local norms is likely explained by the provision of HOT DOCS classes in Spanish. In terms of level of educa tion attained, participants in this study reported similar numbers of high school gradua tes (89% versus 81%), twice the number of college graduates (53% versus 25%), and three times the number of graduate degrees (31% versus 12%) according to census data from 2000 Previous studies of parenting programs have reported similar patterns of higher than expected educational attainment (Fienfield & Baker, 2004; Hartman, Stage, & Webs ter-Stratton, 2003). These studies have hypothesized that the higher mean educational levels may be explained by the additional financial and social supports available to families with higher levels of educational attainment. Researchers have suggested that these resources allow pa rents to participate in and complete training programs, while pare nts with lower educational attainment are often unable to attend and complete training sessions due to issues associated with socioeconomic status, such as lack of transportation, childcare, and time. The use of type of insurance as an indicat or for socioeconomic status (SES) in this study prohibits precise comparisons with lo cal population statistic s, which report SES using ranges of annual household income. However, general comparisons of the
104 proportion of the study sample reporting having Medicaid or no insurance, which were response categories used by the program devel opers to indicate low-SES, were compared with Hillsborough County estimates of adults falling below the poverty line (US Census Bureau, 2000). Approximately one-third, (31%) of HOT DOCS participants reported having no insurance or Medicaid insurance co mpared to 12% of adults in Hillsborough County classified as low-SES. Th is comparison indicates that the HOT DOCS parent training program was provided to a higher pe rcentage of low-SES families than would have occurred simply by chance. Since previ ous research has show n that children of parents who are considered low-SES or low-in come have a greater chance of developing more severe levels of challenging behavi or (Gross et al., 1999 ; Keenan & Wakschlag, 2000; Qi & Kaiser, 2003), the large proportion of participants falling within this category can be considered a positive finding. Comparison of Child and Caregiver De mographics with Previous Studies Demographic information for the caregivers serving as particip ants in this study also was compared with demographic information for participant samples from previous research of group parent training programs. Most of the existing research on parent training programs has focused on female caregiv ers, specifically mothers of children with problem behavior (Bagner & Eyberg, 2003; McNeill, Watson, Hennington, & Meeks, 2002; Phares, Fields, Kamboukos, & Lopez, 2005; Reid et al., 2001). The gender and relationship with target child of participants in this study differs notably from previous research on parent training interventions, sp ecifically by encouragi ng participation of fathers, non-related caregivers and professionals. Participan ts in the sample were 68% female and 32% male, including 54% mothers, 29% fathers, 8% professionals (i.e., early
105 interventionists, service coordi nators), and 7% grandparents. In comparison, research on the Incredible Years parent training program indicates th e majority of participants were mothers (98-100% mothers, small number of grandmother/aunt and fathers), including three studies with 100% female participan ts (Hartman et al., 2003; Reid, WebsterStratton, & Baydar, 2004; Reid et al., 2001). One study on the Incredible Years program did report a significant proportion (43%) of fathers as partic ipants (Webster-Stratton & Hammond, 1997). Research on the e ffectiveness of the PCIT pa rent training intervention has been conducted mainly with mothers or other female caregiver s (Boggs et al., 2004; Hood & Eyberg, 2003) with the exception of a study specifically designed to target fathers participation in PCIT (Bagner & Eyberg, 2003). The ma jority of studies using the targeting children with ADHD have not reporte d data specifying the gender of parents and caregivers participating in training pr ograms (Barkley et al., 2000; Weinberg, 1999). However, one study of the Defiant Childre n Parenting Program reported 100% of participants being mothers (A nastopoulos et al., 1993). In co ntrast to the majority of studies of behavioral parent training including the current study, investigations of the Triple P-Positive Parenting Program (Sanders, 1999) have reported participation by both parents of target children (B or, Sanders, & Markie-Dadds, 2002; Sanders et al., 2000). Research on parenting interventions using the principles of positive behavior support (PBS) have used mostly single subject designs conducted with mothers or female teachers (Conroy et al., 2005; Duda et al., 2004; McNeill et al., 2002). Participant race/ethnicity for this study was compared with demographic information from other parent tr aining programs, including the Incredible Years (Webster-Stratton, 2001), PCIT (Eyberg, 1988), Triple P-Positive Parenting Program
106 (Sanders, 1996), Defiant Children Parenting Program (Barkley et al., 2000) and others. Compared with participants completing othe r training programs, the caregiver sample completing HOT DOCS training was composed of fewe r White caregivers (44% versus an average of 51-98%) and more non-White car egivers (56% versus an average of 249%) (Barkley et al., 2000; Fienfield & Bake r, 2004; Sanders et al., 2004). Specifically, the HOT DOCS participant sample included nearly five times the percentage of Hispanic caregivers (35%) as previous studies. Similar to findings from the current study, one previous study of PCIT had a notably larger percenta ge of Hispanic participants compared to the majority of existing pare nting research (McCabe, Yeh, Garland, Lau, & Chavez, 2005). Just as HOT DOCS was translated to Spanish to increase Hispanic caregiver participation, McCa be and colleagues (2005) modified and translated the original PCIT program to meet the unique needs of Mexican-American families. Preexisting diagnoses of target children of participants in this study also were compared with demographic information from previous research. The majority (66%) of target children in this study did not have a preexisting medical, psychological, or behavioral diagnosis as reporte d by caregiver participants at the time of par ticipation. In contrast, the majority of pr evious studies of parent training programs have specified inclusion criteria requ iring that target children have preexisting mental, emotional or behavioral diagnoses to part icipate in study. Few published, evidence-based interventions target parents of children with non-clinical le vels of challenging beha vior (Lundahl et al., 2006; Maughan et al., 2005; Schumann et al ., 1998). Several investigations of the Incredible Years parent training program and several studies of PCIT specify that children must have preexisting diagnosis of Oppositional Defiant Disorder (ODD) and/or
107 Conduct Disorder (CD) (Harman, Stage, & Webster-Stratton, 2003; Webster-Stratton & Hammond; 1997). Parent training research c onducted by Barkley a nd colleagues (2000) stipulates that all children included in the st udies meet diagnostic criteria for ADHD. In the field of PBS, the majority of parent trai ning research has been conducted with parents of children with clinical dia gnoses of autism spectrum disord ers or intellectual deficits (Conroy et al., 2005). These fi ndings indicate that the HOT DOCS parent training program provided early intervention services as preventative measures for children exhibiting non-clinical levels of challenging behaviors. As indicated by several decades of research, intervention provided before cha llenging behaviors reach chronic and severe levels is more likely to effectively treat and prevent negative lifelong emotional and behavioral impact (Marchant et al., 2004; Wa lker et al., 1998; We bster-Stratton, 1998). Caregiver Knowledge Research Question 1. What is the impact of pa rticipation in the 6-week HOT DOCS parent training program on caregiver knowl edge as measured by preand posttest scores on the HOT DOCS Knowledge Test ? Results of this study indicat ed a significant increase in participants scores on the HOT DOCS Knowledge Test from pretest to posttest. A lthough the difference in mean score from pretest to posttest differed by fewe r than two correct answers, the effect size of the statistical difference was large, indi cating significant and mean ingful increases in the number of correct answers provided by participants. Several features of the HOT DOCS Knowledge Test prevent further interpretati on of the increase in scores. Specifically, due to the small number of items on the test (e.g., 20 items), the lack of reliability and validity data for the measure, and the lack of variation in response type
108 (e.g., all true/false), further analyses are re stricted. Despite these limitations, the items used on the test represent specific concepts, skills, or practices gui ded by the theoretical framework of the HOT DOCS parent training program. Knowing and understanding these skills and concepts may be consider ed ideal outcomes of the parent training program. Therefore, an increase in the numbe r of items correct ma y indicate successful delivery of skills and concepts. Changes in caregiver knowledge as indi cated by these results are similar to outcomes reported by previous research of pa rent training interventions (Anasopoulos et al., 1993; Weinberg, 1999). Anastopoulos and colleagues (1993) id entified changes in parent knowledge as a dependent variable in their investigation of a six-week parent training program for parents of children with ADHD. Results of their study also reported significant increases in parent knowledge from preto posttest us ing a knowledge test created by the researchers specifically for th is purpose. Weinberg (1999) also reported significant increases in parent knowledge of the features of ADHD and behavioral management strategies following participati on in a behavioral pa rent training program. Caregiver Perceptions of Severity of Child Behavior Research Question 2. Do caregivers perceive their child as having more problem behavior than a normative sample prior to pa rticipation in the 6week parent training program? Participants were expected to report high levels of pe rceived challenging behavior in target children. Expectati ons of high levels of problem behavior were based on the method of participant re cruitment. Caregivers either se lf-referred to the program after seeing community advertisements or hearing about the program from friends or were
109 referred to the program following a comprehe nsive psychoeducational evaluation of their child. Previous studies of parent training programs for children with challenging behavior have used parent reported data su ch as the Child Behavior Checklist (CBCL; Achenbach, 2001) and Eyberg Child Behavior Inventory (ECBI; Eyberg & Pincus, 1999). Many studies have cited inclusion criteria fo r participation in the study, stipulating that caregivers must have children who score in the clinically significant range on these measures (Bagner & Eyberg, 2003; Barkley et al., 2000; Harman, Stage, & WebsterStratton, 2003; Webster-Stratton & Hamm ond; 1997). Because many of the published studies of parent training pr ograms have inclusion criteria such as these, the overall frequencies of caregivers repo rted perceptions of child beha vior as being more severe and problematic is higher than expected fo r a normative sample of the general population. Although the current study did not base participant inclusion on pre-test beha vior rating scale scores, it was hypothesized that most of the caregivers seeking to participate in the program would report that their children had more severe levels of problem behavior than a normative sample. Results of this study suppor ted this hypothesis by indi cating that participants reported significantly more severe levels of child problem behavior at pre-test than was predicted for a normative sample of the populat ion. Statistical analyses revealed that nearly twelve times as many caregivers in th e participant sample perceived their childs problem behaviors to be within the clinica lly significant range on both the Internalizing and Externalizing subscales of the CBCL (A chenbach, 2001) than was expected given a normal distribution. These results indicate that the majority of caregivers who elected to
110 participate in HOT DOCS perceived their children as havi ng clinically significant levels of problem behavior prior to beginning the training program. Research Question 3. Do caregivers perceive their child as having less adaptive behavior than a normative sample prior to pa rticipation in the 6week parent training program? As with caregiver perceptions of severity levels of ch ild problem behavior, it was expected that caregivers would also perceive their children as having lower than expected levels of adaptive behavior. Although caregiver s often cite challengi ng behavior as their primary concern, children likely have comorb id deficits in adaptive or prosocial behaviors (Conroy et al., 2005). Despite the lack of available research using parent perceptions of childrens adap tive behavior as inclusion cr iteria or outcome measures, initial studies have indicated that high le vels of problem behavior interfere with childrens ability to develop and maintain appr opriate levels of adap tive behavior (Carr et al., 2002; Conroy et al., 2005; Dunlap, 2006; F ox, Dunlap, & Powell, 2002). Therefore, it was expected that caregiver participants would report lower leve ls of child adaptive behavior than expected in a normative sample of the population. One area of parenting research that has in cluded measures of adaptive behavior is positive behavior support (PBS). Most PBS interventions use adaptive or prosocial behaviors as outcome measures, as these interventions are designed to teach and reinforce adaptive replacement behaviors in place of challenging behaviors (Dunlap, 2006). However, these studies often do not report pre-in tervention levels of adaptive behavior or pretest/posttest comparisons (Conroy et al., 2005). Instea d, they generally report postintervention levels of adaptive behaviors or rate of skill gain.
111 Results of the current stu dy supported the hypothesis by in dicating that the sample participants reported significantl y more severe deficits in ch ild adaptive behavior at pretest than were predicted for a normative sa mple of the population. Statistical analyses revealed that nearly ten times as many car egivers in the participant sample perceived their childs adaptive behaviors to be within the clinically significant or deficit range on the Conceptual, Social, and Practical subs cales of the ABAS-II (Harrison & Oakland, 2003) than was expected given a normal dist ribution. These results indicate that the majority of caregivers who elected to participate in HOT DOCS perceived their children as having clinically deficient levels of adap tive behavior prior to beginning the training program. Changes in Child Problem & Adaptive Behavior Research Question 4. To what extent do caregivers perceive a decrease in child problem behavior following parent participa tion in the 6-week parent-training program ? Comparisons of pretest and posttest caregi ver ratings of child problem behavior using the CBCL have frequently been used in research on behavioral parent training programs (Barkley et al., 2000; Cartwright-Hatton, McNally, & White, 2005; Connolly, Sharry & Fitzpatrick, 2001; Feinfield & Bake r, 2004; Hartman et al., 2003; Nixon et al., 2003; Reid et al., 2001; Thompson, Ruma Schuchmann, & Burke, 1996; WebsterStratton, 1998; Webster-Stra tton & Hammond, 1997). Most studi es reported significant decreases in the severi ty of child behavior from pret est to posttest as reported by caregivers. Despite frequent use of the CBCL in be havioral parent tr aining research, significant limitations have been identified by the majority of researchers using CBCL as
112 an outcome measure. The primary limitation is that the CBCL measures child behaviors through parent report and not through direct observation (Connolly, Sharry & Fitzpatrick, 2001; Feinfield & Baker, 2004; Thompson et al., 1996; Webster-Stratton, Reid, & Hammond, 2004). Thus, pretest and posttest co mparisons might really be measuring increases in parent perceptions of competence, increases in parent percepti ons of social support or normality of child problem behavior, or decreases in parenting stress and not actual changes in child behavior. Severa l studies have overcome this limitation by supplementing the use of parent report ratings of child behavior with direct observations of child behavior, which is thought to provide a more accurate measure of changes in child problem behavior by eliminating th e potentially confoundi ng self-report bias (Barkley et al., 2000; Hartma n et al., 2003; Nixon et al., 2003; Reid et al., 2001). Results of this study indicat e significant reductions in the severity of child problem behavior as perceived by caregivers. It could not be determined from the data available whether child behavior actually impr oved or, as suspected in previous studies, changes in scores were due to reductions in parent stress and in creases in parenting competency. Results of the pretest/posttest comparisons made in this study should be interpreted with caution due to a low return rate of posttest scales (<25%). Information about which caregivers returned posttest rati ng scales (e.g., caregivers of children whose behavior drastically improved or those whose behavior remained the same or worsened) may better explain these results. It may be bene ficial to modify data collection procedures in the future to ensure a more comprehensive return rate of posttest rating scales, such as offering a booster session during which scales could be completed or offering incentives for completing and returning rating scales.
113 Research Question 5. To what extent do caregivers pe rceive an increase in child adaptive behavior following parent participa tion in the 6-week parent-training program? Using a measurement tool with a very limited research base, such as the ABAS-II or any other parent-report m easure of adaptive behavior precludes the development of evidence-based hypotheses for this research question. At present, there are limited published data available to determine whethe r the ABAS-II is sensit ive to short-term gains in adaptive behavior. Despite this lack of existing research, si gnificant gains across all adaptive skill areas were expected based on theories of behavior and positive behavior support (Carr et al., 2002; Conroy et al., 2005; Dunlap et al., 2003). Expectations also were based on the theoretical framework of HOT DOCS, which focuses on teaching children positive, prosocial replacement behavi ors and specifically training parents to shift their focus and attention to praising and rewarding positive behavior (Armstrong, Hornbeck et al., 2006). Larger changes in rating scale scores in adaptive skills from pretest to posttest were predicted compared to behavior problems because early emerging behavior problems have been shown to be stable over time and somewhat resistant to intervention (Campbell & Ewing, 1990; Coie & Dodge, 1998; Dishion et al., 1995; Kazdin, 1995; Moffitt, 1993; Reid, 1993; Tremblay 2000). Results of this study indicat ed non-significant levels of perceived change in the severity of deficits in child adaptive behavior on the part of caregivers. It could not be determined from the data available whethe r child adaptive behavior actually did not change from pretest to posttest or whet her other confounding vari ables, such as low return rate of posttest scales (<25%) could explain the non-significant findings. Differential return rates may be explained by actual changes in childrens adaptive
114 behavior. For example, caregivers whose children increased their adaptive skills may have been perceived as no longer having a prob lem, in which case caregivers may have had less motivation to complete lengthy rating sc ales (Barkley et al., 2000). Caregivers of children whose behavior did not improve or worsened following participation in the program may also have avoided completing and returning the posttest rating scales. Alternative measurement instrume nts for adaptive skills that are more caregiver-friendly (e.g., fewer items) and have more sound ps ychometric properties should be researched. Caregiver Skills at Home Research Question 6. What is the frequency and ease of use of the weekly parenting tips as reported by caregivers? Is there a relation between fr equency of use and ease of use as measured by the HOT DOCS Tip Tracker sheets? Participant caregivers were expected to report high frequencies of skill use, since skills were designed to fit into existing family routines and to be compatible with most parenting styles (Armstrong, Lilly, & Curtiss, 2006). It was also expected that different skills would be perceived by caregivers as more difficult to implement at home than other skills. Specifically, it was e xpected that the skills Use Positive Words Catch Them Being Good and Take 5 for Yourself would be rated by caregivers as being easier to implement at home than the skills Use Calm Voice Use Preventions and Follow Through Results of this study indicated that caregiv ers reported high overall frequencies of use of each skill as well as differential rate s of ease for various skills. However, the differential ratings of ease or difficulty of use did not follow the expected pattern. Caregivers reported Catch Them Being Good as easiest to use, followed by Use Preventions Use Calm Voice Follow Through and Use Positive Words Follow-up
115 interviews with caregivers may be beneficial to investigate why some of the skills were more difficult to implement at home than other skills. In terms of the relation between frequency and ease of skill use, caregivers were expected to report lower levels of difficulty implementing skills at home the more days they reported using the skill. However, results of statistical analyses revealed no significant relationships between frequency of use and ease of use. These findings may be explained by the restricted range of rati ngs of ease or difficulty (e.g., choices only 1 through 4) and the restricted range of days it was possible for caregivers to use skill (e.g., seven days maximum). Anothe r possible confounding variable is the differential number of caregivers completing weekly Tip Tracker sheets. Fewer participants completed and turned in Tip Tracker sheets for each se ssion than the previous sessions (i.e., 106 participants completed Tip Tracker 1 ; 93 participants completed Tip Tracker 3 ; 73 participants completed Tip Tracker 5 ). Results were predicted to show a peak in level of difficulty of skill use during the middle of the week, due to extinction burst of child behavior (Cooper et al., 1987). For example, the first day or two parents used th e skill at home it was expected that children would initially be compliant with parent di rection. However, once children perceived a change in caregiver behavior, childrens challenging behavior was expected to temporarily increase (e.g., testing the limits) and then decrease if caregivers remained consistent in their use of the new skill. Give n the behavioral concep t of extinction bursts, a hypothesized pattern would be for caregivers to initially report easier use of skills, followed by more difficulty using skills, and then a return to reports of more ease of use by the end of the seven-day period.
116 Results supported the hypothesi zed pattern of reported ease or difficulty of use as predicted by the presence of extinction burst s in childrens behavior Four of the five skills followed the expected pattern of reported ease of use, followed by a peak in difficulty, and then a decrease in difficulty. However, caregivers ratings for each of the four skills that followed this pattern were varied. Caregiver ratings for Use Preventions most clearly followed the anticipated pattern. Follow Through Use Calm Voice and Catch Them Being Good followed the pattern to a lesser degree. Caregiver ratings for ease of use of Use Positive Words did not follow the expected pattern. Instead caregivers rated the skill as being initially more difficu lt and progressively get ting easier throughout the week. The pattern of perceived difficulty of Use Positive Words may be explained by the placement of this skill as the fi rst skill assigned as homework in the HOT DOCS program. Caregivers may have reported use of this skill to be more difficult than later skills because they were adjusting to making changes in their overall parenting practices and not necessarily because the skill itself wa s more difficult to use. Future research should include parent interviews to further investigate the reasons for differences in caregiver perceptions of skill use at home. Future research also should investigate possible relations between reported frequency of skill use at home and changes in caregiver perceptions of severity of child s challenging behavior (i.e., correlation between days used and pretest/posttest scores on CBCL). Caregivers Overall Perceptions of the HOT DOCS Program Research Question 7. What are caregivers overall perceptions of the HOT DOCS parent training program as measured by the HOT DOCS Program Evaluation Survey?
117 Results of a previous evaluation of participant satisfaction with the HOT DOCS program (Armstrong, Hornbeck et al., 2006) using surveys and focus groups indicated that caregivers reported high levels of sa tisfaction with program In light of these findings, it was expected that participants in the current study also would report high levels of satisfaction. With few exceptions, th e majority of caregi vers (95%) indicated that they Agreed or Strongly Agreed that the HOT DOCS program met their expectations, was beneficial to their familie s, and positively impacted their behavior as caregivers. The few statements on the survey with which caregivers Disagreed or Strongly Disagreed related to the ability to implemen t specific skills at home and the programs impact on child behavior. These findings are not surprising, given that many parent training interventions struggle with accomplishing transfer of skills taught in the classroom to the home setting (Eyber, 1998; Sanders, 1999). In light of the overwhelmingly positive response to these ite ms, those few participants who were not satisfied with the program were provided in dividual consultation and possible referrals for further assessment and treatment strate gies. These results were interpreted as exceptions to a program perceived as effective, rather than proof that the program is not effective. The majority of caregivers (70%) reported th at they were using the skills learned in the program at home or in the community and had shared the information they learned with others (95%), including spouses, fa mily, and friends. When asked to provide suggestions for future HOT DOCS classes, 40% of caregiv ers answered Nothing, the program is fine as is, and 25% answered More time, (e.g., more classes, longer sessions, booster sessions). These results support caregiver ratings of satisfaction with the
118 program, by indicating that there were no significant ch anges or improvements that should be made to the program. When asked what they valued most from the training, the majority (60%) of caregivers indicated the specific skills taught in the sessions. Implications for Practitioners The results of this study suggest several implications for practitioners. First, the study provided preliminary evidence for the potential effectiveness of the HOT DOCS parent training program as an early interv ention technique, allowing practitioners to tentatively add this program to their list of promising treatment strategies for children displaying early-emerging challenging behavi or. These findings are consistent with several decades of previous research on othe r parent training programs in demonstrating the effectiveness of behavioral parent trai ning as an intervention (Eyberg, 1988; Feinfield & Baker, 2004; Kazdin, 1995; Webster-Strat ton, 1998). The effectiveness of using a group-delivered parent training program to a ddress early-emerging challenging behavior allows psychologists to serve as indirect serv ice providers or consul tants, enabling them to provide information and skills to caregive rs, which they can use to problem-solve and address their own childrens behavior. The indi rect provision of serv ices is in direct contrast with the traditional medical mode l of service delivery, in which children are referred to a professional, an evaluation is conducted, and depending on the results, the professional directly applies treatment to th e child in a one-on-one format. While this traditional treatment model has been shown to be effective in producing desired outcomes it has also been shown to be less cost-effective and have poorer long-term outcomes than group-delivered, consultation model tr eatment strategies (Kazdin, 1995).
119 Results of this study also provide practi tioners with an earl y intervention program that has been successful in reaching previ ously underserved porti ons of the population, specifically, Hispanic or Spanish-speaki ng families and caregivers from low SES families. These early findings suggest that the HOT DOCS training program is promising as an effective intervention for Hispanic or Spanish-speaking families mainly because it has been translated into Spanish in both printed and orally delivered presentations (Armstrong, Lilly, Curtiss, Salinas, Chiraboga, & Ortiz, 2006). HOT DOCS has also been made available to a large proportion of low SES families because it funded by a grant from the Childrens Boar d of Hillsborough county. All materials and supplies are provided for caregiv ers, removing previously identified financial barriers to parent participation in parent training programs (Barkley et al., 2000; Webster-Stratton & Taylor, 2001). Although initially discouraging, the findings in this study identifying the underrepresentation of Black/African American caregivers in the HOT DOCS program also provide practitioners with a specific ta rget for recruiting participants for future HOT DOCS trainings. This might be accomplishe d through increased advertising and recruitment directly targeted at reaching this racial/ethnic group as well as through making adjustments in scheduling of future cl asses, such as offering the trainings at locations within the Black/African American community. Finally, findings from this study provide practitioners with preliminary evidence on the effectiveness of incor porating the principles of PB S into a behavioral parent training program. While previous research has demonstrated the effectiveness of PBS interventions with specific popul ations (e.g., older children with intellectual disabilities or autism spectrum disorders), the current study has applied PBS techniques to a wider
120 segment of the population (Buschbacher et al ., 2004; Conroy et al., 2005). In contrast with earlier research, the result s of this study indicate that providing parents intervention strategies based on the principles of PB S can effectively address early-emerging challenging behaviors in young child ren with sub-clinical levels of challenging behavior. These results also provide initial support for the use of PBS in tervention techniques delivered in a group format, which has not b een demonstrated in previous research. Limitations This study has several significant limitations. The first is the use of archival data, which does not allow the researcher any cont rol over data collecti on procedures and the type of data originally collect ed. The second limitation is the absence of a control or waitlist control group to use as a normative comparison group for the participants who received training. The archival data analy zed were gathered using a pretest-posttest design. This design has several threats to in ternal validity of th e study, including history, maturation, testing, instrumenta tion, mortality, and regression to the mean. The use of a control group would strengthen the internal validity of fu ture investigations of the HOT DOCS program. A third limitation to this study is the small sample size, which is a component of the pilot study design, but will limit the statistical power of results. A fourth limitation is the low return rate of several outcome measures used, including weekly Tip Tracker sheets and posttest beha vior rating scales. Fi nally, several of the measurement instruments used as outcome i ndicators were designed by the authors of the HOT DOCS parent training program. There is no evid ence of reliability or validity data available that these measures accurately or truthfully measure the constructs they were designed to assess.
121 Directions for Future Research Several areas of future research were generated from the results of this study. First, follow-up surveys, possibly through phone interviews, could be conducted to collect further evidence, such as caregiver statements or expl anations of behavior, and to investigate patterns of results, such as rates of attendance an d attrition, reported ease or difficulty of skill use at home, and caregiver perceptions of child behavior following program completion. For example, researchers should investigate why caregivers sign up for class and do not attend; why caregiver s attend one or two sessions but do not complete training; and why a large percent of caregivers did not return posttest rating scale packets. Additional analyses speci fically focusing on rates and patterns of attendance in relation to outcome variab les should be conducted. For example, did participants who attended speci fic sessions (e.g., sessions 3, 4 and 5) show greater gains in knowledge or problem solving skills and did they perceive their childrens skills as improving more than participants who atte nded different sessions (e.g., 1, 2, and 6). Future evaluations of the HOT DOCS program also should incorporate the use of a comparison or control group. Specificall y, caregivers on the waiting list could be asked to complete pre-/postKnowledge te sts and pre-/posttest rating scales while waiting for treatment. In order to increase the re liability and validity of findings related to caregiver knowledge, the HOT DOCS Knowledge Test should be revised and validated. For example, a panel of experts in child deve lopment, PBS, and othe r related field should evaluate test items, the items should be ba lanced for true/false responses, and there should be at least four questions per topic area (e.g., child development, positive behavior support).
122 Another area of future research should focus on a more thorough investigation of the positive behavior support prin ciples incorporated into the HOT DOCS program. Specifically, an outcome measure assessing ca regiver satisfaction with and knowledge of the functions of behavior and the proble m solving process should be included. This investigation should focus more specifically on to what extent caregivers learn and are able to implement the problem solving process. Conclusion Results of this study suggest successf ul outcomes for careg ivers and children participating in the HOT DOCS program, including increases in caregiver knowledge, frequent use of skills at home, high leve ls of satisfaction with the program, and reductions in the perceived severity of child behavior problems. Re sults also indicated several modifications that could be made to the program to improve participant outcomes and increase the validity and reliability of program evaluations, including changes to measurement instruments (e.g., knowledge te st, adaptive skill measure, evaluation survey) and data collection procedures (e.g., wa itlist control group, low rate of return of posttest rating scales). Overall, the HOT DOCS parent training program appears to be a promising early interventi on program that could be delivered in group format.
123 References Achenbach, T. (2001). Manual for the Achenbach system of empirically based assessment (ASEBA) Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families. Achenbach, T. (1991). Manual for the Child Behavior Checklist/4-18 and 1991 Profile Burlington, VT: University of Ve rmont, Department of Psychiatry. Achenbach, T. & Edelbrock, C. (1983). Manual for the Child Behavior Checklist and the Revised Behavior Problem Profile. Burlington, VT: University of Vermont. American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders: Text revision (DSM-IV-TR) (4th ed.). Washington, D.C.: Author. American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders: Fourth Edition (DSM-IV) (4th ed.). Washington, D.C.: Author. American Psychiatric Association. (1987). Diagnostic and statistical manual of mental disorders: Third Edition (DSM-III) (3rd ed.). Washington, D.C.: Author. Anastopoulos, A., Shelton, T., DuPaul, G., & Gu evremont, D. (1993). Parent training for attention deficit hyperactivity disord er: Its impact on parent functioning. Journal of Abnormal Child Psychology, 21 581-596. Armstrong, K. & Hornbeck, M. (2005). Helping our Toddlers: Le ssons for parents of young children with challenging behaviors Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, Department of Child and Family Studies.
124 Armstrong, K., Hornbeck, M., Beam, B., Mack, K., & Popkave, K. (2006). Evaluation of a curriculum designed for caregivers of young children with challenging behavior. Journal of Early Childhood and Infant Psychology, 2 51-61. Armstrong, K., Lilly, C., & Curtiss, H. (2006). Helping our toddlers developing our childrens skills Tampa, FL: University of South Florida, Department of Pediatrics, Division of Child Development. Armstrong, K., Lilly, C., Curtiss, H., Salinas A., Chiraboga, D., & Ortiz, C. (2006). Helping our toddlers developing our childrens skills: En Espanol Tampa, FL: University of South Florida, Department of Pediatrics, Division of Child Development. Bagner, D., & Eyberg, S. (2003). Father i nvolvement in parent training: When does it matter? Journal of Clinical Child and Adolescent Psychology, 32, 599605. Barkley, R. (1997). Defiant children: A clinicians manual for assessment and parent training. 2nd ed New York: Guilford Press. Barkley, R., Shelton, T., Crosswait, C., M oorehouse, M., Fletcher, K., Barrett, S., Jenkins, L., & Metevia, L. (2000). Multi -method psycho-educational intervention for preschool children with disruptive behavior: Preliminary results at posttreatment. Journal of Child Psychology and Psychiatry, 3 319-322. Blackwell, P. (2005). Parent sour ce: A practice in enrichment. Zero to Three, 26 4-8.
125 Bor, W., Sanders, M., & Markie-Dadds, C. ( 2002). The effects of the Triple P-positive Parenting Program on preschool children with co-occurring disruptive behavior and attentional/hyperactivity difficulties. Journal of Abnormal Child Psychology, 30 571-587. Bradley, S., Jadaa, D., Brody, J., Landy, S ., Tallett, S., Watson, W., Shea, B., & Stephens, D. (2003). Brief psychoeducatio nal parenting program: An evaluation and 1-year follow-up. Journal of the American Acade my of Child Adolescent Psychiatry, 42, 1171. Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design. Cambridge, MA: Harvard University Press. Buschbacher, P., Fox, L., & Clarke, S. (2004) Recapturing desired family routines: A parent-professional behavioral collaboration. Research and Practice for Persons with Severe Disabilities, 29 25-39. Buss, D. (1981). Predicting pare nt-child interactio ns from children's activity level. Developmental Psychology, 17, 59-65. Campbell, S. (1997). Behavior problems in preschool children: Developmental and family issues. In T. H. Ollendick & R. J. Prinz (Eds.), Advances in clinical child psychology (Vol .19, pp.1-26). New York: Plenum. Campbell, S. (1995). Behavior problems in preschool children: A review of recent research. Journal of Child Psychology & Psychiatry & Allied Disciplines, 36, 113-149.
126 Campbell, S. & Ewing, L. (1990). Follow-up of hard-to-manage preschoolers: Adjustment at age 9 and predic tors of continuing symptoms. Journal of Child Psychology and Psychiatry and Applied Disciplines, 31, 871889. Carr, E., Dunlap, G., Horner, R., Koegel, R., Turnbull, A., Sailor, W., et al. (2002). Positive behavior support: E volution of an applied science. Journal of Positive Behavior Interventions, 4 4-16. Cartwright-Hatton, S., McNally, D., & White, C. (2005). A new cognitive behavioural parenting intervention for families of young, anxious children: A pilot study. Behavioural and Cognitive Psychotherapy, 33 243-247. Chamberlain, P. & Reid, J. (1987). Parent observation and report of child symptoms. Behavioral Assessment, 9 97-109. Coie, J. & Dodge, K. (1998). Aggression a nd antisocial behavior. In W. Damon (Editor in Chief) and N. Eisenberg (Vol. Ed.), Handbook of child psychology, 5th edition, Volume 3, Social, emoti onal, and personalit y development NY: John Wiley & Sons. Connolly, L., Sharry, J., & Fitzpatrick, C. (2001). Evaluation of a group treatment programme for parents of childre n with behavioural disorders. Child Psychology & Psychiatry Review, 6 159-165. Conroy, M., Dunlap, G., Clarke, S., & Alter, P. (2005). A descriptive analysis of positive behavioral intervention research with young children with cha llenging behavior. Topics in Early Childhood Special Education, 25 157-166. Cooper, J., Heron, T., & Heward, W. (1987). Applied behavior analysis Upper Saddle River, NJ: Merrill.
127 Cummings, E. & Davies, P. (1994). Children and marital conflict: The impact of family dispute and resolution. New York, London: Guilford. Dadds, M. & Powell, M. (1991). The relationshi p of interparental conflict and global marital adjustment to aggression, anxi ety, and immaturity in aggressive and nonclinic children. Journal of Abnormal Child Psychology, 19 553-567. Denham, S. & Burton, R. (1996). A social-emo tional intervention for at-risk 4-year olds. Journal of School Psychology, 32 225-245. Denham, S., Workman, E., Cole, P., Weissbr od, C., Kendziora, K., & Zahn-Waxler, C. (2000). The role of parental so cialization and emotion expression. Developmental and Psychopathology, 12 23-45. Dishion, T., French, D., & Patterson, G. (1995). The developmental and ecology of antisocial behavior. In D. Cicchetti & D. J. Cohen (Eds.), Developmental Psychopathology, Vol. 2: Risk, disorder, and adaptation (pp. 421-471). New York: John Wiley & Sons. Dodge, K. (1993). The future of research on conduct disorder. Development and Psychopathology, 5, 311-320. Dodge, K., Bates, J., & Pettit, G. (1990). Mechanisms in the cycle of violence. Science, 250 1678-1683. Duda, M., Dunlap, G., Fox, L., Lentini, R., & Clarke, S. (2004). An experimental evaluation of positive behavior suppor t in a community preschool program. Topics in Early Childhood Special Education, 24 143-155. Dunlap, G. (2006). The applied behavior analyt ic heritage of PBS: A dynamic model of action-oriented research. Journal of Positive Behavior Support, 8 58-60.
128 Dunlap, G., Conroy, M., Kern, L., DuPaul, G., VanBrakle, J., Strai n, P., et al. (2003). Research synthesis on effective inte rvention procedures: Executive summary Tampa, FL: University of South Florida Center for Evidence-Based Practice: Young Children with Challenging Behavior. Dunlap, G. & Fox, L. (1996). Early interven tion and serious behavior problems: A comprehensive approach. In L. K. Koegel R. L. Koegel, & G. Dunlap (Eds.), Positive behavior support: Including pe ople with difficult behavior in the community (pp. 31-50). Baltimore, MD: Brookes Publishing Co. Dunst, C., Trivett, C., & Cutspec, P. (2002). Toward an operational definition of evidence-based practice. Retrieved December 3, 2006, from the Center for Evidence-Based Practice: Young Children with Challenging Behaviors website: http://challengingbehavior.fmhi.usf.edu Egeland, B., Kalkoske, M., Gottesman, N., & Erikson, M. (1990). Preschool behavior problems: Stability and factors accounting for change. Journal of Child Psychology and Psychiatry, 31 891-909. Eyberg, S. (1988). Parent-child interaction therapy: Integration of traditional and behavioral concerns. Child & Family Behavior Therapy, 10 33-46. Eyberg, S., Bessmer, J., Newcomb, K., Edwa rds, D., & Robinson, E. (1994). Dyadic parent-child interaction coding system-II: A manual. Social and behavioral science documents (Ms. No. 2897). San Rafae l: CA: Select Press. Eyberg, S. & Pincus, D. (1999). Eyberg Child Behavior Invento ry and the Sutter-Eyberg Student Behavior Scale-Revised Psychological Assessment Resources, Inc.: Lutz, FL.
129 Eyberg, S., Funderburk, B., Hembree-Kigin, T., McNeil, C., Querido, J., & Hood, K. (2001). Parent-child interact ion therapy with behavior problem children: One and two year maintenance of treatment effects in the family. Child & Family Behavior Therapy, 23 1-20. Feil, E., Walker, H., Severson, H., & Ball, A. (2000). Proactive screening for emotional/behavioral concerns in Head Start preschools: Promising practices and challenges in applied research. Behavioral Disorders, 26 13-25. Feinfield, K. (1995). Predicting mother-child interactions. Unpublished master's thesis. University of Ca lifornia, Los Angeles. Feinfield, K. & Baker, B. (2004). Empirical support for a treatment program for families of young children with externalizing problems. Journal of Clinical Child and Adolescent Psychology, 33 182-195. Fischer, M., Rolf, J., Hasazi ,, J., & Cummings, L. (1984). Follow-up of a preschool epidemiological sample: Cross-age continuities and predictions of later adjustment with internalizing and ex ternalizing dimensions of behavior. Child Development, 55 137-150. Forehand, R., & Long, N. (1988). Outpatient treatment of the acting out child: Procedures, long term follow-up data, and clinical problems. Advanced Behavior Research Therapy, 10 129-177. Fox, R. (1994). Parent Behavior Checklist Department of Counse ling and Educational Psychology, Marquette, University, firstname.lastname@example.org.
130 Fox, L., Dunlap, G., & Cushing, L. (2002). Earl y intervention, positive behavior support, and transition to school. Journal of Emotional and Behavioral Disorders, 10 149157. Fox, L., Dunlap, G., & Powell, D. (2002). Y oung children with cha llenging behavior: Issues and considerations for behavior support. Journal of Positive Behavior Interventions, 4 208-217. Fox, L. & Little, N. (2001). Starting early: De veloping school-wide behavior support in a community preschool. Journal of Positive Be havior Interventions, 3 251-254. Fox, R. & Nicholson, B. (2003). Parenting Young Children: A Facilitators Guide Sopris West Educationa l Services: Longmont, CO. Frea, W. (2004). Families and positive behavior support: Addressing problem behaviors in family contexts. American Journal of Mental Retardation, 109 264. Gall, M., Gall, J., & Borg, W. (2007). Educational research : An introduction (8th ed.). Boston, MA: Allyn & Bacon. Gross, D., Fogg, L., Webster-Stratton, C., Ga rvey, C., Julion, W., & Grady, J. (2003). Parent training of toddlers in day ca re in low-income urban communities. Journal of Consulting & Clinical Psychology, 71 261-278. Hartman, R., Stage, S., & Webster-Stratton, C. (2003). A growth-curve analysis of parent training outcomes: Examining the influen ce of child risk factors (inattention, impulsivity, and hyperactivity problems), parental and family risk factors. Journal of Child Psychology and Psychiatry, 44 388-398. Hebbler, K. Spiker, D., Mallik, S., Scar borough, A., Simeonsson, R., & Collier, M. (2001). A first look at the characteristics of children and families entering early
131 intervention services. Menlo Park, CA: SRI International. Heller, T., Baker, B., Henker, B., & Hinsha w, S. (1996). Externalizing behavior and cognitive functioning from preschool to first grade: Stabil ity and predictors. Journal of Clinical Child Psychology 25 376-387. Hieneman, M., Childs, K., & Sergay, J. (2006). Parenting with positive behavior support: A practical guide to resolving your child's difficult behavior Baltimore, Maryland: Brookes Publishing Company. Kazdin, A. (1997). Parent management tr aining: Evidence, outcomes, and issues. Journal of the American Academy of Child and Adolescent Psychiatry, 36 1349-1356. Kazdin, A. (1995). Conduct disorder. In F. C. Verhulst & H. M. Koot (Eds.), The epidemiology of child and adolescent psychopathology, (pp. 258-290). New York: Oxford University Press. Kazdin, A. (1993). Adolescent mental hea lth: Prevention and treatment programs. American Psychologist, 48 127-141. Kazdin, A. & Kendall, P. (1998). Current pr ogress and future plans for developing effective treatments: Comments and perspectives. Journal of Clinical Psychology, 27 217-226. Keenan, K. & Wakschlag, L. (2000). More than th e terrible twos: The nature and severity of behavior problems in clin ic-referred preschool children. Journal of Abnormal Child Psychology, 28, 33-46. Kellem, S. (1989). Johns Hopkins Preventative Intervention Research Center [NIMH grant proposal]. Baltimore: Johns Hopkins University.
132 Kendziora, K. & O'Leary, S. (1993). Dysfuncti onal parenting as a focus for prevention and treatment of child behavior problems. In T. H. Ollendick & R. J. Prinz (Eds.), Advances in Clinical Child Psychology, (Vol. 15, pp 175-206). New York: Plenum. Knapp, P., Ammen, S., Arstein-Kerslake, C ., Poulsen, M., & Mastergeorge, A. (2007). Feasibility of expanding services for very young children in the public mental health setting. Journal of the American Academy of Child and Adolescent Psychiatry, 46 152-161. Kumpfer, K. & Alvarado, R. (2003). Family-s trengthening approaches for the prevention of youth problem behaviors. The American Psychologist, 58 457-465. Lavigne, J., Gibbons, R., Christoffel, K. K., Arend, R., Rosenbaum, D., Binns, H., Dawson, N., Sobel, H., & Isaacs, C. (1996) Prevalence rates and correlates of psychiatric disorders am ong preschool children. Journal of the American Academy of Child & Adolescent Psychiatry, 35 204-214. Lipsey, M. & Derzon, J. (1998). Predictors of violent or serious delinquency in adolescence and early adulthood: A synthe sis of longitudinal research. In R. Loeber & D. P. Farrington (Eds.), Serious & violent juvenile offenders: Risk factors and successful interventions (pp. 86-105). Thousand Oaks, CA: Sage. Loeber, R. & Dishion, T. (1983). Early pred ictors of male delinquency: A review. Psychological Bulletin, 94 68-99. Lundahl, B., Risser, H., & Lovejoy, C. (2006) A meta-analysis of parent training: Moderators and follow-up effects. Clinical Psychology Review, 26 86-104. Lutzker, J. & Campbell, R. (1994). Ecobehavioral family interventions in developmental disabilities. Pacific Grove, CA: Brooks/Cole Thomson Learning.
133 Marchant, M., Young, R., & West, R. (2004). The effects of parental teaching on compliance behavior of children. Psychology in the Schools, 41 337. Maughan, D., Christiansen, E., Jenson, W., Olym pia, D., & Clark, E. (2005). Behavioral parent training as a treatment for extern alizing behaviors and disruptive behavior disorders: A meta-analysis. School Psychology Review, 34 267-286. McCabe, K., Yeh, M., Garland, A., Lau, A., Chavez, G. (2005). The GANA program: A tailoring approach to adapting parent-c hild interaction therapy for Mexican Americans. Education and Treatment of Children, 28 111-129. McMahon, R. & Forehand, R. (2003). Helping the non-compliant child: Family-based treatment for opposit ional behavior (2nd ed.). New York, NY: Guillford Press. McNeil, C., Capage, L., Bahl, A., & Bl anc, H. (1999). Importance of early intervention for disruptive behavior probl ems: Comparison of treatment and wait list-control groups. Early Education and Development, 10, 445-454. McNeill, S., Watson, T., Henington, C., & Meek s, C. (2002). The effects of training parents in functional behavior assess ment on problem identification, problem analysis, and intervention design. Behavior Modification, 26 499-515. Minor, L., Onwuegbuzie, A., Witcher, A., & James, T. (2002). Presersvice teachers educational beliefs and thei r perceptions of characterist ics of effective teachers. Journal of Educational Research, 96 116-127. Mrazek, D., Mrazek, P., & Kli nnert, M. (1995). Clinical assessment of parenting. Journal of the American Academy of Ch ild & Adolescent Psychiatry, 34 272-282.
134 Nelson, C. (1995). A meta-analysis of parent ed ucation programs for children two to nine years. Dissertation Abstracts International: Section B: The Sciences and Engineering, 56, 1686. Nicholson, B., Fox, R., & Johnson, S. (2005). Pa renting young children with challenging behaviour. Infant and Child Development, 14 425-428. Nixon, R., Sweeny, L., Erickson, D., & Touyz, S. (2003). Parent-child interaction therapy: A comparison of standard and abbreviated treatments for oppositional defiant preschoolers. Journal of Consulting an d Clinical Psychology, 71 251260. Olweus, D. (1991). Bully/victim problems among school children: Basic facts and effects of a school-based intervention program In D. Pepler & K. Rubin (Eds.), The development and treatment of childhood aggression (pp. 411-446). London: Lawrence Erlbaum. Onwuegbuzie, A. & Teddlie, C. (2002). A fr amework for analyzing data in mixed methods research. In A. Ta shakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 351-383). Thousand Oaks, CA: Sage. Patterson, G. (1982). Coercive family process. Eugene, OR: Castalia. Patterson, G., DeBaryshe, B., & Ramsey, E. (1989). A developmental perspective on antisocial behaviour. American Psychologist, 44 329-335. Patterson, G. & Fleishman, M. (1979). Main tenance of treatment effects: Some considerations concerning family systems and follow-up data. Behavior Therapy, 10 168-185.
135 Patterson, G., Reid, J., & Dishion, T. (1992). Antisocial boys Eugene, OR: Castalia. Pettit, G., Bates, J., & Dodge, K. (1993). Fa mily interaction patterns and children's conduct problems at home and school: A longitudinal perspective. School Psychology Review, 22 403-420. Phares, V., Fields, S., Kamboukos, D., & Lopez, L. (2005). Still looking for Poppa. American Psychologist, 60 735-736. Qi, C. & Kaiser, A. (2003). Behavior problem s of preschool children from low-income families: Review of the literature. Topics in Early Childhood Special Education, 23 188-216. Ramey, C. & Ramey, S. (1998). Early intervention and early experience. American Psychologist, 53 109-120. Reid, J. (1993). Prevention of conduct disorder before and after sc hool entry: Relating interventions to developmental findings. Development and Psychopathology, 5 243-262. Reid, M., Webster-Stratton, C., & Baydar, N. (2 004). Halting the development of conduct problems in Head Start children: The effects of pa rent training. Journal of Clinical Child and Adolescent Psychology, 33 279-291. Reid, M., Webster-Stratton, C., & Beauchaine, T. (2001). Parent trai ning in head start: A comparison of program response among African American, Asian American, Cuacasian, and Hispanic mothers. Prevention Science, 2 209. Rutter, M. (1991). Childhood experiences and ad ult psychosocial functioning. In G. R. Book & J. Whelan (Eds.) The Childhood Environment and Adult Disease, Ciba Foundation Symposium No. 1 (pp 189-208). Chichester, England: Wiley.
136 Sandall, S., & Ostrosky, M. (Eds.). (1999). Young exceptional children: Practical ideas for addressing challenging behaviors. (1st ed.). Denver: Division for Early Childhood of the Council for Exceptional Children. Sanders, M. (1999). Triple P-positive pare nting program: Towards an empirically validated multilevel parenting and family support strategy for the prevention of behavior and emotional problems in children. Clinical Child and Family Psychology Review, 2 71. Sanders, M. (1996). New directions in behavioral family interv ention with children. In T. H. Ollendick & R. H. Prinz (Eds.), Advances in Clinical Child Psychology, (Vol 18, pp. 283-330). New York: Plenum. Sanders, M., Markie-Dadds, C., Tully, L., & Bor, W. (2000). The Triple P-Positive Parenting Program: A comparison of e nhanced, standard, and self-directed behavioral family intervention for pare nts of children with early onset conduct problems. Journal of Consulting and Clinical Psychology, 68 624-640. Sanders, M., Mazzucchelli, T., & Studman, L. (2004). Stepping stones Triple P: The theoretical basis and development of an evidence-based positive parenting program for families with a child who has a disability. Journal of Intellectual Developmental Disability, 29 265-283. Schuhmann, E., Foote, Eyberg, S., Boggs, S., & Algina, J. (1998). Efficacy of parentchild interaction therapy: Interim report of a randomized trial with short-term maintenance. Journal of Clinical Child Psychology, 27 34-45.
137 Shaw, D., Emery, R., & Turner, M. (1993) Parental functioning and children's adjustment in families of divorce: A prospective study. Journal of Abnormal Child Psychology, 21 119-134. Shonokoff, J., & Phillips, D. (Eds.). (2000). From neurons to neighborhoods: The science of early childhood development Washington, DC: National Academy Press. Skinner, B. (1953). Science and human behavior New York: Macmillan Free Press. Smith, B. J. & Fox, L. (2003, January). Systems of Service Deli very: A Synthesis of Evidence Relevant to Young Children at Risk for or Who Have Challenging Behavior Retrieved September 21, 2006, from the Center for Evidence-Based Practice: Young Children with Challenging Behaviors website: http://challengingbehavior.fmhi .usf.edu/resources/smith-fox-jan03.pdf Stevens, J. (1999). Intermediate statistics: A modern approach (2nd edition). Hillsdale, NJ: Lawrence Erlbaum. Stormshak, E., Bierman, K., McMahon, R., & Lengua, L. (2000). Parenting practices and child disruptive behavior problems in early elementary school. Joumal of Clinical Child Psychology, 29. 17-29. Strain, P., Lambert, D., Kerr, M., Stra gg, V., & Lenker, D. (1983). Naturalistic assessment of childrens compliance to teachers requests and consequences for compliance. Journal of Applied Behavior Analysis, 16 243-249. Strain, P. & Timm, M. ( 2001). Remediation and preven tion of aggression: An evaluation of the RIP Program over a quarter century. Behavioral Disorders, 26 297-313.
138 Straus, M. (1989). Manual for the Conflict Tactics Scales (CTS) Durham: University of New Hampshire. Strayhorn, J. & Weidman, C. (1988). A parent pr actices scale and its relation to parent child mental health. Journal of the American Academy of Child & Adolescent Psychiatry, 27 613-618. Thompson, R., Ruma, P., Schuchmann, L., & Bu rke, R. (1996). A cost-effectiveness evaluation of parent training. Journal of Child & Family Studies, 5 415-429. Tremblay, R. (2000). The development of a ggressive behavior dur ing childhood: What have we learned in the past century? International Journal of Behavioral Development, 24 129-141. Vaughn, B., White, R., Johnston, S., & Dunlap, G. (2005). Positive behavior support as a family-centered endeavor. Journal of Positive Behavior Interventions, 7, 55-58. Wahler, R. & Dumas, J. E. (1986). A chip off the old block: Some interpersonal characteristics of coercive children acro ss generations. In P. Strain, M. Guralnick, & H. M. Walker (Eds.), Childrens soci al behavior: Development, assessment, and modification (pp. 49-91). Or lando, FL: Academic Press. Walker, H., Colvin, G., & Ramsey, E. (1995). Antisocial behavior in school: Strategies and best practices Belmont, CA: Brooks/Cole. Walker, H., Kavanaugh, K., Stiller, B., Goll y, A., Severson, H., & Feil, E. (1998). First Step to Success: An early interv ention approach for preventing school antisocial behavior. Journal of Emotional and Behavioral Disorders, 6 66-80.
139 Webster-Stratton, C. (2001). The Incredible Years: Parent, teacher, and child training series. Boulder, CO: Center for the Study and Prevention of Violence, Institute of Behavior Science, University of Colorado at Boulder. Webster-Stratton, C. (2000). Oppositional-defi ant and conduct disordered children. In M. Hersen & R. T. Ammerman (Eds.), Advanced abnormal child psychology (2nd ed., pp. 387412). Mahwah, NJ: Erlbaum. Webster-Stratton, C. (1998). Preventing c onduct problems in Head Start children: Strengthening parenting competencies. Journal of Consulting and Clinical Psychology, 66 715-730. Webster-Stratton, C. (1990). Long-term follow-up of families with young conductproblem children: From preschool to grade school. Journal of Clinical Child Psychology, 19 144-149. Webster-Stratton, C. (1985). Mother percep tions and mother-child interactions: Comparison of a clinic-r eferred and a non-clinic group. Journal of Clinical Child Psychology, 14, 334-339. Webster-Stratton, C., & Hammond, M. (1997). Treating children with early-onset conduct problems: A comparison of child and parent training interventions. Journal of Consulting and Clinical Psychology, 65 93-109. Webster-Stratton, C., Reid, M., & Hammond, M. (2004). Treating children with earlyonset conduct problems: Intervention outcomes for parent, child, and teacher training. Journal of Clinical Child and Adolescent Psychology, 33 105-124.
140 Webster-Stratton, C. & Taylor, T. (2001). Ni pping early risk factors in the bud: Preventing substance abuse, delinquen cy, and violence in adolescence through interventions targeted at young children (0 to 8 Years). Prevention Science, 2 165-192. Weinberg, H. (1999). Pare nt training for attent ion-deficit hyperactivit y disorder: Parental and child outcome. Journal of Clinical Psychology, 55 907-913.
142 Appendix A The HOT DOCS Parent Training Curriculum Session Topic Parenting tip homework Special play activity 1 Early childhood development Use positive words Bubbles 2 Routines and rituals Catch them being good Reading 3 Behavior and development Use a calm voice Coloring 4 Preventing problem behavior Use preventions Fun Dough 5 Teaching new skills Follow-through Balls 6 Managing parent stress Take time for yourself Free choice
143 Appendix B Relation between Research Questions and Variables Research Question Dependent Variable Design What is the impact of participation in the HOT DOCS program on parent knowledge of the principles of behavior, positive behavior support, child development, and parenting practices? HOT DOCS Knowledge Test Preand Posttests Do childs problem behaviors decrease following parent particip ation in the 6-week parent-training course? CBCL Preand Posttests Do childs adaptive behaviors increase following parent particip ation in the 6-week parent-training course? ABAS-II Preand Posttests What is the frequency a nd the ease of use of the weekly parenting tips as reported by participant parents and is ther e a relati on between frequency of use and ease of use? HOT DOCS Tip Tracker Forms Weekly evaluation What are parents overall perceptions of their participation in the HOT DOCS parent training program? HOT DOCS Program Evaluation Survey Posttest
144 Appendix C HOT DOCS Demographics Form (English version)
145 Appendix D HOT DOCS Demographics Form (Spanish version)
146 Appendix E HOT DOCS Knowledge Test (English version)
147 Appendix F HOT DOCS Knowledge Test (Spanish version)
148 Appendix G HOT DOCS Tip Tracker #1 (English version)
149 Appendix H HOT DOCS Tip Tracker #1 (Spanish version)
150 Appendix I HOT DOCS Program Evaluation Survey (English version)
151 Appendix J HOT DOCS Program Evaluation Survey (Spanish version)