USF Libraries
USF Digital Collections

Helping our toddlers, developing our children's skills (HOT DOCS)

MISSING IMAGE

Material Information

Title:
Helping our toddlers, developing our children's skills (HOT DOCS) an investigation of a parenting program to address challenging behavior in young children
Physical Description:
Book
Language:
English
Creator:
Williams, Jillian Leigh
Publisher:
University of South Florida
Place of Publication:
Tampa, Fla
Publication Date:

Subjects

Subjects / Keywords:
Behavioral parent training
Group-delivered
Early intervention
Challenging behavior
Problem solving
Dissertations, Academic -- Psychological and Social Foundations -- Doctoral -- USF   ( lcsh )
Genre:
non-fiction   ( marcgt )

Notes

Abstract:
ABSTRACT: This study investigated outcomes of a parent training curriculum: Helping Our Toddlers Developing Our Children's Skills (HOT DOCS), using secondary analyses of existing data collected between May 2007 and March 2009. The evaluation studied the impact of specific components of the parent training program on both participants' knowledge and attitudes and their perceptions of target children's behavior. Caregivers (n = 334) of children between the ages of 18 months and 5.11 years of age who were participants in the parent training program were included in the study. Measures included a pre/post knowledge test, pre/post rating scales of child problem behavior, and a program evaluation survey. Results indicated significant increases in caregiver knowledge following participation in the program, but non-significant differences between groups of participants based on various demographic variables. Prior to participation in the program, caregivers' perceptions of the severity of child problem behaviors were significantly different from that of the normative population. Following participation in the program, results showed a significant decrease in caregiver perceptions of the severity of child problem behaviors, regardless of caregiver/target child demographic variables. Caregiver feedback indicated high levels of satisfaction with the program.
Thesis:
Dissertation (Ph.D.)--University of South Florida, 2009.
Bibliography:
Includes bibliographical references.
System Details:
Mode of access: World Wide Web.
System Details:
System requirements: World Wide Web browser and PDF reader.
Statement of Responsibility:
by Jillian Leigh Williams.
General Note:
Title from PDF of title page.
General Note:
Document formatted into pages; contains 204 pages.
General Note:
Includes vita.

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
aleph - 002063768
oclc - 557284498
usfldc doi - E14-SFE0003010
usfldc handle - e14.3010
System ID:
SFS0027327:00001


This item is only available as the following downloads:


Full Text
xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam 2200409Ka 4500
controlfield tag 001 002063768
005 20100317091655.0
007 cr bnu|||uuuuu
008 100317s2009 flu s 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0003010
035
(OCoLC)557284498
040
FHM
c FHM
049
FHMM
090
LB1051 (Online)
1 100
Williams, Jillian Leigh.
0 245
Helping our toddlers, developing our children's skills (HOT DOCS) :
b an investigation of a parenting program to address challenging behavior in young children
h [electronic resource] /
by Jillian Leigh Williams.
260
[Tampa, Fla] :
University of South Florida,
2009.
500
Title from PDF of title page.
Document formatted into pages; contains 204 pages.
Includes vita.
502
Dissertation (Ph.D.)--University of South Florida, 2009.
504
Includes bibliographical references.
516
Text (Electronic dissertation) in PDF format.
3 520
ABSTRACT: This study investigated outcomes of a parent training curriculum: Helping Our Toddlers Developing Our Children's Skills (HOT DOCS), using secondary analyses of existing data collected between May 2007 and March 2009. The evaluation studied the impact of specific components of the parent training program on both participants' knowledge and attitudes and their perceptions of target children's behavior. Caregivers (n = 334) of children between the ages of 18 months and 5.11 years of age who were participants in the parent training program were included in the study. Measures included a pre/post knowledge test, pre/post rating scales of child problem behavior, and a program evaluation survey. Results indicated significant increases in caregiver knowledge following participation in the program, but non-significant differences between groups of participants based on various demographic variables. Prior to participation in the program, caregivers' perceptions of the severity of child problem behaviors were significantly different from that of the normative population. Following participation in the program, results showed a significant decrease in caregiver perceptions of the severity of child problem behaviors, regardless of caregiver/target child demographic variables. Caregiver feedback indicated high levels of satisfaction with the program.
538
Mode of access: World Wide Web.
System requirements: World Wide Web browser and PDF reader.
590
Co-advisor: Kathy Bradley-Klug, Ph.D.
Co-advisor: Kathleen Armstrong, Ph.D.
653
Behavioral parent training
Group-delivered
Early intervention
Challenging behavior
Problem solving
690
Dissertations, Academic
z USF
x Psychological and Social Foundations
Doctoral.
773
t USF Electronic Theses and Dissertations.
4 856
u http://digital.lib.usf.edu/?e14.3010



PAGE 1

Helping Our Toddlers, Developing Ou r Childrens Skills (HOT DOCS): An Investigation of a Pare nting Program to Address Challenging Behavior in Young Children by Jillian Leigh Williams A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy Department of Psychological and Social Foundations College of Education University of South Florida Co-Major Professor: Kathy Bradley-Klug, Ph.D. Co-Major Professor: Kathleen Armstrong, Ph.D. Constance Hines, Ph.D. Carol Lilly, M.D. Date of Approval: July 10, 2009 Keywords: behavioral parent traini ng, group-delivered, early in tervention, challenging behavior, problem solving Copyright 2009, Jill ian L. Williams

PAGE 2

Dedication This disse rtation is dedicated to the me mories of my grandfathers, Willis Barton Conable and David Edward Williams, who taught me through their words and their actions that this little girl could grow up to become anything she wanted as long as she put her mind to it and did well in school. I believe both of these great men will be watching from above as I become Dr. Willia ms and it is in their honor that I have persevered to achieve this distinction. This dissertation is also dedicated to my parents, David and Jana Williams, who have never wavered in their belief that I would achieve greatness. Throughout my life, they have not only encouraged me to pursue my education but have provided the supports necessary to allow me to dedicate myself completely to my education. Because you have taught me to set my expectations high, I hope I will continue to make you proud as I finish my education and become a professi onal. I love you both and thank you so much for your help in achieving my dreams. Finally, I dedicate these 222 pages to my partner and my best friend, Shane Childres, who has put up with endless hour s of typing, revising, and worrying as I prepared this dissertation. W ho, in return, gave me the love, laughter, and excitement necessary to balance the rigors of finishing my doctoral de gree and beginning my big people career.

PAGE 3

Acknowledgements My thanks and appreciation to my a dvisor and co-chairperson, Dr. Kathy Bradley-Klug who provided consistent training and support throughout my doctoral studies and was the first to re alize that my true calling was in pediatric sc hool psychology and directed me to the Child Development C linic during my first semester at USF. My gratitude also extends to my mentor and co-chairpers on, Dr. Kathleen Armstrong who afforded me the opportunity to gain ha nds-on experience in the development, implementation, and evaluation of the HOT DO CS parent training program and provided me with equal doses of s upervision and friendship throughout my doctoral studies. I thank the members of my dissertation comm ittee, Dr. Constance Hines and Dr. Carol Lilly who have generously given their time and patience in the process of preparing this dissertation for publication. My gratitude also extends to Dr. Ann Cranston-Gingras and Dr. David Allsopp, who volunteered to serve as Outside Chairpersons for the dissertation proposal and defense meetings, respectively. Fi nally, I am forever grateful for the support of the HOT DOCS team, led by Dr. Heather Cur tiss, without whom there would not be a parent training program for me to evaluate.

PAGE 4

i Table of Contents List of Tables v List of Figures viii Abstract ix Chapter 1 Introduction Statement of the Problem 1 Theoretical Framework 4 Overview of the HOT DOCS Parent Training Program 5 Purpose of the Study 6 Research Questions 7 Significance of the Study 8 Definition of Terms 8 Chapter 2 Review of Related Literature Overview 10 Prevalence of Young Children with Challenging Behavior 10 Outcomes Associated with Early Emerging Behavior Problems 13 Role of Parenting in Child Behavior Problems 15 Outcomes Associ ated with Early Intervention 22 Limited Resources for Prevention & Early Intervention 23 Parent Training as an Intervention 25 Parent and Child Charac teristics Affecting Outcomes of Parent Training 36 Parent/Caregiver Education Level 37 Parent/Caregiver Social Support 39 Childs Age 39 Childs Diagnosis 41 Preliminary Investigati on of the HOT DOCS Parent Training Program 42 Rates and Patterns of Caregiver Attendance 44 Comparis on of Participant and Local Demographics 45 Comparis on of Child and Caregiver Demographics with Previous Studies 47 Caregiver Knowledge 48 Caregiver Perceptions of Severity of Child Behavior 48 Changes in Child Problem & Adaptive Behavior 50 Caregiver Skills at Home 51

PAGE 5

ii Caregivers Overall Perceptions of the HOT DOCS Program 52 Conclusions 53 Summary 54 Chapter 3 Methods Introduction 56 Participants 56 Description of Caregivers 58 Description of Target Children 62 Settings 64 HOT DOCS Parent Training Program 64 Session One 65 Session Two 65 Session Three 66 Session Four 66 Session Five 67 Session Six 67 Booster Session 67 HOT DOCS Trainers 68 Desc ription of Train-theTrainers Process 68 Measures 69 HOT DOCS Caregivers Demographics Form 69 HOT DOCS Knowledge Test 70 Achenbach Child Behavior Checklist 70 HOT DOCS Program Evaluation Survey for Caregivers 71 Data Collection Procedures 72 Data Analysis 75 Caregiver Knowledge 75 Caregiver Perceptions of Severity of Child Behavior 76 Changes in Ca regiver Perceptions of Seve rity of Child Problem Behavior 78 Over all Perceptions of th e HOT DOCS Program 79 Chapter 4 Results Overview 81 Caregiver Knowledge 81 Research Question #1a 81 Research Question #1b 82 Caregivers Level of Education 83 Caregivers Social Support 87 Target Childs Age 90 Target Childs Diagnosis 92 Caregiver Perceptions of Severity of Child Behavior 95 Research Question #2a 95

PAGE 6

iii Research Question #2b 98 Caregivers Level of Education 99 Caregivers Social Support 103 Target Childs Diagnosis 106 Changes in Child Problem Behavior 109 Research Question #3a 109 Research Question #3b 113 Caregivers Social Support 114 Target Childs Diagnosis 117 Target Childs Age 121 Caregivers Overall Perceptions of the HOT DOCS Program 125 Research Question #4 125 Summary 133 Chapter 5 Discussion Overview 135 Demographic Characteristics 136 Rate s and Patterns of Care giver Attendance 136 Comparis on of Caregiver Demograp hics with the Local Community and Previous Studies 140 Caregivers Gender/Relationship to Target Child 141 Caregivers Race/Ethnicity 142 Caregivers Level of Education 145 Type of Insurance 146 Caregivers Social Support 147 Target Childs Preexisting Diagnosis 149 Caregiver Knowledge 150 Caregiver Perceptions of Severity of Child Behavior 156 Changes in Child Problem Behavior 159 Caregivers Overall Perceptions of the HOT DOCS Program 163 Implications for Practitioners 165 Limitations 167 Directions for Future Research 169 Conclusion 171 References 173 Appendices 196 Appendix A: HOT DOCS Parent Training Program 197 Appendix B: HOT DOC S Demographic Form for Caregivers (English) 198 Appendix C: HOT DOC S Demographic Form for Caregivers (Spanish) 199 Appendix D: HOT DOCS Knowledge Test (English) 200

PAGE 7

iv Appendix E: HOT DOCS Knowledge Test (Spanish) 201 Appendix F: HOT DOCS Pr ogram Evaluation Survey for Caregivers (English) 202 Appendix G: HOT DOCS Pr ogram Evaluation Survey for Caregivers (Spanish) 203 Appendix H: Relation be tween Research Questions and Variables 204 About the Author End Page

PAGE 8

v List of Tables Table 1 Attendance Record of Initial Caregiver Participant Sample 58 Table 2 Breakdown of Participant Sample by Gender, Race/Ethnicity, Education Level, and Social Support 59 Table 3 Breakdown of Participant Sample by SES Indicator 61 Table 4 Relation of Caregiver to Ta rget Child by Participant Gender 62 Table 5 Number and Percent of Target Children by Preexisting Diagnosis 63 Table 6 Means and Standard Deviati ons for Participant Scores on the Knowledge Test 82 Table 7 Means and Standard Deviations of Preand Posttest Knowledge Test Scores by Caregive rs Education Level 83 Table 8 Analysis of Variance of HOT DOCS Preand Posttest Knowledge Test Scores by Ca regivers Education Level 85 Table 9 Means and Standard Deviations of Preand Posttest Knowledge Test Scores by Caregi vers Social Support 88 Table 10 Analysis of Variance of HOT DOCS Preand Posttest Knowledge Test Scores by Ca regivers Social Support 89 Table 11 Means and Standard Deviati ons of Preand Posttest Knowledge Test Scores by Target Childs Age 90 Table 12 Analysis of Variance of HOT DOCS Preand Posttest Knowledge Test Scores by Target Childs Age 91 Table 13 Means and Standard Deviati ons of Preand Posttest Knowledge Test Scores by Target Childs Diagnosis 93 Table 14 Analysis of Variance of HOT DOCS Preand Posttest Knowledge Test Scores by Ta rget Childs Diagnosis 94

PAGE 9

vi Table 15 Means and Standard Deviati ons for Pretest Scores on the CBCL 96 Table 16 Observed and Expected Fr equencies for CBCL Internalizing & Externalizing Subs cale T-Scores 96 Table 17 Means and Standard Devi ations of CBCL Internalizing and Externalizing Scores by Care givers Education Level 100 Table 18 Analysis of Variance of Pr e-Test CBCL Scores by Caregivers Education Level 102 Table 19 Means and Standard Devi ations of CBCL Internalizing and Externalizing Scores by Care givers Social Support 104 Table 20 Analysis of Variance of Pr e-Test CBCL Scores by Caregivers Social Support 105 Table 21 Means and Standard Devi ations of CBCL Internalizing and Externalizing Scores by Ta rget Childs Diagnosis 106 Table 22 Analysis of Variance of Pre-Test CBCL Scores by Target Childs Diagnosis 107 Table 23 Means and Standard Deviat ions of Preand Posttest CBCL Scores by Scale 110 Table 24 Analysis of Variance of CBCL Preand Posttest Scores 111 Table 25 Means and Standard Deviat ions of Preand Posttest CBCL Scores by Caregivers' Social Support 114 Table 26 Analysis of Variance of CBCL Preand Posttest Scores by Caregivers Level of Social Support 116 Table 27 Means and Standard Deviat ions of Preand Posttest CBCL Scores by Target Child's Diagnosis 118 Table 28 Analysis of Variance of CBCL Preand Posttest Scores by Target Child's Diagnosis 120 Table 29 Means and Standard Deviat ions of Preand Posttest CBCL Scores by Target Child's Age 122

PAGE 10

vii Table 30 Analysis of Variance of CBCL Preand Posttest Scores by Target Child's Age 124 Table 31 Ratings of Participant Satisfaction with the HOT DOCS Training Program 127 Table 32 How are you using the inform ation you learned in HOT DOCS? 129 Table 33 Have you shared informa tion from HOT DOCS with? 130 Table 34 What can we do to improve HOT DOCS? 131 Table 35 What did you value most a bout taking the HOT DOCS class? 132

PAGE 11

viii List of Figures Figure 1. Mean Knowledge Test scores by caregivers ed ucation level. 87 Figure 2. Number of expected and observed CBCL T-scores by descriptive category. 98 Figure 3. Mean CBCL Internalizing a nd Externalizing subscale scores by caregivers education level. 103 Figure 4. Mean CBCL Internalizing a nd Externalizing subscale scores by target childs diagnosis. 109 Figure 5. Preand posttest mean scores for CBCL scales. 112

PAGE 12

ix Helping Our Toddlers, Developing Ou r Childrens Skills (HOT DOCS): An Investigation of a Pare nting Program to Address Challenging Behavior in Young Children Jillian Leigh Williams ABSTRACT This study investigated outcomes of a parent training curriculum: Helping Our Toddlers Developing Our Child rens Skills (HOT DOCS) using secondary analyses of existing data collected between May 2007 and March 2009. The evaluation studied the impact of specific components of the pare nt training program on both participants knowledge and attitudes and thei r perceptions of target child rens behavior. Caregivers ( n = 334) of children between the ages of 18 months and 5.11 years of age who were participants in the parent training program were included in the study. Measures included a pre/post knowledge test, pr e/post rating scales of child problem behavior, and a program evaluation survey. Results indicat ed significant increases in caregiver knowledge following participat ion in the program, but non-significant differences between groups of participants based on various demographic variables. Prior to participation in the program, caregivers perceptions of the severi ty of child problem behaviors were significantly different from that of the normative population. Following participation in the program, results show ed a significant decrease in caregiver perceptions of the severity of child problem behaviors, regardless of caregiver/target child demographic variables. Caregiver fee dback indicated high le vels of satisfaction with the program.

PAGE 13

1 Chapter 1 Introduction Statement of the Problem After nearly three decades of cross-disc iplinary research, professionals in the fields of psychology, education, and medicine are no longer surprised that their client lists, student rosters, and appointment schedul es are filled with young children displaying challenging behaviors. The most comm only cited challenging behaviors in young children (between the ages of 18 months and 5.11 years old) include sleeping difficulties, mealtime and feeding issues, toilet training, temper tantrums, aggression, sibling rivalry and noncompliance. Recent research has s hown that approximately 15%-25% of all typically developing preschool children have ch ronic levels of behavior problems that fall within the mild to moderate range (Cam pbell, 1995; Keenan & Wakschlag, 2000; Knapp, Ammen, Arstein-Kerslake, Poulsen, & Ma stergeorge, 2007; La vigne et al., 1996). Prevalence rates of chronic behavior problem s for minority children and/or children in low-income families have been identified as ranging between 25% and 35% of typically developing children (Gross, Sambrook, & Fogg, 1999; Webster-Stratton, 1998). A more recent community study of approximately 8,000 families indicated that 10% of infants between the ages of birth and one-year scored within the significan t range on a screening instrument for emotional and behavioral prob lems and 6% of one-year-old children were

PAGE 14

2 scored as significantly high by parents on the Difficult Child scale of the Parenting Stress Index (Beernink, Swinkels, & Buitelaar, 2007). The long-term outcomes associated with early onset challeng ing behavior in young children have been well-documented (Coie & Dodge, 1998; Dishion, French, & Patterson, 1995; Kazdin, 1995; Moffitt, 1993; Reid, 1993; Tremblay 2000). In general, the earlier the problem behavior develops th e more stable and intense the associated negative outcomes are over time. Dishion and colleagues (1995) found that early appearing behavior problems in a childs pres chool career are the single best predictor of delinquency in adolescence, gang membership, and adult incarceration. Other researchers have identified similarly poor long-term outcomes related to academic and school performance. Kazdin (1993) and Tremblay (2000) concluded from their research that preschoolers with challenging behaviors are at a greater risk of experiencing school failure than typically developing children. Several studies have investigated the poor social and interpersonal outcomes associated with developing challenging behavior s at an early age. Coie and Dodge (1998) found that preschoolers with challenging behavi ors were more likely to experience early and persistent peer rejecti on. Strain and his colleagues ( 1983) reported that preschoolers with challenging behaviors also were more likely to experience more punitive interactions with teachers th an their typically developing pe ers. Reid (1993) found that early appearing aggressive be havior is the single best predictor of juvenile gang membership and violence. In response to research demonstrating the rapid and enduring increase in the prevalence rates of young children with cha llenging behaviors and the associated

PAGE 15

3 negative long-term outcomes, professionals acr oss disciplines have developed a variety of treatments to help prevent and treat th ese behaviors. For example, psychotropic medications (Barkley, 1997), individual clinical therapy or counseling with the child (Barkley et al., 2000; Foreha nd & Long, 1988), individual consultation with the family (Anastopoulos, Shelton, DuPaul, & Guevrem ont, 1993; Feinfield & Baker, 2004), play therapy (Blackwell, 2005; McNeil, Capage, Bahl, & Blanc, 1999; Nixon, Sweeny, Erickson, Touyz, 2003), and behavioral parent training (Kazdin, 1997; Sanders, Mazzucchelli, & Studman, 2004; Webster-Stratton, 1998) have all been evaluated for their efficacy in reducing challenging behavi or in young children. Of these interventions and treatments, behavioral pare nt training delivered in a group format has been shown to be an effective treatment for challenging behavior in young children, while utilizing the least amount of resources and empowering pare nts to prevent the de velopment of future problem behaviors (Barlow & Parsons, 2002; Lundahl, Risser, & Lovejoy, 2006; Maughan, Christiansen, Jenson, Olympia, & Clark, 2005; Nelson, 1995; Sandall & Ostrosky, 1999; Smith & Fox, 2003). Despite the available evidence supporting the effectiveness of early intervention, there is a lack of services, resources, and empirically-supported interv entions available to caregivers of young children displaying challe nging behavior (Kazdin & Kendall, 1998; Knitzer, 2007; Walker et al., 1998). Based on the abundance of research supporting the primary role of parents a nd caregivers in young childrens emotional and behavioral development, it follows that the most logical target for prevention a nd early intervention efforts would be improving caregiving sk ills and enhancing th e caregiver-child relationship (Knitzer, 2007). Thus, group-base d parent training woul d be an economical

PAGE 16

4 and ecologically-based system for providing children and families with the knowledge, skill, and support they need to prevent a nd correct challenging behaviors (Smagner & Sullivan, 2005). The ability of one child care professional to simultaneously meet the needs of multiple families and children at once dramatically increases the efficiency of limited resources, professionals, and funding sources. Researchers also have investigated differential outcomes for caregivers who attend behavioral parent training programs based on demographic variables and social characteristics. Specifically, studies have s hown that variables, such as caregivers educational level and degree of social support, as well as childs age and severity of symptoms or existing diagnosis may impact the degree of success in a given parent training program (Bakermans-Kranenburg et al., 2003; Reyno & McGrath, 2006; Stolk et al., 2008). In order to maximize the effectiven ess of an intervention, it is important to identify factors that will allo w practitioners to appropriately match clients with treatments in which they are likely to succeed (Sm ith, Landry, & Swank, 2005). More research is needed to specifically identify parent a nd child characteristics which may affect successful completion and outcomes of participating in a parent training program. Theoretical Framework Historically, one of the major theories guiding the inquiry into chronic behavior problems in young children is Skinners (1953) theory of behavioris m. At its foundation, behaviorism postulates that all behavior is ob servable and functional. Behaviorism relies on the manipulation of antecedents and conse quences and the effects of reinforcement and punishment as a means of changing and shaping behavior. In addition to approaching the study of challenging behavior in young children from a behavioral theoretical

PAGE 17

5 framework, it is necessary to view the problem through an ecological model of child development (Bronfenbrenner, 1979). An ecological model takes into account biological, sociological, and psychological dom ains of child development and functioning (Sontag, 1996). From an ecological perspective, manipul ation of a childs environment, including the behavior of caretakers, will directly impact the childs behavior (Bronfenbrenner, 1979). Given the above, an intervention progr am designed with the principles of behaviorism and ecology in mind would seem promising. Overview of the HOT DOCS Parent Training Program HOT DOCS, or Helping Our Toddlers Developing Our Childrens Skills (Armstrong, Lilly, & Curtiss, 2006) is a be havioral parent training program, which incorporates both behavioral and ecological perspectives in its theoretical framework. HOT DOCS meets the following criteria for a behavioral interventi on: 1) centers around an operant model of behavior 2) provides caregivers wi th detailed information on effective caregiving strategies, 3) focuses on control of antecedents instead of punitive consequences, and 4) enhances generalizati on from the training setting to the home setting. HOT DOCS was designed to teach pa rents a problem-solving process based upon the foundation of behavioral principles (e.g., antecedents, consequences, and function of behavior) delivered in parent-friendly langua ge. Unlike other parent training programs that focus on teaching parents to fix speci fic behavior problems, HOT DOCS teaches caregivers to use a step-by-step method to identify features of the environment and interpersonal interactions that may contribute to the reinforcement or maintenance of current and future problem behaviors. HOT DO CS also focuses on instructing caregivers to recognize that children may use challe nging behaviors because they lack the

PAGE 18

6 knowledge or skills to use more appropriate behaviors. Program developers note that HOT DOCS is different from other existing parent training programs because it directs caregivers to teach children new or replacemen t skills instead of focusing on contingency management strategies designed to el iminate problem behaviors (Armstrong & Hornbeck, 2005). HOT DOCS also is unique in that th e same curriculum can be delivered to parents, relatives, other caregiv ers, and child service profes sionals all in the same group at the same time. This allows parents to bring their support network w ith them to classes, which enhances the consistency of skill implementation across caregivers and across settings outside of the training session. Othe r programs (e.g., Incredible Years, WebsterStratton, 1998; Triple P-Positive Parenti ng Practices, Sanders, 1999) have separate curricula for parents, childre n, teachers, and caregivers. Another major difference between HOT DOCS and existing parent training programs is the total time required for pr ogram delivery. The HOT DOCS program is delivered over six consecutive weekly sessions of two hours each, with a 2-month post training booster session, for a total of 14 hour s of classroom-based instruction. Other programs average 12-30 weekly sessions or a total of 24-60 hours of training. The program is in its fourth year of impl ementation, but no comprehensive study of the impact of the intervention on caregivers' knowledge and perceptions of target children's behavior has been undertaken. Purpose of the Study The current study was designed to serve as an evaluation of participants' knowledge and attitudes and as an investigation of participants' perceptions of target

PAGE 19

7 children's behavior following participation in the HOT DOCS parent training program. In addition, this study investigated the extent to which select demographic variables (i.e., caregivers' level of education, caregivers' so cial support network, targ et child's age, and target child's diagnosis) moderated pa rticipants' outcomes and perceptions. Research Questions The following research questions were addressed in the study: 1. a. What is the impact on caregiver know ledge of child development, behavioral principles, and parenting stra tegies as a result of pa rticipation in the HOT DOCS parent training program? b. Is there a difference in participant caregivers knowle dge of child development, behavioral principles, and parenting strategies base d on caregivers' level of education, caregivers' social support network, the target child's age, and the target child's diagnosis? 2. a. Do caregivers perceive their child as having more problem behavior than a normative sample prior to participation in the HOT DOCS program? b. Are there significant differences in careg iver perceptions of the severity of child problem behaviors based on caregivers level of education, caregivers' social support network, and the ta rget child's diagnosis? 3. a. To what extent do caregivers perceive a decrease in child problem behavior following their participation in the HOT DOCS program? b. Are there differential per ceptions of child behavior change based on caregivers' social support network, the target child's diagnosis, and the target child's age?

PAGE 20

8 4. What are caregivers overall perceptions of the HOT DOCS parent training program? Significance of the Study This study investigated whether or no t the HOT DOCS program was an effective intervention for increasing car egiver knowledge of behavioral practices, whether or not participation in the program impacted careg ivers' perceptions of child behavior, and whether there were differential perceptions and outcomes for specific groups of caregivers. Anticipated contributions to the general knowledge base included: 1) a better understanding of the utility of a group-delivered, behavioral parent training program, which was specifically designed to teach caregivers to use problem-solving strategies to prevent and address challenging behavior in young children while maximizing resource allocation and cost-effectiven ess; and 2) information a bout differential outcomes for various groups of participants based on dem ographic characteristics. In addition, results of this study will be shared with the HOT DO CS program developers in order to facilitate the process of modifying and improving the inst ruments and procedures used to evaluate outcomes of the HOT DOCS parent training program and to help improve and refine the content and delivery of the program. Definition of Terms Young children is defined for the purposes of th is study as children between the ages of 18 months and 5.11 years of age. Behavioral parent training is defined as an intervention technique in which professionals provide training in specific behavioral pare nting skills and techniques to parents and caregivers of young children. Behavi oral parent training programs generally

PAGE 21

9 have four common elements: 1) centers around an operant model of behavior, 2) provides parents with detailed information on appropriate and effective pare nting strategies, 3) focuses on control of antecedents instead of punitive consequences, and 4) enhances generalization from the training setting to th e home and community settings (Fienfield & Baker, 2004). Challenging behavior is defined as a pattern of repeated behaviors that place children at risk of poor developmental outco mes in learning and social interactions (Dunst, Trivett, & Cutspec, 2002). Challenging behavior is therefore defined on the basis of its effects. Caregivers is an inclusive term used throughout this study to refer to all parents, relatives, and child care professionals w ho participated in the HOT DOCS program. Child care professional refers to participants who indicate that they attend HOT DOCS classes in their role as a service provider in a field addressing early childhood development, including early intervention spec ialists, medical and psychiatry students and residents, occupational/physical/speech therapists, behavioral analysts, daycare providers, and teachers. The designation of child service provider is made by the participants themselves when asked to indi cate their relationship to the target child. While providers may also have children of their own and use the sk ills and techniques learned in the class with their own families, the term provider indicates that their primary purpose in attending the HOT DOCS classes is to devel op knowledge and skills to support their clients and patients. Trainers is used to refer to professionals who provided leadership for HOT DOCS parent training classes.

PAGE 22

10 Chapter 2 Review of Related Literature Overview This chapter provides a review of the lit erature relevant to this study. Challenging behavior in young children is discussed, including prevalence rates, negative outcomes associated with early emerging behavior problems, and the role of parenting skills in the development of challenging behavior. Res earch supporting the impor tance of prevention and early intervention is reviewed, as well as the effectiveness of parent training as an intervention and differential outcomes based on demographic and social variables. This chapter concludes with a discussion of the importance of providing effective behavioral parent training through an ecological-behav ioral framework to enable parents and caregivers to prevent and corre ct challenging behavior in young children as early as possible. Prevalence of Young Children with Challenging Behavior Numerous studies conducted over the past 30 years have shown a dramatic increase in the number of young children w ho are referred to professionals due to challenging behaviors (Campbell, 1995; Jolive tte, Gallagher, & Morrier, 2008; Knapp, et al., 2007; Lavigne et al., 1996). Studies report that up to 75% of all psychological referrals for children are rela ted to disruptive and noncompliant behavior (Feinfield & Baker, 2004). Researchers also have found th at the proportion of children meeting the

PAGE 23

11 criteria for a clinical diagnosis of oppositio nal defiant disorder (ODD) ranges between 7% and 25%, depending on the age of the populat ion surveyed (Webster-Stratton, 2000). Overall, the prevalence rate for challenging behaviors in young children varies between 10% and 16% for the general popul ation (Campbell, 1995; Schuchmann, Foote, Eyberg, Boggs, & Algina; 1998; Webster-St ratton, 2000) and between 25% and 30% of the 7.6 million infants and children living be low the poverty level (Gross et al., 2003; Keenan & Wakschlag, 2000; Knitzer, 2007; Qi & Kaiser, 2003). More recently, researchers have begun to investigate the pr evalence rates of emotional and behavioral problems in infants and children younger than preschool age. Results of these preliminary projects indicate that up to 10% of infants and 6% of 12-month-old children had significant emotional and behavioral probl ems (Beernink et al., 2007). In the same survey, 70% of the parents surveyed whose infants and young child rens behavior did not reach clinically significan t levels reported that their ch ildren frequently displayed behaviors such as quickly sh ifts activities, angry moods, and demands must be met immediately (Beernink et al., 2007). Gross and colleagues (2003) conducted a cro ss-sectional study of 2and 3-yearold children from low-income families to describe the prevalence rates and correlates of challenging behaviors in preschool childre n. The study included parents of 133 young children from 10 daycare centers in an urban city. Most of the parents included in this study were African American (64%) or Latino (25%) and were categorized as being low in socio-economic status based on income level (e.g., 50% of participants earned less than the states median income). Parents co mpleted measures of type and intensity of child behavior problems, parenting self-e fficacy, parental discip line strategies, and

PAGE 24

12 parental stress. Findings from the study showed that 32% of the young children had clinically significant levels of problem behaviors in the home setting. These results should be interpreted with caution given that the sample was composed of two minority ethnic groups of low socio-economic status (SES). Results from these findings should only be generalized to similar populations. In 2003, Qi and Kaiser conducted a review of research pertaini ng specifically to challenging behaviors in young children from low-income families. These researchers reviewed and summarized research on this topic published between 1991 and 2002 with the goal of synthesizing prevalence rates of behavior problems and identifying risk factors for behavior problems. Results of this study showed that children whose families are poor are significantly more likely than middleor upper-class families to develop behavior problems. Findings from this revi ew were similar to previous reports in estimating that prevalence rate of challe nging behavior for children from low-SES families is approximately 30% (Gross et al ., 1999; DelHomme, Sinclair, & Kasari, 1994; Feil, Walker, Severson, & Ball, 2000). Keenan and Wakschlag (2000) conducted a study to examine the severity of challenging behaviors exhibi ted by preschool-aged childre n. The authors completed comprehensive psychological evaluations with 79 clinic-referred preschoolers from a primarily low-SES, urban setting. The comprehensive evaluations included semistructured diagnostic parent interview (Schedule for Affec tive Disorders for School-Age Children-epidemiological 5th version; Orvaschel & Puig -Antich, 1995), child behavior rating scales (Child Behavior Checklist; Achenbach, 1991), direct obser vations of parentchild interactions, developmental assessment (Differential Abilities Scales; Elliot, 1983),

PAGE 25

13 and overall clinical impairment ratings (Child Global Assessm ent Scale; Setterberg, Bird, Gould, Shaffer, & Fisher, 1992). Results indi cated that nearly 80% of the preschool children met Diagnostic a nd Statistical Manual-4th Edition (DSM-IV, American Psychiatric Association, 1994) criteria for a disruptive behavior or AttentionDeficit/Hyperactivity Disorder. Specifically, 60% of the children met criteria for Oppositional Defiant Disorder (ODD) and 42% met criteria for Conduct Disorder (CD). These findings support the growing body of re search identifying increasing prevalence and severity rates of disrup tive behaviors in young children. Outcomes Associated with Early Emerging Behavior Problems Preschool years are a time identified by immense developmental challenges, which may include temporary bouts of problem behaviors, many of which resolve without any professional help or targeted intervention (Magee & Roy, 2008). However, unresolved early emerging behavior problems are early warning signs of much more serious future behaviors (Magee & Roy 2008). The problem of increasing prevalence rates of challenging behavior in young children becomes more significant when the longterm outcomes associated with early-emerging behavior problems are taken into account. Children who are identified as ha rd to manage at ages 3 and 4 years of age are twice as likely as their typically-developing peers to c ontinue to display problem behavior into adolescence (Beernink et al., 2007; Cam pbell & Ewing, 1990; Egeland, Kalkoske, Gottesman, & Erikson, 1990; Fischer, Rolf Hasazi & Cummings, 1984; Magee & Roy, 2008). Egeland and colleagues (1990) conducted a longitudinal study in which they assessed the stability of beha vior problems in children beginning in preschool and

PAGE 26

14 following-up again when children reached the 3rd grade. Parents of 118 children between the ages of 4 and 5 years of age completed child behavior rating s cales and measures of parental stress and mental health when th eir child turned 3 years old and entered preschool and again when they began 3rd grade. Assessments also included direct observations of child behavior and semi-structu red parent interviews. Ninety-six children met criteria for problem behaviors includi ng acting out, withdraw al, or inattention. Twenty-two children did not meet criteria and served as the control group. Results indicated a high degree of stability in th e presence of child problem behaviors. A limitation of this study was that the assessments of children's behaviors were only conducted at two points in time (3 years and 5 years of age), which excludes a critical period in the development of early emerging behavior problems occurring between 2 and 3 years of age. A similar study conducted by Campbell and Ew ing (1990) tracked the stability of behavior problems first identified in the pr eschool years; however, in this study, followup assessments were conducted at age 6 years and again at 9 years of age and focused specifically on the children who were exclude d from the age range in the previous study. Parents of 51, three-year-old children comple ted behavior rating sc ales, parenting stress indices, semi-structured interv iews and participated in dir ect observations of behavior. Assessments were conducted at three point s in time, first when children entered preschool at 3 years of age, ag ain at 6 years of age, and agai n at 9 years of age. Twentynine of the children were classified as hard-to-manage and 22 children served as developmentally appropriate control group p eers. Results of this study showed that children who exhibited clinically significant problem behavior at 3 years of age were

PAGE 27

15 more likely than same-aged peers who did not exhibit problem behavi ors to continue to demonstrate problem behaviors at ages 6 and 9 years of age. Results also showed that the majority (67%) of children who had clinically significant behavior problems at 6 years of age met Diagnostic and Statistical Manual-3rd Edition (DSM-III; Am erican Psychiatric Association, 1987) criteria for externalizing disorders at age 9 (e.g., ADHD, OCD, CD). Young children who demonstrate challenging behavior in the preschool years are more likely to experience school failure (Kazdin, 1993; Tremblay, 2000), peer rejection (Coie & Dodge, 1998), punitive teacher interactions (Strain, Lambert, Kerr, Stragg, & Lenker, 1983), and unpleasant family inte ractions (Patterson & Fleischman, 1979). Preschoolers with early-emerging challenging behavior are also more likely to develop adult lives characterized by violence, abuse, loneliness, psychiatric illness, injury, unemployment, divorce, and early death (Coie & Dodge, 1998; Kazdin, 1995; Lipsey & Derzon, 1998; Olweus, 1991; Walker, Colvin, & Ramsey, 1995). Role of Parenting in Child Behavior Problems Much of the recent research conducted in the fields of psychology and education has focused on the etiology of challenging be havior in young children. A major theme to emerge in this body of research is that pare nting style and parent-c hild relationships are significant determinants of child mental he alth problems, including challenging behavior (Jolivette et al., 2008; Loeb er & Dishion, 1983; Patterson, DeBaryshe, & Ramsey, 1989; Rutter, 1991; Stormshak, Bierman, McMahon, Le ngua, 2000). Studies have shown that a common factor in the etiology of most childhood behavior problems and social-emotional disorders is difficulty in the parent-ch ild relationship (Kendziora & OLeary, 1993; Mrazek, Mrazek, & Klinnert, 1995 ; Patterson et al., 1989; Ru tter, 1991; Shaw, Emery, &

PAGE 28

16 Turner, 1993). Magee and Roy (2008) found th at even when all environmental riskfactors were accounted for, children of mothers with less parenting ability were 41% more likely than children of mothers with mo re adequate parenting skills to display challenging behaviors by the time they entere d school. Negative parent-child interaction styles are more frequently observed in families with young children with behavior problems and are predictive of more persis tence in disruptive behaviors (Buss, 1981; Feinfield, 1995; Pettit, Bates, & Dodge 1993; Webster-Stratton, 1985). Pattersons (1982) coercion model explains how negative parent-child interactions lacking warmth and negotiation serve to exacerbate a childs problem behaviors, especially aggression. Parenting skill deficits produce combinations of oppositional and avoidant behaviors in children, which in turn increase parental negativity towards the children (Bradley et al., 2003; Brenner & Fox, 1998, Cummings & Davies, 1994). The result of prolonged coercive interactions is a st rained parent-child relationshi p and persistent challenging child behavior (Patterson, 1982). In contra st, recent research involving the study of resiliency in children's early development indi cated that effective parenting is the most powerful protective factor (Lut har, 2006; Singer, Ethridge, & Aldana, 2007). Thus, harsh and inconsistent parenting places children at risk for problematic development, while positive and consistent parenting can serve as a protective factor against other environmental risks. Denham and colleagues (2000) conducted a study to examine the contribution of parental emotions and behaviors to the emergence of disruptive and noncompliant behaviors in preschool child ren. The study included 79 mothers and fathers and their children, who met criteria for being at-ris k for development of disruptive behavior

PAGE 29

17 disorders. Children involved in this study range d in age from 2 years to 5 years of age, with a mean age of 4 years. Participants in this study were predominantly Caucasian (96%) and from a middleor upper-class soci o-economic status (96%). Families were evaluated at four times duri ng the 4-year longitudinal st udy, including a pretest, two progress monitoring evaluations, and a postt est. Researchers assessed childrens externalizing behavior thr ough parent and teacher report s using Achenbachs (1991) Child Behavior Checklist (CBCL) and Teacher Report Form (TRF), as well as Youth Self-Reports (YSF). Parenting skills were a ssessed at the first and fourth assessment through direct observation of parents interactions with their children in naturalistic play activities. Parenting patterns were coded for patterns of behavior, including supportive presence, limit setting, allowance of autonomy, negative affect, quality of instruction, and confidence. Parenting patterns also were coded for emoti onal expression, including anger and happiness. Results of the study indicated that children with externalizing problems evident during the pre-test cont inued to have behavior proble ms at the 2-year and 4-year follow-up evaluations. Results also demonstrat ed that proactive parenting techniques (e.g., being supportive, giving clear directions, setting limits) predicted decreased behavior problems overtime, especially for ch ildren with clinically significant levels of problem behaviors at pre-test. Conversely, ch ildren of parents who frequently expressed anger were more likely to have continued or worsening externalizing behaviors at the follow-up evaluations. The results of this study should be interpreted with caution, given the limited diversity in ethnicity and SES of the participants included and the small sample size.

PAGE 30

18 Other studies have shown that parent s of young children with externalizing behaviors use more frequent verbal and corporal punishment than parents of young children without challenging behaviors (Nicholson, Fox, & Johnson, 2005). Nicholson and colleagues (2005) conducted a study invest igating the difficu lties of parenting children with challenging behavior as well as the protective f actors that may exist in these families. Preschool teachers identified 30 ch ildren (ages 2 to 5 years) who displayed challenging behaviors and a matched gr oup of 30 children who did not display challenging behaviors to serve as the comp arison group. Teacher classification of child behavior problems was confirmed using the Sutter-Eyberg Student Behavior Inventory (Eyberg & Pincus, 1999). The final sample consisted of 60 children and their mothers who were mostly Caucasian (93%), married (78%), and had a minimum of a high school diploma (72%). Each mother was asked to co mplete a self-report measure of parenting behavior (Parent Behavior Checklist; Fox, 1994) and two rating scales of child behavior (Child Behavior Scale (CBS); Fox & Nichol son, 2003; Eyberg Child Behavior Inventory (ECBI); Eyberg & Pincus, 1999) during a hom e interview. With regard to parent behavior, significant results were found (p<.05) in the differences between the parenting practices of mothers of children with challe nging behavior and moth ers of children with typical behaviors. Specifical ly, mothers of children with challenging behavior reported more frequent use of verbal and corporal punishment than mothers in the control group. No differences were found between the mo thers use of nurturing behaviors or expectations. With regard to child beha vior, mothers of children with challenging behavior rated their children s behavior at home to be significantly more problematic than mothers in the control group on both the ECBI and CBS. Results of this study

PAGE 31

19 indicated that mothers of children with teac her-identified challengi ng behavior interact with their children differently than mothers of children without challenging behaviors. This study provided evidence of differences in parenting practices in families of children with typical and challenging behavior, how ever, generalization of these results are limited due to a small sample size and homogenous participant demographics. The conclusions of this study are also limited by the use of only self-report measures and no direct observations of pa rent or child behavior. A similar study by Stormshak and colleagues (2000) also investig ated differences in parent-child interactions in families with children with challenging behavior, but avoided the problem of limite d generalizability in the prev ious study by selecting a more diverse sample. This study was conducted with a large population-based sample of at-risk and diverse 1st grade students from four locatio ns across the United States (North Carolina, Tennessee, Washington, and Pennsylvania). The sample included 631 kindergartners (mean age 6.45 years) with chal lenging behavior from various ethnic and racial groups (49% minority-predominantly Af rican American, 51% European American) and socio-economic status levels as well as a matched comparison sample of 387 children without challenging behaviors. Measures used in this study incl uded parent (Child Behavior Checklist; Achenbach, 1991) and teacher reports (Teach er Observation of Classroom Adaptation-Revised; Kellem, 1989) of child behavior and several self-report measures of parenting pract ices (Conflict Ta ctics Scale; Straus, 1989; Parent Questionnaire; Strayhorn & Weidman, 1988; Pa renting Practices Inventory; CPPRG, 1996). Results indicated that parents who re ported that their children had challenging behaviors also reported significantly more frequent use of punitive discipline strategies

PAGE 32

20 and aggressive parenting styl es (e.g., yelling, spanking, thr eatening) than parents who reported their childrens behavior to be w ithin normal limits. Punitive discipline and inconsistent parenting were significantly a ssociated with child oppositional, aggressive, and hyperactive behaviors. With the excepti on of a stronger relationship between punitive discipline strategies and child problem beha viors for African American than European American parents, there were no significant differences between ethnic groups across parenting practices or child behavior found in this study. Th is lack of significant group differences suggests a high degree of consistenc y in the influence of parenting practices on child behavior across ethnic groups in Amer ica. Similar to previous studies, the absence of direct assessment of child beha vior, parenting practi ces, and parent-child interactions presents a limitation to the results of the study. While negative parenting practices can pr oduce or exacerbate problem behavior in children, child problem behaviors can also lead to increased le vels of parent stress, and marital conflict (Forehand & Long, 1988; Pa tterson, Reid, & Dishion, 1992; WebsterStratton & Hammond, 1997). Following the cyclic model, elevated levels of chronic parental stress are associated with the maintenance of extern alizing behavior problems in children (Campbell, 1997; Heller, Baker, Henker, & Hinshaw, 1996). Recent research also has shown that nurturing, authoritative, responsive parenting th at utilizes positive behavioral interventions can improve child be havior, enhance child development, reduce the need for professional services in the futu re and reduce parent st ress (Hebbler et al., 2001; Nicholson et al., 2005; Ramey & Ramey, 1998; Shonokoff & Phillips 2000). Pettit and colleagues (1993) conducted a longitudinal study investigating the family interaction variables that were predictive of childrens externalizing problems

PAGE 33

21 during the transition from kindergarten to 1st grade. Specifically, the researchers investigated the hypothesis that positive-proactive and negati ve-coercive parenting styles would make independent, non-overlapping contributions to the prediction of conduct problems in children. The sample included 165 families who were recruited from a larger, ongoing study (see Dodge, Bates, & Pett it, 1990). The sample consisted of a range of social classes (high, middle, and low in come families) and equal numbers of boys ( n = 82) and girls ( n = 83). The sample was predominantly White (84%) and represented twoparent families (70%). The children were stra tified into groups of high, medium, and low aggression based upon mothers ratings of child aggression on the Child Behavior Checklist (Achenbach & Edelbr ock, 1983). All children were observed in their homes during the summer prior to beginning kindergarten using a focused-narrative observational system to code various family interactions. Observations were conducted on two separate occasions for each family, lasting approximately two hours each, and were typically conducted during or near dinner time. Families were instructed to proceed with their normal routines and behaviors and attempt to ignore the observers as much as possible. In addition to the di rect observations, parents comp leted child behavior rating scales. All three data collection methods (hom e observations, parent rating scale, and teacher rating scale) were completed again a year later, in the su mmer prior to children beginning 1st grade. Results indicated a strong correlation (p < .05) between negativecoercive parenting by mothers and child ex ternalizing behavior problems in and 1st grade (behaviors rated by both parents and teacher s). Correlations between negative-coercive parenting by fathers and child externalizing behavior problems were not significant at the kindergarten or first grade levels. This study also found that early, positive parent-child

PAGE 34

22 and family interactions predicted lower leve ls of externalizing behavior problems in kindergarten and first grade. These results pr ovide support for the significant influence of parenting styles and parent-child interac tion patterns on child behavior problems. Outcomes Associated with Early Intervention Despite the projections of negative shor tand long-term outcomes for children who develop challenging behaviors at an earl y age, research has shown that the use of evidence-based intervention techniques can pr event and alleviate many of the associated negative outcomes (Jolivette et al., 2008; Marchant, Young, & West, 2004; Walker et al., 1998; Webster-Stratton, 1998). Marchant and co lleagues (2004) recently demonstrated that prevention strategies implemented as ear ly as the preschool years helped children avoid more severe problems later in life. In this study, four 4-year-old children who were considered to be at-risk for developing antis ocial behavior and thei r parents participated in an intervention training program. During the training phase, the parent coach (first author) developed a collaborative relationship with parents, tr ained parents to use specific parenting skills, and provided parents with im mediate feedback on their use of the skills. Specific skills included a direct teaching sequence aimed at increasing child compliance with multi-step directions and a corrective teaching sequence used when the child was non-compliant with adult direction. The dire ct teaching sequence included describing the skill (compliance) and the steps the child s hould follow, giving reasons that show the benefit of compliance, showing or modeling the steps of compliance for the child, and giving the child feedback in the form of praise or correction. The corrective teaching sequence included being positive (praise), describing the incorrect behavior, prompting the correct behavior (role play if necessary ), and praising the child for listening and

PAGE 35

23 trying again. The study used a multiple baselin e design across the four parent-child dyads to investigate parent and child behaviors in baseline, training, coaching, and follow-up phases. Results of the study showed that ch ildren as young as 4 year s old were able to show improvements in their behavior fo llowing a brief parent-child intervention. Limitations of this study in cluded the small sample size and a homogenous sample in terms of ethnicity (all four families were Ca ucasian). Despite its limited generalizability, the results of this study sugge st that early intervention for challenging behaviors in young children can be effective with chil dren as young as 4 years of age. When parents use responsive parenti ng practices and positive behavioral interventions in the early years, behavior problems are less entrenched, easier to treat, and the potential impact upon future developmenta l trajectories is greater (Dunlap & Fox, 1996; Lutzker & Campbell, 1994; WebsterStratton, 1998). In other studies, early intervention has been associat ed with a decreased risk of withdrawal, aggression, noncompliance, teen pregnancy, juvenile deli nquency, and special education placement (Strain & Timm, 2001). The application of evid ence-based treatment approaches has also been associated with increas ed self-control, self-monitori ng, self-correction, and socialemotional health (Webster-Stratton, 1990); mo re positive peer relationships and social skills (Denham & Burton, 1996); and improved academic success (Walker et al., 1998). Limited Resources for Prevention & Early Intervention Despite the available evidence supporting the effectiveness of early intervention, there is a lack of services, resources, and empirically-supported interv entions available to caregivers of young children displaying challe nging behavior (Kazdin & Kendall, 1998; Knitzer, 2007; Walker et al., 1998). Recent esti mates have shown that fewer than 10% of

PAGE 36

24 young children who show early signs of problem behavior receive services for their difficulties (Kazdin & Kendall, 1998). For t hose children who do receive services, the outcomes may still be bleak, considering resear ch findings that the developmental course of challenging behavior is predictably negativ e for children who are not treated or who receive poor treatment (Lipsey & Derzon, 1998; Patterson & Fleishman, 1979; Wahler & Dumas, 1986). Kumpfer and Alvarado (2003) also suggested that a lack of professional training in evidence-based intervention approaches may be contributing to small effect sizes in prevention and interventi on research. The lack of available services is even more dismaying in the light of re search findings showing that if challenging behaviors are not altered by the time a child r eaches the age of nine years, the behavior problems are considered chronic and will re quire continuing and costly intervention (Dodge, 1993). Investigations into federal, state, an d local policies have shown that obtaining funding to procure early interv ention services for families of children with challenging behavior is one of the key barriers to positive outcomes (Knitzer, 2007; Smagner & Sullivan, 2005). Knitzer (2007) commented that despite the incontrovertible evidence offered in support of the positive outcomes a ssociated with addressing early emerging challenging behavior and the life-long negative outcomes associated with delayed intervention services, federal and state f unding is most usually provided long after behavior problems reach a severe enough level to meet diagnostic criteria for behavioral and psychological disorders. Even the Part C program of the Individu als with Disabilities Education Act (IDEA, 2004), which is purporte d to serve infants and young children who are at risk for developmental delays requires that childrens emotiona l and behavioral

PAGE 37

25 problems reach a level of severity intense enough to qualify for a clinical diagnosis (Knitzer, 2007). Given that federal, state, and local f unding sources are insufficient to meet the needs of children and families with early-e merging behavior problems, policy-makers and child service agencies should prioriti ze prevention strategies and intervention techniques that yield the greatest retu rn on investment, (Knitzer, 2007, p.238). Based on the abundance of research supporting the primary role of parents and caregivers in young childrens emotional and behavioral development, it follows that the most logical target for prevention and early intervention efforts would be improving caregiving skills and enhancing the caregiver-child relationshi p (Knitzer, 2007). Thus, group-based parent training would be an economical and ecologically-based system for providing children and families with the knowledge, skill, and support they need to prevent and correct challenging behaviors (Smagner & Sullivan, 2005). The ability of one child care professional to simultaneously meet the needs of multiple families and children at once dramatically increases the efficiency of limited resources, professionals, and funding sources. Parent Training as an Intervention In order to maximize available resources and maintain a cost-effective method of service delivery, intervention techniques re aching the most children using the fewest resources have recently drawn attention. The most promising and effective of these costreducing interventions is behavioral pare nt training (Barlow & Parsons, 2002; Kazdin, 1995; Knitzer, 2007). Parent trai ning involves professionals teaching parents and other caregivers the basics in behavioral princi ples and behavior ma nagement techniques,

PAGE 38

26 which the parents can then apply with thei r children (Tiano & McNeil, 2005). Parent training programs have been shown to be e ffective when delivered to individual parents or to groups of parents (Barlow & Parsons, 2002; Feinfield & Baker, 2004). Many researchers have provided evidence supporti ng the use of behavior al parent training programs to reduce the development and persistence of problem behavior and improve the quality of parent-child interactions (Armstrong & Hornbeck, 2005; Gross et al., 2003; Maughan et al., 2005; McMahon & Fore hand, 2003; Nixon et al., 2003). The majority of empirically-supported pa rent training programs have four common components: a) center on an operant model; b) provide detailed information on the effective and appropriate use of time-out procedures; c) focus on antecedent control instead of punitive consequences; and d) program for generalization from the training setting to natural settings, in cluding home and community c ontexts (Feinfield & Baker, 2004). Research has also shown that programs that focus on changing parenting behavior have a stronger effect on child behavior outcomes than do programs that focus on changing parents attitudes (Sanders, 1996). In an analysis of parent training research conducted by Webster-Stratton and Taylor ( 2001), available evidence suggested that parent training produced the gr eatest effects with children between the ages of 3 and 10 years; created clinic-based changes that gene ralized to the home setting (but not often to the school setting); created clinically signi ficant and meaningful improvements in two thirds of targeted children; and resulted in changes in childrens behavior lasting up to four years. Recent research on parent trai ning with low-income families showed that well implemented, family-focused two-generational comprehensive programs for infants

PAGE 39

27 and toddlers can reverse the predictable ne gative developmental trajectories for many low-income infants and toddlers (Knitzer, 2007, p. 238). In 2005, Maughan and colleagues conducted a meta-analysis of the existing body of literature and research available regarding behavioral parent training as a treatment for externalizing behavior problems in children. The meta-analysis provi ded a description of studies, summarized the effects of the treatmen t studied, described variables that affected the treatment effects, and calculated an eff ect size to indicate the significance of each treatments effects. Studies which were included in the meta-analysis were: a) conducted between 1966 and 2001; b) target ed at least one externaliz ing behavior; c) targeted children who did not have autism or deve lopmental delays; d) included treatment procedures such as training parents or caregiv ers in the use of reinforcement and/or timeout and one additional parenti ng procedure; e) targeted ch ildren between the ages of 3 and 16 years old; f) used at least one outcome measure on chil ds behavior; g) used either between-subjects group design, within-subjects group design, or single-subject design; and h) incorporated graphs displaying raw data representing baseline data with at least 5 data points if single subject design was us ed. To find research studies, the authors searched using internet tool s and journal databases looking for all studies on behavioral parent training conducted within the specified time period. Th e search resulted in 294 studies, of which 79 (26%) met th e remaining inclusion criteria. Each study was coded for specific in formation related to participant demographics, research design and methods, training program com ponents, and outcome assessment. Effect sizes were calcu lated using statistics such as t, F or p values when means and standard deviations were not avai lable. For between-subjects designs, effect

PAGE 40

28 sizes were calculated based upon differences between pretest and posttest scores between the control and treatment group participants. For within-sub jects designs, effect sizes were calculated based upon difference between pretest and posttest scores for a single sample, divided by the pretest standard de viation (producing a standardized mean change). For single-subject designs, effect sizes were calculated using the ITSACORR computer program. After an effect size was computed for each individual study, a composite effect size with a 95% confidence inte rval was calculated for each of the three research design types (between-subjects, wi thin-subjects, and single-subjects designs). Potential bias for studies not included in th e meta-analysis, which may not have been available due to null results, no effect or lack of publishing, was corrected for by calculating a Fail Safe N which represented the number of studies that would have had to be included in the meta-analysis if all the possible studies were included. For the 79 studies included, 108 separate effect sizes were calculated. Most of the studies used a group training format ( n = 32), some used individual consultation ( n = 20), some used controlled learning techniques ( n = 10), and the remaining studies used mixed methodology ( n = 17). There were 2,083 participants in the between-subjects groups; 1,088 participants in the with in-subjects groups. There were 15 single-subject studies, which yielded 1,482 data points. The unweighted mean effect size for between-subjects studies was d = .58 (each study contributes equally to overall mean) and the weighted mean effect size was d = .30 (95% CI .21 to .39). There were no signifi cant outliers in the between-subjects group. Because the confidence interval did not incl ude zero, it was assumed that behavioral parent training conducted in a group format had a significant eff ect on the criterion

PAGE 41

29 variable. Differences in effect size were found when studies were analyzed separately based on the coded criteria variab les. Studies with parents of children between the ages of 3 and 5 years had an effect size of .40 while studies with parents of children between the ages of 6 and 8 years had an average effect size of .19 and children between the ages of 9 and 11 years had an average effect size of 1.36. Studies with training programs using 1 to 5 sessions had a mean effect size of .96; t hose using 6 to 10 sessions had a mean effect size of .50; those using 11 to 15 sessions had a mean effect size of .45; and those using more than 15 sessions had a mean effect size of .08; indicating that larger effects were found when fewer sessions were used, although no further explanation or interpretation of these differences were provided. In su mmary, variables significantly impacting the effect size of between-subjects studies included method of outcome assessment, child age, method of program delivery, number of sessions, method of assignment to conditions, and use of reliability assessments. The unweighted mean effect size for within-subjects studies was d = .74 and the weighted mean effect size was d = .68 (95% CI .59 to .77). Th e confidence interval for the within-subjects groups di d not include zero, indicati ng that the studies had a significant impact on outcome measures. There was one outlier present in this group, which was removed from further statistical analyses. Studies deliv ering training in an individual consultation format had an averag e effect size of .43, while studies using a group format had an average effect size of .70. This finding supported previous research in demonstrating larger effects when traini ng was delivered in a group format, which has been explained by the positive effects of peer support and modeling (Lundahl, et al.,

PAGE 42

30 2006). In summary, variables significantly impa cting the effect size of within-subjects studies included method of outcome asse ssment and method of program delivery. The unweighted mean effect size for single-subjects studies was d = .59 and the weighted mean effect size was d = .54 (95% CI .43 to .65). There were no significant outliers in the single-subj ects group. The confidence inte rval did not include zero, implying the treatment had a significant eff ect on the criterion variable. In summary, variables significantly impacting the effect si ze of single-subjects st udies included child age and method of program delivery. Results of the meta-analysis suggest that be havioral parent trai ning is an effective intervention for reducing externalizing problem behaviors in children; however, the effectiveness of this intervention is not as la rge as it was hypothesized to be prior to the meta-analysis. The overall mean weighted effect sizes for between-subjects, withinsubjects, and single-subject rese arch designs were all within the small to moderate range and were considered potentia lly significant (between-subj ects and single-subjects) and compelling (within-subjects). The authors cauti oned over-interpretation of the superior average effect size for with in-subjects design ove r between-subjects a nd single-subjects designs, citing previous research showing that this type of res earch design causes inflated effect sizes, regardless of actual treatment ef fects on outcomes. The authors also caution against over-interpretation of differences in effect size based on method of outcome assessment, citing a potential for parent biases in self-reported outcome measures versus direct observation. Suggestions for future rese arch included coding studies for treatment integrity and social validity measures. Limitations of the meta-analysis included

PAGE 43

31 variability in the methodological quality of studies reviewed and methodological limitations in calculating effect sizes for outcomes in single-subject designs. Over the past 20 years, research ers have conducted numerous studies investigating the effectiveness of various parent training programs, including the Incredible Years (Webster-Stratton, 2001), Pa rent Child Interaction Therapy (Eyberg, 1988), and Triple P-Positive Parenting Practices (Sanders, 1999). Despite differences in training components, duration, and research methodology, several meta-analyses have shown that much of the outcome research available reported si milar findings supporting the effectiveness of behavioral parent trai ning programs in impr oving behavior in young children (Conroy, Dunlap, Clarke, & Alter, 2005; Lundahl et al., 2006; Maughan et al., 2005). In one examination of the Incredible Y ears parent training se ries, Scott (2005) tested the effects of this program in a clinical practice setting. Participants were 59 parents of children ages 3 to 8 years re siding in London and Southern England. All children were referred for antisocial behavi or to their local comm unity mental health agency. The Parent Account of Child Symptoms was used as a semi-structured interview to gather parents' reports of childrens an tisocial behavior preand post-intervention. Parents also completed the Strengths and Difficulties Questionnaire (SDQ) as a selfreport of their childs conduct problems, hyperactivity, peer relationships, and prosocial behavior. Parents received the 12-week BASIC parent training program of the Incredible Years series, which was administered accordi ng to the manual. A control (waiting list) group was used for comparison purposes. Faci litators of all sessions were trained therapists from each local health agency. Immediately following the end of intervention,

PAGE 44

32 parent reports of child behavior as meas ured by the interview showed significant decreases in antisocial behavior; similar findings were shown for negative behavior reports on the SDQ, but with smaller effect sizes. Similar or even greater decreases in antisocial behavior and hypera ctivity were found at the oneyear follow-up as compared to controls. Peer relationships did no t show significant improvement following intervention. The researchers also found that risk factors such as ethnic minority, single parent families, and low SES did not redu ce treatment effectiveness. Demographic information did not include the percentages of participants who were mothers versus fathers. This would be valuable informa tion to report regardi ng whether or not the program was effective for both parents. It is necessary to evaluate research conducted with American children and families and diverse ethnic populations to determine whether this training series will be as effective with American children and families as it was for English participants. The Incredible Years parenting progra m was also evaluated among 634 ethnically diverse mothers of childre n enrolled in Head Start (Reid, Webster-Stratton, & Beauchaine, 2001). The CBCL was used to as sess externalizing be haviors including aggression and antisocial behaviors from parent reports. Parents of all ethnic groups receiving intervention were observed to be more positive, less inconsistent, and use less harsh discipline in their parenting (as measured via the Dyad ic Parent-Child Interactive Coding System Revised (DPICS-R) compared to parents in the control group, who were exposed to only the regular Head Start program. Additionally, children of parents receiving the intervention were observed vi a the DPICS-R to exhibit fewer behavior problems at one-year follow-up; however, CBCL reports were not significantly improved

PAGE 45

33 for the intervention group. Importantly, fe w differences were reported across ethnic groups and significant differences were only found among the use of positive parenting and use of critical statements to children as measured by the DPICS-R. These results indicate the applicabil ity of this program for ethnically divers e populations. The large sample size and randomized, controlled design add statistical streng th to the positive findings of this study. Schuchmann and colleagues (1998) conducte d a randomized, controlled trial of Parent Child Interaction Therapy (PCIT) with 64 clinic-referred families. Participants were assigned to a PCIT treatment condition ( n = 37) or a waitlist control group ( n = 27). Criteria for inclusion specified that all families referred had a child who was of preschool age (3 to 5 years) with a DSM-IV diagnosis of conduct disorder. Fam ilies in the treatment condition participated in PCIT sessions wh ile control group families were evaluated using the outcome measures, but had no other c ontact with the therapists or researchers. Outcome measures included direct observation of the quality of parent-child interactions using the Dyadic Parent-Child Interaction Coding System-II (DPCICS-II; Eyberg, Bessmer, Newcomb, Edwards, & Robinson, 1994) the Parental Locus of Control Scale (PLOC; Campis, Lyman, & Prentice-Dunn, 1986) and the Dyadic Adjustment Scale (DAS; Spanier, 1976). Assessments were re-administered every four months during treatment and at a follow-up assessment four months after the fina l PCIT session. Results showed that parents participating in PCIT sessions had more positive interactions with their children, and children demonstrated more frequent compliance with parent direction as compared to the parents in the waitlist control group. Parents in the PCIT group also reported lower levels of parental stress and gr eater internal locus of control in parenting

PAGE 46

34 practices compared to the waitlist control group. Finally, parents in the PCIT group reported greater improvements in their children s behavior following the therapy sessions than did the control group parents. Differentially positive outcomes for the PCIT group were maintained at the 4-month follow-up a ssessment. A limitation of this study was the relatively brief follow-up period, as researcher s determined maintenance of outcomes at four months post-treatment. Further research assessing trea tment maintenance at longer intervals following treatment termination woul d strengthen the efficacy reports for PCIT. A more recent study provided support for the long-term maintenance of treatment outcomes for PCIT (Eyberg et al., 2001). Eyberg and colleagues (2001) studied the maintenance of treatment outcomes for 13 fa milies with preschoolers diagnosed with conduct disorder at oneand two-years pos t-treatment. Treatment effectiveness was measured by the DPCICS-II (Eyberg et al., 1994), the Parenting Stress Index (PSI; Abidin, 1995), the PLOC (Campis et al ., 1986), and the DAS (Spainier, 1976). Significant differences (p < .05) were found between the PCIT families and the control group families on all measures. Eight of the 13 families maintained positive treatment effects at the oneand two-year follow-up assessments. Sanders, Markie-Dadds, Tully, and Bor (2000) conducted a controlled trial of Triple P-Positive Parenting Practices (TPP) in which three variants of the program ranging in levels of intensity were compar ed on 305 preschool-aged children (mean age = 3 years) at risk for developing conduct pr oblems. Families were randomly assigned to one of four conditions: (a) enha nced level, (b) standard leve l, (c) self-directed, and (d) wait-list control. The various conditions varied from practitioner-assis ted to self-directed using booklets and videos at the familys ho me. The standard program involved teaching

PAGE 47

35 parents 17 core child management strategies Ten of the strategies were designed to increase childrens competence and devel opment (e.g., talking with children; physical affection; praise; at tention; engaging activities; setting a good example). The remaining seven strategies were designed to help parents manage challenging behaviors by engaging in positive parenting practices (e.g., se tting rules; directed discussion; planned ignoring; clear, direct inst ructions; logical c onsequences; and time-out). Parents were taught a six-step planned activ ities routine to enhance the generalization and maintenance of parenting skills (e.g., plan ahead; decide on rule s; select engaging activities; decide on rewards and consequences; and hold follow-up di scussions with the child). Parents were taught to apply parenting skill s to a broad range of target behaviors in both home and community settings with the target child and their sibli ngs. Short-term and long-term follow-up data were collected on the effectiveness of the intervention. Various measures were utilized to collect frequency and intensity of behavior information for each child in order to ascertain the level of behavior ch ange pre and post-intervention. Specifically, the Parent Daily Report (PDR; Chamberlain & Reid, 1987), Parenting Scale (PS; Arnold, O'Leary, Wolff, & Acker, 1993), and the Pa rent Problem Checklist (PPC; Dadds & Powell, 1991) were utilized. The results show ed that all levels of the TPP produced significant results for the children and families taking part in the study, however, the enhanced (most intensive) version produced the greatest results. In summary, group-delivered, behavioral parent training has been shown to be an effective intervention method to prevent and address early emerging challenging behaviors in young children as well as improve the quality of family relationships. The positive effects of behavioral parent training have been demonstrated across a variety of

PAGE 48

36 racial/ethnic groups, families in various so cio-economic groups, and for children with a variety of diagnoses and conditi ons. Finally, behavior al parent training is a cost-effective intervention technique that maximizes time and resources. Parent and Child Characteristics Affec ting Outcomes of Parent Training Although parent training has been shown to be an effective treatment or intervention for a variety of behavior problem s, not all families who participate benefit equally (Reyno & McGrath, 2006). Over the past 20 years researchers ha ve investigated a variety of demographic and social variab les that might explain these differential outcomes (Knapp & Deluty, 1989; Oltmanns et al., 1977; Strain, Young, & Horowitz, 1981; Webster-Stratton & Hammond, 1990). As a result of this research, several variables have been identified as potential predictors of the likelihood families will experience success in parent training progr ams. These variables include caregivers' educational attainment level a nd social support network as we ll as target child's age and severity of symptoms or existing diagnos es (Bakermans-Kranenburg et al., 2003; Reyno & McGrath, 2006; Stolk et al., 2008). In or der to maximize the effectiveness of an intervention, it is important to identify factors that will allow practitioners to appropriately match clients with treatments in which they are likely to succeed (Smith, Landry, & Swank, 2005). Despite early evidence that specific pa rent, family, and child characteristics impact the effectiveness parent training progr ams for different participants, very little research has been conducted to specifically analyze predictors of success beyond basic demographic data of the parent and/or child (S mith et al., 2005). The majority of research studies report differential program outcomes based on either socio-economic status or

PAGE 49

37 racial/ethnic group. However, resu lts of these studies are diffi cult to interpret due to the broad operational definitions and measurement c onstructs used to de fine variables such as "socio-economic status" and "symptom se verity" and "social s upport." For instance, socio-economic status has been defined th rough a combination of measures, including family income, parent educational level, parent occupation, and/ or geographic location (Reyno & McGrath, 2006). "Symptom severity" has been conceptualized as the number of symptoms reported by the parent (Sanders et al., 2000), the number of comorbid psychological or medical diagnoses a child has when entering treatment (Scott, 2005), and/or scores on a standardized testi ng instrument (Webster-Stratton, 1998). More research is needed to specifically identify parent and child characteristics which may affect successful completion and outcomes of participating in a parent training program. The current study will anal yze the following demographic and social variables: 1) caregiver's e ducation level (i.e., earned a high school diploma or less, completing technical training or a 2-year college program, or earned a 4-year or graduate college degree); 2) caregiver's social suppor t (i.e., attending HOT DOCS classes with someone or attending alone); 3) target child's age (i.e., under three years of age or over three years of age); and 4) child's preexis ting psychoeducational or medical diagnoses (i.e., no preexisting diagnosis, children with autism spectrum disorders, developmental delays, speech/language impairments, or medical/genetic disorders). Parent/Caregiver Education Level Previous studies of parenting programs have reported higher than expected educational attainment for participants (F ienfield & Baker, 2004; Hartman, Stage, & Webster-Stratton, 2003). Researchers hypothesize that the higher mean educational levels

PAGE 50

38 of their participant samples may be explai ned by the additional financial and social supports available to these families with high er levels of educati onal attainment. These resources allow parents to participate in a nd complete training programs, while parents with lower educational attainment are often unable to attend and complete training sessions due to issues associated with socioeconomic status, such as lack of transportation, childcare, and time (Lundahl et al., 2006). It is difficult to analyze the impact of parent/caregiver edu cation level on parent training outcomes because this specific va riable is often combined with other demographic characteristics to form the mo re generalized variable of socio-economic status. For example, a meta-analysis conduc ted by Reyno and McGrath (2006) identified a combined socio-economic variable of low education/low income as a statistically significant predictor for premature discon tinuation of treatment. In contrast, Cunningham, Bremner, and Boyle (1995) found th at a combined variable consisting of parents educational level and family f unctioning, defined as so cio-economic status, accounted for 23% of the variance in attend ance. In this study, parents with higher educational backgrounds and better functioning families attended more sessions than families with lower educational levels and poorer family functioning. For those studies that did specifically an alyze participant data based on parents educational attainment as a di stinct variable, it was found th at parents with lower levels of education demonstrated greater gains fo llowing participation in a parenting program (Beauchaine et al., 2005; Berlin et al., 1998; Lundahl et al., 2006; Ol ds et al., 1999; Reid, Webster-Stratton, & Baydar, 2004; van Zeijl et al., 2006). Researchers hypothesized that parents with lower education levels experienced more adverse family circumstances,

PAGE 51

39 which increased their overall need for suppor t and therefore their motivation to attend and complete intervention program s (Beauchaine et al., 2005). Parent/Caregiver Social Support Most studies of parent training program outcomes have reported data on social support by defining families as single-parent versus father-involved families (Holden, Lavigne, & Cameron, 1990; Smith, Landr y, & Swank, 2000). Recently, however, research has emerged focusing on the importan ce of resiliency and protective factors for families of children with challenging beha vior (Luthar, 2006; Singer et al., 2007). Included in the research on family resiliency is a broader definiti on of social support, including extended family and community memb ers in addition to spouses or partners. Some research studies repor t that participants in pa rent training programs who have fewer social supports had less positive outcomes as a result of intervention than participants with stronger or wider social support netw orks (Kazdin & Wassell, 1999; Smith et al., 2005; Webster-Stratton & Hammond, 1990). Researchers have hypothesized that parents with available social supports are better able to utilize information and skills learned in parent training programs because their own emotional needs are met by their supporters (Smith et al., 2005). Child's Age In addition to caregiver characteristics that affect outcomes of participation in parent training programs, research ers have suggested that the target childs age or level of development also affect success (For ehand & Wierson, 1993; Ruma, Burke, & Thompson, 1996). However, treatment research on the effectiveness of parent training programs has generally defined age groups of target children for comparisons as young

PAGE 52

40 children (e.g., 0 to 5 years of age), older children (e.g., 6 to 12 years of age), and adolescents (e.g. 13 to 18 years of age) (Ruma et al., 1996). Little re search is available on the differential outcomes for parents of in fants, toddlers, and young children (e.g., 0 to 1 year, 1 year to 2 years, and 2 year s to 6 years of age) (Kazdin, 1993). A few studies have reported childs age as variable affecting treatment outcome, however, age groups in these studies were young children ages 2.5 to 6.5 years and older children 6.5 to 12.5 years, with younger children having more positive outcomes than older children and adolescen ts (Bath, Richey, & Haapal a, 1992; Dishion & Patterson, 1992). In another study, researchers measured treatment effectiveness as the number of sessions participants required in order to obtain specific skills (H olden et al., 1990). In this study, participants with younger children (3.3 years of age) required more sessions of a parent training to have positive outcomes than parents of older children (3.8 years). For this particular parent training program, parent s of older children benefitted more than parents of younger children (age range for sa mple was 18 months to 70 months of age; mean age of 36 months). Overall, the existi ng research on the influence of child's age on parent training program outcomes suggests that parents of older children, between the ages of 3 and 12 years of age, have more positive outcomes than pa rents of children who are older than 12 or younger than 3 years of ag e. However, conclusions about the benefits of parent training for caregivers of children between the ages of 2 and 6 years of age are difficult to draw due to the paucity of res earch focusing on differences within this age group.

PAGE 53

41 Child's Diagnosis Another factor that has b een identified as a moderato r of treatment outcomes in parent training programs is the childs dia gnosis (Holden et al., 1990; Lundahl et al., 2006; Ruma et al., 1996). However, published research findings on the impact of the childs diagnosis are difficult to compare and interpret because the majority of empirical studies have specified inclusi on criteria requiring that targ et children have preexisting mental, emotional or behavioral diagnoses to participate in study (e.g., ADHD, ODD/CD, attachment disorder). Few published, eviden ce-based interventions target parents of children with non-clinical levels of challeng ing behavior (Lundahl et al., 2006; Maughan et al., 2005; Schuma nn et al., 1998). Another difficulty in interp reting existing research on pa rent training programs for children with different conditi ons or diagnoses is that mo st parenting programs do not include parents of children with a variety of diagnoses (Lundahl et al., 2006; Maughan et al., 2005). Most parenting programs have been created to meet the specific needs of children and families with a particular diagnosis or condition. For example, programs based on the principles of applied behavior analysis (ABA) have been offered almost exclusively to children with autism spectrum disorders (Conroy et al., 2005); PCIT and Incredible Years are mainly offered to families of children with Oppositional Defiant Disorder and/or Conduct Disorder; while th e Defiant Child program (Barkley et al., 2000) was designed for parents of children with Attention Deficit/Hype ractivity Disorder. In summary, further research is needed in order to definitively identify parent and child demographic and social characteristics th at might serve as predictors of differential levels of success following participation in parent training programs. At present,

PAGE 54

42 researchers have not yet come to a consensus on whether variables such as parent level of education (Beauchaine et al., 2005; Berlin et al., 1998; Lunda hl et al., 2006), childs age (Forehand & Wierson, 1993; Ruma et al., 1996), or childs diagnos is (Holden et al., 1990; Lundahl et al., 2006; Ruma et al., 1996) accurately predict participants outcomes. They do, however, seem to agree that parents wi th higher levels of so cial supports benefit more from any type of parent training pr ogram than parents with limited or no social support network (Holden et al., 1990; Lundahl et al., 2006; Smith et al., 2005). It is clear that further research is needed to add to th e literature base regard ing the impact of these variables on parent training outcomes. Preliminary Investigation of the HOT DOCS Parent Tr aining Program The original Helping Our Toddlers ( H.O.T.) curriculum (Armstrong & Hornbeck, 2005) was developed through a U.S. Department of Education grant, with funds matched by the Childrens Board of Hillsborough County, Florida (Fox, Dunlap, & Powell 2002). The grant was provided to fund research to investigate the effectiveness of positive behavior support (PBS) applied to toddlers with challenging be havior and was referred to as the Early Intervention Positive Behavi or Support (EIPBS) project. The H.O.T. curriculum was developed by the EIPBS project director and a parent of a young child diagnosed with autism spectrum disorder to assist parents on the waitlist for the more intensive, individually developed EIPBS in tervention. The H.O.T. curriculum was based on the principles of PBS (i.e ., understanding the function of be havior, its antecedents and consequences, and teaching replacement be haviors). The parent training program consisted of six weeks of group instruction conducted in community settings, such as churches and the YMCA.

PAGE 55

43 The original H.O.T. curriculum was deliv ered to four cohorts of parents and caregivers of young children with challenging behaviors, av eraging 8-12 individuals per group occurring between 2005 and 2006. Data co llected during these initial trainings included demographic information, caregiver satisfaction with the program, knowledge of basic behavioral principles, and use of parenting skills taught in class. These data were used to refine the curriculum and genera te outcome reports required by the funding agency. Data gathered through focus groups and follow-up surveys conducted upon completion of the fourth cohort of participan ts, showed that 100% of caregivers who participated in the program reported improvements in their own parenting skills and their childs behavior (Armstrong, Hornbeck, B eam, Mack, & Popkave, 2006). Following the first four cohorts of H.O.T. parent training, several revisions to materials, procedures, and data collection were made to the curric ulum. Subsequently, th e original H.O.T. curriculum evolved into a manualized trai ning program called Helping Our Toddlers, Developing Our Childrens Skills (HOT DOCS; Armstrong, Lilly, et al., 2006). While evaluation data from a pilot study indicat ed promising outcomes of the HOT DOCS curriculum (Williams, 2007), a more rigorous an d standardized evaluation is needed to advance the evidence-base. Preliminary reports suggest the potential effectiveness of the Helping Our Toddlers, Developing Our Childrens Skills ( HOT DOCS) parent trai ning program as a means of reducing challenging behavior in young children and improving parent-child relationships (Armstrong, Hornbeck, et al ., 2006; Williams, 2007). Preliminary reports are based on the results of a pilot study of the programs effectiveness completed by Williams (2007). The pilot study was designed to evaluate the HOT DOCS parent

PAGE 56

44 training program using archival data co llected between August 2006 and April 2007. The evaluation used a one-group, pre/posttest design to study the impact of specific components of the parent training program on both participants knowledge and attitudes and their perceptions of targeted childrens behavior. One-hundred-for ty-six caregivers of children between the ages of 14 months and 10 years of age participated in the parent training program and were included in th e analyses. Measures included a pre/post knowledge test, rating scales of child probl em behavior, weekly progress monitoring forms for caregiver behavior at home, and a program evaluation survey. Rates and patterns of caregiver attendance, comparisons of caregiver demogr aphics with local demographics and with previous research on parent training programs, changes in caregiver knowledge, caregiver perceptions of childrens problem and adaptive behaviors, skill use at home, and overall per ceptions of caregiver participation in the program were analyzed. The pilot study had several significant limitations. These limitations included the use of archival data, the absence of a c ontrol or wait-list cont rol group to use as a normative comparison group for the participants who received training, the small sample size, and the low return rate of several outco me measures used. Therefore, results should be interpreted with caution. Rates and Patterns of Caregiver Attendance Rates and patterns of caregiver attend ance and attrition were analyzed and compared with findings from previous studi es of group-delivered behavioral parent training. Overall patterns of attendance and ra tes of attrition found in the pilot study were similar to those found in previ ous research (Eyberg et al., 200 1; Feinfield & Baker, 2004;

PAGE 57

45 Kazdin, 1997; Sanders et al., 2000). Of th e 189 caregivers attending the first of six sessions of HOT DOCS training, 146 comple ted the program (e.g., attended three or more sessions), resulting in an attrition rate of 23%. Eyberg and colleagues (2001) reported similar rates of attrition in an eval uation of the Parent-Child Interaction Therapy (PCIT) intervention. Specifically, of the original 20 participants, 13 completed the training, resulting in a 30% attrition rate. Comparison of Participant and Local Demographics Demographic information for the caregiver s serving as participants in the pilot study was compared with local demographic information provided by the United States Census Bureau for Hillsborough County, which is the local community where HOT DOCS was developed and delivered to families. The participant sample consisted of 15% fewer caregivers reporting their ethnicity as Caucasian (44% versus 59%), 11% fewer caregivers reporting their ethnicity as Black/A frican American (5.5% versus 16.3%), and 14% more caregivers reporting their ethnicity as Hispanic (35% versus 21.2%) than adults residing in Hillsborough County in 2005 (United States Census Bureau, http://quickfacts.census.gov/qfd/states/12/ 12057.html). These results suggest that the HOT DOCS program reached more Hispanic caregivers, who have been underserved by other parenting programs. However, a disproportionately low percentage of Black/African American caregivers particip ated in the HOT DOCS program. Preliminary analysis of the caregivers signing up to participate in th e program but not completing training (e.g., drop-outs) did not indicate differential rates of attrition for caregivers reporting their race/ethnicity as Black/Af rican American. The underrepresentation of Black/African American caregivers in the HOT DOCS program may be related to a

PAGE 58

46 decreased number of families from this race/ethnic category who self-refer and/or are referred by professionals to pa rticipate in parent training or perhaps identify with a different racial or ethnic cat egory. The high percentage of sample participants reporting their race/ethnicity as Hispan ic as compared to local nor ms is likely explained by the provision of HOT DOCS classes in Spanish. Participants reported level of education attained was compared to census data from 2000. The participant sample reported a si milar percentage of high school graduates (89% versus 81%), twice the number of colle ge graduates (53% versus 25%), and three times the number of graduate degrees (31% ve rsus 12%) as compared to census data. The use of type of insurance as an indicator fo r socioeconomic status (SES) in the pilot study prohibits precise comparisons with local population statistics, which report SES using ranges of annual household income. However, general comparisons of the proportion of the study sample reporting having Medicaid or no insurance, which were response categories used by the program developers to indicate low-SES, were compared with Hillsborough County estimates of adults fa lling below the poverty line (US Census Bureau, 2000). Approximately 31% of HOT DOCS participants reported having no insurance or Medicaid insurance compared to 12% of adults in Hillsborough County classified as low-SES. This comparison i ndicates that the HOT DOCS parent training program was provided to a higher percentage of low-SES families than would have occurred simply by chance. Since previous re search has shown that children of parents who are considered low-SES or low-income have a greater chance of developing more severe levels of challengi ng behavior (Gross et al., 1999; Keenan & Wakschlag, 2000; Qi

PAGE 59

47 & Kaiser, 2003), the large proportion of partic ipants falling within this category can be considered a positive finding. Comparison of Child and Caregiver De mographics with Previous Studies Demographic information for the caregiver s serving as partic ipants in the study also was compared with demographic information for participant samples from previous research of group parent training programs. Most of the existing research on parent training programs has focused on female caregiv ers, specifically mothers of children with problem behavior (Bagner & Eyberg, 2003; McNeill, Watson, Hennington, & Meeks, 2002; Phares, Fields, Kamboukos, & Lopez, 2005; Reid et al., 2001). The gender and relationship with target child of participants in this application differs notably from previous research on pare nt training interventions, specifically by encouraging participation of fathers, non-related caregive rs, and professionals. Participants in the sample were 68% female and 32% male, including 54% mothers, 29% fathers, 8% professionals (i.e., early interventionists, se rvice coordinators), a nd 7% grandparents. Preexisting diagnoses of target children of participants in the pilot study were compared with demographic information from previous research. The majority (66%) of target children in this study did not have a preexisting medical, psychological, or behavioral diagnosis as repor ted by caregiver participants at the time of HOT DOCS participation. In contrast, the majority of previous studies of pa rent training programs have specified inclusion criter ia requiring that target chil dren have preexisting mental, emotional or behavioral diagnoses to part icipate in study. Thus, the HOT DOCS parent training program may provide early interventi on services that may serve as preventative measures for children exhibiting non-cli nical levels of challenging behaviors.

PAGE 60

48 Caregiver Knowledge Results of the pilot study indicated a significant increase in participants scores on the HOT DOCS Knowledge Test from pretest to posttest, t(1,111) = 8.45, p < .001. Although the difference in mean score from pr etest to posttest differed by fewer than two correct answers, the effect size of the statistical difference was large ( d = 1.13), indicating significant and meaningful increases in the number of correct answers provided by participants. Knowing and unde rstanding these skill s and concepts may be considered ideal outcomes of the parent training program Therefore, an increase in the number of items correct may indicate successful delivery of skills and concepts. Caregiver Perceptions of Severity of Child Behavior Participants were expected to report high levels of pe rceived challenging behavior in target children. Expectati ons of high levels of problem behavior were based on the method of participant re cruitment. Participants may have self-referred to the program after seeing community advertisements or he aring about the program from friends, or were referred to the program by professional service providers. Although the pilot study did not base participant inclusion on pretest behavior rating scale scores, it was hypothesized that most of the caregivers seek ing to participate in the program would report that their children had more severe le vels of problem behavior than a normative sample. Results of the study suppor ted this hypothesis by indi cating that participants reported significantly more severe levels of child problem behavior at pre-test than was predicted for a normative sample of the population (Internalizing subscale, 2 (1, n = 101) = 252.24, p < .01; Externalizing subscale, 2 (1, n = 101) = 335.66, p < .01). Statistical

PAGE 61

49 analyses revealed that nearly twelve times as many caregivers in the participant sample perceived their childs problem behaviors to be within the clinically significant range on both the Internalizing and Exte rnalizing subscales of the CBCL (Achenbach, 2001) than was expected given a normal distribution. Thes e results indicate that the majority of caregivers who elected to participate in HOT DOCS perceived their children as having clinically significant levels of problem behavior prior to beginning the training program. As with caregiver perceptions of severity levels of ch ild problem behavior, it was expected that caregivers would also perceive their children as having lower than expected levels of adaptive behavior. Although caregiver s often cite challengi ng behavior as their primary concern, children likely have comorb id deficits in adaptive or prosocial behaviors (Conroy et al., 2005). Despite the lack of available research using parent perceptions of childrens adap tive behavior as inclusion cr iteria or outcome measures, initial studies have indicated that high le vels of problem behavior interfere with childrens ability to develop and maintain appr opriate levels of adap tive behavior (Carr et al., 2002; Conroy et al., 2005; Dunlap, 2006; Fox et al., 2002). Therefore, it was expected that caregiver participants would report lo wer levels of child adaptive behavior than expected in a normative sample of the population. Results of the pilot study supported the hypothesis by i ndicating that the sample participants reported significantly more severe deficits in ch ild adaptive behavior at pretest than were predicted for a normative sample of the population (Conceptual domain, 2 (2, n = 106) = 306.04, p < .01; Social domain, 2 (2, n = 106) = 354.11, p < .01; Practical domain, 2 (2, n = 103) = 525.04, p < .01). Statistical analyses revealed that nearly ten times as many caregivers in the participant sample perceived their childs

PAGE 62

50 adaptive behaviors to be within the clinic ally significant or deficit range on the Conceptual, Social, and Practical subscales of the ABAS-II (Harrison & Oakland, 2003) than was expected given a normal distribution. Th ese results indicate th at the majority of caregivers who elected to participate in HOT DOCS perceived their children as having clinically deficient levels of adaptive beha vior prior to beginning the training program. Changes in Child Problem & Adaptive Behavior Results of the pilot study indicate signifi cant reductions in the severity of child problem behavior as perceived by caregivers, F (1, 27) = 8.489, p < .01. It could not be determined from the data available whethe r child behavior actually improved or, as suspected in previous studies, changes in sc ores were due to reduc tions in parent stress and increases in parenting competency. Results of the pretest/posttest comparisons made in the pilot study should be inte rpreted with caution due to a low return rate of posttest scales (less than a 25% return rate). Results of the pilot study indicated non-significant levels of perceived change in the severity of deficits in child adaptive behavior on the part of caregivers. It could not be determined from the data available whethe r child adaptive behavior actually did not change from pretest to posttest or whet her other confounding vari ables, such as low return rate of posttest scales (<25%) could explain the non-significant findings. Differential return rates may be explained by actual changes in childrens adaptive behavior. For example, caregivers whose children increased their adaptive skills may have been perceived as no longer having a pr oblem, in which case caregivers may have had less motivation to complete lengthy rating sc ales (Barkley et al., 2000). Caregivers of

PAGE 63

51 children whose behavior did not improve or worsened following participation in the program may also have avoided completing and returning the postt est rating scales. Caregiver Skills at Home Caregivers reported high overall freque ncies of use of each skill as well as differential rates of ease for various skills. Caregivers reported Catch Them Being Good as easiest skill to use, followed by Use Preventions Use Calm Voice, Follow Through and Use Positive Words Results of statistical anal yses revealed no significant relationships between frequency of use and eas e of use. These findings may be explained by the restricted range of ratings of ease or difficulty (e.g., choices only 1 through 4) and the restricted range of days it was possible fo r caregivers to use skill (e.g., seven days maximum). Another possible confounding vari able is the differential number of caregivers completing weekly Tip Tracker sh eets as fewer participants completed and turned in Tip Tracker sheets for each session than the previous sessions. Results were predicted to show a peak in level of difficulty of skill use during the middle of the week, which may be explained by the extinction burst phenomenon of child behavior (Cooper, Heron, & Heward, 1987). For example, the first day or two parents used the skill at home children would initia lly be compliant with parent direction. However, once children perceived a change in caregiver behavior, childrens challenging behavior temporarily increased (e.g., test ing the limits) and then will decrease if caregivers remained consistent in their use of the new skill. Given the behavioral concept of extinction bursts, a hypothesized pattern would be for caregivers to initially report easier use of skills, followed by more difficulty using skills, and then a return to reports of more ease of use by the e nd of the seven-day period.

PAGE 64

52 Results supported the hypothesi zed pattern of reported ease or difficulty of use as predicted by the presence of extinction burst s in childrens behavior Four of the five skills followed the expected pattern of reported ease of use, followed by a peak in difficulty, and then a decrease in difficulty. Ho wever, caregivers rati ngs for each of the four skills that followed this pattern were varied. Caregiver ratings for Use Preventions most clearly followed the anticipated pattern. Follow Through, Use Calm Voice, and Catch Them Being Good followed the pattern to a lesser degree. Caregiver ratings for ease of use of Use Positive Words did not follow the expected pattern. Instead caregivers rated the skill as being initially more difficu lt and progressively get ting easier throughout the week. The pattern of perceived difficulty of Use Positive Words may be explained by the placement of this skill as the first skill assigned as homework in the HOT DOCS program. Caregivers may have reported use of this skill to be more difficult than later skills because they were adjusting to making changes in their overall parenting practices and not necessarily because the skill itself was more difficult to use. Caregivers Overall Perceptions of the HOT DOCS Program With few exceptions, the majority of caregivers (95%) indicated that they Agreed or Strongly Agreed that the HOT DOCS program met th eir expectations, was beneficial to their families, and positively impacted their behavior as caregivers. The few statements on the survey with which caregivers Disagreed or Strongly Disagreed related to the ability to implement specific sk ills at home and the programs impact on child behavior. These findings are not surprisi ng, given that many parent trai ning interventions struggle with accomplishing transfer of skills taught in the classroom to the home setting (Eyberg, 1988; Sanders, 1999). In light of the overwhelmingly positive response to these items,

PAGE 65

53 those few participants who were not satisfied with the program were provided individual consultation and possible referrals for further assessment and treatment strategies. These results were interpreted as exceptions to a pr ogram perceived as effective, rather than proof that the program is not effective. The majority of caregivers (70%) reported th at they were using the skills learned in the program at home or in the community and had shared the information they learned with others (95%), including spouses, fa mily, and friends. When asked to provide suggestions for future HOT DOCS classes, 40% of caregivers answered Nothing, the program is fine as is, and 25% answered More time, (e.g., more classes, longer sessions, booster sessions). These results support caregiver ratings of satisfaction with the program, by indicating that there were no significant ch anges or improvements that should be made to the program. When asked what they valued most from the training, the majority (60%) of caregivers indicated the specific skills taught in the sessions. Conclusions Results of the pilot study suggest suc cessful outcomes for car egivers and children participating in the HOT DOCS program, in cluding increases in caregiver knowledge, frequent use of skills at home, high leve ls of satisfaction with the program, and reductions in the perceived severity of child behavior problems. Re sults also indicated several modifications that could be made to the program to improve participant outcomes and increase the validity and reliability of program evaluations, including changes to measurement instruments (e.g., knowledge te st, adaptive skill measure, evaluation survey) and data collection procedures (e.g., wa itlist control group, low rate of return of

PAGE 66

54 posttest rating scales). Overall, the HOT DOCS parent training program appeared to be a promising early interventi on program that could be delivered in group format. Although these preliminary findings are en couraging, federal mandates, such as the Individuals with Disabilities Educa tion Act (IDEA, 2004, U.S. Department of Education) and the No Child Left Behind Act (NCLB, 2001), emphasize the importance of selecting only those interv entions that are empirically-s upported through rigorous and competent research. Therefore, a more exte nsive evaluation of the HOT DOCS parent training curriculum must be completed. Summary The past three decades of research have indicated an alarming and ever-growing need for effective and economically feasible interventions that address challenging behavior in young children. Studies have consistently demons trated prevalence rates of challenging behavior upwards of 25% in the 3to 5-year age gr oup and more recent research has begun to demonstrate prevalence rates near 10% in the infant and toddler age group. Longitudinal research also has clearly demonstrated the profuse, long-term negative outcomes associated with early em erging behavior problems. Following the research on increasing prevalence rates and long-term negative outcomes, researchers and practitioners have developed a multitude of strategies for preventing and treating behavior problems in children and families. Of these interventions, behavioral parent training has been supported by numerous, re peated, well-designed studies and is generally considered the best-practices approach to preventing and remediating challenging behavior in young children. The past three decades of research has clearly indicated a need for cost-efficient, empiri cally-supported, evidence-based parent training

PAGE 67

55 interventions, which enable parents and car egivers to prevent and correct challenging behavior in young children as early as possible.

PAGE 68

56 Chapter 3 Methods Introduction The purpose of the current study was to evaluate participants' knowledge and attitudes and investigate of participants' perceptions of target children's behavior following participation in the HOT DOCS pare nt training program. In addition, this study investigated the extent to which select dem ographic variables (i.e., caregivers' level of education, caregivers' social support network, target child' s age, and target child's diagnosis) moderated particip ants' outcomes and perceptions. The study was a secondary analysis of existing data drawn from the HOT DOCS database, which was developed for storing data on program partic ipants for purposes of program evaluation. For purposes of this study, a one-group, pretest/posttest design was utilized. The design was dictated by the type of data collected by program develope rs. This chapter presents information about participants, trainers, trai ning settings, the HOT DOCS parent training curriculum, measurement tools, and methods of data collection and analysis. Participants The participants in this study were caregi vers of children between the ages of 18 months and 5.11 years of age identified as di splaying challenging behaviors. The sample was a convenience sample, as participati on in the training pr ogram was voluntary. Participant caregivers include d biological, adoptive, and fo ster parents, grandparents,

PAGE 69

57 other relatives, and child service provider s. Participants we re recruited through community advertisements or were referre d by their pediatrician, psychologist, or therapist to participate in a universitya nd community-based parent training program for families and service providers of children displaying challenging or disruptive behavior. As referrals were made or caregivers res ponded to public advertisements, caregivers names were added to a wait-list for future training sessions. Altogether, 662 caregivers were invited and scheduled to participate in the parent training program. Of these, only 465 (70%) attended the first training session. The remaining 197 caregivers did not return remi nder telephone calls and did not participate in the program. Thus, the initial sample fo r the study consisted of 465 caregivers who attended at least the firs t session. As is shown in Ta ble 1, 102 (21.9%) caregivers attended fewer than three of the training sessions and were considered program dropouts, 29 (6.2%) caregivers atte nded three or more sessions but elected not to sign the Internal Review Board (IRB) release form and therefore were not included in data collection for the purposes of th is study, although they did co mplete the course. The final participant sample for this study consisted of 334 caregivers who a ttended at least three sessions conducted between Ma y 2007 and March 2009 and cons ented to participate in the evaluation of the program by signing the IRB release form. Brief analyses were conducted to inves tigate potential differe nces between those caregivers who attended three or more sessions and were considered program completers ( n = 334) and those caregivers who attende d fewer than three sessions and were considered drop-outs ( n = 61). Independent-sample t-tests were calculated to compare the groups of completers and drop-outs for severa l demographic variables and two measures

PAGE 70

58 collected at pretest. No si gnificant differences (p > .05) were found between program completers and drop-outs for the following de mographic variables: caregivers age, race/ethnicity, level of educa tion, and relationship to target child, and target childs age. No significant differences were observed be tween the completers and the drop-outs for the outcome measures administered duri ng the first session in cluding the Knowledge Pretest ( n = 51) and the CBCL pretest (n = 23). Table 1 Attendance Record of Initial Caregiver Participant Sample Attendance record # Caregivers Percent Attended fewer than 3 training sessions 102 21.9 Attended 3 or more sessions but did not sign IRB 29 6.2 Attended 3 or more sessions and signed IRB 334 71.8 Note : n = 465. Description of Caregivers A breakdown of the final participant samp le by gender, race/ethnicity, education level, and social support netw ork is shown in Table 2. Par ticipants were 25.7% male (n = 86) and 74.3% female ( n = 248). They ranged in age from 14 to 69 years ( M = 35.9, SD = 8.63). The majority of the sample (88%) consisted of caregiv ers reporting their race/ethnicity as White (46.7%) or Hispanic (41.3%). African American/Black and Asian caregivers contributed only a bout 8% of the sample. Care givers reported level of education varied from less than a high school diploma to a graduate level degree; 52.6% of the sample reported having an undergra duate or graduate degree. The largest percentage of participants (28.7%) had a degree from a 4-year college ( n = 96) and 23.9%

PAGE 71

59 had a graduate degree. Approximately one-half of the participants ( n = 164) attended the program with another caregiver. The remaining participants ( n = 170) attended the program alone. Table 2 Breakdown of Participant Sample by Gender, Race/Ethnicity, Education Level, and Social Support Variable Number Percent (%) Gender Female 248 74.3 Male 86 25.7 Race/Ethnicity White 156 46.7 Hispanic 138 41.3 African American/Black 20 6.0 Asian 7 2.1 Other 6 1.8 Not Reported 7 2.1 Caregiver Education Level Less than HS 14 4.2 HS Diploma 67 20.1 Technical Training 25 7.5 2-Year College Degree 40 12.0 4-Year College Degree 96 28.7 Graduate Degree 80 23.9 Not Reported 12 3.6 Caregiver Social Support Attended alone 170 50.9 Attended with caregiver 164 49.1 Note n = 334.

PAGE 72

60 The variable caregivers social support, was measured in terms of whether or not a caregiver was accompanied by another individual representing that same target child. Participants attending th e program with another caregi ver were assumed to have a constant source of teamwork, partnership, a nd encouragement to attend weekly sessions, complete homework assignments, and use th e skills and techniques taught during class sessions. Although caregivers who attended th e program alone may have had social support from other caregivers outside of the training session, therefore affording these participants with the same le vel of support and encouragemen t, for the purposes of this study, only those caregivers who attended at least three sessi ons with another caregiver were counted as attending together for the variable caregivers level of social support. Caregivers who attended the program together were aske d to complete homework assignments together, focusing on the same pa renting skills and techniques each week. Caregivers attending the program together also were prompted to complete child behavior rating scales together. Within the context of this study, type of insurance was used as a general indicator of socio-economic status (S ES), with private insuran ce representing higher SES and Medicaid or no insurance representing lo wer SES. HOT DOCS program developers originally used type of insurance as a genera l indicator of participants SES because other previous studies in educational or school-based research have used childrens eligibility for free or reduced-price lunch as a gene ral indicator of SES. However, making inferences about participants SES based on type of health insurance prohibits precise comparisons with previous research, which ma inly defines SES through measures such as

PAGE 73

61 annual household income. Despite the limitati ons of such a definition, the HOT DOCS program developers made the decision to use type of insurance as an indicator because they believed participant careg ivers would more readily provide information about health insurance than they would about annual household income. As is shown in Table 3, 184 (55.1%) pa rticipants reporte d having private insurance, 92 (27.5%) participants reporte d having Medicaid insu rance, and 21 (6.3%) participants reported having no insurance. Thirty-seven (11%) participants did not respond to this item. Table 3 Breakdown of Participant Sample by SES Indicator Type of Insurance Number Percent (%) Private 184 55.1 Medicaid 92 27.5 No Insurance 21 6.3 Not Reported 37 11.1 Note n = 334 The majority of participants (85.6%) reported being the target childs biological, adoptive, or foster parent ( n = 286). Of the remaining par ticipants, 34 (10.2%) described their role as a child servi ce provider, 12 (3.6%) participan ts reported being the target childs grandparent, and 2 (0.6%) particip ants reported being another relative. As shown in Table 4, of the female pa rticipants, 205 (82.7%) reported being the childs biological/adoptive/foster mother, 30 (12.1%) reported being child service providers, 11 (4.4%) reported being the child s grandmother, and 2 (0.8%) reported being

PAGE 74

62 another relative (e.g., aunt, great-grandmot her). Of the male participants, 81 (94.2%) reported being the childs biological/adoptive/ foster father, 4 (4.7%) reported being a child service provider, and 1 (1.1%) re ported being the childs grandfather. Table 4 Relation of Caregiver to Target Child by Participant Gender Relation Number Percent (%) Females (n = 248) Mother & Adoptive/Foster Mother 205 82.7 Child Service Provider 30 12.1 Grandmother 11 4.4 Other Female Relative 2 0.8 Males (n = 86) Father & Adoptive/Foster Father 81 94.2 Child Service Provider 4 4.7 Grandfather 1 1.1 Note n = 334 Description of Target Children Target children ( n = 309) ranged in age from 18 months to 5.11 years ( M = 38.08 months, SD = 13.38). Approximately one-half (50.8 %) of the target children had preexisting medical and/or ps ychological diagnoses, as reported by caregivers during collection of demographic information duri ng the first session. Caregiver-reported diagnoses were not verified through review of records or consultation with physicians. Many of the remaining children had recently been evaluated by pediatricians or

PAGE 75

63 psychologists due to parent or teacher concer ns with development and behavior, but did not meet criteria for a diagnosis according to the Diagnostic and Statistical Manual-4th Edition, Text Revision (DSM-IV-TR, American Psychiatric Association, 2000). As is shown in Table 5, 40.8% (n = 126) of the target children did not have a preexisting diagnosis. Of the children in the sample with preexisting diagnoses, 56 (18.1%) were children with speech or langua ge impairments, 38 (12.3%) were children with developmental delays, 36 (11.7%) were children with a diagnosis on the autism spectrum including Pervasive Developmental Disorder (ASD/PDD), and 27 (8.7%) were children with medical or geneti c disorder (i.e., Attention-De ficit/Hyperactivity Disorder, Down syndrome, cerebral palsy, failure to thrive, premature birth). Twenty-six participants (8.4%) did not report whether or not their child had a preexisting diagnosis. Table 5 Number and Percent of Target Chil dren by Preexisting Diagnosis Childs Preexisting Diagnosis Number Percent (%) None 126 40.8 Speech-Language Impairment 56 18.1 Developmental Delay 38 12.3 ASD/PDD 36 11.7 Medical/Genetic Syndrome 27 8.7 Not Reported 26 8.4 Note n = 309

PAGE 76

64 Settings The HOT DOCS program was provided through the Childrens Medical Services (CMS) clinic, run by the Department of Pediatri cs at a large University in West Central Florida. The training groups were held in conference rooms located within a campus clinic and several community settings, incl uding Head Start classrooms, churches, and community centers. HOT DOCS Parent Training Program The HOT DOCS parent training program is designed to be delivered in seven sessions (refer to Appendix A for a summary of each session). Each of the sessions lasts approximately two hours. The first session in cludes thirty minutes of socialization, including a light dinner provided by trainers and brief introductions; twenty minutes during which caregivers completed the demographics form and knowledge pretest (see description of measures below for details); and one hour of behavior al parent training. The second, third, fourth, and fifth sessions include 30 minutes of socialization, peer support and group problem solving, and review followed by an hour of new instruction. The sixth session follows a similar format and then concludes with twenty minutes during which caregivers complete the knowledge postte st and a program evaluation survey (see description of measures below for details). The final session is a booster or refresher session conducted two months after the sixth se ssion and follows a similar format as the previous sessions. The activities for each session include lecture, practice exercises, role playing, and video vignettes to address the learni ng objectives for each session. Each training session also includes a Parenting Tip and a Special Play Activity Parenting Tips are

PAGE 77

65 specific skills caregivers are asked to practice throughout the following week, which they recorded using the HOT DOCS Tip Tracker shee ts. Tip Trackers recorded the number of days caregivers used the skill, to rate how difficult or easy the skill was to use each day, and prompted caregivers to provide specific ex amples of how they used the skill with their children each week. The Special Play Ac tivities are 5-minute routines caregivers are asked to engage in daily with their child. Inexpensive items such as bubbles are provided each week to caregivers along with a worksheet with guidelines describing how to use the activities to teach their child motor, comm unication, and social-emotional skills. A more detailed description of e ach training session follows. Session One. The first session provides participants with an overview of the HOT DOCS program and an introduction to early childhood development. Caregivers are instructed in brain development, typical ages for achievement of developmental milestones and warning signs for delays in development, school read iness skills, and an overview of the problem-solving process. The Parenting Tip for the first session is Use Positive Words, which is explained to caregiv ers as telling children what to do instead of what not to do. For example, caregivers s hould say, Feet on th e floor, instead of Stop jumping on the couch. A class activ ity is conducted in which caregivers brainstorm positive ways to rephrase twenty of the most common behaviors caregivers usually respond to with No! or Stop! The Special Play Activity for session one is Bubbles. Each participant is given a cont ainer of bubbles to use for this activity. Session Two. The second session focuses on teaching caregivers about the importance of healthy routines and ritual s in promoting positive development and adaptive behavior in young children. Sleep routines, or the activit ies surrounding

PAGE 78

66 bedtime, are highlighted, since this is th e most common problematic routine for most caregivers and children. The Parenting Tip for this sessi on is Catch Them Being Good, which prompts caregivers to focus on the pos itive behaviors or skills their children exhibit each day and to respond with specific, labeled praise for these behaviors. The Special Play Activity for this session is reading, for which caregivers are provided instruction, examples, and a detailed worksheet of activities. Each participant is given a developmentally appropriate storybook. Session Three The third session introduces caregivers to the basics of behavior development in young children, including the co ncepts of social learning, modeling, antecedents and consequences, reinforcement and the function of behavior. In this session, caregivers are introduced to the probl em-solving chart, which includes triggers, behaviors, consequences, preventions, new skills, and new responses. In this session, caregivers learn to complete the first three se ctions. The Parenting Tip for this session is Use Calm Voice, which reminds caregivers to use a calm, quiet voice in response to their childs behavior, especially in respons e to challenging or noncompliant behavior. The Special Play Activity is coloring, for which each participant is given a coloring book and a box of crayons. Session Four The fourth session provides caregive rs with training in the use of various preventative strategi es, including using timers, providing prompts, clarifying expectations, visual schedules or prompts, and personalized stories. The Parenting Tip for this session is Use Preventions, which pr omotes caregivers use of the preventative techniques taught in the session. The Special Play Activity is fun dough, for which each participant is provided with one color or tub of dough and a durable placemat.

PAGE 79

67 Session Five The fifth session provides caregiver s with training in how to teach their children new skills and replacement skills for challenging behaviors. In this session caregivers begin to complete the second half of the problem solving chart, including the preventions and new skills sections. Caregi vers are also provided instruction in the appropriate uses and steps for Time-Out from Positive Reinforcement and what to do when children misbehave or are non-compliant The Parenting Tip for this session is Follow Through, which provides caregivers wi th a brief script to use whenever their children did not comply with a direction or task. The Speci al Play Activity is playing with a ball, which each participant is provided before leaving the session. Session Six. The sixth and final session focuse s on helping caregivers understand and manage their own stress as well as provi ding a summary and review of the content of the previous sessions. Caregivers complete the final categories of the problem solving behavior chart by listing the vari ety of new responses caregiver s can have to their childs appropriate behaviors. These new responses include specific praise prompting, validation and redirection, and follow through. The Paren ting Tip for this session is Take 5 for Yourself, which reminds caregivers to focu s on their own health and stress levels each day. For this week, caregivers are prompted to use one of the five previously learned Special Play Activities each day. Booster Session A Booster session is held two months after the sixth session and focuses on reviewing the content of the first six sessions and checking in with caregivers on their progress with using skills and tec hniques learned in HOT DOCS with their children. Caregivers complete a nd turn in the posttest behavi or rating scale before leaving

PAGE 80

68 the session. No new Parenting Tips or Special Play Activities are introduced, although each tip and play activity are reviewed. All of the HOT DOCS materials, pr esentations, and handouts have been linguistically translated and culturally adapte d from English to Spanish to allow for the program content to be delivered in both languages. The Spanish adaptation was created and field tested by a team of bilingual U SF university students and staff including a fellow in internal medicine and pediatrics w ho was originally from Ecuador; a master of public health graduate student with a medi cal degree who was originally from Nicaragua; a doctoral intern in school psychology, who spoke Spanish as a second language; and a parent and HOT DOCS gradua te, who was originally from Columbia (Curtiss Salinas, Williams, Armstrong, & Ortiz., 2009). HOT DOCS Trainers The following section describes the pro cedures used to train the HOT DOCS Trainers. HOT DOCS Trainers are those i ndividuals who are employed by or volunteer their time to the program to de liver the program to caregiver s. Trainers are required to have served as facilitato rs of at least one class (7 sessions) of HOT DOCS. Description of Train-the-Trainers Process In order to become a trainer of the HOT DOCS curriculum, trainees first observe or shadow an experienced trainer (someone who has taught the classes at least twice without direct superv ision) through the durat ion of one set of se ven sessions of HOT DOCS. During the observation period, train ees are provided with a copy of the HOT DOCS providers manual, which they review before sessions begin and follow along with during each session. Throughout th e observation period, the expe rienced trainer involves

PAGE 81

69 the trainee in the process of preparing for classes and answers any questions the trainee may have about the curriculum, group manageme nt strategies, and methods of interacting with caregivers. After observing a complete set of seven se ssions, the trainee then serves as an assistant to a more experienced trainer. Du ring this phase of pr eparation, the trainee delivers some of the content during weekly se ssions, interacts with caregivers, and helps with material preparation and data collect ion. Throughout this phase, the trainee meets with the experienced trainer before and/or after each weekly session to discuss any questions or concerns. During this supervisi on meeting, the experien ced trainer provides the trainee with specific f eedback on his or her perfor mance. Once the trainee has assisted a more experienced trainer throughout a complete set of seven sessions, the trainee takes on the role of co-leader with another trainer. During these sessions, the trainee should equally share the duties and re sponsibilities of prep aring and leading the sessions with another trainer. Finally, the trainer would co-lead a class with another experienced trainer while the class is obs erved and/or assisted by a new trainee. Measures HOT DOCS Caregiver Demographics Form The Caregiver Demographics Form was developed by the HOT DOCS authors in order to collect demographic information about the caregivers and the target child ren. This form includes 10 questions asking caregivers to indicate their address, ge nder, age, relationshi p to target child, race/ethnicity, and level of education. In a ddition, they are asked to indicate the target childs age, preexisting medical and/or psychological diagnoses, type and name of health

PAGE 82

70 insurance, and the age(s) of other childre n in the home. The demographics form is available in both English and Sp anish (see Appendices B and C). HOT DOCS Knowledge Test The Knowledge Test developed by the HOT DOCS authors, is designed to assess caregivers knowledge of child development, behavioral principles, and parenting strate gies. The test consists of 20 statements, which caregivers are instructed to mark as being "True or False." Each item is scored 1 for the correct answer and 0 for the incorrect answer. The ma ximum possible score on the test is 20. The test takes approximately ten minutes to complete. Although the test includes items reflecting various learning objectives in HOT DOCS, there were not enough items per objectiv e to investigate subscale scores. Thus, for the purposes of this study, only total scores were used for data analysis purposes. No information on reliability or validity was available for this instrument. The knowledge test is available in both English and Spanish (see Appendices D and E). Achenbach Child Behavior Checklist The Child Behavior Checklist (CBCL; Achenbach, 2001) was developed to assess childhood behavior problems. There are multiple versions of the CBCL that are used depending on the childs age and the source of information. The CBCL 1-5 was developed for use with children between the ages of 18 and 71 months of age and can be co mpleted by parents/caregivers and/or teachers/caregivers. The CBCL problem beha vior scores are grouped into two broadband factors (internalizing and externalizing problems), a total broad-band score derived by averaging weighted scores from the br oad-band factors, and eight narrow-band subscales. The narrow-band subscales include aggressive behavior, anxious/depressed, attention problems, delinquent behavior, so cial problems, somatic complaints, thought

PAGE 83

71 problems, and withdrawn behavior. All versions of the CBCL are av ailable in English and Spanish. The CBCL 1-5 is a 99 items-questionnaire th at asks parents/caregivers to rate their childs behavior in the previous 2 months by rating each item on a three-point scale: 0 = not true of the child, 1 = somewhat or sometimes true and 2 = very true or often true. For some items, in addition to rating the child' s behavior on the 3-point scale, caregivers are prompted to provide brie f descriptions of problems, disabilities, most significant parent concerns, and to list their childs strengths. Completing the CBCL takes approximately 20 minutes. Responses are scored using a computerized scoring software program. Scores are expressed as T-scores with a mean of 50 and a standard deviation of 10. A T-score of 64 or below is in the normal range; 65-69 is in the borderline range; and 70 or above is in the clinical range. Scores in the borderline or clinical range indicate that a childs behavior problems are more signi ficant than other children the same age and gender. The CBCL 1-5 was normed on a national sample of 700 children The manual reports median internal consistency coefficients for the Internalizing and Externalizing scales that range from .76 to .92. Studies of the CBCL subscales indicated high retest reliability (Withdrawn: r = .82; Somatic Complaints: r = .95; Anxious/Depressed: r = .86; Social Problems: r = .87; Internalizing Problems: r = .89) and adequate interrater reliability (Withdrawn: r = .66; Somatic Complaints: r = .52; Anxious/Depressed : r = .77; Social Problems: r = .77; Internalizing Problems: r = .66; Achenbach, 1991). HOT DOCS Program Evaluation Survey for Caregivers The Program Evaluation Survey for Caregivers was developed by the HOT DOCS authors to assess caregiver

PAGE 84

72 participants perceptions of the effectivene ss of the parent training program. The survey consists of eight statements about the benef its of HOT DOCS to caregivers, the skill of HOT DOCS trainers, and HOT DOCS impact on child and family behaviors and relationships. Caregivers were asked to indicate the extent of their agreement with the eight statements on a 4-point Like rt-type scale anchored as 4 = Strongly agree 3 = Agree 2 = Disagree and 1 = Strongly disagree The survey also consists of five questions with multiple response options provided. Caregivers are asked to select the option that best represents their perceptions on the usefulness of the program, what they valued most about the program, as well as any suggestions for future trainings or improvements to the current program. The original survey developed in 2006 c onsisted of five openended questions or prompts, which were completed in free-response form by caregivers. After the pilot study was completed, the HOT DOCS authors modified the survey by replacing the five openended prompts with five statements with response choices provided, which were derived from the thematic analysis of the pilot st udy survey responses. The new survey provides options for caregivers to check the responses which best pertain to them and will allow for more systematic quantitativ e analysis of the data. The su rvey is available in both English and Spanish (see Appendices F and G). Data Collection Procedures Data collected for each caregiver include d a demographics information sheet; a knowledge preand posttest of the basic principles of behaviorism and child development; behavior rati ng scales (CBCL); and a pr ogram evaluation survey on caregivers perceptions of the usefulness a nd effectiveness of the program. Caregivers

PAGE 85

73 completed the Demographics Form and the Knowledge Pretest duri ng the first session. Caregivers were given CBCL be havior rating scales during the first session and were asked to complete and return the forms th e next week. Child care providers were not given behavior rating scales unless they were attending th e HOT DOCS classes with a specific family for whom they were providi ng direct intervention services. Caregivers completed the Knowledge Posttest during the final session of training. Caregivers were given behavior rating scales during the Booster Session and were allowed time to complete the scales before leaving. If caregiver s did not complete the rating scales before leaving they were given a postage-paid enve lope to return the scales upon completion. Caregivers who did not attend the Booster Se ssion were mailed a packet containing the rating scales the day after the Booster Session was held. A postage-paid envelope addressed to the HOT DOCS au thors at the CMS clinic was included for return of the completed instruments. Included in the pack et was a letter detailing the request for information, directions for completing the ra ting scale, and a desc ription of how the information would be used as part of the research project. Re minder postcards were mailed to participants who had not returned th e behavior rating scales two weeks after the original mailing. Each program participant was assigned an identification code before the first training session. Identification codes consisted of a five-dig it sequence of numbers. The first two digits were the same for each participant enrolled in a cohort of classes (e.g., all participants in the August 2008 classes were given identification codes beginning with the number 27). The third and fourth digit we re the same for all members of a group or family attending classes together and focu sing on the same child (e.g., a mother, father

PAGE 86

74 and grandmother from the same family attending in August 2008 would have identification codes beginning with 2705). Finally, the fifth digit indicated the participants' relationship to the target child (e.g., all mothers' codes ended in 0, all fathers' codes ended in 1, grandparents and other re latives codes ended in 2, and providers or professionals codes ended in 3). Data for this study were accessed through the HOT DOCS database housed in the Childrens Medical Services (CMS) clinic. Following each weekly training session, the HOT DOCS project director en tered participant data using identification codes only into a database maintained by a secure passwor d. Two school psychology graduate students serving as HOT DOCS staff conducted integr ity checks comparing the raw data to the data entered into the database to ensure th e accuracy of the data. Integrity checks were conducted on every 10th participant in the database. All raw data were stored in a locked file cabinet in the CMS clinic. The primary researcher in this study was involved in the HOT DOCS program prior to proposing this study in the capacity of a trainer and data entry staff. All data used in this study were collected as pl anned by the HOT DOCS program developers, without input from the primary researcher in regards to the planning of this research project. Specific variables and methods of da ta collection (i.e., type of insurance as an indicator of SES, target ch ilds diagnosis collected thr ough caregiver-report, without further verification) were defined prior to the proposal of this project by the HOT DOCS program developers. In this sense, alt hough the primary research was involved in providing HOT DOCS trainings a nd assisting with data entr y prior to proposing this study, the data used for the purposes of this re search were collected without the control or

PAGE 87

75 contributions of the primary researcher, defi ning the study as a seconda ry analysis of an existing or archival data set. Data Analysis Data from participants who did not comp lete three or more HOT DOCS training sessions or who did not sign an IRB consent fo rm were not included in data analyses for this study. The research questions addressed in the study and the analyses used to answer the questions are given below. Caregiver Knowledge Research Question #1a. What is the impact on caregiver knowledge of child development, behavioral principles, and parentin g strategies as a result of participation in the HOT DOCS parent training program? A dependent means t-test was conducted to determine if there was a significant difference in participant caregiver knowle dge of child development, behavioral principles, and parenting strategi es on completion of the HOT DOCS training program, between participants' pretest and posttest scores on the HOT DOCS Knowledge Test. Scores were reported as total number of items correct on the HOT DOCS Knowledge Test. Research Question #1b. b. Is there a difference in participant caregivers knowledge of child development, behavioral pr inciples, and parenting strategies based on caregivers' level of education, caregivers' so cial support network, the target child's age, and the target child's diagnosis? Repeated measures analysis of variance (ANOVA) was used to determine if there was a significant difference in caregiver know ledge due to partic ipation in the HOT

PAGE 88

76 DOCS training program as measured by preand post-test scores on the HOT DOCS knowledge test for specific groups of particip ants. A separate analysis was conducted for each of the following independent variables: e ducation level of caregivers (i.e., less than a high school diploma, high school diploma, t echnical training, 2-y ear college degree, 4year college, or a graduate degree); social support network of careg ivers (i.e., caregiver attending training alone vs. accompanied by another caregiver(s)); age of target child (i.e., target child under three years of age vs target child over three years of age); and diagnosis of target child (i.e ., no preexisting diagnosis, target child with autism spectrum disorders, developmental delays, speech/la nguage impairments, or medical/genetic disorders). The dependent variables in each anal ysis were the pretest and posttest scores on the HOT DOCS Knowledge Test. Caregiver Perceptions of Severity of Child Behavior Research Question #2a. Do caregivers perceive their child as having more problem behavior than a normative sample prior to participation in the HOT DOCS program? Descriptive statistics were used to anal yze the severity leve ls of child problem behavior as perceived by th e caregiver prior to participating in the parent training program. Caregiver ratings on the CBCL were used as indicators of problem behaviors in children. Caregiver ratin gs were analyzed using the de scriptive categories assigned to specific score ranges as desi gnated in the CBCL manual. Number and percent of standard scor es falling within the non-significant, borderline, and clinically significant categories in the sample were calculated for a) the Internalizing, and b) Externalizing scales of the CBCL. A chi-square goodness of fit test

PAGE 89

77 was employed to determine if there was a signi ficant departure of perceived severity level of child problem behavior in the sample for th e Internalizing and Ex ternalizing scales of the CBCL from that expected in each of the three descriptive categor ies: Non-Significant (T-scores less than 65), Borderline (T-s cores between 65 and 69), and Clinically Significant (T-scores greater than or equal to 70) as expected for a distribution of scores in a national sample. Research Question #2b. Are there significant differen ces in caregiver perceptions of the severity of child problem behaviors based on caregivers' level of education, caregivers' social support network, and the target ch ild's diagnosis? One-factor repeated measures ANOVA was used to determine if there were differences in perceptions among caregivers about the severity of children's problem behaviors on the pretest CBCL for specific groups of caregiver participant. A separate analysis was conducted for each of the fo llowing independent (group) variables: education level of caregiver (i.e., less than a high school diploma, high school diploma, technical training, 2-year college degree, 4-year college, or a graduate degree); caregiver social support networks (i.e., caregiver attending training alone vs. accompanied by another caregiver); type of preexisting diagnosis of targ et child (i.e., no preexisting diagnosis, target child with autism sp ectrum disorders, developmental delays, speech/language impairments, or medical/genet ic disorders). The dependent variable for each analysis was the caregivers' rating of the se verity of the target child's behavior using standard scores on the Interna lizing and Externalizing scales of the CBCL at pretest.

PAGE 90

78 Changes in Caregiver Perceptions of Severity of Child Problem Behavior Research Question #3a. To what extent do caregivers perceive a decrease in child problem behavior following their par ticipation in the HOT DOCS program? A two-factor repeated measures ANOVA was conducted to determine if there were significant differences between caregiv ers pretest and pos ttest scores on the Internalizing and Externalizi ng scales on the CBCL. The two within-subjects (repeated) factors in this analysis were type of scal e (Internalizing vs. Exte rnalizing) and time (T) (i.e., pretest and posttest). The dependent va riable was the T-score on the CBCL scales. Research Question #3b. Are there differential perceptions of child behavior change based on caregivers' so cial support network, the target child's diagnosis, and the target child's age? Two-factor repeated measures ANOVA were conducted to determine if there were differential perceptions of the change in the severity of children's problem behaviors from pretest to posttest on the Internalizing and Externa lizing Scales of the CBCL for specific groups of participant caregivers. A separate analysis wa s conducted for each of the following independent (group) variables: social support network of caregiver (i.e., attending training alone or attending accompanied by another caregiver); type of preexisting diagnosis of target child (i.e., target child with no preexisting diagnosis, with autism spectrum disorders, with devel opmental delays, with speech/language impairments, or with medical/genetic disorders) ; and age of target ch ild (i.e., target child under three years of age vs. over three years of age). The depe ndent variable in each of the analyses was standard scores on the CBCL 's completed by careg ivers at pretest and posttest for the Internalizing and Externalizing scales.

PAGE 91

79 Overall Perceptions of the HOT DOCS Program Research Question #4. What are caregivers overall perceptions of the HOT DOCS parent training program? Caregivers mean ratings of satisfact ion with the HOT DOCS program were computed using quantitative data obtaine d from the HOT DOCS Program Evaluation Survey for Caregivers. Before data analyses were conducted, data were screened for missing data and for the assumptions underlying the various inferentia l statistics used to answer each research question. In the case where a part icipant did not provide data n eeded to answer a specific research questions, the participant was only dr opped from the particular analysis, and not excluded from the entire participant sample Consequently, several of the research questions were conducted with different sample sizes, as participants were included in each research question for which they provided complete information. Assumptions underlying the use of a dependent means t-test included independence of observations, normality of score distribution, and homogeneity of variance. Assumptions underlying the use of repeated measures analysis of variance (ANOVA) included independence of observations, normality of score distribution, homogeneity of variance, and sphericity. Because of the nature of the research design, which relied on the use of pretest and posttest scores from the same group of participants, observations were not independent of one another; consequently, repeated measures ANOVA were employed. To assess data for normali ty of distribution of scores, values of skewness and kurtosis were examined for each variable and followed by the conduct of Shapiro-Wilk tests of normality. To assess data for homogeneity of variance, Levene's

PAGE 92

80 tests were conducted for each variable. Levene s test of the assu mption of homogeneity of variance is not seriously affected by vi olations of the normality assumption (Glass, 1966). Although several subsets of the data were found to violate the assumptions of normality and homogeneity of variance, th e literature indicate s that ANOVA is robust with respect to these types of violations and as long as values of skewness and kurtosis were within acceptable limits, analyses were conducted as planned. The assumption of sphericity was not relevant in this study since the rese arch questions included only within-subjects variables containing two levels of the variable. For all repeated measures ANOVA's sphericity was assumed. Results for analyses for each assumption for individual research questions will be pr esented before the discussion of results. Measures of effect size were calculated to provide information about the strength of the relationship between the independent variable and the dependent variables (Stevens, 1999). The reader is referred to A ppendix H for a visual representation of data sources and analyses for each research question.

PAGE 93

81 Chapter 4 Results Overview The following chapter presents results of various data analyses used to answer each research question. Results are organize d by research question. As previously discussed, prior to beginning data analyses data were scre ened for missing data and for the assumptions underlying the inferential sta tistics used in each research question. Caregiver Knowledge Research Question #1a. What is the impact on caregiver knowledge of child development, behavioral principles, and parentin g strategies as a result of participation in the HOT DOCS parent training program? To determine if there was a significant difference between caregivers pretest and posttest scores of knowledge of child developm ent, behavioral principles, and parenting strategies on completion of the HOT DOCS Training program, a dependent means t -test was conducted using participants pretes t and posttest scores on the HOT DOCS Knowledge Test. Means and standard deviat ions of pretest and posttest scores of caregivers knowledge are report ed in Table 6. Although results of the Shapiro-Wilk test indicated a violation of normality, for scor es on the Knowledge Test at pretest and posttest, given the dependent means t-test's robustness against violations of normality, the analysis was conducted as planned.

PAGE 94

82 Table 6 Means and Standard Deviations for Partic ipant Scores on the Knowledge Test Measure M SD Minimum Maximum Skewness Kurtosis Pre-Test 15.88 2.278 7 20 -.635 .211 Post-Test 17.24 2.009 10 20 -.737 .241 Note. n = 250 The results of the t -test show that the particip ants mean posttest score was significantly higher than their mean pretest score, t (1, 249) = 11.22, p < .001. The effect size for the t -test was large ( d = 0.633). This finding indicat es that caregivers knowledge of child development, behavioral principles, and parenting strategi es was greater after completing the HOT DOCS program than before instruction began. Research Question #1b. Is there a difference in part icipant caregive rs knowledge of child development, behavioral principles, and parenting strategies based on caregivers' level of education, caregivers' social support network, the target child's age, and the target child's diagnosis? A two-factor (one betweensubjects one within-subject s) analysis of variance (ANOVA) was used to determine if there was a significant difference in caregiver knowledge due to participation in the HOT DO CS training program as measured by preand post-test scores on the HOT DOCS Knowledge Test for specific groups of participants. A separate analysis was c onducted for each of the following independent (between-subjects) variables: education level of caregivers; social support of caregivers; age of target child; and diagnosis of target child. The within-subject s variable in each

PAGE 95

83 analysis was time of testing (pretest vs. postte st); and the dependent variable was scores on the HOT DOCS Knowledge Test. Caregivers Level of Education The between-subjects factor was level of caregiver education (L) and the within-subject s (repeated) factor was time (T) (i.e., pretest and posttest). The variable caregivers level of education originally included six levels or categories of highest educati onal attainment. Due to significantly unequal distribution of participants acro ss the six levels, the data for this variable were collapsed into four levels: high school diploma or less, technical training + 2 year college degree, 4 year college degree, and graduate degree. Tw o-hundred-forty-five pa rticipants completed and returned both the pretest and posttest Know ledge Test and reported their highest level of education attained on the Demographics Form. Means and standard deviations of pretest and posttest scores by level of car egiver education are reported in Table 7. Table 7 Means and Standard Deviations of Preand Posttest Knowledge Test Scores by Caregivers Education Level Pretest Posttest Education Level n M SD M SD HS diploma or less 55 14.96 2.37 15.73 2.16 Tech. training/2yr college 51 15.25 2.35 16.92 1.89 4yr college degree 81 16.83 1.72 18.30 1.52 Graduate degree 58 16.00 2.37 17.53 1.63 Marginal Means 245 15.76 17.12 Note n = 245

PAGE 96

84 Results of the Shapiro-Wilk test indicate d a violation of normality, for the pretest score, posttest scores, and th e difference score from pretest to posttest. However, it has been shown through Monte Carlo studies (e.g., Norton, 1952) that analysis of variance is robust to violations of the normality assumption and this holds when group sizes are unequal. Levene's test was employed to determine if the hom ogeneity of variance assumption was violated. The test was stat istically significant for pretest scores, F (3, 241) = 3.62, p = .014, and posttest scores, F(3, 241) = 2.75, p = .044, indicating that the variance in pretest and posttest scores wa s not equally distributed across levels of caregiver's level of educa tion. The ratio of the larg est (Graduate degree, 1.5392 = 2.369) to smallest (4 Year College, 1.2322 = 1.520) group variance was less than 3:1 (2.369/ 1.520= 1.229). Because ANOVA is robust against small to moderate disproportionate levels of inequality of variance between groups, the analyses were conducted as planned (Moore, 1995). The sphericity assumption wa s not applicable as there were only two levels of the within -subjects variable. Results of the ANOVA are reported in Table 8. The data revealed a nonsignificant education level x time interaction effect F (3, 241) = 2.53, p > .05, a statistically significant main effect for time, F (1, 244) = 122.83, p < .001, 2 = 0.34, and a significant main effect for car egivers level of education F (3, 241) = 19.25, p < .001, 2 = 0.19.

PAGE 97

85 Table 8 Analysis of Variance of HOT DOCS Preand Posttest K nowledge Test Scores by Caregivers Education Level Source df SS MS F Between subjects 244 358.56 Education (L) 3 352.46 117.49 19.25* S / L 241 6.10 Within subjects 245 662.59 Time (T) 1 219.12 219.12 122.83* L x T interaction 3 13.54 4.51 2.53 ST/L 241 429.93 1.78 Total 244 1021.15 Note *p < .01 The non-significant interaction between car egiver educational level and time of testing (L x T) suggests that there was not a differential change in caregiver knowledge from preto posttest as measured by th e HOT DOCS Knowledge Test due to the educational level of the caregivers. Follow-up of the significant main effect for Time (T), was done by examining the overall HOT DOCS Knowledge Test pretest and posttest mean scores (i.e., marginal means). The signifi cant main effect for time revealed that regardless of educational leve l, the caregivers who participated in the HOT DOCS program scored significantly higher (p < .001) on the HOT DOCS Knowledge Test upon completion of the program, posttest ( M = 17.12), as compared to their performance prior to the start of the program, pretest ( M = 15.76). This finding indicates that caregivers

PAGE 98

86 knowledge of child development, behavioral principles, and parenting strategies was greater at posttest time as compared to pretest time. Follow-up of the significant main effect for level of education (L) was conducted using Bonferroni adjustment for multiple comp arisons to compare mean difference scores on the Knowledge Test for the four levels of caregivers education. Post-hoc Bonferroni tests indicated that caregivers with a gra duate degree scored si gnificantly higher than caregivers with a high school diploma or less ( p < .01), but their pe rformance did not differ significantly from that of caregivers with technical training/2-year college degree or a 4-year college degree ( p > .05). Caregivers with a 4year college degree scored significantly higher than caregivers with a high school diploma or less ( p<.01) and caregivers with technical training/2-year college degree (p < .01). Caregivers with a high school diploma or less did not si gnificantly (p > .05) differ fr om caregivers with technical training/2-year college degree. Refer to Figure 1 for a visual display of differences in mean scores for caregivers level of education.

PAGE 99

87 13 14 15 16 17 18 19 PretestPosttest TimeKnowledge Test Score High School or Less Tehcnical Training/2Yr College 4 Yr College Graduate Degree Figure 1. Mean Knowledge Test scores by caregivers education level Note n = 245 Caregivers Social Support For the repeated measures ANOVA conducted to examine the differences between caregive rs pretest and postt est scores on the Knowledge Test by level of social support received, the between-subjects factor was caregivers level of social support (A) and the within-subjects (repeated) factor was time (T) (i.e., pretest and posttest). Two-hundred-fifty participants co mpleted and returned both the pretest and posttest Knowledge Test and were coded as attending the HOT DOCS training alone or attending with another caregiver (e.g., spouse, relative, friend, therapist). In order for a caregiver to be coded as attending with another caregiver, both participants must have attended at least th ree sessions and signed the consent form.

PAGE 100

88 Means and standard deviations of pretest and posttest scores by level of caregiver social support are reported in Table 9. Table 9 Means and Standard Deviations of Preand Posttest Knowledge Test Scores by Caregivers Social Support Pretest Posttest Social Support n M SD M SD Attended Alone 126 15.93 2.33 17.20 2.09 Attended Together 124 15.83 2.23 17.27 1.93 Marginal Means 15.88 17.24 Note n = 250 Results of the Shapiro-Wilk test did not indicate a violation of normality for the pretest score, posttest scores, and the difference score from pretest to posttest for either level of caregivers' social support. The homogeneity of variance assumption was assessed by examining results of the Levene's test of equality of error va riance for levels of caregiver social support for pret est and posttest scores. Levene 's test was not significant at the .05 level for pretest, F (1, 248) = 1.88, p = .172, nor posttest scores, F (1, 248) = 0.69, p = .407, indicating that the variance in scores was equally distributed across groups of caregiver social support at pretest and posttest. The sphericity assumption was not applicable as there were only two levels of the within-subjects variable. The ANOVA results reported in Table 10 revealed a non-significant social support x time interaction effect, F (1, 248) = 0.52, p > .05, a non-significant main effect

PAGE 101

89 for caregivers level of social support, F (1, 248) = 0.002, p > .05, and a statistically significant main effect for time, F (1, 248) = 125.75, p < .001. Table 10 Analysis of Variance of HOT DOCS Preand Posttest K nowledge Test Scores by Caregivers Social Support Level Source df SS MS F Between subjects 249 1842.80 Support (A) 1 0.02 0.02 0.00 S/A 248 1842.80 7.43 Within subjects 250 684.72 Time (T) 1 230.06 230.06 125.75* Ax T 1 0.94 0.94 0.52 ST/A 248 453.72 1.83 Total 249 2527.52 Note *p<.01 The observed non-significant interaction eff ect, type of social support by time of testing, indicates that there wa s not a differential performance of participant caregivers on the HOT DOCS knowledge test from preto posttest due to the level of support they received while attending the training program. The non-significant main effect for level of support also revealed that there was no difference in the overall performance on the knowledge test by caregivers regardless of wh ether they attended the training alone, or they were accompanied by another individual who was involved in providing care for the target child. Just as in the pr evious analysis, the significant main effect for time revealed

PAGE 102

90 that regardless of careg ivers' level of support, the caregivers who part icipated in the HOT DOCS program scored significantly higher (p < .001) on the HOT DOCS Knowledge Test at posttest ( M = 17.24) than at pretest ( M = 15.88). Target Childs Age The between-subjects factor wa s target childs age (A) and the within-subjects (repeated) factor was time (T) (i.e., pretest and posttest). Twohundred-twenty-eight participants completed and returned both the pretest and posttest Knowledge Test and reported their target ch ilds age on the Demographics Form. Childs age was coded as being 18 to 35 months or 36 to 72 months (e.g., under three years of age or three years of age and over). Means and standard deviations of pretest and posttest scores by target childs ag e are reported in Table 11. Table 11 Means and Standard Deviations of Preand Post test Knowledge Test Scores by Target Childs Age Pretest Posttest Target Child's Age N M SD M SD 18-35 Months of Age 113 16.02 2.40 17.35 2.02 36-72 Months of Age 115 15.82 2.14 17.21 2.04 Marginal Means 15.92 17.28 Note n = 228 Results of the Shapiro-Wilk test did not indicate a violation of normality for the pretest score, posttest scores, and the difference score from pretest to posttest for either level of target child's age. The homogeneity of variance assumption was assessed by examining results of the Levene's test of equa lity of error variance for levels of target

PAGE 103

91 child's age for pretest and posttest scores. Leve ne's test was not significant at the .05 level for pretest, F (1,226) = 0.71, p > .05, nor posttest scores, F (1,226) = 0.66, p > .05, indicating that the variance in scores was equally distributed across groups of target child's age at pretest and posttest. The spheri city assumption was not applicable as there were only two levels of the within-subjects variable. As shown in Table 12, results revealed a non-significant in teraction effect F (1,226) = 0.07, p > .05, a non-significant main e ffect for target childs age F (1,226) = 0.43, p > .05, and a statistically significant main effect for time, F (1,226) = 117.50, p < .001, 2 = 0.34. Table 12 Analysis of Variance of HOT DOCS Preand Posttest Knowledge Test Scores by Target Child's Age Source df SS MS F Between subjects 227 1693.75 Child's age (A) 1 3.23 3.23 0.43 S/A 226 1690.52 7.48 Within subjects 228 615.90 Time (T) 1 210.64 210.64 117.50* A x T 1 0.12 0.12 0.07 ST/A 226 405.14 1.80 Total 227 2309.65 Note p <.01

PAGE 104

92 The observed non-significant interaction e ffect, target child's age by time of testing, indicates that there wa s not a differential performance of participant caregivers on the HOT DOCS knowledge test from preto pos ttest due to the age of their target child. The non-significant main effect for target ch ild's age also revealed that there was no difference in the overall performance on the knowledge test by careg ivers regardless of whether their target ch ild was under three years of age or over three years of age. Just as in the previous analysis, the significant main effect for time revealed that regardless of target child's age, the caregivers who pa rticipated in the HOT DOCS program scored significantly higher (p < .001) on the HOT DOCS Knowledge Test at posttest ( M = 17.28) than at pretest ( M = 15.92). Target Childs Diagnosis The between-subjects factor was target childs diagnosis (D) and the within-s ubjects (repeated ) factor was time (T) (i.e., pretest and posttest). Two-hundred-twenty-four participants completed a nd returned both the pretest and posttest Knowledge Test and reported th eir target childs pr eexisting medical or psychological diagnosis on the Demographics Form. Childs diagnosis was coded as no diagnosis, autism spectrum disorder, medi cal/genetic disorder speech/language impairment, or developmental delay. Means and standard deviations of pretest and posttest scores by target childs di agnosis are reported in Table 13.

PAGE 105

93 Table 13 Means and Standard Deviations of Preand Post test Knowledge Test Scores by Target Childs Diagnosis Pretest Posttest Child's Diagnosis N M SD M SD No Diagnosis 97 15.60 2.27 17.12 2.03 Autism Spectrum 30 16.03 2.61 17.53 1.68 Medical/Genetic 21 16.52 1.97 16.81 2.44 Speech/Language 49 16.06 2.12 17.67 1.91 Developmental Delay 27 16.07 2.67 17.56 1.87 Marginal Means 15.90 17.32 Note n = 224 Results of the Shapiro-Wilk test indicate d a violation of normality, for the pretest score, posttest scores, and the difference score from pr etest to posttest. Although the assumption of normal distribution of scores was violated for these levels of the independent variable, the analysis was conducted due to ANOVA's robustness against violations of normality. The homogeneity of variance assumption was assessed by examining results of the Levene's test of equa lity of error variance for levels of target child's diagnosis for pretest and posttest scores Levene's test was not significant at the .05 level for pretest, F (4,219) = 0.505, p > .05, nor posttest scores, F (4,219) = 1.19, p > .05, indicating that the variance in scores was equally distributed across groups of target child's diagnosis at pretest and posttest. The sphericity assumption was not applicable as there were only two levels of the within-subjects variable.

PAGE 106

94 As shown in Table 14, results revealed a non-si gnificant interaction effect F (4, 219) = 2.08, p > .05, a non-significant main effect for target childs diagnosis F (4, 219) = 0.79, p > .05, and a statistically significant main effect for time, F (1, 219) = 75.87, p < .001, 2 = 0.26. Table 14 Analysis of Variance of HOT DOCS Preand Posttest Knowledge Test Scores by Target Child's Diagnosis Source df SS MS F Between subjects 223 1605.42 Child's diagnosis (D) 4 22.74 5.68 0.79 S/D 219 1582.68 7.23 Within subjects 224 551.22 Time (T) 1 137.95 137.95 75.87* D x T 4 15.10 3.78 2.08 ST/D 219 398.17 1.82 Total 223 2156.64 Note p <.01 The observed non-significant inte raction effect, target child's diagnosis by time of testing, indicates that there wa s not a differential performance of participant caregivers on the HOT DOCS Knowledge Test from preto posttest due to a preexisting diagnosis of their target child. The non-significant main effect for target child's diagnosis also revealed that there was no difference in th e overall performance on the knowledge test by caregivers regardless of whether their target child had a preexisting diagnosis or not.

PAGE 107

95 Caregiver Perceptions of Severity of Child Behavior Research Question #2a. Do caregivers perceive their child as having more problem behavior than a normative sample prior to participation in the HOT DOCS program? In order to describe and analyze caregiver pe rceptions of the severi ty of child problem behaviors before participation in the program, the frequency and percent of caregiver ratings of child behavior falling within specific descriptive categories on the CBCL administered at pretest were calculated. Fre quencies and percents were calculated using the Internalizing and Externalizing Problems T-scores. The frequencies of scores falling within these ranges were compared to the numbe r of scores expected to fall within each category according to the percentages unde r the normal curve (Achenbach, 2001). On the CBCL, scores classified as norm al or Non-Significant ranged from 0 to 64; scores classified as Borderline ranged from 65 to 70; and scores classified as Clinically Significant are those reach ing 70 and above. In the normative population, 93.94% of scores fell within the NonSignificant range, 3.79% of scores fell within the Borderline range, and 2.27% of scores fell within the Clinically Significant range for the CBCL. To determine whether participant caregivers' ratings on the Internalizing and Externalizing scales deviated from what was expected for the normative population, data were subjected to a chi-square goodness of fit analysis. Chi-square analyses were performed between observed and expected frequencies of scores in each de scriptive category for scores in the Internalizing and Extern alizing subscales of the CBCL. Two-hundredeleven participants completed and returned the CBCL rating scale at pretest. Means and

PAGE 108

96 standard deviations of pretes t scores for the Internalizing and Externalizing scales are reported in Table 15. Table 15 Means and Standard Deviations for Pretest Scores on the CBCL Scale M SD Minimum Maximum Skewness Kurtosis Internalizing 56.69 11.078 29 86 -.114 -.050 Externalizing 60.09 11.654 32 92 .284 -.032 Note. n = 211 A separate chi-square goodness of fit test was performed for the Internalizing scores and for the Externalizing scores at pretest time. The alpha-level used was = .01. Observed and expected frequency distribu tions for Internalizing and Externalizing subscale score comparisons are displayed in Table 16. Table 16 Observed and Expected Frequencies for CBCL Internalizing & Externalizing Subscale TScores Observed f Category Expected f Internalizing Externalizing Non-Significant 198 124 107 Borderline 8 29 26 Clinically Significant 5 58 78 Note n = 211 The resultant overall test for the Interna lizing scale was statistically significant, 2 (1, N = 211) = 645.59. Thus, a significant diffe rence between the expected frequency

PAGE 109

97 of scores in each descriptive category and the actual or obtained frequency of scores in each descriptive category for the CBCL Inte rnalizing subscale was found. Caregivers perceived children in the sample to have highe r frequencies of more severe internalizing problem behavior than would be expect ed for a normative sample. Specifically, significantly more target child rens scores fell within the Clinically Significant and Borderline descriptive categories and significantly fewer childrens scores fell within the Non-Significant descriptive category than were expected. Nearly 11 times the number of children expected to have scores in the C linically Significant range were found in the sample. Effect size was calculat ed to describe the strength of the relationship between the expected and obtained values. The effect size for the chi-square calc ulation for scores on the Internalizing subscale was large ( w = 1.787), indicating that the differences between participants perceptions of th e severity of child problem behavior and expectations for a normative sample were not only statistically si gnificant but also clinically meaningful. The resultant overall test for the Exte rnalizing scale also was statistically significant, 2 (1, N = 211) = 1148.12. Thus, a significant difference between the expected frequency of scores in each descriptive categ ory and the actual or obtained frequency of scores in each descriptive category for the CBCL Externalizing subscale was found. Caregivers perceived a signifi cantly higher proportion of child ren in the sample to have more severe externalizing problem behavior than would be expected for a normative sample. Specifically, significantly more target childrens scores fell within the Clinically Significant and Borderline de scriptive categories and signi ficantly fewer childrens scores fell within the Non-Significant desc riptive category than were expected. The observed number of children in the sample whose Externalizing subscale scores fell

PAGE 110

98 within the Clinically Signifi cant range was nearly 16 times the number expected to fall within that range. Effect size was calculated to describe the strengt h of the relationship between the expected and obtained values. The effect size for the chi-square calculation for scores on the Externalizing subscale was large ( w = 2.386), indicating that the differences between participants perceptions of the severity of child problem behavior and expectations for a normative sample were not only statistically significant but also clinically meaningful. A gra phic comparison of observed and expected frequencies of Tscores for the Internalizing and Extern alizing scales is shown in Figure 2. 198 8 5 124 29 58 107 26 780 50 100 150 200 250Non-Significant Borderline Clinically SignificantStandard Score s Expected Observed Internalizing Observed Externalizing Figure 2 Number of expected and observed CBCL T-scores by descriptive category. Note n = 211 Research Question #2b. Are there significant differences in caregiver perceptions of the severity of child problem behaviors based on caregivers' level of education, caregivers' social support network, and the target ch ild's diagnosis?

PAGE 111

99 One-factor repeated measures ANOVA were used to determine if there were differential perceptions among caregivers a bout the severity of children's problem behaviors on the pretest CBCL for specific groups of caregiver participant. A separate analysis was conducted for each of the fo llowing independent (group) variables: education level of caregiver (i.e., high school diploma or less, vs. technical training/2year college degree, vs. 4-year college, vs. a graduate de gree); caregiver social support networks (i.e., caregiver attending training alone vs. accompanied by another caregiver); and type of preexisting diagnosis of target child (i.e., no preexisting diagnosis, target child with autism spectrum disorders, developmental delays, speech/language impairments, or medical/genetic disorders). The independent variab le, caregivers level of education originally included six levels or categories of highest educational attainment. The data for this variable were collapsed in to four levels: high school diploma or less, technical training + 2 year college degree, 4 year college degree, and graduate degree. The dependent variable for each analysis was th e caregivers' rating of the severity of the target child's behavior using standard scores on the Internalizing a nd Externalizing scales of the CBCL at pretest. Caregivers Level of Education Two-hundred-nine participants completed and returned the CBCL rating scale at pretest and reported thei r highest level of education attained on the Demographics Form. The betwee n-subjects factor wa s level of caregiver education (L) and the within-s ubjects (repeated) factor was s cale (C) (i.e., Internalizing and Externalizing). Means and standard deviations of CBCL Internalizing and Externalizing scale scores by caregivers education leve l are shown in Table 17.

PAGE 112

100 Results of the Shapiro-Wilk test did not indicate a violation of normality for the Internalizing and Externalizing T-scores at pretest on any of the six levels of caregiver education. The homogeneity of variance assumption was assessed by examining results of the Levene's test of equality of error variance for levels of caregiver education for Internalizing and Externalizing scores. Le vene's test was not significant for the Internalizing scale, F (3, 205) = 0.17, p > .05, nor for the Externalizing scale, F (3, 205) = 1.34, p > .05, indicating that the variance in scores was equally distributed across groups of caregiver education on the In ternalizing and Externalizing scales at pretest. The sphericity assump tion was not applicable as ther e were only two levels of the within-subjects variable. Table 17 Means and Standard Deviations of CBCL Inte rnalizing and Externalizing Scores by Caregivers Education Level Internalizing Externalizing Education Level n M SD M SD High school or less 50 60.62 11.70 63.06 12.58 Tech. training/2yr college 43 57.19 11.94 60.98 12.36 4 yr college degree 70 55.26 9.92 59.13 10.80 Graduate degree 46 53.78 10.30 57.83 10.602 Marginal Means 209 56.71 60.25 Note n = 209 As shown in Table 18, results of the ANOVA revealed a non-significant interaction effect F (3, 205) = 0.23, p > .05, a statistically significant main effect for

PAGE 113

101 caregivers' education level F (3, 205) = 3.33, p = .021, 2 = 0.05 and a statistically significant main effect for scale, F (1, 205) = 27.30, p < .001, 2 = 0.12. The observed non-significant interaction eff ect, caregivers' level of education by scale, indicates that there were not differ ential ratings of children's Internalizing or Externalizing behaviors at pr etest on the CBCL due to caregivers' level of education. Follow-up of the significant main effect for Scale (C), was done by examining the overall CBCL Internalizing and Externalizing mean scores (i.e., marginal means). The significant main effect for scale revealed that regardle ss of educational level, the caregivers who participated in the HOT DOCS program rated th eir target child's behaviors significantly higher (p < .001) on the Externalizing scale ( M = 60.25) than on the Internalizing scale ( M = 56.71) at pretest. This finding indicates that caregivers perc eived their target children to have more severe levels of Externalizing behavior s than Internalizing behavior upon beginning the HOT DOCS progr am. On the CBCL, behaviors categorized as Externalizing include attention span, hyperactivity, rule-breaking, and aggression. Behaviors categorized as Internalizing incl ude emotional reactivit y, anxiety/depression, somatic complaints (e.g., stomach aches, headaches, overly concerned with neatness or cleanliness), and withdrawal.

PAGE 114

102 Table 18 Analysis of Variance of Pre-Test CBCL Scores by Caregivers Education Level Source df SS MS F Between subjects 208 44083.21 Education (L) 3 2047.07 682.36 3.33* S/L 205 42036.14 205.05 Within subjects 209 10780.83 Scale (C) 1 1261.96 1261.96 27.30** L x C interaction 3 41.27 13.76 0.30 SC/L 205 9477.60 46.23 Total 208 54864.04 Note *p < .05, ** p < .001 Follow-up of the significant main effect for level of education (L) was conducted using Bonferroni adjustment for multiple comp arisons to compare mean difference scores across the Internalizing scales for the four levels of careg ivers education. Post-hoc Bonferroni tests indicated that caregivers wi th a graduate degree ra ted target children's behavior significantly lower than caregive rs with a high school diploma or less ( p < .01), but did not differ significantly from the mean score of caregivers with technical training/2-year college degree or a 4-year college degree ( p > .05). There were no other significant differences ( p > .05) in mean difference scores between any of the other levels of caregivers' level of education. Refer to Figu re 3 for a visual display of differences in mean Internalizing and Externalizing scale scores by caregivers level of education.

PAGE 115

103 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65Internalizing Externalizing CBCL ScaleT-Score at Pretest High School or Less Tehcnical Training/2Yr College 4 Yr College Graduate Degree Figure 3. Mean CBCL Internalizing and Externaliz ing subscale scor es by caregivers education level. Note n = 209 Caregivers Social Support Two-hundred-eleven participants completed and returned the CBCL rating scale at pretest and were coded by attendance rates either as attending the program alone or attending w ith another caregiver. The between-subjects factor was level of caregiver social support (P) an d the within-subjects (repeated) factor was scale (S) (i.e., Internalizing and Externa lizing). Means and sta ndard deviations of CBCL Internalizing and Externalizing scale scores by caregivers social support are shown in Table 19.

PAGE 116

104 Table 19 Means and Standard Deviations of CBCL Inte rnalizing and Externalizing Scores by Caregivers Social Support Internalizing Externalizing Social Support n M SD M SD Attended Alone 121 55.71 11.47 59.29 11.85 Attended Together 90 58.00 10.45 61.17 11.36 Marginal Means 56.69 60.09 Note n = 211 Results of the Shapiro-Wilk test did not indicate a violation of normality for the Internalizing and Externalizing T-scores at pretest on either of the levels of caregiver social support. The homogeneity of varian ce assumption was assessed by examining results of the Levene's test of equality of error variance for levels of caregiver social support for Internalizing and Externalizing scores. Levene's test was not significant for the Internalizing scale, F (1, 209) = 0.31, p > .05, nor for the Externalizing scale, F (1, 209) = 0.29, p > .05, indicating that the variance in scores was equally distributed across groups of caregiver social support on the Internalizing and Externalizi ng scales at pretest. The sphericity assumption was not applicable as there were only two levels of the withinsubjects variable. As shown in Table 20, results revealed a non-si gnificant interaction effect F (1, 209) = 0.09, p > .05, a non-significant main effect for caregivers' social support F (1, 209) = 2.13, p > .05, and a statistically significant main effect for scale, F (1, 209) = 25.03, p < .001, 2 = 0.11.

PAGE 117

105 Table 20 Analysis of Variance of Pre-Test CBCL Scores by Caregivers Social Support Source df SS MS F Between subjects 211 44483.27 Support (P) 1 448.02 448.02 2.13 S/P 209 44035.25 210.70 Within subjects 10981.92 Scale (C) 1 1174.09 1174.09 25.03* P x C interaction 1 4.38 4.38 0.09 SC/P 209 9803.00 46.90 Total 210 55465.19 Note *p < .01 The observed non-significant interaction e ffect, caregivers' level of support by scale, indicates that there were no differential ratings of children's Internalizing or Externalizing behaviors at pr etest on the CBCL due to the level of support they received while attending the training program. The non-si gnificant main effect for level of support also revealed that there was no difference in the overall ratings on the CBCL at pretest by caregivers regardless of whether they atte nded the training alone, or they were accompanied by another individual who was i nvolved in providing care for the target child. Just as in the previous analysis, the si gnificant main effect for scale revealed that regardless of level of social support, the caregivers who participated in the HOT DOCS program rated their target child's behavior s significantly higher (p < .001) on the Externalizing scale ( M = 60.09) than on the Internalizing scale ( M = 56.69) at pretest.

PAGE 118

106 Target Childs Diagnosis Two-hundred participants co mpleted and returned the CBCL rating scale at pretest and reported th e target childs preexisting diagnosis on the Demographics Form. The between-subjects factor was level of target child's preexisting diagnosis (D) and the within-s ubjects (repeated) factor was scale (S) (i.e., Internalizing and Externalizing). Means and standard deviations of CBCL Internalizing and Externalizing scale scores by target ch ilds diagnosis are shown in Table 21. Table 21 Means and Standard Deviations of CBCL Inte rnalizing and Externalizing Scores by Target Childs Diagnosis Internalizing Externalizing Child's Diagnosis N M SD M SD No diagnosis 85 53.91 10.54 58.04 10.61 Autism spectrum 24 64.50 9.75 67.00 13.46 Medical/genetic 22 55.45 11.47 62.14 12.58 Speech/language 40 58.20 10.54 59.85 10.22 Developmental delay 29 58.31 11.97 61.48 12.65 Marginal Means 56.96 60.43 Note n = 200 Results of the Shapiro-Wilk test did not indicate a violation of normality for the Internalizing and Externalizing T-scores at pretest on any of the five levels of target child's preexisting diagnosis. The homogene ity of variance assumption was assessed by examining results of the Levene's test of equa lity of error variance for levels of caregiver education for Internalizing and Externalizing scores. Levene's test was not significant at

PAGE 119

107 the .05 level for the Internalizing scale, F (4, 195) = 0.38, p > .05, nor for the Externalizing scale, F (4, 195) = 1.13, p > .05, indicating that the variance in scores was equally distributed across groups of target child's diagnosis on the Internalizing and Externalizing scales at pretest. The sphericity assumption was not applicable as there were only two levels of the within-subjects variable. As shown in Table 22, results revealed a nonsignificant interacti on effect F (4, 195) = 0.82, p > .05, a statistically significant main effect for target child's diagnosis F (4, 195) = 4.69, p < .001, 2 = 0.09, and a statistical ly significant main effect for scale, F (1, 195) = 20.08, p < .001, 2 = 0.09. Table 22 Analysis of Variance of Pre-Test CBCL Scores by Target Child's Diagnosis Source df SS MS F Between subjects 199 42863.56 Diagnosis (D) 4 3759.22 939.81 4.69* S/D 195 39104.34 200.54 Within subjects 10076.74 Scale (C) 1 926.83 926.83 20.08* D x C interaction 4 151.12 37.78 0.82 SC/D 195 8998.79 46.15 Total 199 52940.30 Note *p < .01 The observed non-significant inte raction effect, target child's diagnosis by scale, indicates that there were no di fferential ratings of children's Internalizing or Externalizing

PAGE 120

108 behaviors at pretest on the CBCL due to target child's preexisti ng diagnosis. Just as in the previous analysis, the significant main effect for scale revealed that regardless of target child's preexisting diagnosis, the caregivers who participated in the HOT DOCS program rated their target child's behaviors significantly higher (p < .001) on the Externalizing scale (M = 60.43) than on the Internalizing scale ( M = 56.96) at pretest. Post-hoc comparisons of the main eff ect for child's diagnosis were performed using the Bonferroni adjustment for multiple comparisons. Caregivers of target children with no diagnosis rated the ch ildren's behavior across the In ternalizing and Externalizing scales of the CBCL as be ing significantly lower ( p < .01) than caregivers of target children with a diagnosis of autism spectru m disorder (mean difference = 9.78 points). Since higher scores on the CBCL indicate more severe levels of problem behavior, this finding suggests that caregivers of children w ith a diagnosis of autism spectrum disorder perceived their children's behavior problems to be more severe than caregivers of children without a diagnosis before be ginning the HOT DOCS program. No other significant differences were found between th e remaining levels of target child's diagnosis. Refer to Figure 4 for a visual disp lay of differences in mean scores on the Internalizing and Externalizing scales at pretest for various levels of target child's diagnosis.

PAGE 121

109 50 52 54 56 58 60 62 64 66 68 InternalizingExternalizing CBCL ScaleT-Score at Pretest No Diagnosis Autism spectrum Medical/genetic Speech/language Developmental delay Figure 4. Mean CBCL Internalizing a nd Externalizing subscale scores by target childs diagnosis Note. Non-significant interaction effect. Changes in Child Problem Behavior Research Question #3a. To what extent do caregivers perceive a decrease in child problem behavior following their par ticipation in the HOT DOCS program? In order to analyze potentia l changes in the severity of child problem behavior as perceived by caregivers from pretest and pos ttest, a two-factor repeated measures ANOVA was computed. The two within-subject s factors were type of scale, (A, Internalizing and Externalizing) and time, (T pretest and posttest) Means and standard deviations of pretest and pos ttest rating scale scores on th e two subscales of the CBCL are reported in Table 23.

PAGE 122

110 Table 23 Means and Standard Deviations of Preand Posttest CBCL Scores by Scale Pretest Posttest CBCL Scales M SD M SD Internalizing 58.51 9.59 54.70 11.01 Externalizing 63.37 11.76 57.89 11.40 Marginal Means 60.94 56.30 Note n = 84 Results of the Shapiro-Wilk test indicate d a violation of normality for the change from pretest to posttest for the Externaliz ing scale. Although the Shapiro-Wilk test indicated a violation of normality assumption for this dependent variable, the analysis was conducted due to ANOVA's robustness against violations of normality and acceptable levels of skew and kurtosis. The homogeneity of variance assumption was not examined because there were no between-subj ects variables. The sphericity assumption was not applicable as there were only tw o levels of the within-subjects variable. As shown in Table 24, results revealed a nonsignificant interacti on effect F (1, 83) = 3.66, p > .05, a statistically signifi cant main effect for time, F (1, 83) = 36.45, p < .001, 2 = 0.31, and a significant main effect for scale F (1, 83) = 19.00, p < .001, 2 = 0.19.

PAGE 123

111 Table 24 Analysis of Variance of CBCL Preand Posttest Scores Source df SS MS F Scale (A) 1 1360.05 1360.05 19.00* Time (T) 1 1810.71 1810.71 36.45* Subject (S) 83 28580.74 344.35 A x T 1 58.33 58.33 3.66 Error (scale) 83 5942.45 71.60 Error (time) 83 4122.79 49.67 Residual error (SAT) 83 28580.74 344.346 Total 83 28580.74 Note *p < .01 The non-significant interaction effect, time of testing and scale of the CBCL, indicated that there was no difference in caregi vers ratings of target childs behavior due to time of testing or subscale of the CBCL. Follow-up of the significant main effect for Time (T), was done by examining the overa ll CBCL pretest and posttest mean scores (i.e., marginal means). The mean posttest score ( M = 56.30) was significantly lower than the mean pretest score ( M = 60.94). This finding indicates that across both scales of the CBCL, caregivers perceived severity of ch ildrens problem behavior was greater at pretest time as compared to posttest time. On the CBCL, higher scores indicate more severe levels of problem behavi or; therefore, a decrease in scores from pretest to posttest indicates caregivers perceived th eir target child to have signif icantly less severe levels of problem behavior following participation in th e program. Refer to Figure 5 for a graphic

PAGE 124

112 representation of the pretes t and posttest mean scores for the Internalizing and Externalizing scales of the CBCL. 45 50 55 60 65 70 75Pre-Test Marginal Mean Post-Test Marginal MeanStandard ScoreClinically Significant Borderline Non-SignificantSignificant difference between pretest and posttest mean scores Figure 5. Preand posttest mean scores for CBCL scales. Note n = 84 Follow-up of the significant main effect for Scale (S), was done by examining the overall CBCL Internalizing and Externalizing mean scale scores (i.e., marginal means). The mean Internalizing scale score across time ( M = 56.61) was significantly lower than the mean Externalizing scale score across time ( M = 60.63). This finding indicates that for both the pretest and posttest caregivers perceived the severity of their target childs Externalizing problem behaviors was greater than the perceive d severity for Internalizing problem behaviors. On the CBCL, behavior s categorized as Ex ternalizing include attention span, hyperactivity, rule-breaking, and aggression. Behaviors categorized as

PAGE 125

113 Internalizing include emotiona l reactivity, anxiety/depressi on, somatic complaints (e.g., stomach aches, headaches, overly concerne d with neatness or cleanliness), and withdrawal. Research Question 3b Are there differential perceptions of child behavior change based on caregivers' social support network, th e target child's diagnosis, and the target child's age? A one-between-subjects, two-within-subjects repeated measures ANOVA was conducted to determine if there were differentia l perceptions of the change in the severity of children's problem behaviors from pretest to posttest on the Internalizing and Externalizing Scales of th e CBCL for specific groups of participant caregivers. A separate analysis was conducted for each of the following independent (betweensubjects) variables: social support network of caregiver (i.e., atte nding training alone or attending accompanied by another caregiver); type of preexisti ng diagnosis of target child (i.e., target child with no preexisting diagnosis, target child with any preexisting diagnosis); and age of target child (i.e., targ et child under three years of age vs. over three years of age). The independent between-subject s variable child's diagnosis originally included five levels or categories of preex isting diagnoses. Due to small numbers of scores falling within each of the existing categories, the data for this variable were collapsed into two levels: target child wit hout a preexisting diagnosis and target child with any existing diagnosis. The dependent variable in each of the analyses was standard scores on the CBCL's completed by caregivers at pretest and posttest for the Internalizing and Externalizing scales.

PAGE 126

114 Caregivers Social Support. The between-subjects factor was caregiver social support (P, Attended alone vs. Attended togeth er). The within-subject s (repeated) factors were time (T, pretest vs. postte st) and scale (C, Internalizi ng vs. Externalizing). Eightyfour participants completed and returned both the pretest and posttest CBCL and were coded based on attendance either as attending the program alone or attending with another caregiver. Means and standard devia tions of pretest and posttest rating scale scores on the two subscales of the CBCL for each level of caregivers social support are reported in Table 25. Table 25 Means and Standard Deviations of Preand Postte st CBCL Scores by Caregivers' Social Support Internalizing Externalizing Social Support n Pretest Posttest Pretest Posttest Attended Alone 52 M 57.79 54.69 62.75 57.88 SD 9.83 11.36 12.07 11.41 Attended Together 32 M 59.69 54.72 64.38 57.91 SD 9.22 10.61 11.36 11.56 Marginal Means 58.11 54.70 63.37 57.89 Note n = 84 Results of the Shapiro-Wilk test indicate d a violation of normality for the change from pretest to posttest fo r the Internalizing and Externalizing scales. Although the Shapiro-Wilk test indicated a violation of normality assumption for this dependent variable, the analysis was conducted as planned due to ANOVA's robustness against

PAGE 127

115 violations of normality and acceptable levels of skew and kurtosis. The homogeneity of variance assumption was assessed by examining re sults of the Levene's test of equality of error variance for levels of caregiver social support for pretest and posttest scores on the Internalizing and Externalizing scales. Leve ne's test was not significant for pretest Internalizing scores, F (1, 82) = 0.00, p > .05, posttest Internalizing scores, F (1, 82) = 0.23, p > .05, pretest Externalizing scores, F (1, 82) = 0.50, p > .05, nor posttest Externalizing scores, F (1, 82) = 0.30, p > .05, indicating that the variance in scores was equally distributed across groups of caregiver social support at pr etest and posttest for both scales of the CBCL. The sphericity assumption was not applicable as there were only two levels of the within-subjects variable. As shown in Table 26, results revealed a non-significant support by time by scale interaction effect F (1,83) = 0.02, p > .05, a non-significant support by time interaction effect F (1,83) = 1.21, p > .05, a non-significant support by scale interaction effect F (1,83) = 0.01, p > .05, a non-significant main e ffect for caregiver support F (1,82) = 0.18, p > .05, a statistically signifi cant main effect for time, F (1,83) = 37.61, p < .001, 2 = 0.31, and a statistically significant main effect for scale, F (1,83) = 17.56, p < .001, 2 = 0.18.

PAGE 128

116 Table 26 Analysis of Variance of CBCL Preand Posttest Scores by Caregivers Level of Social Support Source df SS MS F Between subjects 83 28580.74 Support (P) 1 63.19 63.19 0.18 Residual between 82 28517.55 347.78 Within subjects 87 4520.48 Time (T) 1 1863.69 1863.69 37.61* P x T interaction 1 59.84 59.84 1.21 Scale (C) 1 1272.39 1272.39 17.56* P x C 1 0.39 0.39 .01 P x T x C 1 0.36 0.36 0.02 Residual within 82 1323.81 16.14 Total 83 33101.22 Note *p < .01 The non-significant interacti on effects for each combination of the variables time of testing, scale of the CBCL, and caregivers level of social suppor t while attending the HOT DOCS program, indicated that there was no difference in caregivers ratings of target childs behavior due to time of te sting, the subscale of the CBCL, or caregivers level of support. The non-significant main eff ect for Scale (S), indicated that across both levels of caregivers support, caregivers perceptions of their target childs behavior were the same across pretest and posttest times, regardless of the scale of the CBCL (e.g.,

PAGE 129

117 Internalizing vs. Externalizing). The non-significant main effect for Support (P), indicated that caregive rs perceptions of their target ch ilds behavior were the same for both scales of the CBCL across pretest and posttest times, rega rdless of caregivers level of support. Follow-up of the significant main effect for Time (T), was done by examining the overall CBCL pretest and posttest mean scores (i.e., marginal means). The mean posttest score ( M = 56.30) was significantly lower than the mean pretest score ( M = 60.75). This finding indicates that across bot h scales of the CBCL, regard less of caregivers level of social support, caregivers perc eived severity of childrens problem behavior was greater at pretest time as compared to posttest ti me. On the CBCL, higher scores indicate more severe levels of problem behavi or; therefore, a decrease in scores from pretest to posttest indicates caregivers perceived th eir target child to have signif icantly less severe levels of problem behavior following participation in the program. Target Childs Diagnosis. The between-subjects factor was target child's diagnosis (D, No existing diagnosis and Existing diagnosis). The within-subjects (repeated) factors were time (T, pretest and posttest) and scale (C, Internalizing and Externalizing). Seventy-nine participants completed and re turned both the pretest and posttest CBCL and reported the target child s preexisting diagnosis on the Demographics Form. Means and standard deviations of pretest and posttest rati ng scale scores on the two subscales of the CBCL for each level of target childs preex isting diagnosis are reported in Table 27.

PAGE 130

118 Table 27 Means and Standard Deviations of Preand Post test CBCL Scores by Target Child's Diagnosis Internalizing Externalizing Child's Diagnosis n Pretest Posttest Pretest Posttest No Existing diagnosis 30 M 55.73 50.10 61.67 53.80 SD 9.40 9.80 10.78 9.23 Existing diagnosis 49 M 60.98 58.24 65.18 60.71 SD 9.10 10.69 12.14 12.00 Marginal Means 58.99 55.15 63.85 58.09 Note n = 79 Results of the Shapiro-Wilk test indicate d a violation of normality for the change from pretest to posttest fo r the Internalizing and Externalizing scales. Although the Shapiro-Wilk test indicated a violation of normality assumption for this dependent variable, the analysis was conducted as planned due to ANOVA's robustness against violations of normality and acceptable levels of skew and kurtosis. The homogeneity of variance assumption was assessed by examining re sults of the Levene's test of equality of error variance for levels of target child's preexisting dia gnosis for pretest and posttest scores on the Internalizing and Externalizing scales. Levene's test was not significant for pretest Internalizing scores, F (1, 77) = 0.04, p > .05, posttest Internalizing scores, F (1, 77) = 0.13, p > .05, pretest Externalizing scores, F (1, 77) = 1.83, p > .05, nor posttest Externalizing scores, F (1, 77) = 3.76, p > .05, indicating that the variance in scores was equally distributed across groups of target child's diagnosis at pretest and posttest for both

PAGE 131

119 scales of the CBCL. The spheri city assumption was not applic able as there were only two levels of the within -subjects variable. As shown in Table 28, results revealed a non-significant diagnosis by time by scale interaction effect F (1, 77) = 0.07, p > .05, a non-significant diagnosis by time interaction effect F (1, 77) = 3.66, p > .05, a non-significant diagnos is by scale interaction effect F (1, 77) = 0.61, p > .05, a statistically significant ma in effect for target child's diagnosis, F (1,7 7) = 8.37, p = .005, 2 = 0.10, a statistically signi ficant main effect for time, F (1, 77) = 39.54, p < .001, 2 = 0.34, and a statistically si gnificant main effect for scale, F (1, 77) = 18.45, p < .001, 2 = 0.19. The non-significant interacti on effects for each combination of the variables time of testing, scale of the CBCL, and target child s preexisting diagnosis, indicated that there was no difference in caregivers ratings of targ et childs behavior due to time of testing, the subscale of the CBCL, or target childs diagnosis. Follow-up of the significant main effect for Scale (S), was done by examining the overall CBCL Internalizing and Externalizi ng mean scale scores across time (i.e., marginal means). The mean Internalizing scale score across time and target childs diagnosis ( M = 57.07) was significantly lower than the mean Externalizing scale score across time and target childs diagnosis ( M = 60.97). This finding indicates that for both the pretest and posttest, across le vels of target childs diagnos is, caregivers perceived the severity of their target childs Externaliz ing problem behaviors was greater than the perceived severity for Internalizing pr oblem behaviors. On the CBCL, behaviors categorized as Externalizing include attention span, hyperactivit y, rule-breaking, and aggression. Behaviors categorized as Inte rnalizing include emotional reactivity,

PAGE 132

120 anxiety/depression, somatic complaints (e.g., stomach aches, headaches, overly concerned with neatness or cleanliness), and withdrawal. Table 28 Analysis of Variance of CBCL Preand Posttest Scores by Target Child's Diagnosis Source df SS MS F Between subjects 78 26935.90 Diagnosis (D) 1 2640.00 2640.00 8.37* Residual between 77 24295.90 315.53 Within subjects 82 4734.12 Time (T) 1 1994.08 1994.08 39.54* D x T interaction 1 184.40 184.40 3.66 Scale (C) 1 1237.00 1237.00 18.45* D x C interaction 1 40.75 40.75 0.61 D x T x C interaction 1 1.16 1.16 0.07 Residual within 77 1276.73 16.58 Total 78 31670.02 Note *p < .01 Follow-up of the significant main effect for Time (T), was done by examining the overall CBCL pretest and posttest mean scores (i.e., marginal means). The mean posttest score ( M = 56.62) was significantly lower than the mean pretest score ( M = 61.42). This finding indicates that across both scales of the CBCL, re gardless of target childs diagnosis, caregivers perceived severity of childrens probl em behavior was greater at pretest time as compared to posttest time. On the CBCL, higher scores indicate more

PAGE 133

121 severe levels of problem behavi or; therefore, a decrease in scores from pretest to posttest indicates caregivers perceived th eir target child to have signif icantly less severe levels of problem behavior following participation in the program. Follow-up of the significant main effect for Diagnosis (D) was done by examining the overall CBCL mean scores for each level of target childs diagnosis (i.e., marginal means). The mean CBCL score across time fo r target children w ithout a preexisting diagnosis ( M = 55.33) was significantly lower than the mean CBCL score across time for target children with a preexisting diagnosis ( M = 61.43). This finding indicates that for both the pretest and posttest, across the Intern alizing and Externalizi ng scales, caregivers whose target child did not have a preexisti ng diagnosis perceived the severity of their target childs behaviors to be less severe th an did caregivers whose target child had a preexisting diagnosis. Target Childs Age. The between-subjects factor was target child's age (A, Under 36 months vs. 36 months and over). The with in-subjects (repeated) factors were time (T, pretest vs. posttest) and scale (S, Internalizi ng vs. Externalizing). Ei ghty-four participants completed and returned both the pretest and posttest CBCL and reported the target childs age on the Demographics Form. Means and standard deviations of pretest and posttest rating scale scores on the two subscales of the CBCL for each level of target childs age are reported in Table 29.

PAGE 134

122 Table 29 Means and Standard Deviations of Preand Postte st CBCL Scores by Target Childs Age Internalizing Externalizing Social Support n Pretest Posttest Pretest Posttest 18-35 Months of Age 43 M 57.91 55.14 62.98 58.53 SD 10.83 10.65 12.72 12.53 36-72 Months of Age 41 M 59.15 54.24 63.78 57.22 SD 8.19 11.50 10.81 10.19 Marginal Means 58.51 54.70 63.37 57.89 Note n = 84 Results of the Shapiro-Wilk test indicate d a violation of normality for the change from pretest to posttest for the Internalizing a nd Externalizing scales of the CBCL. The violation of the normality assumption was assessed by examining the skewness and kurtosis for the pretest score, posttest score, and change in scores on both scales for each level of target child's age. Although the Shapiro-Wilk test indicated a violation of normality assumption for this dependent variable, the analysis was conducted as planned due to ANOVA's robustness against violati ons of normality and acceptable levels of skew and kurtosis. The homogeneity of variance assumption was assessed by examining results of the Levene's test of equality of error variance for levels of target child's preexisting diagnosis for pretes t and posttest scores on the In ternalizing and Externalizing scales. Levene's test was not significant for posttest Internalizing scores, F (1, 80) = 0.45, p > .05, indicating that the variance in scores was equally distributed across groups of target child's diagnosis at postte st on the Internalizing scale.

PAGE 135

123 Levene's test was statistically significant for pr etest Internalizing scor es, F (1, 80) = 9.47, p = .003, pretest Externalizing scores, F (1, 80) = 6.11, p = .016, and posttest Externalizing scores, F (1, 80) = 6.10, p = .016, indicating that the variance in scores was not equally distributed across levels of target child's age for these levels of the dependent variable. However, further analysis of group variances for pretest and posttest scores on the Internalizing and Externalizing scales across levels of target childs age revealed acceptable levels of variance. The rati o of the largest (pretest Internalizing scale for target children under 36 months, 12.9522 = 167.757) to smallest (pretest Internalizing scale for target children under 36 months, 9.4232 = 88.800) group varian ce was less than 3:1 (167.757 / 88.800 = 1.889). B ecause ANOVA is robust agai nst small to moderate disproportionate levels of variance, analys es were conducted as planned (Moore, 1995). The sphericity assumption was not applicable as there were only two levels of the withinsubjects variable. As shown in Table 30, results revealed a non-significant age by time by scale interaction effect F (1,80) = 0.02, p = .891, a non-significant age by time interaction effect F (1,80) = 2.35, p = .130, a non-significant age by scale interaction effect F (1,80) = 0.00, p = .981, a non-significant main effect for target child's age, F (1,80) = 0.24, p = .629, a statistically significant main effect for time, F (1, 80) = 31.17, p<.001, 2 = 0.32, and a statistically significant ma in effect for scale, F (1, 80) = 22.12, p<.001, 2 = 0.22.

PAGE 136

124 Table 30 Analysis of Variance of CB CL Preand Posttest Scores by Target Child's Age Source df SS MS F Between subjects 81 28011.15 Age (A) 1 82.27 82.27 0.24 Residual between 80 27928.88 349.11 Within subjects 85 4812.40 Time (T) 1 1891.18 1891.18 31.17* A x T interaction 1 116.21 116.21 2.35 Scale (S) 1 1538.33 1538.33 22.12* A x S interaction 1 0.04 0.04 0.00 A x T x S interaction 1 0.30 0.30 0.02 Residual within 80 1266.34 15.83 Total 83 32823.55 Note *p < .01 The non-significant interacti on effects for each combination of the variables time of testing, scale of the CBCL, and target childs age, indicated that there was no difference in caregivers ratings of target childs behavior due to time of testing, the subscale of the CBCL, or target childs age. The non-significant main effect for Age (A), indicated that caregive rs perceptions of their target ch ilds behavior were the same for both scales of the CBCL across pr etest and posttest times, regardle ss of target childs age. Follow-up of the significant main effect for Scale (S), was done by examining the overall CBCL Internalizing and Externalizing mean scale scores (i.e., marginal means).

PAGE 137

125 The mean Internalizing scale score across time and target childs age ( M = 56.61) was significantly lower than the mean Externalizi ng scale score across time and target childs age (M = 60.63). This finding indicates that for both the pretest and posttest, across levels of target childs age, care givers perceived the severity of their target childs Externalizing problem behaviors was greater than the perceive d severity for Internalizing problem behaviors. On the CBCL, behavior s categorized as Ex ternalizing include attention span, hyperactivity, rule-breaking, and aggression. Behaviors categorized as Internalizing include emotiona l reactivity, anxiety/depressi on, somatic complaints (e.g., stomach aches, headaches, overly concerne d with neatness or cleanliness), and withdrawal. Follow-up of the significant main effect for Time (T), was done by examining the overall CBCL pretest and posttest mean scores (i.e., marginal means). The mean posttest score ( M = 56.30) was significantly lower than the mean pretest score ( M = 60.94). This finding indicates that across bot h scales of the CBCL, regard less of target childs age, caregivers perceived severity of childrens problem behavior was greater at pretest time as compared to posttest time. On the CBCL, high er scores indicate more severe levels of problem behavior; therefore, a decrease in scores from pretest to posttest indicates caregivers perceived their target child to have significantly less severe levels of problem behavior following partic ipation in the program. Caregivers Overall Perceptions of the HOT DOCS Program Research Question #4. What are caregivers overa ll perceptions of the HOT DOCS parent training program?

PAGE 138

126 Caregivers ratings of sa tisfaction with the HOT DOCS program were analyzed using descriptive statistics. A total of 262 caregivers completed the Program Evaluation Survey. As shown in Table 31, the overal l majority of par ticipants (98.9%) Agreed or Strongly Agreed that the HOT DOCS program met thei r expectations. Mo re specifically, participants Agreed or Strongly Agreed that the program was beneficial to their families (100%), the trainers were knowledgeable a nd effective instructor s (100%), caregivers were able to utilize the stra tegies with their children ( 99.6%), the Parenting Tips were beneficial (100%), the Special Play strate gies promoted positive interactions with children (98.9%), the information learned in HOT DOCS changed caregivers parenting practices (98.9%), and that the program positively impacted childrens behavior (97.7%). Of the eight statements used to gauge pa rticipants perceptions of the usefulness of the program, only two statements were marked as Strongly Disagree by one participant each. In general, these two statements related to the caregivers ability to change their parenting practices using the strategies taught and changes in ch ildrens behavior at home. These data indicate that for one car egiver, this level of intervention was not matched appropriately to the level of severity of problem behavior the child demonstrated in the home. The highest percentage of res ponses endorsed by caregiv ers as being in the Disagree or Strongly Disagree categories were on items related to caregivers ability to effectively implement program strategies in the home and the s ubsequent lack of improvement in child behavior follo wing participation in the program.

PAGE 139

127 Table 31 Ratings of Participant Satisfaction with the HOT DOCS Training Program Strongly Agree Agree Disagree Strongly Disagree n % n % n % n % The HOT DOCS program was beneficial to my family 218 83 44 17 0 0 0 0 The presenter(s) were knowledgeable and effective in communicating this topic 241 92 21 8 0 0 0 0 I am able to utilize these strategies with my children 216 82 45 17 1 <1 0 0 The Parenting Tips are beneficial to me 221 84 41 16 0 0 0 0 The Special Play Activities promoted interactions with my child 188 72 71 27 3 1 0 0 The information I learned in HOT DOCS has changed my parenting practices 185 71 74 28 2 1 1 <1 HOT DOCS strategies have positively impacted my childs behavior 176 67 80 31 5 2 1 <1 Overall, the HOT DOCS program met my expectations 219 84 40 15 3 1 0 0 Note. n = 262

PAGE 140

128 Question #9 on the survey asked participants, How are you using the information you learned in HOT DOCS? and provided eigh t options for participants to endorse. Participants were directed to check all boxe s that applied. The response options for all items on the survey were derived from a thema tic analysis of free re sponses to the same questions provided by participants in a pilot study conducted with previous HOT DOCS cohorts. Response choices for Question #9 includ ed use of Parenting Tips, use of Special Play Activities, use of preven tion strategies, use of the pr oblem-solving chart, sharing information with others, improvements in daily interactions or relationships, change in parenting attitude, and Other. As shown in Table 32, the majority of participants (96%, n = 248) endorsed the response option, using a specific skill. Other response options showed similar endorsement rates, 90 % of participants (n = 232) endorsed improve d daily interactions or relationships, 88% ( n = 227) endorsed shared information with others, 86% ( n = 223) endorsed used preventi on strategies, and 81% ( n = 81) endorsed change in parenting attitude. For caregivers who e ndorsed Other, specif ic verbatim responses included, I have learned that I, as a mother, have to organize myself so I can be an example for my kids; It helped my spouse and I be on the same page since we both attended the class together; and I understand now how I was reinforcing behavior negatively through avoidance so I have been able to focus on problem-solving to predict and prevent.

PAGE 141

129 Table 32 How are you using the informati on you learned in HOT DOCS? Response Options n % Use of Parenting Tips 248 96.1 Improvements in daily interac tions or relationships 232 89.9 Sharing information with others 227 88.0 Use of prevention strategies 223 86.4 Change in parenting attitude 210 81.4 Use of Special Play Activities 203 78.7 Use of the problem-solving chart 171 66.3 Other 50 19.4 Note. n = 258 To respond to question #10 on the survey, Have you shared information from HOT DOCS with ? participants were instructed to check the boxes of all the people with whom they had shared information. As shown in Table 33, approximately 79% of participants ( n = 203) indicated that they had shared information with family members or relatives; 78% of participants ( n = 201) indicated that they had shared information with friends; 72% of participants ( n = 184) indicated that they had shared information with their spouse or partner. Approxi mately 39% of participants ( n = 99) indicated sharing information with a professional, such as an early interventio nist, therapist, or teacher while only 13% of participants ( n = 33) reported sharing information with their pediatrician. Approximately 12% of participants ( n = 30) reported sharing information with other people, specifically indicating th at they had shared information with patients

PAGE 142

130 or clients ( n = 8) or co-workers ( n = 8). Verbatim responses from participants who indicated sharing information with other s also included, Everyone! I carry HOT DOCS fliers everywhere I go! and People I have seen w ith children with behavior problems. Less than 1% of participants ( n = 2) indicated that th ey had not yet shared information with others. Table 33 Have you shared information from HOT DOCS with? Response Options n % Other family members or relatives 203 79.0 Friends 201 78.2 Spouse or Partner 184 71.6 Interventionist/Therapist/Teacher 99 38.5 Pediatrician 33 12.8 Other 30 11.7 Have not shared information 2 0.8 Note. n = 257 Question #11 on the survey, asked participants, What can we do to improve HOT DOCS? Responses options provided included nothing, the program is fine as it is; more time for instruction; offer classes in alternate locations; and other. As shown in Table 34, nearly 50% of participants ( n = 113) indicated that no improvements can or should be made to HOT DOCS and that the pr ogram was fine as it was. Approximately 30% of participants (n = 69) indicated that the program could be improved by offering classes in alternate locations while 29% of participants ( n = 67) indicated that the

PAGE 143

131 program would benefit from increasing the time for training and instruction. Approximately 22% of participants (n = 51) indicated other and offered such suggestions for improvement as offering cl asses at different ti mes of day, on the weekends, or in the summer; specifying training and instruction by childs age, disability, or severity of behavioral problems; and allowing more time for discussion between participants and for instructors to addr ess individual behaviors and concerns. One participant responded, I would have liked to sh are pictures of kids earlier in the classes because you get to hear so much about them. Table 34 What can we do to improve HOT DOCS? Response Options n % Nothing, fine as is 113 49.6 Offer classes in alternate locations 69 30.3 Increase time for instruction 67 29.4 Other 51 22.4 Note. n = 228 Question #12 on the survey (n = 235), asked What did you value most about taking the HOT DOCS class? Response options included learning specific parenting skills, support and inte raction with other caregivers in similar situations, provision of materials without cost, learning skills to pr oblem-solve childs chal lenging behavior, and other. As shown in Table 35, the majority of the participants ( n = 224) who responded to this item (95%) indicated that they va lued specific skills they acquired the most. Approximately 83% of participants ( n = 195) indicated that they valued problem-solving

PAGE 144

132 skills; 68% of participants (n = 160) indicated that they valued support and interaction with other caregivers; a nd 66% of participants ( n = 154) indicated that they valued the provision of materials (e.g., t oys, manuals, timers). Approxima tely 13% of pa rticipants (n = 31) indicated other aspects they valued most, including provision of food, knowledge and compassion from instructors, a nd decreased stress and frustrations with an increase in hope. Verbatim responses provided by these participants included, It helped me to see I need to choose my battles, It was nice to be reminded to stop and take 5 minutes to focus on your child. It's easy to forget when you get busy! I was raised by old school parents and I needed an alternative in handling bad behavior, I learned that even though I was fixing the be havior at that moment I wasn't fixing the problem, and I got a sense of temporacy with behaviors, I know they will be resolved now. Table 35 What did you value most about taking the HOT DOCS class? Response Options n % Learning specific pare nting skills 224 95.3 Learning skills to problem-solve childs challenging behavior 195 83.0 Support and interaction with other caregivers 160 68.1 Provision of materials without cost 154 65.5 Other 31 13.2 Note. n = 235

PAGE 145

133 Summary Results of this study revealed a significan t and meaningful incr ease in caregivers knowledge of child development, behavioral principles, and parenting strategies as a result of participation in HOT DOCS. Statistically significan t differences in changes in participant knowledge between groups of caregivers based on caregivers level of education, caregivers social support, target childs age, or target childs preexisting diagnosis were not observed fo r this participant sample. Data analyses suggested that prio r to beginning HOT DOCS, caregivers perceptions of the severity of their target childs pr oblem behaviors significantly exceeded what was expected in a normative population for both internalizing and externalizing behaviors. Results did not indicate differential perceptions of severity of target childs behavior based on caregivers le vel of education, caregiv ers level of social support, or target childs preexisting diagnosis. Results indicated that upon completion of the HOT DOCS program, caregivers reported a significant and meaningful reduction in their perceptions of the severity of their target childs challenging behaviors as compared to perceptions prior to beginning the program. No significant differences we re found in the cha nges in caregivers perceptions based on caregivers level of social support, target childs preexisting diagnosis, or target childs age. Participants indicated high levels of satisfaction with the HOT DOCS program, with the majority of participants indicating that the program met their expectations and provided useful skills and tec hniques. Most participants reported that they are using the specific parenting skills and problem solving techniques taught in the program; that they

PAGE 146

134 have shared the information they learned w ith family, friends, and/or co-workers; that they desired more time for in struction (e.g., more sessions, lo nger classes); and that they valued the knowledge and support from inst ructors and were encouraged by other caregivers struggling with situ ations similar to their own.

PAGE 147

135 CHAPTER 5 Discussion Overview In response to the increasing number of young children displaying early-emerging challenging behavior, professiona ls have increased their efforts to find evidence-based interventions to address child and caregiver needs. The current study served as an investigation of caregivers perceptions of the effectivenes s of the Helping Our Toddlers Developing Our Childrens Skills (HOT DOCS ) parent training program in addressing challenging behaviors in young children. This study evaluated the impact of specific components of the parent training progra m on caregivers knowledge, attitudes, and perceptions of their target childrens behavior. The study also investigated differential outcomes for various groups of participants based on specific demographic variables, including caregivers level of education and social support and target childrens age and preexisting diagnosis. This chapter begins with a discussion of the unique demographics of this studys participant sample and a comparison of th e sample demographics with the local population as well as with previous studies of established parent training programs. Following the examination of the participant sa mples characteristics, a discussion of the significance of the results of analyses for each research question is presented. The chapter

PAGE 148

136 will conclude with statements regarding implications for practitioners, limitations of the study, and directions for future research. Demographic Characteristics Rates and Patterns of Caregiver Attendance Rates and patterns of caregiver attend ance and attrition were analyzed and compared with findings from previous studi es of group-delivered behavioral parent training. Overall patterns of attendance and rates of attrition found in this study were similar to those found in previ ous research (Eyberg et al., 200 1; Feinfield & Baker, 2004; Kazdin, 1997; Sanders et al., 2000) and in the pilot study of HOT DOCS (Williams, 2007). Of the 465 caregivers attending the first session of HOT DOCS training, 334 caregivers completed the program (e.g., attended three or more sessions and signed IRB), resulting in an overall attriti on rate of 28%. In the pilot st udy of HOT DOCS, the rate of attrition was approximately 23% (Williams, 2007). Eyberg and colleagues (2001) reported similar rates of attrition in an eval uation of the Parent-Child Interaction Therapy (PCIT) intervention. Specifically, of the orig inal twenty participants, 13 completed the training, resulting in a 30% attrition rate. Fienfield and Baker (2004) reported lower levels of attrition in an evaluation of a mu ltimodal, manually guided group treatment for parents of children with challenging behavi or. Of the 56 caregivers enrolled in the program, four dropped out of the treatment group and five dropped out of the waitlist control group, resulting in an overall attrition rate of 16%. Several previous studies of group parent training interventions have reported significantly lower attriti on rates than found in this study (B arkley et al., 2000; Reid et al., 2001; Webster-Stratton & Hammond, 1997). Several of the programs reporting low

PAGE 149

137 rates of caregiver drop -out have provided participants with incentives for attendance and completion of the program. For example, in an evaluation of the Incredible Years parent training program, Reid, Webste r-Stratton, and Beauchaine (2001) reported attrition rates of less than 10%. Parents participating in this study were given $50 for participation in each pre-, post-, and follow-up assess ment. Other training programs offered individualized, child-focused in tervention services to program completers (Barkley et al., 2000; Sanders, 1999), which seemed to serve as an additional incentiv e for attendance. Several unique features of the HOT DO CS program may influence the rates and patterns of attendance for this specific pr ogram and a better u nderstanding of these features may be useful to program develope rs in decreasing rates of attrition. Although the rate of attrition for HOT DOCS (28%) is similar to previously published research on group-delivered parent training (16-30%; Eyberg et al., 2001; Fienfield & Baker, 2004; Williams, 2007), from a clinical or practical viewpoint, the program developers will likely want to maximize participant attendan ce and use of resources, meaning decreasing the drop-out rate. The first of these unique features is that the program is provided to caregivers free of charge. Caregivers may view the program as less important than similar programs or therapies offered for a fee or charged to insurance. Another feature, which may contribute to participant drop-out, is that the program is designed for caregivers of typically developing children displaying challenging be haviors. The program may be viewed by caregivers of children with severe medical, genetic, developmenta l, or behavioral disorders or syndromes as too basic or simplis tic to meet their child s complex social and behavioral needs. Conversely, because there are no inclusion or exclusion criteria for

PAGE 150

138 participation in HOT DOCS (e.g., preexisting diagnosis, clinically significant scores on standardized instruments), each class contains caregivers of children w ith a wide range of disorders, symptoms, and severities of ch allenging behaviors. Those caregivers who are attending in order to address or prevent low levels of cha llenging behaviors in children without preexisting diagnoses may drop-out af ter hearing other care givers describe the severe and complex needs of their children who do have preexisting medical or behavioral diagnoses. As with other parent training programs (Bor, Sanders, & MarkieDadds, 2002; Eyberg et al., 2001; WebsterStratton, 1998), caregivers who drop-out of the HOT DOCS program may do so due to pr essure from everyday stressors, including difficulties with childcare, transportation, and scheduling. Finally, caregivers dropping out of the program before completion may be facilitated by the provision of the full HOT DOCS manual to caregivers during the first session. Caregivers, who are struggling with the stressors mentione d previously, may believe that th ey can simply read through the entire manual on their own and do not need to continue attending weekly sessions. Future studies of the HOT DOCS program should in clude follow-up phone calls or focus groups with caregivers who do not comple te the program in order to identify the barriers these caregivers experience. In addition to the factors discussed previ ously, which may contribute to caregiver attrition, there are several features of HOT DOCS, which may help protect against participant drop-out. First, HOT DOCS activel y encourages and recr uits participation from male caregivers by using examples and videos in the training materials featuring male caregivers and by including male trai ners to deliver the HOT DOCS program. Previous studies have demons trated that involving fathers in any treatment plan or

PAGE 151

139 program results in decreased attrition and be tter long-term outcomes than plans or programs focusing only on mothers (Tiano & McNeil, 2005). In addition to actively recruiting male caregivers, HOT DOCS encourages the attendance of multiple caregivers for each target child, stressi ng the ecological model by including extended family, friends, and other support systems in an effort to increase motivation and encouragement for caregivers to attend weekly sessions. Unlike existing pa rent training programs, HOT DOCS uses the same curriculum and materials to train child care prof essionals as it does to train caregivers. Th is feature allows caregivers to at tend parent training sessions with their early interventionists, therapists, or case workers, thereby providing an additional support network for caregivers and encouragin g consistency of skills and techniques across all adults involved in the target child's life. Another feature that may help decrease drop-out is the comparatively short duration of the HOT DOCS program. Existi ng behavioral parent training programs typically require caregivers to attend between 12 and 36 se ssions (Barkley et al., 2000; Lundahl, Risser, & Lovejoy, 2006; Webster-S tratton & Taylor, 2001), while HOT DOCS consists of 6 weekly sessions and a tw o-month follow-up booster session (Armstrong, Lilly & Curtiss, 2006). Although previously discussed as a possible explanation for increased caregiver drop-out, providing the HOT DOCS program free of charge to all participants has made the program available to a large proportion of low SES families in the community. Funding provided by the Childrens Board of Hillsborough County covers the cost of the traine rs fees, materials, supplies and food for participants, as well as participant registration fees removing the financial barriers to participation in parent training programs identified by previous research (Barkley et al ., 2000; Webster-Stratton

PAGE 152

140 & Taylor, 2001). By eliminating registration fees for families, access increases for all families who are willing and able to participate. Finally, another feature uni que to the HOT DOCS program is that this program has not only been translated to Spanish to address language barrier s of Hispanic/Latino caregivers, HOT DOCS was also culturally adapted to the Hispanic/Latino community (Curtiss et al., 2009). As identified in previ ous research, training pr ovided in Spanish by native Spanish trainers makes Hispanic/La tino families feel more accepted, valued, and understood (Forehand & Kotchick, 1996; Smag ner & Sullivan, 2005), which in turn may decrease the likelihood of caregiver drop-out prior to program completion. Research has also suggested that matching therapist-clie nt cultural values has a greater impact on program completion and treatment outcomes th an matching therapist-client language or ethnicity alone (Cabrera et al., 2002 ; Lewis & O'Brien, 1987; Meyers, 1993). Comparison of Caregiver Demographics with the Local Community and Previous Studies Demographic information for the caregiver s serving as particip ants in this study and their target children were compared w ith local demographic information provided by the United States Census Bureau for Hillsborough County through the 2005-2007 American Community Survey (United States Census Bureau, http://factfinder.census.gov, retrieved May 23, 2009) and with demographic information for participant samples from previous research of groupdelivered parent tr aining programs. Significant differences were identified between the HOT DOCS participant sample, the local community, and previous parent training participant samples in terms of participant gender/relationship to target child, race/ethnicity, level of education, type of insurance, which is used as a rough

PAGE 153

141 estimate of socio-economic status (SES), careg ivers social support, and target childs preexisting diagnoses. Caregivers Gender/Relationship to Target Child Most of the existing research on parent training programs ha s focused on female caregivers, specifically mothers of children with problem behavior (Bagne r & Eyberg, 2003; McNeill, Watson, Hennington, & Meeks, 2002; Phares, Fields, Kamboukos, & Lopez, 2005; Reid et al., 2001; Singer, Ethridge, & Aldana, 2007). The ge nder and relationship with ta rget child of participants in this study differs notably from previous research on parent training interventions, specifically by encouraging par ticipation of fathers, non-re lated caregivers, and child service professionals. Participants in th e sample were 74% female and 26% male, including 86% biological, adoptive, or fost er parents, 10% professionals (i.e., early interventionists, service c oordinators, pediatric and psychiatry residents), 4% grandparents, and 1% other relatives. The pr oportion of male to female participants in this study was similar to the results of the pilot study, which reported 68% female participants and 32% male participants (Williams, 2007). In comparison, research on the Incredible Years parent training program indicates the majority of participants were mothers (98-100% mothers, small number of grandmother/aunt and fathers) including three studies with 100% female participants (Hartman et al., 2003; Reid, Webster-Stratt on, & Baydar, 2004; Reid et al., 2001). One study on the Incredible Years program reporte d a significant proportion (43%) of fathers as participants (Webster-S tratton & Hammond, 1997). Research on the effectiveness of the PCIT parent training inte rvention has been conducted mainly with mothers or other female caregivers (Boggs et al., 2004; Hood & Eyberg, 2003) with the exception of a

PAGE 154

142 study specifically designed to target fathers participati on in PCIT (Bagner & Eyberg, 2003). The majority of studies targeting ch ildren with ADHD have not reported data specifying the gender of parents and careg ivers participating in training programs (Barkley et al., 2000; Weinbe rg, 1999). However, one study of the Defiant Children Parenting Program reported 100% of particip ants being mothers (A nastopoulos et al., 1993). In contrast to the majority of studies of behavioral parent training including the current study, investigations of the Triple P-Positive Parenting Program (Sanders, 1999) have reported participation by both parents of target children (Bor, Sanders, & MarkieDadds, 2002; Sanders et al., 2000). The inclusion of a notably high proporti on of male caregivers in HOT DOCS is significant for a number of reasons demonstrated in recent research on the importance of including fathers in behavior parent training (Lamb, 1997; Lundahl, Tollefson, Risser, & Lovejoy, 2008; Palkovitz, 1996; Parke & Brott, 1999). Results of st udies investigating the effects of father involvement generall y agree that while participation of male caregivers does not necessarily predict cons iderably better outcomes for children and families, when fathers are involved attrition rates are lower, maintenance of treatment gains persisted longer, and perceived pare ntal competence was higher, compared to families without father participation (Lundahl, Tollefson, Risser, & Lovejoy, 2008). Other research has demonstrated that involving both parents in behavioral parent training enhances success following program completion by encouraging consistency in parenting skills across mothers and fathers (Tiano & McNeil, 2005). Caregivers Race/Ethnicity. As compared to adults residing in Hillsborough County in 2005-2007, the participant sample in this study consisted of 11% fewer

PAGE 155

143 caregivers reporting their ethnicity as Cau casian (47% in HOT DOCS versus 58% in Hillsborough county), 19% more caregivers repo rting their ethnicity as Hispanic (41% HOT DOCS versus 22% Hillsborough county), and 9% fewer caregivers reporting their ethnicity as Black/African American (6% HOT DOCS versus 15% Hillsborough county). The participant sample is proportionate to Hillsborough county demographics for the percent of caregivers reporting their race/ethnicity as Asian (2% HOT DOCS versus 3% Hillsborough county). These results suggest th at the HOT DOCS program provided early intervention services to caregivers fr om a racial/ethnic group, which has been underserved by previous parenting programs, specifically Hispanic and/or Spanishspeaking caregivers (Hershcell, Calzada, Eyberg, & McNeil, 2002). However, these results also suggest a disproportionately lo w percentage of Black/African American caregivers participating in the HOT DOC S program. The underrepresentation of Black/African American caregivers in the HOT DOCS program is likely related to the lack of families from this race/ethnic category that self-refer and/or are referred by professionals to participate. The high percen tage of sample participants reporting their race/ethnicity as Hispanic as compared to local norms is likely explained by the provision of the culturally adapted HOT DOCS classes delivered in Spanish. Participant race/ethnicity for this study also was compared with demographic information from other parent training programs, including the Incredible Years (Webster-Stratton, 2001), PCIT (Eyberg, 1988) Triple P-Positive Parenting Program (Sanders, 1996), Defiant Children Parenting Program (Barkley et al., 2000) and others. Compared with participants completing other training programs, the pa rticipant sample of caregivers completing HOT DOCS was com posed of fewer White caregivers (47%

PAGE 156

144 versus an average of 51-98%) and more non-White caregivers (51% versus an average of 2-49%) (Barkley et al., 2000; Fienfield & Baker, 2004; Sanders et al., 2004). Specifically, the HOT DOCS participant sample included nearly eight times the percentage of Hispanic caregivers (41%) as previous studi es (generally about 5% across parenting programs). Similar to findings from the curr ent study, one previous study of PCIT had a notably larger percentage of Hispanic participants compared to the majority of existing parenting research (McCabe, Yeh, Garla nd, Lau, & Chavez, 2005). Just as HOT DOCS was translated to Spanish to increase Hispanic caregiver participation, McCabe and colleagues (2005) modified and translated the original PCIT program to meet the unique needs of Mexican-American families. However, as previously discussed, HOT DOCS was not only translated to Spanish, but was cult urally adapted to meet more than just the language needs of the Hispanic/Lati no community (Curtiss et al., 2009). Percentages of various racial/ethnic groups found in the current study is similar to those found in the pilot study of HOT DOCS, which also identified disproportionately low representation of African American/Black participants and disproportionately high representation of White and Hi spanic when compared to demographic data for the local community at that time (Williams, 2007). Alt hough initially discouraging, the findings in this study identifying the underrepresentation of Black/African American caregivers in the HOT DOCS program provide practitioners with a specific target for recruiting participants for future HOT DOCS trai nings. This might be accomplished through increased advertising and recru itment directly targeted at re aching this racial/ethnic group as well as through making adjustments in schedul ing of future classes, such as offering

PAGE 157

145 the trainings at locations within the Black/African American communities, or offering classes taught by African Amer ican/Black instructors. Caregivers Level of Education. When compared to adults residing in Hillsborough county, according to census data from 2005-2007, the participant sample consisted of a similar percent of participants reporting their highest level of education as a high school diploma (20% HOT DOCS vers us 29% Hillsborough county) and as a twoyear college degree (12% HOT DOCS vers us 10% Hillsborough county). However, the participant sample consisted of 11% fewer careg ivers reporting their le vel of education as less than high school (4% HOT DOCS versus 15% in Hillsborough county), 10% more caregivers reporting their highest level of e ducation as a four-year college degree (29% HOT DOCS versus 19% in Hillsborough county), and 15% more caregivers reporting their highest level of education as a gra duate degree (24% HOT DOCS versus 9% in Hillsborough county). The disproportionality be tween sample participants and the local community was also observed in the pilot study of HOT DOCS (Williams, 2007). In general, participants in the HOT DOC S program have a disproportionately high level of education, which may be explai ned by the program's affiliation with the University of South Florida (USF) and surroundi ng medical facilities Many participants in the program are students or faculty at USF or physicians in local clinics or hospitals. The unexpectedly high mean level of educa tion of HOT DOCS participants may be explained by the preventative nature of the program. As previously discussed, HOT DOCS is designed to address ch allenging behaviors in typically developing children. This program, unlike existing parenting programs (e.g., Incredible Years, Triple-P, PCIT), does not focus recruiting and advertising effo rts entirely on caregivers with identified

PAGE 158

146 risk-factors, such as low e ducation levels, low SES, and/or minority groups. However, this may be a finding that encourages the HOT DOCS program developers to actively seek out caregivers from these especia lly needy populations (e.g., low-income, low educational attainment, minority group identi fication) to participate in the HOT DOCS program. Previous studies of parenting programs have reported similar patterns of higher than expected educational attainment (F ienfield & Baker, 2004; Hartman, Stage, & Webster-Stratton, 2003). These studies ha ve hypothesized that the higher mean educational levels may be explained by th e additional financial and social supports available to families with higher levels of educational attainment. Researchers have suggested that these resources allow parents to participat e in and complete training programs, while parents with lower educationa l attainment are ofte n unable to attend and complete training sessions due to issues asso ciated with socioeconomic status, such as lack of transportati on, childcare, and time. As previ ously discussed, the HOT DOCS program has implemented measures to prev ent financial and social barriers from preventing at-risk families from accessing the HOT DOCS program, such as offering the program free of charge, offering classes on even ings, and encouraging caregivers to bring supporting adults with th em to the classes. Type of Insurance. The use of type of insurance as an indicator for socioeconomic status (SES) in this study pr ohibits precise comparisons wi th local population statistics, which report SES using ranges of annual household income, often in combination with other social or educational variables (Reyno & McGrath, 2006). However, general comparisons of the proportion of the study sa mple reporting having Medicaid insurance

PAGE 159

147 or no insurance, which were response categories used by the program developers to indicate lower SES, were compared with Hillsborough County estimates of families with children under 18 years of ag e with household incomes fa lling below the poverty level (United States Census Bureau, http:/ /factfinder.census.gov, retrieved May 23, 2009). Approximately one-third (34%) of HOT DOCS participants reporte d having no insurance or Medicaid insurance compared to 14% of families in Hillsborough County classified as being below the poverty level. This compar ison indicates that the HOT DOCS parent training program was delivered to a higher pe rcentage of low-SES families than would have occurred simply by chance. Since previ ous research has show n that children of parents who are considered low-SES or low-in come have a greater chance of developing more severe levels of challenging behavi or (Gross et al., 1999 ; Keenan & Wakschlag, 2000; Qi & Kaiser, 2003), the large proportion of participants falling within this category can be considered a positive finding. Percen tages of the participant sample reporting having Medicaid or no insurance were similar to participant demographics in the pilot study of HOT DOCS (Williams, 2007). Caregivers Social Support. As with type of insurance, the use of whether caregivers attended HOT DOCS with another caregiver or attended alone as an indicator for caregivers' social support in this study prohibits precise comp arisons with local population statistics, which repor t social support using single-p arent status as heads of households. However, general comparisons betw een the participant sample and data from the local community indicated that a lesser pr oportion of caregivers attended HOT DOCS with another caregiver than the percentage of married families in the local community. Specifically, 49% of participants attended with another caregiver, while data for

PAGE 160

148 Hillsborough County indicated that 65% of fa milies with children under 18 years of age are two-parent households and another 16% of families have grandparents who live in the household (United States Census Bureau, http://factfinder.cens us.gov, retrieved May 23, 2009). These comparisons indicate that while approximately 81% of local families have multiple adults caring for children in the home, only 49% of caregivers in the participant sample attended HOT DOCS with another caregiver. This discrepancy may be explained by the lack of childcare provided by the HOT DOCS program, which requires that caregiver s arrange for childcare during the seven class sessions. As evidenced by particip ant responses on the HOT DOCS Program Evaluation Survey found in this study a nd in the pilot study (Williams, 2007), many caregivers have indicated that a major im provement to the HOT DOCS program would be to offer childcare during class sessions. Availability of childcare has also been indicated as a barrier to participation in pa rent training programs in previous research (Lundahl, Risser, & Lovejoy, 2006; Maughan et al., 2005). The pr oportionately low number of participants atte nding HOT DOCS without another caregiver as their support, despite the high proportion of local families w ith multiple caregivers in the home, may be occurring because one of the caregivers must stay home to watch the children while the other attends the parent training classes. If it is a goal of the program to increase the number of caregivers who attend with ot her adults as social support, HOT DOCS developers may need to consider ways to overcome the barrier of childcare. Possible strategies would be to provide childcare at the training facility dur ing class time or to provide caregivers with resour ces to find and pay for childca re services outside of the training sessions.

PAGE 161

149 Target Child's Preexisting Diagnosis. Preexisting diagnoses of target children of participants in this study also were co mpared with demographic information from previous research. A large percen tage (41%) of target children in this study did not have a preexisting medical, psychologica l, or behavioral diagnosis as reported by caregiver participants at the time of participation. In contrast, the majo rity of previous studies of parent training programs have specified inclusi on criteria requiring th at target children have preexisting mental, emotional or behavi oral diagnoses to pa rticipate in study. Few published, evidence-based interventions target pa rents of children with non-clinical levels of challenging behavior (Lunda hl et al., 2006; Maughan et al., 2005; Schumann et al., 1998). Several investigations of the Incredible Years parent training program and several studies of PCIT specify that children must have preexisting diagnosis of Oppositional Defiant Disorder (ODD) and/or Conduct Di sorder (CD) (Harman, Stage, & WebsterStratton, 2003; Webster-Stratton & Hammond; 1997). Parent training research conducted by Barkley and colleagues (2000) stipulated that a ll children included in the studies met diagnostic criteria for ADHD. These findings indicate that the HOT DOCS parent training program provided early intervention services as preventative measures for children exhibiting non-clinical levels of challenging behaviors. As indicated by several decades of research, intervention provided before challenging behaviors reach ch ronic and severe levels is more likely to effectively treat and prevent negative lif elong emotional and behavioral impact (Marchant et al., 2004; Walker et al., 1998; Webster-Stratton, 1998). Although the target children of participants in this sample ha d fewer preexisting diagnoses than previous research on other parent trai ning programs, this study had a much lower percentage of

PAGE 162

150 target children without a preexisting diagnos is than found in the pilot study of HOT DOCS. In the pilot study, the majority of targ et children (66%) did not have a preexisting diagnosis (Williams, 2007) compared to 41% of target children in the current study having a preexisting diagnosis. Although the per centage of target ch ildren with autism spectrum disorder diagnoses did not differ between the two studies of HOT DOCS (12% in the current study vs. 14% in the pilot study ), the current study consisted of a higher percentage of children with speech-language impairments (12% current study vs. 5% pilot study) and developmental delays (18% current study vs. 4% pilot study) than found in the pilot study. The in crease in the percentage of target childr en have preexisting diagnoses may be due to more referrals to the HOT DOCS program from child service professionals (e.g., pediatricians, interventionists, therapists) who have participated in the HOT DOCS program and are currently treat ing children with diagnosed conditions. The participant sample for this study represented previously underserved or understudied portions of the population. Speci fically, this study included higher than expected numbers of male care givers, Hispanic/Latino caregive rs, caregivers with higher than average educational attainment, careg ivers with low socio-economic status, and caregivers of children without preexisting diagno sis. Therefore, the implications of the unique demographic characteristics of the partic ipant sample will be related to the results of the specific research questions presented below when applicable. Caregiver Knowledge Research Question 1a. What is the impact on care giver knowledge of child development, behavioral principles, and parentin g strategies as a result of participation in the HOT DOCS parent training program?

PAGE 163

151 Results of this study indicated a significant increase in participants scores on the HOT DOCS Knowledge Test from pretest to posttest. Although the difference in mean score from pretest to posttest differed by fewe r than two correct answers, the effect size of the statistical difference was large, indi cating significant and mean ingful increases in the number of correct answers provided by pa rticipants. The items used on the test represent specific concepts, skills, or practices guided by the theoreti cal framework of the HOT DOCS parent training program. Knowing and understanding these skills and concepts may be considered ideal outcomes of the parent training program. Therefore, an increase in the number of items correct may be cautiously interpreted as an indicator of successful delivery of the specific skills and concepts. Changes in caregiver knowledge as i ndicated by these results are similar to outcomes reported in previous research of parent training interven tions (Anastopoulos et al., 1993; Weinberg, 1999). Anastopoulos and colleagues (1993) id entified changes in parent knowledge as a dependent variable in their investigation of a six-week parent training program for parents of children w ith ADHD. Results of their study reported significant increases in parent knowledge from preto posttest us ing a knowledge test created by the researchers specifically for this purpose and meas ured by counting the number of items correct on th e test. Weinberg (1999) also reported significant increases in parent knowledge of the features of ADHD and behavioral ma nagement strategies following participation in a behavioral parent training program. However, in this study, changes in parent knowledge were measured using parent ratings of their own knowledge of ADHD on a 7-point Likert-t ype scale (e.g., "very little" to "very comprehensive"). This measure of parent knowledge would be more accurately defined as parent

PAGE 164

152 perceptions of their knowle dge gains, rather than actual new information acquired through participation in the training program. The statistically significant results f ound in this study have several significant limitations to interpretation. Pa rticipants answered approximately 16 of the 20 questions correctly before beginning the HOT DOCS training (mean pretest score = 15.88). Given the high score at pretest, results indicated that the test itself was fl awed in its design. The 20 questions were either too easy or the Tr ue/False design allowed participants to accurately guess the correct answers (Frisbie 1974). Several features of the HOT DOCS Knowledge Test prevented furthe r interpretation of the increase in scores. Specifically, due to the small number of items on the test (e.g., 20 items), the lack of reliability and validity data available for the m easure, and the lack of variati on in response type (e.g., all true/false), further analyses were restricted. Due to these limitations, the statistically significant results found in this study should be interpreted ca utiously and used more for informing program developers about changes th at need to be made to the measurement instrument rather than used to make in ferences and draw conclusions about the effectiveness of the HOT DOC S parent training program. These results present several possible m odifications that could be made to the Knowledge Test in order to strengthen its design and therefore increase the test's usefulness in informing program evaluation research (Frisbie, 1974; Hogan & Murphy, 2007; Smith, 2006). First, the items could be reformatted into multiple choice questions, which might prevent participants from simply guessing answers for each item. Second, there should be more questions per topic area in orde r to allow for more in depth analysis of knowledge gains, such as a factor analys is. Adding more items per subject area would

PAGE 165

153 also allow for a check of content-related va lidity and reliability. Third, the items on the test should be rewritten or revised to be mo re challenging for participants at pretest. Before exposure to the curriculum, participants should not be able to correctly answer the majority of test items. Finally, the revise d measurement instrument should be evaluated by a panel of experts and evaluated th rough pilot testing with caregivers. Research Question 1b. Is there a difference in partic ipant caregivers performance on the HOT DOCS Knowledge Test due to pa rticipation in the HOT DOCS training program based on caregivers' level of educa tion, caregivers' social support network, the target child's age, and the target child's diagnosis? Results of this study found no differential performance of pa rticipant caregivers on the HOT DOCS Knowledge Test from preto posttest due to participant classification by specific demographic variables. The demogra phic variables targeted in this research question were caregivers le vel of education (i.e., high school diploma or less, vs. technical training/2-year colle ge degree, vs. 4-year colleg e, vs. a graduate degree); caregivers level of social support (i.e ., caregiver attendi ng training alone vs. accompanied by another caregiver(s)); target ch ilds age (i.e., target child under three years of age vs. target child over three years of ag e); and target child s diagnosis (i.e., no preexisting diagnosis, target ch ild with autism spectrum disorders, developmental delays, speech/language impairments, or medical/genet ic disorders). This finding suggests that unlike previous research (Knapp & Delu ty, 1989; Oltmanns et al., 1977; Reyno & McGrath, 2006; Strain, Young, & Horowitz 1981; Webster-Stratton & Hammond, 1990), all participants who completed the HOT DOCS parent training program appeared to

PAGE 166

154 benefit equally in terms of knowledge gains, regardless of caregivers level of education and support and target childs age and diagnosis. The lack of statistically significant di fferences found in this study should be interpreted cautiously due to the small sample size, which was further impacted by unequal numbers of participants within each level of the demographic variables. The unequal distribution of participants in each le vel of the caregivers level of education variable required that the four lowest levels of educational attainment be collapsed into two levels (i.e., less than high school + high sc hool diploma, technical training + 2 year college degree). By collapsing these levels, data analyses were strengthened, but the practical application of investigating specific differences in participant outcomes based on individual levels of care giver education was lost. These findings should also be interpreted with caution due to the method of data collection employed by the progr am developers, especially in the definitions of the demographic variables of caregivers level of social support and target childs diagnosis. Caregivers level of social s upport was uniquely defined for th e purposes of this study in terms of participant attendance, either a ttending the program alone or with another caregiver. In the majority of literature and re search available on parent training programs, the construct of social support is most often measured by such variables as marital status (Kazdin & Wassell, 1999; Smith et al., 2005), fa ther involvement in parenting or parent training (Holden, Lavigne, & Cameron, 1990; Smith, Landry, & Swank, 2000), ratings of perceived social supports using standardized scales, and more recently, specific measures of resiliency and protective factors (Luthar, 2006; Singer et al., 2007). These differences

PAGE 167

155 in definitions and basic constructs make it difficult to compare the results of this study with the existing research base. The method of defining the variable of ta rget childs diagnosis also limits the interpretations and implications of the result s of this study. Individua l participant data for this variable were collected by the program developers based entirely upon parent or caregiver report. On the HOT DOCS Demogra phics Form for Caregivers, the item which provides data on the target childs diagnosis prom pts participants to Circle all that apply: No diagnosis, ADHD, Developmental Delay, Au tism spectrum disorder (Autism, PDD, Aspergers syndrome, etc.), Speech/Language, Other. Participants were prompted to write in other diagnoses in the space provided. Data were entered directly from the Demographics Form as written by participan ts. Diagnoses supplied by participants were not verified by medical records or formal documentation. A suggested modification to program procedures would be to design a method of verifying target childs diagnoses as provided by caregivers. In addition to the limitations produced by parent reported information for this variable, it is also difficult to make comp arisons between the findings of this study and the available research base on outcomes of pa rent training programs because of the large proportion (41%) of target children in this study whose challenging behaviors fell below the threshold required to meet formal diagnosti c criteria. As previously discussed in the section comparing demographic variables of the current sample with previous studies, the majority of behavior parent training programs require target children to have already been diagnosed with a behavioral, educational, and/or medical diagnosis prior to their caregivers entry into the training program (Barkley at al., 2000; Harman, Stage, &

PAGE 168

156 Webster-Stratton, 2003; Lundahl et al., 2006; Maughan et al., 2005; Schumann et al., 1998; Webster-Stratton & Hammond; 1997). The lack of differential outcomes for part icipants in this study based on various caregiver and child variables may be cautiously interpreted as evidence that the design of the HOT DOCS curriculum and the method of content delivery f acilitates knowledge gains for the majority of caregivers who co mplete the training program. Regardless of caregivers' level of education or social support and target child's age and diagnosis, caregivers made significant and meaningful gains in knowledge following completion of the HOT DOCS program. Caregiver Perceptions of Severity of Child Behavior Research Question 2a. Do caregivers perceive their child as having more problem behavior than a normative sample prior to participation in the HOT DOCS program? It was anticipated that participants would report high levels of perceived challenging behavior in their target children. Expectations of high levels of problem behavior were based on the method of particip ant recruitment. Care givers either selfreferred to the program after seeing commun ity advertisements or hearing about the program from friends or were referred to the program following a comprehensive psychoeducational evalua tion of their child. Previous studies of parent training programs for children with challenging behavior have used parent reported data su ch as the Child Behavior Checklist (CBCL; Achenbach, 2001) and Eyberg Child Behavior Inventory (ECBI; Eyberg & Pincus, 1999) to evaluate the effectiveness of their program s. Many studies have ci ted inclusion criteria for participation in the study, stipulating that caregivers must have children who score in

PAGE 169

157 the clinically significant range on these meas ures (Bagner & Eyberg, 2003; Barkley et al., 2000; Harman, Stage, & Webster-Stratton, 2003; Webster-Stratton & Hammond; 1997). Because many of the published studies of parent training programs have inclusion criteria such as these, the overall frequencies of caregivers reported perceptions of child behavior as being more severe and problematic is higher than expected for a normative sample of the general population. Although th e current study did not base participant inclusion on pretest behavior rating scale scores, it was hypot hesized that most of the caregivers seeking to participate in the progr am would report that their children had more severe levels of problem beha vior than a normative sample. Results of this study suppor ted this hypothesis by indi cating that participants reported significantly more severe levels of child problem behavior at pre-test than was predicted for a normative sample of the population. Statistical analys es revealed that 1116 times as many caregivers in the participan t sample perceived their childs problem behaviors to be within the clinically si gnificant range on both the Internalizing and Externalizing subscales of the CBCL than was expected given the percentages reported for the normative population (Achenbach, 2001). Th ese results indicate that the majority of caregivers who elected to participate in HOT DOCS perceived their children as having clinically significant levels of problem behavior prior to beginning the training program. This finding is important for the program deve lopers to know that th ey are recruiting the population of participants for which the curriculum was designed. Specifically, the majority of participants have target child ren who display challengi ng behaviors that are of great concern to caregivers, but do not necessarily meet thresholds to receive psychological, behavioral, or medical/genetic diagnoses.

PAGE 170

158 Research Question 2b. Are there significant differen ces in caregiver perceptions of the severity of child problem behaviors based on caregivers' level of education, caregivers' social support network, and the target ch ild's diagnosis? Results of this study indicated that caregiver s perceptions of the severity of their target childs problem behaviors did not differ depending on caregivers level of education, caregivers level of support, or target childs diagnosis. Follow-up analyses of the non-significant overall findi ngs indicated that in term s of caregivers level of education, caregivers with a graduate degree rated target children's behavior significantly lower than did car egivers with a high school diploma or less. As found in previous studies, caregivers with more re sources, including their level of educational attainment, tend to have more positive views of their children's behaviors and more positive beliefs about their ability to addre ss these behaviors (Kazdin & Wassell, 1999; Smith et al., 2005; Webster-Stratton & Hammon d, 1990). No other significant differences were found between the remaining le vels of caregivers education. Follow-up analyses also indicated that c aregivers of target children with no diagnosis rated their children's behavior as being significantly le ss severe than did caregivers of target children with a diagnosis of autism spectrum disorder. It is hypothesized that parents of children with a diagnosis of autism spectrum disorder are more aware of the severity of their child's challenging be havior because they have attended various evaluation and treatment a ppointments with child service professionals, in which these behaviors are observed and m easured in great detail (Mansell & Morris, 2004; Singer, Ethridge, & Aldana 2007). It has also been sh own that parents of children with behavioral/developmental diagnoses, su ch as autism spectrum disorder, experience

PAGE 171

159 high levels of stress and anxiety about thei r child's development (Holden et al., 1990; Schumann et al., 1998). The finding from this study that caregivers of children with autism spectrum disorders rated the severity of their children's problem behavior significantly higher than did pare nts of children without a diagnos is or with other specific diagnoses may be a reflection of this gr oup of caregivers' increased awareness of children's behavior, as well as a reflection of the increased severity of challenging behavior displayed by children with autism spectrum disorders (Briegel, Schneider & Schwab, 2008; Lardieri, Blacher, & Sw anson, 2000; Smith, Landry, & Swank, 2005). Changes in Child Problem Behavior Research Question 3a. To what extent do caregivers perceive a decrease in child problem behavior following caregiver participation in the HOT DOCS program ? Comparisons of pretest and posttest caregiver ratings of child problem behavior using the CBCL have frequently been used in research on behavioral parent training programs as indicators of program effectiven ess (Barkley et al., 2000; Cartwright-Hatton, McNally, & White, 2005; Connolly, Sharry & Fitzpatrick, 2001; Feinfield & Baker, 2004; Hartman et al., 2003; Nixon et al., 2003; Reid et al., 2001; Thompson, Ruma, Schuchmann, & Burke, 1996; Webster-Strat ton, 1998; Webster-Stratton & Hammond, 1997). Most studies presented significant decreases in the severity of child behavior from pretest to posttest as reported by caregivers. Results of this study indicate significant reductions in the severity of child problem behavior as perceived by caregivers. It could not be determined from the data available whether child behavior actually impr oved or, as suspected in previous studies, changes in scores were due to reductions in parent stress and in creases in parenting

PAGE 172

160 competency (Maughan et al., 2005; Singer, Ethridge, & Aldana, 2007; Smith, Landry, & Swank, 2005) Despite frequent use of the CBC L in behavioral pare nt training research, significant limitations have been identified by the majority of researchers using CBCL as an outcome measure. The primary limitation is that the CBCL measures child behaviors through parent report and not through direct observation (Connolly, Sharry & Fitzpatrick, 2001; Feinfield & Baker, 2004; Thompson et al., 1996; Webster-Stratton, Reid, & Hammond, 2004). Thus, pretest and posttest co mparisons might really be measuring increases in parent perceptions of competence, increases in parent percepti ons of social support or normality of child problem behavior, or decreases in parenting stress and not actual changes in child behavior. Several studies have overcome this limitation by supplementing the use of parent report ratings of child behavi or with direct observations of child behavior, which is thought to provide a more accurate measure of changes in child problem behavior by eliminating the potentially confounding self-report bias (Barkley et al., 2000; Hartman et al., 2003; Nixon et al., 2003; Reid et al., 2001) HOT DOCS program developers should consider including direct observati on of target child's behavior to provide mo re direct and valid measures of behavior change following caregiver participation in the program, which could be accomplished through indivi dual home visits including videotaped observations of caregivers' skills, children's be haviors, and caregiver-child interactions. Results should be interpreted with cauti on due to a relatively low return rate for posttest rating scales (41%). Several modi fications to HOT DOCS program procedures were enacted following the results of the p ilot study with the goal of increasing the low return rate of posttest CBCL rating scales In the pilot study, approximately 25% of

PAGE 173

161 participants who completed the CBCL rating scale at pretest also completed the CBCL rating scale at posttest. At that time, the posttest scales were mailed home to participants two months after program comp letion and prompted to return the scales in a prepaid envelope. In order to address this wea kness, the program developers created a Booster Session held two months after co mpletion of the final cla ss session. During the booster session participants returned to the training location to review the skills an d techniques taught in the class, discuss victories and ongoing challenge s in implementing specific skills at home, and to support one another in continuing to use the HOT DOCS problem solving strategies. Before leaving the booster session, all participants complete a CBCL in addition to other measures of program out comes. Following the implementation of the booster session, the return rate of CBCL rati ng scales increased nearly two-fold. The return rate for the current study (41%) is more acceptable in terms of statistical significance than the return rate for the pilo t study (<25%). Continued efforts should be made by the program developers to ensure that more participants who complete the program provide outcome data at the two-month follow-up. Research Question 3b. Are there differential perceptions of child behavior change based on caregivers' social support network, th e target child's diagnosis, and the target child's age? Results of this study indicated that careg ivers perceptions of the change in severity of their target childs problem be haviors from prior to beginning the HOT DOCS program to 2-months after completion did not differ depending on caregivers level of support, target childs diagnosis or target childs age. Fo llow-up analyses of the overall non-significant findings i ndicated that in terms of target childs diagnosis, caregivers

PAGE 174

162 whose target child did not have a preexisti ng diagnosis perceived their target childs behaviors to be less severe than did care givers whose target child had a preexisting diagnosis. As previously discussed, these re sults follow expected patterns, given that children with a diagnosis by definition have more significant and severe levels of problem behavior than do children who di splay normal levels of developmentally appropriate challenging behavi or. There were no significant differences found in this study between the changes in perceptions by car egivers depending on their level of social support or their target child's age. These results also should be interpre ted guardedly, given the low (41%) and possibly disproportionate return rate of posttest CBCL ra ting scales, as previously discussed. It is not known as a result of this study, whet her those participants who attended the booster session and completed a posttest rating scale were qualitatively and/or quantitatively different from those part icipants who did not return for the booster session nor return behavior rating scales in the mail. It is possible that the caregivers who return have target children with more inte nse challenging behaviors and therefore return to the booster session in pursuit of additional he lp and support. If this were the case, the posttest sample would be composed of caregiv ers of target children with a higher mean level of challenging behaviors than those caregivers who may have felt the booster was unnecessary because their childs behavior ha d so drastically improved. Further research is needed to answer these questions and m odifications to program procedures could be useful in increasing the return rate of posttest scales from all participants.

PAGE 175

163 Caregivers Overall Perceptions of the HOT DOCS Program Research Question 4. What are caregivers overall perceptions of the HOT DOCS parent training program as measured by the HOT DOCS Program Evaluation Survey for Caregivers? Results of a previous evaluation of pa rticipant satisfaction with the HOT DOCS program (Armstrong, Hornbeck et al., 2006) using surveys and focus groups along with the results of the pilot study of the HOT DOCS program (Williams, 2007) indicated that caregivers reported high levels of satisfaction with program. In light of these findings and modifications made to the program following suggestions from previ ous participants, it was expected that participants in the curre nt study also would report high levels of satisfaction. With few exceptions, the majority of caregivers (98%) indicated that they Agreed or Strongly Agreed that the HOT DOCS program met their expectations, was beneficial to their families, and positively im pacted their behavior as caregivers. The few statements on the survey with which caregivers Disagreed or Strongly Disagreed related to the ability to implement specific skills at home and the programs impact on child behavior. These findings are not surprising, given that many pa rent training interventions struggle with accomplishing transfer of skills taught in the classroom to the home setting (Eyberg, 1998; Sanders, 1999). In light of the overwhelmingly positive response to these items, those few participants who were not satisfied with the program were provided individual consultation and possible referrals for further assessment and treatment strategies. These results were interpreted as exceptions to a program perceived as effective, rather than proof that the program is not effective.

PAGE 176

164 The majority of caregivers reported that they were using the skills learned in the program at home or in the community (96 %), they had experienced improvements in their daily interactions and relationships with family members (90%), they had shared the information they learned with others (88%), including spouses, family, and friends, they had used specific prevention st rategies learned in class (86 %), and had noticed a change in their parenting attitude (81%). When aske d to provide suggestions for improvements to future HOT DOCS classes, 50% of caregivers answered Nothing, the program is fine as is, 30% of caregivers suggested offering classes in alternat e locations or at different times of day, and 29% of caregivers answered More time for instruction, (e.g., more classes, longer sessions, booster sessions). These results s upport caregiver ratings of satisfaction with the program, by indicating that there were no significant changes or improvements that should be made to the program. When asked what they valued most from the training, the majority of caregiver s indicated the specifi c skills taught in the sessions (95%), other caregive rs indicated they valued learning problem-solving skills (83%), support and interaction with other caregivers (68%), and provision of materials without cost (66%). Decades of research have demonstrated th at high levels of treatment acceptability and satisfaction are predictors of more posit ive treatment outcomes, increased treatment adherence, and a greater like lihood of generalization of eff ects outside of the treatment setting (Eckert & Hintze, 2000; Elliott, Tureo, & Gresham, 1987; Foster & Mash, 1999; Wolf, 1978). Research on the impact of tr eatment acceptability suggests that when participants believe that the method of tr eatment was appropriate, fair, and reasonable (Kazdin, 1981), the likelihood of their following the treatment plan, demonstrating

PAGE 177

165 positive outcomes, and generalizing skills to various settings is greater than when participants indicate lower levels of treatment acceptability and social validity (Eisenstadt, Eyberg, McNeil, Newcomb, & Funderburk, 1993; Kratochwill, Elliott, Loitz, Sladeczek, & Carlson, 2003; Nastasi & Truscott, 2000). Therefore, the high ratings of participant satisfaction with the HOT DOCS parent training program reported in this study suggest that caregivers are more lik ely to demonstrate positive outcomes following their participation in the program than they would if the overall re ports of participant satisfaction were lower. Implications for Practitioners The results of this study suggest several im plications for practitioners. First, as in the pilot study of HOT DOCS (Williams, 2007), th e current investigation of participants' perceptions of the effectiven ess of the HOT DOCS parent training program suggested that following participation in the program, participants increased their knowledge of child development, behavioral principles, a nd parenting strategies; perceived a decrease in the severity of challenging behaviors in th eir target children; a nd reported high levels of satisfaction with the program. These findings are consistent with several decades of previous research on other pare nt training programs in demons trating the effectiveness of behavioral parent training as an intervention (Eyberg, 1988; Feinfield & Baker, 2004; Kazdin, 1995; Webster-Stratton, 1998). The e ffectiveness of using a group-delivered parent training program to address earl y-emerging challenging behavior allows psychologists to serve as indi rect service providers or consultants, enabling them to provide information and skills to caregivers which they can use to problem-solve and address their own childrens behavior. The i ndirect provision of se rvices is in stark

PAGE 178

166 contrast with the traditional medical model of service delivery, in which children are referred to a professional, an evaluation is conducted, and depending on the results, the professional directly applies treatment to th e child in a one-on-one format. While this traditional treatment model has been shown to be effective in producing desired outcomes it has also been shown to be less cost-effective and have poorer long-term outcomes than group-delivered, consultation model treatment strategies (Kazdin & Kendall, 1998; Knitzer, 2007; Kumpfer & Alvarado, 2003). Results of this study also provide practi tioners with an earl y intervention program that has been successfully delivered to pr eviously underserved por tions of the population, specifically, Hispanic or Spanish-speaki ng families and caregivers from low SES families. These early findings suggest that the HOT DOCS training program can be utilized as an intervention for challengi ng behaviors in young children by Hispanic or Spanish-speaking families. The disproportionate ly high enrollment of Hispanic families is likely explained by the translation and cultu ral adaptation of both printed materials and orally delivered presen tations into Spanish (C urtiss et al., 2009). HOT DOCS has also been made availa ble to a large proportion of low SES families because it funded by a grant from the Childrens Board of Hillsborough county and offered free of charge to all interested families in the local community. All materials and supplies are provided for caregivers, removi ng previously identified financial barriers to participation in parent training programs (Barkley et al., 2000; Webster-Stratton & Taylor, 2001). These results offer preliminar y support for practitioners' use of the HOT DOCS program to address the needs of chronically underserved and at-risk populations

PAGE 179

167 (e.g., low SES, minority racial/ethnic group) using a group-delivered parent training program that maximizes available resources. Limitations The current research study has a number of significant limitations. The first is the use of a preexisting database, which did not allow the researcher control over data collection procedures and the type of data originally collected. The researcher had to use data as collected by the program developers for purposes of reporting to their funding agency and not necessarily for the purposes of evaluating the intervention's effectiveness. The program developers created several m easurement instruments, which met their specific needs for program evaluation but did not provide sufficient data to allow this study to make direct comparisons to previous evaluations of well-established behavioral parent training programs. Specifically, the HOT DOCS program developers defined the caregivers' social support variable in terms of attendance to classes with or without another caregiver. As previously discussed, this unique definition does not allow for comparisons between the HOT DOCS participan t sample and previous research samples or local community demographic information, as these sources most often define social support through concepts such as marital st atus, or perceptions of access to social resources. Another variable, which through the me thod of data collection presented weaknesses to the current study, was the exclus ive use of parent re port to gather data about target children's preexis ting diagnoses. Relying simply on parent report has been shown to result in overor underreporting of children's symptoms and diagnoses (Connolly, Sharry & Fitzpatrick, 2001; Feinfi eld & Baker, 2004; Thompson et al., 1996;

PAGE 180

168 Webster-Stratton, Reid, & Hammond, 2004). The use of type of insurance as an indicator of caregivers' SES also presented problems in the current study in terms of interpreting results and making comparisons with other parent training research. The second limitation is that this study utilized a one group pretest-posttest design, which has several threats to internal validity of the st udy, including history, maturation, testing, instrumentation, mortal ity, and regression toward the mean. To overcome this limitation, future research shoul d incorporate the use of a waitlist control comparison group, which would provide data simultaneously with participants enrolled in the HOT DOCS program. The addition of a waitlist control group would allow researchers and/or program developers to more accurately evaluate the outcomes of participation in the program by controlling fo r the threats to validity mentioned above. A third limitation is the low return rate of several outcome m easures, specifically, posttest CBCL behavior rating scales. Given the low return rate of several outcome measures as identified in the pilot study (Williams, 2007), several modifications were made to HOT DOCS program procedures with the goal of increasing the overall return rate of participant measures. The modificati on included the addition of a Booster Session, as previously discussed. Additionally, a modi fication to procedures following the pilot study included the development of Reminder Notes and Way to Go cards, which were prepared by HOT DOCS trainers before each session listing either the specific forms each participant needed to complete, or congrat ulating participants on having turned in all material up to that point. Notes were prin ted on postcard paper a nd distributed at the beginning of each session. Distribution of the notes allowed the trainers to publicly recognize and praise those part icipants who had turned in all required measures and

PAGE 181

169 directly remind participants which measures th ey had not yet completed. It is noted that following the modifications made to the program procedures the retu rn rate of posttest measures increased dramatically from the p ilot study (26%) to the current study (41%), however, the overall return rate of posttest be havior rating scales in the this study was still remarkably low. Perhaps the most significant limitati on to this study was the use of nonstandardized instruments used to gather outcome data (Knowledge Test, Program Evaluation Survey). These instruments were designed by the authors of the HOT DOCS parent training program, thus, reliability and validity data are not available for these instruments. Program developers should select standardized outcome measures to collect data or should follow guidelines for designing and validating the ex isting tools currently used for HOT DOCS program evaluati on to address these limitations. Directions for Future Research Although this study provided information about the utility of changes made to HOT DOCS curriculum and procedures in re sponse to the pilot study outcomes (e.g., addition of a booster session to increase posttest return rates) (Williams, 2007), the results of the this study generated additional questions and limitations, which need to be addressed in future research st udies. The information gained from the results of this study gives the HOT DOCS program developers an opportunity to conti nue to refine and improve the HOT DOCS program and the meas ures, tools, and procedures used to evaluate the effectiveness of this prog ram. Program developers should focus on redesigning the HOT DOCS Knowledge Test and revising the definitions and data collection procedures for critical demographi c variables, such as caregivers' social

PAGE 182

170 support, caregivers' socio-economic status, and target child's preexisting diagnoses. In order to facilitate comparisons with exis ting parent training research, the program developers may wish to redefine the variables of social supp ort, SES, and diagnosis in a manner similar to how they have been defined in the literature to date (Lundahl, Risser, & Lovejoy, 2006; Maughan et al., 2005). Future research should address the relative ly high rate of at trition (28%) and low return rate of posttest measures (41%) identified in this study. For example, researchers could conduct follow-up surveys, phone interv iews, and/or focus groups to collect further evidence investigating caregivers' patterns and rates of attendan ce and attrition. For example, researchers should investigate why caregivers sign up for class and do not attend; why caregivers attend one or two sessions but do not complete training; and why a large percent of caregivers did not return posttest rating scale packets. Additional analyses specifically focusing on rates and pa tterns of attendance in relation to outcome variables should be conducte d. For example, did particip ants who attended specific sessions (e.g., sessions 3, 4 a nd 5) show greater gains in knowledge or problem solving skills and did they perceive their childrens skills as improving more than participants who attended different se ssions (e.g., 1, 2, and 6). As previously discussed, future eval uations of the HOT DOCS program should incorporate the use of a comparison or cont rol group and revised or replaced outcome measures with demonstrated reliability a nd validity. In the process of revising and redesigning outcome measures, future research should focus on a more thorough investigation of the problem-solving behavior al principles incor porated into the HOT DOCS program, since this is a unique featur e of the HOT DOCS program not addressed

PAGE 183

171 in the majority of existing behavioral pare nt training programs. Sp ecifically, an outcome measure assessing caregiver sati sfaction with and knowledge of the functions of behavior and the problem solving process should be included. This investigation should focus more specifically on to what extent caregiv ers learn and are able to implement the problem solving process. Although the results of specific investig ations of differential outcomes for caregivers based on various social and dem ographic variables indicated a lack of observed differences, the results of this study provided researchers with preliminary information about the effectiveness of the HOT DOCS program, which should be interpreted cautiously, given the limitations in data collection previously discussed. Results indicated fairly equivalent outcomes for various demographic groups, including caregivers with differing levels of education and social support networks and caregivers of children of various ages with diverse pree xisting diagnoses. Furthe r research is needed to investigate differences in outcomes for participants based on these demographic variables once the variables have been redefined to more closely match the definitions and data found in the existing literature base. Conclusion This study continues to e xpand the knowledge and evidence base of a potentially effective, cost-efficient, and participantendorsed prevention/intervention program for young children with early emerging, challenging be haviors. Results of this study suggest caregivers perceived positive outcomes for themselves and their children after participating in the HOT DOCS program regard less of caregivers' le vel of education and social support and target child's age and di agnosis. Indicators of successful outcomes

PAGE 184

172 included increases in caregiver knowledge, high levels of satisfaction with the program, and reductions in the perceived severity of child behavior problems. Results also indicated several modifications that could be made to the program to improve participant outcomes and increase the validity and reli ability of program evaluations, including changes to measurement instruments (e.g., K nowledge Test, Program Evaluation Survey, lack of direct observation of caregiver/child behaviors) a nd data collection procedures (e.g., waitlist control group; definitions of soci al support, SES, child's diagnosis; low rate of return of posttest rating scales). Over all, the HOT DOCS pa rent training program appears to be a promising early interventi on program that could be delivered in group format to caregivers from a variety of so cial and cultural groups to address early emerging challenging behaviors in young children with and without diagnosed conditions.

PAGE 185

173 References Abidin, R. (1995). Parenting Stre ss Index: Professional Manual (3rd ed.). Florida: Psychological Assessment Resources Inc. Achenbach, T. (2001). Manual for the Achenbach system of empirically based assessment (ASEBA) Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families. Achenbach, T. (1991). Manual for the Child Behavior Checklist/4-18 and 1991 Profile Burlington, VT: University of Ve rmont, Department of Psychiatry. Achenbach, T. & Edelbrock, C. (1983). Manual for the Child Behavior Checklist and the Revised Behavior Problem Profile. Burlington, VT: University of Vermont. American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders: Fourth Edition (DSM-IV) (4th ed.). Washington, D.C.: Author. American Psychiatric Association. (1987). Diagnostic and statistical manual of mental disorders: Third Edition (DSM-III) (3rd ed.). Washington, D.C.: Author. Anastopoulos, A., Shelton, T., DuPaul, G., & Gu evremont, D. (1993). Parent training for attention deficit hyperactivity disord er: Its impact on parent functioning. Journal of Abnormal Child Psychology, 21 581-596. Armstrong, K. & Hornbeck, M. (2005). Helping our Toddlers: Le ssons for parents of young children with challenging behaviors. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, Department of Child and Family Studies.

PAGE 186

174 Armstrong, K., Hornbeck, M., Beam, B., Mack, K., & Popkave, K. ( 2006). Evaluation of a curriculum designed for caregivers of young children with challenging behavior. Journal of Early Childhood and Infant Psychology, 2 51-61. Armstrong, K., Lilly, C., & Curtiss, H. (2006). Helping our toddle rs developing our childrens skills Tampa, FL: University of South Florida, Department of Pediatrics, Division of Child Development. Bagner, D., & Eyberg, S. (2003). Father involvement in parent training: When does it matter? Journal of Clinical Child and Adolescent Psychology, 32, 599605. Barkley, R. (1997). Defiant children: A clinicians manual for assessment and parent training. 2nd ed New York: Guilford Press. Barkley, R., Shelton, T., Crosswait, C., M oorehouse, M., Fletcher, K., Barrett, S., Jenkins, L., & Metevia, L. (2000). Multi -method psycho-educational intervention for preschool children with disruptive behavior: Preliminary results at posttreatment. Journal of Child Psychology and Psychiatry, 3 319-322. Barlow, J. & Parsons, J. (2002). Group-based parent-training programmes for improving emotional and behavioural adjust ment in 0-3 year old children. Cochrane Database of Systematic Reviews 2002, 4 433-467. Bath, H., Richey, C., & Haapala, D. (1992) Child age and outcome correlates in intensive family preservation services. Children and Youth Services Review, 14 389-406.

PAGE 187

175 Beernink, A., Swinkels, S., & Buitelaar, J. ( 2007). Problem behavior in a community sample of 14and 19-month-old childr en: Common versus uncommon behaviors, structure, and stability. European Child & Adolescent Psychiatry, 16 271-280. Blackwell, P. (2005). Parent sour ce: A practice in enrichment. Zero to Three, 26, 4-8. Bor, W., Sanders, M., & Markie-Dadds, C. (2002). The effects of the Triple P-positive Parenting Program on preschool children with co-occurring disruptive behavior and attentional/hyperactivity difficulties. Journal of Abnormal Child Psychology, 30, 571-587. Bradley, S., Jadaa, D., Brody, J., Landy, S., Tallett, S., Watson, W., et al. (2003). Brief psychoeducational parenti ng program: An evaluation and 1-year follow-up. Journal of the American Academy of Child Adolescent Psychiatry, 42, 1171. Briegel, W., Schneider, M., & Schwab, K. ( 2008). 22q11.2 deletion syndrome: Behaviour problems of children and adolescents and parental stress. Child: Care, Health and Development, 34 795-800. Brenner, V. & Fox, R. (1998). Parental discipline and behavior problems in young children. Journal of Genetic Psychology, 159 251-257. Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design. Cambridge, MA: Harvard University Press. Buss, D. (1981). Predicting pare nt-child interactio ns from children's activity level. Developmental Psychology, 17, 59-65.

PAGE 188

176 Cabrera, N. et al. (2002). Bridging research and policy: Including fathers of young children in national studies. In C. S. Tamis-Lemonda & N. Cabrera (Eds.), Handbook of father involvement (pp. 489-524). Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Campbell, S. (1997). Behavior problems in preschool children: Developmental and family issues. In T. H. Ollendick & R. J. Prinz (Eds.), Advances in clinical child psychology (Vol .19, pp.1-26). New York: Plenum. Campbell, S. (1995). Behavior problems in preschool children: A review of recent research. Journal of Child Psychology & Psychiatry & Allied Disciplines, 36, 113-149. Campbell, S. & Ewing, L. (1990). Followup of hard-to-manage preschoolers: Adjustment at age 9 and predic tors of continuing symptoms. Journal of Child Psychology and Psychiatry and Applied Disciplines, 31, 871. Campis, L., Lyman, R., & Prentice-Dunn, S. (1986) The parental locus of control scale: Development and validation. Journal of Clinical Child Psychology, 15, 260-267. Carr, E., Dunlap, G., Horner, R., Koegel, R., Turnbull, A., Sailor, W., et al. (2002). Positive behavior support: E volution of an applied science. Journal of Positive Behavior Interventions, 4 4-16. Cartwright-Hatton, S., McNally, D., & White, C. (2005). A new cognitive behavioural parenting intervention for families of young, anxious children: A pilot study. Behavioural and Cognitive Psychotherapy, 33 243-247. Chamberlain, P. & Reid, J. (1987). Parent observation and report of child symptoms. Behavioral Assessment, 9 97-109.

PAGE 189

177 Coie, J. & Dodge, K. (1998). Aggression a nd antisocial behavior. In W. Damon (Editor in Chief) and N. Eisenberg (Vol. Ed.), Handbook of child psychology, 5th edition, Volume 3, Social, emoti onal, and personalit y development NY: John Wiley & Sons. Connolly, L., Sharry, J., & Fitzpatrick, C. (2001). Evaluation of a group treatment programme for parents of childre n with behavioural disorders. Child Psychology & Psychiatry Review, 6 159-165. Conroy, M., Dunlap, G., Clarke, S., & Alter, P. (2005). A descriptive an alysis of positive behavioral intervention research with young children with challenging behavior. Topics in Early Childhood Special Education, 25 157-166. Cooper, J., Heron, T., & Heward, W. (1987). Applied behavior analysis Upper Saddle River, NJ: Merrill. Cummings, E. & Davies, P. (1994). Children and marital conflict: The impact of family dispute and resolution. New York, London: Guilford. Cunningham, C., Bremner, R. & Boyle, M. (1995). Large group community-based parenting programs for families of presc hoolers at risk for disruptive behavior disorders: Utilization, cost effectiveness, and outcome. Journal of Child Psychology and Psychiatry, 36 1141-1159. Curtiss, H., Salinas, A., Williams, J., Arms trong, K., & Ortiz, C. (2009). Adaptation of a parenting program for Hispanics families: Helping our toddlers, developing our childrens skills ( HOT DOCS )

PAGE 190

178 Dadds, M. & Powell, M. (1991). The relationshi p of interparental conflict and global marital adjustment to aggression, anxi ety, and immaturity in aggressive and nonclinic children. Journal of Abnormal Child Psychology, 19 553-567. Denham, S. & Burton, R. (1996). A social-emo tional intervention for at-risk 4-year olds. Journal of School Psychology, 32 225-245. DelHomme, M., Sinclair, E. & Kasari, E. ( 1994). Preschool children with behavioral problems: Observation in instructional and free play contexts. Behavioral Disorders, 19 221-232. Denham, S., Workman, E., Cole, P., Weissbr od, C., Kendziora, K., & Zahn-Waxler, C. (2000). The role of parental so cialization and emotion expression. Developmental and Psychopathology, 12 23-45. Dishion, T., French, D., & Patterson, G. (1995). The developmental and ecology of antisocial behavior. In D. Cicchetti & D. J. Cohen (Eds.), Developmental Psychopathology, Vol. 2: Risk, disorder, and adaptation (pp. 421-471). New York: John Wiley & Sons. Dishion, T. & Patterson, G. (1992). Ag e effects in parent training outcome. Behavior Therapy, 23, 719-729. Dodge, K. (1993). The future of research on conduct disorder. Development and Psychopathology, 5, 311-320. Dodge, K., Bates, J., & Pettit, G. (1990). Mechanisms in the cycle of violence. Science, 250, 1678-1683. Dunlap, G. (2006). The applied behavior analyt ic heritage of PBS: A dynamic model of action-oriented research. Journal of Positive Behavior Support, 8 58-60.

PAGE 191

179 Dunlap, G. & Fox, L. (1996). Early interven tion and serious behavior problems: A comprehensive approach. In L. K. Koegel R. L. Koegel, & G. Dunlap (Eds.), Positive behavior support: Including pe ople with difficult behavior in the community (pp. 31-50). Baltimore, MD: Brookes Publishing Co. Dunst, C., Trivett, C., & Cutspec, P. (2002). Toward an operational definition of evidence-based practice. Retrieved December 3, 2006, from the Center for Evidence-Based Practice: Young Children with Challenging Behaviors website: http://challengingbehavior.fmhi.usf.edu Eckert, T. & Hintze, J. (2000). Behavioral c onceptions and applica tions of acceptability: Issues related to service delivery and research methodology. School Psychology Quarterly, 15 123-148. Egeland, B., Kalkoske, M., Gottesman, N., & Erikson, M. (1990). Preschool behavior problems: Stability and factors accounting for change. Journal of Child Psychology and Psychiatry, 31 891-909. Eisenstadt, T., Eyberg, S., McNeil, C., Ne wcomb, K., & Funderburk, B. (1993). ParentChild Interaction Therapy with behavior problem children: Relative effectiveness of two stages and overall treatment outcome. Journal of Clinical Child & Adolescent Psychology, 22 42-51. Elliott, S., Tureo, T., & Gresham, F. (1987). Consumers' and clients' pretreatment acceptability ratings of classroom group contingencies. Journal of School Psychology, 25, 145-153. Eyberg, S. (1988). Parent-child interaction therapy: Integration of traditional and behavioral concerns. Child & Family Behavior Therapy, 10 33-46.

PAGE 192

180 Eyberg, S., Bessmer, J., Newcomb, K., Edwa rds, D., & Robinson, E. (1994). Dyadic parent-child interaction coding system-II: A manual. Social and behavioral science documents (Ms. No. 2897). San Rafae l: CA: Select Press. Eyberg, S. & Pincus, D. (1999). Eyberg Child Behavior Invento ry and the Sutter-Eyberg Student Behavior Scale-Revised Psychological Assessment Resources, Inc.: Lutz, FL. Eyberg, S., Funderburk, B., Hembree-Kigin, T., McNeil, C., Querido, J., & Hood, K. (2001). Parent-child interact ion therapy with behavior problem children: One and two year maintenance of treatment effects in the family. Child & Family Behavior Therapy, 23, 1-20. Feil, E., Walker, H., Severson, H., & Ball, A. (2000). Proactive screening for emotional/behavioral concerns in Head Start preschools: Promising practices and challenges in applied research. Behavioral Disorders, 26, 13-25. Feinfield, K. (1995). Predicting mother-child interactions. Unpublished master's thesis. University of Ca lifornia, Los Angeles. Feinfield, K. & Baker, B. (2004). Empirical support for a treatment program for families of young children with externalizing problems. Journal of Clinical Child and Adolescent Psychology, 33 182-195. Fischer, M., Rolf, J., Hasazi ,, J., & Cummings, L. (1984). Follow-up of a preschool epidemiological sample: Cross-age continuities and predictions of later adjustment with internalizing and ex ternalizing dimensions of behavior. Child Developmen t, 55 137-150.

PAGE 193

181 Forehand, R. & Kotchick, B. (1996). Cultural diversity: A wakeup call for parent training. Behavior Therapy, 27 187-206. Forehand, R., & Long, N. (1988). Outpatient treatment of the acting out child: Procedures, long term follow-up data, and clinical problems. Advanced Behavior Research Therapy, 10 129-177. Forehand, R., & Wierson, M. (1993). The role of developmental factors in planning behavioral interventions for children: Disruptive behavior as an example. Behavior Therapy, 24 117-141. Foster, S. & Mash, E. (1999). Assessing social validity in c linical treatment research issues and procedures. Journal of Consulting and Clinical Psychology, 67 308319. Fox, R. (1994). Parent Behavior Checklist. Department of Counse ling and Educational Psychology, Marquette, University, bobfox@usm.edu. Fox, L., Dunlap, G., & Powell, D. (2002). Y oung children with cha llenging behavior: Issues and considerations for behavior support. Journal of Positive Behavior Interventions, 4 208-217. Fox, R. & Nicholson, B. (2003). Parenting Young Children: A Facilitators Guide Sopris West Educationa l Services: Longmont, CO. Frisbie, D. (1974). The effect of item form at on reliability and validity: A study of multiple choice and true-false achievement tests. Educational and Psychological Measurement, 34, 885-892.

PAGE 194

182 Gross, D., Fogg, L., Webster-Stratton, C., Ga rvey, C., Julion, W., & Grady, J. (2003). Parent training of toddlers in day care in low-income urban communities. Journal of Consulting & Clinical Psychology, 71, 261-278. Gross, D., Sambrook, A., & Fogg, L. (1999) Behavior problems among young children in low-income urban day care centers. Research in Nursing & Health, 22 15-25. Hartman, R., Stage, S., & Webster-Stratton, C. (2003). A growth-curve analysis of parent training outcomes: Examining the influence of child risk factors (inattention, impulsivity, and hyperactivity problems), parental and family risk factors. Journal of Child Psychology and Psychiatry, 44 388-398. Hebbler, K. Spiker, D., Mallik, S., Scar borough, A., Simeonsson, R., & Collier, M. (2001). A first look at the characteristics of children and families entering early intervention services. Menlo Park, CA: SRI International. Heller, T., Baker, B., Henker, B., & Hinsha w, S. (1996). Externalizing behavior and cognitive functioning from preschool to first grade: Stabil ity and predictors. Journal of Clinical Child Psychology 25 376-387. Hershcell, A., Calzada, E., Eyberg, S., & Mc Neil, C. (2002). Parent-child interaction therapy: New directions in research. Cognitive Behavioral Practice, 9 9-15. Hogan, T. & Murphy, G. (2007). Recommendations for preparing and scoring constructed-response items : What the experts say. Applied Measurement in Education, 20 427-441. Holden, G., Lavigne, V., & Cameron, A. (1990) Probing the continuum of effectiveness in parent training: Characterist ics of parents and preschoolers. Journal of Clinical Child Psychology, 19 2-8.

PAGE 195

183 Jolivette, K., Gallagher, P., & Morrier, M. (2008). Preventing problem behavior in young children with disabilitie s. Exceptionality, 16, 78-92. Kazdin, A. (1997). Parent management tr aining: Evidence, outcomes, and issues. Journal of the American Academy of Child and Adolescent Psychiatry, 36 1349-1356. Kazdin, A. (1995). Conduct disorder. In F. C. Verhulst & H. M. Koot (Eds.), The epidemiology of child and adolescent psychopathology, (pp. 258-290). New York: Oxford University Press. Kazdin, A. (1993). Adolescent mental hea lth: Prevention and treatment programs. American Psychologist, 48 127-141. Kazdin, A. (1981). Acceptability of child treatment techniques: The influence of treatment efficacy and adverse side effects. Behavior Therapy, 12, 493-506. Kazdin, A. & Kendall, P. (1998). Current pr ogress and future plans for developing effective treatments: Comments and perspectives. Journal of Clinical Psychology, 27, 217-226. Kazdin, A. & Wassell, G. (1999). Barriers to treatment participation and therapeutic change among children referred for conduct disorder. Journal of Clinical Child Psychology, 28, 160-172. Keenan, K. & Wakschlag, L. (2000). More than th e terrible twos: The nature and severity of behavior problems in clin ic-referred preschool children. Journal of Abnormal Child Psychology, 28, 33-46. Kellem, S. (1989). Johns Hopkins Preventative Intervention Research Center [NIMH grant proposal]. Baltimore: Johns Hopkins University.

PAGE 196

184 Kendziora, K. & O'Leary, S. (1993). Dysfuncti onal parenting as a focus for prevention and treatment of child behavior problems. In T. H. Ollendick & R. J. Prinz (Eds.), Advances in Clinical Child Psychology, (Vol. 15, pp 175-206). New York: Plenum. Knapp, P., Ammen, S., Arstein-Kerslake, C ., Poulsen, M., & Mastergeorge, A. (2007). Feasibility of expanding services for very young children in the public mental health setting. Journal of the American Academy of Child and Adolescent Psychiatry, 46 152-161. Knapp, P. & Deluty, R. (1989). Relative effec tiveness of two behavior al parent training programs. Journal of Child Clinical Psychology, 18 314-322. Knitzer, J. (2007). Putting knowledge into po licy: Toward an infant-toddler policy agenda Infant Mental Health Journal, 28 237-245. Kratochwill, T., Elliott, S., Loitz, P., Sl adeczek, I., & Carlson, J. (2003). Conjoint consultation using self-administered manual and videotape parent-teacher training: Effects on children's behavioral difficulties. School Psychology Quarterly, 18 269-302. Kumpfer, K. & Alvarado, R. (2003). Family-s trengthening approaches for the prevention of youth problem behaviors. The American Psychologist, 58 457-465. Lamb, M. (1997). Fathers and child developm ent: An introductory overview. In M. E. Lamb (Ed.), The role of fathers in child development (3rd ed., pp. 1-18). New York: Wiley. Lardieri, L., Blacher, J., & Swanson, H. (2000). Sibling relationships and parent stress in families of children with and without learning disabilities. Learning Disability Quarterly, 23 105-116.

PAGE 197

185 Lavigne, J., Gibbons, R., Christo ffel, K. K., Arend, R., Rose nbaum, D., Binns, H., et al. (1996). Prevalence rates and correlates of psychiatric disorders among preschool children. Journal of the American Academy of Child & Adolescent Psychiatry, 35 204-214. Lewis, C. & O'Brien, M. (1987). Constraints on fathers: Research, theory, and clinical practice. In C. Lewis & M. O'Brien (Eds.), Reassessing fatherhood: New observations on fathers and the modern family (pp. 1-19). Beverly Hills, CA: Sage. Lipsey, M. & Derzon, J. (1998). Predictors of violent or serious delinquency in adolescence and early adulthood: A synthe sis of longitudinal research. In R. Loeber & D. P. Farrington (Eds.), Serious & violent juvenile offenders: Risk factors and successful interventions (pp. 86-105). Thousand Oaks, CA: Sage. Loeber, R. & Dishion, T. (1983). Early pred ictors of male delinquency: A review. Psychological Bulletin, 94 68-99. Lundahl, B., Risser, H., & Lovejoy, C. (2006) A meta-analysis of parent training: Moderators and follow-up effects. Clinical Psychology Review, 26 86-104. Lundahl, B., Tollefson, D., Risser, H. & Lovejo y, M. (2008). A meta-analysis of father involvement in parent training. Research on Social Work Practice, 18 97-106. Luthar, S. (2006). Resilience in development: A synthesis of research across five decades. In D. Cicchetti & D. J. Cohen (Eds.), Developmental psychopathology: Risk, disorder, and adaptation (2nd ed.) New York: Wiley. Lutzker, J. & Campbell, R. (1994). Ecobehavioral family interventions in developmental disabilities. Pacific Grove, CA: Brooks/Cole Thomson Learning.

PAGE 198

186 Mansell, W. & Morris, K. (2004). A survey of parents' reactions to the diagnosis of an autistic spectrum disorder by a local serv ice. Access to information and use of services. Autism, 8, 387-407. Marchant, M., Young, R., & West, R. (2004). The effects of parental teaching on compliance behavior of children. Psychology in the Schools, 41 337. Magee, T. & Roy, C. (2008). Predicting school-a ge behavior problems: The role of early childhood risk factors. Pediatric Nursing, 34 37-44. Maughan, D., Christiansen, E., Jenson, W., Olym pia, D., & Clark, E. (2005). Behavioral parent training as a treatment for extern alizing behaviors and disruptive behavior disorders: A meta-analysis. School Psychology Review, 34, 267-286. McCabe, K., Yeh, M., Garland, A., Lau, A., Chavez, G. (2005). The GANA program: A tailoring approach to adapting parent-c hild interaction therapy for Mexican Americans. Education and Treatment of Children, 28 111-129. McMahon, R. & Forehand, R. (2003). Helping the non-compliant child: Family-based treatment for opposit ional behavior (2nd ed.). New York, NY: Guilford Press. McNeil, C., Capage, L., Bahl, A., & Blanc, H. (1999). Importance of early intervention for disruptive behavior probl ems: Comparison of treatment and wait list-control groups. Early Education and Development, 10, 445-454. McNeill, S., Watson, T., Henington, C., & Meek s, C. (2002). The effects of training parents in functional behavior assess ment on problem identification, problem analysis, and intervention design. Behavior Modification, 26 499-515. Meyers, S. (1993). Adapting parent education prog rams to meet the needs of fathers: An ecological perspective. Family Relations 42, 447-452.

PAGE 199

187 Moore, D. (1995). The basic practice of st atistics. New York, NY: Freeman and Co. Mrazek, D., Mrazek, P., & Kli nnert, M. (1995). Clinical assessment of parenting. Journal of the American Academy of Ch ild & Adolescent Psychiatry, 34 272-282. Nastasi, B. & Truscott, S. (2000). Acceptabil ity research in school psychology: Current trends and future directions. School Psychology Quarterly, 15 117-122. Nelson, C. (1995). A meta-analysis of parent education programs for chil dren two to nine years. Dissertation Abstracts International: Section B: The Sciences and Engineering, 56, 1686. Nicholson, B., Fox, R., & Johnson, S. (2005). Pa renting young children with challenging behaviour. Infant and Child Development, 14 425-428. Nixon, R., Sweeny, L., Erickson, D., & Touyz, S. (2003). Parent-child interaction therapy: A comparison of standard and abbreviated treatments for oppositional defiant preschoolers. Journal of Consulting an d Clinical Psychology, 71 251260. Olweus, D. (1991). Bully/victim problems among school children: Basic facts and effects of a school-based intervention program In D. Pepler & K. Rubin (Eds.), The development and treatment of childhood aggression (pp. 411-446). London: Lawrence Erlbaum. Orvaschel, H., & Puig-Antich, J. (1995). Schedule for Affective Disorders and Schizophrenia for School-Age ChildrenEpidemiologic 5th Version Fort Lauderdale, FL, Nova Univ ersity, 1995. Book Citation Palkovitz, R. (1996). Parenting as a generator of adult development: Conceptual issues and implications. Journal of Social and Personal Relationships, 13, 571-592.

PAGE 200

188 Parke, R. & Brott, A. (1999). Throwaway dads: The myths and barriers that keep men from being the fathers they want to be Boston: Houghton Mifflin Company. Patterson, G. (1982). Coercive family process. Eugene, OR: Castalia. Patterson, G., DeBaryshe, B., & Ramsey, E. (1989). A developmental perspective on antisocial behaviour. American Psychologist, 44 329-335. Patterson, G. & Fleishman, M. (1979). Main tenance of treatment effects: Some considerations concerning family systems and follow-up data. Behavior Therapy, 10, 168-185. Patterson, G., Reid, J., & Dishion, T. (1992). Antisocial boys Eugene, OR: Castalia. Pettit, G., Bates, J., & Dodge, K. (1993). Family interaction patterns and children's conduct problems at home and school: A longitudinal perspective. School Psychology Review, 22 403-420. Phares, V., Fields, S., Kamboukos, D., & Lopez, L. (2005). Still looking for Poppa. American Psychologist, 60 735-736. Qi, C. & Kaiser, A. (2003). Behavior problem s of preschool children from low-income families: Review of the literature. Topics in Early Chil dhood Special Education, 23, 188-216. Ra mey, C. & Ramey, S. (1998). Early intervention and early experience. American Psychologist, 53 109-120. Reid, J. (1993). Prevention of conduct disorder before and after sc hool entry: Relating interventions to developmental findings. Development and Psychopathology, 5, 243-262.

PAGE 201

189 Reid, M., Webster-Stratton, C., & Baydar, N. (2 004). Halting the deve lopment of conduct problems in Head Start children: The effects of pa rent training. Journal of Clinical Child and Adolescent Psychology, 33 279-291. Reid, M., Webster-Stratton, C., & Beauchaine, T. (2001). Parent training in head start: A comparison of program response among African American, Asian American, Caucasian, and Hispanic mothers. Prevention Science, 2 209. Ruma, P., Burke, R., & Thompson, R. (1996). Group parent training: Is it effective for children of all ages? Behavior Therapy, 27, 159-169. Rutter, M. (1991). Childhood experiences and ad ult psychosocial functioning. In G. R. Book & J. Whelan (Eds.) The Childhood Environment and Adult Disease, Ciba Foundation Symposium No. 1 (pp 189-208). Chichester, England: Wiley. Sandall, S. & Ostrosky, M. (Eds.). (1999). Young exceptional children: Practical ideas for addressing challenging behaviors. (1st ed.). Denver: Division for Early Childhood of the Council for Exceptional Children. Sanders, M. (1999). Triple P-positive pare nting program: Towards an empirically validated multilevel parenting and family support strategy for the prevention of behavior and emotional problems in children. Clinical Child and Family Psychology Review, 2 71. Sanders, M. (1996). New directions in behavioral family interv ention with children. In T. H. Ollendick & R. H. Prinz (Eds.), Advances in Clinical Child Psychology, (Vol 18, pp. 283-330). New York: Plenum.

PAGE 202

190 Sanders, M., Markie-Dadds, C., Tully, L., & Bor, W. (2000). The Triple P-Positive Parenting Program: A comparison of e nhanced, standard, and self-directed behavioral family intervention for pare nts of children with early onset conduct problems. Journal of Consulting and Clinical Psychology, 68 624-640. Sanders, M., Mazzucchelli, T., & Studman, L. (2004). Stepping stones Triple P: The theoretical basis and development of an evidence-based positive parenting program for families with a child who has a disability. Journal of Intellectual Developmental Disability, 29 265-283. Schuchmann, E., Foote, Eyberg, S., Boggs, S., & Algina, J. (1998). Efficacy of parentchild interaction therapy: Interim report of a randomized tria l with short-term maintenance. Journal of Clinical Child Psychology, 27 34-45. Setterberg, S., Bird, H., Gould, M., Shaffer, D., & Fisher, P. (1992). Parent and interviewer versions of the Childrens Global Assessment Scale. New York: Columbia University. Shaw, D., Emery, R., & Turner, M. (1993) Parental functioning and children's adjustment in families of divorce: A prospective study. Journal of Abnormal Child Psychology, 21 119-134. Shonokoff, J. & Phillips, D. (Eds.). (2000). From neurons to neighborhoods: The science of early childhood development Washington, DC: National Academy Press. Singer, G., Ethridge, B., & Alda na, S. (2007). Primary and sec ondary effects of parenting and stress management interventions for parents of children w ith developmental disabilities: A meta-analysis. Mental Retardation and Developmental Disabilities Research Reviews, 13 357-369.

PAGE 203

191 Skinner, B. (1953). Science and human behavior New York: Macmillan Free Press. Smagner, J. & Sullivan, M. (2005). Investigatin g the effectiveness of behavioral parent training with involuntary client s in child welfare settings. Research on Social Work Practice, 15 431-439. Smith, E. (2006). Review of Developing and Validating Multiple-Choice Test Items (3rd ed.). Applied Psychological Measurement, 30 69-71. Smith, B. & Fox, L. (2003, January). Systems of Service Deliv ery: A Synthesis of Evidence Relevant to Young Children at Risk for or Who Have Challenging Behavior. Retrieved September 21, 2006, from the Center for Evidence-Based Practice: Young Children with Challenging Behaviors website: http://challengingbehavior.fmhi .usf.edu/resources/smith-fox-jan03.pdf Smith, K., Landry, S., & Swank, P. (2005). The in fluence of decreased parental resources on the efficacy of a resp onsive parenti ng intervention. Journal of Consulting and Clinical Psychology, 73, 711. Smith, K., Landry, S., & Swank, P. (2000). The influence of early patterns of positive parenting on children's preschool outcomes. Early Education & Development, 1 147-169. Spanier, G. (1976). Measuring dyadic adjustment : New scales for assessing the quality of marriage and similar dyads. Journal of Marriage and the Family, 38 15-28. Stevens, J. (1999). Intermediate statistics: A modern approach (2nd edition). Hillsdale, NJ: Lawrence Erlbaum.

PAGE 204

192 Stolk, M., Mesman, J., van Zeijl, J., Ali nk, L., Bakermans-Kranenburg, M., IJzendoorn, M., et al. (2008). Early parenting intervention: Family risk and first-time parenting related to intervention effectiveness. Journal of Child & Family Studies, 17, 55-83. Stormshak, E., Bierman, K., McMahon, R., & Lengua, L. (2000). Parenting practices and child disruptive behavior problems in early elementary school. Journal of Clinical Child Psychology, 29. 17-29. Strain, P., Lambert, D., Kerr, M., Stra gg, V., & Lenker, D. (1983). Naturalistic assessment of childrens compliance to teachers requests and consequences for compliance. Journal of Applied Behavior Analysis, 16 243-249. Strain, P. & Timm, M. ( 2001). Remediation and preven tion of aggression: An evaluation of the RIP Program over a quarter century. Behavioral Disorders, 26, 297-313. Strain, P., Young, C., & Horowitz, J. (1981) Generalized behavior change during oppositional child training: An examinati on of child and family demographic variables. Behavior Modification, 5 15-26. Straus, M. (1989). Manual for the Conflict Tactics Scales (CTS) Durham: University of New Hampshire. Strayhorn, J. & Weidman, C. (1988). A parent pr actices scale and its relation to parent child mental health. Journal of the American Academ y of Child & Adolescent Psychiatry, 27 613-618. Thompson, R., Ruma, P., Schuchmann, L., & Bu rke, R. (1996). A cost-effectiveness evaluation of parent training. Journal of Child & Family Studies, 5, 415-429.

PAGE 205

193 Tiano, J. & McNeil, C. (2005). The inclusion of fathers in behavioral parent training: A critical evaluation. Child & Family Therapy, 27, 1-28. Tremblay, R. (2000). The development of a ggressive behavior dur ing childhood: What have we learned in the past century? International Journal of Behavioral Development, 24, 129-141. Wahler, R. & Dumas, J. E. (1986). A chip off the old block: Some interpersonal characteristics of coercive children acro ss generations. In P. Strain, M. Guralnick, & H. M. Walker (Eds.), Childrens soci al behavior: Development, assessment, and modification (pp. 49-91). Or lando, FL: Academic Press. Walker, H., Colvin, G., & Ramsey, E. (1995). Antisocial behavior in school: Strategies and best practices Belmont, CA: Brooks/Cole. Walker, H., Kavanaugh, K., Stiller, B., Golly, A., Severson, H., & Feil, E. (1998). First Step to Success: An early interv ention approach for preventing school antisocial behavior. Journal of Emotional and Behavioral Disorders, 6 66-80. Webster-Stratton, C. (2001). The Incredible Years: Parent, teacher, and child training series. Boulder, CO: Center for the Study and Prevention of Violence, Institute of Behavior Science, University of Colorado at Boulder. Webster-Stratton, C. (2000). Oppositional-defi ant and conduct disordered children. In M. Hersen & R. T. Ammerman (Eds.), Advanced abnormal child psychology (2nd ed., pp. 387). Mahwah, NJ: Erlbaum. Webster-Stratton, C. (1998). Preventing c onduct problems in Head Start children: Strengthening parenting competencies. Journal of Consulting and Clinical Psychology, 66, 715-730.

PAGE 206

194 Webster-Stratton, C. (1990). Long-term follow-up of families with young conductproblem children: From preschool to grade school. Journal of Clinical Child Psychology, 19, 144-149. Webster-Stratton, C. (1985). Mother percep tions and mother-child interactions: Comparison of a clinic-r eferred and a non-clinic group. Journal of Clinical Child Psychology, 14, 334-339. Webster-Stratton, C., & Hammond, M. (1997). Treating children with early-onset conduct problems: A comparison of child and parent training interventions. Journal of Consulting and Clinical Psychology, 65 93-109. Webster-Stratton, C., & Hammond, M. (1990). Predictors of treatment outcomes in parent training for families with conduct problem children. Behavior Therapy, 21 319-337. Webster-Stratton, C., Reid, M., & Hammond, M. (2004). Treating children with earlyonset conduct problems: Intervention outcomes for parent, child, and teacher training. Journal of Clinical Child and Adolescent Psychology, 33 105-124. Webster-Stratton, C. & Taylor, T. (2001). Ni pping early risk factors in the bud: Preventing substance abuse, delinquen cy, and violence in adolescence through interventions targeted at young children (0 to 8 Years). Prevention Science, 2 165-192. Weinberg, H. (1999). Parent training for attent ion-deficit hyperactivit y disorder: Parental and child outcome. Journal of Clinical Psychology, 55 907-913. Williams, J. (2007). Caregivers Percep tions of the Effectiveness of the Helping Our Toddlers, Developing Our Childrens Skills Parent Training Program: A Pilot

PAGE 207

195 Study. Psychological and Social Foundations Tampa, University of South Florida. Educational Specialist (EdS): 163. Williams, J., Armstrong, K., Bradley-Klug, K, & Curtiss, H. (In preparation). Helping Our Toddlers, Developing Our Childrens Skills (HOT DOCS): A Parenting Intervention to Prevent and Address Challenging Be havior in Young Children. Wolf, M. (1978). Social validity: Th e case for subjective measurement. Journal of Applied Behavior Analysis, 11, 203-214.

PAGE 208

196 Appendices

PAGE 209

197 Appendix A HOT DOCS Parent Training Curriculum Session Topic Parenting tip ho mework Special play activity 1 Early childhood development Use positive words Bubbles 2 Routines and rituals Catch them being good Reading 3 Behavior and development Use a calm voice Coloring 4 Preventing problem behavior Use preventions Fun Dough 5 Teaching new skills Follow-through Balls 6 Managing parent stress Take time for yourself Free choice Booster Review previous sessions Review 1-6 Review 1-6

PAGE 210

198 Appendix B HOT DOCS Demographics Form fo r Caregivers (English version)

PAGE 211

199 Appendix C HOT DOCS Demographics Form for Caregivers (Spanish version)

PAGE 212

200 Appendix D HOT DOCS Knowledge Test (English version)

PAGE 213

201 Appendix E HOT DOCS Knowledge Test (Spanish version)

PAGE 214

202 Appendix F HOT DOCS Program Evaluation Survey for Caregivers (English version)

PAGE 215

203 Appendix G HOT DOCS Program Evaluation Survey for Caregivers (Spanish version)

PAGE 216

204 Appendix H Relation between Research Questions and Variables Question Dependent Variable Design What is the impact of participation in the HOT DOCS program on parent knowledge? HOT DOCS Knowledge Test Preand Posttests Do caregivers perceive th eir child as having more problem behavior than a normative sample prior to participation in HOT DOCS? CBCL Pretest Do childs problem behaviors decrease following parent participation in HOT DOCS? CBCL Preand Posttests What are caregivers overa ll perceptions of their participation in the HOT DOCS parent training program? HOT DOCS Program Evaluation Survey for Caregivers Posttest

PAGE 217

About the Author Jillian L. Williams is a Nationally Certified School Psychologist (NCSP) and a doctoral intern in Pediatric School Psychology at the Universi ty of South Florida. She currently conducts research on the effectiveness of a parent training program based on the principles of behaviorism and problem-solving. Ms. Williams completed her doctoral internship in the Department of Pediatrics at USF, conduc ting developmental evaluations, planning and implementing early interventi on strategies with families, providing professional development training s, and serving as project dir ector and as a facilitator of the Helping Our Toddlers, Developing Ou r Childrens Skills (HOT DOCS) parent training program. Ms. Williams received a Ba chelors degree from the University of Maryland, Baltimore County in 2001, Masters and Certificate of Advanced Studies degrees from Towson University in 2005, co mpleted an APPIC internship at the Louisiana School Psychology Internship Consortium (LAS*PIC) in Baton Rouge, Louisiana in 2004-2005, and receiv ed an Educational Special ist degree from USF in 2007.