USF Libraries
USF Digital Collections

The impact of fidelity and innovations on Healthy Families America programs

MISSING IMAGE

Material Information

Title:
The impact of fidelity and innovations on Healthy Families America programs
Physical Description:
Book
Language:
English
Creator:
Nixon, Ashley E
Publisher:
University of South Florida
Place of Publication:
Tampa, Fla.
Publication Date:

Subjects

Subjects / Keywords:
Program implementation
Organizational expansion
Multi-site programs
Organizational effectiveness
Home visiting programs
Dissertations, Academic -- Psychology -- Masters -- USF   ( lcsh )
Genre:
bibliography   ( marcgt )
theses   ( marcgt )
non-fiction   ( marcgt )

Notes

Abstract:
ABSTRACT: Fidelity to a program model and innovations added to the program model have been found to positively impact large scale program expansion and implementation (Blakely et al., 1987). Research examining the effectiveness of Healthy Families America (HFA) programs has been hindered by the differences that exist among HFA programs. This study examines the impact program fidelity and innovations have on outcome measures of updated immunizations, primary medical care providers, and participant retention for 102 HFA sites. Factor analysis and regression were used to analyze archival data. Results indicate that fidelity was positively related to percentage of participants with updated immunizations and primary medical care providers. Innovations, specifically additional services offered to participants and additional training opportunities for HFA staff, had positive relationships with participant retention.
Thesis:
Thesis (M.A.)--University of South Florida, 2007.
Bibliography:
Includes bibliographical references.
System Details:
System requirements: World Wide Web browser and PDF reader.
System Details:
Mode of access: World Wide Web.
Statement of Responsibility:
by Ashley E. Nixon.
General Note:
Title from PDF of title page.
General Note:
Document formatted into pages; contains 215 pages.

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
aleph - 001914858
oclc - 179622978
usfldc doi - E14-SFE0001988
usfldc handle - e14.1988
System ID:
SFS0026306:00001


This item is only available as the following downloads:


Full Text

PAGE 1

The Impact of Fidelity and Innovations on Healthy Families America Programs by Ashley E. Nixon A thesis submitted in partial fulfillment of the requirements for the degree of Master of Arts Department of Psychology College of Arts and Sciences University of South Florida Major Professor: Carnot E. Nelson, Ph.D. Stephen Stark, Ph.D. Judith Becker Bryant, Ph.D. Date of Approval: February 23, 2007 Keywords: program implementation, organi zational expansion, multi-site programs, organizational effectiveness, home visiting programs Copyright 2007, Ashley E. Nixon

PAGE 2

Table of Contents List of Tables.....................................................................................................................iii List of Figures....................................................................................................................vi Abstract................................................................................................................................v Introduction..........................................................................................................................1 Implementation........................................................................................................3 Fidelity and Innovations..........................................................................................4 Home Visiting Programs ..........................................................................................6 Research on Home Visiting Programs.....................................................................7 Nurse Home Visitation Program..............................................................9 Comprehensive Child Development Program.......................................11 Hawaiis Healthy Start Program...........................................................12 Healthy Families America....................................................................17 Best Beginnings....................................................................................20 Current Contentions..............................................................................22 HFA Implementation..............................................................................................24 The Current Study..............................................................................................................27 Method...............................................................................................................................31 Programs................................................................................................................31 i

PAGE 3

Measures................................................................................................................31 HFA Credentialing Program Self-assessment Tool.............................31 Annual Site Profile Update..................................................................33 Results................................................................................................................................35 Item Analysis..........................................................................................................35 Factor Analysis.......................................................................................................36 Hypothesis Testing, First Study Goal.....................................................................42 Hypothesis Testing, Second Study Goal................................................................45 Exploratory Analysis..............................................................................................49 Discussion..........................................................................................................................51 Limitations.............................................................................................................54 Directions for Future Research..............................................................................56 References..........................................................................................................................58 Appendices.........................................................................................................................65 Appendix A: The Critical Elements of Healthy Families America.......................66 Appendix B: HFA Credentialing Program Self-Assessment Tool: An Initiative of Prevent Child Abuse America (1999)....................................69 Appendix C: HFA Credentialing Program Self-Assessment Tool: An Initiative of Prevent Child Abuse America (2003)..................................136 Appendix D: Figures............................................................................................212 ii

PAGE 4

List of Tables Table 1 Pattern Matrix Loadings for the Four Fidelity Factors.....................................37 Table 2 Means, standard deviation, and intercorrelations of variables..........................41 Table 3 Regression analysis predicting percentage of participants with primary medical providers from fidelity factors......................................43 Table 4 Regression analysis predicting percentage of participants with updated immunizations from fidelity factors...........................................45 Table 5 Hierarchical regression analysis predicting the percentage of participant with medical provider from Staff Orientation and additional training, services, and staff.......................................................47 Table 6 Hierarchical regression analysis predicting the percentage of updated immunizations from fidelity factors and additional training, services, and staff......................................................................48 Table 7 Unstandardized coefficients, R 2 R 2 and squared semi-partial correlations between additional services, additional training and participant dropout rate.....................................................................50 iii

PAGE 5

List of Figures Figure 1 Scree Plot of Eigenvalues..............................................................................212 Figure 2 Histogram of the Participant Dropout Rate...................................................213 Figure 3 Histogram of the Percentage of Participants with Medical Providers...........214 Figure 4 Histogram of the Percentage of Participants with Updated Immunizations........................................................................................215 iv

PAGE 6

v The Impact of Fidelity and Innovations on Healthy Families America Programs Ashley E. Nixon ABSTRACT Fidelity to a program model and innova tions added to the program model have been found to positively impact large scale program expansion and implementation (Blakely et al., 1987). Research examining th e effectiveness of Healthy Families America (HFA) programs has been hindered by the differences that exist among HFA programs. This study examines the impact program fidelity and innovations have on outcome measures of updated immunizations, primar y medical care provide rs, and participant retention for 102 HFA sites. Fact or analysis and regression were used to analyze archival data. Results indicate that fidelity was posit ively related to percentage of participants with updated immunizations and primary medical care providers. Innovations, specifically additional services offered to participants and additional training opportunities for HFA staff, had positive rela tionships with participant retention.

PAGE 7

Fidelity and Innovations 1 Introduction Effective social programs are constantly a ttempting to expand to new locations in diverse areas. When expansion occurs, some pr ograms or individual sites are successful while others fail. Many researchers point to th e implementation techniques utilized by the organization to explain this discrepanc y. Although there are several successful implementation methods, questions concerni ng effective duplication of the original program and the production of the desired e ffects exist regardless of the method used (Yoshikawa, Rosman, & Hseuh, 2000). A major implementation concern for practiti oners and researchers is adherence, or fidelity, to the original program model with regard to the success of the multi-site extensions. Two common opinions dominate the discussion. One, the profidelity side, maintains that adherence to the model is esse ntial to program effectiveness. The other, the proadaptation side, contends that each site needs the flexibility to reach its clients in various locations and should be able to alter the model to meet their perceived needs. Blakely et al. (1987) found that the new site s had more effective implementation when they maintained fidelity to the original program model. Innovations are original ac tivities added to a progra m model, instead of a variation of the components of the model. Blakely et al. (1 987) also found programs that maintain fidelity to the original progra m model and used innovations were more successful than programs that maintained fi delity but did not u tilize innovations. The

PAGE 8

Fidelity and Innovations 2 current study seeks to examine the impact of fidelity to the model and the effects of innovations specifically designed to meet the needs of a community in which a new site in a multi-site program is located. Healthy Families America (HFA) program sites provide an ideal forum for this investigation. HFA is a voluntary program desi gned to reduce child maltreatment, such as abuse and neglect. The HFA model calls for paraprofessionals to provide parents with referrals, social support, a nd a child development curric ulum through intensive home visiting during the early years of the childrens lives. Flexibil ity was seen as an essential component to HFA expansion and formation. In the past 13 years, HFA has expanded to over 300 sites in regionally and culturally di stinct locations (Martin, 1999). HFA sites currently provide services in 30 states in America and areas in Canada. These sites provide services to numerous populations a nd diverse cultures in diverse communities, suggesting the need for great variability betw een sites to adequately meet the needs of their clients. There is a great amount of variation betw een HFA sites. For example, within the continental United States, HFA sites are attemp ting to tailor their services to address the needs of such diverse groups as Caucasians, African Americans, Asians, Hispanics, and Native Americans, in urban, small town and ru ral settings. According to Blakely et al.s (1987) findings, programs exhibiting high fi delity to the HFA model and adding innovations according to their clients needs will be the most effective. This research

PAGE 9

Fidelity and Innovations 3 project examined implementation, fidelity a nd innovations in a home visitation programs. Therefore, further discussion of eac h of these topics is required. Implementation Large-scale expansion and model replic ation are challenging objectives with numerous complications (Gross, Temk in-Greener, Kunitz, & Mukamel, 2004). Successful implementation requires extensiv e planning and monitoring. People assume that program implementation will be su ccessful under ideal conditions. However, programs generally do not have the luxury of developing under ideal circumstances, so it is necessary to determine whether a part icular program model can successfully be implemented at a particular site. There are several demonstrab ly successful methods for expansion (Yoshikawa et al., 2000). Several of these methods are descri bed here. Staged repl ication refers to a process in which an initial site is set up and ev aluated. If it is determined that the original site is successful, additional sites are established and evaluated. This process is continued as the program continues to grow. Franchis e replication is another endorsed method where a single, central organization governs independent sites. The governing agency establishes the performance standards accordi ng to the model, and then monitors the independent sites adherence to the standards. Another impl ementation method is referred to as a multi-site demonstration. This process consists of developing several sites at once and evaluating all of the sites. It is similar to staged replication, except that it lacks the first step in which only one site is developed.

PAGE 10

Fidelity and Innovations 4 Several difficulties are inherent in pr ogram expansion and implementation. One complication concerns the new sites fidelity to the original model. Has the program at the new site successfully duplicated the origin al program? If it has, does the new site produce the same desired effects that the orig inal site produced? If the new program has duplicated the original successfully and the desired effects are not produced, many different explanations may exist. One possible explanation is that cultural differences in the areas of the two sites are dissimilar and the new site may need to adjust to its community. These challenges pertain to th e adaptation/innovation argument mentioned earlier. Specific paradoxes underlie the complications inherent in program expansion and implementation (Yoshikawa et al., 2000). Fi rst, programs are often tested under ideal conditions, while replication occurs under non-ideal circumst ances. Second, evaluation of programs should require a representative samp le of programs. However, this is not generally feasible. Programs willing to participate in evaluations are the ones evaluated, which may cause sampling bias and under samp ling. Finally, the necessity for flexibility often conflicts with the importance of mainta ining fidelity on critical aspects of the model. Program outcomes are dependent on decisions made concerning fidelity and adaptation. Fidelity and Innovations Historically, the concept of fidelity was developed to clarify psychotherapy research (Bond, Evans, Salyers, Williams, & Kim, 2000). Fidelity gave researchers a

PAGE 11

Fidelity and Innovations 5 mechanism to categorize and define treatme nts, which enabled them to produce more accurate outcome studies of specific therapies. Fidelity gained popul arity for its utility when it was applied to the field of education. In reaction to the Soviet Unions launch of Sputnik before an American satellite, US l eaders blamed the failing education system and pushed for its reform (Paisley, 1973). Res earchers involved with educational research and development began to make recomme ndations. When the recommendation reports reached the educators, the US Office of Education identified difficulties in implementation at the school level (Berman, 19 81). It was believed that, if the validated program recommendations reached the teachers, they could easily be implemented. It became clear that this was not the case when the desired effects were not achieved. Upon examination, many schools were found to have implemented programs differing from the recommended programs, thus failing to produ ce the intended results. Fidelity to the program model was thus recognized as essential in education reform. Researchers have also identified fidelity as critical in program evaluations, particularly for multi-site programs (Paulson, Post, Herinchx, & Risser, 2002). Multi-site programs differ along many dimensions, in cluding culture, f unding, and community involvement. Fidelity provides a mechanism for measuring each programs adherence to the program model, which in turns allo ws for comparison among programs despite differences (Paulson et al., 2002). As previously mentioned, Blakely et al. (1987) empirically examined two implementation theories: fidelity to the model and model adaptability. Researchers who

PAGE 12

Fidelity and Innovations 6 support fidelity believe that altering core aspects of a th eoretical model as a program expands to new areas will diminish th e programs effect. Those who support model adaptability, or reinvention, believe that program s must be able to alter the model to fit the needs of new sites. Blakely et al. (1987) assessed program model, innovations, and effectiveness by conducting program evaluations at each site. The results of this investigation indicated that expanding program effectiveness was positiv ely correlated to fidelity to the model. Blakely et al. also found that program adjustments made by adding to the model correlated with higher effectiveness. Programs that adjusted by altering or modifying the model did not demonstrate a positive relations hip. Thus, maintaining fidelity to the model while adding new components to meet the i ndividual needs of the community appears to be the most effective plan for implementing and expanding the programs examined in this study. Home Visiting Programs Home visiting programs began in order to address the lack of available resources for those in need of these resources, as we ll as to increase individuals awareness of community resources that are available. Th ey are voluntary programs aimed at preventing child abuse and neglect as well as promoting positive parenting by increasing healthy parent child interactions. They also seek to build strong supp ort networks between parents who are seen as at risk for child maltreatment and community support organizations available to assist them.

PAGE 13

Fidelity and Innovations 7 Home visiting programs begin by assessing parents in their designated communities at the time of birth or while the mother is pregnant. Parents identified as being high risk for child abuse or neglect are invited to join the program and receive weekly visits from a trained home visitor. Parents can be identified as at risk for a number of factors. Programs have indi vidualized requirements; many screen for substance use, domestic violence, parents ag e, first pregnancy, financial problems, and maternal depression, but may have other risk factors to wh ich they attend. Participants of these types of programs can remain i nvolved with the program with decreasing intensity for up to five years (D aro & Harding, 1999; Martin, 1999). For most home visiting programs, home vi sitors are paraprofessionals trained by HFA certified trainers primarily in parent child interaction, child development and care. The home visitors are hired based on their abi lity to develop trusting relationships with the families involved with the program, as well as their educational and experiential background. The home visitors of ten work from a curriculum th at aids them in educating the parents about their childs developmenta l progress. This creates a forum for the visitor to model positive parent ing practices. Home visitors also help link the family with appropriate community resources for needed services. Research on Home Visiting Programs Early research on home visitation pr ograms aimed at preventing child maltreatment was promising; particularly early research examining Hawaii Healthy Families (Guterman, 1997). Consisting mainly of quasi-experimental, pretest-posttest

PAGE 14

Fidelity and Innovations 8 studies, early examinations of the programs i ndicated positive effects. One concern about early studies was the modest or small sample sizes; many lacked the statistical power to detect intervention effects (Guterman, 1997) While cautiously r eceived due to the limited research designs and scopes of the studies the initial data were seen as promising enough to warrant home visitation program expansion (Gomby, 1999). Home visitation programs, modeled pr incipally on Hawaiis Healthy Start Program, expanded rapidly, as did the body of research examining the effects of the programs. Research reviewing the developing pr ograms remained relatively consistent as to design, but the results covered the gamut fr om positive effects to no identifiable effects to negative effects (Guterman, 1997). These fi ndings led researchers to reconsider their outcome measures and how these measures may confound results. One obvious outcome measure was to evalua te intervention effects on child abuse and neglect directly by cons idering reports filed with child abuse agencies. The unfortunate flaw in this r easoning is that many child abus e and neglect cases are never reported to public agencies (Olds, Eckenr ode, & Kitzman, 2005). Participants in home visitation programs were frequently observed an d were more likely to have a child abuse or neglect report filed. This biased reporting probably led to an underestimation of the effectiveness of the home visiti ng programs (Olds & Kitzman, 1993). Researchers began to look at proxy m easures as a way of overcoming the outcome confounds resulting from direct measurements. Common proxy measures included observed parent-child interaction, pa rental attitudes, and medical indicators.

PAGE 15

Fidelity and Innovations 9 These outcome measures yielded slightly be tter but relatively similar results for home visitation programs, with 50% of the studies demonstrating positive effects and others finding no significant effects. Programs that emphasized health care curriculum for the parents and measured medical indicators as outcome variables demonstrated the most consistent positive effects (Guterman, 1997). Due to the limitations of studies publi shed before the early to mid 1990s, researchers identified the need for large-scale studies and trials that examined several outcome measures to avoid the statistical and measurement confounds plaguing their research (Guterman, 1997). At this point, ma ny large-scale longitudinal studies with rigorous scientific methodologies were underway and the results of these studies began to be published in the following years (D aro & Harding, 1999). The following are summaries of several prominent longitudina l studies on child maltr eatment intervention programs. Nurse Home Visitation Program The Nurse Home Visi tation Program, NHVP, was established in 1977 as a research study and demonstration in Elmira, New York. NHVP employs only nurses as home visitors. Participants in NHVP are required to meet three criteria; they must be less than 19, a single parent, and of low socioeconomic status. When a new NHVP site is started, it is requir ed to establish its program as an exact replica of the sites that are be ing researched. Program developers believe that this fidelity to the model will ensure similar results in the new locations (Hill, 1999). NHVP attempts

PAGE 16

Fidelity and Innovations 10 to improve pregnancy outcomes, child health and development, as well as the financial self-sufficiency of the families it serves. Olds, Henderson, Tatelbaum, et al. publishe d their initial article in 1986. Olds et al. (1986) found significant diffe rences between families that were and were not visited by NHVP nurses. During the first two years, mothers visited by the nurses had fewer verified incidents of child abuse and negl ect and demonstrated improved child-parent interaction (Olds et al., 1986). In foll ow up studies, Olds et al. (1994, 1995, 1997, 1998) found that the mothers receiving regular visits from nurse s demonstrated significantly less criminal behavior, less welfare use, a nd fewer subsequent pregnancies. Improved psychological well-being seems to be a longterm effect for adolescents who received nurse home visits as childr en (Olds et al., 1998). NHVP and Olds et al.s re search was duplicated in Memphis, Tennessee. The results from this study also demonstrated im proved maternal mental and physical health and improved parent-child interactions (Olds et al., 1999). Olds et al (2002) have also been conducting research with an NHVP site in Denver, Colorado. This study examined treatments delivered by nurses and treatments delivered by paraprofessi onals. Olds et al. found that the nurse-visited gr oup showed similar results as the Elmira and Memphis programs, but the paraprofessional-visited group only showed signifi cantly better parentchild interactions. On outcome measures effected by either visitor, the nurse-visited condition averaged twice the effect si ze as the paraprofessional condition.

PAGE 17

Fidelity and Innovations 11 In a follow up study, the mothers in the paraprofessional home visitor condition and the nurse home visitor condition dem onstrated better social adjustment, psychological well-being, and improved parent-c hild interaction than the mothers in the control group (Olds et al., 2004). Olds et al concluded that nurse home visitors have more immediate and long-term impact on mo thers and children than paraprofessional home visitors. They also c oncluded that home visitation programs should concentrate on high-risk populations, where their e ffects are the most evident (2004). Comprehensive Child Development Program In the late 1980s, the Comprehensive Child Development Program, CCDP, was funded as a demonstration project by the Administration on Children, Youth and Families, a branch of the U.S. Department of Health and Human Services. The goals of CCDP were to enhance the physical, social, emotional, a nd intellectual development of children from low-income families, provide support for their families, and help the families become financially selfsufficient (St. Pierre et al. 1997, St. Pierre & Layzer, 1999). CCDP employed paraprofessionals to deliver early childhood cu rriculum to the families during the visits and provide case management (St. Pierre et al. 1997, St. Pierre & Layzer, 1999). CCDP directors arranged for weekly review meetings for all caseworkers with supervisors as well as providing additional assistance from the CCDP professional staff in areas such as health and mental health coordination, early childhood specialists, employment, and adult education (St. Pierre et al. 1997, St. Pierre & Layzer, 1999).

PAGE 18

Fidelity and Innovations 12 CCDP did not have any overall positive eff ects on families over the five years the program ran and was observed (St. Pierre et al. 1997, St. Pierre & Layzer, 1999). One of the 24 CCDP sites demonstrated statistically significant positive effects on childrens cognitive development, families employment, income, and use of federal benefits, as well as parenting attitudes (St. Pierre et al. 1997, St. Pierre & Layzer, 1999). The population at this site had a lower over all level of risk than many other populations examined, high levels of state provided suppor t for the families, and was located in a small city in a rural area, which may ha ve decreased the stereotypes limiting the connection between participan ts and the case managers. Additionally, the site was organized through the school board and had a clear focus on childre ns education (St. Pierre & Layzer, 1999). Ryan et al. (2002) corroborated these results in Pittsburgh. Parents who chose parenting and child goals for their service plans were related to higher mental scores for the children (Ryan et al., 2002). Hawaiis Healthy Start Program Hawaiis Healthy Start Program (HSP) began in 1975 with the aid of the Nati onal Center for Child Abuse and Neglect, NCCAN. The program consisted of early identification of families at risk for child abuse and neglect and home visitations by trained paraprofe ssionals (Duggan et al., 1999, 2000, 2004). An early study indicated child abuse and neglect incidences for this group were minimal when compared to comparison group rates fr om other studies. While considered very promising at the time of its reception, it wa s later thought to have been given too much

PAGE 19

Fidelity and Innovations 13 credence due to flawed methodology, such as a single group pretes t/posttest design and effects of statistical regre ssion (Duggan et al., 1999). Despite the concerns over the pilot st udies design, the positive findings were influential enough for HSP to gain additional and continued support, as well as receive national interest. Home visitation as a me thod for child maltreatment was recommended in a report from the U.S. General Accounti ng Office (1990), and HSP was identified as a positive strategy by the U.S. National Advi sory Board on Child Abuse and Neglect (1991). The attention the HSP gained led to an evaluation by NCPCA, which consisted of a one year randomized trial of the Healthy Start Program. This study assigned families to a control group or a treatment group; the fi ndings indicated that the treatment group had significantly less substantiated reports of child maltreatment. These results were plagued by concerns about dropout in the cont rol and home visitation groups, non-blind evaluators, and complications in evaluati ng certain outcome measurements (Duggan et al., 1999). The Hawaii Medical Association and The John Hopkins University collaborated with the Hawaii Department of Health to produce a rigorous randomized examination of HSP. The evaluation was set up to answer f our questions: Does the program conform to the HSP model? Does fidelity to the model effect outcomes? Does the program achieve the desired outcomes for children and parents? How do outcomes compare with program costs (Duggan et al., 1999)?

PAGE 20

Fidelity and Innovations 14 The researchers collected extensive data through many divers e approaches. They used structured interviews with the mothers, developmental testing of the children, as well as home observation of environment and pa rent-child interaction and archival data (Duggan et al., 1999, 2004). The researchers avoi ded evaluation biases by maintaining an independent staff, not associat ed with HSP, to collect the data. The data were analyzed and reported in a series of articles published in peer-reviewed journals from 1999 to 2004. The first article published by Duggan et al (1999) concerning the previously described study analyzed the data findings af ter the first two year s of a three-year examination. The second paper published in 2000 focused on the challenges involved in the identification, engagement, and service de livery to the families involved in the study. Attrition was identified as an early challenge for HSP, as 51% of the families had left the program within the first year. Although attr ition rate for home visiting programs is generally high, this rate was higher than av erage. Participating families received an average of 22 visits in the first year, a time when they we re supposed to receive weekly visits (Duggan et al., 1999). In 2004, Duggan et al. published their findings based on all three years of evaluation. Analysis revealed that there was not a significant difference between the HSP group and the control group in regards to nonviolent discipline, neglect, minor physical assault, severe physical abuse, psychol ogical aggression, substantiated CPS reports, hospitalizations, or maternal relinquishment of primary care giver role (Duggan et al.,

PAGE 21

Fidelity and Innovations 15 2004). There was not a significant reduction in a ny malleable risks, such as poor maternal mental health, substance use, and partner abuse, for HSP parents when compared to control parents. There was also not a significan t difference between groups in the parents interest in or use of co mmunity services that addr essed the risks measured. Another finding from this investigation wa s that home visitors rarely, if ever, identified situations in which a child was at risk or a victim of severe physical abuse. Based on self-report measures completed by mothers in the HSP group, home visitors identified between 4% and 9% of frequent and severe child abuse incidents (Duggan et al., 2004). There are many important factors to consid er when interpreting these results. One analytical decision by Duggan et al. that had th e potential to distort their results was the treatment of early drop out families. When families stopped receiving services from HSP, they were still considered a part of the HSP group during analysis. Duggan et al. (1999, 2000) reported that 51% of families stopped re ceiving services by the end of the first year. This means that 51% of the treatment group received a year or less of a three year treatment program; yet the data were evaluated as if they had received the anticipated treatment. The families who continued services received fewer visits than anticipated in the original model. Both of these factors may have diminished the impact of HSP in Duggan et als results Other factors may have impacted Duggan et al.s analysis of the data. Many of the control group families were able to access other community resources and services for

PAGE 22

Fidelity and Innovations 16 child and family support (Duggan et al., 2000). Due to this, the study may actually be comparing different family support service orga nizations instead of the impact of families support services in general. Another compli cation for Duggan et al. was the extremely low rate of substantiated abuse and neglect cas es, four in total, for the control group. This rate is well below national average and the extremely small sample size became problematic in statistical anal ysis (Duggan et al., 2004). Contra rily, this was a very large, methodologically rigorous study with plenty of power to detect small differences between groups. Duggan et al. (2004) proposed inadequacies of program implementation and fidelity to the model as key factors for HSP failure to establish significant effects with the families involved. The primary function of the home visitor as established by the HSP model was to develop rapport with a family, he lp identify family needs, and assist the family in finding community resources to address those needs. Training for HSP employees, however, consisted of developm ental knowledge acquisition rather than teaching and developing the skills HSP empl oyees would need to identify and address family risks. Program requirements for em ployee educational and experiential history were not met (Duggan et al., 2004). Formal arrangements were not established with available community service orga nizations to facilitate the linking of families to services. The combination of the lack of skills training and formally arranged community services may have led home visitors to try to assi st the families in isolation, which was never intended in the original progr am model (Duggan et al., 2004).

PAGE 23

Fidelity and Innovations 17 Finally, the HSP model was compromised by meeting the requirements of funding sources (Duggan et al., 2004). The program philosophy was shifted away from risk reduction by funding requirements oriented to parent development as well as program turnover. Directors and supervisors familiar w ith the risk reduction origins of HSP were gradually replaced with managers trained in the strengths based model centered on family goal setting (Duggan et al., 2004). The result of this model modulation is that the HSP and home visitors did not address or impact identified risks for child maltreatment, such as maternal depression, parental substa nce use, and parental physical abuse. Healthy Families America. The format of HFA is primarily based on Hawaii Healthy Start Program and other existing leading family support programs at the time of its inception, but is not an exact replica (Martin, 1999). For instance, since HFA provides services to a much larger area than Hawaii Healthy Start, site flexibility has been integrated into the HFA model to a greater extent than the Hawaii Healthy Start model. Specifically, while HFA site screen all families in its intended population like Hawaii Healthy Start, each HFA site is allowed the fl exibility to define its intended population. In 1994, PCA America established the HF A Research Network, a collaborative group of researchers examining the HFA progr am. Daro and Harding (1999) discuss the findings of 17 completed projects by the HFA Research Network. HFA programs had only been established for three years, on averag e, at the time of res earch publications, and it was observed that the programs had not yet achieved the level of community change or social service integration the HFA program model indicated was optimal for reducing

PAGE 24

Fidelity and Innovations 18 child maltreatment and improving family f unctioning. The outcome measures examined were generally compared to demographic comparison groups, as opposed to randomly assigned control groups (Daro & Harding, 1999). Thirteen studies reported HFA families ha d abuse report rates of about 6%, which the researchers estimated to be about half th e rate of families not involved with HFA but that are similar demographically to HFA families (Daro & Harding, 1999). These findings have not been duplicated in st udies with a randomized control group. Other findings reported were that 94% of HFA fa milies were receiving proper and appropriate health care. These findings were strongest for families who entered the program before the birth of the target child as opposed to after the child was born. Researchers also reported that less than 10% of the children enrolled in HFA suffered from developmental delays. In randomized studies there has not been a signi ficant difference identified between developmental progress of children enrolled with HFA and children not enrolled (Daro & Harding, 1999). A particularly positive area in the HFA research is improvements in parent child interaction and parental skills. HFA parent s indicated in quantitative surveys and qualitative interviews that they felt less stre ss and were less likely to abuse their children than parents not enrolled in HFA. Several programs have demonstrated an improvement in maternal life course, as indicated by the families reduced use of public assistance and housing. None of the studies reviewed reported a difference in social support use in HFA and non-HFA families. In addition, the research ers were not able to identify target

PAGE 25

Fidelity and Innovations 19 populations that might benefit more by part icipation in the HFA program. Different aspects of the HFA program positively impacted diverse populations (Daro & Harding, 1999). High attrition rates have plagued the HFA program and many other home visitation programs have reported similar high rates of attrition. Many factors have been identified as contributing to attrition, incl uding characteristics of the parent, family, provider, and the living circumstances of the family (McCurdy & Daro, 2000). McCurdy, Gannon, and Daro reported in 2003 that older pa rticipants, those w ho started HFA early in their pregnancy, and unempl oyed participants were more likely to remain active with HFA for longer periods and to receive more visits than other participants. In a recent report by Daro, McCurdy, a nd Nelson (2005), HFA engagement and retention was examined. In a retrospective look at enrollment and retention patterns, participant characteri stics related to longe r enrollment and a greater number of home visits were race (African American or Hi spanic), age (older), unemployment, school enrollment, and service provision during pr egnancy. Younger home visitors and those with greater experience were also associated with part icipant duration. In addition, programs that maintained lower case loads for their home visitors, and were able to match participants and home visitors on r ace and parenting status demonstrated longer retention and higher numbers of home visits. Daro, McCurdy, and Nelson (2005) then examined these retrospective findings with new families approached by HFA for predictive value. Providing prenatal services

PAGE 26

Fidelity and Innovations 20 was associated with participant interest a nd acceptance of social services. Perceived infant risk was also related to enrollment, and many of the families declining services expressed reluctance to allow home visitor scruti ny into their homes at such an intimate level. Participant retention was predicted by perceived infant risk, participants social networks support of program, participants relationship with the home visitor, participants belief in the programs efficacy, and community distress and disorganization. The relationship between the home visitor and the participant was strongly related to participant retention and receipt of se rvices. Particularly, African America home visitors had difficulty maintaining Caucasia n and Hispanic participants (Daro et al., 2005). However, Daro et al. also point out a substantial portion of African American home visitors did not have college degrees, wh ile the majority of Caucasian home visitors had at least a bachelors degree. Even though th ere was not a direct relationship between educational level and retention, educational level may still aff ect participants perceptions of home visitor competency. This issue need s more attention, as does the importance of comfortable home visitor and participant matches. These findings imply that cultural relevance and sensitivity should be addressed in hiring of and training provided for home visitors. Best Beginnings. Best Beginnings is a HFA site that was established in 1994 in New York. A recent report by Anisfeld, Sandy, and Guterman (2004) evaluates the programs effectiveness. Families who qualif ied for the Best Beginnings program were

PAGE 27

Fidelity and Innovations 21 randomly assigned to a program group or a c ontrol group. The control group received biyearly home visits and referrals when services were needed. The program group received the typical HFA intervention, beginning with intensive weekly visits that became less frequent over the two year pe riod (Anisfeld et al., 2004). Anisfeld et al.s (2004) study asked if Best Beginnings was able to effectively achieve its four core goals: 1) to assess families for strengths and needs and provide referrals as needed. 2) To enhance mate rnal psychosocial functioning and improve maternal life-course/self-suffi ciency. 3) Promote positive parent-child interaction. 4) Promote healthy childhood growth and developm ent. Anisfeld et al. also examined the extent to which the intervention had been im plemented as specified in the protocols. In regards to Goal 1, Anisfeld et al. found that The Kempe Family Stress Inventory was successful at identifying appr opriate participants for Best Beginnings. Paraprofessionals were able to identify the needs of the families and properly refer services. Families participati ng in the program group were more likely than the control group to receive certain community services (Anisfeld et al., 2004). Goal 2 examined maternal psychosocial functioning and mate rnal life course. Mothers participating in th e program group were significan tly more likely to advance their education while in the program (Anisf eld et al., 2004). There was not a significant program effect for employment or participa tion in public assistance programs. Analysis indicated that there were no significant program effects on ma ternal psychosocial functions. Maternal depressive symptoms were found to be related to the mothers

PAGE 28

Fidelity and Innovations 22 perceived social support network. There were also no significant program effects for parent-child interaction, Goal 3 for Best Beginnings (Anisfeld et al., 2004). Goal 4 concerned child health and devel opment. Mothers who received prenatal Best Beginnings invention were significantly more likely to exclusively breastfeed and have a regular primary care provider at intake and 24 months. There were no group differences between immunization rates. Part icipants in the program group were more likely to visit the Pediatric Emergency De partment, and were more likely to have referrals then the control group. At 24 m onths, boys in the program group scored significantly higher scores on de velopmental screenings than the children in the control group; Anisfeld et al. (2004) indicated that the intensive intervention appears to have raised the development of boys, on average, to that of girls. As for the implementation, Best Beginnings paraprofessionals were able to adhere to protocols. They made an appropriate number of visits participated in proper t ypes of activitie s, and made appropriate referrals for the families (Anisfeld et al., 2004). Current Contentions. In a commentary, Mark Chaffin (2004) asks the question Is it time to rethink Healthy Start/Healthy Fam ilies? Chaffin laments the lack of FDA-style evidentiary regulations for psychosocial prevention interventions, comments on the weakness of non-rigorous experimental resear ch findings, and argues for disregarding a large number of such studies in favor of a few studies that have rigorous research designs. He also contends that direct measures of child abuse and neglect are the only adequate measures, despite many researchers concerns th at these findings are vulnerable to bias as

PAGE 29

Fidelity and Innovations 23 previously discussed. He argues that a recen t CDC report was mistaken when its author considered evidence from studies that did not meet the standards Chaffin believes should be set. Chaffin suggests that HS/HFA revamp their models to focus on the three most robust risks for child maltreatment: partne r violence, drug abuse, and maternal depression. He also suggests that program designers reconsider home-visiting as a method of treatment delivery. Chaffin states that it was previously believed home-visiting would lead to higher retention rates but he argues that higher rete ntion rates have not been reached by using this method. Chaffins commentary prompted several le tters to the Editor in response. In a response from the authors of the CDC re port in question, Hahn, Mercy, and Bilukha (2005) provide information on the continued debate concerning quality of design. They reminded readers that different experimental methods were designed to compensate for other experimental methods weaknesses, all methods do have weakness, and research is more robust if it has a variety of experimental designs contributing to it. They also discuss the process by which they evaluated re search for the CDC meta-analysis, offering considerable research to subs tantiate their decisions. In another letter to the ed itor, Olds, Eckenrode, and Kitzman (2005) agree with Chaffin that the field of home-visiting ch ild maltreatment prevention programs should continue to produce rigorous experimental evaluations to improve program service and development. The authors extensively disc ussed why they believe substantiated CPS reports, as an outcome measure, are fraught wi th bias and are an incomplete measure of

PAGE 30

Fidelity and Innovations 24 program effectiveness. Oshana, Harding, Friedman, and Holtons response addresses Chaffins allegations that HFA is based on temporary fades taken up by advocacy groups. They assert that the HFA was developed and is constantly being reevaluated by a panel of scientists, not advocates (Osh ana et al., 2005). They conte nd that HFA is continuously rethinking its program model as new inform ation, such as that provided by Duggan et al. (2004), becomes available. They point to The Network as evidence that continual improvement is a standard that has been a principle of HFA si nce its inception. The response also discusses met hodological differences between psychosocial and medical experimentation, which makes the FDA-style testing extolled by Ch affin less practical and informative than his commentary indicated. In a response submitted by Daro (2005), she expands upon the ideas expressed in Chaffins editorial, indicating that his estimati on of the research and field in general was quite narrow. Daro reiterates the scientific origin of the home visiting model and its wide ranging support from government officials, scientists, advocates, and even economists. She then discusses what she, and the ma jority of authors responding to Chaffins editorial, perceives as the dangers of li miting the acquisition and use of valuable information by utilizing one-dim ensional experimental design. HFA Implementation One major challenge researchers attempting to evaluate the effectiveness of HFA programs encounter are the programmatic differe nces between sites. As is typical in program expansion, some sites report positive outcomes, while others produce neutral

PAGE 31

Fidelity and Innovations 25 results. When examined at the program level, as opposed to studying the individual sites, the differences between sites may account for more variance in program outcomes that the effect of the HFA program and curriculum. The differences between site variations could be making it difficult to identify positive program effects. To maintain the integrity of new HFA s ites while allowing them the flexibility necessary to effectively serv e their populations, PCA America developed a credentialing process. Twelve critical elements of the original program paradigm were defined as essential to the programs desired outcome s (Appendix A). These 12 elements were identified through a thorough review of guiding pr inciples and theories in pertinent fields as well as relevant research and practice outcomes (Daro & Harding, 1999). A new program must successfully meet expectations on all 12 of the critical elements to be certified as a HFA site. PCA Americas creden tialing process attemp ts to assure each HFA site adheres to these elements and mainta ins a certain level of similarity to other HFA programs and the original HFA model. In a recent presentation Kessler and Nelson (2005) examined HFA program effectiveness while accounting for program matic differences in implementation. The relationship between 103 HFA sites fidelity to the HFA model and the sites positive outcomes on measures of updated immuniza tions and primary care physicians was examined. Fidelity was defined as the sites a dherence to 11 of the 12 critical elements of the HFA model and was measured using the or iginal evaluation panels ratings. Element number seven was not included in the evalua tion of fidelity because it concerned medical

PAGE 32

Fidelity and Innovations 26 outcomes, which were used as outcome measur es. The results indicated that there was a significant positive relationship between the percentage of children with updated immunizations records and the sites fidelity rating. The relationship was particularly strong for element 9, which concerns the select ion of staff. A signi ficant relationship was not found between fidelity ratings and the pe rcentage of children who had a primary care physician, nor were the two outcome measures correlated. The present study attempted to improve upon the analysis reported by Kessler and Nelson (2005) in several ways. First, program fidelity was defined as the sites adherence to elements emphasizing implementation on th e credentialing tool. Elements emphasizing implementation manifest themselves in asse ssable outcomes rather than immeasurable recommendations and requirements. Items concerning training, hiring, and evaluation were considered implementation oriented. Items addressing possession of manuals and recommendations for advisory board memb ers were not considered implementation oriented and were not analyzed in this study. Secondly, this study used the 140 second and third order elements instead of the 12 critical elements. This allowed for a much more through examination of the implementatio n fidelity of the sites. A third addition to this study was the additional outcome variab le, participant retention. As mentioned, participant drop out rate is high for home visiting programs and has been identified as a serious problem for HFA progr ams. Thus it is a valuable outcome measure to assess program effectiveness. Finally, this study at tempted to analyze the effects of several program innovations in addition to pr ogram fidelity to the model.

PAGE 33

Fidelity and Innovations 27 The Current Study The first goal of this study was to develop a scale to measure program fidelity using the second and third order elements of the HFA credentialing tool. Item analysis was used to identify and remove items that did not demonstrate ade quate variance across sites. Exploratory factor anal ysis was used to identify pr imary underlying constructs of the fidelity items. Based on this analysis, sc ales were developed representing the various factors. Once these scales were developed, the impact of program fidelity on outcome measures of up-to-date partic ipant immunization records and participant use of medical care providers was examined. This analysis allowed for a comparison of the newly developed scale of site fidelity and the meas ure utilized in the presentation by Kessler and Nelson (2005). Accounting for all sec ond and third order elements emphasizing implementation was expected to increase the correlations between fidelity and the outcome variables over those that were re ported by Kessler and Nelson (2005), whose original study examined all elements regardless of whether they were implementation oriented. This reasoning led to the following hypotheses: Hypothesis 1: Program ratings on fidelity will correlate positively with the percentage of participants w ho have medical care providers. Hypothesis 2: Program ratings on fidelity will correlate positively with the percentage of participants w ho have up-to-date immunizations.

PAGE 34

Fidelity and Innovations 28 The second goal of this study was to examin e the effects of innovations that have been implemented by sites. Data were gathered from annual site surveys on individual sites use of additional employees, such as nurses, additional services, such as social gatherings for families involved with the HF A site, and additional training opportunities for FSWs and supervisors. The relationships between innovations and program fidelity with outcome measures of up-to-date particip ant immunization records, participant use of medical care providers, and participant drop out rate were then examined. According to Blakely et al.s (1987) researc h, the best results for HFA sites will be achieved when the site maintains high fidelity to the program model and when it has added innovations to meet the needs of its community. The innovations identified here, additional training, services, and staff, may addr ess the needs of the populati ons served by the HFA sites using these innovations and ma y moderate the relationship between fidelity and the outcome variables. This reasoning led to the following hypotheses: Hypothesis 3 : Program sites that demonstrate hi gher fidelity to the HFA program model and provide additional tr aining opportunities to thei r FSWs and supervisors will have a greater percentage of participants who have medical care providers than will program sites that do not demonstrate fideli ty to the HFA program model or provide additional training opportunities to their FSWs and supervisors. Hypothesis 4 : Program sites that demonstrate hi gher fidelity to the HFA program model and provide additional services to th e families involved with their program will have a greater percentage of participants who have medical care providers than will

PAGE 35

Fidelity and Innovations 29 program sites that do not demonstrate fideli ty to the HFA program model or provide additional services to the families involved with their program. Hypothesis 5 : Program sites that demonstrate hi gher fidelity to the HFA program model and provide additional staff services no t required by the HFA model, such as child development specialists, will have a greater pe rcentage of participants who have medical care providers than will program sites that do not demonstrate fidelity to the HFA program model or provide additional staff services. Hypothesis 6 : Program sites that demonstrate hi gher fidelity to the HFA program model and provide additional tr aining opportunities to thei r FSWs and supervisors will have a greater percentage of participants who have up-to-date immunizations than program sites that do not demonstrate fideli ty to the HFA program model or provide additional training opportunities to their FSWs and supervisors. Hypothesis 7 : Program sites that demonstrate hi gher fidelity to the HFA program model and provide additional services to th e families involved with their program will have a greater percentage of participants who have up-to-date im munizations than will program sites that do not demonstrate fideli ty to the HFA program model or provide additional services to the families involved with their program. Hypothesis 8 : Program sites that demonstrate hi gher fidelity to the HFA program model and provide additional staff services no t required by the HFA model, such as child development specialists, will have a greater percentage of participants who have up-to-

PAGE 36

Fidelity and Innovations 30 date immunizations than will program sites that do not demonstrate fidelity to the HFA program model or provide additional staff services. Hypothesis 9 : Program sites that demonstrate hi gher fidelity to the HFA program model and provide additional tr aining opportunities to thei r FSWs and supervisors will have a lower participant drop out rate than will program sites that do not demonstrate fidelity to the HFA program model or provide additional training opportunities to their FSWs and supervisors. Hypothesis 10 : Program sites that demonstrat e higher fidelity to the HFA program model and provide additional services to the families involved with their program will have a lower participant drop out rate than will program sites that do not demonstrate fidelity to the HFA program mode l or provide additional services to the families involved with their program. Hypothesis 11 : Program sites that demonstrat e higher fidelity to the HFA program model and provide additional staff services not required by the HFA model, such as child development specialists, will have a lower participant drop out rate than will program sites that do not demonstrate fidelity to the HFA program model or provide additional staff services.

PAGE 37

Fidelity and Innovations 31 Method Programs This sample consists of 102 HFA sites that were reviewed for accreditation between the years of 1998 and 2003. The same sites that were examined by Kessler and Nelson (2005) were used for this study so the results would be dire ctly comparable. Sites were excluded from Kessler and Nelson (2005) if the site had not undergone the credentialing process or if the site was missi ng relevant outcome data. An additional site was removed from the present study because the credentialing file was unavailable. Measures HFA Credentialing Program Self-assessment Tool The 12 critical elements of the HFA program were established based on exte nsive research in the field of child abuse and neglect. In 1999, the HFA Credentia ling Program Self-assessment Tool was developed based on the 12 critical elements (Appendix B). One hundred and forty items were identified to measure the programs adhe rence to the critical elements. Each item was scored by peer-reviewers of the program on a Likert-type scale ranging for to . A score of indicates outstanding performance, a score of indicates good performance, and a score of indicates a need for improvement in that area. The HFA Credentialing Program Self-assessment T ool was updated in 2003 (Appendix C). The purpose and content of the updated HFA Cred entialing Program Self-assessment Tool

PAGE 38

Fidelity and Innovations 32 were not altered. However, the wording and to tal number of items differ. Data from 58 sites were collected using the 1999 HFA Cr edentialing Program Self-assessment Tool, while data from 44 sites were collected using the 2003 version. The credentialing process begins with a self-study performed by the individual sites. After the self-study is completed, two or more trained peer re viewers review it. The peer reviewers also examine the sites file s and conduct interviews with the staff and clients to assign a rating to the items on th e HFA Credentialing Program Self-assessment Tool. Sites are able to respond to all items the peer reviewers ranked at a , which indicates the site must address and improve on this item to be credentialed. A panel of subject matter experts (including state repres entatives for Prevent Child Abuse America (PCAA), program managers, trainers, research ers, and PCAA Board of Directors) makes a final decision on item ratings. If the pr ogram has not met the minimum threshold dictated by the credentialing process, it is deferred and provided recommendations for improvement. The site is given a specified amount of time to make corrections and submit improvements to the panel. If the si te has met the minimum requirements, it is awarded certification for a period of four years. The data from the HFA Credentialing Program Self-assessment Tool were used to develop a measure of program fidelity to the HFA program model. These data were provided by PCAA. Second and th ird order elements pertaining to implementation, as opposed to policy requirements, were extracte d from the total set of second and third

PAGE 39

Fidelity and Innovations 33 order elements. The remaining items were ex amined using item analysis and factor analysis to develop a scale of fidelity. Additionally, percentage of children with updated imm unizations (Credentialing item 7-2.B) and percentage of children with primary care physicians (Credentialing item 7-1.C/7-1.D) were reported in the HFA Cred entialing Program Self-assessment Tool. To improve the accuracy of their findings, Kessler and Nelson (2005) used the actual percentages of immunized child ren and children with primar y care physicians instead of the ranges used by the HFA Credentialing Pr ogram Self-assessment Tool to categorize programs. The HFA Credentialing review pane l verified the data used. On the few occasions when there was a discrepancy betw een the sites reported percentage and the review panels calculations, Kessler a nd Nelson (2005) used the review panels calculations. The verified percentages we re used in this study as well. Annual Site Profile Update. Each year, PCAA sends a survey to HFA sites to collect data concerning many aspects of the si tes functioning and characteristics. These surveys provide information such as the sites funding information, information on participants and staff, and services offered by the sites. Data were collected using the Annual Site Profile Update in 1998, 1999, 2000, 2001, and 2003. Data were not collected in 2002 because the 2001 survey was not dist ributed until late in 2001. The wording of surveys varied slightly by year, but simila r information was solicited in each version. Response rates for the Annual Site Profile Update vary between 75% and 85%.

PAGE 40

Fidelity and Innovations 34 For the current study, the Annual Site Pr ofile Update was used to provide information on innovations; specifically, it provided information on additional training provided to site staff, additional services provided to HFA site participants, and additional types of staff members available to assist HFA site participants. The total number of additional staff members, training opp ortunities, or services reported by a site were used as the sites scor e on the corresponding variable. Survey data regarding participant drop out rate was also extracted from the Annual Site Profile Update. Data were taken from the survey information collected the year the site was under review for accreditation so that th e data coincide with the measurement of fidelity. A participant was cons idered to have dropped out if he or she was not receiving service one year after they enrolled in the HFA program.

PAGE 41

Fidelity and Innovations 35 Results Item Analysis Initially, 90 second and third order elemen ts were chosen from the 140 total items based on their implementation orientation as described earlier. From these 90 items, 11 items were removed because fewer than 41 programs had been evaluated on the item and 20 items were removed because they showed low variance ( 2 < .40). Hiring process items were added on the 2003 credentialing tool. Thus 41 sites that went through credentialing before this addition were missing the relevant data. This established a natural break in the data with either 41 or fewer programs being evaluated on an item and 92 or more programs being evaluated on an ite m. I removed the items representing 41 or fewer items to account for this occurrence. A rating of was emphasized in this proj ect due to the fact that denotes an inability of the site to meet the requirement s for the item. As previously mentioned, items that receive s are then further examined by the credentialing panel. Therefore, a rating of is distinct from a rating of or and is verified as inadequate by the credentialing panel. To account for this aspect of the credentialing process, I included an additional requirement that at least 12 programs received a on each item. Twelve programs were chosen as a threshold due to a natural gap that existed between items that had at least 12 ratings of and items that had more than 19 ratings of . Three

PAGE 42

Fidelity and Innovations 36 additional items were removed because fe wer than 12 programs received 1s on the element, even though the variance for each of these items was slightly above .40. Fiftysix items were retained for further analysis. Factor Analysis Exploratory factor analysis was used to further examine the data. This analysis is constrained by the low number of programs ( N = 102) used in this study. Due to the already low sample size, missing data were replaced with the means for the items. Examination of the scree plot (Figure 1) indi cated that four factor s appeared to account for a large amount of shared variance; the eigenvalues leveled off after 4 factors. Maximum Likelihood extraction method was used to analyze a fou r-factor model. I expected these factors to be correlated, given the nature of the credentialing items, so I used a Promax oblique rotation me thod with Kaiser normalization. The four factors examined accounted for 53% of the variance in the data. An item was considered to load on the factor when the pattern coefficient was greater than .40 and it was the highest pattern coefficient for th at item. Eleven items had similar pattern coefficients for multiple factors and were not included in any factor. Factor loadings can be found in Table 1. The item notation pres ented in Table 1 coincides with the 2003 credentialing tool.

PAGE 43

Fidelity and Innovations 37 Table 1 Pattern Matrix Loadings for the Four Fidelity Factors Item Number Factor 1 Factor 2 Factor 3 Factor 4 10.4 A .754 .122 10.4 C .662 .135 10.4 D .596 .114 10.4 E .683 10.4 F .725 .105 10.5 A 1.081 -.100 -.131 10.5 B 1.117 -.140 -.127 10.5 C .700 -.141 .140 10.5 D .752 -.153 .160 10.5 E .773 -.104 10.5 F .743 .185 1.2 B .662 1.2 C .586 3.4 B .740 3.4 C .638 5.3 .139 .432 .221 5.4 A -.203 .717 -.103 .111 5.4 B -.177 .914 -.112 5.4 C -.245 .903 -.133 5.4 E .579 11.1 B .432 11.2 A .411 .142 .126 Note. Factor 1 Staff Training; Factor 2 Qu ality Control; Factor 3 FSWs Activities; Factor 4 Staff Orientation. Item loadings less than .10 have not been included in this table. Item numbers coincide w ith 2003-creditialing tool. Table Continues

PAGE 44

Fidelity and Innovations 38 Table 1 Continued Item Number Factor 1 Factor 2 Factor 3 Factor 4 11.2 B .400 .156 11.5 B .153 .402 GA.5 D .431 .182 .154 GA.5 E .167 .492 .114 4.1 B .155 .438 4.2 B .155 .408 -.134 6.2 A -.106 -.201 .819 .214 6.2 B -.331 .834 .224 6.2 D -.199 -.146 .794 .223 6.2 E .797 6.2 F .765 7.1 B -.169 .662 7.3 A .267 .448 -.105 7.3 B .160 .434 10.1 B .137 .267 .427 10.2 A .120 .864 10.2 B .757 10.2 C .806 10.2 D .206 -.171 .720 10.2 E .130 -.120 .864 10.2 F .177 .719 Note. Factor 1 Staff Training; Factor 2 Qu ality Control; Factor 3 FSWs Activities; Factor 4 Staff Orientation. Item loadings less than .10 have not been included in this table. Item numbers coincide w ith 2003-creditialing tool.

PAGE 45

Fidelity and Innovations 39 The four factors identified were easily de scribed and named. Factor 1 consisted of items that pertain to staff training and demonstration of knowledge needed for family support work; this factor was named Staff Tr aining. The Staff Training factor consisted of items such as 10.4 D: Staff (assessment workers, home visitors and supervisors) demonstrated knowledge of infant and child development within six months of the date of hire and 10.5 A: Staff (assessment workers, home visitors and supervisors) demonstrated knowledge of child abuse and neglect within 12 months of the date of hire. Factor 2 consisted of items pertaining to the supervision, evaluation, and followup on FSWs and program functioning; this factor was named Quality Control. The Quality Control factor consisted of items such as 1.2 B: The program analyzes at least annually (i.e., both formally, th rough data collection, and informally through discussions with staff and others involved in assessm ent process) who refused the program among those determined to be eligible for serv ices and the reasons why and item 11.1 B: The program ensures that weekly individual supervis ion is received by all direct service staff. Factor 3 contained items related to the routine activities required of the FSWs; this factor was named FSW Activities. Exampl es of items loading on the FSW Activities factor were 6.2 D: The home vi sitor and participant collaborate to establish a plan with specific strategies/objectives to achieve identi fied goals and 7.1 B: Home visitors provide information, referrals and linkage s to available health care re sources for all participating family members.

PAGE 46

Fidelity and Innovations 40 Factor 4 includes items pertai ning to staff orientation pr ior to direct work with children and families; this factor was name d Staff Orientation. Th e Staff Orientation factor contained items such as 10.2 A: A ssessment workers and home visitors are oriented to their roles as they relate to the programs goals, services, policies and operating procedures, and philosophy of home visiting/family support prior to direct work with children and families and 10.2 D: Staff (assessment workers, home visitors and supervisors) are oriented to child abuse and neglec t indicators and reporting requirements prior to direct work with children and families. To calculate factor sum scores for each program, I used the composite estimate method. Program ratings on retained items were summed to form composite scores for each program on the four fidelity factors. None of the factors used negative item weights. All four factors correlated relatively highly with one a nother, ranging from .369 to .679 (See Table 2 for correlations and other descriptive information).

PAGE 47

Fidelity and Innovations 41 Table 2 Means, standard deviation, and inte rcorrelations of variables Variables Mean S.D. N 2 3 4 1. Staff Training 21.15 7.60 94 .369* .416* .679* 2. Quality Control 30.60 6.91 102 .424* .512* 3. FSW Activities 19.90 4.78 102 .414* 4. Staff orientation 15.49 4.60 102 5. Medical Providers 96.54 10.06 102 6. Updated Immunizations 81.93 20.26 102 7. Participant Dropout Rate .29 .21 91 8. Additional Staff -.24 1.11 78 9. Additional Training 3.59 2.35 99 10. Additional Services 2.68 1.48 99 Tables Continues *. Correlation is significant at the 0.05 level (2-tailed)

PAGE 48

Fidelity and Innovations 42 Table 2 (Continued) Variable 5 6 7 8 9 10 1. Staff Training .097 .264* -.097 -.074 .019 .095 2. Quality Control .095 .199* -.067 .037 .050 .205* 3. FSW Activities .113 .332* -.044 .087 .113 .101 4. Staff orientation .213* .299* -.048 -044 .034 .113 5. Medical Providers .136 -.038 .152 -.054 .021 6. Updated Immunizations .033 -.001 .029 .117 7. Participant Dropout Rate -.060 -.236* -.327* 8. Additional Staff .376* .152 9. Additional Training .422* 10. Additional Services *. Correlation is significant at the 0.05 level (2-tailed) Hypothesis Testing, First Study Goal Table 2 presents the means, standard devi ations, and Pearson correlations for each of the four fidelity factors identified in th e factor analysis as well as the two outcome variables: percentage of part icipants who had medical care providers and percentage of participants who had up-to-date immunizat ions. The outcome variables were not correlated with one another. The strong ceiling effect for the percentage of participants

PAGE 49

Fidelity and Innovations 43 who had medical care providers indicates that there is litt le variance to explain among sites. This limitation will cause attenuation of the correlations between the fidelity factors and this outcome variable. Hypothesis 1 states that program ratings on fidelity will correlat e positively with the percentage of participants who have medical care providers. Four separate correlations indicate the rela tionship between the percentage of participants who had medical care providers and the four distinct factors of fidelity. Staff Orientation was the only fidelity factor that corre lated significantly with the percentage of participants who had medical care providers, R = .213, p < .05. The other three correlations were not significant at p < .05 (Table 2). These findings partially support Hypothesis 1. Table 3 Regression analysis predicting percentage of participants with primary medical providers from fidelity factors Variable R Adjusted R 2 Std. Error Model 1 .213 .036 Staff Orientation .467* .215 Model 2 .258 .024 Staff Training -.187 .197 Quality Control -.059 .193 FSW Activities .187 .265 Staff Orientation .694* .341 *. Correlation is significant at the 0.05 level (2-tailed)

PAGE 50

Fidelity and Innovations 44 All four fidelity factors were regressed on the percentage of participants who had primary medical care providers (see Table 3). The model accounted for a significant amount of variance in the outcome variable, R = .258, primarily due to the strong relationship between the outcome variable and Staff Orientation, = .694. Staff Orientation was regressed on th e percentage of participan ts who had primary medical care providers (see Table 3). Th e model was still significant, R = .213, and the adjusted R 2 increased .012. The regression model containing the Staff Orientation variable only is the best fit for this data, as it explains the most variance and is parsimonious. These findings partially support Hypothesis 1. Hypothesis 2 states that program ratings on fidelity will correlat e positively with the percentage of participants who have up-to-date immunizations. All four fidelity factors correlated significantly with the percentage of par ticipants who ha ve up-to-date immunizations. The strength of the relationships ranged from R = .332, p< .05, for FSW Activities to R = .199, p< .05, for the Quality Control factor. These findings demonstrate support for Hypothesis 2. All four fidelity factors were regressed on the percentage of participants who had up-to-date immunizations (see Table 4). This model accounted for a significant amount of variance in the outcome variable, R = .374, primarily due to the significant amount of variance accounted for by the FSW activities, = 1.069. The FSW activities factor was regressed on the percentage of participants who had up-to-date immunizations. The model was still significant, R = .332, and the adjusted R 2 remained the same. The

PAGE 51

Fidelity and Innovations 45 regression model containing the FSW activities variable only is the best fit for these data, because it explains equal variance and is pa rsimonious. These findings partially support Hypothesis 2. Table 4 Regression analysis predicting percentage of participants with updated immunizations from fidelity factors Variable R Adjusted R 2 Std. Error Model 1 .332 .102 FSW Activities 1.410* .400 Model 2 .374 .101 Staff Training .178 .197 Quality Control -.009 .193 FSW Activities 1.069* .265 Staff Orientation .636 .341 *. Correlation is significant at the 0.05 level (2-tailed) Hypothesis Testing, Second Study Goal To address the hypotheses connected to th e second goal of this study, hierarchical regression was used. The four fidelity factors identified previously were each examined independently in these regressions. Thus ther e were four moderated regressions for each hypothesis. Each hierarchical regression model contained data from at least 75 program sites and no more than 97 program sites.

PAGE 52

Fidelity and Innovations 46 In initial analyses, none of the thr ee outcome variables were significantly correlated. Staff Training, Quality Contro l and FWS Activities were not significant predictors when regressed on the percentage of participants who had medical care providers. Therefore these three scales were not examined further in Hypotheses 3, 4, and 5. None of the fidelity factors were significan t predictors of participant dropout rate, so hypotheses 9, 10, and 11 were not supported and were not analyzed further. The results of the hierarchical regression analysis are presented in Tables 5 and 6. Significant moderating relationshi ps were not found in any of the analyses; these data do not support Hypotheses 3, 4, 5, 6, 7, and 8. Thes e analyses were hindered by the small sample size used in this study.

PAGE 53

Fidelity and Innovations 47 Table 5 Hierarchical regression analysis predicting th e percentage of participants with medical provider from Staff Orientation and additional training, services, and staff Model 1: Fidelity Factor Model 2: Innovation Added Model 3: Interaction Added Variable Standardized regression coefficients ( ) Hypothesis 3 Staff Orientation .486* .490* .331 Additional Training -.273 -1.125 SO x Add Training .052 R 2 .045 .049 .052 Hypothesis 4 Staff Orientation .486* .489* .535 Additional Service -.076 .231 SO x Add Service -.020 R 2 .045 .045 .046 Hypothesis 5 Staff Orientation .585* .603* .312 Additional Staff 1.681 9.841* SO x Add Staff -.573* R 2 .050 .077 .126* *. Significant at the 0.05 level.

PAGE 54

Fidelity and Innovations 48 Table 6 Hierarchical regression analysis predicting the percentage of updated immunizations from fidelity factors and additional training, services, and staff Model 1: Fidelity Factor Model 2: Innovation Added Model 3: Interaction Added Variable Standardized regression coefficients ( ) Hypothesis 6 Staff Training .670* .670* .371 Additional Training .047 -1.940 ST x Add Training .093 R 2 .059 .059 .064 Quality Control .557 .555 .334 Additional Training .123 -1.963 QC x Add Training .066 R 2 .034 .034 .036 FSW Activities 1.350** 1.356** 2.108* Additional Training -.108 3.534 FA x Add Training -.183 R 2 .099 .099 .111 Staff Orientation 1.221** 1.219** 1.226 Additional Training .123 .158 SO x Add Training -.002 R 2 .073 .073 .073 Hypothesis 7 Staff Training .670* .647* .296 Additional Services 1.267 -1.678 ST x Add Services .143 R 2 .059 .066 .071 Quality Control .557 .513 .928 Additional Services .974 6.429 QC x Add Services -.181 R 2 .034 .038 .047 FSW Activities 1.350** 1.318** 1.352 Additional Services 1.022 1.260 FA x Add Services -.012 R 2 .099 .105 .105 *. Significant at the 0.05 level; **. Significant at the 0.10 level. Table Continues

PAGE 55

Fidelity and Innovations 49 Table 6 Continued Model 1: Fidelity Factor Model 2: Innovation Added Model 3: Interaction Added Variable Standardized regression coefficients ( ) Staff Orientation 1.221** 1.183** .727 Additional Services 1.045 -2.049 SO x Add Services .198 R 2 .073 .078 .083 Hypothesis Staff Training .989** .997** .920** Additional Staff .716 8.055 ST x Add Staff -.379 R 2 .110 .111 .130 Quality Control .904* .904* .793* Additional Staff -.039 13.933 QC x Add Staff -.466 R 2 .075 .075 .098 FSW Activities 1.638** 1.647** 1.518** Additional Staff -.485 6.661 FA x Add Staff -.385 R 2 .144 .144 .153 Staff Orientation 1.784** .1789** 1.681** Additional Staff .461 3.487 SO x Add Staff -.212 R 2 .128 .129 .131 *. Significant at the 0.05 level; **. Significant at the 0.01 level. Exploratory Analyses Among the innovations, additional traini ng was significantly correlated with additional services, r = .442, p < .05, and additional staff, r = .376, p < .05. This indicates that programs offering additiona l training opportunities to thei r employees are also more likely to offer additional services to their participants and to employ additional staff members to serve their participants.

PAGE 56

Fidelity and Innovations 50 Participant dropout rate was significantly correlate d to additional training, r = .236, p < .05, and additional services, r = -.327, p < .05. These results can be seen in Table 7. The indicates that HFA programs of fering additional training opportunities to their employees or additional services to thei r participants had a lower participant dropout rate than programs that did not engage in these innovations. Offering additional services to the participants had the gr eatest effect on participant retention, explaining almost 11% of the variance in participan t dropout rate across programs, R 2 = .107. Table 7 Standardized coefficients, R 2 and R 2 between additional serv ices, additional training and participant dropout rate Variable Participant Dropout Rate R 2 R 2 (Std. Error) Confidence Interval Step 1 Additional Services .107 -.049(.015) [-.079, -.019] Step 2 Additional Services .125 .018 -.042(.016) [-.074, -.010] Additional Training -.014(.010) [-.034, .006]

PAGE 57

Fidelity and Innovations 51 Discussion The results of this study i ndicate that there are four underlying factors represented on the Healthy Families America Credentialing Tool and that close adherence to these factors differentially affects desired program outcomes. The underlying factors identified in this study using the HFA Credentialing Tool consist of Staff Training, Quality Control, Family Support Worker Activities, and Staff Or ientation. Fidelity factors related to staff training, orientation, daily activities, and qua lity control were st rongly related to the percentage of participants with up-to-date immunizations. Fidelity factors related to staff orientation were significantly related to the percentage of partic ipants with primary medical providers. These findings support work by Gomby (1999), in which she reported that training and supervision of staff were strongly rela ted to program outcomes. None of the fidelity factors identified in this study were significantly related to a third outcome variable, partic ipant dropout rate. However, th is outcome was significantly related to innovations identified and examined in this study. Specifically, programs that provided additional training for their staff and additional services for their participants were more likely to have lower participan t dropout rates. These results support the supposition presented by Daro, McCurdy, and Ne lson (2005). Daro et al. (2005) indicated that training of the FSW might have an impact on the relationship between the participant and the FSW, which was found to be im portant in participant retention.

PAGE 58

Fidelity and Innovations 52 Findings reported by Kessler and Nelson ( 2005) were partially replicated by this study. This is expected, given that this study used a portion of the same data set used by Kessler and Nelson (2005). Ke ssler and Nelson reported th at adherence to the HFA program model was significantly related to the percentage of participants with up-to-date immunizations but not to the pe rcentage of participants with primary medical providers. This study replicated the first finding and dem onstrated that adherence to specific factors of the HFA program model are si gnificantly related to the percentage of participants with primary medical providers. The co rrelations of the fidelity f actors to the percentage of participants with primary medical pr oviders and updated immunizations, although attenuated due to range restriction and a small sample size, were much larger in this study than the correlations reported in Kessler and Nelson (2005). Using only credentialing items that manifest themselves in measurab le outcomes revealed a stronger relationship between measurable outcomes and fidelity to the HFA model than did including nonimplementation items that may be irrelevant to the outcomes examined. The findings of this study partially s upport the work by Blak ely et al. (1987). Adherence to the program model was related to certain outcome variables of importance to the program; however, the degree to which the programs fidelity was related to the desired outcome varied by the out come examined. This indicate s that specific aspects of fidelity differentially impact program outcomes. PCAA may be able use the relationships identified in the present study and future studies to streamline thei r credentialing process and include only items that ar e relevant to the effective functioning of their programs.

PAGE 59

Fidelity and Innovations 53 Organizations developing models for expa nsion purposes can address concerns of construct validity by empirically establishing re lationships between specific factors in the model and desired program outcomes. Fidelity to the HFA Credentialing Tool was related to the percentage of participants with up-to-date immunizations and primary medical providers, but not to participant dropout rate. Fideli ty to the credentialing tool may increase the likelihood of positive medical outcomes, but does not impact participant retention rates. Program activities that improve participant retention, such as the additional services and training identified in the present study, should be in tegrated into the HFA credentialing tool. These data partially support Blakely et al .s (1987) findings related to innovations as well. Additional services and training were significantly related to participant dropout rate but not to the percentage of participants with up-to -date immunizations and primary medical providers. Furthermore, programs that used one innovation tended to use multiple innovations. It may be that the leaders of these organizations have realized ways that the HFA model can be improved and encour age their members to engage in activities that address problems hindering program f unctioning. These leaders seemed to have identified additional services for the clients, more diverse and extensive training for the staff and specialized staff members can assi st the HFA sites in meeting the needs of clients, particularly with re gard to participant retention. Sites using multiple innovations should be examined in the future to identif y the mechanisms underlying the increased use of innovations and improved f unctioning. Innovations were dire ctly related to outcome

PAGE 60

Fidelity and Innovations 54 variables in this study; mode rating impacts were not identifie d. This was most likely due to limitations in power, wh ich are discussed below. Limitations There are several limitations to this st udy and to the results reported here. Many of these limitations are also discussed in Kessler and Nelson (2005); because I used the same data set, I experienced the same limitations of that data set. Statistically, this study was hindered by insufficient power for several reasons. First, there was range restriction in the dependent variables, as indicated by th e strong ceiling and floor effects depicted in Figures 2, 3, and 4. One reason for these e ffects may be the extensive and expensive credentialing process th e sites must go through. Due to th is, sites only undertake the credentialing process when they believe they will successfully complete it. For example, most home visiting programs experience pa rticipant dropout rates of about 50% during the first year of service (D uggan et al., 2000), where as the programs used in this study experienced only 29% participan t dropout rate during the sa me time period. The quality of the programs used in this study causes both range restriction of th e data set as well as bias in the predictor variables. The range rest riction could have diminished the strength of the relationships between dependent and inde pendent variables, making the relationship much more difficult to identify. The sample used in this study was probably biased, representing many of the most effective HFA sites and very few of the ineffective sites. The analyses in this study were also li mited by the small sample size. I did not have a large enough sample size to adequate ly conduct a confirmatory or exploratory

PAGE 61

Fidelity and Innovations 55 factor analysis, nor was the sample size larg e enough to adequately examine the potential moderating effects of the innovations. Because these data were archival, it was not possible to increase the number of programs ev aluated. It is quite pos sible that a Type II error occurred and effects that exist were not identified in this study. Additionally, the sites themselves supplied the data for both the outcome variables and the innovations used in th is study. I do not believe that single source bias highly impacted the findings, however. Credentiali ng data, the independent variable, were assigned by sources not affiliated with the HFA site. Two of the dependent variables, the percentage of participants with up-to-date immunizations and primary medical providers, were verified by outside sources as well. Single source bias ma y have had a larger effect on the innovation data and the participant dropout rate vari able, which were both taken from surveys supplied by the site. The rela tionships between these variables should be interpreted with caution. There were extensive missing data from th e data set that was used for this study. During the transition from the 1999 credentia ling tool to the 2003 credentialing tool, peer-reviewers did not record information a bout some items on the electronic files that PCAA saves and that were used in this st udy. Primarily, information concerning hiring requirements for FSWs and supervisors was omitted. It was noted in the information files that the old criteria for these items were us ed, because employees had been hired before 2003. Thus the hiring practices were those of the 1999 credentiali ng tool. Many items pertaining to hiring were rem oved due to the missing data. The items may constitute an

PAGE 62

Fidelity and Innovations 56 additional factor in the credentialing tool, thus severely limiting the analyses that are presented here. Finally, these data were cro ss-sectional. The independent variables were collected for the same time period as the dependent vari ables, so I am unable to conclude anything about causation from these data. Practical lim itations affecting PCAA at the time of this study severely limited access to any newer data they might have had. Thus I was limited to the cross-sectional data they were able to provide to me. Directions for Future Research Future research should pursue replicating this study with a larger sample size as more sites attempt the credenti aling process and more data b ecome available. Researchers should also examine the impact of the innovati ons identified in this study by relating the innovations utilized to trends in outcome variables over time. Outcome variables may include the three examined here as well as ot her variables, such as substantiated child maltreatment rates or parental-child interac tion. Additionally, as more sites attempt the credentialing process, re searchers should examine subsets of the innovations identified in this study. The main effects and moderating effects of innovations should be reanalyzed with a data set that provides enough power to conc lusively identify what effects are occurring as well as the actual impact of these effects. Additionally, an exploratory factor analysis should be conducted on a complete data set with a sample size that is large enough not to violate the assumption of the analysis and with the power to generate more accurate

PAGE 63

Fidelity and Innovations 57 results. This data set should include the hiring data that were missing from this examination. The data set should also be e xpanded to reduce or eliminate the biases, range restriction, and ceiling and floor eff ects that affected the outcome variables. It would be worthwhile to reexamine th e moderating effects of innovations with narrower innovation categories. Although the pr esent results indicate that additional training and services are rela ted to participant retenti on, examining which types of training and services have a stronger effect on retention would be important to the growing body of knowledge concerning the HFA programs as well as for policy affecting child maltreatment programs across the countr y. Qualitative methods, like those utilized by Blakely et al. (1987), would be useful in identifying a dditional and more specific innovations. This study has demonstrated the relationship between both fidelity to a model and innovations to program outcomes, and that these variables can differentially affect various program outcomes. Future examination of the relationship betw een fidelity to the program model and the impact on innovations is necessary to advance our knowledge of the program expansion process and how we might improve this process. Continued research on large, multi-site organizations, such as HFA, is essential to developing understanding of the program expansion proces s and what we can do to ensure successful expansion in the future.

PAGE 64

Fidelity and Innovations 58 References Anisfeld, E., Sandy, J., & Guterman, N. B. (2004). Best Beginnings: A randomized controlled trial of a parapr ofessional home visiting program. Final report submitted to the Smith Richardson Foundation and New York State Office of Children and Family Services. Berman, P. (1981). Educational change: An implementation paradigm. In R. Lehming & M. Kane (Eds.), Improving schools: Using what we know (pp. 253-286). Beverly Hills: Sage. Blakely, C.H., Mayer, J.P., Gottschalk, R.G., Schmitt, N., Davidson, W.S., & Roitjman, D.B., (1987). The fidelity-adaptation debate : Implications for the implementation of public sector social programs. American Journal of Community Psychology, 15, 253-268. Bond, G.R., Evans, L., Salyers, M.P., Williams J., & Kim, H.W. (2000). Measurement of fidelity in psychiatric rehabilitation. Mental Health Services Research, 2, 75-87. Chaffin, M. (2004). Is it time to reth ink Healthy/Start/Healthy Families? Child Abuse and Neglect, 28(6), 589-595. Daro, D. (2005). Is it time to rethink Healthy/Start/Healthy Families?: Response to Chaffin (2004). Child Abuse and Neglect, 29(3), 237-240.

PAGE 65

Fidelity and Innovations 59 Daro, D.A. & Harding, K. A. (1999). Healt hy Families America: Using research to enhance practice. The Future of Children Home Visiting: Recent Program Evaluations, 9, 152-175. Daro, D., McCurdy, K., & Nelson, C. (2005). Engagement and Retention in Voluntary New Parent Support Programs: Final Report. Final report submitted to the W. T. Grant and Anne Casey Foundations. Duggan, A.K., McFarlane, E.C., Windham, A. M., Rohde, C.A., Salkever, D.S., Fuddy, L., Rosenberg, L.A., Buchbinder, S.B., & Sia, C. (1999). Evaluation of Hawaiis Healthy Start Program. The Future of Children, 9(1), 66-60. Duggan A.K., Windham, A.M., McFarlane, E.C. Fuddy, L., Rohde, C.A., Buchbinder, S.B., & Sia, C. (2000). Hawaiis Healthy Start Program of Home Visiting for AtRisk Families: Evaluation of Family Identification, Family Engagement, and Service Delivery. Pediatrics, 105, 250-259. Duggan, A.K., Fuddy, L., McFarlane, E., Burre l, L., Higman, S.M., Windham, A.M., & Sia, C. (2004). Randomized trial of a stat ewide home visiting program to prevent child abuse: Impact on preventing child abuse and neglect. Child Abuse & Neglect, 28, 597-622. Duggan, A.K., Fuddy, L., Burrell, L., Higman, S.M., McFarlane, E., Windham, A.M., & Sia, C. (2004). Randomized trial of a stat ewide home visiting program to prevent child abuse: Impact in reduc ing parental risk factors. Child Abuse & Neglect, 28, 623-643.

PAGE 66

Fidelity and Innovations 60 Duggan, A.K., Fuddy, L., McFarlane, E., Burre ll, L., Windham, A.M., Higman, S.M., & Sia, C. (2004). Randomized trial of a stat ewide home visiting program to prevent child abuse in at risk families of new borns: Fathers participation and outcomes. Child Maltreatment, 9, 3-17. Gomby, D.S. (1999). Understa nding evaluations of ho me visitation programs. The Future Of Children: Recent Program Evaluations, 9, 27-43. Gross, D. L., Tamkin-Greener, H., Kunitz, S., & Mukamel, D. B. (2004). The growing pains of integrated health care for the elderly: Lessons from the expansion of PACE. Milbank Quarterly, 2, 257-282. Guterman, N. (1997). Early prevention of physical child abuse and neglect: Existing evidence and future directions. Child Maltreatment, 2, 12-34. Hahn, R.A., Mercy, J., & Bilukha, O. (2005) Assessing home visiting programs to prevent child abuse: Taking silv er and bronze along with gold. Child Abuse & Neglect, 29(3), 215-218. Hassall, I. (2005). Is it time to rethink H ealthy/Start/Healthy Families? Response to Chaffin (2004). Child Abuse & Neglect, 29(3), 235. Hill, P. (1999). Appendix C: The Nurse Home Visitation Program. The Future of Children, 9 (1) 190-191. Kessler, S. & Nelson C. E. (2005). Fidelity and program quality in the healthy families America programs: A review of the adaptation/adoption debate. Poster presented at the 2005 American Psychological Society in Los Angeles, CA.

PAGE 67

Fidelity and Innovations 61 Martin, J. B. (1999). Appendix A: Healthy Families America. The Future of Children, 9 (1) 177-178. McCurdy, K., Gannon, R., & Daro, D. (2003). Participation in home-based, family support programs: Ethnic variations. Family Relations, 52(1) 3-11. McCurdy, K., & Daro, D. (2000). Involvement in child abus e prevention programs: An ecological theory. Paper presented at the Victimization of Children and Youth International Research Conference, Univ ersity of New Hampshire, Durham, NH. Olds, D., Eckenrode, J., Henderson, C.R., Kitz man, H., Powers, J., & Cole, R. (1997). Long-term effects of home visitation on ma ternal life course and child abuse and neglect: 15-year follow-up of a randomized trial. Journal of the American Medical Association, 278, 637-643. Olds, D.L., Henderson, C.R., Chamberlin, R., & Tatelbaum, R. (1986). Preventing child abuse and neglect: A randomized tr ial of nurse home visitation. Pediatrics, 95, 65-78. Olds, D., Henderson, C.R., Cole, R., Eckenrode, J., Kitzman, H., & Luckey, D. (1998). Long-term effects of nurse home visitation on childrens criminal and antisocial behavior: 15-year follow-up of a randomized control trial. Journal of the American Medical Association, 280, 1238-1244. Olds, D., Henderson, C.R., Kitzman, H., & Cole, R. (1995). Effects of prenatal and infancy nurse home visitation on surveillance of child maltreatment. Pediatrics, 95, 365-372.

PAGE 68

Fidelity and Innovations 62 Olds, D.L., Henderson, C.R., Tatelbaum, R, & Chamberlin, R. (1986). Preventing child abuse and neglect: A randomized tr ial of nurse home visitation. Journal of the American Medical Association, 78, 65-78. Olds, D.L., & Kitzman, M. (1993). Review of research on home visiting for pregnant women and parents of young children. Future of Children, 3(3), 53-92. Olds, D.L., Henderson, C.R., & Kitzman, M. (1 994). Does prenatal and infancy nurse home visitation have enduring effects on qualities of parent al care giving and child health at 25 and 50 months of life? Pediatrics, 93(1), 89-98. Olds, D.L., Henderson, C.R., Kitzman, H., Ec kenrode, J.J., Cole, R.E., & Tatelbaum, R.C. (1999). Prenatal and infancy hom e visitation by nurses: Recent Finding. The Future of Children, 9(1), 44-65. Olds, D.L., Kitzman, H., Cole, R., Robinson, J., Sidora, K., Luckey, D., Henderson, C., Hanks, C., Bondy, J., & Holmberg, J. (2004). Effects of nurse home visiting on maternal life-course and child developm ent: Age six follow-up of a randomized trial. Pediatrics, 114, 1550-1559. Olds, D.L., Robinson, J., OBrien, R., Luck ey, D.W., Pettitt, L.M., Henderson, C.R., Ng, R.K., Sheff, K.L., Korfmacher, J., Hiatt, S., & Talmi, A. (2002). Home visiting by paraprofessionals and by nurses: a randomized controlled trial. Pediatrics, 110, 486-496.

PAGE 69

Fidelity and Innovations 63 Olds, D., Eckenrode, J., & Kitzman, H. (2005). Clarifying the impact of the nurse-family partnership on child maltreatment: Response to Chaffin (2004). Child Abuse & Neglect, 29(3), 229-233. Oshana, D., Harding, K., Friedman, L., & Holton, J.K. (2005). Rethinking healthy families: A continuous responsibility. Child Abuse & Neglect, 29(3), 219-228. Paisley, W.J. (1973). Post-Sputnik trends in educ ational dissemination systems. Washington DC: U.S. Department of H ealth, Education, & Welfare National Institute of Education. (ERIC Docume nt Reproduction Service No. ED088496). Paulson, R.I., Post, R.L., Herinchx, H.A., Ri sser, P. (2002). Beyond components: Using fidelity scales to measure and assure choice in program implementation and quality assurance. Community Mental Health Journal, 38, 119-128. Ryan, C.S., McCall, R.B., Robinson, D.R., Gr oark, C.J., Mulvey, L., & Plemons, B.W. (2002). Benefits of the Comprehensive Child Development Program as a function of AFDC receipt and SES. Child Development, 73(1), 315-328. St. Pierre, R.G., Layzer, J.I., G oodson, B.D. & Bernstein, L.S. (1997). National impact evaluation of the Comprehensive Child Development Program: Final Report. Cambridge, MA: Abt Associates. St. Pierre, R.G., & Layzer, J.I. (1999). Using home visits for multiple purposes: The Comprehensive Child Development Program. The Future of Children, 9, 134-151.

PAGE 70

Fidelity and Innovations 64 U.S. General Accounting Office. (1990). Home visiting: A promising early intervention strategy for at-risk families. Washington, DC: United States General Accounting Office. U.S. Advisory Board on Child Abuse and Neglect. (1991). Creating caring communities: Blueprint for an effect ive federal policy on ch ild abuse and neglect. Washington, DC: U.S. Government Printing Office. Yoshikawa, H., Rosman, E.A., & Hsueh, J. ( 2002). Resolving paradoxi cal criteria for the expansion and replication of early childhood care and education programs. Early Childhood Research Quarterly, 17, 3-27.

PAGE 71

Fidelity and Innovations 65 Appendices

PAGE 72

Fidelity and Innovations 66 Appendix A The Critical Elements of Healthy Families America 1. Initiate services prenatally or at birth. 2. Use a standardized (i.e., in a consistent way for all families) assessment tool to systematically identify families who are most in need of services. This tool should assess the presence of various factors associated with increased risk for child maltreatment or other poor childhood outcomes (i.e., social isolation, substance abuse, parental history of abuse in childhood). 3. Offer services voluntarily and use positive, persistent outreach efforts to build family trust. 4. Offer services intensely (i.e., at least once a week) with well-defined criteria for increasing or decreasing intensity of service and over the long term (i.e., three to five years). 5. Services should be culturally competent such that staff understands, acknowledges, and respects cultural differences among participants; staff and materials used should reflect the cultural, linguistic, geographic, racial and ethnic diversity of the population served. 6. Services should focus on supporting the parent(s) as well as supporting parentchild interaction and child development.

PAGE 73

Appendix A Continued Fidelity and Innovations 67 7. At a minimum, all families should be linked to a medical provider to assure optimal health and development (e.g., timely immunizations, well-child care, etc.) Depending on the familys needs, they may also be linked to additional services such as financial, food, and housing assistance programs, school readiness programs, child care, job training programs, family support centers, substance abuse treatment programs, and domestic violence shelters. 8. Services should be provided by staff with limited caseloads to assure that home visitors have an adequate amount of time to spend with each family to meet their unique and varying needs and to plan for future activities (i.e., for many communities, no more than fifteen (15) families per home visitor on the most intense service level. And, for some communities, the number may need to be significantly lower, e.g., less than ten (10). 9. Service providers should be selected because of their personal characteristics (i.e., non-judgmental, compassionate, ability to establish a trusting relationship, etc.), their willingness to work in or their experience working with culturally diverse communities, and their skills to do the job. 10. a. Service providers should have a framework, based on education or experience, for handling the variety of experiences they may encounter when working with at-risk families. All service providers should receive basic training in areas such as cultural competency, substance abuse, reporting

PAGE 74

Appendix A Continued Fidelity and Innovations child abuse, domestic violence, drug-exposed infants, and services in their community. b. Service providers should receive intensive training specific to their role to understand the essential components of family assessment and home visitation (i.e., identifying at-risk families, completing a standardized risk assessment, offering services and making referrals, promoting use of preventive health care, securing medical homes, emphasizing the importance of immunizations, utilizing creative outreach efforts, establishing and maintaining trust with families, building upon family strengths, developing an individual family support plan, observing parent-child interactions, determining the safety of the home, teaching parent-child interaction, managing crisis situations, etc.). 11. Service providers should receive ongoing, effective supervision so that they are able to develop realistic and effective plans to empower families to meet their objectives; to understand why a family may not be making progress and how to work with the family more effectively; and to express their concerns and frustrations so that they can see that they are making a difference and in order to avoid stress-related burnout. 12. The program is governed and administered in accordance with principles of effective management and of ethical practice. 68

PAGE 75

Fidelity and Innovations Appendix B HFA Credentialing Program Self-Assessment Tool: An Initiative of Prevent Child Abuse America (1999) An Overview of the Healthy Families America Credentialing Program The development of this credentialing system was initiated as a result of requests from programs, trainers, and state leaders for a quality assurance process to preserve the standards of excellence of the HFA initiative. The system was developed with the assistance of numerous individuals from the field -program managers, trainers, researchers, and state advocates/leaders -together with the expertise of the Council on Accreditation of Services for Families and Children, Inc. Taken together, this collaboration ensures that the new quality assurance system, a) reflects best practice as established by both research and practice and, b) has a process which reflects best practice in the quality assurance community. The purpose of HFA credentialing is to help each home visiting program monitor and maintain quality over the long term, as well as put into place a mechanism to ensure the quality of the HFA initiative. The goal is to recognize all credentialed sites as quality programs. The pursuit of quality and excellence involves partnerships. The program being credentialed enters into a unique partnership with HFA in its efforts to obtain public recognition as a credentialed program. These organizations share a 69

PAGE 76

Appendix B Continued Fidelity and Innovations commitment to enhancing the quality of home visitation programs and the assumption that the best program can continue to learn and grow. In effect, all parties, the applicant program, and HFA share the job of examining the programs structure and functioning. The Self-Assessment Tool, the centerpiece of HFA credentialing, is based upon the critical elements (a set of best practice standards reflecting over twenty years of research into the best outcomes in home visitation programs). When completed, the self-assessment will provide each program with an analysis of how well it is implementing the critical elements. This forms the basis for the peer review and all subsequent decisions about the programs application for a credential. The process is designed as an objective, external review, which validates for the interested public that the program is following best practice standards. The Credentialing Manual has suggestions about how to structure the work involved, the credentialing process itself, and the design and scoring of the tool. Please refer to the manual before you begin your self-assessment or call Prevent Child Abuse America at (312) 663-3520. Healthy Families America Program Fact Sheet Applicant Program Name Name of Program Director Name of Credentialing Contact (if different) Program Address City State Zip Telephone ( ) Fax ( ) E-mail 70

PAGE 77

Appendix B Continued Fidelity and Innovations Name of Host Agency/Organization (if applicable) Name of Chief Executive Officer (if applicable) Address City State Zip Telephone ( ) program, please use the above space for the administrative center and attach a list with each site in the system, program manager, address, telephone number, fax number, and host site information where applicable. Organizational Structure Briefly describe the following aspects of your organizational structure: Legal status of host agency, i.e., 501(c) 3, Public Agency, etc. Organizational chart of host agency (be sure the applicant program is included in this chart) The structure of the applicant program. Be sure your answer addresses the following points: governance of program; organizations jointly providing program services and nature of the relationship with the applicant program (e.g., job contractor, in-kind services, etc.); list of all program staff and volunteers, his/her title, and a brief description of position function. If the applicant program is a multi-site program, be sure your answer addresses the structure of the entire program, as well as each site. Submitted By Signature, Program Director Date Name, Printed Date Signature, CEO of Host Agency Date Name, Printed Date 1. Initiate services prenatally or at birth. 1-1. Program ensures it identifies families in the target population for services either while mother is pregnant (prenatally) and/or at the birth of baby. 1-1.A. The program has a description of the target population that includes key demographic information such as number of live births per year, number of women of child-bearing age, number of single parents, age of the target population, and race/ethnicity/linguistic/cultural 71

PAGE 78

Appendix B Continued Fidelity and Innovations characteristics of population and identification of places in the community where the target population can be found (e.g., local hospitals, prenatal clinics, high schools, etc.). RATING INDICATORS EVIDENCE 1-1.A. Pre-site: Please submit a description of the programs target population (including demographic characteristics described in standard I-A-1.) On-site: Interview staff assigned to maintain information on target population. 1-1.A. 3. The program has a description of the target population and identifies organizations within the community in which the target population can be found. Both the description and identification are comprehensive and up-to-date. 2. The program has a description of the target population and identifies organizations within the community in which the target population can be found, which, while sufficient for its needs, could be more comprehensive. 1. Any of the following: program does not have a description of the target population; program does not identify organizations within the community in which the target population can be found; and/or either or both the description/identification have major information gaps. 1-1.B. The programs system of formal organizational agreements with community entities (e.g. prenatal clinics, hospitals, etc.) reaches the families in the target population to determine their need for service. RATING INDICATORS 1-1.B. 3. The system of organizational agreements enables the program to screen/assess families in the target population for services. Note: while a program can probably not demonstrate that it has reached every single birth in its target population for screening/assessment, it should be able to demonstrate that its network of community organizations (prenatal clinics, local hospitals, doctor officers, medical clinics, etc.) ensures that only an extremely small number of families fall through the cracks. This is what is being evaluated in this standard. 2. While system of organizational agreements enables the program to screen/assess most families in the target population for services, a major source in the community for screening/assessing families is not currently participating as a collaborator. 1. The system does not ensure screening/assessing the majority of families. 72

PAGE 79

Appendix B Continued Fidelity and Innovations 1-1.C. The programs system of formal and/or informal services in coordination with other entities ensures potential participants are identified and referred to the program in a timely manner (i.e., giving the program the necessary time to locate the family and complete an assessment within two weeks of the birth of the baby). RATING INDICATORS 1-1.C. 3. All entities in the system identify and refer potential participants to program in a timely manner. 2. Most, but not all, entities in the system identify and refer potential participants to program in a timely manner. Program is aware of and addressing this issue. 1. The program either has no system or the system does not enable the identification of potential participants and referral to program in a timely manner. 1-1.D. Screenings/assessments (i.e., to determine eligibility for services) occur either prenatally or within the first two weeks after the birth of the baby. EVIDENCE 1-1.D. Pre-site: Assessment information as completed below, or program may submit its own assessment information. Number of Assessments Percentage of Conducted Assessments Prior to birth In hospital From release of baby from hospital to two weeks after birth More than two weeks after birth + + (Total number of assessments) = = 100% of all assessments EVIDENCE 1-1.C. Pre-site: Please submit the description of the mechanism(s) through which the program tracks information on pregnancies/births with the collaborating partners (may be informal, e.g., phone calls, etc., or formal, memoranda of agreement, etc.). On-site: Review all documentation that will show evidence of the programs system of service coordination (e.g., tickler system, etc.) Interview staff and supervisors assigned to maintain information. 73

PAGE 80

Appendix B Continued Fidelity and Innovations On-site: Review assessment files, data system (if applicable), interview assessment worker and supervisor. RATING INDICATORS 1-1.D. 3. Ninety-five (95%) to one hundred percent (100%) of eligibility assessments occur either prenatally or within the first two weeks after the birth of the baby. 2. Eighty percent (80%) to ninety-four percent (94%) of all eligibility assessments occur either prenatally or within the first two weeks after the birth of the baby. 1. Less than eighty percent (80%) of all eligibility assessments occur either prenatally or within the first two weeks after the birth of the baby. NA Program offers services universally to its target population, so assessments are not used as a method for offering program. 1-2. The program defines, measures, and monitors the acceptance rate of participants into the program on at least a yearly basis. 1-2.A. The program defines and measures the acceptance rate of participants into the program. The definition of its acceptance rate includes all potential participants who were determined to be eligible for services. EVIDENCE 1-1.A. Pre-site: Please submit a copy of your programs definition of acceptance rates and a summary of the programs acceptance rates for the most recent year. On-site: Interview assessment worker and supervisor. RATING INDICATORS 1-2.A. 3. The program defines the acceptance rate of participants into the program and evidence indicates acceptance rates are being measured on an ongoing basis. 2. The program defines its acceptance rate, but evidence indicates acceptance rates are measured inconsistently (i.e., more than a year between measurements). 1. The program either does not define its acceptance rate and/or is not measuring its acceptance rate. 1-2.B. The program analyzes (i.e., either formally, through data collection, or informally through discussions with staff and others involved in assessment process) who refused the program among all those determined to be eligible for services and the reasons why. 74

PAGE 81

Appendix B Continued Fidelity and Innovations 75 RATING INDICATORS 1-2.B. 3. The program analyzes who refused the program and why. The analysis addresses demographic and other factors, and will enhance good program planning. 2. The program has an analysis of who refused the program and why. However, analysis relies on informal sources to identify those who refused. 1. The program does not have an analysis of who refused and why. NA The program did not accept any new participants last year or all individuals who were offered the program accepted. 1-2.C. The program addresses whether and how it might increase its acceptance rate among the group(s) who are not currently choosing to participate in program. RATING INDICATORS EVIDENCE 1-2.B. Pre-site: Please submit an analysis of which group(s) are most likely to refuse the program after being found eligible. On-site: Interview assessment worker and supervisor EVIDENCE 1-2.C. Pre-site: Please submit a discussion of whether and how the program might increase its acceptance rate among the group(s) who are not currently choosing to participate in the program after being found eligible. On-site: Interview assessment worker and supervisor 1-2.C. 3. The programs discussion of whether and/or how it might increase its acceptance rate among the group(s) who are not currently choosing to participate in program addresses demographic and other factors and will enhance good program planning. 2. The program addresses whether and/or how it might increase its acceptance rate among the group(s) who are not currently choosing to participate in program. 1. The program does not address whether and/or how it might increase its acceptance rate among the group(s) who are not currently choosing to participate in program. NA The program did not accept any new participants in the past year or all individuals who were offered the program after assessing positive accepted the program.

PAGE 82

Appendix B Continued Fidelity and Innovations 76 1-3. The program ensures that, for those who accept home visitor services, the first home visit occurs prenatally or within the first three months after the birth of the baby. EVIDENCE 1-3. Pre-site: Information as completed below: First Home Visit Occurs: Number of 1st home visits made: Percent Prenatally Within 1st 3 months of birth of baby After 1st 3 months of birth of baby + Total first home visits = =100% On site: Review participant records, database information (if applicable), interview home visitor and supervisor. RATING INDICATORS 1-3. 3. Ninety-five percent (95%) to one hundred percent (100%) of the first home visits occur within the first three months after the birth of the baby. 2. Eighty percent (80%) to ninety-four (94%) percent of the first home visits occur within the first three months after the birth of the baby. 1. Less than eighty percent (80%) of the first home visits occur within the first three months after the birth of the baby. 2. Use a standardized (i.e., in a consistent way for all families) assessment tool to systematically identify families who are most in need of services. This tool should assess the presence of various factors associated with increased risk for child maltreatment or other poor childhood outcomes (i.e., social isolation, substance abuse, parental history of abuse in childhood). 2-1. The program uses tool(s) (e.g., screening tools, assessment tools, etc.) to identify the families within the target population which are most in need of intensive home visitor services. 2-1.A. The program has screening and/or assessment tool(s) to identify families most in need of intensive home visitor services.

PAGE 83

Appendix B Continued Fidelity and Innovations 77 RATING INDICATORS 2-1.A. 3. No 3 rating for 2-1.A. 2. The program has screening and/or assessment tool(s). 1. The program does not have screening and/or assessment tool(s). 2-1.B. The tool(s) assess for presence of factors including increased risk for child maltreatment or other poor childhood outcomes (e.g., social isolation, substance abuse, parental history of abuse in childhood, etc.). RATING INDICATORS 2-1.B. 3. No 3 rating for 2-1.B. 2. Tool(s) assess for the presence of factors including increased risk for child maltreatment or other poor childhood outcomes (e.g., social isolation, substance abuse, parental history of abuse in childhood, etc.). 1. Tool(s) do not assess for the presence of factors including increased risk for child maltreatment or other poor childhood outcome (e.g., social isolation, substance abuse, parental history of abuse in childhood, etc. 2-1.C. The screening and/or assessment tools(s) are used uniformly and in the same way with the target population to determine eligibility for intensive services. EVIDENCE 2-1.A. Pre-site: Please submit a copy of the screening and/or assessment tools used to identify families as eligible for services. On-site: Interview assessment worker and supervisor. EVIDENCE 2-1.C. Pre-site: Please submit a description of how tool(s) are used to determine eligibility. On-site: Review screening/assessment records, interview assessment worker and supervisor. RATING INDICATORS 2-1.C. 3. Program screening and/or assessment tool(s) are used uniformly and in the same way with the target population. 2. Program screening and/or assessment tool(s) are applied with the target population. Some instances have occurred when tools are not used. 1. The program does not demonstrate that it uses tool(s) with target population. 2-2. The program ensures that all staff and volunteers who use the screening and/or assessment tool(s) have been trained in its use prior to allowing them to administer it.

PAGE 84

Appendix B Continued Fidelity and Innovations 2-2.A. The program has a guideline for training workers who will use the tool, which includes the theoretical background (i.e., its purpose, what it measures, etc.) on the tool(s) and hands-on practice in using the tool(s). RATING INDICATORS 2-2.A. 3. The program has a guideline for training workers who will use the tool(s), which includes both the theoretical background and hands-on practice. 2. The program has a guideline for training workers who will use the tool(s), but the training does not include hands-on practice. 1. The program does not have a guideline for training workers who will use the tool(s) or the training does not include the theoretical background on the tool. 2-2.B. The trainer is qualified, through educational background and completion of training in the use of the tool(s) to train others. R A T I N G INDICATORS EVIDENCE 2-2.A. Pre-site: Please submit a copy of the training guidelines (or narrative) regarding the administration of the assessment tool(s). On-site: Interview assessment workers regarding training. EVIDENCE 2-2.B. Pre-site: Please indicate who can train staff in the use of the tool(s) and indicate his/her qualifications for training others to use the tool. On-site: Interview trainer(s), review relevant documentation regarding qualifications of trainer (e.g., certification of training in use of tool, educational degree, etc.) 2-2.B. 3. The trainer is qualified, through educational background (i.e., degree in human services or related field), experience in administering the tool, and completion of training in use of the tool(s), to train others. 2. The trainer is qualified, through experience in administering the tool and completion of training in use of the tool(s), to train others. 1. The trainer has not completed a training in the use of the tool and/or never administered the tool. 2-2.C. All staff and volunteers who use the tool(s) have been trained in its/their use prior to administering it/them. 78

PAGE 85

Appendix B Continued Fidelity and Innovations RATING INDICATORS 2-2.C. 3. All staff and volunteers who use the tool(s) have been trained in its/their use prior to administering it/them. 2. Isolated instances found when staff not trained prior to administering the tool. 1. Either program does not train staff and volunteers in use of the tool(s) or staff and volunteers routinely use the tool(s) before being trained in its/their use. 2-3. The program uses eligibility criteria to identify families in need of service and documents this in its files. 2-3.A. Criteria indicate the constellation of factors necessary for an individual to demonstrate need for service. R ATING INDICATORS 2-3.A. 3. No "3" rating. 2. The criteria indicate the constellation of factors necessary for an individual to demonstrate need for services. 1. The criteria do not indicate the constellation of factors necessary for an individual to demonstrate need for services. 2-3.B. Criteria are applied uniformly and in the same way. EVIDENCE 2-2.C. Pre-site: Please submit a list of staff/volunteers who use the tool, the date when he/she first used the tool, and the date of training. On-site: Review documentation of training for all those using the assessment tool (e.g., review personnel/volunteer files, training log, etc.) Review assessment files. EVIDENCE 2-3.A. Pre-site: Please submit a copy of the programs eligibility criteria (e.g., scoring for the screening/assessment tool(s) to determine eligibility). EVIDENCE 2-3.B. On-site: Review all relevant documentation that provides evidence that the criteria have been consistently applied (e.g., review screening/assessment files, etc.) and interview assessment RATING INDICATORS 2-3.B. 3. Criteria are consistently applied. 2. Isolated instances when criteria are not applied. 79

PAGE 86

Appendix B Continued Fidelity and Innovations 1. Criteria are not routinely applied. 2-3.C. The program assures that the criteria for eligibility are clearly and uniformly summarized in writing and documented in individual participant files. RATING INDICATORS 2-3.C. 3. The program has guidelines for criteria/assessment summaries/narratives that are both clear and comprehensive and uses the guidelines consistently in completion of assessment summaries/narratives. 2. The program has guidelines for criteria/assessment summaries/narratives. Isolated instances when guidelines were not used in completion of assessment summaries/narratives. 1. The program does not have guidelines for criteria/assessment summaries/narratives and/or the guidelines not routinely followed in completion of assessment summaries/narratives. 3. Offer services voluntarily and use positive, persistent outreach efforts to build family trust. 3-1. Services are offered to families on a voluntary basis. RATING INDICATORS EVIDENCE 2-3.C. Pre-site: Please submit a copy of the programs guidelines for and any paperwork and/or forms used to document assessments. On-site: Review criteria/assessment summaries/narratives in screening/assessment files, interview assessment worker/supervisor. EVIDENCE 3.1 Pre-site: Submit copy of program policy regarding voluntary nature of service. On-site: Review participant files for agreement signed by the families indicating voluntary nature of services. 3-1. 3. Services are offered to families solely on a voluntary basis. 2. Services are offered to families on a voluntary basis. Isolated instances when families are mandated to receive services. 1. Families are routinely mandated to receive services. 80

PAGE 87

Appendix B Continued Fidelity and Innovations 3-2. The staff uses positive outreach methods to build family trust, engage new families, and maintain family involvement in program. 3-2.A. The program has guidelines which specify a variety of positive outreach methods. R ATING INDICATORS 3-2.A. 3. The program has clearly written, comprehensive guidelines that specify a variety of positive outreach methods (e.g., telephone calls, visits, mailings, parenting groups, etc.). 2. The program has guidelines that are sufficient for its needs, but could be more clear or comprehensive. 1. Either the program has no guidelines or the guidelines are insufficient for its needs. 3-2.B. The staff uses the guidelines in order to build family trust and engage them in services. RATING INDICATORS EVIDENCE 3-2.A. Pre-site: Please submit a copy of the guidelines which address engaging families and building trust. EVIDENCE 3-2.B. Pre-site: Outreach guidelines, as submitted in 3-2.A. On-site: Review participant files of families in outreach; interview home visitor and supervisors. 3-2.B. 3. Evidence consistently points to routine use of these guidelines. 2. Isolated instances found when guidelines not followed. 1. Guidelines routinely not followed. 3-3. The program offers outreach under specified circumstances for a minimum of three months for each family before discontinuing services. 3-3.A. The program guidelines specify the circumstances under which a family is placed in outreach status. RATING INDICATORS 3-3.A. 3. The guidelines specify the types of circumstances under which a family is provided outreach. Guidelines are clear and easy to understand. 2. The guidelines specify the types of circumstances under which a family is provided outreach. However, guidelines are lacking in clarity and/or somewhat difficult to understand. 81

PAGE 88

Appendix B Continued Fidelity and Innovations 1. No guidelines or the program does not specify the types of circumstances under which a family is provided outreach. 3-3.B. The program guidelines specify that outreach is continued for families for three months and that outreach is only concluded prior to three months when families have been engaged or re-engaged in services or family has moved from the area. R A T I NG INDICATORS 3-3.B. 3. No "3" for 3-3.B. 2. The program guidelines specify that outreach is continued for families for three months and that outreach is only concluded prior to three months when families have (re)engaged in services or the family has moved from the area. 1. The program either has no guidelines or the guidelines do not address either or both points in 3-3.B. 3-3.C. The program places families in outreach appropriately and continues outreach for three months, only concluding outreach prior to three months when the families have (re)engaged in services or moved from the area. R A T I RATING INDICATORS EVIDENCE 3-3.B. Pre-site: Please submit a copy of the program guidelines which address the amount of time outreach is continued for a family. EVIDENCE 3-3.C. On-site: Review participant files of families in outreach, interview home visitors and supervisors. 3-3.C. 3. The program routinely places families in outreach appropriately (i.e., as specified by its guidelines) and routinely continues outreach for three months. The only instances found when outreach was concluded prior to three months occurred when the families (re)engaged in services or moved from the area. 2. The program follows the guidelines, as indicated in "3" rating indicator. However, a few isolated instances found when the guidelines were not followed. 1. The program does not routinely follow its guidelines as specified in "3" rating indicator. 3-4. The program defines, measures and monitors its retention rate of participants in the program on at least a yearly basis. 3-4.A. The program defines and measures its retention rate. The definition of its retention rates includes all participants who received outreach and/or home visitation from the program. 82

PAGE 89

Appendix B Continued Fidelity and Innovations RATING INDICATORS 3-4.A. 3. The program defines the retention rate of participants in the program and evidence indicates retention rates are being measured on an ongoing basis. 2. The program defines its retention rate, but evidence indicates retention rates are measured inconsistently, but still at least yearly. 1. The program either does not define its retention rate and/or is not measuring it at least yearly. 3-4.B. The program analyzes (i.e., either formally, through data collection, or informally through discussions with staff and others involved in program services) which group(s) dropped out of the program, at what point in services, and why. RATING INDICATORS 3-4.B. 3. The program analyzes who leaves the program and why; the analysis addresses demographic and other factors; and will enhance good program planning. 2. Program analyzes who drops out of the program and why. However, analysis relies on informal sources to identify those who dropped out. 1. The program does not analyzes who refused the program and why. 3-4.C. The program addresses whether and how it might increase its retention rate among participant groups who are currently dropping out of the program after receiving services. RATING INDICATORS EVIDENCE 3-4.A. Pre-site: Please submit a copy of the programs definition of retention rates and a summary of the programs retention rates for the most recent year. On-site: Interview home visitors and supervisor. Review documentation the program uses to collect and maintain retention rates. EVIDENCE 3-4.B. Pre-site: Please submit an analysis of which group(s) are most likely to drop out of program, at what point in service, and why. On-site: Interview home visitors and supervisor. EVIDENCE 3-4.C. Pre-site: Please submit a discussion of whether and how the program might increase its retention rate, based upon your analysis of 3-4.B. Ons ite: Interview assessment worker an d su p ervisor. 83

PAGE 90

Appendix B Continued Fidelity and Innovations 3-4.C. 3. The programs discussion of whether and/or how it might increase its retention rate among the group(s) who are currently dropping out addresses demographic and other factors and will enhance good program planning. 2. The program addresses whether and/or how it might increase its retention rate among the group(s) who are currently disengaging from services. 1. The program does not address whether and/or how it might increase its retention rate among the group(s) who are currently disengaging from services. 4. Offer services intensely (i.e., at least once a week) with well-defined criteria for increasing or decreasing intensity of service and over the long term (i.e., three to five years). 4-1. The program has a well-thought out system for managing the intensity of home visitor services. 4-1.A. The levels of service (i.e., weekly visits, bi-weekly visits, monthly visits, etc.) offered by the program are clearly defined. RATING INDICATORS 4-1.A. 3. Levels of service are clearly defined and home visitors understand the levels of service offered. 2. Levels of service are clearly defined; isolated instances found when home visitor did not understand the levels of service offered by the program. 1. Levels of service are not clearly defined or, home visitors routinely do not understand the levels of service offered by the program. 4-1.B. Participants at the various levels of service (i.e., weekly visits, bi-weekly visits, monthly visits, etc.) offered by the program receive the appropriate number of visits, based upon the level of service to which they are assigned. EVIDENCE 4-1.A. Pre-site: Please submit a description of the various levels of service offered by the program. On-site: Interview with home visitor and supervisor. EVIDENCE 4-1.B. Pre-site: Please submit a summary of all current program participants by home visitor, their start date, their level of service, and a record of the visits (completed only) each has received during the past quarter (e.g., home visitor logs). On-site: Interview with home visitor and supervisor, review participant files. 84

PAGE 91

Appendix B Continued Fidelity and Innovations RATING INDICATORS 4-1.B. 3. Participants routinely receive the appropriate number of home visits based upon the levels of service. 2. Isolated instances found when participants did not receive the appropriate number of home visits based upon the level of service. 1. Participants do not routinely receive the appropriate number of home visits based upon the levels of service. 4-1.C. The criteria for increasing/decreasing the intensity of the service is clearly defined and linked to the levels of service offered by the program. R A T ING INDICATORS 4-1.C. 3. Criteria for moving participant from one level of service to another is clearly defined and linked to the levels of service. Home visitors clearly understand the criteria and the linkage with the levels of service. 2. Criteria for moving participant from one level of service to another is clearly defined and linked to the levels of service. Isolated instances found when home visitors did not understand the criteria and the linkage with the levels of service. 1. Any of the following: criteria for moving participant from one level of service to another is not clearly defined; criteria are not linked to levels of service; home visitors routinely did not understand the criteria and the linkage with the levels of service. 4-1.D. Each participants case is regularly reviewed by the family, home visitor, and supervisor. EVIDENCE 4-1.C. Pre-site: Please submit a copy of the programs criteria for increasing/decreasing intensity of service. On-site: Interview home visitor and supervisor. EVIDENCE 4-1.D. Pre-site: Please submit a copy of the programs policy on case reviews. On-site: Review participant files, interview participants, home visitor and supervisor. RATING INDICATORS 4-1.D. 3. The program regularly reviews cases, and involves, at a minimum, the worker, the family, and the supervisor in the case review. 2. Isolated instances found when either review was not conducted regularly or appropriate individuals were not involved with review. 1. Either reviews were not conducted regularly or appropriate individuals were consistently not involved with review. 85

PAGE 92

Appendix B Continued Fidelity and Innovations 4-1.E. The case review is the basis for the decision to move the family from one level of service to another. RATING INDICATORS 4-1.E. 3. The case review routinely serves as the basis for the decision to move families from one level of service to another. 2. Isolated instances when families moved from one level of service to another in absence of a case review. 1. Families routinely moved from one level of service to another in absence of a case review. 4-2. The program offers home visitation services intensively after the birth of the baby. (NOTE: If the applicant program offers services universally, this standard only applies to those families assessed as needing intensive home visitation services.) 4-2.A. Policy states that families receiving intensive home visitation services receive weekly home visits for a minimum of six months after the birth of the baby. RATING INDICATORS 4-2.A. 3. No "3" rating indicator for 4-2.A. 2. Policy states that the minimum length of time for weekly home visits is at least six months after the birth of the baby. 1. Policy states that the minimum length of time for weekly home visits is less than six months or weekly visits are not indicated in program policy. 4-2.B. Program ensures that all families receiving intensive home visitation services participate in weekly (or more frequent) home visits for a minimum of six months after the birth of the baby. EVIDENCE 4-2.A. Pre-site: Please submit a copy of programs policy on the minimum amount of time it offers weekly visits for families receiving intensive home visitation services. EVIDENCE 4-2.B. On-site: Review participant files and/or other appropriate sources of documentation regarding intensity of service offered to participants, interview home visitors, supervisors, and participants. RATING INDICATORS 4-2.B. 3. Participants routinely receive weekly home visits for a minimum of six months after the birth of the baby. 86

PAGE 93

Appendix B Continued Fidelity and Innovations 2. Isolated instances found when participants did not receive weekly visits for a minimum of six months after the birth of the baby. Program demonstrates that home visits were attempted in these isolated instances. 1. Participants routinely do not receive weekly home visits for a minimum of six months after the birth of the baby (i.e., either the participants are routinely not receiving weekly visits for a minimum of six months or the majority of the visits are attempted, but not actually made). 4-3. The program offers home visitation services to participant family for a minimum of three years after the birth of the baby. 4-3.A. The program policy states that it will offer home visitation services to participant families for a minimum of three years after the birth of the baby. R ATING INDICATORS 4-3.A. 3. The program policy specifies that home visitation services are offered for more than three years after the birth of the baby. 2. The program policy specifies that home visitation services are offered for three years after the birth of the baby. 1. The program policy specifies that home visitation services are offered for less than three years after the birth of the baby. 4-3.B. The program ensures that it offers home visitation services to participant families for a minimum of three years after the birth of the baby (for those families who wish to continue participating). R ATING INDICATORS EVIDENCE 4-3.A. Pre-site: Please submit a copy of the programs policy regarding length of time program it will continue to offer home visitation services to a participating family. EVIDENCE 4-3.B. Pre-site: Please submit a report showing the participants in your program who have been with the program for at least three years. On-site: Interview home visitor and supervisor. 4-3.B. 3. No 3" rating indicator for this standard. 2. The program ensures that home visitation services are offered for more than three years after the birth of the baby to participants who wish to continue with the program. 1. The program, while it has been operational for at least three years, has not demonstrated that it continues to provide services for a minimum of three years to those participants who wish to continue. 87

PAGE 94

Appendix B Continued Fidelity and Innovations NA Program has not been in operation for at least three years. 5. Services should be culturally competent such that staff understands, acknowledges, and respects cultural differences among participants; staff and materials used should reflect the cultural, linguistic, geographic, racial and ethnic diversity of the population served. 5.1. The program has a description of the cultural (i.e., teen parent, etc.), racial/ethnic, and linguistic characteristics of all groups within the current service population. RATING INDICATORS 5-1. 3. Description is clear and well-presented and thoroughly addresses all of the following for service population: Cultural characteristics Racial/ethnic characteristics Linguistic characteristics 2. Description addresses all of the above stated items. However, report is not as clear and/or as thorough as possible, given information available to program. 1. Either there is no description of the characteristics of the service population or the description of the service population does not address the items specified in rating indicator #3 above. 5-2. The program promotes culturally competent practice among program staff. 5-2.A. The program has staff, volunteers, and/or agreements with other, appropriate community entities to provide culturally sensitive services to all group(s) within the target population. EVIDENCE 5.1. Pre-site: Please submit a description of service and target population as indicated in the standard above. On-site: Review relevant information to ascertain accuracy of service population demographic reports, interview program manager, and/or staff assigned to maintain this information. EVIDENCE 5-2.A. Pre-site: Please attach a breakdown of the demographics of staff (e.g., culture, race/ethnicity, language, etc.) in the program, identify the culturally sensitive service gaps that exist for program participants and describe the other ways in which the program can meet the needs of these group(s) (e.g., interagency agreements with appropriate agencies who can provide appropriate linguistic services, volunteer recruitment to help fill the gap(s), etc.). On-site: Interview program manager and direct service staff. 88

PAGE 95

Appendix B Continued Fidelity and Innovations RATING INDICATORS 5-2.A. 3. The program has the appropriate staff, volunteers, and community partners to meet the cultural and linguistic needs of all of the population groups within the target population. 2. The program has the appropriate staff, volunteers, and community partners to meet the cultural and linguistic needs of the most of the population groups within the target population. Program has a plan to ensure that it has adequate staff, volunteers or other community partners to meet the needs of those population groups in the target population not currently being served. 1. The program does not have the appropriate staff, volunteers, and community partners to meet the cultural and linguistic needs of most of the population groups within the target population. Or the programs staff, volunteers, and community partners can meet the cultural and linguistic needs of most of the population groups within the target population, but has no plan to ensure it meets the needs of those not currently being served. 5-2.B. The programs materials for the target population and the general public are participant-centered. EVIDENCE 5-2.B. Pre-site: Please submit copies of all relevant program materials for the target population, program participants, and general public (e.g., annual report, program brochure, flyers announcing program events, etc.). Program may provide a brief description of how it ensures the materials are participant-centered (i.e., language, photos, etc.). On-site: Interview program manager and review additional program materials that were too large to send with pre-site evidence (i.e., curricula, etc.). RATING INDICATORS 5-2.B. 3. All of the programs materials (e.g., annual report, brochures, program specific materials such as curricula, etc.) are participant centered (e.g., photos reflective of diversity of population, materials available in major languages spoken by target population, materials reflect literacy level of participants, etc.). 2. Most of the programs materials (e.g., annual report, brochures, program specific materials such as curricula, etc.) are participant centered (e.g., photos reflective of diversity of population, materials available in major languages spoken by population, materials reflect literacy level of participants, etc.). 1. None of the programs materials are participant centered, or, the programs materials do not reflect the diversity of one or more major groups in the target population. 89

PAGE 96

Appendix B Continued Fidelity and Innovations 5-2.C. The program provides staff training regularly (i.e., offers training at regularly scheduled intervals throughout the year) regarding the unique characteristics of population(s) being served (age related factors, language, culture, etc.) and regarding cultural sensitivity. RATING INDICATORS 5-2.C. 3. The program regularly provides training for direct service staff in both areas listed in standard 5-2.C. That is, all staff receive training in the areas listed and the trained is offered on an on-going basis to ensure all staff participate in it. 2. The program provides training for direct program staff in both areas listed in standard. However, evidence suggests that training is not as regular as could be. Or, program is providing training in one area or the other and evidence indicates that lack of training is an availability issue (i.e., appropriate training is not available in the area). 1. The program is not providing training for direct program staff in either of the areas listed in standard. Or program is providing training in one of the areas only and evidence indicates that the lack of training is not due to the lack of availability. 5-2.D. Ethnic, cultural, and linguistic factors are taken into account in assigning workers to participants and in overseeing home visitor/participant interactions. EVIDENCE 5-2.C. Pre-site: Please submit documentation regarding the training offered to staff on cultural diversity and the particular group(s) represented in the target population. Be sure to include a list of all program direct service staff in attendance, the training(s) attended, and date trainings were completed. On-site: Interview program manager and direct service staff; review personnel files/training log. EVIDENCE 5-2.D. Pre-site: Please submit a description of how the program ensures it takes into account the ethnic, cultural, and linguistic issues of participants and home visitors both during initial assignment of home visitor to participant and ongoing. On-site: Interview supervisors, home visitors, and participants. RATING INDICATORS 5-2.D. 3. Program routinely takes into account ethnic, cultural, and linguistic issues during assignment and ongoing oversight of home visitors. 90

PAGE 97

Appendix B Continued Fidelity and Innovations 2. Isolated instances have occurred when this did not happen. 1. Program does not routinely take into account ethnic, cultural, and linguistic issues during assignment and ongoing oversight of home visitors. 5-3. The program regularly evaluates whether its services accommodate cultural differences and utilize cultural and family strengths and resources. 5-3.A. The review addresses all components of the service delivery system. RATING INDICATORS 5-3.A. 3. The annual review addresses all components of the service delivery system related to cultural differences (e.g., assessment, service planning, and service delivery). 2. The annual review addresses only one or two of the components of the delivery system related to cultural differences. Program has a plan to expand its review to include all components of the delivery system. 1. There is no annual review. 5-3.B. The program has criteria for its review. EVIDENCE 5-3.A. Pre-site: Please submit, in writing, a copy of the most recent service review undertaken by the program to evaluate how well it is accommodating cultural differences and utilizing cultural and family strengths and resources. On-site: Interview program manager. EVIDENCE 5-3.B. Pre-site: Please submit a copy of the criteria against which the program measures its review. On-site: Interview program manager. RATING INDICATORS 5-3.B. 3. No 3" for this standard. 2. The program has criteria for its review. 1. The program does not have criteria for its review or there is no review. 5-3.C. The review uses up-to-date (that is, obtained during the past year) information, including participant input regarding culturally appropriate services in the report. 91

PAGE 98

Appendix B Continued Fidelity and Innovations R R ATING INDICATORS 5-3.C. 3. The review uses up-to-date information and includes direct input from participants (e.g., focus groups, participant surveys, etc.). Use of participant input may include: evaluation of participant input, reporting of participant input to decision-making body, or incorporation of participant input into program services. 2. The review uses up-to-date information and the program uses indirect participant input regarding culturally appropriate services (e.g., information gained during home visits which is relayed to supervisor and used to alter practice of program). 1. The review does not use up-to-date information; does not use participant input; or there is no review. 5-3.D. The review is reported at least annually to the appropriate supervisory or governance group and action is taken (if necessary). RATING INDICATORS 5-3.D. 3. The review is reported semi-annually and action is taken, if necessary. 2. The review is reported annually and action is taken, if necessary. 1. Any of the following: the review is not reported at least annually to the appropriate group; no action was taken, but it was judged to be necessary; or there is no review. 6. Services should focus on supporting the parent(s) as well as supporting parent-child interaction and child development. EVIDENCE 5-3.C. Pre-site: Please submit a copy of the most recent service review undertaken by the program to evaluate how well it is accommodating cultural differences and utilizing cultural and family strengths and resources, as submitted in 5-3.A. Program may want to submit any relevant forms, etc. used to gather information that was used in the review. On-site: Interview program manager, review data collection forms used in i EVIDENCE 5-3.D. Pre-site: Please indicate to whom the review is reported, the frequency of the reporting, and how action is taken (if necessary) on the findings of the report. Also, submit a list of individuals to whom the report is presented. On-site: Interview appropriate individuals (as indicated in pre-site answer) regarding the reporting of and action taken on reports. 92

PAGE 99

Appendix B Continued Fidelity and Innovations 6-1. Delivery of services to families is guided by the Family Support Plan and the process of developing the plan uses family support practices. 6-1.A. The home visitor and family collaborate to identify family strengths and competencies. 93 R RATING INDICATORS EVIDENCE 6-1.A. through 6-1.F. Pre-site: Please submit the programs guidelines (or narrative) describing how the service plan promotes family support practices and how the service plan serves as the basis for service delivery. Be sure to include a copy of the service plan used by the program, as well as any other documentation that would support home visitor-participant collaboration in the planning and delivery of services. On-site: Review participant files, Interview participant, home visitor, and supervisor. 6-1.A. 3. The home visitor and family routinely collaborate to identify family strengths and competencies. 2. The home visitor and family collaborate to identify family strengths and competencies. However, some instances found when collaboration did not occur. 1. The home visitor and family do not routinely collaborate. 6-1.B. The home visitor and family collaborate to assess family needs and the services which are desired to help address these needs. RATING INDICATORS 6-1.B. 3. The home visitor and family routinely collaborate to assess family needs and the services that are desired to help address these needs. 2. The home visitor and family collaborate to assess family needs and the services that are desired to help address these needs. However, some instances found when it did not occur. 1. The home visitor and family do not routinely collaborate.

PAGE 100

Appendix B Continued Fidelity and Innovations 94 6-1.C. The home visitor and family collaborate to set family goals and specific objectives for the home visitor service. RATING INDICATORS 6-1.C. 3. The home visitor and family routinely collaborate to set family goals and specific objectives for the home visitor service. 2. The home visitor and family collaborate to set family goals and specific objectives for the home visitor service. However, some instances found when it did not occur. 1. The home visitor and family do not routinely collaborate. 6-1.D. The home visitor and family collaborate to plan for resolution of identified problems. RATING INDICATORS 6-1.D. 3. The home visitor and family routinely collaborate to plan for resolution of identified problems. 2. The home visitor and family collaborate to plan for resolution of identified problems. However, some instances found when it did not occur. 1. The home visitor and family do not routinely collaborate. 6-1.E. The home visitor, family, and supervisor (as needed) collaborate in the review of the familys service plan and the review of the service plan occurs at regular intervals (i.e., bi-weekly, monthly, quarterly, etc). RATING INDICATORS 6-1.E. 3. The home visitor, family, and supervisor (as needed) routinely collaborate in the review of the familys service plan and the review of the service plan occurs at least semi-annually. 2. The home visitor, family, and supervisor (as needed) collaborate in the review of the familys service plan. However, some instances found when it did not occur. The service plan review occurs at least annually. 1. The home visitor, family, and supervisor (as needed) do not routinely collaborate in the review of the familys service plan and/or the program does not review all service plans at least annually. 6-1.F. The plan serves as the guide for delivering services. RATING INDICATORS 6-1.F. 3. The plan routinely serves as the guide for delivering services.

PAGE 101

Appendix B Continued Fidelity and Innovations 95 2. Isolated instances found when the plan did not serve as the guide for delivering services. 1. The plan does not routinely serve as the guide for delivering services. 6-2. In the first home visit, each family member is informed about his/her rights, including the right to confidentiality both verbally and in writing. (Note: This does not include families in creative outreach.) RATING INDICATORS 6-2. 3. The program policy states that each participant is informed about his/her rights, including the right to confidentiality during the first home visit. There is strong evidence in participant files to indicate that participants are routinely informed about their rights during the first home visit, both verbally and in writing. 2. The program policy states that each participant is informed about his/her rights, including the right to confidentiality at the first home visit. There is evidence in participant files to indicate that participants are being informed routinely about their rights at the first home visit. In some unusual instances, participants informed of their rights at a time later than first home visit. 1. The program either does not have a policy that states that each participant is informed about his/her rights, including the right to confidentiality, during the first home visit and/or there is insufficient evidence in participant files to indicate that participants are being informed routinely about his/her rights during the first home visit. 6-3. The program promotes positive parenting skills and knowledge of child development with participants. 6-3.A. The program has comprehensive guidelines regarding promotion of positive parenting skills and knowledge of child development with families. EVIDENCE 6-2. Pre-site: Please submit a copy of the programs policy regarding informing participants about rights and the form used by the program to signify that the participant has been informed about his/her rights. On-site: Review participant files; interview home visitor, participant, and supervisor. EVIDENCE 6-3.A. Pre-site: Please submit a copy of your programs guidelines regarding promotion of positive parenting skills and knowledge of child development with participants. On-site: Review participant files; interview home visitor, supervisor, and participants.

PAGE 102

Appendix B Continued Fidelity and Innovations 96 RATING INDICATORS 6-3.A. 3. Program has comprehensive guidelines regarding promotion of positive parenting skills and knowledge of child development and consistent evidence indicates guidelines are routinely followed. 2. Program has a narrative regarding promotion of positive parenting skills and knowledge of child development and evidence indicates narrative is being followed routinely. Isolated instances found when narrative not followed. 1. Program does not have either guidelines or narrative and/or insufficient evidence to indicate guidelines are regularly put into practice with \ families. 6-3.B. The program has and uses parenting and/or child development curriculum(a). R A T I N G INDICATORS 6-3.B. 3. Program has a number of parenting/child development curricula that are used, depending upon the individual needs of the family. Consistent evidence indicates that curricula are routinely and appropriately used with all parents. 2. Program has at least one parenting/child development curriculum and consistent evidence indicates that curriculum(a) are routinely and appropriately used with all parents. A few isolated instances found when no curriculum was used with a family. 1. Program either does not have at least one parenting/child development curriculum or evidence indicates that curriculum(a) are not routinely used with families. 6-3.C. Home visitor shares information on appropriate infant and child development with families. EVIDENCE 6-3.B. Pre-site: Please submit a list of the parenting and/or child development curricula used with participants. On-site: Review participant files; review curricula; interview home visitor, supervisor, and participants. EVIDENCE 6-3.C. Pre-site: A copy of your programs guidelines (or narrative) regarding sharing information on appropriate infant and child development with participants. On-site: Review participant files; interview home visitor, supervisor, and participants.

PAGE 103

Appendix B Continued Fidelity and Innovations 97 RATING INDICATORS 6-3.C. 3. Home visitor routinely shares information on appropriate infant and child development with families. 2. Home visitor does this; however, isolated instances found when information not shared routinely with families. 1. Home visitor does not do this routinely. 6-3.D. Home visitor shares information on appropriate activities designed to promote bonding and positive parent-child interaction with families. R ATING INDICATORS EVIDENCE 6-3.D. Pre-site: A copy of your programs guidelines (or narrative) regarding sharing appropriate activities designed to promote bonding and positive parent-child interaction with participants. On-site: Review participant files; interview home visitor, supervisor, and participants. 6-3.D. 3. Home visitor routinely shares information on appropriate activities designed to promote bonding and positive parent-child interaction with families. 2. Home visitor does this, however isolated instances found when information not shared routinely. 1. Home visitor does not do this routinely. 6-3.E. Home visitor models positive adult-child interaction. RATING INDICATORS 6-3.E. 3. No 3" for 6-3.E. 2. Home visitor models positive adult-child interaction; evidence consistently indicates that this occurs routinely. 1. Home visitor does not or does not routinely model positive adultchild interaction. 6-4. The program monitors the development of participating infants and children. 6-4.A. The program has a developmental screen (or other standardized tool) to monitor infant/child development.

PAGE 104

Appendix B Continued Fidelity and Innovations 98 RATING INDICATORS 6-4.A. 3. No 3" rating indicator for 6-5.A. 2. The program has a screen/tool. 1. The program does not have a screen/tool. 6-4.B. The program has guidelines to administer the tool, and guidelines specify that screen /tool is to be used with all children participating in the program. RATING INDICATORS 6-4.B. 3. The program has guidelines to administer the screen/tool. Guidelines specify that screen/tool is to be used with all target children in program. 2. The program has a narrative describing how to administer the screen/tool. Narrative specifies that screen/tool is to be used with all target children in program. 1. The program does not have guidelines or a narrative to administer the screen/tool. Or, guidelines/narrative does not specify that screen/tool is to be used with all target children in the program. 6-4.C. The program uses the screen/tool to monitor child development at specified intervals. EVIDENCE 6-4.A. Pre-site: Please submit a copy of the programs policy stating the developmental screens or other standardized tool used to monitor infant/child development. On-site: Review tool(s); interview home visitor and supervisor. EVIDENCE 6-4.B. Pre-site: Please submit a copy of the programs guidelines (or narrative) for administering the tool(s). On-site: Interview home visitor and supervisor. EVIDENCE 6-4.C. Pre-site: Guidelines, as submitted in 6-4.B. above. On-site: Interview home visitor and supervisor; review participant files (or other relevant documentation) for evidence of tool being used. RATING INDICATORS 6-4.C. 3. The program uses the screen/tool at specified intervals to monitor child development for all target children in the program.

PAGE 105

Appendix B Continued Fidelity and Innovations 99 2. The program uses the screen/tool at specified intervals to monitor child development for all target children in the program. Isolated instances found when screen/tool not used with target children. 1. Any of the following: the program does not use the screen/tool; the program does not use the screen/tool at the specified intervals; the program does not use the tool with all target children in the program. 6-5. Those who administer the development screenings have been trained in the use of the tool before administering the screening. R A T I N G RATING INDICATORS 6-5. 3. All staff using the tool have been trained in its use before administering it. 2. Isolated instances found when staff administered screen before being trained. 1. Either of the following: evidence demonstrates that several staff who administer the tool are not trained in its use and/or evidence exists to indicate that staff routinely administer the tool prior to being trained. 6-6. The program tracks child participants who are suspected of having a developmental delay and follows through with other appropriate interventions (e.g., referrals, follow-up, etc.) as needed. 6-6.A. The program has guidelines which address how it tracks and follows through with appropriate actions for child participants suspected of having a developmental delay. EVIDENCE 6-5. Pre-site: Please submit a copy of the training outline (or narrative describing the training); a list of staff who use the tool, and the date trained, and the date the individual first administered the tool. On-site: Review documentation of training (e.g., personnel files, training log); cross check with participant files to evaluate whether staff is administering developmental screen only after training; interview home visitors and supervisors. EVIDENCE 6-6.A. Pre-site: Please submit a copy of the programs guidelines (or narrative) describing how program tracks, refers, etc. child participants suspected of having a developmental delay. On-site: Interview home visitor and supervisor. RATING INDICATORS 6-6.A. 3. The program has guidelines. 2. The program has a narrative. 1. The program has neither guidelines nor a narrative.

PAGE 106

Appendix B Continued Fidelity and Innovations 6-6.B. The program tracks child participants suspected of having a developmental delay. 100 R RATING INDICATORS 6-6.B. 3. Consistent evidence indicates that the program routinely tracks child participants suspected of having a developmental delay. 2. Evidence indicates the program routinely tracks child participants suspected of having a delay. Isolated instances occurred when this did not happen. 1. Insufficient evidence to indicate that the program routinely tracks child participants suspected of having a developmental delay. 6-6.C. The program follows through with appropriate actions (i.e., referrals, in-depth evaluations, or examinations, treatment or other services). R A T ING INDICATORS EVIDENCE 6-6.B. Pre-site: Please submit a copy of the forms used to track child participants suspected of having a developmental delay. On-site: Interview home visitor and supervisor; review participant files. EVIDENCE 6-6.C. Pre-site: Please submit a copy of the forms used to refer and monitor the interventions needed for child participants suspected of having a developmental delay (program can refer to the forms submitted in 6-7.B, if applicable). On-site: Interview home visitor and supervisor; review participant files. 6-6.C. 3. Consistent evidence indicates that the program routinely follows through with appropriate activities. 2. Evidence indicates that the program routinely follows through with appropriate activities. Isolated instances occurred when this did not happen. 1. Insufficient evidence to indicate that the program routinely follows through with appropriate activities 7. At a minimum, all families should be linked to a medical provider to assure optimal health and development (e.g., timely immunizations, well-child care, etc.) Depending on the familys needs, they may also be linked to additional services such as financial, food, and housing assistance programs, school readiness programs, child care, job training programs, family support centers, substance abuse treatment programs, and domestic violence shelters.

PAGE 107

Appendix B Continued Fidelity and Innovations 7-1. All participating families have a medical provider to assure optimal health and development. 7-1.A. The program has definition of (that is, what does the program mean when a participant has a medical provider) and guidelines to document medical provider(s) for all participating family members. 101 R A RATING INDICATORS 7-1.A. 3. Program has a definition of a medical provider and has guidelines for documenting the medical providers. Consistent evidence indicates that guidelines are being followed. 2. Program has a definition of a medical provider and has a narrative describing how to document the medical providers. Evidence indicates that narrative is being used in documentation. 1. Any one of the following: program does not have a definition of a medical provider; program has neither guidelines nor a narrative to document the medical providers; or insufficient evidence to indicate that guidelines/narrative are being used in documentation. 7-1.B. Home visitors, medical provider (as appropriate), and family collaborate to ensure each participating family member receives optimal health care. RATING INDICATORS EVIDENCE 7-1.A. Pre-site: Please submit a copy of the programs definition of and guidelines (or narrative) for how workers should document the medical provider. On-site: Interview program manager and home visitors if needed. EVIDENCE 7-1.B. Pre-site: Please submit a brief description of how the program, participants, and medical provider collaborate to ensure participants receive optimal health care. On-site: Interview program manager, home visitor, and participants. Review participant files. 7-1.B. 3. Collaboration is occurring between the home visitor, the family and the medical provider (if available and appropriate). 2. Evidence suggests that collaboration is occurring at least between the home visitor and family (e.g., home visitor transports family to medical provider for well-baby check-ups, home visitor and family work together to advocate for necessary health care, etc.). 1. Insufficient evidence to suggest that collaboration is occurring. 7-1.C. Participating parent(s) have a medical provider.

PAGE 108

Appendix B Continued Fidelity and Innovations 102 RATING INDICATORS 7-1.C. 3. Ninety-five percent (95%) to one hundred percent (100%) of participating parent(s) have a medical provider. 2. Eighty (80%) to ninety-four percent (94%) of participating parent(s) have a medical provider. 1. Less than eighty percent (80%) of participating parent(s) have a medical provider. 7-1.D. Participating children have a medical provider. RATING INDICATORS 7-1.D. 3. Ninety-five percent (95%) to one hundred percent (100%) of participating children have a medical provider. 2. Eighty (80%) to ninety-four percent (94%) of participating children have a medical provider. 1. Less than eighty percent (80%) of participant children have a medical provider. 7-2. The program ensures participating children receive timely immunizations. 7-2.A. The program identifies immunization schedule to be met and has guidelines to document immunizations for all child participants. EVIDENCE 7-1.C. Pre-site: Please submit a report detailing the number of all participating parents and the percent who currently have a medical provider. On-site: Interview program manager if needed; review participant files. EVIDENCE 7-1.D. Pre-site: Please submit a report detailing the number of all participating children and the percent who currently have a medical provider. On-site: Interview program manager if needed; review participant files. EVIDENCE 7-2.A. Pre-site: Please submit a copy of the programs definition of and guidelines (or narrative) for how workers should document immunizations. On-site: Interview program manager and home visitors if needed; review participants files. [Please note: Typically programs will evaluate child immunizations when child reaches a certain age, e.g., nine months. This is satisfactory for rating this standard.]

PAGE 109

Appendix B Continued Fidelity and Innovations 103 RATING INDICATORS 7-2.A. 3. Program follows an identified immunization schedule (e.g., American Academy of Pediatrics, Center for Disease Control, State Department of Public Health, etc.) and has guidelines to document immunizations for all child participants. 2. Program follows an identified immunization schedule (e.g., American Academy of Pediatrics, Center for Disease Control, Department of Public Health, etc.) and has a narrative describing how to document immunizations. 1. Any one of the following: Program does not have an identified immunization schedule; program does not have guidelines or a narrative to document immunizations for all child participants. 7-2.B. Participating children receive immunizations on schedule. RATING INDICATORS 7-2.B. 3. Ninety percent (90%) to one hundred percent (100%) of participant children receive immunizations on schedule. 2. Eighty (80%) to eighty-nine percent (89%) of participant children receive immunizations on schedule. 1. Less than eighty percent (80%) of participant children receive immunizations on schedule. 7-3. Using the family support plan as a guide, families are linked to additional services on an as-needed basis. 7-3.A. The program connects participants to appropriate referral sources and services in the community based upon the information gathered during the development and review of the family support plan. EVIDENCE 7-2.B. Pre-site: Please submit a report detailing the number of all participating children and the percent who are up to date on immunizations. On-site: Interview program manager if needed; review participant files. [Please note: Typically programs will evaluate child immunizations when child reaches a certain age, e.g., nine months. This is satisfactory for rating this standard.] EVIDENCE 7-3.A. Pre-site: Please submit your programs guidelines for using the family support plan to connect participants to appropriate referral sources in the community. On-site: Interview home visitor, participant, and supervisor; review participant files for family support plan and referral reports.

PAGE 110

Appendix B Continued Fidelity and Innovations 104 RATING INDICATORS 7-3.A. 3. The family support plan is used to identify appropriate places in the community for the family to receive needed services. Consistent evidence to indicate that families are connected to appropriate services in the community. 2. The family support plan is used to identify appropriate places in the community for the family to receive needed services. Isolated instances found when families needing referral were not connected to appropriate services in the community. Lack of connection is due to the fact that the referral services are full. 1. Either the program does not identify appropriate places in the community for the family to receive services or there is insufficient evidence to indicate that families needing referral are connected to appropriate services in the community. This lack of connection is not due to the fact that referral services are full. 7-3.B. The program follows up with the referral source, service provider, and/or participant to determine if the participant received needed services. EVIDENCE 7-3.B. Pre-site: Please submit a copy of the programs guidelines (or narrative) describing how it follows up with community resources and/or the participant to determine if the participant received needed services. Be sure to include any relevant documentation such as referral tracking forms, memorandum of understanding with other agencies, etc. On-site: Interview home visitor, participant, and supervisor; review iifilffl RATING INDICATORS 7-3.B. 3. The program has a method for tracking and following up on referrals of participant families to other community services. Consistent evidence indicates that program is routinely tracking and following up on referrals. 2. The program has a method for tracking and following up on referrals of participant families to other community services. Evidence indicates that program is routinely tracking and following up on referrals. Some instances were found when tracking and follow-up did not occur. 1. Any of the following: the program does not have a method; the program has a method but there is insufficient evidence to indicate that tracking and follow-up is routinely occurring.

PAGE 111

Appendix B Continued Fidelity and Innovations 8. Services should be provided by staff with limited caseloads to assure that home visitors have an adequate amount of time to spend with each family to meet their unique and varying needs and to plan for future activities (i.e., for many communities, no more than fifteen (15) families per home visitor on the most intense service level. And, for some communities, the number may need to be significantly lower, e.g., less than ten (10)). 8-1. Services are provided by staff with limited caseloads to assure that home visitors have an adequate amount of time to spend with each family to meet their needs and plan for future activities. 8-1.A. The programs policy regarding established caseload size is no more than fifteen (15) families at the most intensive level (i.e., weekly visits) per full time home visitor. 105 R EVIDENCE 8-1.A. 8-1.C. Pre-site: Please submit a copy of the programs policy regarding caseload sizes and a report showing the average caseload size of all current home visitors over the past year, as well as an individual breakdown of each home visitors caseload for the quarter immediately prior to submitting the application for credentialing, the number of families assigned to him/her, and the intensity of service each family is receiving. On-site: Review caseload listings, interview supervisor and home visitors. RATING INDICATORS 8-1.A. 3. No 3" rating indicator for standard 8-1.A. 2. The program policy states that caseload size is no more than fifteen (15) families at the most intensive service level (i.e., weekly visits) per full time home visitor. 1. The program policy states that caseload size is more than fifteen (15) families at the most intensive service level (i.e., weekly visits) per full time home visitor. 8-1.B. The programs policy regarding maximum caseload size is no more than twenty-five (25) at any combination of service levels per full-time home visitor. RATING INDICATORS 8-1.B. 3. No 3" rating for 8-1.B. 2. The programs policy regarding maximum caseload size is no more than twenty-five (25) families at any combination of service levels per full time home visitor. 1. The programs policy regarding maximum caseload size exceeds twenty-five (25) families at any combination of service levels per full time home visitor. 8-1.C. Home visitors are within the caseload ranges, as stated in standard 8-1.A and 8-1.B.

PAGE 112

Appendix B Continued Fidelity and Innovations RATING INDICATORS 8-1.C. 3. No home visitor exceeds the caseload sizes, as stated in standards 8-1.A and 8-1B. 2. Most home visitors do not exceed the caseload sizes as stated in 8-1.A and 8-1.B. Evidence suggests any deviation is temporary. 1. Home visitors routinely exceed the caseload sizes as stated in 81.A. and 8-1.B. 8-2. The programs caseload system ensures that home visitors have an adequate amount of time to spend with each family. 8-2.A. The program uses criteria as specified in the rating indicators (8-2.A) to manage its caseload sizes. 106 EVIDENCE 8-2.A. 8-2.B. Pre-site: Please submit a copy of the criteria the program uses to manage the caseloads and any relevant forms used in managing this system. On-site: Interview supervisor and home visitor; review documentation of caseload management system. RATING INDICATORS 8-2.A. 3. The program has criteria it routinely uses in the management of its caseloads. Criteria must include: the nature and difficulty of the problems encountered; the work and time required to serve each family; the number of families per service provider which involve assessment and/or more intensive intervention; travel and other non-direct service time required to fulfill the service providers responsibilities; and extent of other resources available in the community to meet family needs. 2. The program has criteria for managing its caseload, but isolated instances were found when criteria were not used. 1. The program has no criteria or criteria are not being used routinely. 8-2.B. The caseload management system ensures that each home visitor has adequate time to spend with each family. RATING INDICATORS 8-2.B. 3. Home visitors have an adequate amount of time to spend with each family. 2. Isolated instances found when home visitors have insufficient amount of time to spend with each family. However, program is aware of the situation and taking appropriate steps to remedy it.

PAGE 113

Appendix B Continued Fidelity and Innovations 107 1. Home visitors routinely do not have adequate amount of time to spend with each family. 9. Service providers should be selected because of their personal characteristics (i.e., nonjudgmental, compassionate, ability to establish a trusting relationship, etc.), their willingness to work in or their experience working with culturally diverse communities, and their skills to do the job. 9-1. Service providers and program management staff are selected because of a combination of personal characteristics, experiential, and educational qualifications. 9-1.A. Screening and selection of direct service staff include consideration of personal characteristics, including but not limited to: acceptance of individual differences; ability to establish trusting relationships; and, experience and willingness to work with the culturally diverse populations which are present among the programs target population. EVIDENCE 9-1.A. Pre-site: Please submit a description of the programs screening and selection procedures for direct service staff. Be sure to include any relevant materials used during the screening/selection procedure, such as interview guidelines, job descriptions, etc. On-site: Interview supervisors/managers, direct service staff. RATING INDICATORS 9-1.A. 3. Programs system for screening and selection of direct service staff ensures that it considers personal characteristics of job candidates, including, but not limited to, acceptance of individual differences; ability to establish trusting relationships; and experiences and willingness to work with the diverse population(s) which are present among the programs target population. 2. Isolated instances found when the programs system was not followed when hiring or system design is not as effective in screening for personal characteristics as possible. 1. Any of the following: the program does not have a system for screening and selection of direct service staff that ensures it considers personal characteristics of job candidates, including acceptance of individual differences; ability to establish trusting relationships; and experiences and willingness to work with the diverse population(s) which are present among the programs target population; the system is not followed when hiring; the system does not screen for the characteristics listed above; or, the system, as designed, does not screen for personal characteristics. 9.1.B. Direct service providers: are experienced in working with or providing services to children and families;

PAGE 114

Appendix B Continued Fidelity and Innovations are able to observe and report accurately on the functioning of individuals and families; are emotionally mature and capable of exercising judgment; are able to handle stressful situations; and, meet the educational requirements, as established by the program. RATING INDICATORS 9-1.B. 3. Direct service staff routinely meet the criteria stated. 2. Isolated instances found when direct service staff did not meet all of the criteria. 1. Direct service staff routinely do not meet the stated criteria stated. 9-1.C. Managers/supervisors have: solid understanding and experience in managing/motivating staff as well as providing support in stressful work environments; administrative experience in human service program(s); experience with family services that embrace the concepts of family-centered and strength-based service provision; knowledge of maternal-infant health and concepts of child abuse and neglect; experience in providing services to culturally diverse communities/families; experience in home visitation with a strong background in prevention services to the 0-3 age population; and Masters degree in human services preferred. EVIDENCE 9-1.B. Pre-site: Please attach a list of all direct service staff, along with a summary (e.g., resume) of their qualifications for the position he/she holds. Also, please attach a copy of all the job descriptions for direct service staff. On-site: Interview directservice staff su p ervisors and mana g ers. EVIDENCE 9-1.C. Pre-site: Please attach a summary of the qualifications of all program management staff (e.g., a resume or vitae, for example). Be sure the summary includes work and educational experience. On-site: Interview program management. RATING INDICATORS 9-1.C. 3. Supervisors/managers routinely meet the criteria stated in the standard for 9-1.C. 2. Isolated instances found when supervisors/managers did not meet the criteria. 108

PAGE 115

Appendix B Continued Fidelity and Innovations 1. Supervisors/managers routinely do not meet the criteria stated in the standard. 9-1.D. The same expectations/requirements apply to both direct service staff and volunteers/interns performing the same function. RATING INDICATORS 9-1.D. 3. No "3" rating indicator for this standard. 2. The same expectations/requirements apply to both direct service staff and volunteers/interns performing the same function. 1. The program does not apply the same expectations/requirements to volunteers/interns performing the same function as staff. NA The program does not have volunteers/interns performing the same functions as direct service staff. 9-2. The program actively recruits, employs, and promotes qualified personnel broadly representative of the community it serves and administers its personnel practices without discrimination based upon age, sex, race, ethnicity, nationality, handicap, or religion of the individual under consideration. 9-2.A. The program is in compliance with the Equal Opportunity Act in the United States. EVIDENCE 9-1.D. Pre-site: Please submit a copy of the programs policy on expectations, requirements of skills and characteristics of volunteers/interns who perform the same function as direct service staff. On-site: Interview direct service staff, volunteers/interns, and program managers. EVIDENCE 9-2.A. On-site: Team will review materials, such as correspondence with regulatory authorities indicating that there are no known problems or a legal opinion from their attorney indicating the agencys practices conform to the law. In the absence of such materials, the agency may provide a statement indicating whether there have been any findings or rulings against their practices in the past four years. RATING INDICATORS 9-2.A. 3. There have been no administrative findings or court rulings against the program in this respect. 109

PAGE 116

Appendix B Continued Fidelity and Innovations 2. Status is under review and pending final determination; no major difficulties have been identified in the process of a review conducted by a regulatory authority. 1. The program is in process of remediation of identified difficulty, or the program is not in compliance with the applicable law and has not begun corrective action. 9-2.B. The program has a written equal opportunity policy which clearly states its practices in recruitment, employment, transfer and promotion of employees. R RATING INDICATORS EVIDENCE 9-2.B. 9-2.C. Pre-site: Provide a copy of the policy. On-site: Copies of dissemination materials, such as posters, statements inserted in advertisements, brochures, etc., which specify the non-discriminatory nature of program personnel practices. 9-2.B. 3. Written policy on equal opportunity guides program practices in all four listed areas of personnel administration. 2. Policy, though written, is slightly limited in scope and there are no known violations of equal employment opportunity. Or, despite lack of written policy, practice seems non-discriminatory and generally equitable. 1. The program has no policy. 9-2.C. The program disseminates its equal opportunity policy and uses recruitment materials that specify the non-discriminatory nature of the programs employment practices. RATING INDICATORS 9-2.C. 3. The program posts its equal employment opportunity policy and includes a statement with recruitment material and advertisements which specifies the non-discriminatory nature of the programs employment practices. 2. The program uses limited means of disseminating information on its non-discriminatory hiring practices. 1. The program does not disseminate information internally or externally on its position on equal opportunity. 9-3. The programs recruitment and selection procedures assures that its the human resource needs are met. 110

PAGE 117

Appendix B Continued Fidelity and Innovations 9-3.A. The programs recruitment and selection practices are in compliance with applicable law or regulation and include: notification of its personnel of available positions before or concurrent with recruitment elsewhere; personal interviews with applicants before selection; and, documentation that three references from unrelated persons have been obtained. EVIDENCE 9-3.A. 9-3.B. Pre-site: If the program has an opinion from counsel that guides its practices, provide a copy. If not, provide a list of statutes or regulations governing, restricting, or mandating background checks. Describe briefly the legal requirements of each and the mechanisms the program has in place for compliance. On-site: Interview personnel and review personnel records. Team will review personnel records and interview senior personnel. RATING INDICATORS 9-3.A. 3. The program always posts or otherwise informs its own personnel of career advancement or permanent job opportunities in order that they might apply. For all positions, program procedures assure that a personal interview is conducted with each person in the final pool of applicants. In general, program interviews personally all finalists for positions although there may be occasional exceptions. Three references are routinely documented in records. 2. There have been isolated instances when the program has failed to notify personnel who have the required qualifications for the opening. Program has criteria which insure that positions involving frequent or intensive client contact are interviewed prior to that contact; it does not necessarily interview all volunteers before they are retained, but may use group interviewing for some applicants or some positions in the case of volunteers. Program practice is to obtain three references; in those records with less than three references, those obtained are clear and compelling or additional non-reference information is documented which confirms the applicants' suitability. 1. The program deviates in one or more ways such as the following: No advance notice to give personnel an opportunity to seek positions ahead of others. Program relies solely upon group rather than individual interviews for volunteer personnel; or program practice regarding interviews is uneven or haphazard with no criteria governing omissions and exceptions. Program has retained one or more persons with management or client service responsibility without an interview. Only references sought are in management, direct service, or only one reference is obtained routinely, or, the program practices are disorganized, inconsistent, inefficient and/or careless. 111

PAGE 118

Appendix B Continued Fidelity and Innovations 9-3.B. The agency conducts appropriate, legally permissible and mandated inquiries into the background of prospective employees and volunteers who will have responsibilities where clients are children. RATING INDICATORS 9-3.B. 3. Program records are complete with regard to additional screening allowed by law in all cases when there is likely to be contact with vulnerable participants such as children. The program is knowledgeable about what is legally permissible and usable in screening applicants. It carefully follows all mandates. 2. Program practice suggests that it conducts background checks on all employees and volunteers who deal with vulnerable participants but has not maintained good documentation of their efforts in all cases. 1. Program neglects to conduct legally permissible background checks on some applicants or for personnel dealing with vulnerable participants and/or program fails to conduct mandated background checks in all cases. NA Background checks are not permissible in the state. Program provides copy of law or legal opinion. 9-3.C. The rate of personnel turnover is measured and evaluated regularly and action is taken to correct identified problems. R RATING INDICATORS EVIDENCE 9-3.C. Pre-site: Please submit a description of the way the program measures and evaluates its personnel turnover and submit an analysis of personnel turnover rate for the most recent past two years. This analysis should be done both for the entire program and for the various job categories in the program. Indicate what, if any, action was taken as a result of this analysis. On-site: Interview personnel and program management and review personnel records. 9-3.C. 3. Measurement and evaluation of the turnover rate of employees and, when utilized, volunteers, is a regular and integral part of program personnel planning, and action to correct identified problems is promptly taken. Turnover rates are examined by specific job categories to identify any aberrant levels of turnover specific to certain categories, or sites, for example. Turnover rates are examined in the context of measures of job satisfaction and personnel retention. 2. Recently, program began to evaluate personnel turnover rates, and its is beginning to use this as a means of identifying problems it should address. 112

PAGE 119

Appendix B Continued Fidelity and Innovations 1. Program has experienced personnel turnover and turnover rates are irregularly evaluated, little impact of findings seen in addressing need for change; or, no assessment; or assessment has no evident impact in face of reports to the team by personnel of high turnover. NA Program has not experienced personnel turnover. 10.a Service providers should have a framework, based on education or experience, for handling the variety of experiences they may encounter when working with at-risk families. All service providers should receive basic training in areas such as cultural competency, substance abuse, reporting child abuse, domestic violence, drug-exposed infants, and services in their community. 10.b Service providers should receive intensive training specific to their role to understand the essential components of family assessment and home visitation (i.e., identifying at-risk families, completing a standardized risk assessment, offering services and making referrals, promoting use of preventive health care, securing medical homes, emphasizing the importance of immunizations, utilizing creative outreach efforts, establishing and maintaining trust with families, building upon family strengths, developing an individual family support plan, observing parent-child interactions, determining the safety of the home, teaching parent-child interaction, managing crisis situations, etc.). NOTE: In order to streamline the responses to critical elements 10 and 11 (which address worker skills and training), we have combined the two critical elements and are measuring them as one section. 10-1. The program has a system to ensure it can track all of the trainings it provides to all of the programs service providers (paid staff, interns, and volunteers). R A RATING INDICATORS EVIDENCE 10-1. Pre-site: Please submit a copy of the policy regarding staff participation in orientation (basic) training; guidelines (or narrative) describing the system used by the program to ensure all new staff are trained (i.e., a training log, etc.). On-site: Interview supervisors, review staff personnel files or training log. 10-1. 3. The system used by the program to monitor staff training ensures comprehensive and timely training for all staff and the system is designed in such a way that it is easy to track and access training of staff (e.g., training log). 2. The system ensures comprehensive and timely training for all service providers. The system is designed in such a manner in which trainings are not easily tracked or accessed (e.g., only record of training is in individual personnel files). 1. There is no system or the system does not ensure comprehensive and timely training of all service providers. 113

PAGE 120

Appendix B Continued Fidelity and Innovations 10-2. All service providers (includes paid staff, interns and volunteers), receive orientation training prior to direct work with children and families to familiarize them with the functions of the program. 10-2.A. Service providers are oriented to the programs goals, services, policies and operating procedures prior to direct work with children and families. EVIDENCE 10-2.A. through 10-2.E. Pre-site: Please submit a copy of the orientation curriculum used by the program with new hires and a copy of all current program staff, his/her date of hire, and the date he/she completed the orientation training. On-site: Interview staff and supervisors; review personnel records/training log. RATING INDICATORS 10-2.A. 3. All service providers are routinely oriented to the programs goals, services, policies and operating procedures prior to direct work with children and families. 2. Most service providers are oriented to the programs goals, services, policies and operating procedures prior to direct work with children and families. 1. Service providers are routinely not oriented to the programs goals, services, policies and operating procedures prior to direct work with children and families. 10-2.B. Service providers are oriented to the programs relationship with other community resources prior to direct work with children and families. RATING INDICATORS 10-2.B. 3. All service providers are routinely oriented to the programs relationship with other community resources (i.e., organizations in the community with which the program has working relationships) prior to direct work with children and families. 2. Most service providers are oriented to the programs relationship with other community resources prior to direct work with children and families. 1. Service providers are routinely not oriented to the programs relationship with other community resources prior to direct work with children and families. 10-2.C. Service providers are oriented to child abuse and neglect indicators and reporting requirements prior to direct work with children and families. 114

PAGE 121

Appendix B Continued Fidelity and Innovations RATING INDICATORS 10-2.C. 3. All service providers are routinely oriented to child abuse and neglect indicators and reporting requirements prior to direct work with children and families. 2. Most service providers are oriented to child abuse and neglect indicators and reporting requirements prior to direct work with children and families. 1. Service providers are routinely not oriented to child abuse and neglect indicators and reporting requirements prior to direct work with children and families. 10-2.D. Service providers are oriented to the history and philosophy of home visitation prior to direct work with children and families. RATING INDICATORS 10-2.D. 3. All service providers are routinely oriented to the history and philosophy of home visitation prior to direct work with children and families. 2. Most service providers are oriented to the history and philosophy of home visitation prior to direct work with children and families. 1. Service providers are routinely not oriented to the history and philosophy o home visitation prior to direct work with children and families. 10-2.E. Service providers are oriented to issues of confidentiality. RATING INDICATORS 10-2.E. 3. All service providers are oriented to issues of confidentiality prior to direct work with families. 2. Isolated instances found when service providers were not oriented to issues of confidentiality prior to direct work with families. 1. Service providers are routinely not oriented to issues of confidentiality prior to direct work with families. 10-3. All service providers (paid staff, interns, and volunteers) receive orientation trainings within six months of hire date on a variety of topics necessary for effectively working with over-burdened families. 10-3.A. All service providers receive training on Infant Care within six months of the date of hire. 115

PAGE 122

Appendix B Continued Fidelity and Innovations RATING INDICATORS 10-3.A. 3. All service providers are routinely trained in Infant Care within the specified time frame. (Topics may include basics of baby care; breast feeding; the art of nurturing; understanding attachment; infant development and mental health; infant massage; infant well care/medical risk indicators; therapeutic touch; understanding parent-child interaction.) 2. Isolated instances found when training did not occur in the specified time frame. 1. Service providers are routinely not trained in Infant Care within the specified time frame. 10-3.B. All service providers receive training on Child Development and Health within six months of the date of hire. RATING INDICATORS 10-3.B. 3. All service providers are routinely trained in Child Development and Health within the specified time frame. (Topics may include childhood health issues, wellness, nutrition, child management techniques, etc.) 2. Isolated instances found when training did not occur in the specified time frame. 1. Service providers are routinely not trained in child development and health within the specified time frame. 10-3.C. All service providers receive training on Parental Health and Well-Being within six months of the date of hire. RATING INDICATORS 10-3.C. 3. All service providers are routinely trained in parental health and well-being within the specified time frame. (Topics may include personal health issues, prenatal care, post-partum care, family planning.) 2. Isolated instances found when training did not occur in the specified time frame. 1. Service providers are routinely not trained in parental health and well-being within the specified time frame. 10-3.D.All service providers receive training on Language Development within six months of the date of hire. RATING INDICATORS 10-3.D. 3. All service providers are routinely trained in Language Development within the specified time frame. (Topics may include early literacy, the effects of language on behavior and children.) 116

PAGE 123

Appendix B Continued Fidelity and Innovations 2. Isolated instances found when training did not occur in the specified time frame. 1. Service providers are routinely not trained in Language Development within the specified time frame. 10-3.E. All service providers receive training on the Role of Culture in Parenting within six months of the date of hire. RATING INDICATORS 10-3.E. 3. All service providers are routinely trained in the Role of Culture in Parenting within the specified time frame. (Topics may include culture and prenatal care, family cultural issues, etc.) 2. Isolated instances found when training did not occur in the specified time frame. 1. Service providers are routinely not trained in the Role of Culture in Parenting within the specified time frame. 10-3.F. All service providers receive training on Family Violence within six months of the date of hire. RATING INDICATORS 10-3.F. 3. All service providers are routinely trained in Family Violence within the specified time frame. (Topics may include dynamics of domestic violence, how to support/intervene effectively in cases of domestic violence, incest issues and domestic violence, child abuse and neglect.) 2. Isolated instances found when training did not occur in the specified time frame. 1. Service providers are routinely not trained in Family Violence within the specified time frame. 10-3.G. All service providers receive training on Substance Abuse within six months of the date of hire. RATING INDICATORS 10-3.G. 3. All service providers are routinely trained in Substance Abuse within the specified time frame. (Topics may include dealing with issues of dependency in families, adult children of alcoholics, alcohol and addiction, drug exposed babies, identification of substance abuse, family dynamics of chemical dependency, working with participants with addictions, motivation and addictive behaviors) 2. Isolated instances found when training did not occur in the specified time frame. 117

PAGE 124

Appendix B Continued Fidelity and Innovations 118 1. Service providers are routinely not trained in Substance Abuse within the specified time frame. 10-3.H. All service providers receive training on Parental Issues within six months of the date of hire. RATING INDICATORS 10-3.H. 3. All service providers are routinely trained in Parental Issues within the specified time frame. (Topics may include engaging fathers in parenting, family planning, family systems and life cycles, stress/time management, working with new parents, appropriate discipline, the effects of welfare reform on the family, preparing a family budget and making it work, etc.) 2. Isolated instances found when training did not occur in the specified time frame. 1. Service providers are routinely not trained in Parental Issues within the specified time frame. 10-3.I. All service providers receive training on HIV/AIDS within six months of the date of hire. RATING INDICATORS 10-3.I. 3. All service providers ar e routinely trained in HIV/AIDS within the specified time frame. (Topics may include HIV/AIDS impact on service families, AIDS education, AIDS and STDs, cross cultural aspects of HIV/AIDS, etc.) 2. Isolated instances found when training did not occur in the specified time frame. 1. Service providers are routinely not trained in HIV/AIDS within the specified time frame. 10-3.J. All service providers receive training on Staff-Related Subjects within six months of the date of hire. RATING INDICATORS 10-3.J. 3. All service providers are routinely trained in Staff-Related Subjects/Working with New Parents within the specified time frame. (Topics may include professional boundary issues, limit setting, the effects of welfare reform on the service population, crisis intervention with families, ethical issues in home visiting, problem solving without imposing personal values, communication skill building, grieving, interviewing techniques, developing parent support groups, problem solving and negotiation, avoiding staff burnout, using resource materials and community resources, values clarification, etc.) 2. Isolated instances found when training did not occur in the specified time frame.

PAGE 125

Appendix B Continued Fidelity and Innovations 1. Service providers are routinely not trained in Staff-Related Subjects/Working with New Parents within the specified time frame. 10-4. All service providers and supervisory staff receive intensive training within six months of date of hire specific to their role within the home visitation program to help them understand the essential components of their role within the program. 10-4.A. All assessment workers have received intensive training within six months of date of hire to understand the essential components of family assessment. R A EVIDENCE 10-4.A. through 10-4.C. Pre-site: Please submit a copy of the training outline(s) used to train workers about their specific role within the home visitation program. Also, include a list of all direct service and supervisory staff, his/her date of hire, and the date he/she completed the role-specific training. On-site: Review training certificates in personnel files; training log; interview program staff and supervisors. NOTE:Forpr ogramswhousedtheHFATrainingthroughPCAAmericaevidenceof RATING INDICATORS 10-4.A.3. Assessment workers routinely receive intensive training on the essential components of family assessment within six months of the date of hire. Training covers topics such as the role of family assessment, identifying overburdened families, interviewing skills, conducting risk assessments, completing necessary paperwork and documentation, family-centered support services, communication skills, etc. 2. Isolated instances found when assessment workers did not receive training within the specified time frame. 1. Assessment workers routinely do not receive such training within specified time frame or training does not sufficiently address the role of the assessment worker. 10-4.B. All home visitors have received intensive trainings within six months of date of hire to understand the essential components of home visitation. RATING INDICATORS 10-4.B. 3. Home visitors routinely receive intensive training on the essential components of home visitation within six months of the date of hire. Training covers topics such as establishing and maintaining trust with families, completing necessary paperwork/documentation, the role of the home visitor, communication skills, crisis intervention, etc. 2. Isolated instances found when home visitors did not receive training within the specified time frame. 119

PAGE 126

Appendix B Continued Fidelity and Innovations 1. Home visitors routinely do not receive such training within specified time frame or training does not sufficiently address the role of the home visitor. 10-4.C. All supervisory staff have received intensive trainings within six months of date of hire to understand the essential components of their role within the home visitation program, as well as the role of family assessment and home visitation. RATING INDICATORS 10-4.C. 3. Supervisory staff routinely receive intensive training on the essential components of assessment, home visitation, and supervision within six months of the date of hire. Training covers topics such as quality management techniques, crisis management, understanding the programs policies and procedures, case management, effective supervision, the role of family assessment and home visitation. 2. Isolated instances found when supervisory staff did not receive training within the specified time frame. 1. Supervisory staff routinely do not receive such training within specified time frame or training does not sufficiently address the role of one or more of the following: assessment, home visiting, supervision. 10-5. The program ensures that all program staff receive ongoing training which takes into account the workers knowledge and skill base. R R ATING INDICATORS EVIDENCE 10-5. Pre-site: Please submit a copy of the programs guidelines for training beyond the orientation period; any training outlines of trainings which have been conducted; and a list of all staff, his/her date of hire, and the ongoing training(s) completed by the staff. On-site: Review personnel files for certificates of further training; training log; interview staff and supervisors. 10-5. 3. The program ensures that all program staff routinely receive ongoing training beyond the orientation training. Consistent evidence indicates that: 1) Staff are offered and participate in ongoing training, and 2) Current topics covered in training take into account workers knowledge and skill base. 2. Isolated instances were found when staff did not participate in ongoing training or topics covered in training did not take into account workers' knowledge and skill base. 1. The program does not ensure that program staff routinely receive ongoing trainings or staff does not routinely participate in ongoing training opportunities. 120

PAGE 127

Appendix B Continued Fidelity and Innovations 10-6. Programs which have volunteers and/or student interns working in similar positions as paid staff ensure the volunteers/interns receive the same type of training as paid staff. RATING INDICATORS 10-6. 3. The program requires paid staff, volunteers, and interns performing similar functions to have the same type of training. Consistent evidence indicates that all volunteers, interns and paid staff in similar positions are being trained in a similar manner. 2. The program requires paid staff, volunteers, and interns performing similar functions to have the same type of training. Evidence indicates that most volunteers interns, and paid staff in similar positions are being trained in a similar manner. 1. Any of the following: program does not require paid staff, volunteers, and interns performing similar functions to have the same type of training; insufficient evidence to indicate that most volunteers, paid staff, and interns in similar positions are being trained in a similar manner. NA Program does not have paid staff and volunteers/interns performing similar functions. 11. Service providers should receive ongoing, effective supervision so that they are able to develop realistic and effective plans to empower families to meet their objectives; to understand why a family may not be making progress and how to work with the family more effectively; and to express their concerns and frustrations so that they can see that they are making a difference and in order to avoid stress-related burnout. 11-1. The program ensures that direct service staff receive regular, ongoing, and effective supervision. 11-1.A. The program ensures that weekly supervisory time is provided to all direct service staff. EVIDENCE 10-6. Pre-site: Please submit a copy of the programs policy on training volunteers/interns in both the orientation phase and ongoing phase of training. On-site: Review personnel files/training log of staff, volunteers, and interns; interview staff; interview supervisor; interview volunteer/interns. EVIDENCE 11-1.A. Pre-site: Please submit a copy of the programs policy on supervision of direct service staff. On-site: Review supervision logs and interview direct service staff and supervisors. 121

PAGE 128

Appendix B Continued Fidelity and Innovations RATING INDICATORS 11-1.A. 3. Program policy specifies supervisor provides two hours/week of supervisory time available to each direct service staff and consistent evidence indicates that the program is following these guidelines. 2. Program policy specifies supervisor provides at least one and one half hours/week of supervisory time available to each direct service staff and consistent evidence indicates that the program is following these guidelines. 1. Any of the following: Program has no policy on amount of supervisory time provided to each direct service staff member; program policy on supervisory time provided to each direct service staff is less than 1 2 hours/ week; there is insufficient evidence to indicate that the program is following the acceptable guidelines as outlined in 2 and 3 above. 11-1.B. The ratio of supervisors to direct service staff is sufficient to allow regular, ongoing, and effective supervision to occur. EVIDENCE 11-1.B. Pre-site: Please submit a list of each supervisor and the staff he/she supervises. On-site: Review documentation to support supervisor/staff ratio as submitted above (i.e., supervision logs, etc.), interview direct service staff and supervisors. RATING INDICATORS 11-1.B. 3. The ratio of supervisors to direct service staff is one full time supervisor to five full time direct service staff. Consistent evidence indicates the program is following these guidelines. 2. The ratio of supervisors to direct service staff is one (1) full time supervisor to six (6) full time direct service staff. Consistent evidence indicates the program is following these guidelines. 1. Any one of the following: Program has no policy on the ratio of supervisors to direct service staff; program policy on the ratio specifies more than six (6) full time direct service staff to one (1) full time supervisor; or there was insufficient evidence that the program is following the acceptable guidelines as outlined in 2 and 3 above. 11-2. Direct service staff are held accountable for the quality of their work and provided with skill development and professional support. 11-2.A. The programs supervisory procedures assure that direct service staff are held accountable for the quality of their work. 122

PAGE 129

Appendix B Continued Fidelity and Innovations RATING INDICATORS 11-2.A.3. The programs supervisory procedures ensure that all direct service staff are held to a high standard of quality for their work and evidence suggests that procedures are routinely used. Procedures may include: regular and routine review of assessments and assessment records; regular and routine review of cases and home visitor records; feedback to direct service staff on approaches and interventions used; supervisor regularly monitors all types of documentation used in the program; etc. 2. Programs supervisory procedures ensure that all direct service staff are held to a high standard of quality for their work. Isolated instances found when the procedures were not used. 1. Any of the following: Program has no procedures; procedures do not adequately ensure staff are held to high standard of quality for their work; evidence suggests procedures are not routinely used. 11-2.B. The program has supervisory procedures to assure that direct service staff are provided with the necessary skill development and professional support to continuously improve the quality of their performance. EVIDENCE 11-2.A. Pre-site: Please submit a copy of the programs supervisory procedures which address staff accountability. On-site: Review documentation which would establish that the procedures are followed (i.e., supervisory logs, etc.) And, interviews direct service staff and supervisors. EVIDENCE 11-2.B. Pre-site: Please submit a copy of the programs supervisory procedures which address skill development/professional support. On-site: Review documentation to support that supervisory procedures are being followed (i.e., supervision logs, etc.), interviews direct service staff and supervisors. RATING INDICATORS 11-2.B.3. Program has supervisory procedures which assure that all direct service staff receive skill development and professional support (other than training) to continuously improve the quality of their performance and consistent evidence indicates that the program is following its procedures. Procedures can include a variety of mechanisms, such as regular staff meetings, on-call availability to service providers, creating a nurturing work environment that provides opportunities for respite, scheduling flexibility, providing a career ladder for direct service staff, etc. 2. Isolated instances found when the program not following its procedures. 123

PAGE 130

Appendix B Continued Fidelity and Innovations 1. Any of the following: program has no procedures; procedures do not adequately ensure staff receives skill development and professional support; evidence suggests procedures are not routinely used. 11-3. The programs Policies and Procedures Manual is used to guide newer service providers in the delivery of services. 11-3.A. The program has a Policies and Procedures Manual. RATING INDICATORS 11-3.A. 3. No "3" rating indicator for 11-3.A. 2. Program has a Policies and Procedures Manual. 1. Program does not have a Policies and Procedures Manual. 11-3.B. The program uses Policies and Procedures Manual as a guide in the provision of services. RATING INDICATORS 11-3.B. 3. No "3" rating indicator for 11-3.B. 2. Sufficient evidence to indicate that the program routinely uses the manual as a guide in the provision of services, particularly for newer employees. 1. Insufficient evidence to indicate that the program routinely uses the manual as a guide in the provision of services, particularly for newer employees or program does not have a policies and procedures manual. 11-4. Volunteers and student interns who are performing the same/similar functions as direct service staff are receiving the same type and amount of supervision. EVIDENCE 11-3.A. Pre-site: Please submit a copy of the programs policies and procedures manual. On-site: Interview supervisors and/or direct service workers. EVIDENCE 11-3.B. Pre-site: Orientation protocols (or narrative) describing use of policies and procedures manual, especially for new service providers. On-site: Interview direct service staff and supervisors. EVIDENCE 11-4. Pre-site: Guidelines for volunteer supervision or narrative describing volunteer supervision. Ons ite: Interview with volunteer interview su p ervisor. 124

PAGE 131

Appendix B Continued Fidelity and Innovations RATING INDICATORS 11-4. 3. Supervision of volunteers and interns follow same guidelines as supervision of direct service staff who are performing similar tasks. 2. There have been isolated instances/irregularities in practice when volunteers and/or interns are not supervised in same manner as direct service staff. 1. The program does have volunteers and/or interns performing similar functions to direct service staff, but insufficient evidence exists that supervision of volunteers and/or interns follows same guidelines as supervision of paid employees performing similar tasks. NA The program does not have volunteers or interns performing same functions as direct service staff. 11-5. Program managers and supervisors are held accountable for the quality of their work and are provided with skill development and professional support. EVIDENCE 11-5. Pre-site: Guidelines for supervision of program manager and supervisors or narrative describing supervision. On-site: Interview with supervisor of program manager, interview program manager and supervisor; interview with director of host agency. RATING INDICATORS 11-5. 3. Supervision of program manager and supervisors ensures that they are held accountable for the quality of their work and routinely receive skill development and professional support. 2. There have been isolated instances/irregularities in practice when the program manager and/or the supervisors are not supervised in such a way to ensure quality work, skill development and professional support. 1. There is insufficient evidence to indicate that supervision of program manager and supervisors ensures quality work, skill development and professional support. GOVERNANCE AND ADMINISTRATION The program is governed and administered in accordance with principles of effective management and of ethical practice. GA-1. The program has a written statement of purpose that guides the administration of its services. 125

PAGE 132

Appendix B Continued Fidelity and Innovations GA-1.A. The program has a written statement of purpose that reflects the goals and criteria contained in the critical elements and the needs of children, families, and the community. RATING INDICATORS GA-1.A.3. The program has a written statement of purpose that reflects the goals and criteria contained in the critical elements and the needs of children, families, and the community. 2. The program has a written statement of purpose, however it does not address all of the goals and criteria listed in the standard. 1. Either the program does not have a written statement of purpose or the programs written statement of purpose does not reflect the intent of the standard. GA-1.B. The statement is reviewed formally by the programs organizing group at least every four (4) years. R A T I N RATING INDICATORS GA-1.B.3. The programs organizing body reviews its written statement of purpose every four (4) years. 2. The program reviews its written statement of purpose every four (4) years; however, the review is less formal or rigorous. 1. The program does not review its written statement of purpose at least every four (4) years. GA-2. The program has broadly-based, organized group (e.g., a voluntary Board, governing body, an advisory committee, etc., hereinafter referred to as the organizing group) which serves in a governing and/or advisory capacity in the planning, implementation, and assessment of program services. GA-2.A. The programs organizing group is an effectively organized, active body carrying out the functions specified in GA-2. EVIDENCE GA-1.A. Pre-site: Please submit a copy of the programs written statement of purpose. On-site: Interview member of organizing group, program manager. EVIDENCE GA-1.B. Pre-site: Please indicate how often the written statement of purpose is formally reviewed and by whom. On-site: Interview member of organizing group, program manager. EVIDENCE GA-2.A. Pre-site: Please submit a copy of the organizing groups by-laws. On-site: Interview member of organizing group, program manager, and review meeting minutes. 126

PAGE 133

Appendix B Continued Fidelity and Innovations RATING INDICATORS GA-2.A. 3. The programs organizing group is an effectively organized, active body which carries out the activities of planning, implementation, and assessment of program services. 2. The programs organizing group carries out the specified functions, but could be more active in one area of functioning. 1. The programs organizing group is not active or is ineffective in one or more of the major areas of its responsibilities. GA-2.B. The organizing group has a wide range of needed skills and abilities and provides a heterogeneous mix in terms of skills, strengths, community knowledge, professions, age, race, sex, nationality or ethnicity. R A RATING INDICATORS EVIDENCE GA-2.B. and GA-2.C. Pre-site: Please submit a profile of each member of the organizing group (the affiliation, position in his/her organization, length of time on the board, and a brief summary of experience and/or reason leading to membership in the group). On-site: Interview member of organizing group, program manager. GA-2.B. 3. The organizing body has a wide range of skills, abilities, and provides a heterogeneous mix in terms of skills, strengths, community knowledge, professions, age, race, sex, nationality/ethnicity. 2. The organizing bodys membership with representative skills, knowledge and interests, but is weak in some of the specialized areas listed in the standard. 1. The organizing bodys membership is not diverse. GA-2.C. The organizing group is aware of community issues that affect program participants, program planning, implementation, and assessment, either through direct representation by community members/program participants or another effective alternative. RATING INDICATORS GA-2.C. 3. The organizing group is aware of community issues as they affect program participants, program planning, implementation and assessment through direct representation by community members/program participants in the group. 127

PAGE 134

Appendix B Continued Fidelity and Innovations 128 2. The organizing group is aware of community issues as they affect program participants, program planning, implementation and assessment. However, the representation is not direct. 1. The organizing body does not serve as an effective link with the community. GA-3. The program has a mechanism in place for families (i.e., past or present participants) to serve on the organizing board group or otherwise provide formalized input into the program. R RATING INDICATORS GA-3. 3. The program has a formal mechanism for families to serve on the organizing group or some other means to encourage formalized input into the program. Families are represented and those members report satisfaction with their representation and input in the process. 2. The program has informal mechanisms for family input. Families are able to serve on the organizing group but may or may not be aware of this. 1. There are no means for families to have input into the program. GA-4. The manager of the home visitation program works with the organizing body to plan and develop program policy. GA-4.A. The manager of the program involves, consults, and gives leadership to the organizing body in the planning, policy, and fiscal decision making process. EVIDENCE GA-3. Pre-site: By-laws/operating procedures, profile of organizing body members (as submitted in GA-2.A.), narrative describing how program obtains input into program from families (e.g., through participant surveys, etc.) On-site: Interview family members and/or other families who have provided input into program (formally by serving on the organizing group or informally); interview program manager. EVIDENCE GA-4.A. and GA-4.B. Pre-site: Please submit a copy of the programs policies, guidelines, or narrative describing the role of the program manager and the organizing group. On-site: Interview Program Manager and member of organizing group. RATING INDICATORS GA-4.A.3. The manager of the program involves, consults and gives leadership to the organizing body in the planning, policy, and fiscal decision-making process. 2. The manager plans and consults with the organizing body but does not fully involve them in the decision-making process.

PAGE 135

Appendix B Continued Fidelity and Innovations 129 1. The manager does not share information or decision-making with the organizing body. GA-4.B.The manager and the organizing body work as an effective team with information, coordination, staffing, and assistance provided by the manager to support the Board in its policy making and other oversight functions. RATING INDICATORS GA-4.B 3. The manager and the organizing body work as an effective team in order to accomplish the duties detailed in the standard. 2. The manager does not regularly attend meetings, but sends staff instead. 1. Neither the manager nor the staff provide the support needed to the organizing body and/or do not regularly attend the meetings. GA-5. The program monitors and evaluates quality of services regularly and routinely. GA-5.A. The program analyzes program goals regularly and routinely. R A R A TING INDICATORS EVIDENCE GA-5.A. through GA-5.E. Pre-site: Please submit a copy of the programs quality assurance plan and copies of all data tracking forms which support this plan. On-site: Interview staff, supervisors, program manager and any existing internal quality assurance committee re: monitoring and evaluation mechanisms. GA-5.A. 3. The program conducts regular and routine analysis of program goals. 2. The program conducts analysis of program goals, however, not as regular and routinely as possible. 1. The program does not conduct analysis of program goals. GA-5.B. The program reviews of participant grievances regularly and routinely. RATING INDICATORS GA-5.B.3. The program analyzes participant grievances regularly and routinely. 2. The program analyzes participant grievances, however, not as regularly and/or routinely as possible. 1. The program does not conduct analysis of participant grievances. GA-5.C. The program reviews of participant satisfaction with services regularly and routinely. RATING INDICATORS

PAGE 136

Appendix B Continued Fidelity and Innovations 130 GA-5.C.3. The program reviews participant satisfaction with services regularly and routinely. 2. The program reviews participant satisfaction with services, however, not as regularly and/or routinely as possible. 1. The program does not conduct analysis of participant satisfaction. GA-5.D. The program has a follow-up mechanism to address potential problems identified during quality assurance review. RATING INDICATORS GA-5.D. 3. The program has and uses (if necessary) its follow up mechanism to address potential problems identified during quality assurance review. 2. The program has and uses (if necessary) its follow up mechanism to address potential problems identified during quality assurance review. Isolated instances found when problems identified during quality assurance review were not addressed. 1. The program either does not have and/or does not routinely use its follow up mechanism to address potential problems identified during quality assurance review. GA-5.E.The program has a written standard of expectation regarding participants' services, quality, and outcomes. RATING INDICATORS GA-5.E.3. The program has written standard of expectations regarding all of the following: participants' services, quality, and outcomes. 2. The program has written standard of expectations regarding one or more of the following: participants services, quality, and outcomes. 1. The program does not have written standard of expectations regarding participants services, quality, and outcomes. GA-6. The program assures participant privacy and voluntary choice with regard to research conducted by or in cooperation with the program. EVIDENCE GA-6. Pre-site: Please submit a copy of the programs policy (or guidelines) regarding protecting participant privacy and voluntary choice and copies of any relevant forms used to document this. On-site: Interview staff, program researcher, and participants in the research. RATING INDICATORS

PAGE 137

Appendix B Continued Fidelity and Innovations 131 GA-6. 3. The program has procedures that it follows to carefully protect client identity and privacy throughout any research project conducted by or with the cooperation of the agency, as well as those that assure voluntary informed consent without pressure to participate. 2. The programs procedures are very general and do not address all contingencies although past and current practice show no evidence of violation of client privacy or voluntary participation. 1. Any of the following: the agency has general procedures but is lax in their enforcement; the agency has no procedures, individual researchers follow their own plans, and potential for disclosure of identity or violation of privacy is high; or clients are not provided an opportunity to refuse disclosure. GA-7. The program has a policy and procedure for reviewing and recommending approval or denial of research proposals, whether internal or external, which involve past or present participants. R R ATING INDICATORS GA-7. 3. The program (or the oversight body) has a formal mechanism, such as a written procedure for review and action on any research proposal involving past or present participants or participant information. There are no exceptions. 2. There is evidence of the formal mechanism being used in all cases, but a mechanism to prevent a possible, inadvertent delay or omission is not in place. 1. The program (or oversight body) responds to proposal requests on a caseby-case basis with few established guidelines or procedures to guide the process and insure that activities are consistent with or at least not in conflict with the agencys mission and mandates and that participants rights are protected. Or, there is no formal program review of research proposals. GA-8. Program reports suspected cases of child abuse and neglect. GA-8.A. Program has clear criteria through which to identify suspected cases of child abuse and neglect. EVIDENCE GA-7. Pre-site: Please submit a copy of the programs policy and procedures regarding approval/denial of research proposals. On-site: Interview staff and others who have participated in this procedure. EVIDENCE GA-8.A. Pre-site: Please submit a copy of the programs criteria used to identify suspected cases of child abuse and neglect. On-site: Interview staff and supervisors. RATING INDICATORS

PAGE 138

Appendix B Continued Fidelity and Innovations 132 -8.A.3. No for rating indicator GA-8.A. 2. Program has clear criteria to identify which cases of suspected child abuse/neglect should be reported and follows these criteria routinely. 1. Program does not have criteria to identify and report cases of suspected child abuse/neglect or, program does not routinely use criteria. GA-8.B.Programs reporting procedure specifies notification of program manager and his/her supervisor. Other appropriate staff/supervisors within the program are notified as needed. RATING RATING INDICATORS GA-8.B.3. Programs reporting procedure specifies notification of program manager and his/her supervisor. Other appropriate staff/supervisors within the program are notified as needed. Evidence indicates procedures are followed (if applicable). 2. Programs reporting procedure specifies notification of program manager. Other appropriate staff/supervisors within the program are notified as needed. Evidence indicates procedures are followed (if applicable). 1. Programs reporting procedure does not specify notification of program manager and his/her supervisor. Insufficient evidence to indicate procedures are followed (if applicable). GA-8.C.Programs reporting procedure is in compliance with all applicable laws and regulations regarding reporting of suspected cases of child abuse and neglect. R RATING INDICATORS EVIDENCE GA-8.B. Pre-site: Please submit a copy of the programs policy and procedures for reporting suspected cases of child abuse and neglect. On-site: Interview program manager and his/her supervisor. EVIDENCE GA-8.C. Pre-site: Please indicate whether the programs policies and procedures are in compliance with all applicable laws. NOTE: an attorneys opinion, in writing, is sufficient evidence. On-site: Interview program manager and his/her supervisor. GA-8.C.3. No "3" rating indicator for GA-5.C. 2. Programs reporting procedure is in compliance with all applicable laws and regulations regarding reporting of suspected cases of child abuse and neglect.

PAGE 139

Appendix B Continued Fidelity and Innovations 133 1. Programs reporting procedure is not in compliance with all applicable laws and regulations regarding reporting of suspected cases of child abuse and neglect. GA-9. Program has an internal reporting procedure for reporting participant (especially child) deaths that occur while the participant is in the program. GA-9.A.Program has a procedure that specifies notification of program manager and his/her supervisor in cases of participant deaths. Other appropriate staff/supervisors within the program are notified as needed. R ATRATING INDICATORS EVIDENCE GA-9.A. and GA-9.B. Pre-site: Please submit a copy of the programs policy and procedures for reporting deaths of participants. On-site: Interview program staff, manager and his/her supervisor. GA-9.A. 3. No "3" rating indicator for GA-6.A. 2. Programs reporting procedure specifies notification of program manager and his/her supervisor. Other appropriate staff/supervisors within the program are notified as needed. Evidence indicates procedures are followed (when applicable). 1. Programs reporting procedure does not specify notification of program manager and his/her supervisor. Evidence indicates procedures are not followed (when applicable). GA-9.B.Procedure ensures that staff receive crisis/grief counseling, as needed. RATING INDICATORS GA-9.B. 3. No "3" rating indicator for GA-6.B. 2. Procedure ensures that staff receive crisis/grief counseling, as needed. 1. Procedure does not ensure that staff receive crisis/grief counseling, as needed. GA-10. The program has a written budget and monitors expenditures to manage financial resources and support program activities for the program. RATING INDICATORS GA-10. 3. The program has a detailed written budget and it is used to monitor and manage expenditures for program activities during the year. 2. A written budget is used, but the monitoring and management of fiscal resources could be clearer.

PAGE 140

Appendix B Continued Fidelity and Innovations 1. Any of the following: program written budget is weakened by the lack of detail or clarity; there is no written budget; the budget is not monitored in order to manage fiscal resources for program activities during the year. GA-11. The budget is reviewed and approved by a group (other than program manager) prior to the beginning of the fiscal year. RATING INDICATORS GA-11.3. The organizing body, or the appropriate authority vested with that responsibility, reviews and approves the budget prior to the beginning of the fiscal year. 2. The organizing body (or the appropriate authority vested with the responsibility) reviews the budget before the beginning of the fiscal year, but approval has been delayed. 1. Any of the following: the organizing body approves the budget after the fiscal year has commenced; there is evidence that review has occurred, but no approval is documented. GA-12. The program seeks diversification and balance in its sources of funding. EVIDENCE GA-11. Pre-site: Please submit a narrative describing the approval mechanism for program budget and a list of individuals/groups who participate in the review. On-site: Interview those indicated as participating. EVIDENCE GA-12. Pre-site: Describe programs fund development efforts and its procedures for presenting need to bodies that provide its funding. On-site: Interview program manager and/or development staff. RATING INDICATORS GA-12.3. The program seeks diversification (e.g., fee for service arrangements, third party reimbursement, use of federated or other funding sources as appropriate to program structure and mission) and balance in its sources of funding to avoid excessive dependence on any one major funding source which, if terminated, could result in abrupt cessation of service to clients. 2. The program, despite active efforts to diversify or strengthen resources, is dependent on one major source or is under-funded because of matters beyond its control (as in legislative constraints). 1. Any of the following: program has made minimal efforts to expand; diversity or strengthen its resource base; program and services suffer because of abrupt termination or withdrawal of funding; program has no "fall back" position and has made no effort to protect itself from the consequences of excessive dependence on one source of funding. GA-13. The program (or programs sponsoring agency) makes available to the community an annual report or fiscal, statistical, and service data regarding the program. 134

PAGE 141

Appendix B Continued Fidelity and Innovations RATING INDICATORS GA-13. 3. No "3" rating indicator for GA-13. 2. The program or the programs sponsoring agency produces an annual report and/or fiscal, statistical, and service data regarding the program on an annual basis. And that report is made available to the community. 1. The program or the programs sponsoring agency does not make available an annual report and/or fiscal, statistical, and service data regarding the program. And/or the report is not made available to the community. GA-14. The program (or the programs sponsoring agency) is audited annually by an independent certified public accountant approved by the governing body. EVIDENCE GA-13 Pre-site: Please submit a copy of the programs most recent annual report or review the data the program makes available to the community about its fiscal, statistical, and service information. Please describe how the program disseminates this information to the community. On-site: Check availability of report(s) to community through interviewing organizing group members, participants, staff and supervisors. EVIDENCE GA-14. Pre-site: Please submit a copy of the programs (or host agencys) most recent audit. On-site: Interview fiscal staff, program manager, and his/her supervisor. RATING INDICATORS GA-14. 3. No "3" rating indicator for GA-14. 2. The program (or the programs sponsoring agency) is audited annually by an independent certified public accountant approved by the governing body. 1. The program (or the programs sponsoring agency) is not audited annually by an independent certified public accountant approved by the governing body. End of Assessment Tool 135

PAGE 142

Fidelity and Innovations Appendix C HFA Credentialing Program Self-Assessment Tool: An Initiative of Prevent Child Abuse America (2003) An Overview of the Healthy Families America Credentialing Program The development of this credentialing system was initiated as a result of requests from programs, trainers, and state leaders for a quality assurance process to preserve the standards of excellence of the Healthy Families America (HFA) initiative. The system was developed with the assistance of numerous individuals from the field program managers, trainers, researchers, and state advocates/leaders together with the expertise of the Council on Accreditation of Services for Families and Children, Inc. Taken together, this collaboration ensures that the new quality assurance system a) reflects best practice as established by both research and practice and, b) has a process which reflects best practice in the quality assurance community. The purpose of HFA credentialing is to help each home visiting program monitor and maintain quality over the long term, as well as put into place a mechanism to ensure the quality of the HFA initiative. The goal is to recognize all credentialed sites as quality programs. The pursuit of quality and excellence involves partnerships. The program being credentialed enters into a unique partnership with HFA in its efforts to obtain public recognition as a credentialed program. These organizations share a commitment to enhancing the quality of home visitation programs and the assumption that the best program can continue to learn and grow. In effect, all parties the applicant program and HFA share the job of examining the programs structure and functioning. The HFA Credentialing Site Self-Assessment Tool, the centerpiece of HFA credentialing, is based upon the critical elements, a set of best practice standards reflecting over twenty years of research into the best outcomes in home visitation programs. When completed, the self-assessment will provide each program with an analysis of how well it is implementing the critical elements. This forms the basis for the peer review and all subsequent decisions about the programs application for a credential. The process is designed as an objective, external review, which validates for the interested public that the program is following best practice standards. The Credentialing Manual has suggestions about how to structure the work involved, the credentialing process itself, and the design and scoring of the tool. Please refer to the manual before you begin your self-assessment or call Prevent Child Abuse America at 312. 663.3520. 136

PAGE 143

Appendix C Continued Fidelity and Innovations HEALTHY FAMILIES AMERICA PROGRAM FACT SHEET Applicant Program Name Name of Program Director Name of Credentialing Contact (if different) Program Address City State Zip Telephone ( ) Fax ( ) E-mail Name of Host Agency/Organization (if applicable) Name of Chief Executive Officer (if applicable) Address City State Zip Telephone ( ) Fax ( ) STRUCTURE OF THE HFA PROGRAM Briefly describe the following aspects of the structure of your HFA program, including:: Legal status of host agency, i.e., 501(c) 3, public agency, etc.; Organizational chart of host agency (be sure the applicant program is included in this chart); and The structure of the applicant program. Be sure your answer addresses the following points: governance of program; organizations jointly providing program services and nature of the relationship with the applicant program (e.g., job contractor, in-kind services, etc.); list of all program staff and volunteers, his/her title, and a brief description of position function. 1. I nitiate services p renatall y or at birth. 137

PAGE 144

Appendix C Continued Fidelity and Innovations 1-1. Program ensures it identifies participants in the target population for services either while mother is pregnant (prenatally) and/or at the birth of baby. 1-1.A. The program has a description of the target population that includes key demographic information such as number of resident live births per year, number of women of child-bearing age, number of single parents, age of the target population, and race/ethnicity/linguistic/cultural characteristics of population and places where the population is found (e.g., local hospitals, prenatal clinics, high schools, etc.). 1 -1-1.A. RATING INDICATORS 3 The program has a description of the target population and identifies organizations within the community in which the target population can be found. Both the description and identification are comprehensive and up-to-date. 2 The program has a description of the target population and identifies organizations within the community in which the target population can be found which, while sufficient for its needs, could be more comprehensive. 1 Any of the following: program does not have a description of the target population; program does not identify organizations within the community in which the target population can be found; and/or the description and/or identification have major information gaps. 1-1.B. The programs system of formal organizational agreements with community entities (e.g. prenatal clinics, hospitals, etc.) identifies the participants in the target population to determine their need for service. 1 1 .B. RATING INDICATORS EVIDENCE 1-1.A. Pre-site: Please submit a description of the programs target population (including demographic characteristics described in standard 1-1.A.). On-site: Interview staff assigned to maintain information on target population. EVIDENCE 1-1.B. Pre-site: Please submit a description of the collaboration(s) the program has with community entities in order to provide entry to the target population. Include copies of any relevant documentation, such as memoranda of agreements, etc. On-site: Interview staff and supervisors assigned to develop the collaborations and interview at least one partner in the collaboration. 3 The system of organizational agreements enables the program to identify 85% or more of the participants in the target population for screening or assessment. (If a screening process is not used, this standard refers to assessment). 138

PAGE 145

Appendix C Continued Fidelity and Innovations 2 The system of organizational agreements enables the program to identify at least 75% of the participants in the target population for screening or assessment. 1 The system of organizational agreements does not ensure identification of 75% or more of the participants in the target population for screening or assessment. 1-1.C. The programs system of formal and/or informal services in coordination with other entities ensures potential participants are identified and referred to the program in a timely manner (i.e., giving the program the necessary time to locate the participant and complete an assessment within two weeks of the birth of the baby). R 1 1-1.C. RATING INDICATORS EVIDENCE 1-1.C. Pre-site: Please submit a description of the mechanism(s) through which the program tracks information on pregnancies/births with the collaborating partners (may be informal, e.g., phone calls, etc., or formal, memoranda of agreement, etc.). On-site: Review all documentation that shows evidence of the programs system of service coordination (e.g., tickler system, etc.). Interview staff and supervisors assigned to maintain information. 3 All entities in the system identify and refer potential participants to program in a timely manner. 2 Most, but not all, entities in the system identify and refer potential participants to program in a timely manner. Program is aware of and is addressing this issue. 1 The program either has no system or the system does not enable the identification of potential participants and referral to program in a timely manner. 1-1.D. Screenings/Assessments to determine eligibility for services occur either prenatally or within the first two weeks after the birth of the baby. (Please note: Figures related to screenings are acceptable only in instances where screenings determine eligibility for services). 139

PAGE 146

Appendix C Continued Fidelity and Innovations 1-1D. RATING INDICATORS EVIDENCE 1-1.D. Pre-site: Screening/Assessment information as completed below for the past twelve months, or program may submit its own screening/assessment information. Number of Screenings/ Percentage Assessments Conducted Prenatally Birth to two weeks after birth 3 Ninety-five percent (95%) through one hundred percent (100%) of eligibility screenings or assessments occur either prenatally or within the first two weeks after the birth of the baby. 2 Eighty percent (80%) through ninety-four percent (94%) of all eligibility screenings or assessments occur either prenatally or within the first two weeks after the birth of the baby. 1 Less than eighty percent (80%) of all eligibility screenings or assessments occur either prenatally or within the first two weeks after the birth of the baby. 1-2. The program defines, measures, and monitors the acceptance rate of participants into the program in a consistent manner and on a regular basis. 1-2.A. The program defines, measures and monitors the acceptance rate of participants into the program. 1-2.A. RATING INDICATORS More than two weeks after birth + + (Total # of screens/assessments) = = 100% of all screens/assessments On-site: Review screening/assessment files, data system (if applicable), interview assessment worker and supervisor. EVIDENCE 1-2.A. Pre-site: Please submit a copy of your programs definition of acceptance and method for calculating acceptance rates and a summary of the programs acceptance rates for the most recent year. On-site: Interview assessment worker and supervisor. 3 The program defines, measures and monitors the acceptance 140

PAGE 147

Appendix C Continued Fidelity and Innovations rate of participants into the program and evidence indicates acceptance rates are being measured more than once a year. 2 The program defines, measures and monitors its acceptance rate and evidence indicates acceptance rates are measured in a consistent manner and at least yearly. 1 The program does not define, measure and/or monitor its acceptance rate and/or is not measuring its acceptance rate at least yearly. 1-2.B. The program analyzes at least annually (i.e., both formally, through data collection, and informally through discussions with staff and others involved in assessment process) who refused the program among those determined to be eligible for services and the reasons why. 1-2.B. RATING INDICATORS 3 The program semi-annually uses both formal and informal methods to analyze who refused the program and why. This analysis addresses programmatic, demographic, social and other factors. 2 The program annually analyzes who refused the program and why. However, analysis relies only on demographic and informal sources to identify those who refused. 1 Any one of the following: The program does not have an analysis of who refused services and why, has only an informal analysis, relies solely on demographic sources or a programmatic analysis, or the analysis is not conducted annually. NA The program did not accept any new participants last year or all individuals who were offered the program accepted. 1-2.C. The program addresses how it might increase its acceptance rate based on its analysis of programmatic, demographic, social and other factors related to choosing not to participate in program after being found eligible. EVIDENCE 1-2.B. Pre-site: Please submit an analysis of which individuals are most likely to refuse the program after being found eligible. On-site: Interview assessment worker and supervisor EVIDENCE 1-2.C. Pre-site: Please submit a written plan on how the program has tried to increase its acceptance rate, based upon results of the analysis conducted in 1-2.B., among the individuals who are not currently choosing to participate in the program after being found eligible. On-site: Interview assessment worker and supervisor. 141

PAGE 148

Appendix C Continued Fidelity and Innovations 1-2.C. RATING INDICATORS 3 Based on the analysis, the program has implemented a plan for increasing its acceptance rate among the individuals who are not currently choosing to participate in the program. The plan addresses programmatic, demographic, social and other factors. 2 Based on the analysis, the program addresses how it might increase its acceptance rate and has a plan for doing so. However, the plan has not yet been implemented. 1 The program does not have a plan and/or does not address how it might increase its acceptance rate. NA The program did not accept any new participants in the past year or all individuals with a positive assessment accepted the program. 1-3. The program ensures that, for those who accept home visitor services, the first home visit occurs prenatally or within the first three months after the birth of the baby. 1 -3. RATING INDICATORS EVIDENCE 1-3. Pre-site: Information as completed below for the past twelve months: First Home Visit Occurs: Number of first home visits made: Percent Prenatally Within first three months of birth of baby After first three months of birth of baby + Total first home visits = = 100% Onsite: Review participant records databaseinformation(ifapp licable) 3 Ninety-five percent (95%) through one hundred percent (100%) of first home visits occur within the first three months after the birth of the baby. 2 Eighty percent (80%) through ninety-four percent (94%) of first home visits occur within the first three months after the birth of the baby. 1 Less than eighty percent (80%) of first home visits occur within the first three months after the birth of the baby. 142

PAGE 149

Appendix C Continued Fidelity and Innovations 2-1. The program uses a tool(s) (e.g., screening tools, assessment tools, etc.) to identify the participants within the target population who are most in need of intensive home visitor services. 2-1.A. The program has screening and/or assessment tool(s) to identify participants most in need of intensive home visitor services. 2-1.A. RATING INDICATORS 3 No 3 rating for standard 2-1.A. 2 The program has screening and/or assessment tool(s). 1 The program does not have screening and/or assessment tool(s). 2-1.B. The tool(s) assesses for the presence of factors including increased risk for child maltreatment or other poor childhood outcomes (e.g., social isolation, substance abuse, parental history of abuse in childhood, etc.). 2-1.B. RATING INDICATORS EVIDENCE 2-1.A. Pre-site: Please submit a copy of the screening and/or assessment tools used to identify participants most in need of intensive home visitor services (as defined by the program). On-site: Interview assessment worker and supervisor. NOTE: A programs tool(s) may look either at individuals or the community as a whole. EVIDENCE 2-1.B. Pre-site: Please indicate which factors (see standard above) are measured in each tool used by the program. On-site: Review assessment records and interview assessment worker and supervisor. 2. Use a standardized (i.e., in a consistent way for all families) assessment tool to systematically identify families who are most in need of services. This tool should assess the presence of various factors associated with increased risk for child maltreatment or other poor childhood outcomes (i.e., social isolation, substance abuse, parental history of abuse in childhood). 3 No 3 rating indicator for standard 2-1.B. 143

PAGE 150

Appendix C Continued Fidelity and Innovations 2 Tool(s) assesses for the presence of factors including increased risk for child maltreatment or other poor childhood outcomes (e.g., social isolation, substance abuse, parental history of abuse in childhood, etc.). 1 Tool(s) does not assess for the presence of factors including increased risk for child maltreatment or other poor childhood outcome (e.g., social isolation, substance abuse, parental history of abuse in childhood, etc.). 2-1.C. The screening and/or assessment tools(s) are used uniformly with the target population. 2-1.C. RATING INDICATORS 3 Program screening and/or assessment tool(s) is used uniformly with the target population. 2 Isolated instances occur when tool(s) is not used uniformly with the target population. 1 The program does not demonstrate that it uses tool(s) with target population. 2-2. The program ensures that staff and volunteers who use the screening and/or assessment tool(s) have been trained in its use prior to allowing them to administer it. 2-2.A. The program has guidelines for training workers who will use the tool to ensure that the worker has adequate understanding and knowledge of how to use the tool appropriately. These guidelines require that the training include the theoretical background (i.e., its purpose, what it measures, etc.) on the tool(s) and hands-on practice in using the tool(s). 2 -2.A. RATING INDICATORS EVIDENCE 2-1.C. Pre-site: Please submit a description of how tool(s) are used to determine participants most in need of intensive home visitor services. On-site: Review assessment records, interview assessment worker and supervisor. EVIDENCE 2-2.A. Pre-site: Please submit a copy of the written training guidelines regarding the administration of the assessment tool(s). On-site: Interview assessment workers regarding training. 3 The program has written guidelines for training workers who will use the tool(s) that include both the theoretical background and hands-on practice. 2 The program has written guidelines for training workers who will use the tool(s), but the training does not include hands-on practice. 144

PAGE 151

Appendix C Continued Fidelity and Innovations 1 The program does not have written guidelines for training workers who will use the tool(s) or the training does not include the theoretical background on the tool. 2-2.B. The trainer is qualified, through educational background and completion of training in the use of the tool(s) to train others. 2-2.B. RATING INDICATORS 3 The trainer is qualified, through educational background (i.e., degree in human services or related field), experience in administering the tool, and completion of training in use of the tool(s), to train others. 2 The trainer is qualified, through experience in administering the tool and completion of training in use of the tool(s), to train others. 1 The trainer has not completed training in the use of the tool and/or never administered the tool. 2-2.C. Staff and volunteers who use the tool(s) have been trained in its/their use prior to administering it/them. 2-2.C. RATING INDICATORS EVIDENCE 2-2.B. Pre-site: Please indicate who can train staff in the use of the tool(s) and indicate his/her qualifications for training others to use the tool. On-site: Interview trainer(s), review relevant documentation regarding qualifications of trainer (e.g., certification of training in use ftldtildt) EVIDENCE 2-2.C. Pre-site: Please submit a list of the written guidelines, staff/volunteers who use the tool, the date when he/she first used the tool, and the date of training. On-site: Review documentation of training for all those using the assessment tool (e.g., review personnel/volunteer files, training log, etc.). Review assessment files. 3 Program guidelines require all who use the tool(s) to be trained prior to administering it/them and all staff and volunteers who use the tool(s) to have been trained in its/their use prior to administering it/them. 2 Program guidelines require all who use the tool(s) to be trained prior to administering it/them, however isolated instances found when staff were not trained prior to administering the tool. 1 Program guidelines do not require all who use the tool(s) to be trained prior to administering it/them, the program does not train staff/volunteers in use of the tool(s), or staff and volunteers routinely use the tool(s) before being trained in its/their use. 145

PAGE 152

Appendix C Continued Fidelity and Innovations 2-3. The program uses criteria to identify participants in need of service and documents this in its files. 2-3.A. Criteria indicate the constellation of factors necessary for an individual to demonstrate need for service. 2-3.A. RATING INDICATORS 3 No "3" rating indicator for standard 2-3.A. 2 The criteria indicate the constellation of factors necessary for an individual to demonstrate need for services. 1 The criteria do not indicate the constellation of factors necessary for an individual to demonstrate need for services. 2-3.B. The program assures that the criteria are clearly and uniformly summarized in writing and documented in individual participant files. EVIDENCE 2-3.A. Pre-site: Please submit a copy of the programs written criteria (e.g., scoring for the screening/assessment tool(s)). EVIDENCE 2-3.B. Pre-site: Please submit a copy of the programs written criteria and any paperwork and/or forms used to document assessments. On-site: Review criteria, assessment summaries, and/or narratives in screening/assessment files, and interview assessment worker(s) and supervisor(s). 2-3.B. RATING INDICATORS 3 The program has written guidelines for criteria, assessment summaries, and/or narratives that are both clear and comprehensive, and uses the guidelines consistently in completion of assessment summaries/narratives. 2 The program has written guidelines for criteria, assessment summaries, and/or narratives. Isolated instances when guidelines were not used in completion of assessment summaries/narratives. 1 The program does not have written guidelines for criteria, assessment summaries, and/or narratives and/or the written guidelines are not routinely followed in completion of assessment summaries/narratives. 2-3.C. Criteria are applied uniformly. 2-3.C. RATING INDICATORS 3 Criteria are consistently applied. 146

PAGE 153

Appendix C Continued Fidelity and Innovations 2 Isolated instances occur when criteria are not consistently applied. 1 Criteria are not consistently applied. 3-1. Services are offered to families on a voluntary basis. 3-1. 3-2. 3-3. RATING INDICATORS 3 No 3 rating indicator for standard 3-1. 2 Services are offered to families solely on a voluntary basis. 1 There are instances in which families are mandated to receive services at program entry. 3-2. The staff uses positive outreach methods to build family trust, engage new families, and maintain family involvement in program. 3-2.A. The program has guidelines that specify a variety of positive outreach methods. EVIDENCE 3-1. Pre-site: Please submit copy of program policy regarding voluntary nature of service. On-site: Review participant files for agreement signed by participants indicating voluntary nature of services. EVIDENCE 3-2.A. Pre-site: Please submit a copy of the written guidelines that address engaging families and building trust. 3. Offer services voluntarily and use positive, persistent outreach efforts to build fil 3-2.A. RATING INDICATORS 3 The program has clearly written, comprehensive guidelines that specify a variety of positive outreach methods (e.g., telephone calls, visits, mailings, parenting groups, etc.). 2 The program has written guidelines that are sufficient for its needs, but could be more clear or comprehensive. 1 Either the program has no written guidelines or the guidelines are insufficient for its needs. 3-2.B. The staff uses the guidelines in order to build family trust, engage them in services and maintain family involvement. 147

PAGE 154

Appendix C Continued Fidelity and Innovations 3-2.B. RATING INDICATORS 3 Evidence consistently points to routine use of these guidelines. 2 Isolated instances found when guidelines not followed. 1 Guidelines routinely not followed. 3-3. The program offers outreach under specified circumstances for a minimum of three months for each participant before discontinuing services. 3-3.A. The program guidelines specify the circumstances under which a participant is placed in outreach status. 3-3.A. RATING INDICATORS 3 No 3 rating indicator for standard 3-3.A. 2 The written guidelines specify the types of circumstances under which a participant is provided outreach. Guidelines are clear and easy to understand. 1 One of the following: There are no written guidelines; the program does not specify the types of circumstances under which a participant is provided outreach; the guidelines are lacking clarity and/or are difficult to understand. 3-3.B. The program guidelines specify that outreach is continued for participants for three months and that outreach is only concluded prior to three months when participants have been engaged, re-engaged in services, refused services or have moved from the area. EVIDENCE 3-2.B. Pre-site: Please submit the programs written outreach guidelines, as submitted in 3-2.A. On-site: Review participant files of families in outreach; interview home visitor and supervisors. EVIDENCE 3-3.A. Pre-site: Please submit a copy of the programs written guidelines that address the types of circumstances under which a participant is provided outreach. EVIDENCE 3-3.B. Pre-site: Please submit a copy of the programs written guidelines that address the amount of time outreach is continued for a participant. 3-3.B. RATING INDICATORS 148

PAGE 155

Appendix C Continued Fidelity and Innovations 3 No "3" rating indicator for standard 3-3.B. 2 The written guidelines specify that outreach is continued for participants for three months and that outreach is only concluded prior to three months when participants have (re)engaged in services, refused services or the participant has moved from the area. 1 The program either has no written guidelines or the guidelines do not address all points identified in 3-3.B. 3-3.C. The program places participants in outreach appropriately and continues outreach for three months, only concluding outreach prior to three months when the participants have (re)engaged in services, refused services or moved from the area. EVIDENCE 3-3.C. On-site: Review files of participants in outreach, interview home visitors and supervisors. 3-3.C. RATING INDICATORS 3 The program routinely places participants in outreach appropriately (i.e., as specified by its written guidelines) and routinely continues outreach for three months. The only instances found when outreach was concluded prior to three months occurred when the participants (re)engaged in services, refused services or moved from the area. 2 The program follows the written guidelines, as indicated in "3" rating indicator. However, isolated instances were found when the written guidelines were not followed. 1 The program does not routinely follow its written guidelines as specified in "3" rating indicator. 3-4. The program defines, measures and monitors its retention rate of participants in the program in a consistent manner and on a regular basis. 3-4.A. The program defines, measures and monitors its retention rate. The definition of its retention rates includes all participants who received outreach and home visitation from the program 3-4.A. RATING INDICATORS 3 The program defines, measures and monitors the retention rate of participants in the program and evidence indicates retention rates are being measured more than once a year. 2 The program defines, measures and monitors its retention rate, and evidence indicates retention rates are measured in a consistent manner and at least yearly. 1 The program does not define, measure and/or monitor its retention rate and/or is not measuring it at least yearly. 3-4.B. The program analyzes at least annually (i.e., both formally through data collection and informally, through discussions with staff and others involved in program services) which individuals dropped out of the program, at what point in services, and reasons why. 149

PAGE 156

Appendix C Continued Fidelity and Innovations 3 -4.B. RATING INDICATORS 3 The program semi-annually uses both formal and informal methods to analyze who leaves the program and why. This analysis addresses programmatic, demographic, social and other factors. 2 Program annually analyzes who drops out of the program and why. However, analysis relies on demographic and informal sources to identify those who dropped out. 1 The program does not have an analysis of who dropped out of the program and why, has only an informal analysis, does not include a programmatic analysis, relies solely on demographic sources or programmatic analysis, or the analysis is not conducted annually. NA No participants have dropped out of the program in the past year. 3-4.C. The program addresses how it might increase its retention rate based on its analysis of programmatic, demographic, social and other factors related to dropping out of the program after receiving services. EVIDENCE 3-4.B. Pre-site: Please submit an analysis of which individuals are most likely to drop out of program, at what point in service, and why. On-site: Interview home visitors and supervisor. EVIDENCE 3-4.C. Pre-site: Please submit a written plan on how the program has tried to increase its retention rate, based upon the analysis conducted in 3-4.B., among individuals who are dropping out of the program. On-site: Interview assessment worker and supervisor. 3-4.C. RATING INDICATORS 3 Based on the analysis, the program has implemented a plan for increasing its retention rate among the individuals who are currently dropping out of the program. The plan addresses programmatic, demographic, social and other factors. 2 Based on the analysis, the program addresses how it might increase its retention rate and has a plan for doing so. However, the plan has not yet been implemented. 1 The program does not have a plan and/or does not address how it might increase its retention rate. NA No participants dropped out of the program in the past year. 150

PAGE 157

Appendix C Continued Fidelity and Innovations 4-1. The program has a well-thought-out system for managing the intensity of home visitor services. 4-1.A. The levels of service (i.e., weekly visits, bi-weekly visits, monthly visits, etc.) offered by the program are clearly defined. 4-1.A. RATING INDICATORS 3 Levels of service are clearly defined and home visitors understand the levels of service offered. 2 Levels of service are clearly defined; however there are isolated instances found when home visitor did not understand the levels of service offered by the program. 1 Levels of service are not clearly defined and/or, home visitors routinely do not understand the levels of service offered by the program. 4-1.B. Participants at the various levels of service (i.e., weekly visits, bi-weekly visits, monthly visits, etc.) offered by the program receive the appropriate number of home visits, based upon the level of service to which they are assigned. EVIDENCE 4-1.A. Pre-site: Please submit a description of the various levels of service offered by the program. On-site: Interview with home visitor and supervisor. EVIDENCE 4-1.B. Pre-site: Please submit a summary of all current program participants by home visitor, their start date, the target childs date of birth, their level of service, and separately list a record of the visits conducted in the home (completed only) and the number of visits conducted elsewhere over the past quarter (e.g., home visitor logs). On-site: Interview with home visitor and supervisor, review participant files. 4. Offer services intensely (i.e., at least once a week) with well-defined criteria for increasing or decreasing intensity of service and over the long term (i.e., three to five years). 4-1.B. RATING INDICATORS 3 Participants receive at least ninety percent (90%) of the appropriate number of home visits based upon the individual level of service to which they are assigned, with at least eighty percent (80%) of the visits occurring in the home. 2 Participants receive at least seventy-five (75%) percent of the appropriate number of home visits based upon the individual level of service to which they are assigned, with at least eighty percent (80%) of the completed visits occurring in the home. 151

PAGE 158

Appendix C Continued Fidelity and Innovations 1 Participants receive less than seventy-five (75%) percent of the appropriate number of home visits based upon the individual level of service to which they are assigned or receive less than eighty percent (80%) of the completed visits in the home. 4-1.C. The program analyzes and addresses how it might increase its home visitation completion rate. (Please note: This standard applies regardless of whether the 75% threshold identified above in standard 4-1.B. is being met.) 4 -1.C. RATING INDICATORS 3 Based on the analysis, the program has implemented a plan for increasing its home visitation completion rate. 2 Based on the analysis, the program has a plan that addresses how it might increase its home visitation completion rate, but has not yet been implemented. 1 The program either has not conducted an analysis to address how it might increase its home visitation rate or does not have a plan. NA Based upon the level of service to which they are assigned (i.e., weekly visits, biweekly visits, monthly visits, etc.), participants receive the appropriate number of visits. 4-1.D. The criteria for increasing/decreasing the intensity of the service are clearly defined and linked to the levels of service offered by the program. EVIDENCE 4-1.D. Pre-site: Please submit a copy of the programs criteria for increasing/decreasing intensity of service. On-site: Interview home visitor and supervisor. EVIDENCE 4-1.C. Pre-site: Please submit a copy of the programs analysis and plans for increasing the home visitation completion rate. Ons ite: Interview p ro g ram mana g er and/o r su p ervisor. 4-1.D. RATING INDICATORS 3 Criteria for moving participants from one level of service to another are clearly defined and linked to the levels of service. Home visitors clearly understand the criteria and the linkage with the levels of service. 2 Criteria for moving participants from one level of service to another are clearly defined and linked to the levels of service. Isolated instances found when home visitors did not understand the criteria and the linkage with the levels of service. 1 Any of the following: criteria for moving participants from one level of service to another are not clearly defined; criteria are not linked to levels of service; home visitors routinely did not understand the criteria and the linkage with the levels of service. 4-1.E. Each participants progress is regularly reviewed by the family, home visitor, and supervisor. (Please note: All parties do not have to be present at the same time to conduct this review). 152

PAGE 159

Appendix C Continued Fidelity and Innovations 4-1.E. RATING INDICATORS 3 The program regularly reviews progress made by participants, and involves, at a minimum, the home visitor, the participant, and the supervisor in the review. 2 Isolated instances found when either review was not conducted regularly or appropriate individuals were not involved with review. 1 Reviews were not conducted regularly or were not according to program policy or appropriate individuals were consistently not involved with review. 4-1.F. The progress of the participant is the basis for the decision to move the participant from one level of service to another. 4-1.F. RATING INDICATORS 3 The progress of the participant routinely serves as the basis for the decision to move participants from one level of service to another. 2 Isolated instances when participants moved from one level of service to another in absence of a review of participant progress. 1 Participants routinely moved from one level of service to another in absence of a review of participant progress. 4-2. The program offers home visitation services intensively after the birth of the baby. 4-2.A. Policy states that participants receiving intensive home visitation services are offered weekly home visits for a minimum of six months after the birth of the baby. 4-2.A. RATING INDICATORS EVIDENCE 4-1.F. Pre-site: Programs written policy on reviewing participant progress and service intensity, as submitted in 4-1.E. On-site: Review participant files and any other documentation that addresses participant movement from one level of service to another, interview home visitor and supervisor. EVIDENCE 4-2.A. Pre-site: Please submit a copy of programs written policy on the minimum amount of time it offers weekly visits for participants receiving intensive home visitation services. 3 No "3" rating indicator for standard 4-2.A. 2 The programs written policy states that the minimum length of time for weekly home visits is at least six months after the birth of the baby. 1 The programs written policy states that the minimum length of time for weekly home visits is less than six months or minimum length of time for weekly visits is not indicated in program policy. 153

PAGE 160

Appendix C Continued Fidelity and Innovations 4-2.B. Program ensures that participants remain on the most intensive home visitation level (at least weekly) for a minimum of six months after the birth of the baby. 4-2.B. RATING INDICATORS 3 Participants remain on the most intensive home visitation level for a minimum of six months after the birth of the baby. 2 Isolated instances were found where participants did not remain on the most intensive home visitation level for a minimum of six months after the birth of the baby. 1 Participants consistently do not remain on the most intensive home visitation level for a minimum of six months after the birth of the baby. 4-3. The program offers home visitation services to participants for a minimum of three years after the birth of the baby. 4-3.A. The program policy states that it will offer home visitation services to participants for a minimum of three years after the birth of the baby. EVIDENCE 4-2.B. On-site: Review participant files and/or other appropriate sources of documentation regarding intensity and duration of service offered to participants, interview home visitors, supervisors, and participants. EVIDENCE 4-3.A. Pre-site: Please submit a copy of the programs written policy regarding length of time program it will continue to offer home visitation services to a participating participant. 4-3.A. RATING INDICATORS 3 The program policy specifies that home visitation services are offered for more than three years after the birth of the baby. 2 The program policy specifies that home visitation services are offered for three years after the birth of the baby. 1 The program policy specifies that home visitation services are offered for less than three years after the birth of the baby. 4-3.B. The program ensures that it offers home visitation services to participants for a minimum of three years after the birth of the baby (for those participants who wish to continue participating). 154

PAGE 161

Appendix C Continued Fidelity and Innovations 4-3.B. RATING INDICATORS 3 The program ensures that home visitation services are offered for more than three years after the birth of the baby to participants who wish to continue with the program. 2 The program ensures that home visitation services are offered for at least three years after the birth of the baby to participants who wish to continue with the program. 1 The program, while it has been operational for at least three years, has not demonstrated that it continues to provide services for a minimum of three years to those participants who wish to continue. NA Program has not been in operation for at least three years. 5-1. The program has a description of the cultural, racial/ethnic, and linguistic characteristics of all groups within the current service population. 5-1. RATING INDICATORS EVIDENCE 5-1. Pre-site: Please submit a description of the programs service population as indicated in the standard above. On-site: Review relevant information to ascertain accuracy of service population demographic reports, interview program manager, and/or staff assigned to maintain this information. EVIDENCE 4-3.B. Pre-site: Please submit a report showing the participants in your program who have been with the program for at least three years. On-site: Interview home visitor and supervisor. 3 No 3 rating indicator for standard 5-1. 2 Description is clear and comprehensively addresses all of the following for service population: Cultural characteristics Racial/ethnic characteristics Linguistic characteristics 1 The program does not have a description or the description addresses all of the above stated items, but does not comprehensively address all factors stated above. 5-2. The program demonstrates culturally competent practices in all aspects of its service delivery. 155

PAGE 162

Appendix C Continued Fidelity and Innovations 5-2.A. The program has staff, volunteers, and/or agreements with other, appropriate community entities to provide culturally competent services to all group(s) within the service and target populations. 5 2 A R ATING INDICATORS 3 The program has the appropriate staff, volunteers, and community partners to meet the cultural and linguistic needs of all of the population groups within the service and target populations. 2 The program has the appropriate staff, volunteers, and community partners to meet the cultural and linguistic needs of most of the population groups within the service and target populations. Program has a plan to ensure that it has adequate staff, volunteers or other community partners to meet the needs of those population groups in the target population not currently being served. 1 The program does not have the appropriate staff, volunteers, and community partners to meet the cultural and linguistic needs of most of the population groups within the service and target populations. Or the programs staff, volunteers, and community partners can meet the cultural and linguistic needs of most of the population groups within the service and target populations, but has no plan to ensure it meets the needs of those not currently being served. 5-2.B. The programs materials are reflective of the diversity of the service and target populations. 5-2.B. RATING INDICATORS EVIDENCE 5-2.A. Pre-site: Please attach a breakdown of the demographics of staff (e.g., culture, race/ethnicity, language, etc.) in the program, identify service gaps to be addressed for the program to become more culturally competent and describe the other ways in which the program can meet the needs of these groups (e.g., interagency agreements with appropriate agencies who can provide appropriate linguistic services, volunteer recruitment to help fill the gap(s), etc.). Also, attach description of the programs target population as submitted for standard 1-1.A. On-site: Interview program manager and direct service staff. EVIDENCE 5-2.B. Pre-site: Please submit copies of all relevant program materials for the service and target populations, (e.g., annual report, program brochure, flyers announcing program events, etc.). Also, describe how program ensures materials are participant-centered (i.e., language, photos, etc.). On-site: Interview program manager and review additional program materials that were too large to send with pre-site evidence (i.e., curricula, etc.). 3 Almost all of the programs materials (e.g., annual report, brochures, program specific materials such as curricula, etc.) are participant-centered (e.g., photos 156

PAGE 163

Appendix C Continued Fidelity and Innovations reflective of diversity of population, materials available in major languages spoken by target population, materials reflect literacy level of participants, etc.). 2 A majority of the programs materials (e.g., annual report, brochures, program specific materials such as curricula, etc.) are participant-centered (e.g., photos reflective of diversity of population, materials available in major languages spoken by population, materials reflect literacy level of participants, etc.). 1 Few of the programs materials are participant-centered or the programs materials do not reflect the diversity of one or more major groups in the target population. 5-2.C. Ethnic, cultural, and linguistic factors are taken into account in assigning workers to participants and in overseeing home visitor-participant interactions. (Note: It is not necessary that worker and participant possess the same cultural, racial/ethnic, and/or linguistic characteristics.) 5-2.C. RATING INDICATORS 3 Program routinely takes into account ethnic, cultural, and linguistic issues during assignment and ongoing oversight of home visitors by ensuring that the worker supports and respects the participants cultural, racial/ethnic, and linguistic characteristics. 2 Isolated instances have occurred when ethnic, cultural, and linguistic issues were not taken into account during assignment and ongoing oversight of home visitors. 1 Either the program does not routinely take into account ethnic, cultural, and linguistic issues during assignment and ongoing oversight of home visitors or it does not ensure the worker supports and respects the cultural, racial/ethnic, and linguistic characteristics of the participant. 5-3. The program provides staff training on culturally competent practices based on the unique characteristics of population(s) being served (i.e., age related factors, language, culture, etc.) by the program. EVIDENCE 5-2.C. Pre-site: Please submit a description of how the program ensures it takes into account the ethnic, cultural, and linguistic issues of participants and home visitors both during initial assignment of home visitor to participant and ongoing. On-site: Interview supervisors, home visitors, and participants. EVIDENCE 5-3. Pre-site: Please submit documentation regarding the training offered to staff on cultural diversity and the particular group(s) represented in the target population. Be sure to include a list of all program direct service staff in attendance, the training(s) attended, and date trainings were completed. On-site: Interview program manager and direct service staff; review personnel files/training log. 157

PAGE 164

Appendix C Continued Fidelity and Innovations 5-3 RATING INDICATORS 3 All staff receive training related to the service population at least annually. 2 There are isolated instances when staff do not receive training related to the service population on an annual basis. 1 Program staff routinely do not complete training related to the service population on an annual basis. 5-4. The program regularly evaluates the extent to which all aspects of its service delivery system (i.e., family assessment, service planning, home visitation, supervision, etc.) are culturally competent. 5-4.A. There is an annual review of cultural competency that addresses the following components: materials, training and service delivery system. 5-4.A. RATING INDICATORS 3 The annual review of cultural competency comprehensively addresses materials, training and the service delivery system (i.e., assessment, service planning, home visitation, supervision, etc.) 2 The annual review of cultural competency addresses the components of the service delivery system related to cultural differences. However, the components are not addressed in a comprehensive manner. 1 There is no annual cultural competency review or it does not address one or more of the components listed above. 5-4.B. The annual review of culturally competent practices includes participant input regarding culturally appropriate services. EVIDENCE 5-4.A. Pre-site: Please submit a copy of the most recent annual review of cultural competency undertaken by the program to evaluate how well it is accommodating cultural differences and utilizing cultural and participant strengths and resources. On-site: Interview program manager. EVIDENCE 5-4.B. Pre-site: Please submit a copy of the most recent annual review undertaken by the program to evaluate how well it is accommodating cultural differences and utilizing cultural and family strengths and resources, as submitted in 5-4.A. Program may want to submit any relevant forms, etc. used to gather information that was used in the review. On-site: Interview program manager, direct service staff and participants and review data collection forms used in review. 158

PAGE 165

Appendix C Continued Fidelity and Innovations 5-4.B. RATING INDICATORS 3 The program has a review of its practices for cultural competency and includes direct input from the participants on culturally sensitive practice, materials, communication and staff-participant interaction. The review is comprehensive and up-todate. 2 The review includes direct input from participants on at least three of the four areas identified in the rating above (culturally sensitive practice, materials, communication and staff-participant interaction), but could be more comprehensive. 1 The review does not use participant input; there is no review; it is not completed annually or it covers fewer than three of the items listed in the rating. 5-4.C. The annual review of culturally competency practices includes staff input regarding culturally appropriate services. 5 -4.C RATING INDICATORS EVIDENCE 5-4.C. Pre-site: Please submit a copy of the most recent annual review undertaken by the program to evaluate how well it is accommodating cultural differences and utilizing cultural and family strengths and resources, as submitted in 5-4.A. Program may want to submit any relevant forms, etc. used to gather information that was used in the review. On-site: Interview program manager and other direct service staff and review data collection forms used in review. 3 The program has a review of its practices for cultural competency and includes direct input from the staff on culturally sensitive practice, materials, communication and staff-participant interaction. The review is comprehensive and up-to-date. 2 The review includes direct input from staff on at least three of the four areas identified in the rating above (culturally sensitive practice, materials, communication and staff-participant interaction), but could be more comprehensive. 1 The review does not use staff input; there is no review; it is not completed annually or it covers fewer than three of the items listed in the three rating. 5-4.D. The review is reported at least annually to the appropriate supervisory or advisory/governance group. 159

PAGE 166

Appendix C Continued Fidelity and Innovations 5-4.D. RATING INDICATORS 3 No 3 rating indicator for standard 5-4.D. 2 The review is reported at least annually to the appropriate group. 1 The review is not reported at least annually to the appropriate group; or there is no review. 5-4.E. The appropriate supervisory or advisory/governance group takes action on the recommendations contained within the report. 5 -4.E. RATING INDICATORS 3 No 3 rating indicator for standard 5-4.E. 2 Action was taken by the appropriate group. 1 No action was taken. NA No corrective action identified, therefore no need for supervisory or advisory/governing group to take action. 6-1. Issues identified by the participant in the initial assessment are addressed during the course of home visiting. EVIDENCE 5-4.D. Pre-site: Please indicate to whom the review is reported, the frequency of the reporting, and how action is taken (if necessary) on the findings of the report. Also, submit a list of individuals to whom the report is presented. On-site: Interview appropriate individuals (as indicated in pre-site answer) regarding the reporting of and action taken on reports and review minutes of advisory/governing group meetings. EVIDENCE 5-4.E. Pre-site: Please indicate to whom the review is reported, the frequency of the reporting, and how action is taken (if necessary) on the findings of the report. Also, submit a list of individuals to whom the report is presented. On-site: Interview appropriate individuals (as indicated in pre-site answer) regarding the reporting of and action taken on reports and review minutes of supervisory or advisory/governing group meetings. 6. Services should focus on supporting the parent(s) as well as supporting parent-child interaction and child develo p ment. 160

PAGE 167

Appendix C Continued Fidelity and Innovations 6-1.A. The supervisor and home visitor review the issues identified by the participant in the initial assessment. 6 -1.A RATING INDICATORS 3 Based on the programs written guidelines, the supervisor and home visitor consistently address and review issues identified in the initial assessment. 2 Based on the programs written guidelines, isolated instances were found when the supervisor and home visitor did not address or review issues identified in the initial assessment. 1 Either the program does not have written guidelines or the supervisor and home visitor do not consistently address and review issues identified in the initial assessment. 6-1.B. The home visitor and participant review issues identified in the initial assessment. 6-1.B RATING INDICATORS EVIDENCE 6-1.A. Pre-site: Please submit the programs written guidelines describing how the home visitor and supervisor review issues identified in the initial assessment with participants. On-site: Review participant files and supervision notes/logs and interview home visitor and supervisor. EVIDENCE 6-1.B. Pre-site: Please submit the programs written guidelines describing how the home visitor and participant review issues identified in the initial assessment. On-site: Review participant files and supervision notes/logs and interview participant, home visitor, and supervisor. 3 Based on the programs written guidelines, the home visitor addresses and reviews issues identified in the initial assessment with the participant. 2 Based on the programs written guidelines, isolated instances were found when the home visitor did not address and review issues identified in the initial assessment with the participant. 1 Either the program does not have written guidelines or the home visitor and participant do not consistently address and review issues identified in the initial assessment. 6-2. Delivery of services to participants is guided by the Individual Family Support Plan (IFSP) and the process of developing the plan uses participant support practices. 6-2.A. The home visitor and participant collaborate to identify participant strengths and competencies. 161

PAGE 168

Appendix C Continued Fidelity and Innovations 6 -2.A. RATING INDICATORS EVIDENCE 6-2.A. through 6-2.G. Pre-site: Please submit the programs written guidelines describing how the IFSP promotes participant support practices, how the IFSP serves as the basis for service delivery, how collaboration between the home visitor and participant is documented by the program and how often the IFSP is reviewed. Be sure to include a copy of the IFSP used by the program, as well as any other documentation that would support home visitor-participant collaboration in the planning and delivery of services. On-site: Review participant files and supervision notes/logs and interview participant, home visitor and supervisor. 3 The home visitor and participant collaborate to identify participant strengths and competencies. 2 The home visitor and participant collaborate to identify participant strengths and competencies. However, isolated instances found when collaboration did not occur. 1 The home visitor and participant do not consistently collaborate. 6-2.B. The home visitor and participant collaborate to assess participant needs and the services which are desired to help address these needs. 6-2.B. RATING INDICATORS 3 The home visitor and participant collaborate to assess participant needs and the services that are desired to help address those needs. 2 The home visitor and participant collaborate to assess participant needs and the services that are desired to help address those needs. However, isolated instances found when it did not occur. 1 The home visitor and participant do not consistently collaborate. 6-2.C. The home visitor and participant collaborate to set participant goals for the home visitation service. 6-2.C. RATING INDICATORS 3 The home visitor and participant collaborate to set participant goals, taking into consideration participant strengths, needs and concerns. 2 The home visitor and participant collaborate to set participant goals, taking into consideration participant strengths, needs and concerns. However, isolated instances found when collaboration did not occur. 1 The home visitor and participant do not consistently collaborate or participant strengths, needs and concerns are not taken into consideration. 162

PAGE 169

Appendix C Continued Fidelity and Innovations 6-2.D. The home visitor and participant collaborate to establish a plan with specific strategies/objectives to achieve identified goals. 6-2.D. RATING INDICATORS 3 The home visitor and participant collaborate to establish a plan with specific strategies/objectives to achieve identified goals. 2 The home visitor and participant collaborate to establish a plan with specific strategies/objectives to achieve identified goals. However, isolated instances found when it did not occur. 1 The home visitor and participant do not consistently collaborate or the strategies/objectives are inconsistently linked to identified goals. 6-2.E. The home visitor, and supervisor review IFSP progress at regular intervals (i.e., bi-weekly, monthly, quarterly). 6-2.E. RATING INDICATORS 3 The home visitor and supervisor review the progress of the participant toward meeting identified goals at regular intervals. 2 The home visitor and supervisor routinely review the progress of the participant toward meeting identified goals at regular intervals. However, isolated instances were found when it did not occur. 1 The home visitor and supervisor do not consistently review the progress of the participant toward meeting identified goals at regular intervals. 6-2.F. The home visitor, participant and supervisor collaborate to update each participants IFSP at regular intervals. (All parties do not have to be present at the same time to conduct this review). 6-2.F. RATING INDICATORS 3 The home visitor, participant and supervisor collaborate to update the IFSP and review occurs at least quarterly. 2 The home visitor, participant and supervisor routinely collaborate to update the IFSP at least semi-annually. However, isolated instances were found when it did not occur according to this timeframe. 1 The home visitor, participant and supervisor do not consistently collaborate to update the IFSP or the program does not update IFSPs at least semi-annually. 6-2.G. The IFSP serves as the guide for delivering services. 6-2.G. RATING INDICATORS 3 The IFSP serves as the guide for delivering services. 2 Isolated instances were found when the IFSP did not serve as the guide for 163

PAGE 170

Appendix C Continued Fidelity and Innovations delivering services. 1 The IFSP inconsistently serves as the guide for delivering services. 6-3. Before or on the first home visit, the participant is informed about their rights, including confidentiality, both verbally and in writing. 6 3 RATING INDICATORS 3 The written policy states that the participants (as defined by the program) are informed about the right to confidentiality before or on the first home visit. There is evidence in participant files to indicate that participants are informed about their rights, including confidentiality, before or on the first home visit, both verbally and in writing. 2 The written policy states that the participants are informed about their rights, including confidentiality, before or on the first home visit. There is evidence in participant files to indicate that participants are being informed routinely about their rights before or on the first home visit. Isolated instances were found where participants were informed of their rights at a time later than first home visit. 1 The program either does not have a written policy that states that the participants are informed about their rights, including confidentiality, before or on the first home visit or there is insufficient evidence in participant files to indicate that participants are being informed about rights before or on the first home visit. 6-4. The program promotes positive parenting skills, parent-child interaction and knowledge of child development with participants. 6-4.A. The program has comprehensive guidelines regarding promotion of positive parenting skills, parent-child interaction and knowledge of child development with participants. EVIDENCE 6-3. Pre-site: Please submit a copy of the programs written policy regarding informing participants about their rights, including confidentiality; the form used by the program to signify that participants have been informed of their rights; and a description of who the program defines as a participant. On-site: Review participant files; interview home visitor, participant, and supervisor. EVIDENCE 6-4.A. Pre-site: Please submit a copy of your programs written guidelines regarding promotion of positive parenting skills and knowledge of child development with participants. On-site: Interview home visitor and supervisor. 164

PAGE 171

Appendix C Continued Fidelity and Innovations 6-4.A. RATING INDICATORS 3 No 3 rating indicator for standard 6-4.A. 2 Program has written guidelines regarding promotion of positive parenting skills and knowledge of child development. 1 Program does not have written guidelines. 6-4.B. Home visitor shares information with participants on appropriate activities designed to promote positive parenting skills. 6 -4.B. RATING INDICATORS 3 Home visitor routinely shares information with participants on appropriate activities designed to promote positive parenting skills. 2 Home visitor does share information routinely on appropriate activities designed to promote positive parenting skills, however isolated instances were found when information was not shared routinely with participants. 1 Home visitor does not share information routinely on appropriate activities designed to promote positive parenting skills. 6-4.C. Home visitor shares information with participants on appropriate activities designed to promote positive parent-child interaction. EVIDENCE 6-4.B. Pre-site: Please submit a copy of your programs written guidelines regarding sharing information on appropriate parenting with participants; a list of the parenting curricula used with participants; a description of how the curricula is used with participants and how staff document use within participant files. On-site: Review participant files; review curricula; and interview home visitor, supervisor and participants. EVIDENCE 6-4.C. Pre-site: Please submit a copy of your programs written guidelines regarding sharing appropriate activities with participants designed to promote positive parent-child interaction; a list of the curricula used with participants to promote positive parent-child interaction; a description of how the curricula are used with participants and how staff document use within participant files. On-site: Review participant files; review curricula; and interview home visitor, supervisor and participants. 165

PAGE 172

Appendix C Continued Fidelity and Innovations 6-4.C. RATING INDICATORS 3 Home visitor routinely shares with participants information on appropriate activities designed to promote positive parent-child interaction. 2 Home visitor does share information on appropriate activities designed to promote positive parent-child interaction, however isolated instances were found when information was not shared routinely with participants. 1 Home visitor does not share information routinely on appropriate activities designed to promote positive parent-child interaction. 6-4.D. Home visitor shares information on appropriate infant and child development with participants. 6-4.D. RATING INDICATORS 3 Home visitor routinely shares information on appropriate infant and child development with participants. 2 Home visitor does share information on appropriate infant and child development; however, isolated instances were found when information was not shared routinely with participants. 1 Home visitor does not share information routinely on appropriate infant and child development. 6-4.E. Home visitor shares information with participants on appropriate health and safety related issues. EVIDENCE 6-4.E. Pre-site: Please submit a copy of your programs written guidelines regarding sharing information on health and safety related issues. On-site: Review participant files; interview home visitor, supervisor and participants. EVIDENCE 6-4.D. Pre-site: Please submit a copy of your programs written guidelines regarding sharing information on appropriate infant and child development with participants; a list of the child development curricula used with participants; provide a description of how the curricula are used with participants and how staff document use within participant files. On-site: Review participant files; interview home visitor, supervisor, and participants. 6-4.E RATING INDICATORS 3 Home visitor routinely shares information on appropriate health and safety related issues. 166

PAGE 173

Appendix C Continued Fidelity and Innovations 2 Home visitor does share information on appropriate health and safety related issues, however isolated instances were found when information was not shared routinely with participants. 1 Home visitor does not share information routinely on appropriate health and safety related issues. 6-5. The program monitors the development of participating infants and children with a standardized developmental screen. 6-5.A. The program has guidelines for administration of a standardized developmental screen/tool that specify how the tool is to be used with all children participating in the program, unless developmentally inappropriate. 6 -5.A. RATING INDICATORS 3 No 3 rating indicator for standard 6-5.A. 2 The program has written guidelines to administer the screen/tool. Guidelines specify that the standardized developmental screen/tool is to be used with all target children in program, unless developmentally inappropriate. 1 The program does not have written guidelines to administer the standardized developmental screen/tool. Or, guidelines do not specify that screen/tool is to be used with all target children in the program, unless developmentally inappropriate. 6-5.B. The program ensures that a standardized developmental screen/tool is used to monitor child development at specified intervals, unless developmentally inappropriate. 6-5.B. RATING INDICATORS EVIDENCE 6-5.A. Pre-site: Please submit a copy of the programs written guidelines regarding use of a developmental screen or other standardized tool(s) to monitor infant/child development, including guidelines for administering the tool(s) at specified intervals and indicate the name of the tool. On-site: Review tool(s); interview home visitor and supervisor. EVIDE N CE 6-5.B. Pre-site : Please subm it a copy of y our program s written guidelines as subm itted in 6-B.A. above. On-site : Interview hom e visitor and supervisor; review participant files (or other relevant docum entation) for evid ence of tool being used. 3 The progra m u s e s the sta ndar d ized dev e lop m ental scre en/too l at specified in terval s to m o nitor ch ild develop m e n t for all target children i n the progra m unless devel o p m entally inappropriate. 167

PAGE 174

Appendix C Continued Fidelity and Innovations 2 The program uses the standardized developmental screen/tool at specified intervals to monitor child development for all target children in the program. Isolated instances were found when screen/tool was not used with target children, unless developmentally inappropriate. 1 The program does not use the standardized developmental screen/tool; the program does not use the standardized developmental screen/tool at the specified intervals; or the program does not consistently use the standardized developmental tool with all target children in the program. 6-6. Those who administer developmental screenings have been trained in the use of the tool before administering it. 6-6. RATING INDICATORS 3 All staff using the tool have been trained in its use before administering it. 2 Isolated instances were found when staff administered screen before being trained. 1 Evidence demonstrates that several staff who administer the tool are not trained in its use or evidence exists to indicate that staff routinely administer the tool prior to being trained. 6-7. The program tracks target children who are suspected of having a developmental delay and follows through with appropriate interventions (e.g., referrals, follow-up, etc.) as needed. 6-7.A. The program has guidelines that address how it tracks and follows through with appropriate actions for child participants suspected of having a developmental delay. EVIDENCE 6-7.A. Pre-site: Please submit a copy of the programs writtenguidelines describing how program tracks, refers, and follows up on child participants suspected of having a developmental delay. On-site: Interview home visitor and supervisor. 6-7.A RATING INDICATORS 3 No 3 rating indicator for standard 6-7.A. 2 The program has written guidelines to track and follow through with appropriate actions for child participants suspected of having a developmental delay. 1 The program does not have written guidelines to track and follow through with appropriate actions for child participants suspected of having a developmental delay. 6-7.B. The program tracks target children suspected of having a developmental delay. EVIDENCE 6-7.B. Pre-site: Please submit a copy of the forms used to track child participants suspected of having a developmental delay. On-site: Interview home visitor and supervisor; review participant files. 6-7.B RATING INDICATORS 168

PAGE 175

Appendix C Continued Fidelity and Innovations 3 Consistent evidence indicates that the program routinely tracks target children suspected of having a developmental delay. 2 Evidence indicates the program routinely tracks target children suspected of having a delay. Isolated instances were found when this did not happen. 1 Insufficient evidence exists to indicate that the program routinely tracks target children suspected of having a developmental delay. NA No children were identified with a developmental delay. 6-7.C.The program follows through with appropriate actions (i.e., referrals, in-depth evaluations, or examinations, treatment or other services) for target children suspected of having a developmental delay. 6-7.C RATING INDICATORS 3 Consistent evidence indicates that the program routinely follows through with appropriate activities. 2 Evidence indicates that the program routinely follows through with appropriate activities. Isolated instances were found when this did not happen. 1 Insufficient evidence exists to indicate that the program routinely follows through with appropriate activities. NA No children were identified with a developmental delay. 7-1. Participating family members (as defined by the program) have a medical/health care provider to assure optimal health and development. 7. At a minimum, all families should be linked to a medical provider to assure optimal health and development (e.g., timely immunizations, well-child care, etc.) Depending on the familys needs, they may also be linked to additional services such as financial, food, and housing assistance programs, school readiness programs, child care, job training programs, family support centers, substance abuse treatment programs, and domestic violence shelters. EVIDENCE 6-7.C. Pre-site: Please submit a copy of the forms used to refer and monitor the interventions needed for child participants suspected of having a developmental delay (program can refer to the forms submitted in 6-7.B, if applicable). On-site: Interview home visitor and supervisor; review participant files. 169

PAGE 176

Appendix C Continued Fidelity and Innovations 7-1.A. The program has guidelines for documenting medical/health care provider(s) for all participating family members. 7-1.A RATING INDICATORS 3 No 3 rating indicator for standard 7-1.A. 2 Program has written guidelines for documenting medical/health care providers for all participating family members. Consistent evidence indicates that guidelines are being followed. 1 The program has no written guidelines to document medical/health care providers or guidelines are not being followed. 7-1.B. Home visitors provide information, referrals and linkages to available health care resources for all participating family members. 7-1.B. RATING INDICATORS 3 Evidence indicates that home visitors consistently provide information, referrals and linkages to participating family members on available health care resources. 2 Isolated instances were found when home visitors did not provide information, referrals and linkages to participating family members on available health care resources. 1 Insufficient evidence exists to suggest that home visitors are providing information, referrals and linkages to participating family members on available health care resources. 7-1.C. Target children have a medical/health care provider. EVIDENCE 7-1.A. Pre-site: Please submit a copy of the programs definition of participating family members and medical/health care provider and written guidelines for how workers should document each. On-site: Interview program manager and home visitors if needed. EVIDENCE 7-1.B. Pre-site: Please submit a brief description of how the program works with participants to link them to health care providers. On-site: Interview program manager, home visitor, and participants. Review participant files. EVIDENCE 7-1.C. Pre-site: Please submit a report detailing the number of target children and the percent who currently have a medical/health care provider. On-site: Interview program manager (if needed); review participant files. 7-1.C. RATING INDICATORS 170

PAGE 177

Appendix C Continued Fidelity and Innovations 3 Ninety-five percent (95%) through one hundred percent (100%) of target children have a medical/health care provider. 2 Eighty percent (80%) through ninety-four percent (94%) of target children have a medical/health care provider. 1 Less than eighty percent (80%) of target children have a medical/health care provider. 7-2. The program ensures that immunizations are up to date for target children. 7-2.A. The program identifies an immunization schedule to be met and has guidelines to document immunizations for all target children. 7 -2.A RATING INDICATORS EVIDENCE 7-2.A. Pre-site: Please submit a copy of the programs immunization schedule and writtenguidelines for how workers should document immunizations. On-site: Interview program manager and home visitors if needed; review participant files. 3 No 3 rating indicator for standard 7-2.A. 2 Program follows an identified immunization schedule (e.g., American Academy of Pediatrics, Center for Disease Control, State Department of Public Health, etc.) and has written guidelines to document immunizations for all target children. 1 The program does not have an identified immunization schedule or program does not have written guidelines to document immunizations for all target children. 7-2.B. Immunizations for target children are up to date. (Please note: the percentage should not include children whose family beliefs preclude immunizations. Evidence of this must be documented in the participant file.) EVIDENCE 7-2.B. Pre-site: Please submit a report detailing the number of all target children and the percent whose immunizations are up to date. On-site: Interview program manager if needed; review participant files. 7-2.B. RATING INDICATORS 3 Ninety percent (90%) through one hundred percent (100%) of target children have up-to-date immunizations. 171

PAGE 178

Appendix C Continued Fidelity and Innovations 2 Eighty percent (80%) through eighty-nine percent (89%) of target children have upto-date immunizations. 1 Less than eighty percent (80%) of target children have up-to-date immunizations. 7-3. Participants are linked to additional services on an as-needed basis taking into account one or more of the following: information gathered in the assessment process, through the development of the IFSP, through home visits, from other service providers, etc. 7-3.A. The program connects participants to appropriate referral sources and services in the community based upon the information gathered. 7-3.A. RATING INDICATORS 3 Consistent evidence indicates that participants are connected to appropriate services in the community. 2 Isolated instances were found when participants needing referral were not connected to appropriate services in the community. 1 There is insufficient evidence to indicate that participants needing referral are connected to appropriate services in the community. This lack of connection is not due to the fact that referral services are full. 7-3.B. The program follows up with the referral source, service provider, and/or participant to determine if the participant received needed services. EVIDENCE 7-3.A. Pre-site: Please submit your programs written guidelines (including instances when referral sources are full, families opt not to follow up on referral(s), referral sources are not allowed to confirm participant involvement, etc.) for using information gathered in the assessment process, through the development of the IFSP, through home visits, and from other service providers, etc. to connect participants to appropriate referral sources in the community. On-site: Interview home visitor, participants, and supervisor; review participant files for family support plan and referral reports. EVIDENCE 7-3.B. Pre-site: Please submit a copy of the programs written guidelines describing how it follows up with community resources and/or the participant to determine if the participant received needed services. Be sure to include any relevant documentation such as referral tracking forms, memoranda of understanding with other agencies, etc. On-site: Interview home visitor, participant, and supervisor; review participant files for referral reports. 172

PAGE 179

Appendix C Continued Fidelity and Innovations 7-3.B RATING INDICATORS 3 The program has a method for tracking and following up on referrals of participants to other community services. Consistent evidence indicates that program is tracking and following up on referrals. 2 The program has a method for tracking and following up on referrals of participants to other community services. Evidence indicates that program is tracking and following up on referrals. Isolated instances were found when tracking and follow-up did not occur. 1 Either the program does not have a method or the program has a method but there is insufficient evidence to indicate that tracking and follow-up is occurring. 8-1. Services are provided by staff with limited caseloads to assure that home visitors have an adequate amount of time to spend with each participant to meet their needs and plan for future activities. 8-1.A. The programs policy regarding established caseload size is no more than fifteen (15) participants at the most intensive level (at least weekly visits) per full time home visitor. 8-1.A. RATING INDICATORS EVIDENCE 8-1.A. 8-1.C. Pre-site: Please submit a copy of the programs written policy regarding caseload sizes and a report showing the average caseload size of all current home visitors over the past year (include breakdowns for each home visitors caseload for the quarter immediately prior to completing the self-assessment, the number of participants assigned to him or her, and the intensity of service each participant is receiving). On-site: Review caseload listings; interview supervisor and home visitors. 8. Services should be provided by staff with limited caseloads to assure that home visitors have an adequate amount of time to spend with each family to meet their unique and varying needs and to plan for future activities (i.e., for many communities, no more than fifteen (15) families per home visitor on the most intense service level. And, for some communities, the number may need to be significantly llh(10)) 3 No 3" rating indicator for standard 8-1.A. 2 The programs written policy states that caseload size is no more than fifteen (15) participants at the most intensive service level (at least weekly visits) per full time home visitor. 1 The programs written policy states that caseload size is more than fifteen (15) participants at the most intensive service level (at least weekly visits) per full time home visitor. 8-1.B. The programs policy regarding maximum caseload size is no more than twenty-five (25) at any combination of service levels per full-time home visitor. 173

PAGE 180

Appendix C Continued Fidelity and Innovations 8-1.B. RATING INDICATORS 3 No 3" rating indicator for standard 8-1.B. 2 The programs policy regarding maximum caseload size is no more than twenty-five (25) participants at any combination of service levels per full time home visitor. 1 The programs policy regarding maximum caseload size exceeds twenty-five (25) participants at any combination of service levels per full time home visitor. 8-1.C. Home visitors are within the caseload ranges, as stated in standard 8-1.A and 8-1.B. 8-1.C. RATING INDICATORS 3 No home visitor exceeds the caseload sizes, as stated in standards 8-1.A and 8-1B. 2 Most home visitors do not exceed the caseload sizes as stated in 8-1.A and 8-1.B. Evidence suggests any deviation is temporary. 1 Home visitors routinely exceed the caseload sizes as stated in 8-1.A. and 8-1.B. 8-2. The programs caseload system ensures that home visitors have an adequate amount of time to spend with each participant. 8-2.A. The program has guidelines for managing its caseloads. 8-2.A RATING INDICATORS EVIDENCE 8-2.A. Pre-site: Please submit a copy of the written guidelines the program uses to manage caseloads and any relevant forms used in managing this system. 3 No 3 rating indicator for standard 8-2.A. 2 The program has written guidelines which include the following criteria: Experience and skill level of the home visitor assigned; Nature and difficulty of the problems encountered; Work and time required to serve each participant; Number of participants per service provider which involve assessment and/or more intensive intervention; Travel and other non-direct service time required to fulfill the service providers responsibilities; Extent of other resources available in the community to meet participant needs; and Other assigned duties. 1 The program does not have written guidelines or the guidelines do not include all the criteria listed above in the rating indicator. 174

PAGE 181

Appendix C Continued Fidelity and Innovations 8-2.B. The program uses the guidelines identified above in 8-2.A. to manage its caseload sizes. 8-2.B. RATING INDICATORS 3 The program has guidelines it routinely uses in the management of its caseloads. Criteria must include: Experience and skill level of the home visitor assigned; Nature and difficulty of the problems encountered; Work and time required to serve each participant; Number of participants per service provider which involve assessment and/or more intensive intervention; Travel and other non-direct service time required to fulfill the service providers responsibilities; Extent of other resources available in the community to meet participant needs; and Other assigned duties. 2 The program uses the above criteria for managing its caseload, but isolated instances were found when criteria were not used. 1 The program has no criteria or criteria are not being used routinely. EVIDENCE 8-2.B. On-site: Interview supervisor and home visitor; review documentation of caseload management system. 9. Service providers should be selected because of their personal characteristics (i.e., non-judgmental, compassionate, ability to establish a trusting relationship, etc.), their willingness to work in or their experience working with culturally diverse communities, and their skills to do the job. 9-1. Service providers and program management staff are selected because of a combination of personal characteristics, experiential, and educational qualifications. 9-1.A. Screening and selection of program managers includes consideration of characteristics including, but not limited to: A solid understanding of and experience in managing staff; Administrative experience in human service or related program(s), including experience in quality assurance/improvement and program development; 175

PAGE 182

Appendix C Continued Fidelity and Innovations A bachelors degree in human services administration or related field required (Masters degree preferred). 9-1. A RATING INDICATORS 3 Programs system for screening and selection of program managers ensures that it considers personal characteristics of job candidates, including, but not limited to: A solid understanding and experience in managing programs; Administrative experience in human service or related program(s), including experience in quality assurance/improvement and program development; and A bachelors degree in human services administration or related field required (Masters degree preferred). 2 The programs system screens and selects for a majority, but not all of the characteristics listed above. 1 The program does not screen for a majority of the characteristics listed above; the system is not followed when hiring; or the program does not have a screening or selection system for hiring program managers. 9.1-B. Program managers have: A solid understanding and experience in managing staff; Administrative experience in human service or related program(s), including experience in quality assurance/improvement and program development; and A bachelors degree in human services or related field required (Masters degree preferred). EVIDENCE 9-1.A. Pre-site: Please submit a description of the programs screening and selection procedures for program managers. Be sure to include any relevant materials used during the screening/selection procedure, such as interview guidelines, job descriptions, qualifications required at hire, etc. On-site: Interview program management staff responsible for the recruitment and hiring process for the program manager position; interview program manager. EVIDENCE 9-1.B. Pre-site: Please attach a summary of the qualifications of all program management staff (i.e., a resume or vitae). Be sure the summary includes work and educational experience. On-site: Interview program manager and management staff responsible for or involved in selection process. 176

PAGE 183

Appendix C Continued Fidelity and Innovations 9-1.B. RATING INDICATORS 3 Program managers routinely meet a majority of the criteria stated in standard 9-1.A. 2 Instances found when program managers did not meet the criteria, but a staff development or mentoring plan for managers is in place. 1 Program managers routinely do not meet the criteria stated in the standard and there is no development or mentoring plan. 9-1.C. Screening and selection of supervisors includes consideration of characteristics, including but not limited to: A solid understanding of and experience in supervising and motivating staff, as well as providing support to staff in stressful work environments; Knowledge of infant and child development and parent-child attachment; Experience with participant services that embrace the concepts of family-centered and strength-based service provision; Knowledge of maternal-infant health and dynamics of child abuse and neglect; Experience in providing services to culturally diverse communities/participants; Experience in home visitation with a strong background in prevention services to the 0-3 age population; and, Bachelors degree in human services or related field required (Masters degree preferred). EVIDENCE 9-1.C. Pre-site: Please submit a copy ofthe programs screening and selection procedures for supervisors. Be sure to include any relevant materials used during the screening/selection procedure, such as interview guidelines, job descriptions, qualifications required at hire, etc. On-site: Interview program management staff responsible for screening and selection process for supervisors; interview supervisory staff. 9-1.C RATING INDICATORS 3 Programs system for screening and selection of supervisors ensures that it considers personal characteristics of job candidates, including, but not limited to: A solid understanding and experience in supervising staff; Knowledge of infant and child development and parent child attachment; Experience with participant services that embrace the concepts of familycentered and strength-based service provision; Knowledge of maternal-infant health and dynamics of child abuse and neglect; Experience in providing services to culturally diverse communities/participants; Experience in home visitation with a strong background in prevention services to the 0-3 age population; and, 177

PAGE 184

Appendix C Continued Fidelity and Innovations Bachelors degree in human services or related field required (Masters degree preferred). 2 The programs system screens and selects for a majority of the characteristics listed in standard 9-1.C. 1 The program does not screen for a majority of the characteristics listed above; the system is not followed when hiring; or the program does not have a screening or selection system for hiring supervisors. 9-1.D. Supervisors have: A solid understanding and experience in supervising and motivating staff as well as providing support in stressful work environments; Knowledge of infant and child development and parent child attachment; Experience with participant services that embrace the concepts of family-centered and strength-based service provision; Knowledge of maternal-infant health and concepts of child abuse and neglect; Experience in providing services to culturally diverse communities/participants; Experience in home visitation with a strong background in prevention services to the 0-3 age population; and, Bachelors degree in human services or related field required (Masters degree preferred). 9 -1.D. RATING INDICATORS EVIDENCE 9-1.D. Pre-site: Please attach a summary of the qualifications of all supervisory staff (i.e., a resume or vitae). Be sure the summary includes work and educational experience. On-site: Interview program management and supervisory staff. 3 Supervisors routinely meet all the criteria stated in standard 9-1.C. 2 Instances found when supervisors did not meet all of the criteria, but there is a supervisor development plan in place. 1 Supervisors routinely do not meet the criteria stated in the standard. 9-1.E. Screening and selection of direct service staff include consideration of personal characteristics, including but not limited to: Are experienced in working with or providing services to children and participants; An ability to establish trusting relationships; Acceptance of individual differences; Experience and willingness to work with the culturally diverse populations that are present among the programs target population; Knowledge of infant and child development; and Are experienced in working with or providing services to children and 178

PAGE 185

Appendix C Continued Fidelity and Innovations participants. 9-1.E RATING INDICATORS 3 Programs system for screening and selection of direct service staff ensures that it considers personal characteristics of job candidates, including, but not limited to: Are experienced in working with or providing services to children and participants; An ability to establish trusting relationships; Acceptance of individual differences; Experiences and willingness to work with the diverse population(s) that are present among the programs target population; and Knowledge of infant and child development; Meet the educational requirements, as established by the program. 2 Isolated instances found when the programs system was not followed when hiring or system design is not as effective as possible in screening for personal characteristics. 1 The program does not screen for a majority of the characteristics listed above; or the system is not followed when hiring. 9.1-F. Direct service providers: Are experienced in working with or providing services to children and participants; Have demonstrated ability to establish trusting relationships; Demonstrate acceptance of individual differences; Have experience with and willingness to work with the culturally diverse populations that are present among the programs target population; Are knowledgeable about infant and child development; and, Meet the educational requirements, as established by the program. EVIDENCE 9-1.E. Pre-site: Please submit a description of the programs screening and selection procedures for direct service staff. Be sure to include any relevant materials used during the screening/selection procedure, such as interview guidelines, job descriptions, etc. On-site: Interview supervisors/managers, direct service staff. EVIDENCE 9-1.F. Pre-site: Please attach a list of all direct service staff, along with a summary (i.e., resume or vitae) of their qualifications for the position he/she holds. Also, please attach a copy of all the job descriptions for direct service staff. On-site: Interview direct service staff, supervisors, and managers. 179

PAGE 186

Appendix C Continued Fidelity and Innovations 9.1-F RATING INDICATORS 3 Direct service staff routinely meet all of the criteria listed in Standard 9-1.F. 2 Isolated instances found when direct service staff did not meet all of the criteria, and these issues are being addressed in supervision and/or staff development. 1 Direct service staff routinely do not meet the stated criteria stated. 9-1.G. The same expectations/requirements apply to both direct service staff and volunteers and interns performing the same function. 9-1.G. RATING INDICATORS 3 No "3" rating indicator for standard 9-1.G. 2 The same expectations/requirements apply to both direct service staff and volunteers and interns performing the same function. 1 The program does not apply the same expectations/requirements to volunteers and interns performing the same function as staff. NA The program does not have volunteers and interns performing the same functions as direct service staff. 9-2. The program actively recruits, employs, and promotes qualified personnel and administers its personnel practices without discrimination based upon age, sex, race, ethnicity, nationality, handicap, or religion of the individual under consideration. 9-2.A. The program is in compliance with the Equal Opportunity Act in the United States. EVIDENCE 9-1.G. Pre-site: Please submit a copy of the programs written policy on expectations, requirements of skills and characteristics of volunteers and interns who perform the same function as direct service staff; materials used in screening and selecting volunteers and interns. On-site: Interview direct service staff, volunteers and interns, and program managers. EVIDENCE 9-2.A. On-site: Review materials, such as correspondence with regulatory authorities indicating that there are no known problems or a legal opinion from counsel indicating the agencys practices conform to the law. (In the absence of such materials, the agency may provide a statement indicating whether there have been any findings or rulings against their practices in the past four years.) 180

PAGE 187

Appendix C Continued Fidelity and Innovations 9-2.A RATING INDICATORS 3 There have been no administrative findings or court rulings against the program in this respect. 2 Status is under review and pending final determination; no major difficulties have been identified in the process of a review conducted by a regulatory authority. 1 The program is in process of remediation of identified difficulty, or the program is not in compliance with the applicable law and has not begun corrective action. 9-2.B. The program has a written equal opportunity policy that clearly states its practices in recruitment, employment, transfer and promotion of employees. EVIDENCE 9-2.B. 9-2.C. Pre-site: Provide a copy of the policy. On-site: Copies of dissemination materials, such as posters, statements inserted in advertisements, brochures, etc., which specify the non-discriminatory nature of program personnel practices. 9-2.B. RATING INDICATORS 3 Written policy on equal opportunity guides program practices in all four areas of personnel administration listed above and there are no known violations. 2 Policy, though written, does not include all areas of personnel administration and there are no known violations of equal employment opportunity. 1 The program has no written policy and/or the program has violated its equal opportunity policy. 9-2.C. The program disseminates its equal opportunity policy and uses recruitment materials that specify the non-discriminatory nature of the programs employment practices. 9-2.C. RATING INDICATORS 3 The program posts its equal opportunity policy and includes a statement with recruitment material and advertisements that specifies the non-discriminatory nature of the programs employment practices. 2 The program uses limited means of disseminating information on its nondiscriminatory hiring practices. 1 The program does not disseminate information internally or externally on its position on equal opportunity. 9-3. The programs recruitment and selection procedures assure that its human resource needs are met. 9-3.A. The programs recruitment and selection practices are in compliance with applicable law or regulation and include: 181

PAGE 188

Appendix C Continued Fidelity and Innovations Notification of its personnel of available positions before or concurrent with recruitment elsewhere; Personal interviews with applicants before selection; and, Documentation that three references from unrelated persons have been obtained. EVIDENCE 9-3.A. 9-3.B. Pre-site: If the program has an opinion from counsel that guides its practices, provide a copy. If not, provide a list of statutes or regulations governing, restricting, or mandating background checks. Describe briefly the legal requirements of each and the mechanisms the program has in place for compliance. On-site: Interview human resource personnel and review personnel records. 9-3.A. RATING INDICATORS 3 The programs recruitment and selection procedures contain all three practices identified in the standard for both staff and volunteers. 2 Isolated instances were found where the programs recruitment and selection procedures did not contain all three practices identified in the standard for both staff and volunteers. 1 The programs recruitment and selection procedures consistently do not include all three practices identified in the standard for both staff and volunteers. 9-3.B. The agency conducts appropriate, legally permissible and mandated inquiries into the background of prospective employees and volunteers who will have responsibilities where clients are children. 9-3.B. RATING INDICATORS 3 Program records are complete with regard to additional screening allowed by law in all cases when there is likely to be contact with vulnerable participants such as children. The program is knowledgeable about what is legally permissible and usable in screening applicants. It carefully follows all mandates. 2 Program practice suggests that it conducts background checks on all employees and volunteers who deal with vulnerable participants but has not maintained good documentation of their efforts in all cases. 1 Program neglects to conduct legally permissible background checks on some applicants or for personnel dealing with vulnerable participants and/or program fails to conduct mandated background checks in all cases. NA Background checks are not permissible in the state. Program provides copy of law or legal opinion. 182

PAGE 189

Appendix C Continued Fidelity and Innovations 9-3.C. The rate of personnel turnover is measured and evaluated regularly and action is taken to correct identified problems. EVIDENCE 9-3.C. Pre-site: Please submit a description of the way the program measures and evaluates its personnel turnover and submit an analysis of personnel turnover rate for the most recent past two years. This analysis should be done both for the entire program and for the various job categories in the program. Indicate what, if any, action was taken as a result of this analysis. On-site: Interview human resource personnel and program management and review personnel records. 9-3.C. RATING INDICATORS 3 Measurement and evaluation of the turnover rate of employees and, when utilized, volunteers, is a regular and integral part of program personnel planning, and action to correct identified problems is promptly taken. Turnover rates are examined by specific job categories to identify any aberrant levels of turnover specific to certain categories or sites. Turnover rates are examined in the context of measures of job satisfaction and personnel retention. 2 Recently, program began to evaluate personnel turnover rates, and it is beginning to use this as a means of identifying problems it should address. 1 Program has experienced personnel turnover and turnover rates are irregularly evaluated, little impact of findings seen in addressing need for change; no assessment; or assessment has no evident impact in face of reports to the team by personnel of high turnover. NA Program has not experienced personnel turnover. 183

PAGE 190

Appendix C Continued Fidelity and Innovations NOTE: In order to streamline the responses to critical elements 10 and 11 (which address worker skills and training), we have combined the two critical elements and measure them as one section. 10-1. The program has a system for assuring that the following trainings are made available for all staff (assessment workers, home visitors, and supervisors): Orientation (10-2.A-E); Intensive role specific training (10-3.A-C); Additional training within six months of hire (10-4.A-F.); Additional training within twelve months of hire (10-5.A-F.); and On-going training topics (10-6.) 10-1.A. The program has a training plan that assures access to required trainings in a timely manner for all staff (home visitors, assessment workers and supervisors). EVIDENCE 10-1.A. & B. Pre-site: Please submit a copy of the policy regarding staff participation in training and a description of the system used by the program to ensure all new staff are trained (i.e., a training log, documentation in individual personnel files, etc.). On-site: Interview supervisors, review staff personnel files or training log. 10.a Service providers should have a framework, based on education or experience, for handling the variety of experiences they may encounter when working with at-risk families. All service providers should receive basic training in areas such as cultural competency, substance abuse, reporting child abuse, domestic violence, drug-exposed infants, and services in their community. 10.b Service providers should receive intensive training specific to their role to understand the essential components of family assessment and home visitation (i.e., identifying at-risk families, completing a standardized risk assessment, offering services and making referrals, promoting use of preventive health care, securing medical homes, emphasizing the importance of immunizations, utilizing creative outreach efforts, establishing and maintaining trust with families, building upon family strengths, developing an individual family support plan, observing parent-child interactions, determining the safety of the home, teaching parent-child interaction, managing crisis situations, etc.). 184

PAGE 191

Appendix C Continued Fidelity and Innovations 10-1.A. RATING INDICATORS 3 The program has a comprehensive training plan that ensures timely access of training for all staff. 2 While the training plan is comprehensive, it does not address how it will ensure timely access to all required trainings. 1 There is no plan or the plan does not address all required trainings. 10-1.B. The program has a system to monitor staff training. 10-1.B. RATING INDICATORS 3 The system used by the program to monitor staff training is designed in such a way that it is easy to determine if all training needs and requirements are met. 2 The program has a system in place, while sufficient for its needs, it could be more comprehensive and easier to determine if training needs and requirements are met. 1 There is no system or the system is insufficient to determine if all training needs and requirements are met. 10-2. Staff (assessment workers, home visitors and supervisors), receive orientation (separate from intensive role specific training) prior to direct work with children and families to familiarize them with the functions of the program. [Please note: All interns and volunteers who perform the same duties as assessment workers, home visitors and supervisors should receive the same type of training as paid staff. Also, self-study training (i.e., reading manuals, watching videos, listening to tapes, etc.) can qualify as training when coupled with competency based testing and/or supervisory follow-up to ensure understanding of materials.] 10-2.A. Assessment workers and home visitors are oriented to their roles as they relate to the programs goals, services, policies and operating procedures, and philosophy of home visiting/family support prior to direct work with children and families. EVIDENCE 10-2.A. through 10-2.F. Pre-site: Please submit an outline of the orientation topics covered by the program with new hires and a copy of all current program staff, his or her date of hire, and the date he or she completed the orientation training. On-site: Interview staff and supervisors; review personnel records/training log. 10-2.A. RATING INDICATORS 3 All staff are oriented to their roles as they relate to the programs goals, services, policies and operating procedures, and philosophy of home visiting/family support prior to direct work with children and families. 185

PAGE 192

Appendix C Continued Fidelity and Innovations 2 Most staff are oriented to their roles as they relate to the programs goals, services, policies and operating procedures, and philosophy of home visiting/family support prior to direct work with children and families. 1 Staff are routinely not oriented to their roles as they relate to the programs goals, services, policies and operating procedures, and philosophy of home visiting/family support prior to direct work with children and families. 10-2.B. Supervisors are oriented to their role as it relates to the programs goals, services, policies and operating procedures, and philosophy of home visiting/family support prior to supervision of staff. 10-2.B. RATING INDICATORS 3 All supervisors are oriented to their roles as they relate to the programs goals, services, policies and operating procedures, and philosophy of home visiting/family support prior to supervision of staff. 2 Most supervisors are oriented to their roles as they relate to the programs goals, services, policies and operating procedures, and philosophy of home visiting/family support prior to supervision of staff. 1 Supervisors are routinely not oriented to their roles as they relate to the programs goals, services, policies and operating procedures, and philosophy of home visiting/family support prior supervision of staff. 10-2.C. Staff (assessment workers, home visitors and supervisors) are oriented to the programs relationship with other community resources prior to direct work with children and families. 10-2.C. RATING INDICATORS 3 All staff are oriented to the programs relationship with other community resources (i.e., organizations in the community with which the program has working relationships) prior to direct work with children and families. 2 Most staff are oriented to the programs relationship with other community resources prior to direct work with children and families. 1 Staff are routinely not oriented to the programs relationship with other community resources prior to direct work with children and families. 10-2.D. Staff (assessment workers, home visitors and supervisors) are oriented to child abuse and neglect indicators and reporting requirements prior to direct work with children and families. 10-2.D. RATING INDICATORS 3 All staff are oriented to child abuse and neglect indicators and reporting requirements prior to direct work with children and families. 2 Most staff are oriented to child abuse and neglect indicators and reporting requirements prior to direct work with children and families. 186

PAGE 193

Appendix C Continued Fidelity and Innovations 1 Staff are routinely not oriented to child abuse and neglect indicators and reporting requirements prior to direct work with children and families. 10-2.E. Staff (assessment workers, home visitors and supervisors) are oriented to issues of confidentiality. 10-2.E. RATING INDICATORS 3 All staff are oriented to issues of confidentiality prior to direct work with families. 2 Most staff are oriented to issues of confidentiality prior to direct work with families. 1 Staff are routinely not oriented to issues of confidentiality prior to direct work with families. 10-2.F. Staff (assessment workers, home visitors and supervisors) are oriented to issues related to boundaries. 10-2.F. RATING INDICATORS 3 All staff are oriented to issues related to boundaries prior to direct work with families. 2 Most staff are oriented to issues related to boundaries prior to direct work with families. 1 Staff are routinely not oriented to issues related to boundaries prior to direct work with families. 10-3. Staff (assessment workers, home visitors and supervisory) receive intensive training within six months of date of hire specific to their role within the home visitation program to help them understand the essential components of their role within the program. [Please note: All interns and volunteers who perform the same duties as assessment workers, home visitors and supervisors should receive the same type of training as paid staff.] 10-3.A. Staff conducting assessments have received intensive role specific training within six months of date of hire to understand the essential components of family assessment. 10-3.A. RATING INDICATORS EVIDENCE 10-3.A. through 10-3.C. Pre-site: Please submit a copy of the training outline(s) used to train workers about their specific role within the home visitation program. Also, include a list of all direct service and supervisory staff, his or her date of hire, and the date he or she completed the role-specific training. On-site: Review training certificates in personnel files; training log; interview program staff and supervisors. 3 All staff conducting assessments routinely receive intensive training on the essential components of family assessment within six months of the date of 187

PAGE 194

Appendix C Continued Fidelity and Innovations hire. Training covers topics such as the role of family assessment, identifying overburdened families, interviewing skills, conducting risk assessments, completing necessary paperwork and documentation, family-centered support services, communication skills, etc. 2 Isolated instances were found when staff conducting assessments did not receive intensive role specific training within six months of hire. 1 Staff conducting assessments routinely do not receive such training within specified time frame or training does not sufficiently address the role of the assessment worker. 10-3.B. Home visitors have received intensive role specific training within six months of date of hire to understand the essential components of home visitation. 10-3.B. RATING INDICATORS 3 All home visitors receive intensive training on the essential components of home visitation within six months of the date of hire. Training covers topics such as establishing and maintaining trust with families, completing necessary paperwork/documentation, the role of the home visitor, communication skills, crisis intervention, etc. 2 Isolated instances were found when home visitors did not receive intensive role specific training within six months of hire. 1 Home visitors routinely do not receive such training within specified time frame or training does not sufficiently address the role of the home visitor. 10-3.C. Supervisory staff have received intensive role specific training within six months of date of hire to understand the essential components of their role within the home visitation program, as well as the role of family assessment and home visitation. 10-3.C. RATING INDICATORS 3 All supervisory staff receive intensive training specific to their role on the essential components of assessment, home visitation, and supervision within six months of the date of hire. Training covers topics such as: the role of family assessment and home visitation, effective supervision, quality management techniques, crisis management, understanding the programs policies and procedures, and case management. 2 Isolated instances were found when supervisory staff did not receive intensive role specific training within six months of hire. 1 Supervisory staff routinely do not receive training specific to their role within specified time frame or training does not sufficiently address the role of one or more of the following: assessment, home visiting, supervision. 10-4. Staff (assessment workers, home visitors and supervisory) demonstrate knowledge on a variety of topics necessary for effectively working with families and children within six months of hire. [Please note: All interns and volunteers who perform the same duties as assessment workers, home visitors and supervisors should receive the same type of training as paid staff. Knowledge can be demonstrated by attendance at trainings, formal education, certification, licensure, competency-based 188

PAGE 195

Appendix C Continued Fidelity and Innovations testing and previous professional experience in each of the specified areas below. Also, self-study training (i.e., reading manuals, watching videos, listening to tapes, etc.) can qualify as training when coupled with competency based testing and/or supervisory follow-up to ensure understanding of materials.] 10-4.A. Staff (assessment workers, home visitors and supervisors) demonstrated knowledge of infant care within six months of the date of hire. 1 0-4.A. RATING INDICATORS EVIDENCE 10-4.A. through 10-4.F. Pre-site: Please submit documentation listing all current program staff, his or her date of hire, and the date he or she demonstrated knowledge on each of the topics listed in standards 10-4.A. through 10-4.F. Evidence of completed trainings should include date received, content (i.e., detailed outline, curriculum, etc.) and length of training. Training and experience must have occurred within three years prior to hire in the HFA program. On-site: Interview staff and supervisors; review personnel records/training log. 3 All staff have demonstrated knowledge on all the recommended content areas on infant care within six months of hire. Recommended content areas include: Sleeping, Feeding/Breastfeeding, Physical care of the baby, and Crying and comforting the baby. 2 All staff have demonstrated knowledge on a majority of the recommended content areas identified above within six months of hire. 1 A majority of staff have not demonstrated knowledge on a majority of the recommended content areas identified above within six months of hire. 10-4.B. Staff (assessment workers, home visitors and supervisors) demonstrated knowledge of child health and safety within six months of the date of hire. 10-4.B. RATING INDICATORS 3 All staff have demonstrated knowledge on all the recommended content areas on child health and safety within six months of hire. Recommended content areas include: Home safety, Shaken baby syndrome, SIDS, Seeking medical care, Well-child visits/immunizations, Seeking appropriate child care, Car seat safety, and 189

PAGE 196

Appendix C Continued Fidelity and Innovations Failure to thrive. 2 All staff have demonstrated knowledge on a majority of the recommended content areas identified above within six months of hire. 1 A majority of staff have not demonstrated knowledge on a majority of the recommended content areas identified above within six months of hire. 10-4.C. Staff (assessment workers, home visitors and supervisors) demonstrated knowledge of maternal and family health within six months of the date of hire. 10-4.C. RATING INDICATORS 3 All staff have demonstrated knowledge on all the recommended content areas on Maternal and Family Health within six months of hire. Recommended content areas include: Family Planning, Nutrition, Pre-natal/Post-natal healthcare, and Pre-natal/Post-Partum Depression; 2 All staff have demonstrated knowledge on a majority of the recommended content areas identified above within six months of hire. 1 A majority of staff have not demonstrated knowledge on a majority of the recommended content areas identified above within six months of hire. 10-4.D. Staff (assessment workers, home visitors and supervisors) demonstrated knowledge of infant and child development within six months of the date of hire. 10-4.D. RATING INDICATORS 3 All staff have demonstrated knowledge on all the recommended content areas on infant and child development within six months of hire. Recommended content areas include: Language and literacy development, Physical and emotional development, Identifying developmental delays, and Brain development. 2 All staff have demonstrated knowledge on a majority of the recommended content areas identified above within six months of hire. 1 A majority of staff have not demonstrated knowledge on a majority of the recommended content areas identified above within six months of hire. 10-4.E. Staff (assessment workers, home visitors and supervisors) demonstrated knowledge of the role of culture in parenting within six months of the date of hire. 10-4.E. RATING INDICATORS 190

PAGE 197

Appendix C Continued Fidelity and Innovations 3 All staff have demonstrated knowledge on all the recommended content areas on the role of culture in parenting within six months of hire. Recommended content areas include: Working with diverse cultures/populations (age, religion, gender, sexuality, ethnicity, poverty, dads, teens, gangs, disabled populations, etc. Culture of poverty; and Values clarification. 2 All staff have demonstrated knowledge on a majority of the recommended content areas identified above within six months of hire. 1 A majority of staff have not demonstrated knowledge on a majority of the recommended content areas identified above within six months of hire. 10-4.F. Staff (assessment workers, home visitors and supervisors) demonstrated knowledge of supporting the parent-child relationship within six months of the date of hire. 10-4.F. RATING INDICATORS 3 All staff have demonstrated knowledge on all the recommended content areas on supporting the parent-child relationship within six months of hire. Recommended content areas include: Supporting attachment; Positive parenting strategies; Discipline; Parent-Child interactions; Observing parent-child interactions; and Strategies for working with difficult relationships. 2 All staff have demonstrated knowledge on a majority of the recommended content areas identified above within six months of hire. 1 A majority of staff have not demonstrated knowledge on a majority of the recommended content areas identified above within six months of hire. 10-5. Staff (assessment workers, home visitors and supervisors) demonstrate knowledge on a variety of topics necessary for effectively working with families and children within 12 months of hire. [Please note: All interns and volunteers who perform the same duties as assessment workers, home visitors and supervisors should receive the same type of training as paid staff. Knowledge can be demonstrated by attendance at trainings, formal education, certification, licensure, competency-based testing and previous professional experience in each of the specified areas below. Also, self-study training (i.e., reading manuals, watching videos, listening to tapes, etc.) can qualify as training when coupled with competency based testing and/or supervisory follow-up to ensure understanding of materials.] 10-5.A. Staff (assessment workers, home visitors and supervisors) demonstrated knowledge of child abuse and neglect within 12 months of the date of hire. 191

PAGE 198

Appendix C Continued Fidelity and Innovations EVIDENCE 10-5.A. through 10-5.F. Pre-site: Please submit documentation listing all current program staff, his or her date of hire, and the date he or she demonstrated knowledge on each of the topics listed in standards 10-5.A. through 10-5.F. Evidence of completing trainings should include date received, content (i.e., detailed outline, curriculum, etc.) and length of training. Training and experience must have occurred within three years prior to hire in the HFA program. On-site: Interview staff and supervisors; review personnel records/training log. 10-5.A. RATING INDICATORS 3 All staff demonstrated knowledge on all the recommended content areas on child abuse and neglect within twelve months of hire. Recommended content areas include: Etiology of child abuse and neglect, and Working with survivors of abuse. 2 All staff demonstrated knowledge on a majority of the recommended content areas identified above within 12 months of hire. 1 A majority of staff have not demonstrated knowledge on a majority of the recommended content areas identified above within 12 months of hire. 10-5.B. Staff (assessment workers, home visitors and supervisors) demonstrated knowledge of family violence within 12 months of the date of hire. 10-5.B. RATING INDICATORS 3 All staff demonstrated knowledge on all the recommended content areas on family violence within 12 months of hire. Recommended content areas include: Indicators of family violence, Dynamics of domestic violence, Intervention protocols, Strategies for working with families with family violence issue, Referral resource for domestic violence, Effects on children, and Gangs. 2 All staff demonstrated knowledge on a majority of the recommended content areas identified above within 12 months of hire. 1 A majority of staff have not demonstrated knowledge on a majority of the recommended content areas identified above within twelve months of hire. 10-5.C. Staff (assessment workers, home visitors and supervisors) demonstrated knowledge of substance abuse within twelve months of the date of hire. 192

PAGE 199

Appendix C Continued Fidelity and Innovations 10-5.C. RATING INDICATORS 3 All staff demonstrated knowledge on all the recommended content areas on substance abuse within 12 months of hire. Recommended content areas include: Etiology of substance abuse, Culture of drug use, Strategies for working with families with substance abuse issues, Smoking cessation, Alcohol use/abuse, Fetal alcohol syndrome, Street drugs, and Referral resources for substance abuse. 2 All staff demonstrated knowledge on a majority of the recommended content areas identified above within 12 months of hire. 1 A majority of staff have not demonstrated knowledge on a majority of the recommended content areas identified above within 12 months of hire. 10-5.D. Staff (assessment workers, home visitors and supervisors) demonstrated knowledge of staff related Issues within 12 months of the date of hire. 10-5.D. RATING INDICATORS 3 All staff demonstrated knowledge on all the recommended content areas on staff related Issues within 12 months of hire. Recommended content areas include: Stress and time management, Burnout prevention, Personal safety of staff, Ethics, Crisis intervention, and Emergency protocols. 2 All staff demonstrated knowledge on a majority of the recommended content areas identified above within 12 months of hire. 1 A majority of staff have not demonstrated knowledge on a majority of the recommended content areas identified above within 12 months of hire. 10-5.E. Staff (assessment workers, home visitors and supervisors) demonstrated knowledge of family issues within 12 months of the date of hire. 10-5.E. RATING INDICATORS 3 All staff have received training on all the recommended content areas on Family Issues within 12 months of hire. Recommended content areas include: Life skills management, Engaging fathers, Multi-generational families, 193

PAGE 200

Appendix C Continued Fidelity and Innovations Teen parents, Relationships, and HIV and AIDS. 2 All staff demonstrated knowledge on a majority of the recommended content areas identified above within 12 months of hire. 1 A majority of staff have not demonstrated knowledge on a majority of the recommended content areas identified above within 12 months of hire. 10-5.F. Staff (assessment workers, home visitors and supervisors) demonstrated knowledge of mental health within 12 months of the date of hire. 10-5.F. RATING INDICATORS 3 All staff demonstrated knowledge on all the recommended content areas on Mental Health within 12 months of hire. Recommended content areas include: Promotion of positive mental health, Behavioral signs of mental health issues, Depression, Strategies for working with families with mental health issues, and Referral resources for mental health. 2 All staff demonstrated knowledge on a majority of the recommended content areas identified above within 12 months of hire. 1 A majority of staff have not demonstrated knowledge on a majority of the recommended content areas identified above within 12 months of hire. 10-6. The program ensures that all program staff receive ongoing training which takes into account the workers knowledge and skill base. 1 0 6 RATING INDICATORS EVIDENCE 10-6. Pre-site: Please submit a copy of the programs written guidelines for training beyond the orientation period; any training outlines of trainings which have been conducted; and a list of all staff, his/her date of hire, and the ongoing training(s) completed by the staff. On-site: Review personnel files for certificates of further training; training log; interview staff and supervisors. 3 The program ensures that all program staff routinely receive ongoing training beyond the trainings identified in 10-2, 10-3, 10-4 and 10-5. Consistent evidence indicates that: 1) staff are offered and participate in ongoing training, and 2) current topics covered in training take into account workers knowledge and skill base. 2 Isolated instances were found when staff did not participate in ongoing training or topics covered in training did not take into account workers' knowledge and skill base. 194

PAGE 201

Appendix C Continued Fidelity and Innovations 1 The program does not ensure that program staff routinely receive ongoing training or staff does not routinely participate in ongoing training opportunities. 11-1. The program ensures that direct service staff receive regular, and ongoing supervision. 11-1.A. The programs policy states that weekly individual supervision is provided to all direct service staff (i.e., assessment and home visitation staff). 11-1.A. RATING INDICATORS 3 Program policy specifies all direct service staff receive a minimum of two hours per week of regularly scheduled individual supervision. 2 Program policy specifies all direct service staff receive a minimum of one and a half hours per week of regularly scheduled individual supervision. (For less than full-time staff at least one hour of individual supervisory time is required). 1 There is no policy or the policy does not meet the requirements specified in the rating. 11-1.B. The program ensures that weekly individual supervision is received by all direct service staff. (Please note: supervisory sessions should not be split into more than two regularly scheduled sessions). 11. Service providers should receive ongoing, effective supervision so that they are able to develop realistic and effective plans to empower families to meet their objectives; to understand why a family may not be making progress and how to work with the family more effectively; and to express their concerns and frustrations so that they can see that they are making a difference and in order to avoid stress-related burnout. EVIDENCE 11-1.A. Pre-site: Please submit a copy of the programs policy on supervision of direct service staff. EVIDENCE 11-1.B. On-site: Review supervision logs and interview direct service staff and supervisors. 11-1.B. RATING INDICATORS 3 Evidence indicates that all direct service staff receive a minimum of two hours of individual supervisory time per week. (Supervisory sessions are not split into more than two regularly scheduled meetings and less than full-time staff receive a minimum of one hour of individual supervision.) 195

PAGE 202

Appendix C Continued Fidelity and Innovations 2 Evidence indicates that all direct service staff receive a minimum of one and a half hours of individual supervisory time per week. (Supervisory sessions are not split into more than two regularly scheduled meetings and less than full-time staff receive a minimum of one hour of individual supervision.) 1 There is insufficient evidence to indicate that the program is following the acceptable guidelines as outlined in and above. 11-1.C. The ratio of supervisors to direct service staff is sufficient to allow regular, ongoing, and effective supervision to occur. 11-1.C. RATING INDICATORS 3 The ratio of supervisors to direct service staff is one full time supervisor (with full-time supervisor responsibilities for the HFA program) to five full time direct service staff. Consistent evidence indicates the program is following these guidelines. 2 The ratio of supervisors (with full-time supervisor responsibilities for the HFA program) to direct service staff is one full time supervisor to six full time direct service staff. Consistent evidence indicates the program is following these guidelines. 1 Program has no policy on the ratio of supervisors (has full-time supervisor responsibilities for the HFA program) to direct service staff; program policy on the ratio specifies more than six full time direct service staff to one full time supervisor; or there was insufficient evidence that the program is following the acceptable guidelines as outlined in and above. 11-2. Direct service staff (i.e., assessment and home visitation staff) are provided with skill development and professional support and held accountable for the quality of their work. 11-2.A. The program has supervisory procedures to assure that direct service staff (i.e., assessment and home visitation staff) are provided with the necessary skill development to continuously improve the quality of their performance. EVIDENCE 11-1.C. Pre-site: Please submit a list of each supervisor and the staff he/she supervises. Be sure to include any additional roles in which the supervisor serves (i.e., program manager, assessment worker, etc.) and include full-time equivalency information for each person. On-site: Review documentation to support supervisor to staff ratio as submitted above (i.e., supervision logs, etc.), interview direct service staff and supervisors. EVIDENCE 11-2.A. Pre-site: Please submit a copy of the programs supervisory procedures that address staff skill development. On-site: Review documentation that would establish that the procedures are followed (i.e., supervisory logs, etc.); interview direct service staff and supervisors. 196

PAGE 203

Appendix C Continued Fidelity and Innovations 11-2.A RATING INDICATORS 3 Program has supervisory procedures that assure all direct service staff receive skill development (other than formal training) and consistent evidence indicates the program is following its procedures. Procedures can include a variety of mechanisms such as: Feedback on strength-based approaches and interventions used (i.e., problem-solving, crisis intervention, etc.), Shadowing, Feedback on documentation, Integration of results of tools used (i.e., developmental screens, evaluation tools, etc), Integration of quality assurance results, Guidance provided on use of curriculum, Guidance provided on techniques and approaches, Identification of areas for growth, Identification of potential boundary issues, and Sharing of information related to community resources. 2 -Isolated instances were found when the procedures were not used. 1 Program has no procedures; procedures do not adequately ensure staff receive skill development; or evidence suggests procedures are not routinely used. 11-2.B. The program has supervisory procedures to assure that direct service staff (i.e., assessment and home visitation staff) are provided with the necessary professional support to continuously improve the quality of their performance. 1 1 2.B. RATING INDICATORS EVIDENCE 11-2.B. Pre-site: Please submit a copy of the programs supervisory procedures that address professional support. On-site: Review documentation to support that supervisory procedures are being followed (i.e., supervision logs, etc.), interview direct service staff and supervisors. 3 Program has supervisory procedures that assure that all direct service staff receive professional support (other than formal training) to continuously improve the quality of their performance and consistent evidence indicates that the program is following its procedures. Procedures can include a variety of mechanisms, such as: Regular staff meetings, On-call availability to service providers, Exploration of impact of the work on the worker, Employee assistance program, Clinical supervision, acknowledgement of performance, provision of tools for performing job; Creating a nurturing work environment that provides opportunities for respite, 197

PAGE 204

Appendix C Continued Fidelity and Innovations Scheduling flexibility, and Providing a career ladder for direct service staff. 2 Isolated instances were found when the program did not follow its procedures. 1 Any of the following: program has no procedures; procedures do not adequately ensure staff receives professional support; or evidence suggests procedures are not routinely used. 11-2.C. The programs supervisory procedures assure that direct service staff (i.e., assessment and home visitation staff) are held accountable for the quality of their work. 11-2. C. RATING INDICATORS 3 The programs supervisory procedures ensure that all direct service staff are held to a high standard of quality for their work and evidence suggests that procedures are routinely used. Procedures may include: Regular and routine review of assessments and assessment records; Regular and routine review of cases and home visitor records; and Regular monitoring by the supervisor of all types of documentation used in the program. 2 Programs supervisory procedures ensure that all direct service staff are held to a high standard of quality for their work. Isolated instances found when the procedures were not used. 1 Any of the following: program has no procedures; procedures do not adequately ensure staff are held to high standard of quality for their work; or evidence suggests procedures are not routinely used. 11-3. The programs policies and procedures manual is used to guide newer service providers in the delivery of services. 11-3.A. The program has a policies and procedures manual. EVIDENCE 11-2.C. Pre-site: Please submit a copy of the programs supervisory procedures that address staff accountability. On-site: Review documentation that would establish that the procedures are followed (i.e., supervisory logs, etc.); interview direct service staff and supervisors. EVIDENCE 11-3.A. Pre-site: Please submit a copy of the programs policies and procedures manual. On-site: Interview supervisors and/or direct service workers. 198

PAGE 205

Appendix C Continued Fidelity and Innovations 11-3.A. RATING INDICATORS 3 No "3" rating indicator for standard 11-3.A. 2 Program has a policies and procedures manual. 1 Program does not have a policies and procedures manual. 11-3.B. The program uses policies and procedures manual as a guide in the provision of services. 11-3.B. RATING INDICATORS 3 No "3" rating indicator for standard 11-3.B. 2 Sufficient evidence to indicate that the program routinely uses the manual as a guide in the provision of services, particularly for newer employees. 1 Insufficient evidence to indicate that the program routinely uses the manual as a guide in the provision of services, particularly for newer employees or program does not have a policies and procedures manual. 11-4. Volunteers and student interns who are performing the same/similar functions as direct service staff are receiving the same type and amount of supervision. 1 1-4. RATING INDICATORS EVIDENCE 11-3.B. Pre-site: Please submit orientation protocols (or narrative) describing use of policies and procedures manual, especially for new service providers. On-site: Interview direct service staff and supervisors. EVIDENCE 11-4. Pre-site: Please submit written guidelines for volunteer supervision or narrative describing volunteer supervision. On-site: Interview with volunteer, interview supervisor. 3 No 3 rating indicator for standard 11-4. 2 Supervision of volunteers and interns follow same guidelines as supervision of direct service staff who are performing similar tasks. 1 The program does have volunteers and/or interns performing similar functions to direct service staff, but insufficient evidence exists that supervision of volunteers and/or interns follows same guidelines as supervision of paid employees performing similar tasks. NA The program does not have volunteers or interns performing same functions as direct service staff. 11-5. Supervisors receive regular, on-going supervision which holds them accountable for the quality of their work and provides them with skill development and professional support. 11-5.A. The program has procedures to assure that supervisors receive regular and on199

PAGE 206

Appendix C Continued Fidelity and Innovations going supervision which holds them accountable for the quality of their work and provides them with skill development and professional support. 11-5.A. RATING INDICATORS 3 The programs written procedures assure that supervisors are held accountable for the quality of their work, receive skill development and professional support. Procedures can include a variety of mechanisms such as: Addressing boundary issues, Feedback to supervisors, Review of supervisors documentation, Strategies to promote professional development/growth, and Quality oversight. 2 The programs written procedures assure that supervisors are held accountable for the quality of their work and receive skill development and professional support. However, isolated instances were found in practice when supervisors are not supervised in such a way to ensure quality work, skill development and professional support. 1 The programs written supervisory procedures do not assure that supervisors are held accountable for the quality of their work and receive skill development and/or professional support. Or, there is insufficient evidence to indicate that supervision of supervisors ensures quality work, skill development and professional support. 11-5.B. Program ensures that supervisors receive regular, on-going supervision. 11-5.B. RATING INDICATORS 3 Based on the programs written policy, supervisors receive supervision on a consistent basis. 2 Isolated instances in practice were found when supervisors did not receive supervision as identified in their written program policy. 1 There is insufficient evidence to indicate that the program is following its written policy. 11-6. Program managers are held accountable for the quality of their work and are provided with skill development and professional support. 11-6.A. RATING INDICATORS EVIDENCE 11-6. Pre-site: Please submit written procedures for supervision of program managers. On-site: Interview with supervisor of program managers and interview supervisor. 3 The programs written procedures assure that program managers are held accountable for the quality of their work, and receive skill development and professional support. 2 The programs written procedures assure that program managers are held accountable for the quality of their work, and receive skill development and professional support. 200

PAGE 207

Appendix C Continued Fidelity and Innovations However, isolated instances in practice were found when program managers are not supervised in such a way to ensure quality work, skill development and professional support. 1 The programs written supervisory procedures do not assure that program managers are held accountable for the quality of their work, or receive skill development and/or professional support. Or, there is insufficient evidence to indicate that supervision of program managers ensures quality work, skill development and professional support. GA-1. The program has a written statement of purpose that guides the administration of its services. GA-1.A. The program has a written statement of purpose that reflects the goals and criteria contained in the critical elements and addresses the needs of children, families, and the community. GA-1.A. RATING INDICATORS 3 The program has a written statement of purpose that reflects the goals and criteria contained in the critical elements and the needs of children, families, and the community. 2 The program has a written statement of purpose, however it does not address all of the goals and criteria listed in the standard. 1 Either the program does not have a written statement of purpose or the programs written statement of purpose does not reflect the intent of the standard. GA-1.B. The statement is reviewed formally by the programs advisory/governing group at least every four (4) years. GOVERNANCE AND ADMINISTRATION The program is governed and administered in accordance with principles of effective management and of ethical practice. EVIDENCE GA-1.A. Pre-site: Please submit a copy of the programs written statement of purpose. On-site: Interview member of organizing group, program manager. EVIDENCE GA-1.B. Pre-site: Please indicate how often the written statement of purpose is formally reviewed and by whom. On-site: Interview member of advisory/governing group, program manager. GA-1.B. RATING INDICATORS 3 The programs advisory/governing body reviews its written statement of purpose every four years. 2 The program reviews its written statement of purpose every four years; however, the review is less formal or rigorous. 201

PAGE 208

Appendix C Continued Fidelity and Innovations 1 The program does not review its written statement of purpose at least every four years. GA-2. The program has broadly-based, advisory/governing group (e.g., a voluntary Board, governing body, an advisory committee, etc.) which serves in a advisory and/or governing capacity in the planning, implementation, and assessment of program services. GA-2.A. The programs advisory/governing group is an effectively organized, active body carrying out the functions specified in GA-2. GA-2.A. RATING INDICATORS 3 The programs advisory/governing group is an effectively organized, active body which carries out the activities of planning, implementation, and assessment of program services. 2 The programs advisory/governing group carries out the specified functions, but could be more active in one area of functioning. 1 The programs advisory/governing group is not active or is ineffective in one or more of the major areas of its responsibilities. GA-2.B. The advisory/governing group has a wide range of needed skills and abilities and provides a heterogeneous mix in terms of skills, strengths, community knowledge, professions, age, race, sex, nationality or ethnicity. GA-2.B. RATING INDICATORS EVIDENCE GA-2.A. Pre-site: Please submit a copy of the advisory/governing groups operating procedures or by-laws. On-site: Interview member of advisory/governing group, program manager, and review meetin g minutes. EVIDENCE GA-2.B. and GA-2.C. Pre-site: Please submit a profile of each member of the advisory/governing group (the affiliation, position in his/her organization, length of time on the board, and a brief summary of experience and/or reason leading to membership in the group). On-site: Interview members of advisory/governing group, program manager. 3 The advisory/governing group has a wide range of skills, abilities, and provides a heterogeneous mix in terms of skills, strengths, community knowledge, professions, age, race, sex, nationality or ethnicity. 2 The advisory/governing groups membership has some of the representative skills, knowledge and interests necessary to represent the community. 1 The advisory/governing groups membership does not represent the skills, knowledge and interests of the population it serves or is not diverse. 202

PAGE 209

Appendix C Continued Fidelity and Innovations GA-2.C. The advisory/governing group is aware of community issues that affect program participants, program planning, implementation, and assessment, either through direct representation by community members/program participants or another effective alternative. GA-2.C. RATING INDICATORS 3 The advisory/governing group is aware of community issues as they affect program participants, program planning, implementation and assessment through direct representation by community members/program participants in the group. 2 The advisory/governing group is aware of community issues as they affect program participants, program planning, implementation and assessment. However, the representation is not direct. 1 The advisory/governing group does not serve as an effective link with the community. GA-3. The program has a mechanism in place for families (i.e., past or present participants) to provide formalized input into the program. GA-3. RATING INDICATORS 3 The program has formal and informal mechanisms for participants to provide input into the program. Mechanisms used by the program include at least two of the following: participant satisfaction surveys, anecdotal reports, participant service on advisory/governing group/family advisory committee, participant feedback through focus groups, etc. 2 The program has mechanisms for participants to provide input to the program and at least include participant satisfaction surveys. 1 There are no means for participants to have input into the program. GA-4. The manager (or other program representative) and the advisory/governing group work as an effective team with information, coordination, staffing, and assistance provided by the manager to plan and develop program policy. EVIDENCE GA-3. Pre-site: By-laws/operating procedures,; profile of advisory/governing group members (as submitted in GA-2.A.);, narrative describing how program obtains input into program from participants (i.e., through participant surveys, serving on organizing group, serving on family advisory committee, etc.) On-site: Interview family members and/or other families who have provided input into program (formally by serving on the advisory/governing group or informally); interview program manager. 203

PAGE 210

Appendix C Continued Fidelity and Innovations GA-4. RATING INDICATORS 3 The manager (or other program representative) and the advisory/governing group work as an effective team in planning and developing program policy. 2 The manager (or other program representative) and the advisory/governing group plan and consult with one another, but the organizing group is not fully involved in the decision-making process. 1 The program and the advisory/governing group do not work as a team. GA-5. The program monitors and evaluates quality of services. GA-5.A. The program routinely reviews the progress towards its program goals and objectives. GA-5.A. RATING INDICATORS 3 No 3 rating indicator for standard GA-5.A. 2 The program conducts an analysis of program goals and objectives at least annually. 1 The program does not conduct analysis of program goals and objectives or it is not conducted on an annual basis. GA-5.B. The program reviews participant grievances. GA-5.B. RATING INDICATORS EVIDENCE GA-4. Pre-site: Please submit a copy of the programs written policies, guidelines, or narrative describing the role of the program manager (or other program representative) and the advisory/governing group. On-site: Interview program manager (or other program representative) and member of advisory/governing group. EVIDENCE GA-5.A. Pre-site: Please submit a copy of the programs work plan and annual report, evaluation reports, etc. On-site: Interview staff, supervisors, program manager re: analysis program goals and objectives. EVIDENCE GA-5.B. Pre-site: Please submit a copy of the programs grievance policy and follow-up procedures. On-site: Interview staff, supervisors, program manager and review documentation of procedures being followed (if applicable). 204

PAGE 211

Appendix C Continued Fidelity and Innovations 205 3 No 3 rating indicator for standard GA-5.B. 2 The program has a participant grievance policy and in cases where a grievance has occurred the policy has been followed. 1 The program does not have a policy or in cases where there has been a grievance the policy has not been followed. GA-5.C. The program regularly conducts an analysis of participant satisfaction with services. GA-5.C. RATING INDICATORS 3 No rating indicator for standard GA-5.C. 2 The program analyzes participant satisfaction with services at least annually. 1 The program does not conduct analysis of participant satisfaction or it is not conducted on an annual basis. GA-5.D. The program has a formal mechanism for reviewing the quality of all aspects of the program (assessment, home visitation and supervision). GA-5.D. RATING INDICATORS 3 The program has a mechanism for reviewing the quality of its program and is inclusive of all service areas. 2 The review is not as comprehensive as it could be. 1 The program either does not have a mechanism for reviewing the quality of its program; the mechanism for review does not include all service components (assessment, home visitation and supervision); or is not comprehensive. GA-5.E. The program has a follow-up mechanism to address areas for improvement identified during quality assurance review. EVIDENCE GA-5.C. Pre-site: Please submit a copy of the programs participant satisfaction survey and an analysis of its results. On-site: Interview staff, supervisors, and program manager. EVIDENCE GA-5.D. Pre-site: Please submit a copy of the programs quality assurance plan and copies of all data tracking forms that support this plan. On-site: Interview staff, supervisors, program manager and any existing internal quality assurance committee re: monitoring and evaluation mechanisms. Evidence may include shadowing, participant file reviews, supervision reviews, reviews of evaluation results, etc.

PAGE 212

Appendix C Continued Fidelity and Innovations 206 GA-5.E. RATING INDICATORS 3 The program has and uses its follow up mechanism to address areas for improvement identified during quality assurance review. 2 The program has and uses its follow up mechanism to address areas for improvement identified during quality assurance review. Isolated instances were found when problems identified during quality assurance review were not addressed. 1 The program either does not have and/or does not routinely use its follow up mechanism to address areas for improvement identified during quality assurance review. GA-6. The program has a policy and procedure for reviewing and recommending approval or denial of research proposals, whether internal or external, that involve past or present participants. GA-6. RATING INDICATORS 3 The program (or the oversight body) has a formal mechanism, such as a written procedure for review and action on any research proposal involving past or present participants or participant information. There are no exceptions. 2 There is evidence of the formal mechanism being used in all cases, but a mechanism to prevent a possible, inadvertent delay or omission is not in place. 1 The program (or oversight body) responds to proposal requests on a case-by-case basis with few established guidelines or procedures to guide the process and ensure that activities are consistent with or at least not in conflict with the agencys mission and mandates, and that participants rights are protected. Or, there is no formal program review of research proposals. GA-7. The program assures participant privacy and voluntary choice with regard to research conducted by or in cooperation with the program. EVIDENCE GA-5.E. Pre-site: Please submit a copy of the programs quality assurance plan and copies of all data tracking forms which support this plan. On-site: Interview staff, supervisors, program manager and any existing internal quality assurance committee re: monitoring and evaluation mechanisms. EVIDENCE GA-6. Pre-site: Please submit a copy of the programs policy and procedures regarding approval/denial of research proposals. On-site: Interview staff and others who have participated in this procedure.

PAGE 213

Appendix C Continued Fidelity and Innovations 207 GA-7. RATING INDICATORS 3 The program has procedures that it follows to carefully protect participant identity and privacy throughout any research project conducted by or with the cooperation of the agency, as well as those that assure voluntary informed consent without pressure to participate. 2 The programs procedures are very general and do not address all contingencies although past and current practice show no evidence of violation of client privacy or voluntary participation. 1 The agency has general procedures but is lax in their enforcement; the agency has no procedures, individual researchers follow their own plans, and potential for disclosure of identity or violation of privacy is high; or clients are not provided an opportunity to refuse disclosure. GA-8. Program reports suspected cases of child abuse and neglect. GA-8.A. Program has clear criteria through which to identify suspected cases of child abuse and neglect. GA-8.A. RATING INDICATORS 3 No for rating indicator standard GA-8.A. 2 Program has clear criteria to identify which cases of suspected child abuse/neglect should be reported and follows these criteria routinely. 1 Program does not have criteria to identify and report cases of suspected child abuse/neglect or, program does not routinely use criteria. GA-8.B.Programs reporting procedure regarding reporting of suspected cases of child abuse and neglect specifies immediate notification of the program supervisor and/or program manager and are in compliance with all applicable laws and regulations. Other appropriate staff/supervisors within the program are notified as needed. EVIDENCE GA-7. Pre-site: Please submit a copy of the programs policy (or guidelines) regarding protecting participant privacy and voluntary choice and copies of any relevant forms used to document this. On-site: Interview staff, program researcher, and participants in the research. EVIDENCE GA-8.A. Pre-site: Please submit a copy of the programs criteria used to identify suspected cases of child abuse and neglect. On-site: Interview staff and supervisors.

PAGE 214

Appendix C Continued Fidelity and Innovations 208 G A-8.B. RATING INDICATORS 3 No rating indicator for standard GA-8.B. 2 Programs reporting procedure specifies immediate notification of program manager and supervisor and is in compliance with all applicable laws and regulations. Other appropriate staff/supervisors within the program are notified as needed. 1 Programs reporting procedure does not specify immediate notification of program manager and supervisor or the procedures are not in compliance with all applicable laws and regulations. GA-8.C.Program follows its procedure regarding reporting of suspected cases of child abuse and neglect. GA-8.C. RATING INDICATORS 3 No "3" rating indicator for standard GA-5.C. 2 There is evidence to indicate that procedures are followed in cases where child abuse or neglect are suspected. 1 Insufficient evidence exists to indicate procedures are followed in cases where child abuse or neglect are suspected. GA-9. Program has an internal reporting procedure for reporting participant (especially child) deaths that occur while the participant is in the program. GA-9.A.Program has a procedure that specifies immediate notification of the program supervisor and/or program manager in cases of participant deaths. Other appropriate staff/supervisors within the program are notified as needed. EVIDENCE GA-8.B. Pre-site: Please submit a copy of the programs policy and procedures for reporting suspected cases of child abuse and neglect and indicate whether the programs policies and procedures are in compliance with all applicable laws. NOTE: an attorneys opinion, in writing, is sufficient evidence. On-site: Interview program manager and his/her supervisor. EVIDENCE GA-8.C. On-site: Interview program manager and his/her supervisor. EVIDENCE GA-9.A. and GA-9.B. Pre-site: Please submit a copy of the programs policy and procedures for reporting deaths of participants. On-site: Interview program staff, manager and his/her supervisor.

PAGE 215

Appendix C Continued Fidelity and Innovations 209 GA-9.A. RATING INDICATORS 3 No "3" rating indicator for standard GA-9.A. 2 Programs reporting procedure specifies immediate notification of the program supervisor and/or program manager. Evidence indicates procedures are followed (when applicable). 1 Programs reporting procedure does not specify immediate notification of program manager and supervisor. Evidence indicates procedures are not followed (when applicable). GA-9.B.Procedure ensures that staff receive crisis/grief counseling, as needed. GA-9.B. RATING INDICATORS 3 No "3" rating indicator for standard GA-6.B. 2 Procedure ensures that staff receive crisis/grief counseling, as needed. 1 Procedure does not ensure that staff receive crisis/grief counseling, as needed. GA-10. The program has a written budget and monitors expenditures to manage financial resources and support program activities for the program. G A G A-10. RATING INDICATORS 3 The program has a detailed written budget and it is used to monitor and manage expenditures for program activities during the year. 2 A written budget is used, but the monitoring and management of fiscal resources could be clearer. 1 Program written budget is weakened by the lack of detail or clarity; there is no written budget; the budget is not monitored in order to manage fiscal resources for program activities during the year. GA-11. The budget is reviewed and approved by a group (other than program manager) prior to the beginning of the fiscal year. EVIDENCE GA-10. Pre-site: Please submit a copy of the programs written budget and description of procedures to monitor expenditures in support of program activities. Ons ite: Interview p ro g ram mana g er and his/her su p ervisor. EVIDENCE GA-11. Pre-site: Please submit a narrative describing the approval mechanism for program budget and a list of individuals/groups who participate in the review. On-site: Interview those indicated as participating in the approval of the budget.

PAGE 216

Appendix C Continued Fidelity and Innovations 210 GA-11. RATING INDICATORS 3 The advisory/governing group, or the appropriate authority vested with that responsibility, reviews and approves the budget prior to the beginning of the fiscal year. 2 The advisory/governing group (or the appropriate authority vested with the responsibility) reviews the budget before the beginning of the fiscal year, but approval has been delayed. 1 The advisory/governing group approves the budget after the fiscal year has commenced; there is evidence that review has occurred, but no approval is documented. GA-12. The program seeks diversification and balance in its sources of funding. GA-12. RATING INDICATORS 3 The program seeks diversification (e.g., fee for service arrangements, third party reimbursement, use of federated or other funding sources as appropriate to program structure and mission) and balance in its sources of funding to avoid excessive dependence on any one major funding source that, if terminated, could result in abrupt cessation of service to clients. 2 The program, despite active efforts to diversify or strengthen resources, is dependent on one major source or is under-funded because of matters beyond its control (as in legislative constraints). 1 Program has made minimal efforts to expand, diversify or strengthen its resource base; program and services suffer because of abrupt termination or withdrawal of funding; program has no "fall back" position and has made no effort to protect itself from the consequences of excessive dependence on one source of funding. GA-13. The program (or programs sponsoring agency) makes available to the community an annual report or fiscal, statistical, and service data regarding the program. GA-13. RATING INDICATORS 3 No "3" rating indicator for standard GA-13. EVIDENCE GA-12. Pre-site: Describe programs fund development efforts and its procedures for presenting need to bodies that provide its funding. On-site: Interview program manager and/or development staff. EVIDENCE GA-13 Pre-site: Please submit a copy of the programs most recent annual report or review the data the program makes available to the community about its fiscal, statistical, and service information. Please describe how the program disseminates this information to the community. On-site: Check availability of report(s) to community through interviewing advisory/governance group members, participants, staff and supervisors.

PAGE 217

Appendix C Continued Fidelity and Innovations 211 2 On an annual basis, the program (or the programs sponsoring agency) produces an annual report or makes available to the community fiscal, statistical, and service data regarding the program. 1 The program (or the programs sponsoring agency) does not produce an annual report or fiscal, statistical, and service data regarding the program or the report is not made available to the community. GA-14. The program (or the programs sponsoring agency) is audited annually by an independent certified public accountant approved by the governing body. G A -14. RATING INDICATORS 3 No "3" rating indicator for standard GA-14. 2 The program (or the programs sponsoring agency) is audited annually by an independent certified public accountant approved by the advisory/governing group. 1 The program (or the programs sponsoring agency) is not audited annually by an independent certified public accountant approved by the advisory/governing group. End of Self Assessment Tool EVIDENCE GA-14. Pre-site: Please submit a copy of the programs (or host agencys) most recent audit. On-site: Interview fiscal staff, program manager, and his/her supervisor.

PAGE 218

Fidelity and Innovations 212 Appendix D Figures 545352515049484746454443424140393837363534333231302928272625242322212019181716151413121110987654321 Factor Number 20151050 Eigenvalue Figure 1. Scree Plot of Eigenvalues.

PAGE 219

Appendix D Continued Fidelity and Innovations 213 1.000.800.600.400.200.00 Participant Retention Rate 14121086420 Frequency Mean =0.7098Std. Dev. =0.21152N =91 Figure 2. Histogram of the Participant Retention Rate variable.

PAGE 220

Appendix D Continued Fidelity and Innovations 100.0080.0060.0040.0020.000.00 Percentage of Participants with a Primary Medical Provider 100806040200 Frequency Mean =96.535Std. Dev. =10.06177N =102 Figure 3. Histogram of the Percentage of Participants with a Primary Medical Provider. 214

PAGE 221

Appendix D Continued Fidelity and Innovations 215 100.00 80.00 60.00 40.00 20.00 0.00 Percentage of Participants with Updated Immunizations 40 30 20 10 0 Frequency Mean =81.9282 Std. Dev. =20.25701 N =103 Figure 4 Histogram of the Percentage of Participants with Updated Immunizations.


xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam Ka
controlfield tag 001 001914858
003 fts
005 20071029123327.0
006 m||||e|||d||||||||
007 cr mnu|||uuuuu
008 071029s2007 flu sbm 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0001988
040
FHM
c FHM
035
(OCoLC)179622978
049
FHMM
090
BF121 (ONLINE)
1 100
Nixon, Ashley E.
4 245
The impact of fidelity and innovations on Healthy Families America programs
h [electronic resource] /
by Ashley E. Nixon.
260
[Tampa, Fla.] :
b University of South Florida,
2007.
3 520
ABSTRACT: Fidelity to a program model and innovations added to the program model have been found to positively impact large scale program expansion and implementation (Blakely et al., 1987). Research examining the effectiveness of Healthy Families America (HFA) programs has been hindered by the differences that exist among HFA programs. This study examines the impact program fidelity and innovations have on outcome measures of updated immunizations, primary medical care providers, and participant retention for 102 HFA sites. Factor analysis and regression were used to analyze archival data. Results indicate that fidelity was positively related to percentage of participants with updated immunizations and primary medical care providers. Innovations, specifically additional services offered to participants and additional training opportunities for HFA staff, had positive relationships with participant retention.
502
Thesis (M.A.)--University of South Florida, 2007.
504
Includes bibliographical references.
516
Text (Electronic thesis) in PDF format.
538
System requirements: World Wide Web browser and PDF reader.
Mode of access: World Wide Web.
500
Title from PDF of title page.
Document formatted into pages; contains 215 pages.
590
Advisor: Carnot E. Nelson, Ph.D.
653
Program implementation.
Organizational expansion.
Multi-site programs.
Organizational effectiveness.
Home visiting programs.
0 690
Dissertations, Academic
z USF
x Psychology
Masters.
773
t USF Electronic Theses and Dissertations.
856
u http://digital.lib.usf.edu/?e14.1988