xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam 2200385Ka 4500
controlfield tag 001 002069424
007 cr bnu|||uuuuu
008 100422s2009 flu s 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0003239
Matos, Paula A.
Using staff management procedures to improve staff adherence with a toileting program at an intermediate care facility for individuals with developmental disabilities
h [electronic resource] /
by Paula A. Matos.
[Tampa, Fla] :
b University of South Florida,
Title from PDF of title page.
Document formatted into pages; contains 31 pages.
Thesis (M.A.)--University of South Florida, 2009.
Includes bibliographical references.
Text (Electronic thesis) in PDF format.
ABSTRACT: The purpose of this study was to evaluate feedback systems for improving staff adherence to a toileting program at an intermediate care facility for individuals with developmental disabilities. The facility uses interoffice memos in order to provide their staff with feedback. Following baseline, we evaluated the effectiveness of the memo (B), memo plus in-service (C), and memo plus feedback (D) in an ABCDBD reversal design across 4 residences. The results showed that memo plus feedback was the most effective intervention.
Mode of access: World Wide Web.
System requirements: World Wide Web browser and PDF reader.
Advisor: Raymond G. Miltenberger, Ph.D.
x Mental Health Studies
t USF Electronic Theses and Dissertations.
Using Staff Management Procedures to Improve Staff Adherence with a Toileting Program at an Intermediate Care Facility for Individuals with Developmental Disabilities by Paula A. Matos A thesis submitted in partial fulfillment of the requi rements for the degree of Master of Arts Department of School of Mental Health Studies College of Behavioral and Community Science University of South Florida Major Professor: Raymond G. Miltenberger, Ph.D. Timothy Weil, Ph.D. Lawrence Schonfeld, Ph.D. Date of Approval: November 13, 2009 Keywords: Supervision, Feedback, Compliance, Performance Management, Reactivity Copyright 2009, Paula A. Matos
Acknowledgements I would like to thank the members of my committee for their constant fee dback and guidance throughout this study, as well as, the staff of Hillsborough County Developmental Center for their assistance and participation
i Table of Contents List of Figures ii Abstract iii Chapter One: Introduction 1 Chapter Two: Method 6 Subjects and Setting 6 Materials 6 Target Behaviors 7 Data Collection 7 Interobserver Agreement 8 Experimental Design 9 Procedures 9 Baseline 9 Memo 9 Memo + Inservice 10 Memo + Feedback 10 Chapter Three: Results 12 House 1 12 House 2 1 3 House 3 1 5 House 4 1 7 Chapter Four: Discussion 21 List of References 2 7 Appendices 2 9 Appendix A : Toileting Data Sheet 30 Appendix B: Staff Adherence Data Sheet 31
ii List of Figures Figure 1 Results from the intervention package for Hous e 1 1 4 Figure 2 Results from the intervention package for House 2 1 6 Figure 3 Results from the intervention package for House 3 1 8 Figure 4 Results from the intervention package for House 4 19
iii Using Staff Management Procedures to Improve Staff Ad herence with a Toileting Program at an Intermediate Care Facility for Individuals with Developmental Disabilities Paula A. Matos ABSTRACT The purpose of this study was to evaluate feedback systems for improving staff adherence to a toileting program at an intermediate care facility for individuals with developmental disabilities. The facility uses interoffice memos in order to provide their staff with feedback. Following baseline, we evaluated the effectiveness of the memo (B) memo plus in service (C) and memo plus feedback (D) in an ABCDBD reversal design across 4 residences. The results showed that memo plus feedback was the most effective intervention.
1 Chapter One Introduction Data collection and accurate treatment implementation are i mportant p art s of applied behavior analysis practice. In an institution or other residential or habilitation setting, most decisions are based on data. These decisions can effect the institution or agency at large, for example decisions with regards to the instituti Lerman, Smith, Iwata, & DeLeon, 1995). Health and other service providers make treatment decisions regarding their clients based on the data that are collected (Parsons, Rollyson, & Reid, 2004). Therefore, it is extremely important to receive data that are legible, accurate, and complete. Data are collected by the individuals responsible for implementing treatment. In long term care facilities, suc h as intermediate care facilities or group homes, these individuals are usually direct care staff members. No treatment can be effectively implemented without competently trained staff and accurate data collection. Although data collection and treatment im plementation are part of the job responsibilities of a direct care staff member, they do not always get done or done correctly (Paterson & Allega, 1999). A number of variables can influence whether a direct care staff member completes all of his or her res ponsibilities. These factors can include high turn over rates, under staffing, heavy work loads, and the nature of supervision received by staff
2 (Macpherson, Eastley, Richards, & Mian, 1994). To increase job performance, research has suggested that agencie s incorporate the use of staff management approaches such as prompt ing supervisory feedback, and self monitoring (Burgio et al 1990; Harchik, Sherman, Sheldon, & Strouse, 1992). Staff management approaches have been used successfully with a variety of beh aviors and populations (Kern Dunlap et al, 1992). Self monitoring, prompt ing and supervisor feedback are only a few types of the behavior management procedures that have been evaluated to increase performance of desired behaviors and decrease performance of undesired behaviors by staff in institutions and community settings (Brackett, Reid, & Green, 2007; Burgio et al., 1990; Reid, Green, & Parsons, 2003). Self monitoring can help improve behavior while reducing the amount of supervision that may be requir ed ; (Brackett et al., 2007; Kern Dunlap et al, 1992) while prompt ing can be used as reminders for the staff to engage in the behavior at the correct time and feedback can be used to reinforce appropriate staff behavior and reduce incorrect staff behavior. (Austin, Weatherly, & Gravina, 2005; Richman, Riordan, Reiss, Pyles, & Bailey, 1988) and numerous published studies have shown performance feedback to be effective at pro ducing behavior change (Austin et al., 2005; Crowell, Anderson, Abel, & Sergio, 1988; Dib & Sturmey, 2007; Kern Dunlap et al, 1992; Richman et al., 1988). In fact research has shown performance feedback to be so effective that some researchers suggest that feedback is a critical component of programs to increase performance (Arco & du Toit, 2006). However, how much feedback is necessary for behavior to change and how feedback should be delivered are still under debate.
3 Performance feedback ensures that i ndividuals are competently trained (Arco & du Toit, 2006). Feedback contributes to competency by giving staff the necessary information to succeed. Positive feedback will tell staff in which areas they are performing well, while corrective feedback will in form staff what improvements need to be made. The most effective feedback consists of both positive and corrective information Although, feedback is usually given as an evaluation of performance, it is important for staff to receive feedback during and af ter training. Performance feedback needs to be specific and can be delivered in many ways. It can be tailored to an individual or group, it can be given orally or in writing, and it can be given immediately after the behavior or delayed. Although some res earch suggests that feedback is most effective when it is specific and delivered immediately after a skill has been demonstrated correctly or incorrectly or after an opportunity for the skill has been missed (Daniels & Daniels, 2004; Jahr, 1998), there is some evidence that delayed feedback is also effective (Kern Dunlap et al. 1992). No matter how it is delivered, once competency is achieved, feedback should be faded so staff members are performing correctly with a reduced level of feedback (Arco & du Toit 2006). As staff members increase their competency level, feedback should be faded to a lower level because a lower level of feedback requires a lower level of supervision (Arco & du Toit, 2006; Kern Dunlap et al., 1992). Although some amount of supervis ion will always be warranted, if the amount of time spent on supervision can be decreased, supervisors will be more likely to carry out their supervision responsibilities (Arco & du Toit, 2006; Brackett et al, 2007). Decreased supervision is seen as desira ble because it
4 would be impractical and inefficient to have a supervisor observe a staff member for the entire work day. The reasons behind the effectiveness of feedback are varied. Research has suggested that the effectiveness of feedback may be due to t he reinforcing effects of praise, the social recognition staff receive when feedback is given (Crowell et al., 1988), or the fact that rule governed behavior may be formed and reinforcement is received upon following those rules (Kern Dunlap et al., 1992). Whatever the reason behind its effectiveness, there is no questioning that feedback is effective at increasing performance. In addition to establishing the effectiveness of feedback, some research has also shown that feedback is a more powerful procedure than antecedent approaches such as the provision of inservice training. In 2006 Mozingo, Smith, Riordan, Reiss, and Bailey found that a brief in service with staff was not enough to improve the reliability of the data that staff members were collecting. I n fact, improvements were only seen for 2 of the 8 participants. To improve data reliability of all of the participants the authors added supervisory feedback. Performance feedback from the supervisor increased the reliability of data for all of the partic presence alone, after feedback was given, was enough to maintain high levels of reliable data collection. Although feedback may improve staff performance, feedback requires a ence and the effects of feedback may come under the stimulus control of e ry, Miltenberger, & Weil, in press). Mowery et al. showed that staff positive interactions increased when staff received feedback but that the increase w
5 (2007) evaluated the effects of reactivity on staff performance. In particular, the authors observed whether the job coach completed the work task of a supported worker or if they allowed the supported worker to complete the task independently. This study involved both conspicuous and inconspicuous observations and self recording. Results of the study showed that when job coaches were aware that they were being observed, they allowe d the supported worker to independently complete his or her work task (the desired behavior). However, when the job coaches were unaware that they were being observed, they completed most of the work tasks for the supported worker. These findings along wit h the results of Mow e ry et al. demonstrated that, in addition to the positive effects of feedback for improving staff performance, the presence of the supervisor can exert The primary purpose of this study was to e valuate feedback systems for improving staff adherence to a toileting program at an intermediate care facility (ICF) for individuals with developmental disabilities. Prior to this study th e ICF g a ve feedback to its staff and management via memos. It was hy pothesized that the memos were not very effective at changing behavior and maintaining behavior change (e. g., Mozingo et al., 2006; Petscher & bailey, 2006). Therefore, the secondary purpose of this study was to introduce and evaluate an intervention pack age consisting of prompt ing and feedback the study was to evaluate the effects of supervisor presence on staff behavior to see if the effects of feedback persisted when the supervisor wa s present but no longer providing feedback.
6 Chapter Two Methods Subjects and Setting This study was conducted in an intermediate care facility for individuals with developmental disabilities. The facility is made up of one cent ral auditorium and four residential houses. The facility has the capacity to house 64 residents with 16 residents living in each house. During the time of this study, there were 63 residents living in the facility There were also 1 4 residents per house p articipating in the toileting program. The residents participating in the toileting program wore underwear and not depends at the time of the study. Participants included 4 16 direct care staff members. All participants were at least 21 years of age, held at least a high school diploma or GED, and worked during the first and second shifts. During both shifts each house had at least 2 direct care members working. Materials The materials involved in this study included: data sheets, clipboards, and binders. Prior to the beginning of this study, a data sheet was specifically created for staff to use with the toileting procedure (see appendix A for a copy of this data sheet). This data sheet was created for several reasons. Its first and primary purpose was to collect toileting data This being so, the staff were aware that the toileting data sheets would be collected and
7 analyzed, and feedback would be given to management and staff members. A secondary purpose for the toileting data sheet was to serve as both a prompt for the staff to run the toileting program and to serve as a self monitoring form for the staff. A second data sheet was created by the experimenter to assess st appendix B). Clipboards and binders were used by both the staff and experimenters to hold data sheets and record data. Target Behaviors Three target behaviors were selected for this study. The first two rela ted to implementation of the toileting program and the third related to data collection. Pant checks consisted of the staff members checking the residents every 30 minutes for a toileting accident. This behavior involved staff looking to see if the residen ts were wet or Toileting prompts consisted of the staff prompting the resident to use the toilet every hour. Prompting the use of the toilet consisted of either asking the resid ent if he/she needs to use the toilet and/or taking the resident to the toilet. The final target behavior, documentation was for staff members to document on the data sheet every time they conducted a pant check and/or prompted the resident to use the toi let. Data Collection First and second shift staff members were required to run the toileting program and document throughout their entire shift. Therefore, probes were conducted at random times between the hours of 6:00 am and 8:00 pm. Probes were used t o assess whether staff members were completing all aspects of the toileting program (pant checks toileting prompts and documentation). Probes were at least an hour in length to capture multiple
8 opportunities for the behavior to occur, data were collected within 30 minute intervals for pant checks and 60 minute intervals for bathroom visits. The data collector recorded whether the behavior happened in each consecutive interval during the observation period and reported the behavior as the percentage of int ervals. The first author served as the data collector for this study. Data was collected in the common areas of each of the four behavioral intern at this ICF w as to conduct direct observations of residents; therefore the direct care staff members were under the impression that she was observing and collecting data on the residents, not the staff. Interobserver Agreement Interobserver agreement (IOA) was conduc ted during 33% of the probes by having a second observer independently observe and record the target behaviors. The independent observer in this study was a B oard C ertified B ehavior A nalyst that worked as a consultant in the facility. Interval by interval IOA w as calculated by dividing the total number of intervals of agreement on occurrence and nonoccurrence between the two observers by the total number of intervals of observation and multiplying by 100. An agreement was defined as both observers recording the occurrence or nonoccurrence of the behavior in the same interval. Mean IOA for House 1 was 9 2 % for pant check and toileting prompts and 99% for documentation. Mean IOA for House 2 was 98% for pant check and toileting prompts and 96% for documentation. Mean IOA for House 3 was 9 4 % for pant check and toileting prompts and 9 9 % for documentation. Mean IOA for House 4 was 9 3 % for pant check and toileting prompts and 99% for documentation.
9 Experimental Design This study used a multiple baseline across part icipants design with an ABCDBD sequence of phases (A=baseline, B=memo, C=memo + inservice, D=memo + feedback). Because there was no one staff member respon sible for the toileting program the participants were divided by the house they worked for. For exam ple, all of the data collected from House 1 were viewed as data for one participant. Procedures Months prior to the beginning of the study, the house manager ( QMRP ) explained to the staff that they were going to start a toileting program for the resident s and that it documentation. The toileting program consisted of the following responsibilities. First, pant checks occurred every 30 minutes. Secondly, residents were prompted t o go to the bathroom every hour. Lastly, the staff needed to record every pant check, toileting prompt and accident that occurred. Baseline. One week prior to the beginning of data collection, staff members were given the instruction to start running the toileting program on selected residents During base line staff members were expected to r u n the toileting program as they w ere instructed Th e program consisted of a pant check occurring every 30 minutes, a toileting prompt occurring every hour, and docum entation occurring every time these behaviors occurred. During this time the staff was not prompted to conduct the toileting program or prompted to document, and no feedback was given to the staff or QMRP. Memo. During the first phase of intervention, wee kly feedback was given to the QMRP via a standard interoffice memo. The memo consisted of data on observed
10 toileting patterns for the residents and corrective feedback for the staff, and asked the QMRP to relay the information to their staff. It is importa nt to note that the QMRP received feedback via an interoffice memo during every phase. The weekly memo contained information regarding toileting patterns for the residents. Memo + Inservice In the second phase of the intervention, feedback was given to b oth the QMRP and the staff members directly. The QMRP received the corrective feedback via their interoffice memo, just as she did in phase 1. However, during this phase the experimenter discussed the feedback with the QMRP during a brief meeting. Followin g the meeting, both the QMRP and the experimenter inserviced the staff on the toileting program. The inservice consisted of the same corrective feedback that was given to the QMRP during the prior meeting. During the inservice the experimenter and the QMRP reminded the staff that the toileting program needed to be run daily and tha t it consisted of three parts: f irst, pant checks need to occur every 30 minutes; s econd the residents need to be prompted/taken to the toilet every hour; and lastly documentati on needed to occur for every pant check and toileting trip/toileting prompt. Memo + Feedback The third phase of the intervention involved giving corrective feedback to the staff as well as giving feedback to the QMRP via the interoffice memo. The experim correct behavior if they ha d not done so at the appropriate time. For example, at 3:30 pm the exp erimenter asked staff members if they had checked the residents for toileting accidents (if the staff had not done so); at 4:00 pm the experimenter ask ed the staff if they had checked the residents for a toileting accidents and prompted him or her to use t he
11 toilet (if the staff member had not done so). The feedback was provided in this manner in order to make it less obvious that staff behavior was being observed and recorded and in an attempt to avoid reactivity of observation. In addition, to avoid react ivity, the researcher did not provide praise contingent on each instance of appropriate behavior. However, the researcher did make intermittent positive comments about staff appropriate behavior. Fo r example, the experimenter would tell a staff member oticed you asked experimenter kept track of whether the target behaviors were occurring and whether corrective feedback was given. Staff was given feedback at the end of ev ery 30 minutes for pant checks and at every hour for toileting prompts/trips. The fourth phase of the study was a return to the first phase of the intervention (memo). In other words, staff members were no longer given feedback regarding their adherence t o the toileting program and the only feedback given was received by the QMRP via an interoffice memo. This return to the first phase was conducted to assess the wa s withdrawn. The fifth and final phase of the study consisted of a reintroduction of the memo plus feedback phase. This phase was only conducted if the fourth phase resulted in low er levels of target behaviors than in the previous memo+ feedback phase
12 Chapter Three Results H ou se 1 Figure 1 displays the results from the intervention package for House 1. The first panel in figure 1 shows the results for adherence to pant checks and toileting prompts. The data in Figure 1 show that adherence was highest in the memo + feedback pha se, with a slight drop in the second memo phase, and a further increase in the second feedback phase. Throughout baseline there were 2 residents participating in the toileting program. During this time staff members engaged in very low levels of adherence with a mean of 6% (range of 0% to 20%). The first memo phase resulted in a slight increase in adherence, with a mean of 33% (range of 0% to 50%). During the first memo only phase there were 3 4 residents participating in the toileting program. The memo + inservice phase resulted in a similar level of adherence, with a mean of 36% (range of 10% to 50%). Throughout the memo + inservice phase and the first memo + feedback phase there were 3 participants in the toileting program. During the first memo + feedb ack phase, staff members engaged in higher levels of adherence with a mean of 71% (range of 33% to 100%); however, the mean for the second half of the phase was 73%. By removing the feedback, a return to the memo only phase was achieved. The reversal phase or the phase in which immediate feedback to the staff was withdrawn, resulted in a mean adherence level of 70% (range of 60% to 100%) although the level decreased in the
13 second half of the phase to 60%. When the memo plus feedback was re implemented, a q uick rise was seen in staff adherence levels with a mean of 84 % (range of 60 % to 100 %) T he reversal and the second memo + feedback phase both consisted of 1 resident participating in the toileting program. The second panel in Figure 1 shows the results f or adhering to documentation, with a mean of 4% (range of 0% to 20%) in baseline, 25% (range of 0% to 100%) in the memo phase, 8% (range of 0% to 20%) in the memo + in service phase, 2 1 % (range of 0% to 75%) in the memo + feedback phase, 8% (range of 0% to 80%) in the second memo phase, and 24 % (range of 0 % to 80 %) in the second memo + feedback phase House 2 Figure 2 displays the results from the intervention package for House 2. The first panel in figure 2 shows the results for adherence to pant checks a nd toileting prompts. The data in Figure 2 show that the level of adherence was highest in both of the memo + feedback phases with a slight decrease in the second memo phase that separated the two feedback phases. During baseline, staff members engaged in 0% adherence. The mean level of adherence was 15% (range of 0% to 67%) in the first memo phase, 18% (range of 0% to 60%) in the memo + inservice phase, 71% (range of 0% to 100%) in the first memo + feedback phase (although the mean was 75% in the second ha lf of the phase), 60% (range of 33% to 80%) in the reversal to the second memo phase, and 87 % (range of 80% to 100%) in the return to the memo + feedback phase. In house 2 there was 1 resident participating in the toileting program during the baseline, mem o, and memo + inservice phases, 1 2 residents participating in the toileting program during the first memo + feedback phase, and 1 resident participating during both the second memo only
14 Figure 1. Results from the intervention package for House 1. The first panel displays the percent of staff adherence during baseline and each of the intervention phases for pant checks and toileting prompts. The second panel displays the percent of staff adherence for documentation during each of the phases.
15 phase a nd the second memo + feedback phase. The second panel in Figure 2 shows the results for adhering to documentation. The means for adherence were 0% in baseline, 0% in the memo phase, 0% in the memo + inservice phase, 18% (range of 0% to 80%) in the memo + f eedback phase, 58% (range 0% to 100%) in the second memo phase and 36 % (range of 0% to 80%) in the final memo + feedback phase. House 3 Figure 3 displays the results from the intervention package for House 3. The first panel in figure 3 shows the results for adherence to pant checks and toileting prompts. The data in Figure 3 show that the level of adherence was highest in both of the memo + feedback phases with a decrease in adherence occurring in the second memo phase that separated the two feedback pha ses. Throughout baseline there was 1 resident participating in the toileting program. During this time, staff members engaged in very low levels of adherence with a mean of 27% (range of 0% to 100%). The first memo phase resulted in a slight increase in ad herence, with a mean of 37% (range of 0% to 60%), which quickly fell to 0% during the second half of the phase. During this phase there were 2 residents participating in the toileting program. The memo + inservice phase resulted in a similar level of adher ence, a slight increase followed by quick decreases in adherence. The memo + inservice phase had a mean of 16% (range of 0% to 60%) and 1 2 residents participating in the toileting program. During the first memo + feedback phase, staff members engaged in h igher levels of adherence. This phase resulted with a mean of 74% (range of 40% to 100%), although the mean for the second half of the phase was 93%. The reversal resulted in a mean of 66% (range of 40% to 80%). The
16 Figure 2. Results from the interv ention package for House 2. The first panel displays the percent of staff adherence during each of the phases for pant checks and toileting prompts. The second panel displays the percent of staff adherence for documentation during each of the phases.
17 r eimplementation of the memo + feedback resulted in a mean of 91% (range of 60% to 100%). The memo + feedback phase, second memo only phase, and the second memo + feedback phase all consisted of 1 resident participating in the toileting program. The second panel in Figure 3 shows the results for adhering to documentation. The means for adherence were 0% in baseline, 0% in the memo phase, 10% (range of 0% to 50%) in the memo + inservice phase, 4% (range of 0% to 40%) in the memo + feedback phase, 14% (range 0 % to 60%) in the second memo phase, and 16 % (range of 0% to 80%) in the final memo + feedback phase. House 4 Figure 4 displays the results from the intervention package for House 4. The first panel in figure 4 shows the results for adherence to pant check s and toileting prompts. The data in Figure 4 show that the level of adherence was highest in the memo plus feedback phase. During baseline, staff members engaged in 0% adherence. After intervention started the mean levels of adherence started to increase. The mean level of adherence was 28% (range of 0% to 67%) in the memo phase, 37% (range of 0% to 80%) in the memo + inservice phase, and 74% (range of 40% to 100%) in the memo + feedback phase (although the mean was 86% in the second half of the phase). In house 4 there was 1 resident participating in the toileting program during baseline and 2 residents participating in the toileting program throughout the rest of the phases: memo only, memo + inservice, and memo + feedback. The second panel in Figure 4 sh ows the results for adhering to documentation. The means for adherence were 0% in baseline, 11% (range of 0% to 50%) in the memo
18 Figure 3. Results from the intervention package for House 3. The first panel displays the percent of staff adherence dur ing each of the phases for pant checks and toileting prompts. The second panel displays the percent of staff adherence for documentation during each of the phases.
19 Figure 4. Results from the intervention package for House 4. The first panel displays the percent of staff adherence during each of the phases for pant checks and toileting prompts. The second panel displays the percent of staff adherence for documentation during each of the phases.
20 phase, 2% (range of 0% to 40%) in the memo + in service phase, and 5% (range of 0% to 40%) in the memo + feedback phase.
21 Chapter Four Discussion Overall the results of this study indicate that a staff management procedure involving feedback components increased staff adherence to a toileting program at an IC F. The results showed low levels of staff adherence during the following phases: baseline, the first memo only phase, and the memo + inservice phase. The memo + feedback phase resulted in the highest level of staff adherence for the two parts of the toilet ing program that received feedback: pant checks and toileting prompts. The third behavior, correct documentation, did not receive feedback and this behavior did not increase when feedback was delivered for only for pant checks and toileting prompts. It is important to note that even though some adherence was seen throughout all phases, staff adherence did not increase to any meaningful level until after feedback was implemented. Furthermore, only the behaviors receiving feedback increased during the feedbac k phase. This result is consistent with results f r om previous research that ha ve shown staff management procedures involving feedback to be effective (Burgio et al 1990; Kern Dunlap et al., 1992; Mowrey, Miltenberger, & Weil, in press; Mozingo et al., 2006 ; Petscher & bailey, 2006). As was hypothesized, the results also indicated that the memo alone was not sufficient to increase adherence to high levels There was an increase in adherence with the introduction of the first memo; however the increase was no t large and adherence
22 levels decreased shortly after. The memo + inservice proved to have a similar effect: increases in adherence were seen but still below 50%. In order for staff adherence to improve, it was necessary for staff members to receive feedbac k. As in previous studies, high levels of behavior change were only seen after feedback was given (Mozingo et al., 2006; Petscher & bailey, 2006). In order for staff management procedures to be valuable they must produce and maintain behavior change in the absence of supervision. This point was addressed with the withdrawal of feedback Even though percent of staff adherence dropped during the reversal, they remained higher than 50% and higher than the percentages seen during baseline, the first memo only, and memo + inservice phases Experimental control was demonstrated in two ways. First, e xperimental control was demonstrated by a reversal. Once high levels of staff adherence were obtained during the memo + feedback phase, feedback was removed and only t he memo was delivered. During this reversal staff adherence dropped to a slightly lower level than previous performance; however, performance remained at levels higher than baseline and at higher levels than the first memo phase When the memo + feedback p hase was re implemented adherence levels quickly rose again. This finding demonstrated that feedback was effective in increasing the adherence levels of staff members. Secondly, e xperimental control was demonstrated by showing that the behaviors targeted f or feedback (pant checks and toileting prompts) increased with feedback while the third behavior that was not targeted (documentation) did not increase with the feedback delivered to the other behaviors The case can be made that documentation was placed o n an extended baseline because although it received the memo phase and the memo + inservice phase it did not
23 receive immediate feedback. Therefore the only difference between these two components of the program was the manipulation of feedback. The differ ence in levels between behaviors that did and did not receive feedback demonstrated that immediate feedback was a critical component at increasing the adherence of staff. This study showed that a memo was sufficient to maintain behavior change well above baseline levels but only after direct feedback was given. In the first memo phase following baseline, the mean level of adherence for the toileting program was 2 8 % (range of 15% to 3 7 %) while in the second memo phase following feedback, the mean level of a dherence was 65% (range of 60% to 70%). It appears that the prior use of feedback made the memo more effective. Future research should focus on maintenance and generalization. For example, future researchers could ask the following questions: For how long will delayed feedback, delivered only in the form of a memo, be effective? In this study, the memo phase following feedback was too short to answer this question. Other questions that future researchers could ask are: For what other behaviors can feedback, in the form of a memo, be used and can it be effective in lieu of direct supervision? After direct feedback is given, can a memo be used to maintain a complex behavior such as implementing a program for reduction of problem behaviors? In this study, the t arget behaviors were not complex, involving pants check and bathroom prompts delivered on a time based schedule. Although the results of this study are promising, a few issues do arise. First, having the staff inserviced by the experimenter and the QMRP co uld have increased their perception of the importance of the task, and may, therefore, have made it more likely that they would adhere to the toileting program. However, this potential increase in
24 motivation to adhere to the toileting program seems unlikel y due to the low levels of adherence observed during the inservice phase. Secondly, staff members were never informed that data were going to be collected on their adherence. This could explain the low levels of documentation that was seen throughout the s tudy. However, documentation is part of the everyday job responsibilities and staff members were informed that the toileting data w ere going to be collected and analyzed. Finally, phase changes for this study were determined by staff adherence to pant chec k s and toileting prompts and not by staff adherence to documentation. It was decided by the researcher that staff adherence to pant checks and toileting prompts were the two more critical target behaviors and adherence to documentation served as a control condition A couple of limitations in this study should be noted. First, two phase changes were made at times when the behavior was trending upward (the change from Memo to Memo + Inservice for House 2 and the change from Baseline to Memo for House 3). I n each case phase changes should have occurred later after the data had stabilized in these phases. Although the phase change s were implementation as quickly as possible waiting until the data were no longer trending upward would have demonstrated better experimental control A second limitation in the study is that the second memo phases may have been too short to detect maintenance effects in the absence of feedback. Although these phases lasted from 8 to 3 0 sessions (3 to 10 weeks of data collection) and the data were stable in these phases, it is possible that the level of adherence may have decreased over a longer period of time without feedback. Future research should investigate this possibility. In thi s study the decision was made to return to the memo + feedback phase after the second memo phase in order to
25 demonstrate experimental control in a reversal design. A third limitation of this study is that House 4 did not have a second m emo and m emo + f eedb ack phase and, therefore, could not demonstrate experimental control in a reversal design. However, three other houses did complete the reversal and demonstrated experimental control Furthermore, experimental control was demonstrated in House 4 by showing that adherence to pant checks and toileting prompts increased when feedback was applied to these behaviors while adherence to documentation (which did not receive feedback) did not increase at the same time A fourth limitation of the study is that there were some extraneous variable s that could have accounted for the variability seen in the data. First there was no systematic control for when or how the QMRP relayed the information on the written memo to the staff. The manner in which the QRP delivered th e feedback from the memo could have influenced some of the data. However, there was consistency in the fact that the QMRP was relaying the feedback from the memo throughout the entire study Another important note is that although steps were taken to reduc e reactivity, reactivity was not evaluated. A third extraneous variable was that the experimenter did not control for the activities that were taking place during the times of observations or the staff that were present during the times of observations A final limitation is the absence of assessment of these potential extraneous variables to determine their influence on the target behaviors. In summary, this study demonstrated the effects of feedback on adherence to a toileting program by staff in four gr oup homes for individuals with developmental disabilities. The results showed that there was only a moderate increase in adherence with the use of a memo and a memo plus inservice, suggesting that these procedures are not
26 sufficient for producing meaningfu l staff behavior change. However, once immediate feedback was provided and adherence increased to high levels, a memo alone maintained a higher level of adherence than was seen in the prior memo or inservice phases. These results suggest that feedback on s taff performance is a critical component of effective staff management strategies.
27 List of References Arco, L., & du Toit, E. (2006). Effects of adding on the job feedback to conventional analog staff training in a nursing home. Behavior Modification 30, 713 735. Austin, J., Weatherly, N. L., & Gravina, N. E. (2005). Using task clarification, graphic feedback, and verbal feedback to increase closing task completion in a privately owned restaurant. Journal of Applied Behavior Analysis, 38, 117 12 0. Br ackett, L., Reid D. H., & Green, C. W. (2007). Effects of reactivity to observations on staff performance. Journal of Applied Behavior Analysis 40, 191 195. Burgio, L.D., Engel, B.T., Hawkins, A., McCormick, K., Scheve, A., & Jones, L.T. (1990). A staff management system for maintaining improvements in continence with elderly nursing home residents. Journal of Applied Behavior Analysis 23(1), 111 118. Crowell, C. R., Anderson, D. C., Abel, D. M., & Sergio, J. P. (1988). Task clarification, performance feedback, and social praise: Procedures for improving the customer service of bank tellers. Journal of Applied Behavior Analysis 21, 65 71. Daniels, A. C., & Daniels, J. E. 2004. Performance management: Changing behavior that drives organizational effec tiveness (4 th edition) Atlanta, GA: Aubrey Daniels International, Inc. implementation of discrete trial teaching. Journal of Applied Behavior Analysis 40, 339 343. Harchi ck, A. E., Sherman, J. A., Sheldon, J. B., & Strouse, M. C. (1992). Ongoing consultation as a method of improving performance of staff members in a group home. Journal of Applied Behavior Analysis, 25, 599 610. Jahr, E. (1998). Current issues in staff tra ining. Research in Developmental Disabilities, 19 73 87. Kern Dunlap, L., Dunlap, G., Clarke, S., Childs, K. E., White, R. L., & Stewart, M. P. (1992) Effects of a videotape feedback package on the peer interactions f children with serious behavioral and emotional challenges. Journal of Applied Behavior Analysis 25, 355 364.
28 MacPherson, R., Eastleey, R., Richards, H., & Mian, I. (1994). Psychological distress among workers caring for the elderly. International Journal of Geriatric Psychiatry, 9, 381 386. on staff response to tactile prompts and self monitoring in a group home setting. Behavioral Interventions. Mozingo, D. B., Smith, T., Riordan, M. R., Reiss, M. L., & Bailey, J. S. (2006). Enhancing frequency recording by developmental disabilities treatment staff. Journal of Applied Behavior Analysis 39 253 256. Parson, M. B., Rollyson, J. H., & Reid, D. H. (2004). Improving day treatment services for adults with s evere disabilities: a norm referenced application of outcome management. Journal of Applied Behavior Analysis 37, 365 377. Paterson, J. M. & Allega, R. L. (1999). Improving communication between hospital and community physicians feasibility study of a h andwritten, faxed hospital discharge summary. Canadian Family Physician 45, 2893 2899. Reid, D. H., Green, C. W., & Parsons, M. B. (2003). An outcome management program for extending advances in choice research into choice opportunities for supported work ers with severe multiple disabilities. Journal of Applied Behavior Analysis 36, 575 578. Richman, G. S., Riordan, M. R., Reiss, M. L., Pyles, D. A. M., & Bailey, J. S. (1988). The effects of self monitoring and supervisor feedback on staff performance in a residential setting. Journal of Applied Behavior Analysis 21, 401 409. Shore, B. A., Lerman, D. C., Smith, R. G., Iwata, B. A., & DeLeon, I. G. (1995). Direct assessment of quality of care in a geriatric nursing home. Journal of Applied Behavior Anal ysis 28, 435 448.
30 Appendix A: Toileting Data Sheet
31 Appendix B: Staff Adherence Data Sheet