xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam Ka
controlfield tag 001 002001012
007 cr mnu|||uuuuu
008 090427s2008 flu s 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0002533
Mowery, Judith M.
Effects of supervisor's presence on staff response to tactile prompts and self-monitoring in a group home setting
h [electronic resource] /
by Judith M. Mowery.
[Tampa, Fla] :
b University of South Florida,
Title from PDF of title page.
Document formatted into pages; contains 40 pages.
Thesis (M.A.)--University of South Florida, 2008.
Includes bibliographical references.
Text (Electronic thesis) in PDF format.
ABSTRACT: Staff management research in group home settings has involved direct observation of staff performance during pre-treatment and treatment conditions. Collecting accurate research data is crucial to analyze treatment effects; however, reactivity to being observed has been cited as a limitation in several studies. The current study evaluated the use of a tactile prompt, self-monitoring, and feedback to increase positive interaction in a group home setting. Direct support professional staff were trained on the purpose and use of the MotivAider which provided tactile prompts to remind them to engage in positive client interaction. Reactivity was assessed by having a confederate observe staff positive interaction when the supervisor was present and when the supervisor was absent. The effects of supervisor presence were evaluated using an alternating treatment within a multiple baseline across participants' research design. Results showed that 2 of 4 participants increased positive interactions only when a supervisor was present and 2 other participants increased positive interaction only after receiving feedback.
Mode of access: World Wide Web.
System requirements: World Wide Web browser and PDF reader.
Advisor: Raymond Miltenberger, Ph.D.
x Applied Behavior Analysis
t USF Electronic Theses and Dissertations.
Effects of Supervisor's Presence on Staff Response to Tactile Prompts and Self Monitoring in a Group Home Setting by Judy M. Mowery A thesis submitted in partial fulfillment of the requirements for the degree of Master of Arts in Applied Behav ior Analysis College of Graduate S chool University of South Florida Co Major Professor: Raymond Miltenberger, Ph.D. Co Major Professor: Timothy Weil, Ph.D. Kwang Sun Blair, Ph.D. Date of Approval: July 8, 2008 Keywords: staff management, reactivi ty, confederates, positive interaction, MotivAider Copyright 2008, Judy M. Mowery
i Table of Contents List of Figures ii Abstract iii Chapter One: Introduction 1 Chapter Two: Method 8 Participants and Se ttings 8 Materials 11 Target Behaviors and Data Collection 1 2 Interobserver Agreement 1 3 Social Validity Questionnaire 1 5 Experimental Design and Procedure 1 5 Baseline/Pre Training Assessment 1 6 Trainin g 1 6 Post Training Assessment 1 7 Supervisor Present + MotivAider On 17 Supervisor Absent + MotivAider Off 1 8 Supervisor Absent + MotivAider Off 1 8 Supervisor Present + MotivAider On + Feedback 1 8 Chapter Three: Results 1 9 Chapter Four: Discussion 2 6 References 3 2 Appendices Appendix A: Social Validity Questionnaire 3 9
ii List of Figures Figure 1. Results of staf f management package + feedback 20 Figure 2. Results of s taff management package 2 1
iii Effects of Supervisor's Presence on Staff Response to Tactile Prompts and Self Monitoring i n a Group Home Setting Judy M. Mow ery ABSTRACT Staff management research in group home settings has involved direct observation of staff performance during pre treatment and treatment conditions. Collecting accurate research data is crucial to analyze treatment effects; however, reactiv ity to being observed has been cited as a limitation in several studies. The current study evaluate d the use of a tactile prompt self monitoring, and feedback to increase positive interaction in a group home setting. Direct support professional staff we re trained on the purpose and use of the MotivAider which provided tactile prompts to remind them to engage in positive client interaction. Reactivity was assessed by having a confederate observe staff positive interaction when the supervisor wa s present and when the supervisor wa s absent The effects of supervisor presence were evaluated using an alternating treatment within a multiple baseline across participants' research design. Results showed that 2 of 4 participant s increased positive interactions only when a supervisor was present and 2 other participants increased positive interaction only after receiving feedback.
1 Chapter One Introduction Staff management has been an area of interest for many years. Studi es of staff management procedures have been conducted in residential settings ( Green, Reid, Perkins, & Gardner, 1991; Harchik et al., 2001; Harchik, S h erman, Sheldon, & Strouse, 1992; Mozingo, Smith, Riordan, Reiss, & Bailey, 2006 ; Parsons, Reid, & Green, 1996; Parsons, Reid, & Green, 1993; Richman, Rio r dan, Reiss, Pyles, & Bailey, 1988), vocational setting s (Fleming & Sulzer Azaroff, 1992), hospital setting s (Alavosius & Sulzer Azaroff, 1990), nursing homes (Acro & D u Toit, 2006; Burgio et al. 1990); clas srooms (Codding, Feinber g Dunn, & Pace, 20 05; De Pry & Sugai, 2002; Gross & Ekstrand, 1983; Reinke Lewis Palmer, & Martin, 2007; S c heeler & Lee, 2002), retail store s (Pampino, MacDonald, Mullin, & Wilder, 2003), banks (Crowell, Anderson, Abel, & Sergio, 1988), restaurants (Austin, Weatherly, & Gravina, 2005), family home (Harris, Peterson, Filliben, Glassberg, & Favell, 1998), community setting s (Cooper & Browder, 2001), state institutional settings ( Hutchinson, Jarman, & Bailey, 1980; Reid, Sc huh Wear, & Brannon, 1978; Wilson, Reid, & Korabek Pinkowski, 1991), and prison s (Bassett & Blanchard, 1977). Each has provided valuable information related to how staff performance can be improved and maintained beyond the typical classroom setting where initial tr aining is usually provided. Much of this research has been conducted in residential settings for persons with developmental disabilities (DD)
2 Residential group home settings remain a frequent placement for people with DD The staff in such settings prov ides skill acquisition training to increase independence and quality of life for people who require staff support to manage daily routines and schedules. Supports also revolve around inclusion on many levels (home, work, recreational) as well as coordinat ing a multitude of services available to help the person with DD achieve as much independence as possible. Staff interaction with residents with DD then is critically important to achieve the best outcomes with the residents Although frequent p ositive i nteraction and rapport building activities are often included in behavior support plans r esidential providers have reported difficulties in motivating their staff to increase interaction with the people they work with ( Gre en et al., 1991; Parsons, Cash, & Reid, 1989; Quilitch, 1975; Richman et al., 1988; Wilson, et al ., 2006) There is a clear need for strategies to improve staff performance in residential settings. Researchers have shown that a number of strategies may be successful for in creasing staff positive interactions with residents with DD. These strategies include performance feedback from supervisors (Arco & Du Toit, 2006; Austin et al., 2005; Codding et al., 2005; Cook & Dixon, 2005; Crowell, et al. 1988; Gross & Ekstrand, 1983; Hagermoser Sanetti, Luiselli, & Handler, 2007; Pampino et al., 2003; Reinke et al., 2007), self monitoring (Amato Zech, Hoff, & Doepke, 2006; Burgio et al., 1990; Doerner, Miltenberger, & Bakken, 1989; Harris, 1986; Petscher & Bail ey, 2006; Richman et al., 1988), and various prompting strategies, including the use of electronic devices (Amato Zech et al., 2006; Petscher & Bailey, 2006; Scheeler & Lee, 2002).
3 Performance feedback occurs when supervisors observe desired job related be haviors from staff and provide specific praise (and possibly other incentives) for the performance. For example, Codding et al. (2005) used performance feedback as an intervention in a classroom setting. Teachers worked with students with behavior support plans. The authors met with the teacher after class to review the components of the behavior plan implemented by the teachers and to provide feedback on all the components observed during the class. Results of this study show the intervention improved t reatment integrity and the results were maintained up to 15 weeks. Other researchers have evaluated performance feedback as an intervention with staff working with individuals with DD. For example, Doerner et al. (1989) and Suda and Miltenberger (1993) sh owed that staff positive interactions with group home residents and workshop participants increased when they received feedback on their performance. Research on the effects of performance feedback has shown it to be effective in other settings as well. Fo r example, Pampino et al. (2003) looked at weekly versus daily feedback (along with task clarification, goal setting, and access to preferred tangibles contingent on goal attainment) with staff in a retail framing and art store setting. In addition to find ing that the procedures were effective, they found that participants preferred daily feedback as opposed to weekly feedback. In another investigation, Arco and Du Toit (2006) evaluated the effects of feedback on the performance of staff in a nursing home s etting and found improved performance when feedback was part of a staff management intervention. In yet another setting, Austin et al. (2005) used task clarification, graphic feedback, and verbal feedback to improve completion of closing
4 tasks at a restau rant. Hagermoser Sanetti et al. (2007) found that a combination of verbal and graphic performance feedback increased treatment integrity of behavior support plans in a public elementary classroom setting. In one other investigation of feedback as a staf f management procedure, Crowell et al. (1988) showed that the use of task clarification, feedback, and praise as a staff management package increased customer relations target behaviors of bank tellers. These studies demonstrate the utility of performance feedback in a wide variety of different employment settings. Another strategy for promoting staff positive interactions with residents is the use of self monitoring. In self monitoring, staff observe and record their own behavior on the job as a way to i ncrease desirable behaviors. Self monitoring has been found to be a valuable staff management strategy in a number of studies. For example, s elf monitoring and supervisor feedback were the focus of a study conducted by Richman et al. (1988). The authors fo und that on task and on schedule staff behaviors substantially increased with the introduction of a self monitoring strategy using individualized shift schedule cards. Staff behaviors further increased when self monitoring was combined with supervisory fe edback. Doerner et al. (1989) found that the use of a goal setting, self monitoring, self evaluation, and self praise staff management package increased staff positive interaction with clients living in group homes. Across these studies, staff self monit oring has resulted in improved performance when used alone (Richman et al., 1988) or when used in conjunction with other procedures (Doerner et al., 1989). In addition to self monitoring and the use of performance feedback, another staff management strateg y is the use of prompts (Amato Zech et al., 2006; Petscher & Bailey,
5 2006; Scheeler & Lee, 2002). Recent research has evaluated the use of tactile prompts to cue the staff to perform a specific job task. For example, Petscher and Bailey (2006) conducted a study using a vibrating pager to prompt instructional staff in a classroom setting to implement a token economy. Another component to the intervention was the use of a self monitoring system to improve the behavior of staff. Results of this study show t he introduction of the tactile prompt followed by the introduction of self monitoring produced clear increases in target behaviors. Another tacti le prompting device is called the MotivAider. This device emits a pulsating vibration following a continuous o r fixed time schedule that is programmable to meet spe cific needs. Amato Zech et al. (2006) evaluated the MotivAider in an elementary school with three students. The study included the use of the MotivAider and a paper and pencil recording system to self monitor the students' on task behavior. When the MotivAider vibrated, the student s checked whether they were paying attention at that moment. The r esults show ed that on task behavior i ncrease d of 35% during intervention. There are no other studies conduc ted using vibrating prompts as a staff management procedure. Although only a few studies have evaluated tactile prompting, the results are promising. The use of a vibrating pager may be an effective way to promote successful performance by staff in a varie ty of settings. While a number of staff management strategies have been proven effective, a common limitation cited in many staff management studies involves the possible effect of reactivity as a variable influencing the outcome. That is to say, the impr ovement in staff
6 performance may be due to the presence of the supervisor or data collector rather than the staff management procedure itself (e.g., Bassett & Blanchard, 1977). Few s tudies have evaluated the effects of reactivity Brackett, Reid, and Gree n (2007) conducted a study in a vocational setting evaluating job coaches behavior when a supervisor was conducting conspicuous versus inconspicuous observations. The y found that the job coaches engaged in a higher percentage of correct behavior when the y knew they were being observed than when they did not know they were being observed. Because staff behavior was likely to have been influenced by conspicuous observation the authors implemented a self recording procedure to promote staff behavior when we re not being observed. The results showed that their performance increased when they engaged in self monitoring even when there was no conspicuous observation. The results of Bracket et al. clearly showed the effects of supervisor presence on staff perf ormance. This finding is potentially significant as all previous research has demonstrated the effectiveness of staff management procedures while a data collector or supervisor was present as a possible confounding variable. It is not known how well staff would have performed in the absence of an observer. Because the Bracket et al. study is the only experimental demonstration of the effects of supervisor presence on staff behavior, t he purpose of th e proposed study wa s to replicate the research conducted by Brackett et al. (2007) in a group home or cluster setting with individuals with DD This study evaluated the influence of a cost effective staff management package involving self monitoring and tactile prompts during supervisor present and supervisor a bsent conditions to evaluate the effectiveness of the procedures and the possible reactivity that
7 would result with the supervisor present. An additional condition involving supervisor feedback was implemented for participants who did not benefit from the staff management package.
8 Chapter Two Method Participants and Settings Four direct suppo rt professional staff were recruited from area group homes The group homes housed up from 6 to 8 persons with DD Staff to client ratios varied from 1:2 to 1:3 throughout the day. Staff participants were recruited based on their shifts so that training given to some staff did not influence other staff who had not yet received training. Consent to participate was obtained prior to the initiation of the study. Nancy was a 22 year old female with 1 year and 5 months experience working with people with DD She gained this experience working at her current place of employment. Nancy had a high school education. The group home she worked in had four bedrooms one wheelchair accessible bathroom, a living room, dining room, family/activities room, and a screened in outdoor patio/lanai area. There were five adults living there during the course of the study. There were two single occupancy bedrooms and two roo ms were shared by four residents. Three used wheelchairs for mobility, one person used a walker to assist with mobility, and the fifth person was ambulatory. Two people were non vocal and used gestures and simple sign language to communicate wants and ne eds; a third was non vocal and used a communication board to spell out what she wanted to say; a fourth person had approximately 30 recognizable words in his verbal repertoire but primarily used gestures and simple signs to communicate to others,
9 and the f ifth person was non vocal with very limited gestures she use d to communicate to others. All required assistance from staff to meet their daily living needs. The living room, dining room, family/activities room, lanai area, and backyard were areas design ated for leisure activities to occur. Some staff interactions in this home were positive while others had infrequent positive interaction and were limited to meeting basic care needs of the persons living in the home. Luke was a 33 year old male with 3 years experience working with people with developmental disabilities, all at his current place of employment. He was a college graduate. Luke worked in a group home that had four bedr ooms two were single occupancy rooms while the other two were double bedrooms. The people who lived there were all male with age ranging from 11 years to 26 years of age. They were all mobile and had some form of communicating their wants and needs. T h ree of the people living in this home were non vocal and used gestures and simple signs to communicate choices. All were capable to accessing preferred items and activities. The areas were Luke was observed included the dining room, family room, lanai, a nd backyard. The backyard was included because leisure activity choices include backyard activities such as playing basketball, riding bikes, and playing soccer. Positive interactions observed were typically minimal. Conversations were typically between staff working the same shift. Lisa was a 21 year old female with 10 months experience working with people with DD She obtained a medical tech certificate post high school graduation. Lisa's group home location had three bedroo ms and two bathrooms. The people who lived in
10 this home were all male, t wo were ambulatory and two used a wheelchair for mobility. A fifth male resident moved in to the home in the middle of the study. He was ambulatory. One was non vocal and the other four males had receptive and expressive communication skills. Areas designated for observations include the family room, kitchen/dining room, and lanai. Flora was a 26 year old female with a total of 3 years experience. She had been employed at her cu rrent place of employment for 1 year. Flora had a high school diploma. Flora's work site was a home that had four bedrooms. The people who resided at this home were non ambulatory and required the use of a wheelchair for mobility. In addition, the peop le at this home were non vocal, had extensive medical needs, and required staff supports to manage their daily routine. A common eating area that also functioned as an activity and positionin g was the area designated for positive interactions to be observ ed and recorded by the supervisor and/ or confederates. The bedrooms, bathroom, and laundry room were located in another area of the house. The residents all share d bedroom s Staff were infrequently engaged in positive inte ractions with the residents in this house Typically, conversations were between other staff working the same shift or others visiting the homes. The supervisor for this study was the principal investigator. She is a Board Certified Associate Behavior An alyst and has worked in that capacity for three years at the same place of employment where the participants were recruited. The behavior analyst and group home manager worked directly with the staff and facilitated staff meetings together.
11 Materials The materials included the MotivAider vibrating device (www.motiv aider.com; Amato Zech et al., 2006) The MotivAider is an electronic vibrator (does not beep) that can be programmed to vibrate on a continuous or intermittent schedule and is small enou gh to be attached to waistbands or placed in pockets. It weights 2.9 oz. and measures 2.5 inches in height, 2.3 inches in width, and 0.6 inches in depth. Signal duration can also be programmed for 2, 3, or 4 seconds It uses one AA battery. For the purp oses of this study, the MotivAider was programmed to vibrate every 1 minute and worn by Flora from 1:00 2:00pm; Nancy from 3:15 4:15pm; Luke from 4:30 5:30pm; and Lisa from 6:00 7:00pm. The MotivAider was also worn by the confederates; however, it was programmed to deliver a signal for data collection purposes every 15 seconds Clipboards or binders were used by all participants including the confederates. A self monitoring checklist and a reference list were included on the clipboards or in t he binders. Program specific data sheets were those that were in place in the residence independent of this study and were also used by staff participants. The self monitoring checklists were divided into 1 minute increments for a total of 60 intervals with time corresponding to the agreed upon leisure time hour. During intervention phases, the staff were expected to self record whether positive interaction occurred or not during the interval. Staff were informed that their self monitoring data sheets would be collected the next day or as soon as possible. Because staff were trained to engage in positive interactions with residents, a reference sheet listing examples of positive interactions was placed on the data collection clipboards or binders for r eview. It guided staff behavior in
12 response to different situations from which positive interaction could be initiated. A data sheet was also provided to the confederate during intervention phases. The confederate recorded the occurrence of target behav iors following 15 s econd interval recording for up to 30 minutes. Although the characteristics and capabilities of people with DD differed from home to home, the expectation for staff to interact with them was the same across homes and individuals Target behaviors (described below) were all possible within the context of leisure time activities Target Behaviors and Data Collection Positive i nteraction was assessed in all conditions. Positive interaction was defined as a comment ( e.g., responding to a qu estion from a client using more than one word ); positive comment ( e.g., "Looking good!" and "Nice outfit today!"); praise ( e.g., "That's great!" and "I like how you take your dishes to the sink after meals!" ,); and leisure activity involving client interac tion (i.e., engaging in a game of checkers, playing a game of cards etc.). Praise could also include providing physical contact such as pats on the back or high fives. The number of clients present in the room was also noted on the data sheet. Data were collected for up to 30 minutes during the designated leisure hour, Monday through Friday. The participants were scheduled to work a minimum of 2 days and up to 4 days during the week. The 30 minutes varied within the hour. For example, one sessi on was held 4:00 4:30pm while another session was held between 4:25 4:55pm. Leisure time was scheduled during the hour and data were collected in common
13 areas such as the living room or lanai where leisure activities occurred. The leisure activity al so occurred in the backyard to accommodate choices. The supervisor and/or confederate independently recorded whether positive interaction occurred during this time period. Partial interval recording in 15 s econd increments was used. A + was scored if p ositive interaction occurred and a 0 was scored if positive interaction did not occur during each interval. When staff left the room or leisure area for a work related reason (e.g., assisting a client with toileting), data collection paused and resumed when staff returned. The primary observer used a signal such as stretching arms or scratching head to communicate to the second data collector to pause and to resume recording target behaviors. Data collection also paused when the person with development al disabilities exhibited problem behaviors and resumed when the episode ended. Activities that were observed to occur during the leisure hour included dancing to music, playing a game (Connect 4 Uno ), singing to music, playing basketball, drawing, and wa tching spots on TV to name a few. Leisure time meant that no demands were placed and the people living in the home were free to access preferred activities or received assistance in accessing preferred activities Some demands were placed when the person was asked to make choices regarding what they would like to do ; however training included following through with verbal praise on m aking a good choice or some other positive comment about the choice made Interobserver Agreement 31% of the assessments in baseline and intervention phases include d observation and data collection by an independent observer (a confederate) in order to evaluate
14 interobserver agreement The independent observer was a fellow graduate or undergraduate student recruited fr om the Applied Behavior Analysis Masters Program. Confederates presented themselves as "social work" students who are volunteers for the same agency that employed the staff participants. They told the staff and people living in the homes if appropriate t hat they were learning about developmental disabilities and wanted to find out about satisfaction with community placement. The focus was on the people living at the homes and not on the staff participants or other staff The confederates recorded data o n the whether the participants engaged in positive interaction without them knowing it Two confederates were present in all three conditions to assess interobserver agreement; however, the supervisor also recorded data when permitted and served as a seco nd data collector to assess interobserver agreement. The percentage of agreement was calculated for all observation s by dividing the number of intervals with agreements for the target behavior by the number of intervals with agreements plus disagreements. An agreement was scored for an interval when both observers agreed that the target behavior occurred or did not occur. Percentage of agreement for each observation in baseline and intervention conditions was averaged to produce a mean for each condition. An interobserver agreement mean of 97% (range 81% to 100%) was obtained for intervals of positive interactions in 29% of total baseline sessions. An interobserver agreement mean of 87% (range 72% to 99%) was obtained for intervals of positive interactio ns during a total of 33% of sessions during the staff management intervention.
15 The primary data collector was either the supervisor or on e of the confederates. It was not always possible for the supervisor to be the primary data collector, especially in t he no supervisor present condition. Social Validity Questionnaire A social validity questionnaire, consisting of 6 questions answered with a 5 point Likert Scale was given to the participants after the study (see Appendix A). The questions asked the participants to rate the helpfulness of the procedures, the nature of their positive interactions before and after the study, the importance of supervisor presence on their interactions with clients, and whether they would recommend the staff management p rocedures to others. Experimental Design and Procedure A t least one confederate was on site in the group homes at different times prior to the implementation of training. Staff were told that the confederates were social work students learning about devel opmental disabilities and community placement and that they would be observing the individuals with disabilities. Baseline assessment was conducted by the confederate without the knowledge of the participant. Following baseline for each p articipant, training in an office setting was conducted on the importance of, and how to increase, positive interactions with clients. T he purpose and use of the MotivAider was also discussed. The study utilized an alternating treatments design (presence or absence of the supervisor and baseline probe) within a multiple baseline across participants design An ABAB design was used to evaluate the effects of
16 feedback for one participant. The participants were blind to the purpose of the study but briefed at its completion. Baselin e A ssessments of participants' positive interactions in the group homes were recorded through data collection by the confederate Participants did not receive feedback on their performance d uring these assessments. Baseline data collection continued until there were at least 3 stable data points in the supervisor absent condition and 2 stable data points in the supervisor present condition Obtaining baseline data in the supervisor present condition was necessary to compare to the supervisor present condition when the staff management package was implemented The supervisor did not collect any data in this condition. The supervisor appeared to be doing other routine tasks in the area simi lar to the Brackett et al. (2007) study. T raining. The supervisor conduct ed individual training by first discussing the importance of positive client interaction outlined in individual behavior plans. The supervisor then introduce d the MotivAider for sta ff to us e as a reminder to interact with the clients in a positive manner N ext the supervisor demonstrated a number of examples of positive interactions and asked the participant to rehearse a number of different positive interactions during role plays wi th the supervisor portraying a client. Each participant demonstrated a high rate of positive interactions (up to 8 per minute) during the role play s indicating that positive interaction skills were already a part of their repertoire In addition, a refer ence list of positive interactions was given to staff so it was available for staff to use in their interactions with clients. Staff were informed the MotivAider was programmed to vibrate every minute to remind them to interact with
17 clients in a positive manner. However, they were also informed that they should provide positive interaction more often than once per minute. Staff were also informed that the supervisor would be on site sometimes to monitor positive interaction with clients. Finally, staff were informed that feedback would not be provided by the supervisor at each of these observation sessions. Only one training session was provided in an office area and lasted approximately 1 hour and 30 minutes. Post Trainin g Assessment. P ost training a ssessments were conducted in the participant's assigned group home or cluster Data were collected in the same manner, time, and place as in baseline. During this phase, staff positive interactions were measured in three conditions to evaluate the presen ce of a supervisor and the use of the MotivAider. In each post training condition, the confederate was present with one of the following conditions in effect; supervisor present and MotivAider on, supervisor absent and MotivAider on, and baseline probe. T he first data point for each participant was the supervisor present and MotivAider on condition to compare baseline performance to performance in this condition immediately after training. If an improvement was noted, the order of the three conditions (de scribed below) was determined through a draw from a hat. Supervisor P resent + MotivAide r On In this condition, the MotivAider was activated while the supervisor was on site in the same room or area of the residence as the participant.
18 Supervisor Absen t + MotivAider On. In this condition, the staff wore the MotivAider at the specified time; however, the supervisor was not present during this time period. Supervisor Absent + MotivAider Off In this condition the staff was observed by the confederate at a time when neither the supervisor was present nor MotivAider activated (equivalent to baseline conditions). Supervisor Present + MotivAider On + Feedback When increases in positive interaction were not observed, an addition feedback phase was added. A feedback phase was provided by the supervisor to 2 participants who met this criterion. In the feedback phase the supervisor was present with the participant wearing the activated MotivAider. The supervisor provided feedback to the staff person approxima tely every 1 minute. If the participant had just engaged in positive interactions, the supervisor provided descriptive praise. If the participant missed opportunities to engage in positive interactions, the supervisor provided instructions, telling the par ticipant how he or she could have engaged in a positive interaction appropriate to the situation.
19 Chapter Three Results Results are shown in Figure s 1 and 2 During baseline, participants engaged in low levels of positive interactions when the supervi sor was present and when the supervisor was absent. With the implementation of the staff management package, two participants (Lisa and Flora) engaged in an increased level of positive interactions but only when the supervisor was present. Positive interac tions were not consistently higher when the supervisor was absent even when the MotivAider was activated. Two participants (Nancy and Luke) did not increase positive interactions until feedback was provided by the supervisor. The top panel of Figure 1 sho ws the results for Nancy. During baseline, Nancy engaged in a mean of 5% (range 2% to 10%) of intervals of positive interactions when the supervisor was absent and a mean of 3.5% (range 0% to 7%) when the supervisor was present. During the staff management package phase, positive interactions were at a mean of 9.3% (range 2% to 21%) of intervals during the supervisor present condition. During the one baseline probe, Nancy engaged in 1% of intervals of positive interactions. Because there was scant increase in positive interactions with the staff management package, feedback was added. Nancy engaged in positive interactions during 40% (range 23% to 64%) of intervals during the first feedback phase, 13% (range 12% to 16%) of
20 intervals during the return to no f eedback, and 51% (range 39% to 73%) of intervals when the final feedback phase was conducted. Figure 1 Results of staff management intervention package + feedback. The second panel of Figure 1 shows the results for Luke. During baseline, he enga ged in a mean of 2.7% (range 1% to 5%) of intervals with positive interactions when the supervisor was present and a mean of 4% (range 2% to 11%) when the supervisor was absent. When the staff management package was implemented, Luke increased intervals 0 % 1 0 % 2 0 % 3 0 % 4 0 % 5 0 % 6 0 % 7 0 % 8 0 % 9 0 % 1 0 0 % N a n c y S u p e r v i s o r P r e s e n t S u p e r v i s o r P r e s e n t M o t i v A i d e r O n B a s e l i n e S t a f f M a n a g e m e n t P a c k a g e F e e d b a c k N o F e e d b a c k F e e d b a c k 0 % 1 0 % 2 0 % 3 0 % 4 0 % 5 0 % 6 0 % 7 0 % 8 0 % 9 0 % 1 0 0 % 1 2 3 4 5 6 7 8 9 1 0 1 1 1 2 1 3 1 4 1 5 1 6 1 7 1 8 1 9 2 0 2 1 S e s s i o n s L u k e S u p e r v i s o r A b s e n t Percent Positive Interactions
21 o f positive interactions to a mean of 7.8% (range 2% to 16%) in the supervisor present condition. Because his scores were not substantially increased over baseline, the feedback phase was added. With feedback, Luke's level of positive interactions increa sed to a mean of 53% (range 38% to 63%). Figure 2. Results of staff management intervention package. 0 % 1 0 % 2 0 % 3 0 % 4 0 % 5 0 % 6 0 % 7 0 % 8 0 % 9 0 % 1 0 0 % L i s a S u p e r v i s o r A b s e n t B a s e l i n e S t a f f M a n a g e m e n t P a c k a g e S u p e r v i s o r P r e s e n t M o t i v A i d e r O n 0 % 1 0 % 2 0 % 3 0 % 4 0 % 5 0 % 6 0 % 7 0 % 8 0 % 9 0 % 1 0 0 % 1 2 3 4 5 6 7 8 9 1 0 1 1 1 2 1 3 1 4 1 5 1 6 1 7 1 8 S e s s i o n s F l o r a S u p e r v i s o r P r e s e n t S u p e r v i s o r A b s e n t M o t i v A i d e r O n Percent Positive Interaction s
22 The first panel of Figure 2 shows Lisa's results. During baseline, she engaged in a mea n of 2% (range 0% to 4%) of intervals of positive interactions when the supervi sor was present and a mean of 5% (range 1% to 8%) when the supervisor was not present. When the staff management package was implemented, Lisa engaged in a mean of 23% (ran ge 12% to 33%) of intervals of positive interactions in the supervisor present condition and a mean of 4% (range 3% to 5%) in the supervisor absent condition. Baseline probe sessions were held post training and Lisa's levels of positive interactions at a mean of 11% (range 5% to 23%). Finally, the second panel of Figure 2 shows Flora's results. Flora engaged in a mean of 0.7% (range 0% to 1%) of intervals with positive interactions with supervisor present and 0% of intervals when the supervis or was absent during baseline. In the staff management phase, Flora engaged in a mean of 42% (range 30% to 77%) of positive interactions when the supervisor was present, an increase of 41.3% compared to baseline. When the supervisor was absent, Flora eng aged in a mean of 12 % (range 2% to 29%) of positive interactions. Her level dropped to 0% of positive interaction intervals when two baseline probe sessions were conducted Social validity was also assessed with the use of a questionnaire given to the participants and was not anonymous. Responses were scored on a Likert S cale ranging from 1 to 5. Nancy rated the staff management interventio n a 2 in increasing her positive interaction with people she works with. In addition, she did not indicate a difference in the level of positive interactions before or after the staff management intervention was implemented rating each a 4). Nancy rated the presence of a supervisor as making no
23 difference at all. She would not continue using the intervention on the job and rated her recommendation for other group homes to implement the staff management intervention as a 2. Because scant increases in pos itive interactions were observed when the supervisor was prese nt and the MotivAider was on, a feedback phase was necessary to increase her levels to a higher rate. When the feedback was withdrawn, Nancy's level of positive interaction dropped to closer to her baseline levels even when the supervisor was present. In this case, she accurately self reported that a supervisor on site did not make a difference in her positive interactions with people living in the group home. Luke found the staff management in tervention extremely helpful and rated his level of positive interactions prior to its introduction as a 3 and after it was implemented as a 4. He said that having a supervisor on site made a big difference in his positive interactions with people he work s with and said he would recommend that other group homes also implement the staff management intervention. When asked if he would continue to use this intervention on the job, Luke said that he was not at all likely. He commented that he could also lear n more if another staff was using it. Overall, he commented that this staff management intervention was a good thing and helped the people he works with feel good about themselves. When compared to the results of his positive interactions, Luke's positiv e interactions were not significantly more when the staff management package was implemented. Like Nancy, a feedback phase was also added and more positive interactions were observed as a result. The supervisor's presence after training was conducted did not make a considerable difference in
24 providing more positive interactions and it was only after feedback was added that the higher levels occurred. Lisa rated the helpfulness of tactile prompts and self monitoring a 4. She rated her positive interaction s with people with developmental disabilities residing at her place of work a 4 before the introduction of the staff management intervention and a 5 after its introduction. She rated the supervisor's presence making a big difference in her positive intera ctions. Finally, Lisa reported that she is likely to continue using the staff management intervention on the job and highly recommends that other group homes also implement the staff management intervention. An increase in positive interactions was obser ved after training was conducted. Her data also show that more positive interactions were observed when the supervisor was on site while the MotivAider was activated. The levels in the supervisor absent condition and baseline probes were lower. Flora fou nd the staff management intervention extremely helpful (a score of 5). She rated her positive interactions a 3 prior to its introduction and rated herself a 5 after the staff management intervention was implemented. Interestingly, Flora reported that she did better when the supervisor was not on site and rated this question a 2 (supervisor presence did not make a big difference). She was not likely to continue to use this staff management intervention on the job and recommended its use at other group hom es "only if they are willing to do it." F lora 's positive interactions were almost non existent prior to training. Levels were indeed higher when the staff management package was implemented. Her data also suggest that the when the supervisor was absent, her level of positive interaction were at lower rates (she did not do better).
25 Based on the answers provided by the participants, i t is not clear why the participants agreed to take part in this study. They did not state needing improvements in this area of their job performance.
26 Chapter Four Discussion Overall, the results o f this study showed that the staff management package (training to increase positive interaction and the use of tactile prompts via the MotivAider) increased the occurrence of positive interaction Lisa and Flora had with people with developmen tal disabilities but did not significantly improve the performance of Nancy and Luke. Furthermore, the increase in positive interactions for Lisa and Flora only occurred when a supervisor was present. When they were observed in the absence of a supervisor positive interactions were much lower. Feedback was added with Nancy and Luke when there was minimal change in their positive interactions during the staff management intervention compared to baseline. It was only after feedback was introduced that a s ignificant increase in their positive interactions was seen. It is a noteworthy finding from this study that none of the four participants increased their positive interactions with clients to any meaningful degree following staff training and the use of the tactile prompt and self monitoring when a supervisor was not present. Even though they demonstrated the skills during training and wore the MotivAider to prompt their behavior when working with their clients, they failed to increase their positive int eractions when they did not know they were being watched. For staff training and management procedures to be valuable they must produce changes in staff behavior that persist in the absence of direct supervision. Given that staff are
27 frequently present wit h their clients while a supervisor is not (especially during night shifts), it is important for staff management procedures to work in the absence of the stimulus control of a supervisor's presence. It is equally important foe staff management procedures to be cost effective. One of the strengths of this study was the use of confederates for data collection. The confederates were present in the group home to collect data on staff interactions with their clients without any awareness by the staff that they were being observed. The use of confederates allowed us to collect data on the "true" level of positive interactions that was not influenced by the confounding presence of a data collector. Considering that the vast majority of staff management studies uti lize individuals present in the site who are known by the staff to be collecting data on staff behavior, the results of these studies are likely confounded to some degree by reactivity to the presence of the data collectors. This study and the study by Bra cket at al. (2007) are the only studies to our knowledge to use surreptitious data collection. Future research on staff management should use similar forms of data collection. This type of data collection, termed in situ assessment, is used widely in othe r research areas such as child safety skills training (e.g., Miltenberger, 2008). In safety skills training studies, children are observed without their knowledge to determine whether they will use the safety skills when a parent, teacher, or other adult i s not present. In this way the safety skills are not under the stimulus control of the presence of an observer, parent, or teacher. It is important for staff management research to continue to use confederates to conduct in situ assessments to determine wh ether staff continue to use important skills in the absence of an observer or supervisor.
28 The findings of this study expand on the Brackett et al. (2007) study by showing that the presence of a supervisor had an effect on staff performance for two partici pants. For Lisa and Flora, even without any feedback or a history of feedback, higher rates of positive interaction were seen when the supervisor was present as compared to other conditions. Although Nancy and Luke did not increase positive interaction w ith the staff management procedures, even when a supervisor was present, we speculated that positive interactions might increase in the supervisor's presence once there was a history of feedback from the supervisor. To investigate this possibility, an ABAB design was used with Nancy to evaluate feedback and to see if positive interactions would be maintained in the absence of feedback. However, even with a history of supervisor feedback, Nancy did not show an improvement in positive interaction when feedbac k was removed. All four staff in this study demonstrated increases in positive interactions by the end of the study. Anecdotally, these particular staff demonstrated different styles at increasing positive interactions. For example, Luke was observed to rotate attention to persons in his immediate area while Nancy focused on one person at a time. In the final data points for Luke, he positioned himself in between the people he was supervising and went from one person to the next engaging them in convers ation or an activity and providing verba l and physical praise. It would be interesting to find out whether better results would have been achieved if more than one staff us ed the MotivAider. If multiple staff used the MotivAider they may have provided eac h other with support to interact with the people they serve with in a positive manner. Perhaps utilizing lead staff to model the desired
29 staff behav ior could be useful in promoting behavior change in other staff Another interesting research que stion is whether adding one or more intermittent supervisor feedback session s post training would promote the desired staff performance of providing more positive interactions. The results of the social validity questionnaire showed that each of the four participan ts rated their positive interactions before intervention as fairly high (when in reality they were quite low). Based on their perception that they already engaged in high levels of positive interactions, it is not clear why they agreed to participate in t his study with a stated goal of increasing positive interactions. I t would be interesting to find out reasons why participants agree to take part in a study involving staff management p rocedures Perhaps selecting only those who are motivated to make som e improvements in job related tasks would produce different results. There were several challenges faced by the author in conducting this study. First, due to a number of logistical problems and unavoidable conflicts, data collected in some sessions could not be used. In one case, the group home manager/supervisor was present in a no supervisor condition, thus violating the integrity of the condition. In another case, the MotivAider should have been on for a "supervisor absent and MotivAider on" condition, but the participant forgot to bring the device to work for that session. One data point could not used because the activity was not a leisure activity and instead consisted of a self care activity. Decisions to exclude data were made by authors of this study when the integrity of the condition was violated.
30 An additional challenge faced by the author was the possibility of observer drift. Observer drift was noted with data collectors who had similar training in another project and may have been one possi ble explanation for some low interobserver agreement scores (i.e., less than 90%). Another factor that may have contributed to relatively lower interobserver agreement scores, mainly during intervention, was that the MotivAiders worn by the two observers were not perfectly synchronized. Starting and stopping the MotivAiders at exactly the same time was not easily done and may have contributed to slight differences in the interval data collected by the two observers. A final challenge in conducting this s tudy involved the logistics of getting the confederates and participants together at the right times. Confederates were required to travel to the different sites and coordinate with each other for reliability sessions. Not all opportunities to collect da ta could be utilized due to schedule conflicts. Furthermore participants were not always were they needed to be and therefore some sessions could not be conducted. Communication with the managers or scheduling coordinators with the primary author of this study was not consistent and switching participants to work at another site or perform another duty (van runs) resulted in canceled sessions. In spite of the challenges faced by the author in conducting this study, the study produced interesting and impor tant findings. The results of this study demonstrated that staff management procedures consisting of prompts and self monitoring may not be effective in promoting desired staff behavior in a group home setting, even though other studies suggest that they m ay be effective (Petscher & Bailey, 2006; Scheeler & Lee, 2002)). These procedures did not work for any of the four participants in this study unless
31 a supervisor was present on site, and even then it was effective for only two of the four. These are disco uraging findings suggesting that it may be difficult to motivate staff to engage in relatively simple positive interaction with clients when they do not know they are being watched (i.e., unless a supervisor is present). Although feedback was effective whe n implemented with two of the participants, the effects did not persist when feedback was withdrawn for Nancy. Although the demonstration that feedback was effective was valuable, the fact that the behavior did not persist in the absence of feedback sugges ts the need for better staff management strategies that can motivate staff performance in the absence of a supervisor who must be present to deliver feedback.
32 References Alavosius, M.P. & Sulzer Azaroff, B. (1990). Acquisition and maintenance of health care routines as a function of feedback density. Journal of Applied Behavior Analysis 23(2), 151 162. Arco, L. & Du Toit, E. (2006). Effects of adding on th e job feedback to conventional analog staff training in a nursing home. Behavior Modification 30(5), 713 735. Amato Zech, N.A., Hoff, K.E., & Doepke, K.J. (2006). Increasing on task behavior in the classroom: Extension of self monitoring strategies. Ps ychology in the Schools 43(2), 211 221. Austin, J., Weatherly, N.L., & Gravina, N.E. (2005). Using task clarification, graphic feedback, and verbal feedback to increase closing task completion in a privately owned restaurant. Journal of Applied Behavior A nalysis 38(1), 117 120. Bassett, J.E. & Blanchard, E.B. (1977). The effect of the absence of close supervision on the use of response cost in a prison token economy. Journal of Applied Behavior Analysis 10(3), 375 379. Brackett, L., Reid, D.H., & Gree n, C.W. (2007). Effects of reactivity to observations on staff performance. Journal of Applied Behavior Analysis 40(1), 191 195. Burgio, L.D., Engel, B.T., Hawkins, A., McCormick, K., Scheve, A., & Jones, L.T. (1990). A staff management system for main taining improvements in continence
33 with elderly nursing home residents. Journal of Applied Behavior Analysis 23(1), 111 118. Codding, R., Feinberg, A.B., Dunn, E.K., & Pace, G.M. (2005). Effects of immediate performance feedback on implementation of beha vior support plans. Journal of Applied Behavior Analysis 38(2), 205 219. Cook, T. & Dixon, M.R. (2005). Performance feedback and probabilistic bonus contingencies among employees in human service organizations. Journal of Organizational Behavior Managemen t 25(3), 45 63. Cooper, K. J. & Browder, D. M. (2001). Preparing staff to enhance active participation of adults with severe disabilities by offering choice and prompting performance during a community purchasing activity. Research in Developmental Disab ilities 22, 1 20. Crowell, C.R., Anderson, D.C., Abel, D.M., & Sergio, J.P. (1988). Task clarification, performance feedback, and social praise: Procedures for improving the customer service of bank tellers. Journal of Applied Behavior Analysis 21(1), 65 71. De Pry, R.L., & Sugai, G. (2002). The effects of active supervision and pre correction on minor behavioral incidents in a sixth grade general education classroom Journal of Behavioral Education 11(4), 255 267. Doerner, M., Miltenberger, R.G., & Bakk en, J. (1989). The effects of staff self management on positive social interactions in a group home setting. Behavior Residential Treatment 4(4), 313 330.
34 Fleming, R. & Sulzer Azaroff, B. (1992). Reciprocal peer management: Improving staff instruction in a vocational training program. Journal of Applied Behavior Analysis 25(3), 611 620. Green, C.W., Reid, D.H., Perkins, L.I., & Gardner, S.M. (1991). Increasing habilitative services for persons with profound handicaps: An application of structural analysi s of staff management. Journal of Applied Behavior Analysis 24(3), 459 471. Gross, A.M. & Ekstrand, M. (1983). Increasing and maintaining rates of teacher praise: A study using public posting and feedback fading. Behavior Modification 7(1), 126 135. Hag ermoser Sanetti, L.M., Luiselli, J.K., & Handler, M.W. (2007). Effects of verbal and graphic performance feedback on behavior support plan implementation in a public elementary school. Behavior Modification 31(4), 454 465. Harchik, A.E., Anderson, M., Tho mpson, R., Forde, K., Feinberg, L., Rivest, S., & Luiselli, J.K. (2001). Evaluation of a participatory competency based model of staff training in a community habilitative setting. Behavioral Interventions 16, 1 13. Harchik, A.E., Sherman, J.A., Sheldon, J.B., & Strouse, M.C. (1992). Ongoing consultation as a method of improving performance of staff members in a group home. Journal of Applied Behavior Analysis 25(3), 599 610.
35 Harris, K.R. (1986). Self monitoring of attentional behavior versus self monitor ing of productivity: Effects on on task behavior and academic response rate among learning disabled children. Journal of Applied Behavior Analysis 19(4), 417 423. Harris, T.A., Peterson, S.L., Filliben, T.L., Glassberg, M., & Favell, J.E. (1998). Evaluat ing a more cost efficient alternative to providing in home feedback to parents: The use of spousal feedback. Journal of Applied Behavior Analysis 31(1), 131 134. Hutchison, J.M., Jarman, P.H., & Bailey, J.S. (1980). Public posting with a habilitation team : Effects on attendance and performance. Behavior Modification 4(1), 57 70. Miltenberger, R. (2008). Teaching safety skills to children: Prevention of firearm injury as an exemplar of best practice in assessment, training, and generalization of safety ski lls. Behavior Analysis in Practice 1, 30 36. Mozingo, D.B., Smith, T., Riordan, M.R., Reiss, M.L., & Bailey, J.S. (2006). Enhancing frequency recording be developmental disabilities treatment staff. Journal of Applied Behavior Analysis 39(2), 253 256. Pa mpino, R.N., Jr., MacDonald, J.E., Mullin, J.E., & Wilder, D.A. (2003). Weekly feedback vs. daily feedback: An application in retail. Journal of Organizational Behavior Management 23(2/3), 21 43. Parsons, M.B., Cash, V.B., & Reid, D.H. (1989). Improving r esidential treatment services: Implementation and norm referenced evaluation of a comprehensive management system. Journal of Applied Behavior Analysis 22(2), 143 156.
36 Parsons, M. B., Reid, D. H., & Green, C. W. (1993). Preparing direct service staff to teach people with severe disabilities: A comprehensive evaluation of an effective and acceptable training program. Behavioral Residential Treatment 8(3), 163 185. Parsons, M. B., Reid, D. H., & Green, C. W. (1996/0). Training basic teaching skills to comm unity and institutional support staff for people with severe disabilities: A one day program. Research in Developmental Disabilities 17(6), 467 485. Petscher, E.S. & Bailey, J.S. (2006). Effects of training, prompting, and self monitoring on staff behav ior in a classroom for students with disabilities. Journal of Applied Behavior Analysis 39(2), 215 226. Quilitch, H.R. (1975). A comparison of three staff management procedures. Journal of Applied Behavior Analysis 8(1), 59 66. Reid, D.H., Schuh Wear, C .L., & Brannon, M.E. (1978). Use of group contingency to decrease staff absenteeism in a state institution. Behavior Modification 2(2), 251 266. Reinke, W.M., Lewis Palmer, T., & Martin, E. (2007). The effect of visual performance feedback on teacher use of behavior specific praise. Behavior Modification 31(3), 247 263. Richman, G.S., Riordan, M.R., Reiss, M.L., Pyles, D.A.M., & Bailey, J.S. (1988). The effects of self monitoring and supervisor feedback on staff performance in a residential setting. Journ al of Applied Behavior Analysis 21(4), 401 409.
37 Scheeler, M.C., & Lee, D.L. (2002). Using technology to deliver immediate corrective feedback to preservice teachers. Journal of Behavioral Education 11(4), 231 241. Suda, K.T., & Miltenberger, R.G. (1993) Evaluation of staff management strategies to increase positive interactions in a vocational setting. Behavioral Residential Treatment 8(2), 69 88. Wilson, P.G., Reid, D.H., & Korabek Pinkowski, C.A. (1991). Analysis of public verbal feedback as a staff management procedure. Behavioral Residential Treatment 6(4), 263 277.
39 Appendix A: Social Validity Questionnaire Thank you for participating in a study evaluating a staff management package to increase positive client intera ction. Please take a few minutes to complete this questionnaire about your experience. 1. How helpful was the use of tactile prompts, self monitoring, and supervisor feedback to increase your positive interaction with your clients? 1 2 3 4 5 Not at al l helpful Extremely helpful Comments: 2. How would you rate your positive interaction with clients prior to the introduction of this staff management package? 1 2 3 4 5 Very few positive Many positive interactions interactions Comments: 3. How would you rate your positive interaction with clients after you started using the staff management package? 1 2 3 4 5 Very few positive Many positive interactions interactions Comments:
40 4. Do you feel that having a supervisor on site makes a difference in your positive interactions with residents? 1 2 3 4 5 Made no difference Made a big difference at all Comments: 5. How likely are you to continue to use this staff management package on the job? 1 2 3 4 5 Not at all likely Extremely likely Comments: 6. Would you recommend that other group homes implement this package? 1 2 3 4 5 Not at all recommended Highly recommended Comments: Thank you!