xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam 2200361Ka 4500
controlfield tag 001 002317644
007 cr bnu|||uuuuu
008 100903s2009 flua ob 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0003108
Hybza, Megan M.
Increasing Medicaid dollars billed for services by school psychologists using a performance improvement package
h [electronic resource] /
by Megan M. Hybza.
[Tampa, Fla.] :
b University of South Florida,
Title from PDF of title page.
Document formatted into pages; contains 41 pages.
Thesis (M.A.)--University of South Florida, 2009.
Includes bibliographical references.
Text (Electronic thesis) in PDF format.
ABSTRACT: Organization Behavior Management (OBM) is a subfield within the discipline of Applied Behavior Analysis (ABA) involving the application of ABA principles and practices to organizational settings. One successful intervention used in OBM is the implementation of a performance improvement package. This approach has proven to be an effective problem solving strategy in a variety of settings, based on effective components such as goal-setting, prompting, and feedback. In this endeavor a performance improvement package was applied within an educational setting. In this school district, school psychologists are required to complete documentation for Medicaid reimbursement to the district, but were inconsistent in doing so. The purpose of this study was to improve the consistency of billing for Medicaid reimbursement by 74 school psychologists serving 102 schools. A multiple baseline across three areas was used to assess the effectiveness of the intervention introduced in a sequential manner.
Mode of access: World Wide Web.
System requirements: World Wide Web browser and PDF reader.
Advisor: Trevor F. Stokes, Ph.D.
Performance improvement package
x Child and Family Studies
t USF Electronic Theses and Dissertations.
Increasing Medicaid Dollars Billed for Services by School Psychologists Using a Performance Improvement Package by Megan M. Hybza A thesis submitted in partial fulfillment of the requirements for the degree of Master of Arts Department of Child and Family Studies College of Behavioral and Community Sciences University of South Florida Major Professor: Trevor F. Stokes Ph.D. Marilee Hayman, MS, Tracy Schatzberg Ed S Date of Approval: Jun e 18 2009 Keywords: performance impr ovement package, goal setting, prompting, feedback Copyright 2009 Megan M. Hybza
Dedication I dedicated this manuscript to my family: my father, mother, and two younger brothers. My father has given me the drive to take on all my professio nal goals, and he reminded me to never let anyone/anything bring me down. My mother has been my constant support throughout my schooling, and she helped me stand upright when the world felt sideways. My brothers have been my constant reminders to smile a nd laugh through the good and bad. Thank you all for your continuous encouragement!
Acknowledgements First and foremost, I would like to start off by thanking Dr. Trevor Stokes for ughout my entire graduate school process, he has been my other half, my mentor, friend, and taught me the I NEED to thank Marilee Hayman for EVERYTHING. She opened ever door I tried to get through, and without her my education and practicum experiences would have never be so great. She taught me to simplify, the art of writing, and the importance of networking. Dr. Tracy Schatzberg, I want to thank for believing in my thesis and giving me an opportunity to watch it grow from beginning to end. She was very receptive of all my suggestions and wonderful to work with. Harold Daniels was at the heart of my thesis, sending me the data as it came in. This thesis would be nothing without him and his positive outl ook on the study. I would also like to thank Lindsey Jones, everyday she has a positive attitude that is contagious and makes me smile. Lastly, to my high school English teacher, Sam Joseph for his consistency in making me write paper after paper. Bec ause of him I felt prepared to tackle my thesis and literature review with a stress free attitude.
i Table of Contents List of Tables ii List of Figures iii Abstract i v Introduction 1 Feedback Alone 2 Performance Improvement Package 7 Current St udy 15 Me thod 16 Participants and Settings 16 Performance Analysis 17 Medicaid System 17 System Analysis 19 Data Collection and Dependent Variable 20 Interobserver Agreement 20 Social Validity 21 Experimental Design 21 14 Procedures 22 Baseline 22 Performance Improvement Package 22 Results 26 Dependent Variables 26 Dollars Billed 26 Total number of school psychologists 28 Interobserver Agreement 30 Social Validity 30 Discussion 32 References 37 Appendices 4 0 Ap pendix A: School Psychologist Social Validity Questionnaire 41
ii List of Tables Table 1. Percentage of Social Validity Questionnaire Results 31
iii List of Figures Figure 1. The total dollars billed to Medicaid for reimbursement by school psychologists 27 Figure 2 The number of school psychologists who turned in documentation for Medicaid billing. 29
iv Increasing Medicaid Dollars Billed for Services by School Psychologists Using a Performance Improvement Package Megan M. Hybza A BSTRACT Organization Behavior Management (OBM) is a subfield within the discipline of Applied Behavior Analysis (ABA) involving the application of ABA principles and practices to organizational settings. One successful intervention used in OBM is the implementa tion of a performance improvement package. This approach has proven to be an effective problem solving strategy in a variety of settings, based on effective components such as goal setting, prompting, and feedback. In this endeavor a performance improvem ent package was applied within an educational setting. In this school district, school psychologists are required to complete documentation for Medicaid reimbursement to the district, but were inconsistent in doing so. The purpose of this study was to im prove the consistency of billing for Medicaid reimbursement by 74 school psychologists serving 102 schools. A multiple baseline across three areas was used to assess the effectiveness of the intervention introduced in a sequential manner
1 Introduction O rganizational Behavior Management (OBM), a subfield of Applied Behavior Analysis, is effective when implemented in organizational settings. Daniels and Daniels (2004, 4 th .ed), prominent OBM consultants, described the value of performance management within organizations. When performance management processes are utilized positive results can be seen immediately or over time. Performance management is user friendly and does not require staff members to have a specialized college degree to use the OBM proced ures. Organizational Behavior Management has the flexibility to be used in a variety of settings, such as industrial plants (Goltz, Citera, Jensen, Favero, & Komaki,1989), health facilities (Jones, Morris, & Barnard,1986; Hawkins, Burgio, Langford, & Enge l,1992; Slowiak, Madden, & Mathews, 2005), and universities (Wilk & Redmon, 1990; Tittelbach, DeAngelis, Sturney, &Alvero, 2007. The behaviors of people are observed and modified; therefore, where ever there are people, OBM has made its mark. Organization al Behavior Management is not the only scientific field within psychology that claims to be effective in changing organizational settings. Industrial/Organizational psychology takes its place alongside OBM. Both fields can claim to improve the organizati on, but OBM has the scientific advantage because change procedures are more frequently guided by data. Aubrey Daniels International sums up the OBM advantage best in a quote on the OBM Network website (www.obmnetwork.com):
2 almost any technology, process, or innovation can be replicated, leaving most organizations without a decisive competitive advantage. However, leaders who embrace Organization Behavior Management (OBM) are gaining a competitive advantage that is nearly i mpossible to emulate. Why? Behavior is difficult to replicate more so than any other aspect of business. Unless you have a strong In order to have a su ccessful organization, a business needs to focus on the behaviors of all employees through careful assessment. A business can use data to change problem behaviors to see the results they are looking for. Within the OBM literature, research data has shown that performance feedback is an effective intervention. Feedback Alone about performance 2004, p.171). A study by Goltz, Citera, Jensen, Favero, and Komaki (1989) asked the question, product while working with the product in a microelectronics plant. The feedback effectiveness was assessed using an ABCB reversal design, where A was baseline; B involved group feedback; C consisted of group and individual feedback; B was a reversal phase in which only group feedback was received.
3 The group feedback was displayed on a chart and cumulative graph. The chart also contained the handling behaviors being obser the product. The graphic feedback displayed the overall percentage for all the behaviors combined for that day. During the group plus individual feedback condition, the feedback chart also provided individual information on performance for each handling behavior, overall individual percentage for his or her percentages with the group percentages. Results indicated that there was an improvement when individual feedba ck was added to group feedback compared to group feedback alone. However, when individual feedback was withdrawn and group feedback continued, little decrease in performance was observed. Handling behavior during the group only feedback condition maintai ned at a level similar to group plus individual feedback. Because the experimental design may have a problem of sequence effects in the condition order of ABCB, this study results are suggestive though not definitive in concluding that the individual feed back condition improved handling behavior. Feedback was also examined in a study by Jones, Morris, and Barnard (1986), participants. The staff included psychiatrists, psych iatric medical residents, psychiatric social workers, and psychiatric nurses. The intervention consisted of didactic instruction and grouped graphic feedback. The state required that three forms (notice of rights, imminent harm applications, and witness lists) be filled out before a patient could be
4 detained involuntary for observation and treatment. Accuracy in completing these forms was the dependent measure for the study. During baseline, a state required training session was conducted to inform staff of new civil commitment procedures and train them to accurately complete forms. Intervention included a feedback package consisting of instructions and group graphic feedback. Intervention was assessed using a multiple baseline across the three required forms. Instructions were given during three meetings, at which the mental health coordinator met individually or with up to three staff members at one time. During the meetings, participants were told about the study being conducted. Then, participants a nd the coordinator reviewed the civil commitment process to become comfortable talking to one another. Next, each staff member was given a folder that included a job aid on how to fill out each of the three forms and a rationale as to why the forms needed to be filled out completely and correctly. Following instructions, staff was given a graph displaying group mean percentages for the three forms completed correctly. Then, the staff was asked if they would like to receive an updated weekly graph. Resu lts indicated that correct completion of the three forms increased during intervention and was maintained during a follow up condition when no graphic feedback was provided. This study adds to the literature on the effectiveness of group feedback. The ef fect of sources of feedback was explored in a study by Tittelbach, Fields, & Alvero in their 2007 study. The study examined the effects of supervisor verse peer verbal feedback on accuracy and speed of a typing task behavior. The study included 63 underg raduate students and was conducted in a laboratory room. In the room, cubicles were set up containing a computer and printer inside. A pretest was given on the
5 with minimal mistakes in a 100 word document. There were two independent variables: verbal feedback by the supervisor or by the peer, with feedback based on accuracy, speed, and combined. There was also a control group, which was given no feedback. A question naire was given at the end of the study to determine if the participants distinguished between the supervisor and peer, which determined that the participants were able to distinguish between the persons. Speed and accuracy were the dependent variables fo r the study. The design was a 2X3X2 repeated measures mixed factional design. Prior to baseline, participants were given instructions as they sat at the computers. They were told to type the letters as they appeared on the screen in both capital and lowe rcase performance. Feedback was given on the number of number blocks (4 minutes) completed, number completed accurately, and number copied inaccurately. The baseline period was 35 minutes long, and the intervention condition consisted of eight, four minute blocks. The experimental condition contained separate supervisor and peer feedback phases, while the control phase was the same as baseline. Results indicated ther e was no difference between the effects of supervisor and peer feedback on an increase in typing speed and accuracy with verbal feedback. The Hawkins, Burgio, Langford, & Engel (1992) study examined the effects of adding written evaluative feedback to a ve rbal feedback system on the use of a prompted voiding procedure by geriatric nursing assistants (GNAs). This study took place in a nursing home with 47 GNAs as participants.
6 Prior to intervention, the GNAs participated in a maintenance program that set go als, gave graphed feedback on staff performance, and provided verbal feedback by supervisor. Part of the maintenance program involved the use of a GNA self monitoring form, supervisor monitoring, and individual performance feedback, given to each GNA by t he head nurse. GNAs were given verbal and written instructions on how to implement the prompted voiding procedure. The self monitoring forms were kept in the scheduled ti me and if the resident was wet or dry. Each GNA signed the self monitoring form. During staff meetings the head nurse gave individual verbal feedback and graphic feedback as a group. The head nurse praised individuals who completed 60% or more of the pro mpted voids that were assigned and corrective feedback to anyone under 60%. During the intervention condition consisting of written evaluative feedback, GNAs were given a memo to inform them that a letter would be sent out every two weeks describing their performance, which would be placed in their personal file and used in their annual performance evaluation. Letters contained praise to those who completed 60% or more of their assigned prompted voids, or a statement that there was need for improvement fo r those GNAs fewer than 60% performance. A design with sequential introduction of individual feedback with different units was used, without repeated measures in baselines, technically not a multiple baseline design. The intervention effectively increased prompted voiding procedures across the three units. Results indicated that the addition of written evaluative feedback to verbal feedback can increase staff performance.
7 Performance Improvement Package Review of the literature concerning feedback sug gests that it alone does not improve performance to the fullest extent; the addition of other procedures along with feedback tends to improve its effects (Balcazar, Hopkins, & Suarez, 1985; Alvero, Bucklin, & Austin 2001). One addition to performance fee dback is goal setting. The study by Wilk & Redmon ( 1990 ) included three participants from an undergraduate admissions office and focused on the number of tasks they completed, overtime costs, and absenteeism. The tasks consisted of loading an application recalculating a GPA, typing a label with a name and putting it on a folder. Data were recorded on a data sheet that the participants signed During the intervention condition, the superviso r met with each employee to talk about daily goals. These goals were specific and contained the precise number of tasks to reviewed and office needs were taken into cons ideration. Feedback was given by the supervisor a minimum of 2 times per day focused on the completion of meeting or approaching goals. Praise was given if an employee was working and prompts were given if there was a need to focus back on the goals. Th e study used a multiple baseline across participants design, with baseline data being self monitored. Results indicated an increase in the number of tasks completed. Also, overtime cost decreased to $0.00, and hours absent from the eight hour work day de creased. A second addition to performance feedback is recognition and praise by the supervisor, Brown et al. (1981) examined the effects of supervisor verbal feedback and verbal feedback plus praise on staff at a residential facility for individuals with h andicap
8 and intellectual disabilities during a morning and afternoon shift. Three categories of staff behaviors were assessed: social interaction defined as positive or neutral verbal, gestural, or physical contact with a resident, direct care stimulation defined as working directly with the resident in care work such as dressing and talking with them at one time, and off task was defined as a staff member engaged in behaviors such as talking with other staff or reading a book and not engaged in behavior d ealing with the unit or residents. The supervisor recorded data at hourly intervals using time sampling procedures. During baseline, staff members were to perform their jobs as usual. They were told that observations were being conducted to provide info rmation on how much time was spent on different tasks. Before the feedback condition began, staff members were informed that they would be receiving feedback on their performance. During this condition, the supervisor provided feedback to each staff memb er individually. Supervisors did not provide approval or disapproval of staff performance at the time. During the feedback plus praise condition, the supervisor used provided positive gories of behaviors which were the focus of the study. A multiple baseline across staff and a reversal design was used to assess the effects of the intervention. During intervention, mean percentages of feedback alone did little to improve performance; t he addition of feedback plus praise increased direct care/stimulation across both shifts and decreased off task behavior. Another performance improvement component that has been added to feedback is task clarification. Task clarification, goal setting, a nd feedback have been combined in several studies (Tittelbach, DeAngelis, Sturney, &Alvero, 2007; Amigo, Smith, &
9 Ludwig, 2008). The effects of the combined intervention components were used to decrease table busing times in a franchise pizza restaurant i n the study by Amigo, Smith, & Ludwig, 2008. The study used an ABC design, indicating that phase A was baseline, B was task clarification and goal setting, and lastly phase C was group and individual feedback. During phase B, the participants were given a memo instructing them on the correct steps to be used when bussing a table. A goal was set to reduce the time to bus a table to three minutes or less. Participants were reminded of the goal throughout the study. During phase C, the manager informed p articipants individually of their busing performance by providing graphic feedback displaying his or her average time; the goal was also reviewed at this time. Group graphic feedback of the combined average busing time for all participants was posted in th e back of the restaurant. Results indicated that both intervention phases reduced table busing time to less than three minutes, meeting the established goal. The nature of an ABC design made interpretation of data difficult; however, the B phase was an i mprovement on baseline, while the C phase was associated with maintenance. In a more sophisticated experimental design, Tittelbach et al. (2007) examined the effects of task clarification, feedback, and goal custom er service behaviors. These behaviors included: punctuality (arrival 5 minutes correct front desk behavior (sitting up correctly and facing clients while behind th e desk). Unlike the Amigo, Smith, and Ludwig, 2008 study, this study combine all three components in a multiple baseline design across behaviors design. The data were made up of the average percentage across all 10 participants. During baseline, role p lays were
10 shown for correct office and customer service behaviors. During intervention, a goal was stated by the supervisor for future performance. A task clarification sheet was given describing average group performance and verbal feedback was also giv en to the group. Results indicated a strong effect across all behaviors, especially greeting. Another study, by Anderson, Crowell, Hontula, and Siroky (1988) took place at a university bar in need of major cleaning components. Thirty bar employees partic ipated in the study. The bar was divided into eleven work areas with check lists indicating completion of cleaning tasks for each area. After the employees cleaned, data were collected with the number of check marks producing a percentage, the dependent variable for the study. Intervention included a task clarification checklist that was always visible to employees. The check list was explained for the required cleaning tasks. The checklists were located in every work area. Employees were then divided into 3 groups and given visual feedback through charts placed on a wall in the bar. Feedback was staggered between the three groups. A multiple baseline across groups design was used to assess the intervention. Baseline data displayed a downward trend, which was reversed during the task clarification phase and continued to increase with feedback phase. In addition to the three components previously mentioned (feedback, goal setting, task clarification), performance contingent consequences have also been added to intervention packages. In a study by Slowiak, Madden, and Mathews (2005), the effects of the intervention package were examined in relation to telephone customer service in a medical clinic. Greeting, friendly voice tone, and closing were the t hree customer service behaviors targeted.
11 An ABAB reversal design was used to assess the effects of the intervention package. The intervention consisted of a job aid, placed as a visual reminder, the location of the three target behaviors. Task clarifica tion was given as a handout to participants describing the telephone customer service standards of the clinic. Definitions of the three target behaviors were given as well. Goals were also set for all three behaviors based on baseline data. Performance f eedback was also part of the intervention package. Feedback was given twice a week by email to each individual, in the form of a bar graph of the three behaviors and goals. If goals were met the individual was given a list of bonus items given as a rewar d. Data indicated an obvious increase from baseline to the first intervention phase; when reversed to baseline, there were still increased effects from the previous phase at the same level as improvement during intervention. A contingent consequence withi n a lottery system was added in a study by Cook & Dixon (2005), which extended the effects of verbal feedback to graphic feedback and a lottery for financial rewards to assess their effects on the completion of forms. The participants included three adult supervisors in an agency that served individuals with developmental disabilities by providing group homes. Direct care staff completed the forms that were used during the study. The four forms included a shift summary report, daily observation report, b ehavioral tracking sheet, and a program task analysis sheet. A multiple baseline across participants design was used to assess the effects of the three different intervention conditions. During baseline, the participants were to perform their normal dail y duties. In the verbal feedback condition the investigator and participant would meet once per week and give their score on the level of completeness on each form for that week. During the comparative graphic feedback condition, a graph was given to the participant weekly, displaying their progress along with the other two
12 participants who names were not reported. When a lottery system was added in the third condition, whichever participant had the highest score on their level of completeness for the we ek received three lottery tickets, then 2 tickets for second highest score, and one for the last participant. At the end of each week, a $50 cash prize was given to the winner whose ticket was drawn. Results indicated that the verbal feedback condition increased the mean percentage of forms completed. When graphic feedback was added to verbal feedback in the second condition no significant increases in percentages were seen. The highest increases were seen in the lottery condition for all participants. Performance feedback with or without the other two conditions added showed improvements in the each A study by Loewy & Bailey (2007) incorporated all these successful interventions to assess their effects on customer service behaviors. The study took place in two national home improvement chain store locations; each location had about 150 employees. Greeting, eye contact, and smiling were the dependent measures for this study. Observations were recorded in the front of the store at other high traffic areas within the store. Data were collected on an average of once out of every three days. The intervention was assessed using an A B C and multiple baseline design. During the graphic feedback condition, group performance levels were posted outside their break room. Mangers prompted staff members to look at the graphs. The goal was also posted on the graphs, marked by a red dashed line. When the goal had been met for the posted feedback, the manager would give written feedback on the graph
13 itself. Managers also gave verbal praise to individual staff members when they were observed engaging in one of the targeted beha viors (greeting, eye contact, and smiling). Results indicated a slight improvement in performance during the graphic feedback condition, and then a decrease. Little change was indicated during the second condition. The researches propose that this sligh t change in performance may have been due to a lack of consequences. A study by Godbey & White (1992), examined the effects of behavior monitoring, setting objectives, giving feedback, and praise on the accuracy of computerized summaries of court case ac tivity. Five staff members of the local court system served as participants in the study. Audit procedures were used as the measurement procedure for the study, since during the audits the accuracy of the first 100 filing documents would be examined. Dur ing baseline, participants were instructed to improve the accuracy of computerized summaries of court activity. In this condition, two audits were conducted and used for baseline data. An ABCA design was used to assess the intervention. During the firs t intervention phase, group meetings were held. At this time, staff members were asked to identify behaviors they could utilize in to reduce the errors shown in the computerized summaries. Participants were given assignments for each week and then praise d upon completing assignments in an individual memo. The author of the study provided prompts and verbal praise for working on assignments. After six weeks, the study procedures reversed to baseline. The intervention condition was re established and mod ified so there were no more meetings or memorandums. The intervention still consisted of monitoring computerized summaries and informal weekly conversations. Results of the study indicated an increase in accuracy during the first intervention phase. Whe n the study reversed to baseline the percentages stayed the same and showed an
14 increase during the second intervention study. The ABAC design leads to sequence effects which were likely demonstrated in the study. Prompting is yet another component that h as been added to feedback packages. The Milligan and Hantula (2004) study found that prompting alone can effect performance. The study had only one participant who was owner/operator of a pet grooming store. The owner recorded grooming and sales on index cards in the store. These cards were then used as prompts for additional purchases. The prompts were written on the back of the cards. and turned over so that prompt was visible to the owner. A research assistant recorded data while dressed as a grooming assistant. An treatments phase, the owner re corded normal store procedures on index card, along with whether or not a prompt was used. During the full prompt phase, the no prompt cards were taken out. Results indicated that the use of the index cards increased the owners prompting behavior for as king customers to buy additional products. In the 2007, Squires et al. study examined 10 employees of a restaurant. The target behaviors were greeting (verbal acknowledgement of customer within 3 sec) and up selling (asking the customer to purchase addit ional items). Students observed and recorded data about the two b ehaviors while seated at a near by table pretending to read. A multiple baseline across behaviors with a reversal design was used to assess the effects of task clarification, prompts, and f eedback on target behaviors. During task clarification, both greeting and up selling behaviors were described and examples of correct behavior were modeled. The visual prompts were two posters; one for greeting
15 and another for up selling. The posters we (greet/up percentage of group performance. The graph was updated daily. All three intervention conditions were effective in increas ing the two target behaviors. When the intervention reversed back to baseline, target behaviors decreased. Structure of the design was open for sequence effects. The performance improvement package procedures, including written and graphic feedback, pro mpting, praise, and contingent consequences have been effect in the management of behaviors. Current Study The purpose of this current study was to expand upon the previous performance improvement literature. A performance improvement package with the components of goal setting, prompting, and feedback was used to improve dollars billed to Medicaid for services provided by school psychologists, and to increase the number of school psychologists who turned in their required documentation for Medicaid bil ling. Experimental control was demonstrated within a multiple baseline design across three large service areas of a county school system.
16 Method Participant s and Setting The study took place within the Hillsborough School District which has an enr ollment of 192,749 students. The school district was divided into 7 geographical areas of schools that include the pre k through secondary level. School psychologists were assigned schools within areas to provide psychological services, and three of the seven areas were the focus for the study. Area A employed 24 school psychologists. Area C had 25 school psychologists and Area B had 25 school psychologists, for a total of 74 psychologists. A school psychologist must have had at least a mast for the three areas selected included the potential for funds to be generated because the schools had a substantial number of ESE students enrolled. Ac cording to the September 2008 enrollment count there were 6,468 ESE students enrolled in the district, which is 3.4% of the total students. The settings for the study included the various schools that were within each of the three areas, from which the sch ool psychologists were assigned. Another location was the school district administrative building where the supervisor for Psychological Services was located, along with the Medicaid building where Medicaid functions were performed. 34 individual schools were located within Area A, Area B had 38 schools, while Area C included 30 individual schools, for a total of 102 schools.
17 Prior to the beginning of this research, approval was obtained from the Hillsborough School District and The Institutional Review Board (IRB) at the University of South Florida Performance Analysis A performance analysis was conducted prior to baseline. The Performance Diagnostic Checklist (PDC) (Austin, 2000) was used to assess areas for performance improvement within duties of b illing among the school psychologists. The analysis was based on direct observation and interviews with several school psychologists, the Supervisor of Psychological Services, and an employee of the Medicaid budget department. The PDC included four secti ons; antecedents and information, equipment and processes, knowledge and skills, and consequences. Results of the assessment indicated that the school psychologists did not have a system in place for antecedents, information, and consequences. The scho ol psychologists were not given frequent antecedent stimuli, such as there were no prompts or reminders to turn in billing. Also, goals were not set for the completion of billing. The school psychologists did not have adequate information on where the do llars for billing were dispersed once reimbursed. Finally, school psychologists were provided with infrequent and inconsistent feedback on their billing performance, along with no performance contingent consequences. Medicaid System Hillsborough School D istrict participated in a state wide program for schools known as Medicaid Certified School Match (MCSM) program. The program matched reimbursement for services provided to a student were:
18 1. Student must be in the ESE program: To be an ESE student they must meet the criteria of deaf/hard of hearing, emotional/behavioral disability, autism spectrum disorder, physically impaired with orthopedic impairments, physically impaire d with other health impairments, or physically impaired traumatic brain injury. 2. Student identified as ESE must have an Individual Education Plan (IEP). The IEP is developed with parental involvement. The IEP identifies individual goals that need to be ad dress based on assessment data. Program monitoring is required, and services must be provided in the least restrictive environment. Other sections of the IEP include present level of performance, academic goals, and teaching strategies. 3. Student must be u nder 21 4. Disabled under IDEA: determined to have handicap condition under guidelines for IDEA federal legislation. 5. Student must be Medicaid eligible: parent/guardian has completed a Medicaid application at the Florida Department of Children and Family. The department receiving Social Security Income (SSI ) is automatically eligible for Medicaid. The MCSM program covered a variety of services, such as nursing, social work, and behavio r analysis. This study focused on the behavior services provided by school psychologists. The school psychologists billed under three categories: group service, individual service evaluation, or individual service all else. The MCSM Coverage and Limitat Medicaid eligible student, regardless of which service or combinations of services are being rendered, the school district must bill for individual behavioral services...a grou p of
19 psychologist was required to document service activity on an electronic or paper billing form. A single form could contain multiple students/services provided so that it was not necessary to fill out a form for each service activity. System Analysis The billing procedures operated as follows: First, the school psychologists ascertain if the student was on the Medicaid eligible list. Once a child was found to be Medicai system. At the end of each month Information Systems (IS) sent the Budget Department at Medicaid a list of the current Exceptional Student Education (ESE) students, the inform ation was downloaded into the Medicaid tracking system. The Medicaid Budget Department downloads the list from the MTS, downloads the information into an Access file and put the file into the IDEAS system, which was the computer communications system wi thin the school district. There was a Medicaid section located in IDEAS (Internal District Electronic Access System), where the school psychologist could electronically retrieve and pull for a current list of Medicaid eligible students for billing purpose s. The school psychologist could also download the Medicaid billing form from the Medicaid section in IDEAS. Once downloaded, the school psychologist completed the form and sent it by email or mail to the Medicaid budget department. The completion of al l relevant forms would take each participating school psychologist 5 15 minutes each fortnight. When a completed form was received at the Medicaid office, an employee inputted the information from the forms into a spreadsheet. Information on the spread
20 name, date of service, procedure code, units, and diagnosis code. Numerous forms were to Tallahassee, FL, which was the fiscal agent. The information was processed in Tallahassee and a check was sent electronically back to Medicaid for the school district of Hillsborough County containing 58% of the amount billed. For example, when Medic aid batched on Wednesday, the money would arrive on Monday. That money was priorities. Data collection and Dependent V ariable s Every two weeks (2 fortnights) data was collected through the Medicaid tracking system. The amount of billed Medicaid dollars was the primary dependent variable. The amount was calculated by multiplying the units of time by the procedure code amount. 15 minutes was equivalent to 1 unit. The three procedural codes; group services, individual evaluation, and individual all else had designated fees. The groups service and individual all else were $9.66 per unit, while group service was $4.95 per student, per unit. There was a proto col to round up the time when billing the units. For example, a school psychologist billed for 16 units (4 hours) for an individual evaluation which cost $10.00 a unit. The billed amount calculated to $160.00. A secondary dependent variable was the numb er of school psychologists who turned in billing Inte robserver Agreement The data were assessed for interobserver agreement by having a second person independently score the data from 6 of the 19 two week periods (32%). The reliability observer scored da ta sets in an order determined from a table of random numbers with the
21 requirement that interobserver assessment was assessed on at least two data points in each experimental condition for each area. The order of scoring; Area A: 10, 8, 9. 12, 11, 1. Are a B: 4, 8, 2, 10, 17, and 16 Area C: 19. 9, 2, 5, 3, 17. The IOA score was calculated by two week periods as a percentage. The percentage was calculated by taking the smallest total score divided by the largest total score total multiplied by 100, and there were 6 scores for each Area, for a total of eighteen scores. The eighteen scores were added and divided by eighteen for the mean scores and the overall reliability scores. IOA scores were calculated for dollars billed to Medicaid for reimbursement by school psychologists and the number of school psychologists who turned in billing. The Medicaid tracking system also includes a Quality Control file, which takes 30 forms at random to check the billing amount with the system. IOA was calculated agre ement by two week periods, as a percentage. Social Validity A social validity questionnaire was administered by the Supervisor of Psychological Services to the school psychologist after the study had been conducted. The questionnaire contained six questio ns for the school psychologists to rank on a five point scale: 5=Agree, 4=somewhat agree, 3=neutral, 2=somewhat disagree, 1=disagree. The questions related to information provided by the supervisor, acceptability of performance improvement package, value of completing billing, and the likelihood of continuity of participation in the program. Experimental Design A multiple baseline across the three school district areas was used to evaluate the effects of the intervention. The intervention was implemented in a staggered manor across the areas to demonstrate experimental control. Experimental control was
22 demonstrated when an intervention was implemented, and a behavior change was displayed for billing by psychologists in that area only. The logic of singl e case design could be applied to between group comparisons using the repeated measures within a multiple baseline design according to Kazdin (1982, pg. 229). Procedure Baseline. At the beginning of the school year all school psychologists received tra ining in Medicaid documentation procedures. Training was conducted by a senior basics of Medicaid, instructions on how to retrieve the Medicaid eligibility list online th rough IDEAS and how to complete the billing form. During baseline, the school psychologists were required to turn in billing within a one year period. The Supervisor of Psychological Service did not provide any feedback to the school psychologists on the ir billing activity. Performance Improvement Package. During the intervention phase a performance improvement package was implemented. The package was include goal setting, prompting, and feedback. 1. Goal setting. An email was sent out to the school ps ychologists in the areas targeted by the intervention by the supervisor of psychological services stating that a new goal for Medicaid billing activity was to be put into place. The new goal required school psychologists to complete documentation for Medi caid billing on a weekly basis; this differed from baseline, which required billing on a yearly basis. The email content follows: Hello Area___
23 Over the past 4 years, Medicaid reimbursement dollars have been lower than previous years. I would like to s ee those dollars increase, but need your help. I ask that everyone who has provided services to a Medicaid eligible, ESE student please complete billing in a timely fashion. Our goal is to consistently complete required documentation for Medicaid billing at the end of each week. Last year I sent a similar email out to school psychologist in Area 2, by providing a goal, prompting, and feedback on my end, they were able to increase their monthly billing by thousands of dollars! I understand that this may require and me, I know your Area will also increase dollars billed. An increase in dollars billed means more money is reimbursed for the school district. I appreciate t hose who are already extending the effort to consistently complete documentation. I look forward to meeting our goals of completing required documentation on a weekly basis, and most importantly increasing Medicaid reimbursement dollars. Best Supervisor o f Psychological Services 2. Prompting. Prompts were sent out every Friday morning by the Supervisor of Psychological Services. Prompts were sent electronically to the school k. The prompt content was as follows: Hello Everyone This is just a reminder to complete the required documentation for Medicaid billing. Thank you!
24 3. Feedback. Group written and graphic feedback was given to the school psychologists every two weeks base d on their billing performance. A written statement of praise or need for improvement was sent out electronically by the Supervisor of Psychological Services which contained two graphs, one displayed the billed Medicaid dollars, and the second graph displ ayed the number of school psychologist who turned in billing. Positive feedback was provided routinely, except on those occasions participation or dollars billed were judged by the experimenter as not showing improvement consistent with the goal set or sh owed a decline below the level set by the goal. The email content included: Dear Area___ I appreciate your effort in the Medicaid billing these last few weeks. I want to thank you for completing your documentation for Medicaid billing and keeping our g oal in mind. dollars billed for Medicaid reimbursement. Thanks! Supervisor of Psychological Service Dear Area ___ I appreciate your effort in completing billing each week. From the information given to us by Medicaid, it seems that very little billing has been sent in. I know that documentation activity requires extra work, but I do hope you keep our goal in mind. I have attached a graph, which displays our billed dollars to Medicaid. Thanks Supervisor of Psychological Services. The additional time required by the Supervisor of Psychological Services for implementation of these procedures was minimal, less than a few minutes per fortnight. The supervisor received a report in a word document containing a summary of graphs and
25 information. The supervisor attached the report in an email and sent the group feedback to each area as appropriate. The experimenter received a summary of information in a spread sheet format from the Medicaid budget department, collated the information and graphed it. The graphs and the content of feedback were sent by email to the Supervisor of Psychological Services. This process required no more than thirty minutes of the ime each fortnight. The fiscal analyst who was responsible for Medicaid billing continued his activities without change, except that batching was placed on a routine. The batched information was sent to the experimenter each fortnight to be reviewed and sent to the Supervisor of Psychological Services. Additional activities required only a few minutes.
26 Results Dependent Variables Dollars Billed Based on the data collected across 19 billing periods, the performance improvement package increased the amount of dollars billed to Medicaid for all three Areas. Figure 1 displays the total amount of dollars billed to Medicaid for all three Areas. The x axis spans across 19 billing periods, each billing period equals 2 weeks. The y axis is t he total in dollars billed. In Area A, the baseline mean= $322.62; sequential introduction of the performance improvement package, billing consistently increased followi ng the change of experimental conditions. During the performance improvement package the average of dollars billed in all three Areas increased; Area A, M= $1,984.96; Area B, M= $4,293.53; Area C, M= $19,106.87. Overall the package lead to an improvement from a baseline data mean of $1,028.12 to a performance improvement package mean on $23,226.47, which meant that under the performance improvement package billing averaged $22,198.35 more per billing period in the performance improvement package condition s. In fact, the 38 baseline periods resulted in $40, 538.00 totals in billing, whereas the 19 intervention periods totaled in $98,770.42 total. Medicaid billing is reimbursed by the state 58% of the amount billed.
27 Figure 1. Dollars Bi lled by School Psychologists across three Areas. Area C Area B Area A Baseline Performance Improvement Package
28 Number of School Psychologists Based on the data collected throughout the study, the performance improvement package increased the number of school psychologists who turned in billing to Medicaid within a two week billing period. Figure 2 displays the number of school psychologists who turned in billing to Medicaid. During baseline the average number of school psychologist turning in billing was; Area A, M= .6; Area B, M= 1.08; Area C, M=1.88. The ave rage number of school psychologists turning in billing increased to Area A, M=4.33; Area B, M= 6.33; Area C, M=5.67 during the intervention phase. Therefore, during baseline conditions 38% of psychologists participated per billing period, whereas during t he performance package, 58% of the psychologists participated actively in the program per billing period.
29 Figure 2. Number of school psychologists who turned in documentation for Medicaid b illing. Area C Area B Area A Baseline Performance Improvement Package
30 Interobserver Agreement The IOA score was calculated as a percentage in two week periods. The percentage was calculated by taking the smallest total score divided by the largest total score and multiplied by 100; there were 6 scores for each Area, for a total of eighteen scores. Two disagr eements were identified for the dollar amount scores, a difference totaling $.24 and $106.26, with an overall IOA score of 99.77%. There was one disagreement in the number of school psychologists who billed, a difference of one, with an overall IOA score of 99.3%. Social Validity Based on the results of the School Psychologist Questionnaire, the school psychologists agreed that the Supervisor of Psychological Services provided them with information on how to complete billing, feedback and graphs were u nderstood, and it was recommended that the supervisor continue to provide feedback on billing activity. When asked if the goal was acceptable and obtainable, 46% of the school psychologists agreed or somewhat agreed, 23% were neutral, and 30% somewhat dis agreed. 65% of the school psychologists said they would continue to bill on a weekly basis, while one school psychologist reported the goals were unacceptable and unobtainable, and would not continue to bill Medicaid on a weekly basis.
31 Table 1 Social Va lidity Questi onnaire for School Psychologists Completed by 18% of school psychologists Average Percentage Scores N=13 5 Agree, 4 Somewhat Agree, 3 Neutra, 2 Somewhat Disagree, 1 Disagree a. My supervisor provided me with information I needed to complete my Medicaid billing. 5 4 3 2 1 92% 7% 0% 0% 0% b. I understood the feedback and graphs that my supervisor provided for me. 5 4 3 2 1 76% 7% 7% 7% 0% c. The goal to complete my Medicaid billing on a weekly basi s is acceptable and obtainable. 5 4 3 2 1 23% 23% 23% 23% 7% d. performance. 5 4 3 2 1 46% 54% 0% 0% 0% e. It is important to be informed on the Medicaid bil led amounts. 5 4 3 2 1 54% 31% 15% 0% 0% f. I will continue to complete my Medicaid billing on a weekly basis. 5 4 3 2 1 23% 38% 31% 0% 7%
32 Discussion The current study evaluated the effects of a performance improveme nt package on the documentation of Medicaid billing by school psychologists across ten months of the school year. The results of the study showed that, consistent with the multiple baseline design, the sequential introduction of the performance improvemen t package was followed by prompt increases in the dollars billed to Medicaid by school psychologists. In addition, the number of school psychologists who turned in billing for Medicaid reimbursement increased. These changes resulted in a substantial incr ease in income for the school district. The increase in the income for the district averaged $12,875 per fortnight with the total investment of additional time of two four hours of all personnel involved. These benefits were obtained in a cost efficient manner even though only 58% of school psychologist participated during the intervention condition. These results demonstrate an excellent return on investment of time and resources. The results of introducing the performance improvement package consisti ng of goal setting, prompts, and group written and graphic feedback improved billing to Medicaid by school psychologists support the findings by Balcazar, Hopkins, & Suarez (1985) and by Alvero, Bucklin, & Austin (2001) who showed that the additional compo nents of graphic feedback, prompts, and praise added to feedback will increase success. This study was valuable because the education system is in need of additional funding and the performance improvement package utilized the money the school systems may have readily available. This study was the first to assess the effects of a
33 performance improvement package on documentation for Medicaid billing within a school system. The findings of this study support a performance improvement package as an effectiv e, and social valid intervention to increase the dollars billed to Medicaid, and the number of school psychologists who turned in billing. The study was also valuable in the context of shrinking school budgets. In fact, during the conduct of this study s the accompanied reduction in salary and associated decline in morale. Experimental control was demonstrated within the multiple baseline design. Stable baselines were establishe d prior to intervention, with Area B showing greatest variability. Following the introduction of the intervention, the first data point showed small if any effect and a much larger increase in the second data point of intervention. After the initial subs tantial increases the level of the data decreased and stayed at a level consistently higher than baseline. The increase from the first to the second intervention data point appears to be a result of having received the first feedback on performance, even though the intervention began two weeks earlier with the introduction of goal setting and prompting. Thus, the intervention package had been fully implanted prior to the second data point of the intervention. It should also be noted that there were large increases in the dollars billed at the beginning of the intervention, and these effects were transitory. Thus, the initial increase of dollars billed were likely the result of the school psychologists having months of billing documentation yet to be comp leted and turned in to Medicaid. Prior to the performance improvement package the school psychologists were required to turn in Medicaid billing documentation within a one year time frame, while the intervention package set a goal for billing to be turned in at the end of each
34 week. Once the pile up of documentation had been billed there would be a decrease in the dollar amount, which was demonstrated in the data in figure 1. A benefit of this study was that the performance improvement package component s fit within the system without excessive additional time needed from all participants involved. The time required by each school psychologist was less than 15 minutes per fortnight, and the additional time required for administration and supervisor time was 35 minutes per fortnight. The Medicaid budget department suggested the use of the performance improvement package to be utilized within the social work and nursing departments that also bill to Medicaid for reimbursement. Before the performance impr ovement package is adapted to a broader area a recommendation would be to bill on a monthly basis rather than a weekly basis. A pilot study was conducted prior to this study containing a similar performance improvement package, during the pilot study scho ol psychologists were required to turn in billing on a monthly basis rather than weekly during the intervention phase. Results from the pilot study showed that school psychologist averaged $5,000 billed to Medicaid each month. In the current study, the d ollars billed to Medicaid maintained around an average of $2,000 every two weeks, totaling an average of $4,000 every month. The two studies came to similar results, one of which billed monthly and the other weekly showing that the school psychologists ap parently do not need to bill on a weekly basis in order to see successful results. Based upon the outcomes of this research the following recommendations can be made. The performance improvement packages components of goal setting, prompting, and feedb ack should be incorporated in ongoing programs. Feedback to a group rather than to individuals is adequate and recommended. The goal setting should be for performance improvement rather than absolute numbers for consistency across areas
35 where they may be varying possibilities for billing, e.g. special education center versus regular school. Forms completed by school psychologist for Medicaid billing should allow for multiple students in recording efficiently on the same page for optimal use of time. So cial validity data showed that the program was well received, informative, provided relevant information, important, and the school psychologists were likely to continue the program. However, the percentage of school psychologists who returned the social validity questionnaire was low at 13 percent of school psychologists who participated. This was probably due to timing of administration, being after the school year had concluded. The reaction school district administration may be characterized by one a dministrators comment that such a large change in dollars generated that it cannot be ignored. One limitation to the study was the experimenter analyzed and graphed the data as it came through the Medicaid tracking program. If the performance improvement package is to be maintained, this may create the need for a systems consultant or training of a current employee to analyze and graph data for feedback. Only a small amount of additional time is needed to analyze and graph the data, approximately one hou r each month. The additional employee would need to be trained and skilled in Microsoft Excel or a similar spreadsheet and graphing program. Another limitation to this study was the performance improvement package itself. The results of this study cann ot determine if one component (goal setting, prompting, or feedback) was attributed to the increase in the two dependent variables. The data show that when goal setting and prompting had been implemented alone prior to the first intervention data point th e outcomes were smaller than during subsequently billing
36 periods after which the feedback component was added. However, given that the package of procedures was easy to administer with little time involve and the effects were robust, then component analys is does not need to be warranted. Future research should collect data on the day billing was received at the Medicaid budget department and compare those data to the day the school psychologist provided services to the Medicaid eligible student. The fu ture research idea would offer a detailed system analysis of the date billing and services were completed. Future research could also assess the maintenance of effects. During the pilot research, it was found that following withdrawal of the performance improvement package dollars billed declined to baseline level. This suggests that the program needs to continue or other maintenance strategies need to be examined.
37 References Alvero, A., Bucklin, B., & Austin, J. (2001). An objective re view of the effectiveness and essential characteristics of performance feedback in organizational settings (1985 1998). Journal of Organizational Behavior Management, 21 3 29. Amigo, S., Smith, A., & Ludwig, T. (2008). Using task clarification, goal set ting, and feedback to decrease table busing times in a franchise pizza restaurant. Journal of Organizational Behavior Management, 28, 176 187. Anderson, D.C., Crowell, C.R., Hantula, D.A., & Siroky, L.M. (1998). Ta sk clarification and individual perform ance posting for improving cleaning in a student managed university bar. Journal of Organizational Behavior Management, 9 73 90. Aubrey Daniels International (ADI). Why should business leaders use organizational behavior management? Retrieved March 23 2008, from http://www.obmnetwork.com/leader/whyuseobm/#adi Austin, J. (2000). Performance analysis and performance diagnostics. In J.Austin & J.E. Carr (Eds) Handbook of applied behavior analysis (pp.321 349). Reno, NV: Context Press. Balcazar, F., Ho pkins, B.L., & Suarez, Y. (1985). A critical, objective review of performance feedback. Journal of Organizational Behavior Management, 7 65 89. Brown, K. M., Willis, B.S., & Reid, D.H. (1981). Differential effects of supervisor verbal feedback and feed back plus approval on institutional staff performance. Journal of Organizational Behavior Management, 3 57 68.
38 Calpin, J. P., Edelstein, B., & Redmon, W.K. (1988). Performance feedback and goal setting to improve mental health center staff productivity. Journal of Organizational Behavior Management, 9 35 58. Cook, T., & Dixon, M.R. (2005). Performance feedback and probabilistic bonus contingencies among employees in a human service organization. Journal of Organizational Behavior Management, 25 45 63. Daniels, A.C., & Daniels, J.E. (2008). Performance Management: Changing Behavior That Drives Organizational Effectiveness Atlanta, GA: Aubrey Daniels International, Inc. Department of Student Support Services (2005). Medicaid Certified School Match Co verage and Limitations Handbook. Godbey, C. L., & White, A.G. (1992). Increasing the accuracy of computerized summaries of court case activity. Journal of Organizational Behavior Management, 13 113 127. Goltz, S.M., Citera, M., Jensen, M., F avero, J.,& Komaki, J.L. (1989). Individual feedback: does it enhance effects of group feedback? Journal of Organizational Behavior Management, 10 77 92. Hawkins, A. M., Burgio, L. D., Langford, A., & Engel, B. T. (1992). The effects of verbal and writt en supervisory feedback on staff compliance with assigned prompted voiding in a nursing home. Journal of Organizational Behavior Management, 13 137 150. Jones, H., Morris, E.K. & Barnard J.D., (1986). Increasing staff completion of civil commitment for ms through instructions and graphed group performance feedback. Journal of Organizational Behavior Management, 7 29 43.
39 Kazdin, A, E. (1982). Single Case Research Designs New York, NY: Oxford University Press. Loewy, S., & Bailey, J.S. (2007). The eff ects of graphic feedback, goal setting, and manager praise on customer service behaviors. Journal of Organizational Behavior Management, 27 15 26. Milligan, J. & Hantula, D.A. (2005). A prompting procedure for increasing sales in a small pet store. Journ al of Organizational Behavior Management, 25 37 44. Slowiak, J.M., Madden, G.J., & Matthews, R. (2005). The effects of a combined task clarification, goal setting, feedback, and performance contingent consequence intervention package on telephone custome r service in a medical clinic environment. Journal of Organizational Behavior Management 15 35. Squires, J., et al. (2007). The effects of task clarification, visual prompts, and graphic feedback on customer greeting and up selling in a restaurant. Jour nal of Organizational Behavior Management, 27 1 13. Tittelbach, D., DeAngelis, M., Sturmey, P., & Alvero, A.M. (2007). The effects of and customer service. Journal of O rganizational Behavior Management, 27 27 37. Tittelbach, D., Fields, L., & Alvero, A.M. (2007). Effects of performance feedback on typing speed and accuracy. Journal of Organizational Behavior Management, 27 29 52. Wilk, L.A. & Redmon, W.K (1990 ). A daily adjusted goal setting and feedback procedure for improving productivity in a university admissions department. Journal of Organizational Behavior Management, 11 55 75.
41 Appendix A: A social validity questionnaire given to the school psychologists Name:______________________________ Date:_______________________________ Questions for the School Psychologists to Answer: 5 Agree, 4 Somewhat Agree, 3 Neutra, 2 Somewhat Disagree, 1 Disagree Please circle the rate number fo r each of the following questions. a. My supervisor provided me with information I needed to complete my Medicaid billing. 5 4 3 2 1 b. I understood the feedback and graphs that my supervisor provided for me. 5 4 3 2 1 c. The goal to complete my Medicaid billin g on a weekly basis is acceptable and obtainable. 5 4 3 2 1 d. performance. 5 4 3 2 1 e. It is important to be informed on the Medicaid billed amounts. 5 4 3 2 1 f. I will continue to comple te my Medicaid billing on a weekly basis. 5 4 3 2 1