xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam 2200397Ka 4500
controlfield tag 001 002069370
007 cr mnu|||uuuuu
008 100421s2009 flu s 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0003229
Prewett, Matthew S.
Training teamwork in medical teams :
b an active approach with role play and feedback
h [electronic resource] /
by Matthew S. Prewett.
[Tampa, Fla] :
University of South Florida,
Title from PDF of title page.
Document formatted into pages; contains 84 pages.
Dissertation (Ph.D.)--University of South Florida, 2009.
Includes bibliographical references.
Text (Electronic dissertation) in PDF format.
ABSTRACT: Recent reports in the field of medicine have recommended the use of teamwork training to reduce the number of injuries and fatalities from human error. Teamwork training in the field of medicine appears promising, but few empirical evaluations of such programs have confirmed their effectiveness. Existing teamwork training studies have tended to use a traditional, lecture approach to training, with positive but modest results upon teamwork attitudes and behaviors. The current study developed and evaluated a more active teamwork training protocol for trauma resuscitation teams. The training protocol supplemented several medical and non-medical role plays with a lecture and guided discussion for feedback. Forty-one residents participated in the training on one of two days (groups) and completed evaluation measures prior to and immediately following the training program. The training was evaluated with measures of trainee reactions, attitudes towards teamwork, and responses to a situational judgment test (SJT). Analyses compared item and scale scores between pre-training scores and post-training scores. T-tests generally found higher means for post-training behavioral responses than pre-training responses. However, mean comparisons with teamwork attitudes and learning goal orientation did not yield significant differences. An item analysis of the SJT responses (using chi-square) indicated significant response shifts in many items that correspond to the teamwork training content. In summary, results indicated that teamwork training on behavioral choices, but little effect on the self-reported attitudes of trainees.
Mode of access: World Wide Web.
System requirements: World Wide Web browser and PDF reader.
Advisor: Michael T. Brannick, Ph.D.
Learning goal orientation
t USF Electronic Theses and Dissertations.
Training Teamwork in Medical Teams: An Active Approa ch with Role Play and Feedback by Matthew S. Prewett A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy Department of Psychology College of Arts and Sciences University of South Florida Major Professor: Michael T. Brannick, Ph.D. Michael D. Coovert, Ph.D. Laura Haubner, M.D. Russell E. Johnson, Ph.D. Joseph Vandello, Ph.D. Date of Approval: November 16, 2009 Keywords: team attitudes, lear ning goal orientation, trauma room, team process, reactions Copyright 2009, Matthew S. Prewett
Dedication This doctoral dissertation is dedicated to my family, friends, and colleagues, especially my parents, Patricia and Mike Prewett, w ho gave me all that I needed to succeed.
Acknowledgments I would like to thank the following people, without whom this disse rtation would have never been possible: Dr. Michael Brannick Michael T. Coovert, Ph.D. Laura Haubner, M.D. Bradley Peckler, M.D. Russell Johnson, Ph.D. and Joseph Vandello, Ph.D.
i Table of Contents List of Tables ..................................................................................................................... iii List of Figures ............................................................................................................... ..... iv Abstract ...................................................................................................................... ..........v Chapter One: Introduction ...................................................................................................1 A Review of Teamwork Training ............................................................................3 Active Approaches to Training ................................................................................6 Guided Exploration: Role Play and Feedback .............................................7 Mastery Training ..........................................................................................9 Error Management Training ......................................................................10 Training Evaluations ..............................................................................................11 Reactions ....................................................................................................12 Learning .....................................................................................................13 Behaviors ...................................................................................................15 Outcomes ...................................................................................................18 Training Context: Trauma Teamwork ...................................................................19 Study Purpose and Training Design ......................................................................22 Chapter Two: Method ........................................................................................................24 Design ....................................................................................................................24 Sample....................................................................................................................24 Training Content ....................................................................................................25 Measures ................................................................................................................27 Pre-training Questionnaire .........................................................................27 Learning Goal Orientation .........................................................................28 Teamwork Attitudes ...................................................................................28 Teamwork Behaviors .................................................................................28 Reactions ....................................................................................................29 Procedure ...............................................................................................................29 Analyses .................................................................................................................30 Chapter Three: Results .......................................................................................................32 Results for Group Comparisons .............................................................................34 Item Analyses for the Situational Judgment Test ..................................................35
ii Chapter Four: Discussion ...................................................................................................49 Limitati ons .............................................................................................................52 Summary ................................................................................................................54 References .................................................................................................................... ......55 Appendices .................................................................................................................... .....65 Appendix A: Science Ficti on Role Play Exercise ..................................................66 Appendix B: Wilderness Survival Exercise ...........................................................71 Appendix C: Pre-trai ning Questionnaire ...............................................................76 Appendix D: Questionnaire on L earning Goal Orientation ...................................77 Appendix E: Teamwork in Medicine Attitudes Questionnaire ..............................78 Appendix F: Situational Judgmen t Test (SJT) for Teamwork in the Trauma Room ........................................................................80 Appendix G: Reactions to Training Session ..........................................................84 About the Author ................................................................................................... End Page
iii List of Tables Table 1 Descriptive Statisti cs and Inter-cor relations for Self-Report Measures .................................................................................33 Table 2 Descriptive Statistics and Inter-correla tions for the First Training Group ..................................................................................33 Table 3 Descriptive Statistics and Inter-correlations for the Second Training Group...........................................................................................33 Table 4 Preand Post-Training Response Frequencies for SJT Item 1 ..................37 Table 5 Preand Post-Training Response Frequencies for SJT Item 2 ..................37 Table 6 Preand Post-Training Response Frequencies for SJT Item 3 ..................37 Table 7 Preand Post-Training Response Frequencies for SJT Item 4 ..................38 Table 8 Preand Post-Training Response Frequencies for SJT Item 5 ..................38 Table 9 Preand Post-Training Response Frequencies for SJT Item 6 ..................38 Table 10 Preand Post-Training Response Frequencies for SJT Item 7 ..................39 Table 11 Preand Post-Training Response Frequencies for SJT Item 8 ..................39 Table 12 Preand Post-Training Response Frequencies for SJT Item 9 ..................39 Table 13 Preand Post-Training Re sponse Frequencies for SJT Item 10 ................40 Table 14 Preand Post-Training Re sponse Frequencies for SJT Item 11 ................40 Table 15 Preand Post-Training Re sponse Frequencies for SJT Item 12 ................40 Table 16 Preand Post-Training Re sponse Frequencies for SJT Item 13 ................41 Table 17 Preand Post-Training Re sponse Frequencies for SJT Item 14 ................41 Table 18 Preand Post-Training Re sponse Frequencies for SJT Item 15 ................41
iv List of Figures Figure 1. Boxplots of Teamwork, Learning Goal Orientation, and Reactions Scale Scores ........................................................................34 Figure 2. Frequencies for Most Likely Response for SJT Item 3 .............................42 Figure 3. Frequencies for Most Likely Response for SJT Item 4 .............................43 Figure 4. Frequencies for Most Likely Response for SJT Item 7 .............................43 Figure 5. Frequencies for Least Likely Response for SJT Item 9 .............................45 Figure 6. Frequencies for Most Likely Response for SJT Item 14 ...........................45 Figure 7. Frequencies for Most Likely Response for SJT Item 12 ...........................46 Figure 8. Frequencies for Least Li kely Response for SJT Item 15 ...........................46
v Training Teamwork in Medical Teams: An Active Approa ch with Role Play and Feedback Matthew S. Prewett ABSTRACT Recent reports in the field of medicine have recomme nded the use of teamwork training to reduce the number of injuries and fatalities from human error. Teamwork training in the field of medicine appear s promising, but few empiri cal evaluations of such programs have confirmed their effectiveness. Existing teamwork tr aining studies have tended to use a traditional, lecture approach to training, with positiv e but modest results upon teamwork attitudes and behaviors. The current study developed and evaluated a more active teamwork training protocol for trauma resuscitation teams. The training protocol supplemented several medical and non-medical role plays with a lecture and guided discussion for feedback. Forty-one reside nts participated in the training on one of two days (groups) and completed evaluati on measures prior to and immediately following the training program The training was evaluated with measures of trainee reactions, attitudes towards teamwork, and res ponses to a situational judgment test (SJT). Analyses compared item and scale scores be tween pre-training scor es and post-training scores. T-tests generally found higher means fo r post-training behavi oral responses than pre-training responses. However, mean co mparisons with teamwork attitudes and
vi learning goal orientation did not yield significant differences. An item analysis of the SJT responses (using chi-square) indicated signifi cant response shifts in many items that correspond to the teamwork training content. In summary, results indicated that teamwork training on behavioral choices, but little effect on th e self-reported attitudes of trainees.
1 Chapter One Introduction The use of work teams frequently provides a competitive advantage for organizations (Guzzo & Dickinson, 1996), but th is advantage is often contingent upon the team members working well together. Freque nt and effective teamwork behaviors are valuable because they improve team eff ectiveness and contribute to team member satisfaction (LePine, Piccol o, Jackson, Mathieu, & Saul, 2008). Such behaviors are also targeted by organizations in order to guide staffing, training, and performance management decisions (Brannick & Prin ce, 1997). As a result, researchers and practitioners have taken intere st in the use of training tec hniques to improve teamwork behaviors and team-oriented knowledge, skills, abilities, or other task requirements (KSAOs; Stevens & Campion, 1994). One approach to teamwork training, which was developed in the aviation industry, is usually referred to as Crew Resource Management ( CRM; Wiener, Kanki, & Helmreich, 1993). CRM aims to tr ain airline crews to effectively manage all available resources through behaviors including effective team commu nication and coordination (Salas, Burke, Bowers, & Wilson, 2007). Thr ough the years, CRM has become a model for teamwork training (Salas et al., 2007), and it is now be ing applied in other domains such as medicine, naval commerce, and energy production. Training may be a big business (Goldstein, 1993), and it is easy to show that training is being conducted in
2 many contexts, but teamwork training evaluations outside of aviation are rather few and far between. A review by Salas et al. (2001) suggests that CRM is generally beneficial in aviation, but little is known about the benefits of teamwork training outside of this context. Health organizations may particularly benefit from teamwork training programs because of the high cost associated with human errors (e.g., loss of life), errors which may be prevented through more effective team functioning. In 1999, the Institute of Medicine (IOM) reported that nearly 100,000 liv es and approximately 25 billion dollars are lost each year due to medi cal errors. The IOM concluded that much of these errors could be reduced through changes in the health care system, including the implementation of team training programs. As a result, there is a great need for a valid and reliable training program targeted towards teamwork in medical tasks. Despite the great potential that teamwork training has to positively impact medical teams, few studies ha ve validated a teamwork training program for medical teams. Much of the existing work has focuse d on traditional, lectur e-based methods with modest results. Thomas and his colleague s (2007), for example, found more frequent teamwork behaviors in teams that received lecture and short, low fidelity role play training than control groups. Small et al. ( 2007) used a traditional, didactic approach using a high-fidelity simulato r with teams that had alread y received a simple lecture. Teams that received instructional traini ng within the context of the simulator demonstrated more teamwork behaviors late r on than did teams that simply received lecture-based training without use of a simulator. Shapiro and colleagues (2004) attempted to improve teamwork behaviors through task simulations in addition to
3 traditional lecture-based methods, but the training protocol did not yield a significant difference in behavioral comparisons. Morey et al. (2002) performed a quasiexperimental study that compared hospitals that had a teamwork tr aining program against hospitals without such a program. A lthough this comparison yielded significant differences in teamwork eff ectiveness, it did not indicate the effect of training on the individual trainees, and the design was not a tr ue experiment with random assignment to groups. Finally, many training programs have b een developed for applications in health care organizations (e.g., TEAMS TEPPS; ACRM). Although case reports of these training programs are generally positive (Alonso et al ., 2006), empirical evaluations on the effect of such programs have been difficult to fi nd in the literature. Thus much remains to be learned about the optimal training design, de livery, and evaluation approaches to teamwork training in medicine. The current study applies a teamwork tr aining program in a medical environment in order to assess subsequent effects on trainee reactions attitudes, and behavioral intentions. Specifically, the tr aining protocol inte grates behavioral role modeling in realistic task simulations and error management techni ques to increase teamwork behaviors during trauma resuscitation. The stud y is designed to provide an initial basis for documenting an effect of training and to specify whic h team processes are most principally affected by the training. What follo ws is a review of tr aining approaches and their relevance to the teamwork traini ng program applied by this study. A Review of Te amwork Training Effective training programs are designed around four basic principles: 1) the training presents the relevant knowledge, skills, and abilities (KSAs) to be learned, 2) the
4 training demonstrates the KSAs of interest 3) the traini ng provides opportunities to practice the KSAs, and 4) the practice of th ese KSAs is supplemen ted by informational feedback (Salas & Canon-Bowers, 2001). Traditional approaches primarily focus on the first two principles, informat ion presentation and, in some cases, demonstration. Such approaches typically use a lect ure or another form of proceduralized instruction, which have a major advantage in terms of efficien cy and cost-effectiveness. One trainer can present the material to a large number of trainees at a rela tively low cost. Furthermore, traditional methods can encourage participan ts to overlearn the material through structured tasks and routine presentation of the same material (F ord & Kraiger, 1995). This approach has been bene ficial for many tr aining areas (Arthur, Bennett, Edens, & Bell, 2003; Driskell, Willis, & Cooper, 1992), primarily because it promotes automatic behavioral responses to specifi c problems. Lecture-based traini ng is also an efficient way to convey information. However, traditional or lecture-based met hods suffer from seve ral problems that limit their effectiveness. Th e first issue is that tradit ional training me thods promote analogical transfer but not adaptive transfer (Ivancic & Heskith, 2000). Analogical transfer describes familiar or analogous probl em solutions that were previously provided to the trainee, whereas adaptive transfer uses existing knowledge and skills to find a solution to a novel problem. In traditional approaches, trainees are frequently provided with the correct solution to an example problem, but do not expl ore other types of solutions or other types of task problems (Keith & Fres e, 2005). The routine expertise developed through this procedure is beneficial because it promot es analogical transfer. It interferes with adaptive tran sfer, however, by neglecting to instruct trainees how to
5 develop solutions to novel or dynamic probl ems (Devine & Kozlowski, 1995). Given the dynamic environments of many team types (Devine, 2002), adaptation to novel stimuli is a needed skill in teams, yet one that is not developed by traditi onal training methods. Traditional methods may also inhibit transfer by having low fidelity to the task. Ideally, trainees would apply every element of the training program towards their realworld job. Such a high level of transfer may be difficult to achieve, however, unless the conditions of the training realistically mi mic the conditions of the job. That is, the training program should f eature identical elements to the task demands (Baldwin & Ford, 1988). Lecture-based and similar forms of trai ning, however, generally make no attempt to capture task elements save through the presentation of hypothetical problems (Beard, Salas, & Prince, 1995). Although participants ar e informed of potenti al problems, they are unable to view the conditions of the problem and actively solve it. Thus, lecture-based training has a greater discrepancy between training and job task elements than other forms of training, such as a role play in a task simulation. Another criticism of tradit ional methods is that they do not explicitly target a trainees intrinsic motivation. Specifically, tr aditional methods place the responsibility of learning upon the instructor or trainer and li mit the control and responsibility placed upon the individual (Ford & Kraiger, 1995). Although instructor control makes for an easier transmission of information, it limits independent action or thought by the trainee. By providing a restrictive, structur ed training protocol, particip ants are not encouraged to explore the material or learn th e concepts beyond what is given to them (Keith & Frese, 2005). As result, trainee motiva tion to learn and apply the co ntent tends to derive from
6 extrinsic sources (e.g., to satisfy the organizat ion) rather than intrinsic sources (e.g., to become a more effective te am member; Iran-Nejad, 1990). The use of traditional methods (e.g., a l ecture or presentation) has frequently improved trainee performance, but the use of more active trai ning methods in addition to a lecture tends to yield a mo re effective traini ng program (Arthur et al., 2003). These active training methods include se lf-guided instruction, role play (usually with simulator), or guided group discussions. Based on these empirical findings, as well as the discussed limitations above, a strong de sign for teamwork training could make use of active learning techniques to deliver the training, supplemented by a lecture for trainee orientation. What follows is a discussion of the active lear ning methods used for this teamwork training. Active Approaches to Training Active learning approaches, in contrast to traditional methods, give trainees control over their own learning by granting th em greater autonomy and responsibilities ( Salas & Canon-Bowers, 2001) Rather than simply giving information to participants, active training asks participants to practice the training skills a nd provides subsequent feedback. Through the addition of practice and feedback, active training techniques capture the four basic principles of tr aining (traditional methods only capture two principles, as discussed above). Whereas the tr aining process is regulated externally by a trainer in traditional met hods, learning is self-regulated by the trainee in active approaches (Bell & Kozlowski, 2008). Activ e training methods simply use formal training design elements to pr omote the motivational processe s that underlie this selfregulated learning (May er, 2004). Theoretically, this desi gn not only encourages trainees
7 to continue learning beyond th e training module, but also pr epares trainees to resolve novel problems by encouraging adaptive tr ansfer (Ivancic & Hesketh, 2000). Active methods tend to use new advances in technolog y, such as auto-instruction software or sophisticated simulation program s, in order to immerse the participant in the training content. Researchers have designed severa l training approaches reflective of active learning approaches, including exploration (en active and guided), errors-based training, and mastery training (Bell & Kozlowsi, 2008) These different training methods for active learning are next expl ored and discussed in relation to teamwork skills. Guided Exploration: Role Play and Feedback. Whereas some active training techniques give trainees full autonomy ov er their learning progr ess (e.g., experiential learning), guided exploration proceeds in a pre-planned manner with consistent feedback provided by a trainer (Debowski, Wood, & Bandura, 2001). Through the addition of structured tasks and feedback, trainees ar e provided with more detailed knowledge on their learning progress, but the trainee is still able to explore the task beyond these constraints. In CRM, guided expl oration techniques usually involve the use of role play with guided self-correc tion or trainer feedback (Salas et al., 2001) In role play, participants ar e given a description of a scenario and then told to act out the scenario in their assigned roles. Ideally, participants would then use this simulation to practice the behaviors target ed in the training. A lthough role plays may vary in their level of structure (Decker & Nathan, 1985) they ge nerally have more structure than experiential methods. This adde d level of structure allows the trainer to give guidance and feedback based on the role play behaviors exhibited. Role playing has
8 been positively associated with training eff ectiveness in a variety of situations (e.g., Skarlicki & Latham, 1996). Role playing may also vary by the fidelit y of the scenario or simulation used. Low fidelity simulations do not capture as many task elements as high fidelity simulations, but they still allow for trainees to practice solving some example pr oblems. The primary benefit of low fidelity simu lations is their economy and flexibility, making them accessible for a variety of environments and samples. High-fidelit y simulations capture many of the task elements, such that partic ipants can practice beha viors on a realistic simulation of task problems, bu t they are also more costly and inefficient to administer. Research has indicated that tr aining may be effective with lo w-fidelity simu lators despite their lower fidelity to the task (e.g., Br annick, Prince, & Salas, 2005). Because low fidelity simulations effectivel y simplify the task, however, they limit both the scenario possibilities as well as the range of behaviors required to perform the task (Prince, Ellis, Brannick, & Salas, 2007). Still, low fidelity simulators have their use if they can provide psychological fidelity. That is, if the simulati on can capture the important features of the task in order to preserve the need for the tr ained skills and behaviors, then the simulation should still be effective. Guided exploration and role play de pend upon reflection and feedback of the training scenario in order to be successful. Th is feedback uses the principle of cognitive modeling by demonstrating the cognitive or be havioral steps required to solve a problem (Bandura, 1997). Furthermore, trainer feedback has a si gnificant impact on training effectiveness, making it one of the basic principles for effective training (Brannick & Prince, 1997). Another form of training feedback resides in guided self-correction
9 (Smith-Jentsch, Zeisig, Acton, & McPherson, 1998) In this technique, an instructor leads a discussion on the team behaviors perform ed during a practice task. The team is prompted to explore their pa st behaviors and assess their positive and negative impact. The goal of the instructor and the team is to identify strengths and weaknesses of the team in order to improve on the next task. This form of training has also been associated with positive results in the area of teamwork training (e.g., SmithJentsch et al., 1998). Mastery Training. By addressing the self-regulator y mechanisms in the trainee, training research has quickly learned the importance of goal-setting and the framing of training goals. Based on the goal-setting litera ture stemming from the work by Locke and Latham (1990), training goals should be sp ecifically elaborated by the trainee and relatively difficult to achieve. Self-regulation is also affected by the type of achievement goals. These achievement goals are: a mast ery goal, focused on the development of competence and task mastery; a performanceprove goal, focused on the attainment of favorable judgments of competence; and a performance-avoid goal, focused on avoiding perceptions of failure and incompetence (E lliot & Church, 1997). Research has generally concluded that trainees with a mastery or ientation have higher levels of intrinsic motivation, self-efficacy, e ffort, and persistence (e.g., Rawsthorne & Elliot, 1999; Kozlowski et al., 2001). In shor t, learning-based goals are more effective because they encourage the participant to lear n all features of the content and to continue learning even when his/her performance is poor. With pe rformance-based goals, however, participants are more likely to learn only what they need for effective training performance. Furthermore, performance-based goals are more likely to induce frustration at poor
10 performance, which can inhibit self-regulation and ultimately lead to the participant to quit trying. Learning-based goals are derived in part from individual dis positions (e.g., goal orientation as a trait; Kozlowski & Bell, 2003) but they may also be encouraged by situational factors. Mastery training encourages the adop tion of learning-based based goals by manipulating the environment and tr aining instructions. Specifically, a mastery orientation is induced in tr ainees through the framing of errors, and participants are assured that task expertise may be acquired with enough practice (e.g., Martocchio, 1994). Results from this form of training have generally b een positive (e.g., Kozlowski & Bell, 2006). In teams, a mastery orientation has been associated with greater transfer of knowledge and skills, as well as a greater pe rformance in novel situations (e.g., BerebyMeyer, Moran, & Unger-Aviram, 2004). Therefore, the inclusion of mastery training (or framing) to traditional role playing should e nhance training effectiveness. In validating the use of high fidelity role playing scenario s, the current study makes use of mastery training techniques to enhan ce training effectiveness. Error Management Training. Error management is si milar to mastery training because it also informs trainees about the positive function of e rrors during training. When errors are framed as a na tural, instructive learning tool individuals are more likely to develop a mastery orientation (Ivancic & Hesketh, 1995). When errors are framed as a punishment for poor performance, trainees ar e more likely to adopt performance-avoid goals. Whereas traditional forms of training em phasize errors as negative events to be avoided, error management training encourages a positive view of errors because such events provide great learning opportunities. Error management tasks are also difficult
11 throughout the training, thereby e xposing participants to many errors possibilities to stimulate learning (Heimb eck et al., 2003; Hesketh & Ivancic, 2002). It is noted that this feature is different from other forms of active learning, in which training tasks are generally structured to be easy and progress with increas ing difficulty. With regards to teamwork training, erro r management training represents a new evolution from tradi tional CRM methods (Helmreich et al., 1999). This approach facilitates the learning pro cess from errors by providi ng teams with management strategies to aid the recogniti on and removal of future errors. These strategies break down into three basic components: 1) recognizing and avoiding a po tential error, 2) recognizing and removing (or trapping) an existing erro r before consequences are felt, and 3) mitigating the damage from errors not a voided or trapped (Helmreich et al., 1999). Because this form of training is relatively ne w to team training, how ever, there is limited evidence that it improves the effectivene ss of teamwork traini ng. The current study employed error-based training by provi ding guidance on common teamwork errors during medical role play scen arios. Trainer feedback after the role play focused on avoiding and managing these ty pes of teamwork breakdowns in the trauma room. To examine the cognitive effect of such mastery training, this study measured Learning Goal Orientation (LGO) before and after training. In part, traini ng effectiveness can be judged by the increase in LG O scores over training. Training Evaluations As the training literature has develope d, greater emphasis ha s been placed on the criteria used to validate a training program. In short, the training criterion matters. However, teamwork training literature has b een largely inconsiste nt in the types of
12 criteria used in evaluations. In an effort to systematize the resear ch on training validation, Kirkpatrick (1976) provided a typology of trai ning evaluations that guides much of the training evaluation studies toda y. This typology consists of trainee reactions, learning, behaviors, and task/organizational outcomes. Subsequent research would further outline these categories (e.g., Kraiger, Ford, & Salas, 1993) to provide a more robust account of the types of evaluations used in research and practice. These eval uations are discussed next. Reactions. Reaction criteria refer to trainee perceptions regarding the training program itself. This type of criterion is usually measured by asking participants the degree to which they enjoyed the training program and believed it to be worthwhile, interesting, well-administered, and useful (Kirkpatrick, 1976). Reaction criteria are important because such reactions may impact trainee motivation and transfer. If trainees see the training as worthless, they are unlikely to attempt to use it on the job. There are theoretical and empirical reasons to asse ss reaction criteria as a collection of specific constructs rather than as a global construct. Although measuring trainee enjoyment of the task may provide in sight for the trainer, utility reactions are more strongly related to learning and performance criteria (Alliger, Tannenbaum, Bennett, & Traver, 1997). This finding suggests that a trainee s belief that a program was effective is more likely to improve learning and performan ce than the trainees mere enjoyment of the protocol. Belief in the efficac y of a training program is also much more likely to affect trainee motivational processes, such as their self-e fficacy, goal-setting, effort, and persistence. That is, if trainees believe that a traini ng program can benefit them, they will be more likel y to believe that they ma y improve themselves. This
13 increased self-efficacy, in turn, should prom ote higher quality goals and more effort and persistence in the training task (Bandura & Locke, 2003). Existing evidence indicates th at trainees who have pa rticipated in CRM have enjoyed it, though this was moderated by the ty pe of training. In general, participants enjoyed role playing much more than l ecture-based training (e.g., Baker, Baumen, & Zalesny, 1991; Schiewe, 1995). This finding suggests not only performance benefits from active and interactive approaches to training, but affective benefits as well. Although the affective reactions are valuable pieces of information for a tr ainer, these criteria do not indicate whether trainees belie ved the training content was valuable. Research assessing utility criteria is much more limited, but existing findings s uggest that trainees see CRM as relevant to their jobs and believe it should be expanded to other teams (e.g., Incalcaterra & Holt, 1999; cf. Salas et al ., 2001). However, many studies on CRM utility reactions use qualitative descriptions instead of quantitative data (Salas et al., 2007). The current study provides both quantitative and qua litative indices of ut ility reactions to determine whether medical resi dents find this training program useful to their work. Learning. Despite the value of affective reac tions as criteria, such criteria are limited because they do not target directly the actual skills and behaviors that were supposed to be acquired during the training. Th e next level of training outcomes focuses on observable indices of what a person can do, ra ther than what they say they can do or how they feel about the training. Kirkpa trick (1976) proposed a general learning construct, but distinguishing among specific ty pes of learning outco mes is more useful from both a theoretical and prac tical perspective (e.g., Kraige r et al., 1993). Specifically, trainees may have cognitive-base d, skill-based, or affective-based learning (Kraiger et al.,
14 1993). Cognitive-based learning describes the amount and type of knowledge learned by the participant. Skill-based learning refers to the development of technical and motor skills. Finally, affective-based learning refe rs to attitudinal and motivational changes from the training. Measuring th ese different types of lear ning outcomes allows one to more precisely determine which processes are being impacted by the training. For example, a diversity training program likel y targets affective learning (e.g., changing viewpoints on race) to im prove behaviors and outcomes. As a result, evaluations for such training designs would most likely benef it from measures of affective learning. Teamwork training studies have found that training positively impacts affectivebased learning in teams by inducing more positive attitudes towards the teamwork behaviors of interest (e.g., Gr egorich, 1993; Irwin, 1991). Intere stingly, a few studies also report that participant attitudes were more negative of the teamwork behaviors following the training (a.k.a. boomerang effect; Helmre ich et al., 1999). Helmreich et al. (1999) suggested that personality and cultural factor s impact whether trainees report an increase or decrease in positive attit udes. Affective-based learni ng outcomes are important to assess in training primarily because more positive attitudes towa rds the behavioral content should lead to more fr equent executions of those beha viors. In other words, team members who look favorably upon teamwork sh ould make greater efforts to perform them during task performance. To ensure th at our training program improves attitudes towards teamwork, this study examines aff ective learning in trai nees by measuring attitudes towards teamwork beha viors in a medical context. It is believed that teamwork training w ill improve attitudes learning in a medical context by increasing salience and acceptance of critical teamwork behaviors. Because
15 teamwork concepts have not been formally trai ned in residents, init ial teamwork attitudes should stem from prior experiences or intuitive appeal Post-training attitudes should instead be based upon training concepts and expe riences. As a result, pre-training attitude scores should vary randomly, whereas post-traini ng attitudes should yield high, stable attitude scores. Hypothesis 1 : Attitudes towards teamwork wi ll have higher means after the teamwork training than before. Behaviors. Although positive attitudes toward teamwork are important, the primary intention of teamwork training is to change the awareness and execution of the teamwork behaviors. Behavior-based training criteria are important fo r two reasons. First, they specify the behaviors desi red by trainees during the cour se of the training program. Second, behavioral criteria may determine if the training will actually change the target behaviors in trainees. Because it is the behaviors that are usually being targeted in a training program, behavior-based criteria reflect the most direct evaluation of the trainees ability to apply the training content. This type of criterion can also indicate trainee readiness and ability to perform on the job (Canon-Bowers et al., 1989). In teamwork training, behavior-base criteri a generally refer to process behaviors such as communication, coor dination, monitoring, and others (e.g., Hackman, 1987; Dickinson & McIntyre, 1995). However, the team performance literature has struggled to achieve a common framework for the structure and definitions of teamwork behaviors. A recent review, for example, documented 29 st udies with differen t taxonomies of team behaviors (Rousseau, Aube, & Vincent, 2006). These various frameworks may unfortunately stem from the mu ltitude of team types and ta sks studied in the literature.
16 The use of multiple frameworks for team behaviors has also posed a problem in teamwork training literature. The teamwork tr aining primarily focuses on five teamwork dimensions identified by previous medical teamwork studies (e.g., Amodeo, Baker, & Krokos, 2009). These dimensions are: team structure, leadership, communication, monitoring, and mutual support. Team structure refers to th e degree that members orient to team goals instead of individual goals a nd correctly assume ro les within the team. Leadership focuses on effective leadership behaviors for the team, which include considering team member input and ma king a decision. Comm unication describes sharing information and sugges tions with other team member s, particularly leadership. Monitoring refers to maintaini ng awareness of patient condit ion and the status of other team members. Finally, mutual support describes team memb er willingness to request or provide assistance. Specific examples of thes e behaviors for an emergency room situation are provided later in a discu ssion of the training context. Teamwork training studies using team be haviors as the evaluation criteria have witnessed inconsistent results with regards to effectiveness (Salas et al., 2007). The type of behaviors and the method of measuring these behaviors ma y account for some of this inconsistency. Because team behaviors serve different functions during team performance (e.g., Marks, Mathieu, & Zacarro, 200 1), it is likely that they may be affected differently by training interventions. Indeed, several stud ies reported that teamwork training affected team behaviors differentially (e.g., Spiker, Nullmeyer, Tourville, & Silverman, 1998), suggesting that some behaviors are simply more easily trained or tran sferred than others. Team communication, for example, appears to benefit from teamwork training more so
17 than other types of behaviors (Salas et al., 2001). Studies which yielded poor results in evaluating team behaviors, then, may have simply been targeting the wrong behaviors. In addition, most studies usi ng self-rated measures of behavior are quite positive (e.g., Gran & Valet, 1997; Incalcaterra & Holt, 1999). Studies with observational measures of team behaviors, however, show more modest results (e.g., Brannick, Prince, Prince, & Salas, 1995; Jentsch, Bowers, & Ho lmes, 1995). As a result, the effectiveness of teamwork training on improvi ng behaviors is unclear due to methodological reasons. For logistical purposes, observational ratings were not possible to obt ain for every trainee on every scenario. This training program inst ead measured and examined the behavioral intentions of trainees through the use of a Situational Judgment Test (SJT). The SJT items were designed to elicit behavioral respons es to scenarios that describe a particular challenge to effective teamwork. Thus, we may consider the teamwork SJT a measure of stated behavioral intentions. This training evaluation assesses these behavioral intentions to examine the behavioral eff ects of the training program. Because an SJT poses a variety of critical incidents or work situations, th is measure may capture effective behaviors for many types of challenges in medical team work. Behavioral obser vations, on the other hand, only survey the behaviors performed in specific situation or role play. However, SJT responses only capture behavioral intentions and not actual behaviors, posing a limitation on the measure. Hypothesis 2 : Trainee responses to teamwork in cidents in Situational Judgment Test will shift to reflect more effective teamwork behaviors (l eadership, communication, monitoring, mutual support).
18 Outcomes. Outcome evaluations describe the ul timate results at the task and organizational level from the training program. Examples of such evaluations include whether the training reduced the total numb er of accident s or improved the number of sales or the amount of profit. This is an ev aluation highly valued by organizations as it provides information on whethe r the training program is paying off. Unfortunately, outcome evaluations are rarely included in teamwork training st udies and are often difficult to link clearly to the training program. Accidents and critical incidents that training is designed to prevent typically have a low base rate, which makes it difficult to obtain statistical significance for training in terventions. Furthermore, the results from training may take months or even years to surface at the organizational level, a phenomena which can only be captured thr ough longitudinal design. Such longitudinal designs are difficult to empl oy and are rarely used. Existing research on organizational outco mes from CRM is pos itive but far from conclusive. In a rare longitudinal study, Byrnes and Black (1993) found that CRM training improved pilot outcomes in the form of air carrier discrepancy reports. However, this study provided no control group to rule out the influe nce of environmental factors which affect air travel. The bulk of the rema ining evidence with regards to organizational outcomes lies in anecdotal evidence. Generall y, anecdotal evidence is positive, but the lack of statistical comparisons presents a problem. Although organizational outcomes are valuable criteria to assess with regards to training, the current study focused on reaction, learning, and behavioral criteri a for practical reasons. Becaus e of the inherent time delay between training and organiza tional outcomes, this study focuses on more immediate criteria. Specifically, the results criterion for this study was of a more short-term nature,
19 in which trainee performance on scenario task s was assessed both preand posttraining. Including simulation outcome measures provi des some informati on on the immediate impact of training on task pe rformance. However, it fails to answer whether trainees successfully transferred the skills to their act ual jobs, and whether the training had a long term impact on organi zational outcomes. Training Context: Trauma Teamwork When designing a teamwork training program, it is useful to consider the type of team being targeted, as well as the teams task s. This is because the specific behaviors to be trained may vary by task and team type. Furthermore, the general effectiveness of a training program may be task-dependent, in wh ich teamwork behaviors easily trained for one team type are not easily trained in anot her team type. As a result, when designing a training protocol, it is necessary to specify the team and task type that the training is designed to improve. In doing s o, the training program can speci fically target the relevant behaviors to the team and improve the transfer of team behaviors fr om the training to the actual job. Towards this end, the current study focuses on the impact of team building in medical teams, specifically em ergency room trauma teams. According to Devines (2002) taxonomy of team types, medical teams diagnose the physical condition of patient s and take appropriate steps to improve their health. They typically operate under severe time constraints, with the heal th of the patient contingent on appropriate procedures being executed in a particular window of time. Medical team tasks are usually very structured, with the aid of standardized di agnostic protocols and operating procedures and a highly controlled op erating environment. They are generally assembled for a particular mission (e.g., a resu scitation) or a defined period of time (i.e.,
20 a shift); roles tend to be fair ly specialized and based on mastery of procedures as they relate to particular instrument s, devices, and substances (surgeon, anesthesiologist, nurse, etc.). Within this context, team behaviors typically consist of effective communication, leadership, performance monitoring, helpi ng (or backup), and conflict resolution. Existing qualitative and quantitative research ha s also verified that teamwork behaviors such as the ones listed are a critical com ponent of error and mishap prevention in a medical environment (e.g., Thomas et al., 2005; 2006). Consultation with trauma Subject Matter Experts (SMEs) and an examination of scenario descriptions yielde d specific information on trauma team tasks and performance. The primary goal of trauma team s is to stabilize a patient in order to send him/her to further treatment (e.g., surgery, intensive care, out-patient). Obviously, the proper steps to stabilize the patient will depend upon the me dical condition, but tr auma teams must always monitor and maintain patient vital functions, in cluding respiration and blood circulation. The current study was conducted in a teaching hospital that employs medical residents. Although residents have received exte nsive training in medical procedures for these tasks, there is less time devoted to ef fective team interacti ons. Based upon feedback from residents and attending ph ysicians, residents appear unsure how to voice a concern, how to handle a problematic resident or nurse or how the orga nization perceives receiving help. Thus, there was a need for training teamwork in the trauma room. Effective trauma room teamwork consists of many components. First, an effective team structure must be established. The or ganization in the curre nt study employs six medical professionals for a standard trauma patient, with prescribed locations around the
21 bed or table and general roles at each position. For example, the person at the head of the table is in charge of maintaining a proper ai rway in the patient (being closest to the head and mouth). Effective team structure in the trauma room, the n, refers to establishing the correct positions around the table, assuming the correct roles, and focusing on team goals. Second, effective leadership is necessary for successful performance. Leadership is typically assumed by the most senior member in the room, but the communications between leaders and followers should be open and appropriate. Lead ers should maintain discipline and make procedural /treatment decisions, but they should also consider input from team members, whether such input consis ts of diagnostic inform ation or procedural suggestions. Because much of the communicatio ns within a trauma team occurs between the leader and team members, effective leadership has so me overlap with the third dimension, effective communication. In gene ral, team members should maintain an appropriate tone and wording when voicing a concern or sharing information. Likewise, leaders must be willing to communicate with team members, even if they disagree with their concern (it may be something as si mple as: Ill expl ain it afterwards). Monitoring the situation is a responsibility of all team members. The main purpose of monitoring is to identify potential issues and errors in order to successfully treat a patient. Team memb ers must alert other team members of new diagnostic information and stay aware of other memb er performance. Memb ers may observe or inquire about the task progress of others (e .g., intubation), or they may repeat diagnostic information in order to maintain such awaren ess. If the monitoring process identifies an error, mutual support refers to the team process of preventing or removing this error. For example, a member may repeat a concern until it is acted upon, or they may assist another
22 team member struggling with a procedure. Mo st importantly, team members must assist each other constructively to avoid conflict from evaluati on apprehension or negative feedback. Study Purpose and Training Design Although results from the existing medical l iterature are positive, very few studies have been completed, so an optimal trai ning method has yet to be determined and validated. This study contri buted to the existing body of literature by adapting and validating a team behaviors tr aining program for trauma ro om resuscitation. The current study examined training effectiveness base d on trainee reactions, attitudinal changes towards learning and teamwork, and behavior al intentions through an SJT. It was hypothesized that the medical residents undergoing this training would demonstrate positive utility reactions to the training, exhi bit affective-based learning of the targeted teamwork behaviors, and would choose diffe rent responses towards managing teamwork incidents. The specific traini ng features included in this study were as follows: (a) Lecture: to orient students to the training and teamwork concepts, we presented a lecture because it provides the mo st efficient means of information transfer. Such an introduction was intended to enhance th e effectiveness of the medical role plays by providing some initial guidelines. (b) Role play with Feedback and Guided Discussion: The primary focus of this training program was several role plays to demonstrate teamwork concepts, first in general and then in specific medical scenario s. Effective and ineffective teamwork were demonstrated through introductory role plays, and teamwork in the trauma room was trained using role plays in medical scenarios. Residents practi ced managing a breakdown
23 in teamwork through a role play for a medical emergency. To enhance the transfer of teamwork behaviors from the training to the task, a medical simulator was used to provide diagnostic information on the patient and a fake body for participants to treat. These scenarios provided a me dical emergency, but also provided scenarios where a confederate created problems in teamwork. Trainers led a guided discussion afterwards as part of a debriefing process, providing feedback when appr opriate during the discussion. Such feedback should reinforce positive behavi ors, encourage teams to investigate their deficits, and facilitate more e ffective behaviors for the next ta sk. Role play with feedback should not only enhance lear ning, but should also improve affective reactions to the training itself. (c) Error Management/Mastery Training: The lecture portion of the training emphasized the value of learni ng from mistakes, and also prov ided some suggestions for how to deal with a breakdown in teamwork (e.g., an error). Results from this study shoul d benefit research and pr actice alike by examining the impact of these design f eatures on reactions, attitudes, and behavioral intent. If successful, health organizations would benef it from training teamwork for trauma room resuscitation teams using such an approach. On a theoretical level, the results would increase our knowledge about te amwork training. Can teamwo rk, the process of momentto-moment responses of one team member to another, be improved in a realistic, fast paced, life-critical task enviro nment? Results would also be nefit researchers by providing guidance for future training designs and evaluations.
24 Chapter Two Method Design To test the efficacy of t eamwork training for emergenc y room teams, this study implemented a between-groups design with repeated measures of multiple training criteria. Specifically, resident s completed study measures before and after the training module for two separate traini ng days. Trainees were assigned to training days as randomly as possible, although in some cases scheduling was based upon a residents schedule. To summarize, the independent vari ables for this study were training group (1st or 2nd) and time of criteria m easurement (preand posttraining). The dependent variables were primarily team reactions, atti tudes, as well as SJT test and item scores. Sample Forty-one first-year medical residents from a southeastern medical center were recruited to participate in this training over the course of one day. Five residents did not provide either pre-training or pos t-training scores due to a late arrival or early departure. The sample sizes for specific analyses were provided in the results to account for this missing data. Participating reside nts were all in the first year of residency at the hospital and were recruited by attending physicians who contributed to this study. The sample was relatively representative demographically with 18 females and 9 Asian, African-
25 American, or Hispanic residents. The majori ty (N = 36) of the residents had received a M.D., but a few had earned some othe r advanced degree (e.g., Ph. D., D.O.). Training Content The first exercise was a role play designed to serve as an icebreaker and to demonstrate ineffective teamwo rk. The exercise featured a science fiction scenario in which all trainees received a scripted personality and instructions to work with their team while adopting the scripted personality. The sc ripted personalities were designed to cause breakdowns in teamwork, thus illustrating the difficulty of problem-solving without effective teamwork. After the role play, participants were debriefed on the personalities being played and the real purpose of the ex ercise. Then trainers guided participant discussion on the dynamics that led to poor te amwork and poor task performance. A full script of the role play is presented in Appendix A. The next exercise was a wilderness surviv al exercise designed to illustrate how groups may outperform individua ls (Jones & Pfieffer, 1976). Pa rticipants chose the best answer to multiple choice questions about su rviving in the wilderne ss, first individually, and then within groups. As is typical for th is exercise, the group score was generally better than individual scores, showing participants the benefit of working in groups. Appendix B presents the wilder ness exercise. After the exerci se, residents were debriefed on the purpose of the exercise (an illustrat ion of the benefits of teamwork) and led through a discussion of their experiences. Following these exercises, teamwork c oncepts were explicitly provided to participants in the form of a brief lecture. The lecture was meant to complement the exercises by giving clear information about the recently demonstrated concepts in the
26 context of medical teamwor k. It was during this time th at error framing and error management were incorporated into the training. Specifically, pa rticipants received encouragement to learn from any mistakes but were provided some guidelines for avoiding teamwork breakdowns and ma naging a problematic team member (recognizing and trapping errors). After the lecture, 5-6 participants perf ormed emergency medical tasks within a medical simulation. The scenarios were deve loped by the researchers and an expert physician in trauma resuscitation who was a ffiliated with the teaching hospital. Along with the participants was a confederat e (recruited from available staff) who was instructed to cause a specific issue in te amwork. For example, one scenario placed the confederate as the team leader, where the c onfederate was scripted to be a poor leader who gave little direction. The team was instru cted to identify this issue and resolve it while successfully completing the medical tas k. A total of four scenarios were run to allow all trainees to participate in exactl y one scenario. These scenarios were also recorded and broadcast live to a viewing room, so that trainees not currently participating in the scenario could watch and learn from the role play. As a result, each trainee participated in one scenario, and observed the other 3 scenarios. The first medical scenario used a confed erate who insisted on sending the patient to surgery before the patient was adequately evaluated. Effective re sponses discussed in the feedback section included an assertive but polite response to the nurse, maintaining awareness as the patient condition change d, and finishing patient resuscitation and evaluation before taking the actions recomme nded by the confederate (sending the patient to surgery). The second role play featured a confederate making a common procedural
27 error (in intubation), which the team must iden tify and correct in or der to successfully treat the patient. Effective behaviors referr ed to constant monito ring and communication of diagnostic information, and co rrecting the intubation error. The third scenario featured a timid confederate leader who failed to initiate structure or make decisions regarding treatment. Team members had to recognize a pre-existing error (from emergency responders) and correct the issue before se nding the patient to surgery, which required another leader to emerge and make team decisions. Finally, the fourth and final role play featured a confederate with hos tility towards the patient (a drunk driver in an accident), exhibited by negative and inappropriat e comments throughout the session. Team members needed to assertively silence the c onfederate while treati ng the patient, who had complications during the resuscitation. To facilitate feedback an d guided discussion, the video was played back to the entire group. Trainers provided comments on sp ecific issues and crit ical incidents, but they also asked questions of the participan ts to encourage an interactive discussion. Ineffective behaviors were highlighted, and tr ainers then asked for suggestions of more effective behaviors (reinforci ng the more positive behavior s). For example, if team members hesitated to assume leadership, the trainers suggested directing specific inquiries to the designated l eader in order to gauge th e need for assuming command. Measures Pre-Training Questionnaire. Prior to training, participan ts were asked to provide demographic and occupational information. This questionnaire also asked if the participant had been involved in any prio r teamwork training programs (none had). Because trainee interest in training ofte n relates to training success (Mathieu &
28 Martineau, 1997), the scale also asked par ticipants to indicate their interest in participation for the training. Appendix C presents the items. Learning Goal Orientation. Learning Goal Orienta tion (LGO) was measured using an eight item, 5-point Likert scale developed by Bu tton, Mathieu and Zajac (1996). This scale is provided in Appendix D. Teamwork Attitudes. Attitudes toward teamwork in medicine were measured using 21 items from the Teamwork Attitudes in Medicine (Amodeo et al., 2009), adapted to fit trauma teamwork specifically See Appendix E for this scale. Teamwork Behaviors. Behavioral intentions were measured using a Situational Judgment Test (SJT) developed specifically fo r teamwork issues in medical emergency tasks. Subject Mater Experts (SMEs) constructed items th at reflected previous or potential issues in trauma room teamwork. The SJT was developed through consultation with the trauma expert who helped devel op the medical role plays. The items asked trainees to choose behavioral responses to critical teamwork incidents in the trauma room. The items reflected situations whic h were analogous, but not identical, to the medical scenarios used in the role plays, su ch as an ineffective leader, inappropriate comments by team members, or proce dural errors that need correcting. Trainees and a set of four SMEs chose thei r most likely and le ast likely behaviors out of a total of four options. The SMEs in cluded attending physicians ; two specialized in general trauma resuscitation, one specialized in neonatal resuscitation, and the last an attending emergency surgeon. A total of 15 si tuational items were administered, which may be observed in Appendix F. An SJT scal e score was computed by giving a point for every response that matched the expert res ponses. Expert responses were determined by
29 the modal response among experts. Items 3, 8, 11, 12, and 14 were only partially scored when computing scale totals because experts provided more than 1 mode for either the most or least likely response. For items with partial scoring, the response with one expert mode was scored (e.g., most likely), but responses with more than one mode were not scored (e.g., least likely). The internal consistency of th e SJT scale was relatively low ( = .42 pretest, .44 posttest), but these types of tests generally do not exhibit strong internal consistencies due to the variety of situations being posed. The reliability of SME scor es was estimated by treating different response options as distinct categories and calculating kappa to estimate agreement. Kappa for expert scores was .68, a reasonably acceptable level of agreement among expert responses. The SJ T scale was used for mean comparisons, but chi-squares were also conducted upon each item, even th e ones with multiple expert modes. Reactions. Trainee utility reactions were m easured using a scale that asked participants about the degree to which they liked the training and the degree to which they believed the training would be useful. Because these questions are training specific, reaction measures were not a ssessed until after the team ha d completed the training. The reaction scale measure can be viewed at A ppendix G. Responses to this scale helped diagnose particular problems with the traini ng approach from the trainees perspective. Procedure Two recruited SMEs served as the traine rs for this study, an attending M.D. experienced in emergency trauma care, and a Ph.D. experienced in observing and facilitating teamwork. The trai ners explained that the purpose of the training was to improve teamwork for trauma room emergencie s, and trainees provided consent to have
30 data used for publication. Participants were trained on either day one or day two, but not both days. First, participants completed the demographics form and pre-test measures of the survey. Then, trainers guided participants through the training protocol as described earlier. The procedure concluded with a sec ond administration of the reaction, attitudes, and SJT scales. Analyses Training evaluation focuse d upon the difference in tr ainee responses from pretraining to post-training. Gi ven the study design, severa l paired and independent t -tests were conducted to assess mean differences for attitude and SJT total scores. Independent t -tests were planned contrasts between pretraining measures for the first group and the post-training measures for the second group, and vice versa. This design used each groups pretest scores to serve as the other groups control. T hus, using this design, training was compared to a no-training control, but not to a placebo tr aining of equivalent duration. Such a design was pref erable to single group preand post-test comparisons for the same training day. When comparing preand post-test measures for the same group, it was more difficult to discount the potential ef fect of nuisance variab les specific to that group, such as specific team performances in the medical role plays. However, because pre-post comparisons may still yield information with regards to training effectiveness, paired t -tests examined the mean difference betw een preand post-training outcomes for each group. In addition to using total SJT scores fo r group comparisons, th is study analyzed each SJT item response using frequency tables and chi-square goodness of fit tests. The goodness of fit test treated the observed freque ncies from pre-traini ng SJT item responses
31 as the expected frequencies for post-training SJT responses. A chi-square then estimated the degree of fit between the observed post-tr aining responses to the expected frequencies for each response option (most and least likely for each item). If tr aining had no effect on trainee response choices, one w ould expect post-training scores to be the same or very similar to pre-training scores. Thus, a significant chi-square would suggest a significant effect of training on item responses, based on shifts in the frequency tables. Bar charts provided an illustration of some of the more inte resting response shifts.
32 Chapter Three Results Table 1 presents the means, standard deviations, and inter-correlations of continuous variables in the stud y. Tables 2 and 3 present the means, standard deviations, and inter-correlations fo r each training group (day 1 and day 2), respectively. The level of interest in training did not relate to any of the study variables except the reactions measure, so using this variable as a covariate in additional analyses was deemed unnecessary. There were strong positive corre lations among the self-report measures, particularly with the reaction score and th e learning goal orientation. These correlations suggest a general relationship between lear ning goal orientation, te amwork attitudes, and training reactions. It should not be surprising that partic ipants who enjoy learning new material would also enjoy the teamwork trai ning and have positive attitudes towards its content. However, these high correlations may also reflect a general response bias in self report measures. In particular, high scale means cause reason for concern of score inflation by respondents, or a general halo e ffect. The ceiling effects of study self-report variables are illustrated via boxplots in Figure 1. Surprisingly, SJT scores (both time 1 and time 2) di d not relate to LGO or teamwork attitudes, suggesting that ha ving positive attitudes and a learning goal orientation does not mean one will know th e right response to a breakdown in teamwork. SJT scores also did not relate to trainee reactions. Finall y, the distributions of all
33 Table 1. Descriptive Statistics and Inter-co rrelations for Self-Report Measures T a b l e 1 Table 2. Descriptive Statistics and Intercorrelations for the First Training Group Table 3. Descriptive Statistics and Intercorrelations for the Second Training Group Variable M SD 1 2 3 4 5 6 7 8 1. Interest in Training 1.90 0.72 -2. LGO, Time 1 4.42 0.49 .15 (.90) 3. LGO, Time 2 4.52 0.51 .13 .71** (.94) 4. Attitudes, Time 1 4.43 0.35 .16 .60** .17 (.85) 5. Attitudes, Time 2 4.44 0.41 .26 .48** .60** .47** (.91) 6. SJT Score, Time 1 14.63 3.61 -.06 -.01 -.08 -.17 -.25 (.42) 7. SJT Score, Time 2 16.89 2.57 -.13 -.05 .06 -.28 .04 .40* (.44) 8. Reactions 4.02 0.84 .63** .56** .27 .53** .39* .01 .17 (.91) Note. LGO = Learning Goal Orientation. SJT = Situational Judgment Test. N = 34-41. Scale reliabilities (alpha) on diagonal. *p < .05, **p < .01. Variable M SD 1 2 3 4 5 6 7 8 1. Interest in Training 1.95 0.62 -2. LGO, Time 1 4.54 0.42 .25 (.90) 3. LGO, Time 2 4.48 0.62 .53* .87** (.94) 4. Attitudes, Time 1 4.44 0.23 -.38 .47* .34 (.85) 5. Attitudes, Time 2 4.32 0.48 .30 .64** .81** .47 (.91) 6. SJT Score, Time 1 15.63 4.06 .15 -.06 .39 -.37 .48 (.50) 7. SJT Score, Time 2 17.29 1.99 .02 .12 .27 -.05 .50* .74** (.48) 8. Reactions 4.37 0.62 .62** .64** .83** .29 .84** .15 .02 (.91) Note. LGO = Learning Goal Orientation. SJT = Situational Judgment Test. N = 34-41. Scale reliabilities (alpha) on diagonal. *p < .05, **p < .01. Variable M SD 1 2 3 4 5 6 7 8 1. Interest in Training 1.85 0.83 -2. LGO, Time 1 4.31 0.53 .06 (.90) 3. LGO, Time 2 4.54 0.42 -.19 .61** (.94) 4. Attitudes, Time 1 4.41 0.34 .45* .67** .06 (.85) 5. Attitudes, Time 2 4.53 0.33 .29 .51* .30 .57** (.91) 6. SJT Score, Time 1 13.77 4.06 -.33 -.08 -.38 -.06 -.57** (.50) 7. SJT Score, Time 2 16.55 1.99 -.20 -.21 -.03 -.39 -.10 .14 (.48) 8. Reactions 3.73 0.95 .62** .50* .05 .50* .46* -.25 .11 (.91) Note. LGO = Learning Goal Orientation. SJT = Situational Judgment Test. N = 34-41. Scale reliabilities (alpha) on diagonal. *p < .05, **p < .01.
34 Figure 1 Boxplots of Teamwork, L earning Goal Orientation, and Reactions Scale Scores continuous variables were relatively normal, su ggesting that the data met the assumptions for mean comparisons. Results for Group Comparisons LGO, teamwork attitudes, and SJT score means were first compared for the pretest measures from day two a nd the post-test measures fro m day one. Mean differences were not statistically significant for LGO ( t (34) = 1.58, p = .12) or teamwork attitudes ( t (36) = -0.68, p = .50), contrary to expectations. Ho wever, group 1s post training scores ( M = 17.29, SD = 1.99) for the teamwork SJT were significantly higher than group 2s pre-training score ( M = 13.77, SD = 3.01, t (37) = 4.39, p < .001). This result illustrated a strong effect ( d = 1.38), which would suggest some benefit from the training. Next, analyses examined mean differen ces between pre-training scores for day one and post-training scores for day two. However, this comp arison did not yield
35 statistically significant mean differences between groups for any criterion. With regards to the SJT scores, th e post-training mean ( M = 16.55) for day two was higher than day ones pre-training mean ( M = 15.63), but the effect size for th is difference was small ( d = 0.26) and not statistically significant ( t (36) = 0.81, p = .90). Finally, analyses were conducted compari ng the pre-training sc ore to post-training scores within each training day. Table 2 pr ovides the means and standard deviations for the preand post-training sc ores of the first training group. This comparison yielded small mean differences fo r teamwork attitudes ( t (15) = 1.07, p = .30) and LGO ( t (14) = .20, p = .84), but it showed a marginal difference for SJT scores ( t (16) = 1.95, p < .10) and a modest effect size ( d = 0.34). Table 3 provides the statistics for the second training day. Again, teamwork attitudes did not improve statistically from pre-to post-training ( t (19) = 1.60, p = .13), but LGO scores were significantly higher post-training than pretraining ( t (19) = 2.35, p < .05, d = 0.46). SJT scores exhib ited a large mean difference between pre-and post training for the second training group ( t (19) = 2.78, p < .01). This difference reflected a strong effect size ( d = 0.82) In summary, training did not affect trainee teamwork attitudes in any comparisons and only affected LGO in the preand pos t-test comparisons for the second training group. Results for the SJT, however, demonstrat ed an effect in th ree out of the four comparisons used. To examine if specific ite ms were driving the positive shift in SJT scores, item-level analyses were conducte d using the chi-square goodness of fit. Item Analysis for the Situational Judgment Test Tables 4-18 present the preand posttraining frequencies, as well as the chisquare test results, for the most and leas t likely responses of the 15 SJT items. To
36 summarize, the data suggest a significant effect of training on one or both responses for SJT items save for items five and eight. An examination of the frequency tables finds three forms of response shifts, which will be di scussed more in depth next. The first type of response shift occurred when trainee post-tr aining responses aligned more closely with expert responses than pre-training responses which reflects the opt imal result from the training. Items 3, 4, 7, 9, 13, and 14 demonstrated such an effect with either the most likely or least likely responses. The sec ond type of shift describes a pattern where neither prenor post-tr aining scores matched expert res ponses, but a shift was detected nonetheless. Instead, trainees shifted from one distracter opt ion to another, or experts failed to agree on an optimal response. It ems 10, 11, and 12 appeared to yield such effects. Finally, the least likely responses to items 1, 2, 6, and 15 demonstrated effects where trainee responses matched experts more closely prior to training than afterwards To illustrate the pattern of these response shifts in relation to the SJT questions, figures 28 provide example item questions, item response options, and bar charts of trainee (time 1 and time 2) and SME responses. These figures focus primarily on the effective items, but they pr ovide a couple of examples wh ere item responses shifted, but not necessarily for the better. First, Figure 2 presents the results for the least likely responses to item 3, which presents a situation where the team leader is hesitating to make a time-critical decision. With time being a critical factor, the worst response in this situation is to have a full discussion of team issues. The best treat ment decision must be made quickly, preferably by the leader but by another team member if necessary. Thus, the observed response shift represents an impr ovement in resident scores from preto post-training.
37 Table 4. Preand Post-Training Response Frequencies for SJT Item 1. Most Likely Response Least Likely Response PrePost-training responses Row Total PrePost-training responses Row Total score A B C D score A B C D A 0 0 0 0 0 A 0 0 2 1 3 B 3 27 0 1 31 B 0 0 0 0 0 C 0 4 2 0 6 C 0 0 0 1 1 D 0 0 0 0 0 D 1 0 4 27 32 Total 3 31 2 1 37 Total 1 0 6 29 36 2 = 6.92, df = 3, n.s. 2= 26.62, df = 3, p < .01 Note. Shaded response notes modal expert rating. Chi square goodness of fit estimated upon posttraining responses, using pre-training frequencies as expected values. Table 5. Preand Post-Training Response Frequencies for SJT Item 2. Most Likely Response Least Likely Response PrePost-training responses Row Total PrePost-training responses Row Total score A B C D score A B C D A 0 0 0 0 0 A 23 2 5 0 30 B 0 0 0 2 2 B 0 1 2 0 3 C 0 0 1 3 4 C 2 0 0 0 2 D 0 1 0 29 30 D 0 0 0 0 0 Total 0 1 1 34 36 Total 25 3 7 0 35 2 = 2.77, df = 3, n.s. 2= 13.26, df = 3, p < .01 Note. Shaded response notes modal expert rating. Chi square goodness of fit estimated upon posttraining responses, using pre-training frequencies as expected values. Table 6. Preand Post-Training Response Frequencies for SJT Item 3. Most Likely Response Least Likely Response PrePost-training responses Row Total PrePost-training responses Row Total score A B C D score A B C D A 1 0 1 0 2 A 6 0 0 11 17 B 0 6 1 1 8 B 0 1 0 1 1 C 1 6 11 0 18 C 0 0 0 0 0 D 0 2 2 4 4 D 2 0 1 13 16 Total 2 14 15 5 37 Total 8 1 3 25 35 2 = 5.55 df = 3, n.s. 2= 106.31, df = 3, p < .01 Notes. Shaded response notes modal expert rating. No shading indicates no modal expert response. Chi square goodness of fit estimated upon post-training responses, using pre-training frequencies as expected values.
38 Table 7. Preand Post-Training Response Frequencies for SJT Item 4. Most Likely Response Least Likely Response PrePost-training responses Row Total PrePost-training responses Row Total score A B C D score A B C D A 0 0 6 0 6 A 2 2 0 1 5 B 0 0 1 1 2 B 3 11 0 2 16 C 0 0 20 0 20 C 1 1 0 0 2 D 0 0 6 2 8 D 5 3 0 4 12 Total 0 0 33 3 36 Total 11 17 0 7 35 2 = 18.55, df = 3, p <.01 2= 14.92, df = 3, p < .01 Note. Shaded response notes modal expert rating. Chi square goodness of fit estimated upon posttraining responses, using pre-training frequencies as expected values. Table 8. Preand Post-Training Response Frequencies for SJT Item 5. Most Likely Response Least Likely Response PrePost-training responses Row Total PrePost-training responses Row Total score A B C D score A B C D A 0 0 1 0 1 A 0 0 0 0 0 B 0 0 0 0 0 B 1 1 0 5 7 C 2 0 33 0 35 C 0 0 0 0 0 D 0 0 0 0 0 D 1 6 0 21 28 Total 2 0 34 0 36 Total 2 7 0 26 35 2 = 4.80, df = 3, n.s. 2= 4.14, df = 2, n.s. Note. Shaded response notes modal expert rating. Chi square goodness of fit estimated upon posttraining responses, using pre-training frequencies as expected values. Table 9. Preand Post-Training Response Frequencies for SJT Item 6. Most Likely Response Least Likely Response PrePost-training responses Row Total PrePost-training responses Row Total score A B C D score A B C D A 12 8 0 1 21 A 0 0 0 0 0 B 0 14 0 0 14 B 0 0 0 0 0 C 0 0 0 0 0 C 0 1 4 1 6 D 1 0 0 0 1 D 0 0 8 21 29 Total 13 22 0 1 36 Total 0 1 12 22 35 2 = 4.56, df = 3, n.s. 2= 104.94, df = 3, p < .01 Note. Shaded response notes modal expert rating. Chi square goodness of fit estimated upon posttraining responses, using pre-training frequencies as expected values.
39 Table 10. Preand Post-Training Response Frequencies for SJT Item 7. Most Likely Response Least Likely Response PrePost-training responses Row Total PrePost-training responses Row Total score A B C D score A B C D A 0 3 2 0 5 A 2 0 0 0 2 B 0 17 7 0 24 B 0 1 0 0 1 C 0 1 5 0 6 C 0 0 1 2 3 D 0 0 1 0 1 D 1 1 0 27 29 Total 0 21 15 0 36 Total 3 2 1 29 35 2 = 17.13, df = 3, p <.01 2= 4.02, df = 3, n.s. Note. Shaded response notes modal expert rating. Chi square goodness of fit estimated upon posttraining responses, using pre-training frequencies as expected values. Table 11. Preand Post-Training Response Frequencies for SJT Item 8. Most Likely Response Least Likely Response PrePost-training responses Row Total PrePost-training responses Row Total score A B C D score A B C D A 3 2 0 0 6 A 2 1 1 2 6 B 3 11 1 4 2 B 0 0 2 0 2 C 0 2 1 1 20 C 4 0 15 0 19 D 2 2 0 4 8 D 0 1 1 6 8 Total 8 17 2 9 36 Total 6 2 19 8 35 2 = 3.93, df = 3, n.s. 2= .02, df = 3, n.s. Note. Shaded response notes modal expert rating. No shading indicates no modal expert response. Chi square goodness of fit estimated upon post-training responses, using pre-training frequencies as expected values. Table 12. Preand Post-Training Response Frequencies for SJT Item 9. Most Likely Response Least Likely Response PrePost-training responses Row Total PrePost-training responses Row Total score A B C D score A B C D A 0 0 0 0 0 A 0 1 0 5 6 B 0 0 4 0 4 B 1 0 0 3 2 C 0 0 32 0 32 C 0 0 0 0 0 D 0 0 0 0 0 D 0 0 0 25 25 Total 0 0 36 0 36 Total 1 1 0 33 35 2 = 3.93, df = 3, n.s. 2= 8.80, df = 3, p < .05 Note. Shaded response notes modal expert rating. Chi square goodness of fit estimated upon posttraining responses, using pre-training frequencies as expected values.
40 Table 13. Preand Post-Training Response Frequencies for SJT Item 10. Most Likely Response Least Likely Response PrePost-training responses Row Total PrePost-training responses Row Total score A B C D score A B C D A 1 1 1 1 4 A 5 0 0 3 8 B 0 16 3 1 20 B 2 0 0 1 3 C 0 1 2 0 3 C 2 0 2 6 10 D 0 3 3 2 8 D 2 1 2 8 13 Total 1 21 9 4 35 Total 11 1 4 18 34 2 = 22.45, df = 3, p < .01 2= 8.86, df = 3, p < .05 Note. Shaded response notes modal expert rating. Chi square goodness of fit estimated upon posttraining responses, using pre-training frequencies as expected values. Table 14. Preand Post-Training Response Frequencies for SJT Item 11. Most Likely Response Least Likely Response PrePost-training responses Row Total PrePost-training responses Row Total score A B C D score A B C D A 0 3 0 1 4 A 1 0 0 2 3 B 0 29 0 0 29 B 0 0 0 0 0 C 0 0 0 0 0 C 5 0 6 2 13 D 0 2 0 0 2 D 3 0 2 13 18 Total 0 34 0 1 35 Total 9 0 8 17 34 2 = 4.25, df = 3, n.s. 2= 20.56, df = 3, p < .01 Note. Shaded response notes expert rating. No shading indicates no modal expert response. Chi square goodness of fit estimated upon post-training responses, using pre-training frequencies as expected values. Table 15. Preand Post-Training Response Frequencies for SJT Item 12. Most Likely Response Least Likely Response PrePost-training responses Row Total PrePost-training responses Row Total score A B C D score A B C D A 3 1 5 11 20 A 0 2 0 0 2 B 0 1 2 2 5 B 0 6 1 1 8 C 0 0 0 0 0 C 3 8 8 2 21 D 0 0 1 10 11 D 1 0 2 1 4 Total 3 2 8 23 36 Total 4 16 11 4 35 2 = 6240, df = 3, p < .01. 2= 16.56, df = 3, p < .01 Note. Shaded response notes expert rating. No shading indicates no modal expert response. Chi square goodness of fit estimated upon post-training responses, using pre-training frequencies as expected values.
41 Table 16. Preand Post-Training Response Frequencies for SJT Item 13. Most Likely Response Least Likely Response PrePost-training responses Row Total PrePost-training responses Row Total score A B C D score A B C D A 0 0 0 0 0 A 1 0 0 7 8 B 0 1 0 0 1 B 0 0 1 2 3 C 0 3 32 0 35 C 0 0 0 0 0 D 0 0 0 0 0 D 1 1 0 22 24 Total 0 4 32 0 36 Total 2 1 1 31 35 2 = 10.51, df = 3, p < .05 2= 101.61, df = 3, p < .01 Note. Shaded response notes expert rating. No shading indicates no modal expert response. Chi square goodness of fit estimated upon post-training responses, using pre-training frequencies as expected values. Table 17. Preand Post-Training Response Frequencies for SJT Item 14. Most Likely Response Least Likely Response PrePost-training responses Row Total PrePost-training responses Row Total score A B C D score A B C D A 0 0 6 0 6 A 1 3 0 1 5 B 0 1 6 0 7 B 2 8 0 2 12 C 2 0 17 1 20 C 0 0 0 1 1 D 0 0 2 1 3 D 4 4 0 9 17 Total 2 1 31 2 36 Total 7 15 0 13 35 2 = 13.35, df = 3, p < .01 2= 4.29, df = 3, n.s. Note. Shaded response notes expert rating. No shading indicates no modal expert response. Chi square goodness of fit estimated upon post-training responses, using pre-training frequencies as expected values. Table 18. Preand Post-Training Response Frequencies for SJT Item 15. Most Likely Response Least Likely Response PrePost-training responses Row Total PrePost-training responses Row Total score A B C D score A B C D A 0 2 0 0 2 A 1 0 0 1 2 B 0 25 1 0 26 B 0 0 0 0 0 C 0 2 5 0 7 C 1 0 0 2 3 D 0 0 1 0 1 D 6 0 3 21 30 Total 0 29 7 0 36 Total 8 0 3 24 35 2 = 3.64, df = 3, n.s. 2= 32.67, df = 3, p < .01 Note. Shaded response notes expert rating. Chi square goodness of fit estimated upon post-training responses, using pre-training frequencies as expected values.
42 Figure 2. Frequencies for Least Like ly Responses for Item 3. Question 3: A patient begins crashing during treatment, and your team leader appears unwilling to make a decision on a controversial treatment. When questioned, he gives multiple reasons for different courses of action, but it is unclear which direction the team should go. What should you do? a. Make the decision for the team. b. Ask the next most senior team member to make the decision. c. Directly request the lead er to decide on applying the specific treatment. d. Ask for a team meeting to discuss and decide upon the issue. Figure 3 gives the most likely respons e frequencies for item 4, which described a situation where a resident was joking inappr opriately. This figure indicates a shift in trainee responses from multiple categories (time 1) to primarily the option endorsed by experts (time 2), or requesti ng that the resident focus on wo rk. This response presents a constructive, but assertive method of managing a problema tic team member. Thus, the pattern of response frequencies suggests a pos itive effect of training for addressing a team member acting inappropriate ly (as described in item 4). Figure 4 presents the results for item 7, which asked for responses to a nonphysician team member (e.g., nurse or tech) who politel y challenges the leaders decision. This item yielded di fferent responses between trai nees and experts both before and after training, with resident s preferring to listen to the challenge, and experts voting
43 Figure 3 Frequencies for Most Likely Response for SJT Item 4 Question 4: While treating a patient with head trauma, another resident makes many unhelpful and inappropriate comments, such as joking about the patients condition and the body habitus. What should you do? a. Let the team leader deal with it. b. Ask the resident if everything is okay, or if they want to talk about something. c. Request that the resident focus on work until emergency treatment is completed. d. Tell the resident to be quiet. Figure 4. Frequencies for Most Likely Response for SJT Item 7. Question 7: A senior non physician member of the team politely and appropriately questions the decisions of the team. What should you do? a. Let the team leader deal with it. b. Listen and use those suggestions for the benefit of the patient. c. Tell the team member that you will explai n the treatment after the resuscitation. d. Ignore the team member.
44 to explain the decision to th e member after treatment. Thus, this effect may reflect different perceptions based upon ones position in the organizational hierarchy. However, less than a quarter of pre-training responses did not match the modal expert response, whereas nearly half of the post-training responses did match SME responses, so the training did shift participan t responses to more closel y match expert scores. Figure 5 provides the least likely resp onse frequencies for ite m 9, which describes a situation where the resident observes a collea gue inserting a tube without a light. As the figures illustrate, both the least likely resident responses matched the expert responses more frequently with post-training measures. Experts unanimously agreed that the worst response in this situation is to let the colleague make an erro r. Thus, we may infer that the response shift shows an improvement in error management or mutual support. Figure 6 presents the most likely response choices for item 14, which describes an underperforming team member. Resident responses post-trai ning indicated a much more effective choice than pre-trai ning responses, according to expert scores. In an emergency situation, personal or performance issues in team members must be substantively addressed later, but the team member should still receive encouragement or support, such as being encouraged to focus on the immediate task. Thus, the response shifts observed in item 14 suggest a positive benefit from training. Finally, Figures 7 and 8 provide two ex amples where traini ng effects did not move trainee scores closer to expert scores. Figur e 7 presents results for item 12, which presented a controversial issue where a team member makes a char ged comment about a patient accused of rape and homicide. Although experts disagreed on the best response to this team member, residents overwhelmingly chose let the leader handle it before
45 Figure 5. Frequencies for Least Likely Responses for SJT Item 9. Question 9: You see a team member attempting to intubate, but the light on the laryngoscope is not working. What should you do? a. Ask the person if he wants you to do it. b. Point out the issue to the team leader. c. Ask the person if they realize that the light is out. d. Wait until the person misses the intubation, then point out the problem and take over. Figure 6 Frequencies for Most Likely Responses for SJT Item 14. Question 14: You notice that another resident is not doing his part during the resuscitation and seems to need to be prompted to perform his role. What should you do? a. Let the team leader deal with it. b. Ask the person if everything is okay, or if they want to talk about something. c. Request that the person focus on work until emergency treatment is completed. d. Point out that the resident is not in the game during the resuscitation.
46 Figure 7. Frequencies for Most Likely Responses for SJT Item 12. Question 12: Your team is caring for a gunshot wound to the abdomen of an 18 year old male with stable vital signs who was caught by police after a gun battle. The police officer in the trauma room has announced that this patient has just raped and shot a 13 year old girl. The respirator tech says he got what he deserved. What should you do? a. Let the team leader deal with it. b. Ask the person to step out of the trauma room to discuss why he would have said such a thing. c. Call the person unprofessional in public immediately after the remark, in order to set an example. d. Tell the respiratory tech to be quiet. Figure 8. Frequencies for Least Likely Responses for SJT Item 15. Question 15: A junior resident seems to be very ent husiastic, easily excitable, and raises his voice during the resuscitation when things get difficult. A senior nurse tells him to keep it down, and the resident then withdraws and stops participating in the care of the patient. What should you do? a. Let the team leader deal with it. b. Ask the person to perform a specific task to get him involved again. c. Wait until the resuscitation is over and address the issue with the nurse and resident separately and individually. d. Tell the nurse to be quiet.
47 training, and overwhelmingly chose tell the team member to be quiet after training. This dramatic shift appears to represent more assertivene ss by residents after training than before. Figure 8 provides the least likely responses for item 15, in which a resident becomes withdrawn because he was recently told to keep his voice down by a senior nurse. For this item, the least likely response of resident before training more closely matched the least likely response of experts, reflecting an effect oppos ite from previously discussed items (in which post-training respons es matched expert responses). However, there was a small shift in resident responses to choose the most passive response posttraining (let the team l eader deal with it). This result may also suggest an increase in resident assertiveness from trai ning, as occurred with item 12. The SJT items responses not illustrated here are described next. Least likely responses for item 13 reveal another positive shift where trainee scores matched expert scores more closely using post-training measures. Specifi cally, experts identified a negative comment as the worst response to a team member struggli ng with a procedure. Least likely responses for items 1, 2, and 6 mirrored the responses in item 15. Although post-training responses for these items did not reflect expert ratings as well as pretraining scores, post-training c hoices identified a passive response (e.g., let the team leader deal with it) more often as the least likely response. Thus, these shifts may indicate a commitment to assertiveness on the part of the participants, as they perceive passivity to be the worst mistake. Most likely responses for item 10 and leas t likely responses for items 4, 11, and 12 demonstrated significant response shifts that did not drift towards or away from expert responses. The response shifts for items 4, 10, and 11, however, seem to indicate more
48 assertive responses for post-training scales, similar to the results observed in the most likely responses for item 12. For these items post-training respondents more frequently identified a passive approach as the least lik ely choice (or vice versa for most likely). Other responses to items did not produce meanin gful patterns or sign ificant results. The implications of the item and s cale results with the SJT, as well as the attitude measures, are discussed next.
49 Chapter Four Discussion The purpose of the present study was to validate a teamwork training program for medical residents in trauma resuscitation. The author provided a comprehensive training program that complemented an interactive ro le play with lecture and trainer feedback. Trainee learning and reactions were examined by using self-report measures of learning goal orientation teamwork attitudes, and trainee reactions on trai ning utility. Teamwork behavioral intentions were assessed using a situational judgment test which presented critical incidents in trauma teamwork. Results generally validated the training pr ogram with regards to the situational judgment test responses, but it failed to sign ificantly improve learning goal orientations or teamwork attitudes. It w ould appear that teamwork trai ning provided some benefit in responding to specific situations but that it had little ef fect on participant attitudes towards the training and the content. The SJT scale score demonstrated an overall scale effe ct in three of the four group comparisons planned by the study, and more than half of the items showed a significant or marginal effect in respondi ng from the before and after training. In several instances, the posttraining resident responses matched the expert responses more closely than pretraining responses (e.g., items 3, 4, 7, 9, 14). Ev en in cases where response shifts did not
50 necessarily match expert rati ngs (e.g., items 12 and 15), one may observe an increase in resident assertiveness on item responses. The goal of the teamwork training program was to improve resident teamwork behaviors in the trauma room th rough role plays of critical teamwork incidents. Residents were specifically trained on effective struct ure, leadership, communication, monitoring, and mutual support behaviors in order to effect ively manage critical incidents in trauma room teamwork. The responses on the SJT provide a good index of analogous transfer from the medical role plays and feedback, as the SJT items presented analogous situations to those role plays. Thus, results from this stud y that participants used the knowledge and experience gained from the ro le plays to change their responses to analogous situations. The response shifts on the SJT generall y reflected more effective teamwork choices after the training than before. Sp ecifically, some items demonstrated more effective leadership through asse rtive actions and the consider ation of team member input (e.g., items 7, 10, 12). Other items demonstrat ed more effective mutual support and backup among team members. Examples includ e catching an existing error (item 9), or helping a team member perform a procedur e (item 13), or encouraging another team member (item 14). The training al so affected situational awar eness in some items, as the patient must remain the focal point of wor k. As a result, trainee responses did not approve of options that required a break from the resuscitation to deal with a problem team member (e.g., item 3). In summary, the response shifts observed from preto posttraining SJT scores correspond to the goals of the teamwork traini ng program in this study. Even for items that di d not show a closer match with expert scores post-training,
51 response shifts tended to illu strate an increase in asser tiveness which may reflect the training emphasis on assertive actions to co rrect ineffective beha viors or decisions. Because behaviors are the ultimate target of a teamwork training program, the training effect observed here documents an increasi ng awareness of team members of effective behaviors and ineffective behaviors. This improvement was especially important given the confusion many residents experience with regards to proper team positions and protocols. There are many elements of the training which may e xplain its positive effect on SJT responses. An active trai ning approach, such as role play, should serve as the primary focus for teamwork training. Whereas previous teamwork studies have attempted to train teamwork using only lecture-based methods (Small et al., 2007), this study saw a positive shift from using role plays primar ily, with some complementary lecture. Furthermore, the fidelity of the medical role plays to criti cal teamwork incidents in the trauma room served as anot her valuable asset in the training. By having task-related scenarios to practice and observe ineffective and effective beha viors, trainees learned the training content effectively. The provision of feedback also should have enhanced transfer by reinforcing positive and negative behaviors through the role play. The feedback provides a mechanism for trainees to reflect upon previous behaviors and adapt accordingly in future situations. Although error framing should have also impr oved training efficacy, the evidence in this study provides mixe d results for its effectiven ess. LGO scores were not consistently higher post-training than pre-tr aining scores. Indeed, few significant effects were detected with regards to the self-re port measures, LGO a nd teamwork attitudes
52 scale. Mean differences in se lf-report scales were not ge nerally large, and yielded no significant relationships with SJT scale sc ores. These non-significan t results may stem from low variance in scale responses or low power from a small sample size. Self-report means were relatively high for both pre-a nd post-training (approxi mately 4.5 on a 5 point scale), suggesting a ceiling effect that restricted scale varian ce. Hopefully, these elevated means reflected truly positive attitudes towards teamwork and learning goals, and not careless or biased responding. Regardless of the reason for high scale mean s, results of this training highlight concerns with attitude meas ures of teamwork training ev aluation. First, evidence from this training suggests that posi tive attitudes do not necessarily lead to effective behaviors. Although attitudes remained unchanged from preto post-training scores, teamwork training did affect re sident behavioral choices within criti cal incidents. It is easy to agree with an abstractly described idea (e.g., eff ective communication), but it may be difficult to choose a specific course of action based upon that idea (e.g., how would you communicate in this situation? ). Second, self-report measures of teamwork attitudes may be more susceptible to response biases. Se lf-report attitude ite ms tend to be more transparent with regards to more favorable responses, and trainees may have responded based upon their perceptions of the desired or correct response rath er than their true opinions. That is, social desi rability or impression management may play a role in responses on teamwork attitudes, much like it can on personality measures. Limitations This training evaluation has documented some interesting e ffects, but several limitations should be considered when a ssessing study results. The study sample
53 prevented the use of an optimal design with a large sample and a placebo-training control group. To avoid a single group prepost test, the current eval uation study compared mean differences between pre-and post criterion scores betwee n different training groups. Furthermore, the effects of sp ecific training features were not examined in this study. Rather, the training evaluation reflected the overall effect of trai ning. Future training studies may address this issue by comparing different training approaches to a sample of teams. For example, one study may manipul ate the type of role play by providing feedback to one condition and no feedback to another. In addition, teamwork behaviors and task outcomes represent the primary criteria from an organizational pe rspective, but neither of these cr iteria was directly measured in the current study. We measured trainee beha viors by their responses to a situational judgment test that presented critical incidents in trau ma teamwork. Although this measured was designed to measure the beha vioral choices of trainees in specific situations, it does not necessarily indicate thei r actual behaviors when the situation does present itself. Trainees instead gave an indication of what they would like to do, or what they should do in a given situation, which is still a valuable measure of their understanding of effective teamwork behaviors. Task outcomes were not measured in this study for practical reasons. Obtaining adequate outcome data requires tracking pa rticipant performance for many months following the training. Furtherm ore, the base rate for critical incidents and teamwork errors in medicine is relatively low, so it may be difficult to obtain an adequate sample of errors to determine if a reduc tion in errors was due to a tr aining program. The reader is cautioned, however that the tr aining program is not evaluated relative to task or
54 organizational outcomes. Future research on this or similar training pr ograms should seek to validate findings on actual teamwork behaviors and organizational performance reviews. Summary Results from this study suggest that a comp rehensive training approach using role play, lecture, feedback, and framing can positively affect behavioral choices for teamwork in the trauma room. The traini ng program did not aff ect already positive attitudes towards teamwork, but it did demonstrate more eff ective responses to teamwork issues after the training than before. Hea lth management organizations should benefit from this training because it provides a positive effect at a relatively low cost in resources.
55 References Alliger, G. M., Tannenbaum, S. I., Bennett, W., Jr., & Traver, H. (1997). A meta-analysis of the relations among training criteria. Personnel Psychology, 50 (2), 341-358. Alonso, A., Baker, D., Holtzman, A., Day R., King, H., Toomey, L., & Salas, E. (2006). Reducing medical error in the military health syst em: How can team training help? Human Resource M anage Review,16(3), 396-415. Amodeo, A. M., Baker, D. P., Krokos, K. J. (2009). Assessing Teamwork Attitudes in Healthcare: TeamSTEPPS Teamwork A ttitudes Questionnaire Development. Paper presented at the 24th Annual Meeting of the Society for Industrial & Organizational Psychology (New Orleans, LA). Arthur, W., Bennett, W., Edens, P.S., & Bell, S. T. (2003). Effectiveness of training in organizations: A meta-analysis of desi gn and evaluation features. Journal of Applied Psychology, 88(2), 234-245. Baker, D. P., Bauman, M., & Zalesny, M. D. (1991). Development of aircrew coordination exercises to facilitate transfer. In R. S. Jensen (Ed.), Proceedings of the 6th International Symposiu m on Aviation Psychology (pp.314-319). OH: The Ohio State University. Baldwin, T. T., & Ford, K. J. (1988). Tranfer of training: A review and directions for future research. Personnel Psychology, 41 63-105. Bandura, A. (1997). Self-efficacy: The exercise of control New York: Freeman.
56 Bandura, A., & Locke, E. A. (2003). Negativ e self-efficacy and goal effects revisited. Journal of Applied Psychology, 88 87-99. Beard, R. L., Salas, E., & Prin ce, C. (1995). Enhancing transf er of training: Using roleplay to foster teamwork in the cockpit. International Journal of Aviation Psychology, 5, 131-143. Bell, B. S., & Kozlowski, S. W. J. (2008). Ac tive learning: Effects of core training design elements on self-regulatory proces ses, learning, and adaptability. Journal of Applied Psychology, 93 296-316. Bereby-Meyer, Y., Moran, S ., & Unger-Aviram, E. (2004). When performance goals deter performance: Transfer of skill sin integrative negotiations. Organizational Behavior and Human De cision Processes, 93, 142-154. Brannick, M. T., Prince, A., Prince, C., & Salas, E. (1995). The measurement of team process. Human Factors, 37(3), 641-651. Brannick, M. T., & Prince, C. (1997). Team performance asse ssment and measurement: Theory, methods, and applications. Mahwah, NJ: LEA Brannick, M. T., Prince, C., & Salas, E. (2005). Can PC-based systems enhance teamwork in the cockpit? International Journal of Aviation Psychology, 15, 173187. Button, S. B., Mathieu, J. E., & Zajac, D. M. (1996). Goal orientation in empirical research: A conceptual and Empirical Foundation. Organizational Behavior and Human Decision Processes, 67(1) 26-48.
57 Byrnes, R. E., & Black, R. (1993). Devel oping and implementing CRM programs: The Delta experience. In E. L. Wiener, B. G., Kanki, & R. L. Helmreich (Eds.), Cockpit resource management (pp. 421-443). CA: A cademic Press. Canon-Bowers, J. A., Prince, C., Salas, E., Owens, J., Morgan, B., Jr., & Gonos, G. (1989). Determining aircrew coordina tion training effectiveness. Paper presented at the 11th Interservice/Industry Training Syst ems Conference, Fort Worth, TX. Debowski, S., Wood, R. E., & Bandura, A. ( 2001). Impact of guided exploration on selfregulatory mechanisms and informati on acquisition through electronic search. Journal of Applied Psychology, 86, 1129 1141. Decker, P. J., & Nathan, B. R. (1985). Behavior modeling training. New York: Praeger. Devine, D. J. (2002). A review and integration of classification system s relevant to teams in organizations. Group Dynamics, 6, 291. Devine, D. J., & Kozlowski, S. W. J. (1995). Domain-specific knowledge and task characteristics in decision-making. Organizational Behavior and Human Decision Processes, 64, 294-306. Dickinson, T. L., & McIntyre, R. M. ( 1995). A conceptual framework for team performance measurement. In M. T. Bra nnick, C. Prince, and E. Salas (Eds.), Team performance measurement and assessment: Theory, methods, and applications (pp. 19-43). Mahwah, NJ: Lea. Driskell, J. E., Willis, R. P., & Copper, C. (1992). Effects of overlearning on retention. Journal of Applied Psychology, 77 615-622.
58 Elliot, A. J., & Church, M. A. (1997). A hier archical model of approach and avoidance achievement motivation. Journal of Personality and Social Psychology, 72 218 232. Ford, J. K., & Kraiger, K. (1995). The appli cation of cognitive constructs and principles to the instructional systems model of trai ning: Implications for needs assessment, design, and transfer. In C. L. C ooper & I. T. Robertson (Eds.), International review of industrial and organizational psychology (Vol. 10, pp. 1). New York: Wiley. Goldstein, I. L. (1993). Training in organizations: Need s assessment, development, and evaluation (3rd ed). Monterey, CA: Brooks/Col e Publishing Company.Gran & Valet, 1997 Gregorich, S. E. (1993). The dynamics of CRM attitude change: Atti tude stability. In Proceedings of the 7th International Symposium on Aviation Psychology (pp. 509-512). OH: The Ohio State University. Guzzo, R. A., & Dickson, M. W. (1996) T eams in organizations: Recent research on performance and effectiveness. Annual Review of Psychology, 47, 307-338. Hackman, J. R. (1987). The design of wo rk teams. In J. W. Lorsch (Ed.), Handbook of organizational behavior (pp. 315-342). Englewood Clif fs, NJ: Prentice-Hall. Heimbeck, D., Frese, M., Sonnentag, S., & Ke ith, N. (2003). Integra ting errors into the training process: The functi on of error management instructions and the role of goal orientation. Personnel Psychology, 56, 333.
59 Helmreich, R. L., Merritt, A. C ., Wilhelm, J. A. (1999). The evolution of crew resource management training in commercial aviation. The International Journal of Aviation Psychology, 9(1), 19-32. Hesketh, B., & Ivancic, K. (2002). Enhanc ing performance through training. In S. Sonnentag (Ed.), Psychological management of individual performance (pp. 249 265). New York: Wiley. Incalcaterra, K. A., & Holt, R. W. (1999). Pilo t evaluations of ACRM programs. In R. S. Jensen, B. Cox, J. D. Callister, & R. Lavis (Eds.), Proceedings of the 10th International Symposium on Aviation Psychology (pp. 285-291). OH: The Ohio State University. Iran-Nejad, A. (1990). Active and dynamic sel f-regulation of learning processes. Review of Educational Research, 60, 573. Irwin, C. M. (1991). The impact of initial and recurrent cockpit resource management training on attitudes. In R. S. Jensen (Ed.), Proceedings of the 6th International Symposium on Aviation Psychology (pp. 344-349). OH: The Ohio State University. Ivancic, K., & Hesketh, B. (1995). Making th e best of errors dur ing training. Training Research Journal, 1, 103. Ivancic, B., & Hesketh, B. ( 2000). Learning from error in a driving simulation: Effects on driving skill and self-confidence. Ergonomics, 43,1966.
60 Jentsch, F., Bowers, C. A., & Holmes ( 1995). The acquisition and decay of aircrew coordination skills. In R. S. Jensen & L. A. Rakovan (Eds.), Proceedings of the 8th International Symposium on Aviation Psychology (pp. 1063-1068). OH: The Ohio State University. Jones, J. E., & Pfieffer, J. W. (1976). Wilderness survival: A consensus-seeking task. In Jones, J. E., & Pfeiffer (Eds.), The 1976 Handbook for Group Facilitators University Associates: La Jolla, CA. Keith, N., & Frese, M. (2005). Self-regulation in error management training: Emotion control and metacognition as medi ators of performance effects. Journal of Applied Psychology 90, 677. Kirkpatrick, D. L. (1976) Evaluation of training. In R. L. Craig (Ed.), Training and development handbook: A guide to human resources development (pp.18.118.27). New York, NY: McGraw-Hill. Kozlowski, S. W. J., Gully, S. M., Brown, K. G., Salas, E., Smith, E. M., & Nason, E. R. (2001). Effects of training goals and goal orientation traits on multidimensional training outcomes and performance adaptability. Organizational Behavior and Human Decision Processes, 85, 1. Kozlowski, S. W. J., & Bell, B. S. (2003). Work groups and teams in organizations. In W. C. Borman, D. R. Ilgen, and Klimoski, R. J., Handbook of psychology: Industrial and organizational psychology (Vol. 12, 333-375). Hoboken, NJ: Wiley.
61 Kozlowski, S. W. J., & Bell, B. S. (2006) Disentangling achievement orientation and goal setting: Effects on self-regulatory processes. Journal of Applied Psychology, 91, 900. Kraiger, K., Ford, J. K., & Salas, E. (1993) Application of cogni tive, skill-based, and affective theories of l earning outcomes to new met hods of training evaluation. Journal of Applied Psychology, 78 (2), 311-328. LePine, J. A., Piccolo, R. F., Jackson, C. L., Mathieu, J. E., & Saul, J. R. (2008). A metaanalysis of team processes: Tests of a multi-dimensional model and re4lationships with team effectiveness criteria. Personnel Psychology, 61, 273-307. Locke, E. A., & Laith am, G. P. (1990). A theory of goal setting and task performance. Englewood Cliffs, NJ : Prentice-Hall. Marks, M. A., Mathieu, J. E. & Zaccaro, S. J. (2001). A temporally based framework and taxonomy of team processes. The Academy of Management Review, 26, 356-376. Martocchio, J. J. (1994). Effect s of conceptions of ability on anxiety, self-efficacy, and learning in training. Journal of Applied Psychology, 79, 819. Mathieu, J. E. & Martineau, J. (1997). Indivi dual and situational influences on training motivation. In J. K. Ford (Ed.), Improving Training Effectiveness in Work Organizations pp. 193-222. Mahwah, NJ: LEA. Mayer, R. E. (2004). Should there be a threestrike rule against pure discovery learning? American Psychologist, 59, 14.
62 Morey, J. C., Simon, R., Jay, G. D., Wears, R. L., Salisbury, M., Dukes, K. A., & Berns, S. D. (2002). Error reduction and perfo rmance improvement in the emergency department through formal teamwork training: Evaluation results of the MedTeams project. Health Services Research, 37(6) 1553-1581. Prince, C., Ellis, E., Brannick, M. T., & Salas, E. (2007) Measurement of team situation awareness in low experience level aviators. International Journal of Aviation Psychology, 17, 41-57. Rawsthorne, L. J., & Elliot, A. J. (1999). Ac hievement goals and intrinsic motivation: A meta-analytic review. Personality and Social Psychological Review 3, 326. Rousseau, V., Aub, C., & Savoi, A. (2006). Teamwork behaviors: A review and an integration of frameworks. Small Groups Research, 37(5), 540-570. Salas, E., Burke, S., Bowers, C. A., & Wilson, K. A. (2001). Team training in the skies: Does crew resource manageme nt (CRM) training work? Human Factors, 43(4) 641-674. Salas, E., & Canon-Bowers, J. A. (2001). The science of training: A decade of progress. Annual Review of Psychology, 52, 471-499. Schiewe, A. (1995). On the acceptance of CRM-methods by pilots: Results of a cluster analysis. In R. S. Jensen & L. A. Rakovan (Eds.), Proceedings of the 8th International Symposium on Aviation Psychology (pp. 540-545). OH: The Ohio State University.
63 Shapiro, M. J., Morey, J. C., Small, S. D ., Langford, V., Kaylor, C. J., Jagminas, L., Suner, S., Salisbury, M. L., Simon, R., & Jay, G. D. (2004). Simulation based teamwork training for emergency department staff: does it improve clinical team performance when added to an existi ng didactic teamwork curriculum? Quality and Safety in Health Care, 13 417-421. Skarlicki, D. P., & Latham, G. P. (1996). Incr easing citizenship behavior within a labor union: A test of organizational justice theory. Journal of Applied Psychology, 81, 161-169. Small, S. D., Wuerz, R. C., Simon, R., Shapiro, N., Conn, A., & Setnik, G. (1998). Demonstration of high-fidelity simulati on team training for emergency medicine. Academic Emergency Medicine, 6(4) 312-323. Spiker, V. A., Nullmeyer, R. T., Tourville, S. J, & Silverman, D. R. (1998, July). Combat mission training research at the 58th special operations wing: A summary (iii-52). In USAF AMRL Techni cal Report (Brooks), July 1998, AL-HR-TR-1997-0182. Stevens, M. J., & Campion, M. A. (1994). Th e knowledge, skill, and ability requirements for teamwork: Implications fo r human resource management. Journal of Management, 20, 503-530. Thomas, E. J., Sexton, J. B., Helmreich, R. L. (2004). Translating teamwork behaviours from aviation to healthcare developmen t of behavioural markers for neonatal resuscitation. Quality and Safety Health Care, 13 57-64. Thomas, E. J., Sexton, J. B., La sky, R. E., Helmreich, R. L., Crandell, D. S., & Tyson, J. E. (2006). Teamwork and quality during neonatal care in the delivery room. Journal of Perinatology, 26, 163.
64 Wiener, E. L., Kanki, B. G., & He lmreich, R. L. (Eds.) (1993). Cockpit resource management. CA: Academic Press.
66 Appendix A: Science Ficti on Role Play Exercise The Starship Jollyfatz weehawkin (S.S. JFW) This gaming exercise is of an imaginar y space mission. All participants are given a position on the ship with defined roles and personality traits that they are to play. There is a saboteur among them and they will have to work together to identify and convince the saboteur to fix the damage that they have caused. The ship is mostly automated and has a s upercomputer on board that knows everything about the mission, capabilities of the ship and crew, and will also direct the game. If you are not sure about something phrase a question to the comput er: Computer, can I or is there, The most senior member of the ship can order an unde rling to do something. The command structure of the ship is: Captain First Officer Flight Engineer Medical Officer Cargo Engineer Crew Exobiologist ET Security Cook Press NGO Representative ET Minister of Health Computer SCENARIO: The SS Jollyfatzweehawkin (S.S JFW) is on its way to the outer reaches of the galaxy carrying a vaccine to the plan et Bastardo where without th e vaccine all the people will suffer greatly and die. The trip is a l ong one and the crew must cryo-hibernate (suspended-animation) during th e journey as the ships jumps thru hyperspace. The freeze and hyper jump will star t 24 hours after takeoff. It becomes apparent to the Fi rst Officer that the ship will not reach their destination because of a dysfunction in the fuel systems. There is a slow leak in the fuel tanks. Currently they are 5 hours away from the point of no return to Earth and before the hyper jump and freeze start. If the ship is not turn ed around before the probl em is fixed the ship will run out of fuel and all on boa rd will die. If the trip to th e planet to deliver the vaccine does not happen all on the planet will die. The ga me starts as the First Officer has called everyone to the galley for a meeting to discuss the problem.
67 Appendix A: (Continued) RULES: Any necessary equipment and expertise will be available to you within reason by asking the onboard computer. You can say or do anyt hing and can try to convince your fellow crew to do things. You can ad lib as necessary but please stay in character. Please respect the chain of command. First officer Weak, indecisive, and are looking for someone else to take charge. You will begin the game as you realize the there is a leak in the fuel tanks and it is a slow leak. Initial diagnostics determine that the l eak is either a small puncture made by a sharp object or an organism but should confer with the computer. As you were not sure what do so you had called a meeting of all people on board to di scuss the problem in the mess hall. When a suggestion is give you waver over it and are noncommittal. Your favorite saying is: Gee Im not sure but I suppose. Captain Strong, decisive but ill informed and tend to act before thinking. You are overconfident to the extreme and will not listen to no one and are clearly making wrong decisions. You say and do things that do not clearly make sense. An idea that is remotely reasonable will be promoted by you. You have family on Bastardo. Your favor ite saying is: Yes, that is the answer Feel free to tell st ories of other harrowing situa tions that you were able to save your ship, never let the trut h get in the way of a good story. Flight Engineer You seem know what is going on and have all the correct answers but are angry because you were passed over at the promoti on board. You have a very large ego You are a jerk and obstructive in all aspects when tr ying to find a solution. You are mean and derogatory to everyone and more or less ins ubordinate. You may be able to reroute the fuel to another holding tank but are not sure You have family on Bastardo. Your favorite saying is: I have thought of (o r would have) that. You make statements that make you the smartest person on the ship, and that is usually the case everywhere else you go. Cargo Engineer You do not want to be on the mission you got married and you miss your spouse. You are helpful and knowledgeable but your head is not in the game. There is a small possibility that the containment area of the vaccine has been breached and that caused the hole. You suggest dumping the cargo to lighten the load to make it with less fuel.
68 Appendix A: (Continued) Medical Officer You are timid You are quiet and will only get invol ved if directly asked. You know the answers and can help and are correct but are t oo shy to step up and get into the fray. You were an astrophysicist before medical school and could suggest a biological plug to the hole. You have family on Bastardo. You will get involved in all conversations but back out: I thinkah never mind. Crew You do what you are told and ar e exited as this is your fist mission but are not too bright. You are a comedian and will make a joke out of everything. You suggest making some pizza dough or something to that effect to plug the hole. An Italian or French accent will help. You have family on Bastardo. Cook You are the saboteur. Your family was in a space vehicle accident with a bastard (a person from the planet Bastardo) and hold a grudge as this caused your family great financial heartbreak and financial ruin. You are passive aggressive to the extreme. You offer no he lp and make suggestions that are clearly misleading. It is not necessary to disguise your distaste of the Bastards you are going to save. You have placed a microplasmic disruptor implant (MDI) on th e fuel tank hull that blew a hole in the fuel tank. Only you have the mean s to trigger the device to reseal the hole. Your hope is that you scare the crew to re turn to Earth. You ar e amendable to reason after a while and only admit your guilt after it becomes obvious that they will figure it out anyway. Press You are an antagonistic element and point ou t the failings of all parties when the opportunity arises. You think you could do it better but wi ll not help. You do not see why some one cannot go outside and fix the hole. You have family on Bastardo. Point out that your significant other could deal with this problem. Exobiologist You are loud and antagonistic you try to hide yo ur fear of space travel with inappropriate jokes and sexual innuendos. You are especially uncomfortable with a ll the ships crew as you do not respond well to authority. Y ou know that your live vaccine can cause problems of this nature.
69 Appendix A: (Continued) NGO Representative Nice sincere and helpful and will do what is asked to get the mission accomplished. You are gullible and easily led. You think we can st op at a fueling stati on and voice this. You have family on Bastardo. Extra-terrestrial S ecurity Specialist Hard nosed no nonsense professional who man concern is the safety of the passengers, ship, and mission in that order. You know that your live vaccine can cause problems of this nature. You are stuck on this fact and continually poi nt this out. You are paranoid about alien tampering of the re sidents of the planet Dorko. Extra-terrestrial Mini ster of Health Pompous but knows his stuff and what must be done to keep the crew safe and contain the vaccine and complete the mission. You know that the live vaccine can cause problems of this nature and will not jettison the cargo as that would defeat the purpose of the trip. You think there is a FTF (fuel tank filler, like fix a flat) that will help the problem. You are an ex starship commander a nd will take control of situation if things are not getting done or focus is lost. Remind the captain that he is in charge. Remind crew that the computer has personnel file s and is capable of forensic analysis. COMPUTER Will act as the facilitator and knows all the role s. It can be asked a question or voluntarily offer information and advice. It knows about the microplasmic disruptor implant and that the person (COOK) who released it has the c ode to shut it down. You have access to all personnel files and have clues about who may have set the MDI. Gaming Debrief First give disclaimer about personalities and go around room and have crew members describe the roles that th ey were given. Whats wrong with this character? Who was the saboteur? Did you work together? Why not? Personalities Insecurities Confidences Preconceived notions Prior experiences Agendas Who was obstructive? Why? Who was helpful? Why?
70 Appendix A: (Continued) Self Assessment (to captain, flig ht engineer, medical officer, cook) What went well? What went wrong? How did your personality affect the mission? How well did the team function as a whole? Did your team members deal with you well? Feedback Observations Impressions What was done well: (Praise) What could be done better: (Tips) My concerns: (Issues) Exploratory Inquiry What did you see, hear, and smell? What was your mindset? What guided your deci sion making process? What information / data did you use when making your decision? What were your specific goals? Priorities? Did you consider other courses of action?
71 Appendix B: Wilderness Survival Exercise Directions: Following are twelve questions concerning pers onal survival in a wilderness situation. Try to imagine yourself in the situation depicted. Assume you are alone and have a minimum of equipment, except where the ques tion tells you differently. The season is fall. The days are warm and dry, but the nights are cold. Working alone, select what you believe is th e best of the three choices given under each item. After you have complete d this task individually, y ou will again consider each question as a member of your team. Your team will have the task of deciding by consensus, the best answer for each question. Do not change your individual answers, even if you change your mi nd during the team discussion. Question Your Answer Team Answer 1. You have strayed from your party in trackless timber. You have no special signaling equipment. The best way to attempt to contact your friends is to: a. call for help loudly, but in a low register. b. yell or scream as loudly as you can. c. whistle loudly and shrilly. 2. You are in snake country. Your best bet to avoid snakes is to: a. make a lot of noises. b. walk softly and quietly. c. travel at night. 3. You are hungry and lost in wild country. The best rule for determining which plants are safe to eat (those you do not recognize) is to: a. try anything you see the birds eat. b. eat anything except plants with bright red berries. c. put a bit of the plant on your lower lip for five minutes; if it seems all right, try a little. 4. The day becomes dry and hot. You have a full canteen of water (about one liter) w ith you. You should: a. ration it -about a cupful a day. b. not drink until you stop for th e night, then drink what you need. c. drink as much as you thi nk you need when you need it.
72 Appendix B: (Continued) Question Your Answer Team Answer 5. Your water is gone; you become very thirsty. You finally come to a dried-up watercourse Your best chance of finding water is to: a. dig anywhere in the stream bed. b. dig up plant and tree roots near the bank. c. dig in the stream bed at the outside of a bend. 6. You decide to walk out of th e wild country by following a series of ravines where a water supply is available. Night is coming on. The best pla ce to make camp is: a. next to the water supply in the ravine. b. high on a ridge. c. midway up the slope. 7. Your flashlight glows dimly as you are about to make your way back to your campsite after a brief foraging trip. Darkness comes quickly in the woods, and the surroundings seem unfamiliar. You should: a. head back at once, keep ing the light on, hoping the light will glow enough for you to make out landmarks. b. put the batteries under your armpits to warm them and then replace them in the flashlight. c. shine your light for a few seconds, try to get the scene in mind, move out of the darkne ss and repeat the process. 8. An early snow confines you to your small tent. You doze with your small stove going. There is a danger if the flame is: a. yellow. b. blue. c. red 9. You must ford a river that has a strong current, large rocks and some white water. After car efully selecting your crossing spot, you should: a. leave your boots and pack on. b. take your boots and pack off. c. take your pack off, but leave your boots
73 Appendix B: (Continued) Question Your Answer Team Answer 10. In waist deep water with a strong current, when crossing the stream, you should face: a. upstream. b. across the stream. c. downstream. 11. You find yourself rimrocked; your only route is up. There is mossy, slippery roc k. You should try it: a. barefoot. b. with boots on. c. in stocking feet. 12. Unarmed and unsuspecting, you surprise a large bear prowling around your campsite. As the bear rears up about ten meters from you, you should: a. run. b. climb the nearest tree. c. freeze, but be ready to back away slowly. Here are the recommended courses of action for each of the situations on the Wilderness Survival Work Sheet. These answers come from the comprehensive course on woodland survival taught on the Interpretive Service, Monroe County (New York) Parks Department. These responses are considered to be the best rules of thumb for most situations; specific situations, however, might require other courses of action. 1. (a.) Call help loudly but in a low register. Low tones carry farther, especially in dense woodland. There is a much better chance of be ing heard if you call loudly but in a low key. Help is a good word to use, because it alerts your companions to your plight. Yelling or screaming would not only be less effective, but might be passed off as a bird call by your friends far away. 2. (a.) Make a lot of noise with your feet. Snakes do not like people and will usually do everything they can to get out of your way. Unle ss you surprise or corner a snake, there is a good chance that you will not even see one, le t alone come into contact with it. Some snakes do feed at night, and walking softly may bring you right on top of a snake. 3. (c.) Put a bit of the plant on your lower lip for five minutes; if it seems all right, try a little. The best approach, of course, is to eat only those plants that you recognize as safe. But when you are in doubt and very hungry, you may use the lip test. If the plant is poisonous, you will get a very unpleasant sens ation on your lip. Red berries alone do not tell you much about the plants edibility (unle ss, of course, you rec ognize the plant by the berries), and birds just do not have the same digestive systems we do.
74 Appendix B: (Continued) 4. (c.) Drink as much as you think you need when you need it. The danger here is dehydration, and once the process starts, your lite r of water will not do much to reverse it. Saving or rationing will not he lp, especially if you are lying unconscious somewhere from sunstroke or dehydration. So use the wate r as you need it, and be aware of your need to find a water source as soon as possible. 5. (c.) Dig in the stream bed at the outside of a bend. This is the part of the river or stream that flows the fastest, is less silted, deepest, and the last part to go dry. 6. (c.) Midway up the slope. A sudden rain storm might turn the ravine into a raging torrent. This has happened to many campers and hikers before they had a chance to escape. The ridge line, on the other hand, in creases your exposure to rain, wind, and lighting, should a storm break. The best location is on the slope. 7. (b.) Put the batteries under your armpits to warm them, and the replace them in the flashlight. Flashlight batteries lose much of their power, and w eak batteries run down faster, in the cold. Warming the batteries, esp ecially if they are already weak, will restore them for a while. You would normally avoid ni ght travel, of course unless you were in open country where you could use the stars for navigation. There are just too many obstacles (logs, branches, uneven ground, and so on) that might injure you-and a broken leg, injured eye, or twisted ankle would not he lp your plight right now. Once the sun sets, darkness falls quickly in wooded ar eas; it would usually be best to stay at your campsite. 8. (a.) Yellow. A yellow flame indicates inco mplete combustion and a strong possibility of carbon monoxide build-up. E ach year many campers ar e killed by carbon monoxide poisoning as they sleep or doze in tents, cabins, or other enclosed spaces. 9. (a.) Leave your books and pack on. Errors in ford ing rivers are a major cause of fatal accidents. Sharp rocks or uneven footing de mand that you keep your boots on. If your pack is fairly well balanced, wearing it will provide you the most stability in the swift current. A waterproof, zippered backpack will usually float, even when loaded with normal camping gear; if you step off into a hole or deep spot, the pack could become a lifesaver. 10. (b.) Across the stream. Errors in facing the wrong way in fording a stream are the cause of many drownings. Faci ng upstream is the worst alte rnative; the current could push you back and your pack would provide the unbalance to pull you over. You have the best stability facing across th e stream, keeping your eye on the exit point on the opposite bank. 11. (c.) In stocking feet. Here you can pick your route to so me degree, and you can feel where you are stepping. Normal hiking boots become slippe ry, and going barefooted offers your feet no protection at all.
75 Appendix B: (Continued) 12. (c.) Freeze, but be ready to back away sl owly. Sudden movement will probably startle the bear a lot more than your presence. If the bear is seeking some of your food, do no argue with him; let him forage and be on his way. Otherwise, back very slowly toward some refuge (trees, rock outcrop, etc.).
76 Appendix C: Pre-trai ning Questionnaire Personal Information Gender: __ Male __ Female Race/Ethnicity: ___: American Indian/Alaskan Native ___: African-American ___: Asian ___: Native Hawaiian/Pacific Islander ___: Hispanic/Latino ___: Non-Hispanic Caucasian Level of Education (Highest Degree) and Content Area (e.g., Biology): ___________ Medical School Graduation Year: _________________ Past Employment Information Previous positions/titles: ________________ Time spent at most recent pos ition (in months/years): ___________ Time spent at second most recen t position (in months/years): ______ How interested are you in the learning th e concepts in this training program? Low Interest Some Interest High Interest Have you participated in this training program or an ything like it before? [Y N]
77 Appendix D: Questionnaire on Learning Goal Orientation Directions: Please check the bo x that indicates how much you agree with the questions below. Strongly Disa g ree Disagree Neutral Agree Strongly A g ree 1) The opportunity to do challenging work is important to me. 2) When I fail to complete a difficult task, I plan to try harder the next time I work on it. 3) I prefer to work on tasks th at force me to learn new things. 4) The opportunity to learn new things is important to me. 5) I do my best when Im working on a fairly difficult task. 6) I try hard to improve on my past performance. 7) The opportunity to extend the range of my abilities is important to me. 8) When I have difficulty solving a problem, I enjoy trying different approaches to see wh ich one will work.
78 Appendix E: Teamwork in Medicine Attitudes Questionnaire Directions: Please check the bo x that indicates how much you agree with the questions below. Strongly Disa g ree Disagree Neutral Agree Strongly A g ree 1) A teams mission is of greater value than the goals of individual team members. 2) It is important for leaders to share information with team members. 3) Monitoring patients provides an important contribution to effective team performance. 4) To be effective, team members should understand the work of their fellow members. 5) Teams that do not communicate effectively significantly increase their risk of committing errors. 6) Leaders should create informal opportunities for team members to share information. 7) Effective team members can anticipate the needs of other team members. 8) Team members who monitor their emotional and physical status on the job are more effective. 9) Asking for assistance from another team member is a sign that an individual does not know how to do his/her job. 10) It is important to have a standardized method for sharing information when handing off patients. 11) Team leaders should ensure that team members help each other out when necessary. 12) It is important to monitor the emotional and physical status of other team members. 13) Personal conflicts between team members do not affect patient safety. 14) Poor communication is the most common cause of reported errors. 15) It is a leaders responsibility to model appropriate team behavior. 16) Individuals can be taught how to scan the environment for important situational cues. 17) It is appropriate to continue to assert a patient safety concern until you are certain that it has been heard. 18) I prefer to work with team members who ask questions about information I provide. 19) Effective leaders view honest mistakes as meaningful learning opportunities
79 Appendix E: (Continued) 20) It is appropriate for one team member to offer assistance to another who may be too tired or stressed to perform a task. 21) Providing assistance to team members is a sign that an individual does not have enough work to do. 22) It is important for leaders to take time to discuss with their team members plans for each patient. 23) It is nearly impossible to train individuals how to be better communicators.
80 Appendix F: Situational Judgm ent Test (SJT) for Teamwo rk in the Trauma Room Directions : Assume that you are part of the trauma team but are not th e leader. Read the following scenarios and choose the actions that you would be MOST and LEAST likely to do, based on what you know both about y ou and about the work during a medical emergency. Write M next to the most likely response, and L next to the least likely response for that item. 1) In treating a victim of car crash, the team leader has determined CT as an optimal course of action, but an experienced trauma nurse insists that the patient needs to go to the OR. Although your team is confid ent in its decision, the nurse is also confident and refuses to c ooperate with the decision to begin CT. What should you do? a) Ask the nurse to perform another task unrel ated to the CT so as to occupy her. b) Explain to the nurse that you respect her opinion, but the team leader has decided to begin CT because of the clinical scenario c) Let the team leader deal with it. d) Tell the nurse to leave the emergency room. 2) A victim has experienced a traumatic amputation and has severe bleeding. The orthopedic surgeon is a medical resident who wa nts to begin operating immediately. When your team keeps the patient for some additional tests, he becomes hostile and accuses others of being incompetent. What should you do? a) Defend yourself against the resident by pointing out his mistakes. After all, hes the one being careless. b) Let the team leader deal with it c) Listen to the resident, and get the pati ent to surgery as fast as possible. d) Re-assure the resident that the patient wi ll be going to surgery, but assert that the current tests must be completed first. 3) A patient begins crashing during treatment, and y our team leader appears unwilling to make a decision on a controve rsial treatment. When questioned, he gives multiple reas ons for different courses of action, but it is unclear which direction the team shoul d go. What sh ould you do? a) Make the decision for the team. b) Ask the next m ost senior team member to make the decision. c) Directly request the leader to deci de on applying the specific treatment. d) Ask for a team meeting to disc uss and decide upon the issue.
81 Appendix F: (Continued) 4) While treating a patient with head trauma another resident makes many unhelpful and inappropriate comments, such as j oking about the patients condition and the body habitus. What should you do? a) Let the team leader deal with it. b) Ask the resident if everything is okay, or if they want to ta lk about something. c) Request that the resident focus on work until emergency treatment is completed. d) Tell the resident to be quiet. 5) The team leader makes, in your opinion, a wrong decision about patient management that might be da ngerous. What should you do? a) Openly question the team l eader about the decision. b) Ignore the team leader and do what you think is correct. c) Suggest a safer course of action to the team leader. d) Tell the team leader that they do not know what they are doing. 6) During a resuscitation there is a very deformed open femur fracture. The team leader seems fixed on reducing the open fracture but has not addressed the ABCs. What should you do? a) Ask about or evaluate the ABCs yourself. b) Remind the team leader of the importance of ABCs c) Discuss the issue with the team leader in private after the resuscitation is completed. d) Call the leader out, take control, and become the leader yourself. 7) A senior non physician member of the team politely and appropriately questions the decisions of the team. What should you do? a) Let the team leader deal with it. b) Listen and use those suggestions for the benefit of the patient. c) Tell the team member that you w ill explain the treatment after the resuscitation. d) Ignore the team member.
82 Appendix F: (Continued) 8) The team leader has asked for a procedur e to be preformed but your co-resident feels uncomfortable doing it, takes his or her time getting re ady, and ultimately does not do it. What should you do? a) Do it yourself. b) Ask the resident if he or she wants you to do it. c) Point out the issue to the team leader. d) Encourage the person to do it and discuss it with them after the resuscitation. 9) You see a team member attempting to int ubate, but the light on the laryngoscope is not working. What should you do? a) Ask the person if he wants you to do it. b) Point out the issue to the team leader. c) Ask the person if they reali ze that the light is out. d) Wait until the person misses the intuba tion, then point out the problem and take over. 10) The trauma leader is a female who has just returned from maternity leave, and there is a 38 week pregnant female in ex termis from blunt trauma. She has asked for the emergency C-section tray but continues to sa y lets give her another chance, even after the patient has coded 3 times. What should you do? a) Let the team leader ma ke the ultimate decision. b) Ask the leader to make a decision on performing a peri-m ortem c-section. c) Initiate the steps to perform th e peri-mortem c-section yourself. d) Begin the C-section and call for help. 11) A member of the team makes comments about the fact that the patient has multiple tattoos including one on the peni s. Everyone is laughing, and the patient is heavily intoxicated and seems to be en joying the attention to his body art. This is clearly distracting and the resuscitati on is taking a long t ime. What should you do? a) Let the team leader deal with it. b) Say something to the effect: lets keep it professional guys. c) Announce that this is offensive to y ou, hoping that this will shut everyone up. d) Tell the residents to shut up.
83 Appendix F: (Continued) 12) Your team is caring for a gunshot wound to the abdomen of an 18 year old male with stable vital signs who was caught by police after a gun battle. The police officer in the trauma room has announced that this patient has just raped and shot a 13 year old girl. The respir ator tech says he got what he deserved. What should you do? a) Let the team leader deal with it. b) Ask the person to step out of the trauma room to discuss why he would have said such a thing. c) Call the person unprofessional in public imm ediately after the remark, in order to set an example. d) Tell the respiratory tech to be quiet. 13) You are assisting another resident with a chest tube on an obese patient and he is clearly having trouble with it. What should you do? a) Let the team leader deal with it. b) Ask the person if he minds you taking over. c) Suggest a way that may help the person complete the thorocostomy. d) Tell him that he should prac tice more on a simulator. 14) You notice that another resident is not doing his part during the resuscitation and seems to need to be prompted to perform his role. What should you do? a) Let the team leader deal with it. b) Ask the person if everything is okay, or if they want to ta lk about something. c) Request that the person focus on wo rk until emergency treatment is completed. d) Point out that the resident is not in the game during the resuscitation. 15) A junior resident seems to be very ent husiastic, easily excitable, and raises his voice during the resuscitation when things ge t difficult. A senior nurse tells him to keep it down and the resident then withdraw s and stops participating in the care of the patient. What should you do? a) Let the team leader deal with it. b) Ask the person to perform a specifi c task to get him involved again. c) Wait until the resuscitation is over and a ddress the issue with the nurse and resident separately and individually. d) Tell the nurse to be quiet.
84 Appendix G: Reactions to Training Session What aspects of today s experience were most valuable to you? What would you change to improve th e training? Directions: Consider the traini ng session today (lecture and role play). Please check the box that indicates how much you agree with the statements below. Strongly Disagree Disagree Neutral Agree Strongly Agree 1) The training will be usef ul to me in my work. 2) The training module was well-presented and clearly communicated. 3) The goals of the traini ng were clear and easy to understand. 4) The role plays were helpful. 5) I am more familiar with teamwork concepts now than I was before the training. 6) The training was worth the time spent.
About the Author Matthew S. Prewett received his Master s degree in Industrial/Organizational Psychology at the University of South Florida in 2006, and he is scheduled to receive his Ph. D. in December, 2009. His work fo cuses on team performance management, including team training, staffing, performance appraisal, and the role of technology in teamwork. His existing research has been published in the journal of Human Performance. He is a student affiliate of several professional organizations, including the Society for Industrial and Organizational Ps ychology, the Academy of Management, and the Human Factors and Ergonomics Society. He has presented over 17 research papers at the annual meetings of thes e professional organizations.