USF Libraries
USF Digital Collections

An evaluation of the influence of case-method instruction on the reflective thinking of MSW students

MISSING IMAGE

Material Information

Title:
An evaluation of the influence of case-method instruction on the reflective thinking of MSW students
Physical Description:
Book
Language:
English
Creator:
Milner, Marleen
Publisher:
University of South Florida
Place of Publication:
Tampa, Fla
Publication Date:

Subjects

Subjects / Keywords:
Decision cases
Reflective Judgment Model
Critical thinking
Social work education
Learning outcomes
Dissertations, Academic -- Social Work -- Doctoral -- USF   ( lcsh )
Genre:
non-fiction   ( marcgt )

Notes

Summary:
ABSTRACT: Social work practice requires that graduates be prepared to deal with complex, multifaceted problems which cannot be defined completely, do not have absolute, correct answers and can be approached from multiple perspectives. This study evaluated the influence of case-based instruction on MSW students' reflective judgment, an aspect of critical thinking associated with the ability to reason through ill-structured problems. (King, Wood, & Mines, 1990). The Reflective Judgment Model, which describes a developmental continuum based upon epistemic assumptions regarding the source and justification of knowledge claims, served as the theoretical framework for the assessment of reflective thinking in this mixed methods study. A quasi-experimental pre-post nonequivalent control group design was utilized to explore whether students who participated in a case method course demonstrated greater increases in reflective judgment than those who did not.MSW students enrolled in a case-based capstone course at a major metropolitan university in the southeast served as the intervention group, while foundation year students enrolled in a research methodology course served as the comparison group. Both groups completed the Reasoning about Current Issues Test (RCI), which is an online, standardized measure that has been widely used to assess reflective judgment (Wood, Kitchener, & Jensen, 2002) at pre and posttest. Content analysis procedures were used to facilitate assessment of students' initial and final case analysis papers for evidence of changes in the reflective thinking skills and problem-solving approaches utilized on initial and final case analysis papers. The case method participants' mean RCI scores remained unchanged between pre and posttest, while RCI posttest scores of participants in the control group decreased significantly.Pre and posttest comparison of students' case analysis papers using a customized rubric based on Wolcott's Steps for Better Thinking (2006) similarly indicated no mean changes in problem-solving approaches between pre and posttest. However, students who began the course using strategies associated with pre-reflective judgment increased their scores on the rubric significantly while those who exhibited higher levels of quasi-reflective judgment at pretest decreased at posttest. Strategies for designing a developmental curriculum to target the reflective judgment levels of MSW students are proposed.
Thesis:
Dissertation (Ph.D.)--University of South Florida, 2009.
Bibliography:
Includes bibliographical references.
System Details:
Mode of access: World Wide Web.
System Details:
System requirements: World Wide Web browser and PDF reader.
Statement of Responsibility:
by Marleen Milner.
General Note:
Title from PDF of title page.
General Note:
Document formatted into pages; contains 320 pages.
General Note:
Includes vita.

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
aleph - 002063941
oclc - 558634977
usfldc doi - E14-SFE0003025
usfldc handle - e14.3025
System ID:
SFS0027342:00001


This item is only available as the following downloads:


Full Text

PAGE 1

An Eval uation of the Influence of Case Method Instruction on the Reflective Thinking of MSW Students by Marleen Milner A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy School of Social Work College of Behavioral and Community Sciences University of South Florida Major Professor: William Rowe, DSW Jessica Cabness, Ph.D. T. Grandon Gill, Ph.D. Michael Rank, Ph. D. Terry Wolfer, Ph.D. Date of Approval: July 1, 2009 Keywords: dec ision cases, Reflective Judgment Model, critical thinking, social work education learning outcomes, case based teaching Copyright 2009, Marleen Milner

PAGE 2

Dedication I would like to dedicate this dissertation to my mother Ofelia Morales, who has always inspired me to be and do my best. Your life long commitment to excellence has encouraged m e to press on to the next hill and let me know you understood when I had to forfeit our favorite Saturday outings. I would also like to dedicate this dissertation to my students who I hope will benefit from what I have learned. Finally I dedicate this work to my Lord and Savior, Jesus Christ, who is the wind beneath my wings.

PAGE 3

A cknowledgments The former pres ident of the university in wh ich I teach is fond o f quoting a Texas proverb that claims, turtle and I would like to acknowledge the many people that have helped me to find my place on this fence post First, I would l ike to acknowledge Mack Milner, my husban d of 31 years whose willingness to sacrifice our comfort able pre doctoral studies life together and has e nabled me to make it through endless weekends and nights of analysis and re visi ons Secondly, I would like to thank Terry Wolfer whose enthusiasm for case method teaching helped conceive this study. Special thanks to Roger Boothroyd and Mary Armstrong who provided patient counsel and instruction in quantitative and qualitative research methods. I am grateful for my colleagues Irv Zieman, Drucella Crutchfield, and Velmarie Albertini who volunteered their time to edit and provide feedback on dissertation chapters as well to Grace V each, my brilliant coder who sacrificed her summer leisur e to code student papers. Special thanks go to my friend and colleague, Pam Criss who has support ed me in my frustrations and shared in my small victories Finally, I would like to thank each of the members of my committee, Dr. Bill Rowe, Dr. Jessica Cabn ess, Dr. Michael Rank, Dr. Grandon Gill, and Dr. Terry Wolfer who each have loaned me their expertise and extensive experience and provided in valuable feedba ck that helped shape this study.

PAGE 4

i Table of Contents List of Tables ................................ ................................ ................................ ..................... vi List of Figures ................................ ................................ ................................ ................... vii Abstract ................................ ................................ ................................ ............................ viii Chapter I ................................ ................................ ................................ ............................. 1 Problem Statement ................................ ................................ ................................ .. 2 Use of the Case Method to Promote Reflective Thinking ................................ ...... 3 Reflective Judgment Model ................................ ................................ .................... 5 Research Questions ................................ ................................ ................................ 8 Chapter II ................................ ................................ ................................ ............................ 9 Critical Thinking as an Educational Outcome ................................ ........................ 9 Defining Critical Thinking ................................ ................................ ........ 10 Critical Thinking and Social Work Education ................................ ...................... 12 Relationship between Critical Thinking and Reflective Thinking ....................... 17 Problem Structure ................................ ................................ ..................... 18 Uncertainty ................................ ................................ ................................ 19 Relationship to Epistemic Assumptions ................................ ................... 20 Reflective Judgment Model ................................ ................................ .................. 22 Theoretical Foundation ................................ ................................ ............. 22 Research on the Reflective Judgment Model ................................ ............ 25 Distinctive construct. ................................ ................................ .... 25 Developmental sequence. ................................ ............................. 26 Age/educational level ................................ ................................ .... 27 Gender ................................ ................................ ........................... 28 Race/Ethnicity ................................ ................................ ............... 29 Time between testing ................................ ................................ .... 29 Discipline ................................ ................................ ...................... 31 Assessing Reflective Judgment ................................ ................................ 31 Reflective Judgment Interview ................................ ..................... 32 Reasoning about Current Issues Test ................................ ............ 34 Summary ................................ ................................ ................................ ... 36 Promoting Reflective Thinking ................................ ................................ ............. 36 Case Method Instruction ................................ ................................ ....................... 38 History of Case Based Instruction in Social Work ................................ ... 40 Foster ing Reflective Thinking through Case Method Instruction ........................ 42

PAGE 5

ii Research on the Case Method and Reflective Thinking ................................ ....... 44 Case Analysis as a Method of Gauging Reflective Thought .................... 44 Case Analysis and Increases in Reflective Thinking ................................ 45 Case Discussion as Integral t o Fostering Reflective Thought .................. 47 Impact of Case Discussion on Epistemology ................................ 48 Case Method Research in Social Work Education ................................ ............... 49 The Case Method as a Model for Meeting Educational Objectives ......... 50 Assessing Perceived Learning Outcomes ................................ ................. 5 2 Limitations of the Case Method ................................ ................................ ............ 54 Summary ................................ ................................ ................................ ............... 55 Chapter III ................................ ................................ ................................ ......................... 57 Hypotheses ................................ ................................ ................................ ............ 57 Variables ................................ ................................ ................................ ... 59 Independent Variables ................................ ................................ .. 59 Dependent Variables ................................ ................................ ..... 59 ................................ ............................... 62 Description of Setting ................................ ................................ ............... 63 Description of Case Method Course ................................ ......................... 64 Participants ................................ ................................ ................................ 67 Intervention Group ................................ ................................ ........ 67 Comparison Group ................................ ................................ ........ 68 Sample Size ................................ ................................ ................... 70 Process and Procedures ................................ ................................ ......................... 72 Recruitment of Participants ................................ ................................ ....... 72 Intervention Group ................................ ................................ ........ 72 Comparison Group. ................................ ................................ ....... 74 Attrition ................................ ................................ ................................ ..... 75 Permissions ................................ ................................ ............................... 78 Testing Procedure ................................ ................................ ..................... 79 Collection of Case Analysis Papers ................................ .......................... 80 Instrumentation ................................ ................................ ................................ ..... 80 Description of RCI ................................ ................................ .................... 81 Reliability. ................................ ................................ ..................... 82 Validity. ................................ ................................ ........................ 83 Content Analysis Procedures ................................ ................................ ................ 83 Development of Content Analysis Themes ................................ .............. 84 Reflective Thinking Skills ................................ ............................ 87 Rationale. ................................ ................................ ...................... 88 Unit of Analysis ................................ ................................ ........................ 88 System of Enumeration ................................ ................................ ............. 89 Sc oring. ................................ ................................ ......................... 90 Internal Consistency ................................ ................................ .................. 91 Inter Rater Reliability ................................ ................................ ............... 92 App roach to Data Analysis ................................ ................................ ................... 95

PAGE 6

iii Chapter IV ................................ ................................ ................................ ......................... 98 Prescreening of Data ................................ ................................ ............................. 98 Equality of Groups on Demographic Factors ................................ ....................... 99 Gender ................................ ................................ ................................ ....... 99 Race ................................ ................................ ................................ ......... 100 Social Work Experience ................................ ................................ ......... 101 Age ................................ ................................ ................................ .......... 101 Hypotheses Related to Demographic Factors ................................ ..................... 103 Gender ................................ ................................ ................................ ..... 103 Hypothesis 1.1. ................................ ................................ ............ 103 Race/Ethnicity ................................ ................................ ......................... 104 Hypothesis 1.2 ................................ ................................ ............. 104 Age ................................ ................................ ................................ .......... 106 Hypothesis 1.3. ................................ ................................ ............ 106 Years of Socia l Work Experience ................................ ........................... 107 Hypothesis 1.4 ................................ ................................ ............. 107 Hypothesis Related to RCI Pre and Posttest Scores ................................ ........... 108 Hypothesis 2.1 ................................ ................................ ............. 110 Hypothesis 2.2. ................................ ................................ ............ 111 Hypothesis 2.3. ................................ ................................ ............ 113 Hypotheses Related to Content Analysis Procedures ................................ ......... 113 Internal Consistency ................................ ................................ ................ 114 Comparison of Initial a nd Final Rubric Scores ................................ ....... 115 Hypothesis 3.1 ................................ ................................ ............. 115 Comparison by individual item scores ................................ ........ 118 Comparison by Case. ................................ ................................ .. 120 Comparison by Rationale. ................................ ........................... 121 Comparison by Section. ................................ .............................. 125 Correlation between Rubric and RCI Scores ................................ .......... 127 Hypothesis 3.2 ................................ ................................ ............. 127 Qualitative Analysis ................................ ................................ ............................ 128 Case Descriptions and Observations ................................ ....................... 128 Decision Case #1. ................................ ................................ ........ 129 De cision Case #7. ................................ ................................ ........ 131 Decision Case #9. ................................ ................................ ........ 131 Decision Case #10 ................................ ................................ ....... 132 Decision Case #11 ................................ ................................ ....... 133 Decision Case #12 ................................ ................................ ....... 134 Reflective Thinking Performance Patterns ................................ ......................... 134 Pre Reflective Performance Pattern 0 ................................ ..................... 136 Problem Statement ................................ ................................ ...... 137 Contextual Analysis. ................................ ................................ ... 138 Alternative Solutions and Recommendation. ............................. 140 Global Ratings ................................ ................................ ............ 147

PAGE 7

iv Quasi Reflecti ve Performance Pattern 1 ................................ ................. 147 Problem Statement ................................ ................................ ...... 148 Contextual Analysis ................................ ................................ .... 149 Alternative Solutions and Recommendation .............................. 153 Global Ratings ................................ ................................ ............ 156 Quasi Reflective Performance Pattern 2 ................................ ................. 156 Problem Statement ................................ ................................ ...... 157 Contextual Analysis ................................ ................................ .... 158 Alternative Solutions and Re commendation .............................. 163 Global Ratings. ................................ ................................ ........... 166 Reflective Performance Pattern 3 ................................ ........................... 167 Problem Statement ................................ ................................ ...... 168 Contextual Analysis ................................ ................................ .... 169 Alternative Solutions and Recommendation .............................. 172 Global Ratings ................................ ................................ ............ 179 Code Omissions ................................ ................................ ...................... 180 Rationale ................................ ................................ ................................ 181 Chapter V ................................ ................................ ................................ ........................ 187 Summary of the Current Study ................................ ................................ ........... 1 87 Sample Characteristics ................................ ................................ ............ 188 Hypothesis Testing for Demographic Factors ................................ ........ 190 Hypothesis Testing for RCI Scores ................................ ......................... 193 Hypothes is Testing for Content Analysis Procedures ............................ 197 Differential Performance Based on Beginning RJM Level .................... 199 Pre Reflective Performa nce Pattern 0 ................................ ......... 199 Quasi Reflective Performance Pattern 1. ................................ .... 202 Quasi Reflective Performance Pattern 2 ................................ ..... 205 Rationale ................................ ................................ ................................ 209 Differential Performance by Section ................................ ...................... 210 Correlation between Rubric Scores and RCI ................................ .......... 212 Limitations ................................ ................................ ................................ .......... 218 Unbalanced Groups ................................ ................................ ................. 218 Instrumentati on ................................ ................................ ....................... 219 Sample Size ................................ ................................ ............................. 220 Unequivalence of Qualitative Posttest Measure ................................ ..... 220 Fidelity to Case Method Model ................................ .............................. 221 Lack of Empirical Validation of Steps for Better Thinking Rubric ........ 222 Match between Co ntent Analysis and Assignment Description ............. 223 Implications for Social Work Education ................................ ............................. 223 Importance of Assessing Reflective Judg ment ................................ ....... 223 Evidence Based Practice and Case Method Instruction ......................... 226 Tailoring Assignments to the Development Level of Students .............. 229 Social Work Values and Ethical Decision Making ................................ 231

PAGE 8

v Recommendations for Targeting Reflective Judgment through Case Method Instruc tion ................................ ................................ ................................ ........... 232 Recommendations for Further Research ................................ ............................. 237 References ................................ ................................ ................................ ....................... 241 Appendices ................................ ................................ ................................ ...................... 265 Appendix A: Reflective Judgment Interview (RJI) ................................ ............ 266 Appendix B: Longitudinal Studies of Reflective Judgment .............................. 268 Appendix C: RCI Sample ................................ ................................ ................... 270 Appendix D: Course Syllabus ................................ ................................ ............. 273 Appe ndix E1: Initial Intervention Group Invitation ................................ ........... 288 Appendix E2: Intervention Group Reminder ................................ ...................... 289 Appendix E3: Intervention Group Identifier & Link ................................ .......... 290 Appendix E4: Intervention Group Posttest Invitation ................................ ........ 291 Appendix E5: Intervention Group Reminder 1 ................................ ................... 292 Appendix E6: Intervention Group Reminder 2 ................................ ................... 293 Appendix E7: Comparison Group Invitation ................................ ...................... 294 Appendix E8: Comparison Group Reminder ................................ ...................... 295 Appendix E9: Website Problem Notice ................................ .............................. 296 Appendix E10: Person al Reminder ................................ ................................ ..... 297 Appendix E11: Personal Reminder re: Case Analysis ................................ ........ 298 Appendix F: Permission to use RCI ................................ ................................ .... 299 Appendix G: Permission from IRB ................................ ................................ ..... 301 Appendix H: Reflective Judgment Study Survey ................................ ............... 305 Appendix I: Steps for Better Thinking Rubric ................................ .................... 308 Appendix J: Coding Rubric ................................ ................................ ................ 309 Appendix K: Coded Paper ................................ ................................ .................. 312 Appendix L: Performance Pattern Frequencies ................................ .................. 316 About the Author ................................ ................................ ................................ ... End Page

PAGE 9

vi List of Tables Table 1. Summary of the Reflective Judgment Model ................................ ................... 24 Table 2. Demographic Characteristics of Intervention and Control Groups .................. 70 Table 3. Participation of Intervention Group by Section ................................ ................ 76 Table 4. Comparison between Steps for Better Thinking and Case Analysis ................ 86 Table 5. RCI Scores by Age and Experience ................................ ................................ 108 Table 6. Paired Samples T Tests on Rubric Items ................................ ........................ 119 Table 7. Comparison of Initial an d Final Scores by Final Case ................................ ... 121 Table 8. Use o f Rationale and Rubric Means ................................ ............................... 122 Table 9. Initial and Fi nal Rubric Scores by Section ................................ ..................... 126 Table 10. Recomme ndations for Future Research ................................ .......................... 23 9

PAGE 10

vii List of Figures Figure 1.Epistemic Assumptions and Ill Structured Problem Resolution ........................ 21 Figure 2. Research Timeline ................................ ................................ ............................. 63 Fi gure 3. Change in RCI Scores ................................ ................................ ..................... 112 Figure 4.Mean Rubric Change b y Initial Ca se Performance Pattern Level .................... 118 Figure 5. Relationship between Adjusted RCI and Rubric Scores ................................ 128

PAGE 11

viii An Evaluation of the Influence of Case Method Instruction on the Reflective Thinking of MSW Students Marleen Milner A BSTRACT Social work practice requires that graduates be prepared to deal with complex, multifaceted problems which cannot be defined completely, do not have absolute, correct answers and can be approached from multiple perspectives This study evaluated the influence of case critical thinking associated with the ability to reason through ill structured pr oblems. (King Wood, & Mines, 1990). The Reflective Judgment Model, which describes a developmental contin uum based upon epistemic assumptions regarding the source and justification of knowledge claims served as the theoretical framework for the assessment of reflective thinking in this mixed methods study A quasi experimental pre post nonequivalent control group design was utilized to explore whether students who participated in a case method course demonstrated greater increases in reflective judgme nt than those who did not. MSW students enrolled in a case based capstone course at a major metropolitan univ ersity in the southeast served as the intervention group, while foundation year students enrolled in a research methodology course served as the comparison group. Both groups completed the Reasoning about Current Issues Test (RCI), which is an online, stan dardized measure that

PAGE 12

ix has been widely used to assess reflective judgment (Wood Kitchener, & Jensen, 2002 ) at pre and posttest Content analysis procedures were used to facilitate assessment of final case analysis papers for evidence of changes in the reflective thinking skills and problem solving approaches utilized on initial and final case analysis papers. and postte st, while RCI posttest scores of participants in the control group decreased sing a customized rubric based on similarly indicated no m ean changes in problem solving approaches between pre and posttest. However, students who began the course using strategies associated with pre reflective judgment increa sed their scores on the rubric significantly while those who exhibited higher levels o f quasi reflective judgment at pretest decreased at posttest. Strategies for designing a developmental curriculum to target the reflec tive judgment levels of MSW students are proposed.

PAGE 13

1 Chapter I Introduction As professional problem solvers, social work ers must have skills to assess multifaceted problems systemically based on information that is often ambiguous, inconc lusive and variable P ractitioners r outinely make vital decisions regarding such conundrums utions. Therefore, schools of social work are enjoined with the primary task of preparing l Work Education, 2003, p. 33). However, identifying effective teac hing strategies that foster the types of reasoning skills required in social work practice has remained elusive. Many educators in a number of disciplines have endorsed t he case method of instruction, which is a student centered approac h that involves the analysis of open ended, realistic practice situations, as a leading teaching strategy for preparing students to deal with ill structured problems. While well structured problems can be described with a high degree of certainty and solved using deductive lo gic, ill structured problems can be understood from multiple perspectives, cannot be described completely, and do not have an absolute, correct answer (Altshuler & Bosch, 2003; King, Wood & Mines, 1990) This study evaluates the influence of case based instruction on Master of Social is linked to the ability to reason through ill structured problems. Reflect ive thought involves

PAGE 14

2 carefully considering beliefs or knowledge claims in the light of supporting evidence in order to bring closure to situations that are controversial or problematic (Dewey, 193 3; King & Kitchener, 1994) Problem Statement A primary purpose of social work education is to empower students to alleviate perplexing problems such as poverty, oppression, a nd social injustice. These complex problems are not clearly understood; consequ ently, proposed solutions are base d upon varying perspectives. Social workers routinely face such complex problems and make decisions that require the use of reflective reasoning. For example, social workers are required to make decisions regarding placing children who are at risk, intervening in the lives of the chronically mentally ill, and addressing the impact of social policy on individual lives and communities The weight and significance of such decisions is clear. Because effective problem solving i s so integral to routine practice, s ocial work employers often cite the critical thinking abilities of future employees as a top concern (Jones, 2003; Wingo, Perry & Orton, 20 03; Dalton & Wright, 1999 as cited in Wolfer Freeman, & Rhodes, 2001) A related concern is that it is estimated that only 10% of course based learning is t ransferred to on the job performance (Holten & Baldwin, 2 000) The current emphasis on evidence based practice is expected to facilitate better decision making, maximize service to vulnerable populations, and minimize judgment errors H owever the consistent use of evidence to support practice decisions requir es a level of cognitive complexity that research strongly suggests is uncharacteristic of the average college senior and beginning graduate student (King & Kitchener, 2002;

PAGE 15

3 Kohlberg, 1969; Kuhn, Ho, & Adams, 1979). Researchers have noted that most college seniors and adults in general fail to provide evidence to support their decisions (King et al., 1990). Many college students do not see the relationship among interpretations, judgments, and evidence, believing that interpretations are simply equally valid opinions. rong from weak evidence, and often make j udgments based on personal opinion rather than logic when faced with competing claims to truth ( Brabeck & Welfel, 1985) Students using these problem solving approaches will find themselves ill prepared to make sound judgments in a field which is characterized by problems which rarely are understood completely or have easy answers, yet require careful decision making in order to avoid further harm to populations already at risk (Gambrill, 1990; Gibbs, 1991) Use of the Case Method to Promote Reflective Thinking The case method has b een promoted as a useful strategy to prepare graduates for the real world where solutions to complex problems are not found in textbooks and there is often not agreement regarding the correct solutions to difficult questions (Lynn, 1999) Adherents of the case m ethod argue that it fosters critical and reflective thinking, develop deal with the ambiguity of real world pro blems and assists them in clarifying their own beliefs and how those beliefs impact their decision making (Lundeberg, Levin, & Harrington, 1999; Macaulay & Cree, 1999, p. 189) Noting the widespread use of the case method in other disciplines, Cossom (1991) endorsed case

PAGE 16

4 social work ambiguity . and difficult ols of Law and Business in the 1870 s, and was later adopted by other disciplines including schools of medicine, education nursing, psychology, and social work (Gullahorn, 1959; Webb, Gill, & Poe, 2005) While there are numerous variations of the case method, this study will use the description offered by Wo lfer (2006, p. 3): The case method . involves in depth class discussions based on detailed, open ended accounts of actual practice situations. These accounts, referred to as decision cases, require students to formulate problems and decide on potential courses of action. Although the case method has been used extensively in business schools for over a century, the majority of empirical studies of case method outcomes have only been published over the last 20 years in the area of teacher education (Alle n, 1995; Harrington, 1999; Lundeberg, 1999; Lunde berg, Levin, & Harrington, 1999 ; Lynn 1999 ) Several authors have promoted case based instruction in social work education ( e.g., Cossom, 1991; Seelig, 1991), but it is only in the last decade that a handfu l of authors have published material regarding the process and outcomes of utilizing the case study method in social work ( Gray, Wolfer, & Maas, 2006; Jones, 2003; Jones, 2005; Wolfer et al., 2001; Wolfer & Gray, 2007) These works, which will be discussed in greater detail in the review of the literature, have advanced meaningful rationales for the use of case

PAGE 17

5 based instruction in social work education and provided preliminary evidence of positive learning outcomes This study contribute s to the literature by providing empirical measures of the extent to which participatio n in a case based course affects the reflective thinking of MSW students. In addition, this will be the first study in social work to utilize the Reflective Judgment Model, a cognitive development al framework that has been widely arning. After searching several databases including Social Work Abstract Plus, Social Sciences Full Text, and Soc I ndex, no publications were located within s ocial work literature that utilized the Reflective Judgment Model The following section provides a brief overview of the model. Reflective Judgment Model the most rigorously and extensively researched model of epistemology (Hofer & Pintrich, 1997) and the best known model of ad ult cognitive development (Pascarella & Terenzi, 1991) the way that they reason about and justify their own judgments when conside ring ill structured problems Although this model has not previously been utilized in social work education, its emphasis on how individuals approach decision making about problems that cannot be defined or resolved with absolute certainty, makes it a particul arly compelling model for assessing and encouraging the most critical reasoning competencies demanded in daily social work practice (Teare & S heafor, 1995)

PAGE 18

6 The Reflective Judgment Model (RJM) operationalizes seven developmental stages of reflective thinking based upon epistemic assumptions. The first three stages, which are based to varying degrees on the assumption that knowledge is a bsolute and comes from authoritative sources, are Learners do n ot perceiv e complex issues as problematic because knowledge is certain and issues are right o r wrong, black or white. When uncertainty is evident these individu als believe that it is temporary and will be resolved when those in authority discover unavaila ble information or are able to resolve the problem conclusively. Stages 4 and 5 which are more typical of college seniors and graduate students, asi students perceive the uncertainty of ill structured prob lems. However, they are uncertain how to deal with the ambi guity and believe that c ompeting pe rspectives espouse them. Consequently students with Stage 4 assumptions tend to use evidence selectively to support their own opinion, rather than considering neutral or disconfirming evidence. At S tage 5, students understand that knowledge claims are subject to interpretation and contextual realities. Students demonstrate the ability to analyze complex problems compreh ensively and to use evidence objectively and consistently. However, they are unable to establish criteria for selecting between viable alternatives and therefore have difficulty defending their conclusions. Stages 6 and 7 represent beginning and advanced l evels of At S tage 6 of reflective thinking, individuals understand that although knowledge is not certain, conclu sions can be reached based on interpretations of the available evidence

PAGE 19

7 T he implications and consequences are subjected to overarching principles that can be applied across contexts At Stage 7, individuals assume responsibility for constructing and evaluating knowledge claims on an ongoing basis and use evidence to reach decisions based on r compelling understanding of an The R eflective Judgment Model Stages two through seven are summarized in Table 1 in Chapter II. Stage 1 is not included because it represents reasoning approaches that are common to y oung children. The Reflective Judgment Model (RJM) was selected as a theoretical framework for this research f or the following reasons: (1) RJM delineates the levels of reasoning utilized in thinking through i ll structured problems, which are the types of problems most frequently enc ountered by social workers ; (2) i t is well suited for the assessment of the effectiveness of fostering reflective thinking by analyzing decision cases, which by defini tion are ill structured; (3) t he stages of the RJM have been rigorously tested in longitudinal and cross sectional studies that validated the stages as organize d, hierarchical, and sequential; and (4) t he Reasoning about Current Issues Test, which is based on the RJM, provides a standardized measure of reflective r easoning. A description of the model, research supporting it, and the corresponding instrument will be discussed in the review of the literature. In summary, a lthough the limited number of studies regarding case method instruction show promise regarding i ts potential to enhance the reasoning aptitudes and skills of future social work practitioners, empirical studies using objective measures to the majority of

PAGE 20

8 publications across discipl ines are descriptive, emphasizing methods rather than outcomes Because of the urgency of producing graduates that are able to grapple with complex, multi faceted problems, the need to develop evidence based strategies that will encourage the development o is clear. Therefore, the purpose of this study is to address a significant gap in the urther ing knowledge regarding how the academy can best pre pare graduates for the significant challenges ahead of them Research Questions This study considers the following questions : 1 Do MSW students participating in a case method course demonstrate improvement in reflective thinking on a standardized measure of reflective judgment? 2 Do MSW students participating in a case method course demonstrate greater gains in their reflective thinking skills than graduate students who are not exposed to a case method course? Are the gains greater than those that might be expe cted based on educational experience and maturation? 3 Do final written case analyses, by MSW students participating in a case method course, reflect changes in the way they reason about ill structured problems when compared with their initial case analys e s ? 4 What if any, demographic factors are associated with Reflective Thinking ?

PAGE 21

9 Chapter II Review of Selected Literature This chapter will review selected literature regarding the significance of critical thinking as an educational outcome, the relationship between critical and reflective thinking the use of case ritical and reflective thinking, and the use of the Reflective Judgment Model to assess how students reason through ill structured problems Critical Think ing as an Educational O utcome Educational literature provides ample evidence that critical thinking has become the single most prized student outcome at all levels of education (Blai, 1992; Boostrom, 2005; Facione, 1998; Halx & Reybold, 2005; Norris, 1985; Paul & Elder, 2006; Phillips & Bond, 2004) Literature regarding the i mportance of critical thinking has proliferated sin ce the early 1990s. A search for full text scholarly articles available through the EBSCO Academic Search Complete Database with critical thinking in the title or abstract published between 1990 and 2009 y ields 1882 articles. A Go ogle search uncovers dozens of u niversity websites dedicated to the topic and more than 26,0 00,000 matches. National concern regarding the diminishing educational outcomes of American schools has resulted in a growing critical thi nking movement and the initiation of n ational and statewide reforms (Facione, 1998; Facione, Facione, & Giancarlo, 2000; Paul, Elder, & Bartell, 1997) For example, in 1989 Goals 2000 charged colleges and universities to

PAGE 22

10 devise strategies to improve the abilities of students to think critically, solve problems and communicate (Halonen, 1995) Yet, in spite of concerted effor ts to address these concerns on a national level, effective strategies for the achievement and assessment of these fundamental educational outcomes have remained elusive (Ennis, 1993; Halonen, 1995; Halx & Reybold, 2005). Nearly 10 years after Goals 2000 w as conceived t he Boyer Commission (1998) which was tasked with making recommendations for the reconstruction of undergraduate ed ucation reported that many graduate s were un able to integrate learning between courses, think logically, write clearly, or sp eak coherently. Defining Critical Thinking The most frequently cited impediment to increasing critical thinking among students is the lack of agreement among educators about an operational definition of critical thinking (Bissell & Lemons, 2006; Boostrom, 2005; Brookfield, 1987; Ennis, 1993 ; Ennis, 1991 ; Facione, 1998; Halonen, 1995; Paul et al., 1997; Shermis, 1992; South Carolina Higher Education Assessment Network, 1996) According to Halonen (1995), the comple xity and famil iarity of critical thinking qu alify concept understood (Minnich, 1990, p. 51, as cited by Halonen, 1995). A number of researchers contend th at although most educators give lip service to the importance of teaching critical thinking, few can clearly define it, and fewer still can demonstrate that they are teaching it (Bissell & Lemons, 2006; Boostrom, 2005; Browne & Freeman, 2000; Halx & Reybold, 2005; Paul et. al., 1997)

PAGE 23

11 The complexity of the construct of critical thinking is evidenced by the fact that the NP EC Sourcebook on Assessment (U. S. Department of Education, National Center for Education Statistics, 2000) identified sixty nine distinct skill sets and fourteen dispositions that are included in the twelve most widely used measures for the assessment of critical thinking. Definitions range from the minimalist to the elaborate, but many authors agree that commonalities across definitions can be clearly identified (Allegretti & Frederick, 1995; Bissell & Lemons, 2006; Boostrom, 2005; Paul et. al., 1997; Plath, English, Connors, & Beveridge, 1999; Shermis, 1992) Frequently cited themes inc lude the ability to frame problems, identify and evaluate assumptions, analyze and synthesize information, make correct inferences from data, assess the credibility of arguments, consider alternate perspect ives, deal with ambiguity, support claims with evi dence, and (Mumm & Kersting, 1997; Paul et. al., 1997; Plath et a l., 1999; Ringel, 2003; Shermis, 1992; Terenzi, Springer, Pascarella, & Noram, 1995) Based on the difficulty of reducing the construct to a few clearly defined skills, numerous authors have asserted that it is contingent upon the specific disciplines to come to a consensus as to a definition that best fits the requirements for reasoning skills and dispositions of that field (South Carolina Higher Education Assessment Network, 1 996) Others have argued that thinking skills, per se, are domain specific, and can only be defined and developed within the context in which they are used (Glaser, 1984; McPeck, 1981; Smith, 2002)

PAGE 24

12 Critical Thinking and Social Work Education In spite of the fact that critical thinking skills social work practice (Gibbons & Gray, 2004), relatively little has been written about it in professional literature when compared to other helping professions such as teaching and nursing Early in the 1990s, S e elig (1991) noted the failure of social work education to contribute to the expanding criti cal thinking movement and challenged the profession to recognize the importance of critical thinking and to emphasize it as a component skill of social work practice. Gambrill and Gibbs began addressing the gap in the literature by authoring a number of books and journal articles that argued the importance of encouraging critical thinking skills in social work students and practitioners by training them to reason scientifically (Gambrill, 1990; Gambrill, 1997; Gambrill, 1999; Gambrill, 2006; Gibbs, 1991; Gibbs et al., 1995; Gibbs & Gambrill, 1999) Gambrill defined critical thinki ng as well reasoned (1997, p. 125) She further described the process as and taking respons ibility for our claims and arguments, critically evaluating our views no (p. 126) Social work models for teaching critical thinking run the gamut from a post positivist perspective (Gambrill, 2006; Gibbs, 2007; Kersting & Mumm, 2001) to a constructivist paradigm (Gibbons & Gray, 2004; Plath et al., 1999). Those who focus on a post positivist approach tend to bu ild critical thinking skills around the use of scientific reasoning rational decision making and the concept of evidence based practice (Gibbs,

PAGE 25

13 2007) while those with a constructivist approach center on experience based l earning and reflective practice (Laird, 1993) While they share a common focus on the role of criticism in the evaluation of knowledge claims, the approaches adopted to facilitate the development of reasoning sk ills from these perspectives are based on disparate epistemologies. From the post positivist perspect ive, although knowledge is subject to change, (Gambrill, 19 97, p. 83) can be used to acquire knowledge and minimize judgment errors. Consequently efforts to foster critical thinking skills focus on error elimination strategies by identifying common fallacies in logic, increasing objectivity, teaching rational pr oblem solving methods, and honing the skills necessary to carefully scrutinize knowledge claims (Gambrill, 1997) Constructivists view knowledge a s a s ocial construction that is limited, contextual, and relative, a nd therefore agree that knowledge claims must be examined critically. However, the underlying assumption that people must construct or make sense of reality for themselves results in a focus on experiential learning rather than error elimination strategies According to Gibbons and Gray (2004) only be learned and refined through practice within a particular discipline, through doing and reflecting on what we have done and why we did it that Strategies for fostering critical thinking from this perspective emphasize the structuring of tasks or e xperiences that will trigger perplexity encouraging the learner to engage in reflectiv e thought considering knowledge claims carefully ( Dewey, 1933)

PAGE 26

14 Gambrill and Gibbs have been major proponents of the post positivist approach, which challenges educators to equip social work students with the skill s to use sound logic and reasoning strategies to examine knowledge claims, test assumptions, identify fallacies, and make optimal S ocial work educators have proposed numerous strategies to foster critical thinking skills based on this perspective. These include providing specific content on the use of inductive and deductive logic and argumentation, research e valuating the quality of online resources, analyzing social work theories, and using logic models, logic ga mes and exercise s (Alter & Egan, 1997; Gambrill, 19 97; Gibbs, 1991; Gibbs & Gambrill, 1999; Lister, 2004; Lynch, Vernon, & Smith, 2001; Mumm & Kersting, 1997; Vandsburger, 2004) A review of th e literature indicates that the majority of the methods proposed to teach critical thinking in social work use a post positivist perspective and focus on instruction regarding the use of logic and identification of reasoning errors. For example, the workbook, Critical thinking for Social Workers: Exercises for the Helping Profession (Gibbs & Gambrill, 1999) presents students with numerous exercises focused on identifying fallacies in thinking and developing skills in logic and argumentation. The PR IDE1 (Gibbs et al., 1995 ) and the Profe ssional Thinking Form (Gibbs & Gambrill, 1999) require that students correctly identify fallacies in reasoning. These tasks require the type of critical thinking skills associated with solving well structured proble m s i.e., students must d iscover the correct answer by applying course content on the use of logic (King & Kitchener, 1994)

PAGE 27

15 While these basic skills are required to evaluate knowledge claims effectively, they do no t adequately prepare students to think reflectively when addressing issues that cannot be resolved with certainty based on existing knowledge (Gi bbons & Gray, 2004; Gould, 1996; King & Kitchener, 1994; Sung Chan & Yuen Tsang, 2008) Critics of the post positivist paradigm argue that social work practitioners are often faced with complex problems that cannot be resolv ed by applying professional pri nciples based on existing knowledge (Gibbons & Gray, 2004; Sung Chan & Yuen Tsang, 2008) According to Schn (1987) professionals regularly e ncounter problems that cannot be solved with governed existing professional knowledge b ut require practitioners to generate new theories that are subsequently tested and revised (p. 34) Luitgaarden (2009) has argued that the exten sive complexities of social work practice make rational decision making models that rely on deductive and statistical reasoning unsuitable models for practice. B ased on a constructivist paradigm, social work educators at The University of Newcastle in Aus tralia have utilized an integrative curriculum utilizing a problem based and experie n tial learning model that infuses critical thinking assignments throughout the BSW curriculum An intensive critical thinking unit at the end of the curriculum teaches crit ical thinking as a specific social work skill set. Faculty assess critical thinking awareness, ability to make well reasoned arguments, and ability to communicate their views effectively in consideration of al ternate perspectives (Gibbons & Gray, 2004) They expressed concern that i n spite of a strong emphasis on meaning

PAGE 28

16 survey of students indicated that they con tinued to associate critical thinking with a n objective, or scientific view of the reasoning process rather than a strategy for dealing with the uncertainties involved in social work practice. Altshuler and Bosch (2003) and Cole man, Collins and Baylis (2007) proposed Problem Based Learning to simulate situations that social workers will inevitably encounter in the field In this model, students are presented with situations that they do not have sufficient knowledge to resolve, requiring them to search for solutions. Instructors serve as consultants rather than authorities and learning takes place in a collaborative, small group environment. Sung Chan & Yuen Tsan g (2008) proposed an action research approach to bridge the gap between theory and practice in social work education. They criticized the prevailing educational models in social work education in light of the complexity and uncertainty of social work pract ice, especially in the context of non Western cultures such as mainland China. A reciprocal reflection and experimentation cycle based on Schn espoused practice frames and the development and testing of new solutions to narrow the gap. Other str ategies that have been suggested to target critical thinking based on a constructivist model include self reflection, student journals, the development of portfolios and the use of deci sion cases (Coleman, Rogers, & King, 2002; Haulotte & K retzschmar, 2001; Jones, 2003; Jones, 2005; Nesoff, 2004; Ringel, 2003; Scales et al., 2002; Wolfer et al., 2001)

PAGE 29

17 In summary, while ability to evaluate knowledge claims critically, and to assume personal responsibility as consumers of knowledge, the methods used differ and may target different aspects of critical thinking Post positivist approaches focus on teaching students principles that support effective argumentation and the avoidance of com mon fallacies in reasoning while constructivist approaches tend to focus on experiential or transformative learning, self awareness, and integration of theory wit h practice. Based on the distinctions proposed by and Brabeck (1980) between critical and reflective thinking, the post positivist pedagological approaches may target general critical thinking skills, while constructivist approaches target the development of reflective thinking. Relati onship between Critical Thinking and Reflective Thinking In his seminal work, How We Think, Dewey (1933) defined reflective thinking as f knowledge in the light of the grounds that support it and the further conclusions to which This necessarily involves many of the commonly agreed upon elements of critical thinking such as framing problems, identifying assumptions, analy zing and synthesizing information, and making inferences from data However, although it shares some commonalities with critical thinking and at times is used interchangeably, there are important distinctions between the two constructs Based on a comparis on of student performance on critical thinking measures and the RJI, Bra beck (1980) concluded that critical thinking is necessary but insufficient for reflective thinking. The distinctions betw een the two are addressed below.

PAGE 30

18 Problem Structure A primary difference between critical thinking and reflective thinking is that critical thinking skills may focus on resolving well structured problems, which can be resolved with certainty, regardless of the level of difficulty, using deductive or inductive logic. These problems require that the learner find an applicable problem solving procedure to discover, compute or recall the solutions. In contrast, ill structured problems, which have no verifiable c orrect answers, canno t be resolved with logic alone. Problems such as child abuse, mental illness, poverty, juvenile delinquency and racism cannot be resolved conclusively with logic or specific knowledge They cannot be resolved with certainty by referrin g to the claims of authorities as authorities frequently disagree as to the best solutions for these types of issues. Instead, they require that inquirers identify the facts and theories that may apply to the situation evaluate their credibility and rele vance within the current context and generate potential solutions. These solutions must then be evaluated in the light of existing information and contextual realities, and decisions must be made based on the best available evidence /information Gill and Hicks (2006) note that a primary distinction between ill structured and well structured problems is the relationship between the task complexity and the amount of discretion called for in order to fulfill the req uirements of task performance Task performance that call s for individuals to rely primarily on domain specific knowledge such as formulas or proven principles allow little discretion in the number of viable paths that may be chosen In contrast, ill str uctured problems are characterized by a significant amount of discretion in the number of acceptable paths one may chose to resolve the

PAGE 31

19 problem or complete the task. For example, a clinical social worker may rely upon domain specific knowledge to come to a IV diagnosis of clinical depression but will have considerably more discretion in determining an appropriate treatment plan based on client characteristics, contextual factors, and available resources. However, discret ion alone may not elicit reflective thought as many tasks that allow for discretion become ro utine as the practitioner comes to rely on increasing knowledge, experience, and expertise (Crook, 200 1; Gill, 2006) Uncertainty Perceived uncertainty regarding problem formulation and resolution triggers the processes involved in reflective thought. Dewey (1933) argued that t and uncertainty are the genesis of reflective thinking Acco rding to Dewey, to a child (or a grown up) to think irrespective of the existence in his own experience of some difficulty that troubles him and disturbs his equilibrium, are as futile as advice to li ft himself by his boot (p. 15) The role of conflict and controversy as essential to cognitive growth, learning and conceptual change has been wi dely espoused in the literature. bration postulates that the experiences that promote cognitive develop ment are thos e that not only incite curiosity but also create a state of conflict that the individual seeks to resolve (Piaget, 1964) Similarly Kohlberg (1969) whose stages of moral d ieved that movement from one stage to the next occurr ed as ones views were challenged through the discussion of moral dilemmas with others Sch n (1983) a p roponent of reflective practice, described at involved developing

PAGE 32

20 an aw areness of a problem that could not be resolved through previously employed means, reframing the problem from a new perspective, and gen e rating new hypotheses, which were subsequently tested in practice Echoing themes in Dewey a nd Sch s work, Mezirow (1998 ) also asserted that reflection followed led to the cri tique of previously held beliefs in the light of alternative explanations of experience Relationship to Episte mic Assumptions An additional difference between critical and reflec tive thinking is the central role of epistemic assumptions in the internal l ogic used to resolve ill structured problems. King and Kitchener (1994 ) argue that traditional attempts to define critical thinking based on skill sets involving basic logic and problem solving fail to account for the differing worldviews that impact how i ndividuals approach problem solving. According to cognitive theorists (King & Kitchener, 1994; Kuhn & Dean, 2004; Perry, 1970) the process of making judgments about ill stru ctured problems involves the construction of beliefs, which requires individuals to utilize underlying cognitive structures related to their understanding of the limits, certainty, and criteria for knowing. These underlying beliefs based think The Reflective Judgment Model is based upon empirical observations of a distinct developmental progression in the epistemic assumptions and related r easoning strategies of learners as they become increasingly effe ctive in dealing with uncertainty. Figure 1 depicts the relationship between epistemic assumptions, uncertainty, and problem solving strategies used when individuals between stages 3 and 6 encounter an ill structured problem.

PAGE 33

21 Figure 1 Epistemic Assumpti ons and Ill Structured Problem Resolution

PAGE 34

22 Reflective Judgment Model Theoretical Foundation cognitive developmental theories of Piaget (1964) and Kohlberg (1969) as well as the orig inal work of Dewey (1933) on reflective thought (1970) research on the link between the epistemolog ies and cognitive development of college students. The d King and Kitchener (1994) discovered consistent patterns that revealed an internal logic to the way that individuals approached complex problems using an interview protoco l with trained interviewers and raters described nine epistemic positions, King and Kit stage developmental progression of epistemic assumptions as individuals become increasingly able to deal with uncer tainty, evaluate knowledge claims and justify their beliefs and conclusions. The stages are sequential and hierarchical, meaning that previous stages provide the foundation for subsequent ones. Stages 2 and 3 are considered the pre reflective stages, whil e 4 and 5 are considered quasi reflective stages, and 6 and 7 describe true reflective judgment. (1984; 2002) King and Kitchener describe the mode l as a complex stage model, meaning that the stages are not necessarily static but represent the range of cognitive complexity of which a person is capable. According to Fischer (1984) the environment in which the skill is required influences the level of skill

PAGE 35

23 a person demonstrates. Environments that provide support such as prompts, feedback, elicit the functional level. T the stage at which the individual functions without contextual support. Based on a complex stage model, growth in reflective thinking occurs in waves, with the person functioning in a range of thinking, often spanning two adjacent stages, and rarely, three. Growth spurts, characterized by inconsistent use of the stage based on the level of support provided, are common. Although the model has been extensively tested in the United States, and found to be consistent across cultures and ethnicities, King and Kitchener do not make any claims as to its universality. In the only reported testing of its use abroad a mong German university students, the findings were consistent with patterns observed in the U.S.

PAGE 36

24 Table 1. Summary of the Reflective Judgment Model Stages (King & Kitchener, 1994, pp.14 16) Stage View of Knowledge Source of Knowledge Justification of Knowledge 2 Absolutely certain but may not be immediately available Direct observation or claims of authorities Unexamined or justified based on direct observations or information from authorities; issues are assumed to have a right answer 3 Absolutely certain or temporarily uncertain Authorities in some areas; through personal beliefs when knowledge is uncertain Information from authorities or personal opinion 4 Uncertain and ambiguous due to situ ational variables idiosyncratic to the individual logic Personal or situational variables, unevaluated beliefs, anecdotal evidence; confirmatory bias 5 Contextual and subjective, open to interpretation Interpretations of ev idence, events, or issues Rules of inquiry within a particular context or context specific interpretations of evidence 6 Constructed into individual conclusions based on information from a variety of sources Personal assessment of evidence or evaluation s of opinions of experts Rules of inquiry, comparing evidence and evaluating options from various perspectives, evaluating views of experts 7 Tentatively certain and based on reasonable inquiry; solutions are constructed and their adequacy can be evaluate d and revised Critical inquiry or synthesis; re evaluated when new evidence, perspectives or tools of inquiry become available Evaluating, re evaluating, and integrating evidence and arguments from multiple perspectives, more or less reasonable conjectures about the reality of the world based on available evidence

PAGE 37

25 Research on the Reflective Judgment Model Extensive research efforts, including both cross sectional and longitudinal studies, have addressed the validity of the Reflective judgment Model a s a distinc tive construct the sequential nature of the stages and group differences and similarities in Reflective Judgment scores based on gender, ethnicity, educational level and age (King & Kitchener, 1994) The Reflective Judgment Interview (RJI) was the primary vehicle used to research the validity of the Reflective Judgment Model and to inform theory development. Over 1700 people of all ages and educational levels from high school to graduate students, as well as non student adults completed the R eflective J udgment I nterview in va rious cross sectional studies over a 20 year period. See Appendix A for the interview protocol. Distinctive c onstruct Reflective Judgment has been differentiated from similar construc ts, such as critical thinking, intelligence, or scholastic aptitude by its unique relationship to the resoluti on of ill structured problems. Critical thinking is necessary but insufficient for the development of reflective judgment (Brabeck, 1980) Bra back compared critical thinking skills as measured by the Watson Glaser Critical Thinking Appraisal (WGCTA) with scores on the RJI She reported that 1) reflective judgment levels increased with education level when critical thinking scores were held constant; 2) high critical thinking subjects outperformed low critical thinking subjects on the RJI; 3) however, while low critical thinking subjects were homogeneously low in RJI levels, high critical thinking subjects had a greater degree of variability on RJI sc ores King, Wood, and Mines (1990) also examined the relationship between scores on two critical

PAGE 38

26 thinking measures, the WGCTA and the Cornell Critical Thinki ng Test (CCTT) and RJI scores. They found that educational level differences could be accounted for by academic aptitude for both critical thinking measures, but not for the RJI. Reflective Judgment has also been differentiated from intelli gence or scholastic aptitude. Correlations between RJI scores and tests frequently used to assess intelligence bas ed on verbal reasoning (Concept Mastery Test, WISC R, or WAIS R) have been low to moderate in various studies, ranging from .37 to 55 (King & Kitchener, 1994) Correlations between scores on the RJI and measures of scholastic aptitude such as the SAT or A CT have been lower, ranging between .17 for the composite SAT to .26 for th e composite ACT (King & Kitchener, 1994) These low to moderate correlations indicate that reflective judgment is related to but distinctive from intelligence, and only minimally related to academic aptitude. Developmental s equence. The findings of n umerous longitudinal studies provide evidence that the RJM describes a clear developmental sequence that is organized and hierarchical. Several longitudinal studies were completed by Ki ng & Kitchener to validate the developmental sequence including a 10 year longitudinal study of a cohort of 80 individuals and another of 120 individuals the majority of whom were involved in formal education (2004) Seven other longitudinal studies revi ewed involved an additional 180 individuals who were evaluated over one to four years The most significant finding is a persistent pattern of growth over longer periods, or stability between testings in shorter periods indicating the gradual emergence of reflective

PAGE 39

27 thinking between adolescence and e arly adulthood (King & Kitchener, 1994) Please refer to Appendix B for an overview of the longitudinal studies. A ge/educational level King and Kitchener (1994) reviewed 25 studies involving over 1500 responde nts from geographic areas across the United States to determine whether the Reflective Judgment Model was sensitive to educational differences. The from high scho ol (M=3.2) to the first year of college (M=3.6) and showed continued growth in the senior year of college (M=4.0) Gradua te students across studies consistently earned the highest RJI scores of any group tested, indicating that their epistemic assumptions were the most consistent with refle ctive thinking ( King & Kitchener, 2004) The highest scores have been reported for advanced doctoral students ( M= 5.86). Across studies they scored nearly three quarters of a stage higher ( M= 5.3) than beginning graduate students ( M= 4.6) who scored a full stage higher than beginning undergraduates did King and Kitchener (1994) also examined the relationship between age and RJI scores for all individuals who had been tested one or more times in the ten year longitudinal study. The modal scores of each age grouping increased predictably, indicating a strong linear relationship b etween age and the RJM stages. The modal scores of the majority of participants ages 36 or older were at S tage 6 or 7, which are the highest stages of reflective thinking In another study involving 156 students, no Stage 6 rea soning appeared before age 22 ( Kitchener, Lynch, Fischer, & Wood, 1993) These findings ap pear to be confounded with education, however, since most of the participants

PAGE 40

28 in both the longitudinal and cross sectional studies, were involved in educational pur suits. Notably, in a separate analysis including data from six studies and 191 participants, the overall mean for adults without college degrees was 3.6, while those with college degree had an overall mean of 4.29 Gender Although, research has suggested that men a nd women reason differently (Baxt er Magolda, 1990) evaluation of RJI scores by gender are inconclusive regarding differential performance based on gender. In reviewing 14 cross sectional studies, King and Kitchener (1994) found that 7 of the 14 had no significant findings and the others had mixed outcomes In six studies, men outperformed women, in the last there was a class by gender interaction favoring women In the 10 year longitudinal study (King & Kitchener, 1994) no significant differences were found based on gender in 1977 and 1979, but the results approached significance in 1983 and 1987, with men scoring slightly higher than women The results were subsequently analyzed for differences based on educational attainment They reported that w hile 47% of the men had attained post baccalaureate degrees by 1987 only 15% of the women had Given the fact that educational level has been shown to be related to RJI scores, they speculate d that the differences in gender noted may be a function of educa tional level. King and Kitchener (2002) found differences based on the collective results of studies using the RCI, with women scoring slightly higher Thomson (1995) noted a slight gender effect favoring women on the Reflective Thinking Appraisal, a paper and pencil precursor to the RCI The authors conclude that the results remain inconclusive based on the level of inconsistency and the wide variety of samp ling strategies used.

PAGE 41

29 Race/E thnicity Because the RJM model has been almost exclusively tested in the United States, King and Kitchener (1994; 2002) do not make any claims regarding the universality of the model. One study that tested the model among 48 Ge rman university students, found results consistent with those of American university students, suggesting the possibility that the sequence may not be simply a function of academic socialization in the United States An e valuation across studies of the imp act of race and ethnicity upon reflective judgment has indicated that both RJI and RCI scores remain consistent across ethnicities and cultures in the U.S. (King & Kitchener, 2002) A cross sectional study comparing Euro Americans to African American college student s found no significant diffe rences (King & Kitchener, 2002) and a study examining RJ scores among Latinos found a consistent developmental pattern on reflective judg ment sco res (Samson, 2000) King and Kitchener (2002) found consistent scores on the RCI across ethnicities after controlling for ACT composite scores T ime between t esting Although findings support the correlati on between educational experience and the development of RJ, the ability of reflective judgment measures to detect differences resulting from educational interventions remains dubious. King and Kitchener (1994) noted that the amount of change in RJI scores appeared to be strongly related to the amount of time between testing. The largest increases were found in the ten year longitudinal study for high school students and the smallest were found in According t o Wood and Kadrash ( Wood & Kadrash, 2002) research designs investigating shorter education intervals require larger sample sizes in order to detect

PAGE 42

30 differences. They argue that while educational interventions may be eff ective in developing epistemology, the effect size is likely to be very small, given the fact that changes over a two year period from the freshman to junior year are modest In spite of these cautions, a few studies have found gains in reflective thinkin g following educational interventions. Thomson (1995) found significant differences between experimental and control groups using the Reflective Thinking Appraisal (RTA) as a pre and post test measure of reflective thinking in a series of natural science c ore course. The experimental courses used specific pedagogical strategies recommended by Kitchener (1994) to increase the reflective thinking of students Although statistically significant, the gains were modest (M=4.55 to M=4.87), reflecting an increase of less than a quarter of a stage Nevertheless, the post test scores positioned the students at the higher range of Stage 4 thinking which is meaningful from a developmental perspective. Kronholm (1996) developed an instructional model, called the Reflective Judgment Developmental Instruction Model to facilitate cognitive growth in undergraduate students Students exposed to the intervention gained .296 of a Reflective Judgment stage over the course of a semester This change, though small, was significant when compared to the control group However, Wood and Kadrash (2002) noted that when t he y compared pre test scores between the control and experimental groups, the experim ental group had lower baseline scores. On retesting, the scores were comparable He speculates that the nonequivalence of the two groups at pretest makes it unclear whether the change may have been a result of a growth spurt in the experimental group, or a direct result of the intervention.

PAGE 43

31 D iscipline King, Wood, and Mines (1990) examined disciplinary differences in RJI scores among graduate students. They found that students in the social sciences scored significantly higher than those in other disciplin es, including the mathematical sciences and medical students. They speculated that this difference may have be en a result of the emphasis on ill structured problems in the social sciences and encourage d more research into strategies for structuring graduat e study to better prepare students to make judgments about complex problems. These differences were not observed for undergraduate students. After searching several databases including Social Work Abstract Plus Social Sciences Full Text, SocIndex and Proq uest Dissertat ions, no publications were located in social work professional literature that referred to or utilized the Reflective Judgment Model. The RJM has been used to assess reflective thinking in numerous disciplines including graduate psychology st udents (Owen, 2005) music education students (Bailey, 2000) educational leadership students (MacDonald, 2003) dental stude nts (Boyd, 2005) and nursing students (Pittman, 2006) Assessing Reflective Judgment According to the South Carolina Higher Education Assessment Network (1996) no single assessment instrument measures the construct in its entirety They conclude that it is essential that groups determine their own definition for critical thinking and then lo ok for instruments that best match that definition and the instructional methods used In addition, they recommend that at least three different types of critical thinking indicators should be used to assess outcomes before making decisions about learning and teaching.

PAGE 44

32 A critical aspect of reflective thinking assessment is the evaluation of proposed measures to determine whether they feature well structured problems, ill structured problems, or both (King & Kitchener, 1994) After reviewing the most widely used instruments, King and Kitchener (1994) concluded that the majority of frequently used and validated measures are unsuited for measuring reflective thinking because they rely on well structured problems, or al ternately, treat ill structured problems as though they were well structured by indicating certain solutions are absolutely correct. Reflective Judgment Interview The Reflective Judgment Interview was originally used primarily as a vehicle for theory deve lopment; however, as educators learned of the Reflective Judgment model, numerous institutions used it as a method of assessing reflective judgment. The RJI required t rained and certified interviewers who asked participants four open ended questions regard ing controversial problems The ill structured problems used we re based on current issues such as the accuracy of new reporting, the safety of chemical additives to food, the building of the Egyptian pyramids the origins of man, the nature of alcoholism, and immigration policy. how they arrived at their point of view, the certainty with which they held that view, the logic by which they explained disagreements between experts on t he topic, and whether those who disagreed were necessarily wrong or right. Trained raters assigned two scores Occasionally, three stages were exhibited. The scores were then w eighted and an overall score calculated. See Appendix A for the interview protocol.

PAGE 45

33 (1984) that individuals are capable of demonstrating a range of thinking based on the level of support in the ir environment was tested using a Prototypic Reflective Judgment Interview. After completing the RJI, participants read prototypical responses by prior respondents to the RJI at each level of the model. Trained interviewers asked participants to respond to a series of questions that served to direct their attention to key elements of the statements, and to explain the statements in their own words. The participants were prompted to consider the various statements prior to the next testing, which occurred wi thin two weeks. The finding that participants scored higher on the PRJI than on the RJI supported the premise that individuals are capable of functioning at higher levels when provided with contextual support. However, an age related ceiling was observed, which suggested that once the optimal level was reached, participants could not exceed their developmental range even when support was provided. The validity and reliability of the Reflective Judgment Interview were consistently high across studies. Accord ing to a report compiled by the National Postsecondary Education Cooperative (2000) internal consistency for the RJI ranged from .75 to .96 across 33 st udies. The inter rater reliability of the interview was reported to be .97 and the more stringent rater agreement ranged from .76 to .90. However, the RJI was impractical for large scale use as it involved certified interviewers and raters. The t raining re quired for certification to administer the RJI is no longer available. The original interview format was replaced by the Reasoning ab out Current Issues Test (RCI), following a series of efforts to create an assessment measure for reflective judgment that w as amenable to l arge scale use. A sample of a previous version of the RCI is included

PAGE 46

34 in Appendix C Although it does not include any of the questions used in the current study, it provides an example of the format and the root questions used to elicit in formation regarding epistemic assumptions. B ecause the measure is under revision, administrators preferred not to have a sample of the current instrument included in the Appendices of this study. Reasoning about Current Issues Test The Reasoning about Current Issues Test (RCI) was developed in response to concerns about the feasibility of using the Reflective Judgment Interview (RJI) for institutional assessment The RJI was expensive to administer as it involved one hour individual interviews of students by trained interviewers and trained raters to score results In contrast, the online instrument takes approximately 30 to 45 minutes to administer and can be taken from any computer with access to the internet, making it suitable for institutiona l use The RCI is an objectively scored instrument modeled after the structure of the Defining Issues Test (DIT) developed by Rest (1979) t o assess moral judgment. It evolved over a period of years as a product of n umerous attempts to develop paper and pencil measures which were amenable to large scale use which were ultimately refined to the current online f ormat ( Wood & Kadrash, 2002) There are two different types of measures u sed to assess cognitive development Those that use production tasks, such as the RJI, require the participant to produce a response spontaneously based on his or her own repertoire of skills The second type of measure uses recognition tasks, which involv e presenting the individual with a series of response options ( Kitchener & Fischer, 1990) Production tasks are usually required in

PAGE 47

35 interviews and essay formats, while recognition tasks are common in multiple ch oice instruments While assessment measures employing production tasks yield richer, more complex information, recognition tasks are not as demanding of the participant, are less expensive and are easier to administer and score The content of the RCI, whi ch uses recognition tasks, is modeled after the Reflective Judgment Interview, which employed production tasks Correlations between the RJI and the RCI have been in the low .40s (King, Lindsay, & Brown) suggesting it me asures an aspect of the construct originally measu red by the RJI ( Wood et al., 2002) This may be due in part to the differences between production tasks and recognitions tasks, which place different types of demands on learners. Rec ognition tasks provide a higher level of support for reflective thinking and levels of Reflective Judgment. Consistent with this assumption, RCI scores have been found to be approximately one stage higher than those found on th e RJI (King & Kitchener, 2004) A meta analysis of all the data available on studies that used the RCI yielded findings similar to those of previous studies using the RJI (Kitchener, Wood, & Jensen, 2002 as cited in King et al., n.d.) The sample of 9.477 students enrolled in undergraduate, graduate and professional progr ams at seven different institutions found significant differences on RCI scores by educational level, even when prior academic achievement and acad emic aptitude were controlled. Significant differences were noted between college freshman, sophomores, and s eniors. Graduate students scored higher than did college students. No significant differences were found based on race or

PAGE 48

36 ethnicity. These findings indicate that the RCI adequately measures changes in Reflective Judgment that have been consistently associa ted with participation in educational programs. Summary To summarize, findings from an extensive number and variety of studies collectively validate the Reflective Judgment Model as a distinct construct that represents an organized, sequential, and hierar chical developmental sequence of cognitive complexity The development of reflective thinking, which is based on epistemic assumptions, appears to be highly correlated with educational experience, but only modestly related to academic aptitude, and verbal ability It correlates positively with age, but educational level is confounded with this variable The Reflective Judgment Model is reliable and consistent across gender and ethnicity The RCI, which has been developed based on the original Reflective Jud gment Interview used to validate the model, has been shown to adequately represent changes in reflective thinking that occur as individuals become better able to reason through ill structured problems Promoting Reflective Thinking Although educators appe ar to agree universally that developing the reasoning skills of students is the single most critical outcome of higher learning, student scores over a period of twenty five years on the Reflective Judgment Model indicate that college seniors and beginnin g graduate students are functioning at the quasi reflective thinking stages At this level, students are unable to use evidence consistently to support their

PAGE 49

37 beliefs and conclusions and are discomfited by the ambiguity of ill structured problems Alternate perspectives are often considered equally valid claims to truth as they merely represent the opinions of those who espouse them In addition, students reasoning at this level are unable to evaluate their own judgments or develop coherent arguments to supp ort their positions. These findings point to the urgent need for educators to devise intentional strategies focus of the social work profession on evidence based practice plac es additional demands Social work educators must acknowledge the developmental stage in which students currently function, while de vising strategies to foster the skills that will be required of them as professionals in a highly demanding and complex field. King and Kitchener (1994) make a number of recommendations for fostering reflective judgment in college students. They stress th e importance of expressing respect for students regardless of the cognitive level at which they are functioning and assessing their current stage as a beginning place for fac ilitating further development. Recommendations include familiarizing students with ill structured problems within their own discipline, creating multiple opportunities for students to consider alternate perspectives, and encouraging students to make well reasoned judgments and explain their own points of view. They suggest grounding e du cationa l experiences emotionally as

PAGE 50

38 well as cognitively. Fina lly, students should be challeng ed within an atmosphere of ample support in order to reach their optimal levels of reflective thinking. Case based instruction seems especially well suited to the implementa tion of these recommendations. Grounded in constructivist theory, the case method is based on the assumption that students are co constructors of meaning and that it is important to approach teaching in an egalitarian and respectful way (Webb et al., 2005) By using decision cases, students are familiarized with ill structured problems within the discipline and multiple opportunities are created for students to examine different points of view, make judgmen ts about what they believe and justify their conclusions. Clas s discussions involving student to student and student to professor interactions provide both challenges and supports that are grounded emotionally and cognitively. Finally, decision cases regar ding controversial issues involve students em otionally and intellectually as they provoke uncertainty regarding ethical and moral decision making. Case discussions are often emotionally charged as students defend their perspectives and are confronted by th e points of view of others The following secti on will examine the use of case based instruction as a method for fostering reflective thinking. Case Method Instruction Numerous versions of the or iginal Harvard case method evolved as its practice was adapte d to suit the purposes of various disciplines. At its core, the case method involves presenting students with a realistic case situation which students are required to analyze critically, identifying relevant issues, recognizing assumptions made, applying the knowledge, skills, and values of the profession, reflecting on ethical decision making,

PAGE 51

39 and proposing alternative solutions to problems (Lundeberg et al., 1999) It differs significantly from lecture based p edagogy in several ways including a) its primary objectives, b) the role of the professor as facilitator rather than information giver, and c) the use of inductive methods rather than the presentation of theoretical frameworks (Webb et al., 2005). A fundam ental goal of case based instruction is to facilitate d iscussion between students (Barnes, Christensen, & Hansen, 1994) In recent years, the method has also been adapted successfully for use in online environments (Gill, 2005; Webb et al., 2005) Interest in case based instruction for the preparation of teachers has grea tly increased in the last 20 years, resulting in a significant number of publication s regarding its use in pre service teacher preparation (Barnett, Tyson, & San Francisco, 1999; Harrington, 1995; Harrington, Quinn Leering, & Hodson, 1996; Harrington, 1999; Lundeberg et al., 1999; Wassermann, 1994) Proponents of the case method in teacher preparation emphasize the role of teachers as decision makers and ar gue th at traditional curricula do not train them for the complex realities of the classroom Leading advocates promote the use of decision cases as a method of developing critical thinking and problem solving skills (Garvin, 2003; Grossman, 1994; Lundeberg & Fawver, 1994; Lundeberg et al., 1999; McBride, Xiang, & Witte nburg, 2 002; McDade, 1995; Wood & Anderson, 2001) The parallel between the preparation needs of teachers and social workers as complex problem solvers and autonomous decision makers is clear. The use of the case method in teacher education most closely mirrors t hat which has been

PAGE 52

40 suggested in the social work literature, both in its purposes and methodology (Jones, 2003; Jones, 2005; Wolfer & Miller Cribbs, 2005 ; Wolfer et al., 2001) History of Case Based Instruction in Social W ork Cases have been used in the training of social workers since the inception of the profession (Cossom, 1991) ; however they have primarily been used primarily to support traditional e ducational methods Towle (1954; 1958) advocated the use of the case method in the 1950s as an effective method for the training of professionals, including social workers. Strategies and ratio nale for applying the method to the training of (1958) In spite of early signs of interest in the case method, the profession did not maintain its early in terest in the case method or embrace it as a primary instructional method as readily as other disciplines Nevertheless, social work has a long history of usi ng cases as instructional tools to facilitate transfer of learning from the classroom to the fi eld (Towle, 1954, 1958). Cases have been used in social work education to illustrat e various stages of the problem solving process, to expose students to the challenges of working with diverse populations, to describe social work intervention methods, intr oduce ethical dilemmas, simulate practice situations, and to conceptualize practice in a variety of contexts ( Gray et al., 2006; Gray & Gibbons, 2007; Haulotte & Kretzschmar, 2001; Jones, 2005; LeCroy, 1999; Rivas & Hull, 2004; Scales et al. 2002; Scales & Wolfer, 2006; Wells, 1998; Wolfer et al., 2001; Wolfer & Gray, 2007)

PAGE 53

41 Merseth (1996) identified three different types of cases used to integrate practice with learning. These include the exemplar case, the reflection case, and the decisi on case. The majority of cases utilized in social work education would best be described as exemplar cases as they (Fauri, Wernet, & Netting, 2000) This type of case illustrate s a concept or the successful resolution of a practice dilemma (Lundeberg, 1999) These cases are commonly used in social work problem from engagement with the client system to its resolution. Other cases may be used to stimulate reflection upon the actions taken by the professional in the case and engage students experientially in considering the consequences and impl ications of professional practice decisions in real world situations. The final type of case used is the decision case, which may also be referred to as a teach ing case (Jones, 2003) or a dilemma based case (Lundeberg 1999) This type of case presents the learner with a problem to be solved rather than an example of ideal practice (Cossom, 1991; Graham & Cline, 1980; Jones, 2003; Lundeberg et al., 19 99) Whi le exemplar cases demonstrate sound practice in a variety of co ntexts, decision case s induce students to engage in problem solving anal ysis, and ethical decision making Herreid ( as cited in Jones 2003) describes a good case as one th at includ es a controversial issue, generates empathy and authenticity by using direct quotations; has direct relevance to the reader, can be generalized to other situations, and encourages decision making An effective teaching case does not suggest an obvious sol ution to the problem it presents (Lynn, 1999 ) and is presented in a narrative account that is

PAGE 54

42 sufficiently detailed, contextualized and complex to allow for multiple levels of analysis and interpretation ( Levin 1995) Ill structured cases encourage students to face the ambiguity of reality and to grapple with the consequences of their choices (Lundeberg, 1999) According to Barnes, Christensen, and Hansen (1994) dealing with specific situations forces the student to present (p. 47). Fostering Reflective Thinking through Case Method Instruction McDade (1995) links the use of decision cases directly to the development of critical thinking by creating opportunities for student to a pply skills in analysis and decision making to realistic problems. She lists the following compelling argumen ts for the use of case method instruction to promote critical thinking 1 ) It models critical thinking and provides a laboratory in which students ca n practice and advance their critical thinking skills. 2 ) It emphasizes the process of analyzing information. 3 ) It is contextually based; that is, students must understand contextual nuances and make references and analyses accordingly. 4 ) It challenges students t o identify and challenge assumptions about situations and about their own beliefs. 5 ) It encourages students to imagine alternatives and explore theses for strengths and weaknesses.

PAGE 55

43 6 ) It helps students to integrate learning by incorporating theory into practice and practice into theory. 7 ) It enables students to develop critical listening skills because listening to and understanding the nuances and diversity of the thinking. 8 ) It provides opportuniti es for students to develop and test theories about how people and organizations function. 9 ) It helps students to develop teamwork and collaborative learning as students work together in small groups and in classroom to solve the problems presented by the cas e with the best means possible to serve the most goals. 10 ) It helps students to experience, explore, and test alternative ways of thinking. 11 ) It facilitates the consideration of different perspectives as other students present ideas, analyses, and solutions tha t no one student may have thought of (p.10) Each of these arguments for the development of critical thinking through case based teaching supports reflective thinking as it relates to reasoning through ill structured problems All but the first two argume nts for critical thinking assume that the problems encountered do not have a clear right or wrong answer, must be understood contextually, and can be interpreted from a variety of perspectives Rather than searching for absolute

PAGE 56

44 solutions, students are giv en opportunities to explore alternate perspectives and weigh the consequence and implications of proposed solutions Luitgaarden (2009) provides a compelling argument for the need to em phasize stimulating the decision making processes which experts use in real life situations to prepare future practitioners for the inherent complexities of social work practice He juxtaposes this against the current emphasis on evidence based practice and rational decision making which assume a high degree of predictabilit y, certainty, measurability and redundancy among cues. The following section will review empirical studies that have examined the relationship between the use of the case method and the development of critical and reflective thinking Research on the Cas e Method and Reflective Thinking Although empirical research exploring the relationship between case method strategies and critical or reflective thinking is limited, a number of educators have evaluated various aspects of reflective thinking in students e nrolled in case based courses. Content analysis procedures using the products of the case based course emerge as the strongest indicator of changes in reflective reasoning. Only one study utilized the Reflective Judgment Model itself to define and assess r eflective thinking. Case Analysis as a Method of Gauging Reflective Thought Harrington, Quinn Leering, and Hodson (1996) examined the degree to which case based instruction could be used to gauge the develo pment of critical reflection in student teachers. Based on an extensive review of the literature on reflective thought, the

PAGE 57

45 validity in other perspectives ( open mindednes s) 2) considering the moral and ethical consequences of choices ( responsibility) and 3) identifying and clarifying limitations in whole heartedness ) Although the authors did not specifically refer to the Reflecti perspectives mirrored the three broad categories of reflective thinking. Examination of a focus on authority an d assumptions about the certainty of knowledge, to a greater acceptance of ambiguity, a willingness to consider various perspectives, and responsibility for how knowledge is used to make decisions. Case Analysis and Increases in Reflective Thinking Harrin gton (1995) assessed the first and final cases analyses of 26 college juniors and seniors enrolled in an education course based on the way they framed problems, identified and grounded alternative perspectiv es on the case, substantiated solutions, identified consequences of action, and demonstrated an awareness of the limit ations in their own thinking. problems on a grounded rationale, to p rovide evidence to warrant solutions, to consider alternative perspectives and to demonstrate reflectiveness. Lundeberg and Fawver (1994) evaluated the extent to which a case based course impacted the abilit y of student teachers to: 1) identify issues and generate alternative approaches ( flexibility) 2) to consider various perspectives ( perspective taking ), 3) to apply theories to situated problems ( connectedness) and 4) to explain how particular

PAGE 58

46 theoretica l principles either confirmed or conflicted with their own beliefs ( meaningfulness) Three sources of data were collected from students: written analyses of a specific case at the beginning and end of the semester, a reflection on changes between their fir st and later written case analysis, and self reported written explanation of changes in their beliefs. Based on this data, the authors reported significant improvement on all measures of cognitive growth. Self so examined qualitatively using The researchers equated b ecoming constructivist in their beliefs in this context ith becoming better able to reason reflectively. Using a mixed methods approach, Allen (1995) compared the reflective thinking of students in an educational psychology class using decision cases to that of students in a traditionally taught educational psychology course. The most significant finding was that students enrolled in the case study class learned significantly more content than those enrolled in more traditional formats. Although no statistical ly significant differences were noted between the groups on the Defining Issues Test (Rest, 1979), a measure of moral reasoning and decision making ability , students in the case based courses made the greatest gains. The author noted that only 25% of the class had been devoted to case discussion and speculated that a greater focus on case analysis may have resulted in greater gains. Ho wever, based on earlier observations regarding the fact that cognitive

PAGE 59

47 development occurs relatively slowly, the length of the intervention may have been insufficient to asses s changes in reasoning. Bailey (2000) responses to cases and case writings in a senior level music edu cation course. The researcher observed that reflective thinking varied depending on the student, the case itself and the context in which the case was presented. Cases that were based on real classroom situations, required resolution of potential problems, preconceived ideas about teaching were the most effective in fostering reflective thought. Case Discussion as Integral to Fostering Reflective Thought According to a number of cognitive development theories (Kohlberg, 1969; Piage t, 1964), discussion of controversial issues serves as an important impetus f or changes in student reasoning. Therefore the case discussion in a case method course is 5) conducted case method discussion has an intensity and Levin (1995) examined the importance of the case discussion in learning outcome s. Using qualitative and quantitative procedures, she compared the pre and post case analyses of students who engaged in case discussion with those who only wrote about the case in dicated important changes in thinking in the discussion group. Comparisons of the discussion based course became clearer, more explicit, and better elaborated. Student

PAGE 60

48 teach ers also improved in their understanding of issues. Beginning teachers displayed an increase in their ability to reflect on their own teaching In contrast, teachers in the comparison group at all three levels of experience raised no new issues, insights, or topics, and tended to summarize their original thinking. An important finding in this study is that greater experience was associated with more complex, multi dimensional analysis of the cases and the ability to be more reflective However, the less ex perienced teachers appeared to profit the most from exposure to the case discussions, providing some preliminary indication that perhaps the cases and discussions provided inexperienced learners with necessary opportunities to for dealing with real world complexities (Macaulay & Cree, 1999, p. 189). Limitations of this study include the small sample size, the lack of inter rater reliability data on the holistic scoring rubric, and the fact that the subjects completed only two c ase analyses and one case discussion. Impact of Case Discussion on Epistemology Both Harrington (1995) and Levin (1995) attributed changes in student reasoning indicative of epistemological growth to participation in case discussions Students thinking i ncreased in cognitive complexity moving from a dichotomous, au thority based view of knowledge, to a more contextualized understanding and to increased responsibility for supporting decisions with evidence. Barnett and Tyson (1999) reported that math teachers engaged in case discussion shifted from viewing the source of knowledge as authority to greater autonomy.

PAGE 61

49 Allen and Razvi (2006) g case study discussions, their levels of epistemological understanding, and their critical thinking (2004) Levels of Epistemological Understanding, the authors evaluated the l evel of thinking demonstrated during case discussions by 19 undergraduate students enrolled in two educational psychology classes. Results indicated a general upward trend in all epistemological levels from the first case discussion to later discussion S tudents offered opinions more frequently and supported their ideas with evidence and logical argument more often. Limitations of this study included the small sample size and the fact that inter rater reliability for the rubric was not established nor the validity of the model addressed. Ratings were established through consensus by the two authors. A further limitation ions This study could be strengthened by analysis would have been less subject to factors such as the amount of time the teacher engaged in instruction and explanation, the level of instructor questions, and the level of student participation. Case Method Research in Social Work Education In the last 10 years, there have been a limited number of publications regarding the use of the case method in social work education. The m ajority of these have focused o n graduate school students. While several are theoretical, a number of the studies reviewed provide d preliminary evidence of promising learning outcomes as a result of

PAGE 62

50 case method instruction. Although reflective thinking has not been the specific focus of any of the studies, the development of critical thinking skills has been one learning outcome evaluated in most of the studies. These studies have numerous limitations as they are conducted in educational settings, without r andomized samples, and by the instructors themselves In addition, they used self report measures, which may or may not provide accurate depictions of actual learning. The Case Method as a Model for Meeting Educational Objectives Wolfer, Freeman and Rhode s (2001) presented the development of a case based MSW capstone course designed to facilitate application of theory to practice and collaborative work between micro and macro students. The authors noted two instructional challenges they faced: 1) fosterin of both faculty and students regarding the new methodology. They concluded that the an effective vehicle for promoting and reinforcing critical thinking and problem its use included providing faculty with tangible support, having a committed cohort of faculty, developing multiple methods of ev aluation, and taking proactive steps to manage student anxiety and resistance regarding unfamiliar teaching methods. Jon es (2003) encouraged case method instruction in graduate social work education as a way to assist collaborative work, demonstrate how po wer and control are shared in relationships, transfer knowledge from the classroom to the field, develop problem solving and decision making skills, and enable students to identify their own

PAGE 63

51 biases regarding clients from diverse economic, racial and ethni c backgrounds. He recommended the case method as a strategy for encouraging critical thinking, problem solving professional decision making, and oral and written communication in MSW students. Wolfer and Gray (2007) recommended the use of decision cases as a strategy for helping students understand the relevance of policy to practice and involving students in policy advocacy make the connecti ons between policy and practice. The authors recommend that educators write decision cases based on local or state issues, which afford greater opportunities for direct student involvement. The analysis of these decision cases is used analytical skills, political skills, interactional skills and value clarifying skills. They report positive student responses to a decision case written by faculty regarding an actual policy dilemma that was occurring in their state Gray, Wolfer and Ma as (2006) recommended case method instruction as a strategy for energizing interest in community organizing and involving students in grass roots efforts. They suggest that using decision cases enables students to critically analyze community problems, develop appropriate solutions and develop self awareness through collaborative work. They suggest that decision case teaching fits well with grass roots philosophy as a method for developing the leadership skills r equired to do grass roots organizing These include listening, questioning, self confidence, understanding motivations, problem solving, and clear articulation of problems.

PAGE 64

52 Assessing Perceived Learning Outcomes An unpublished study completed by Wolfer and Miller Cribbs (2005) described numerous measures used to provide multiple evaluative loops to assess learning outcomes for the earlier publication (2001). A Case Method Lear ning Outcomes Scale (CMLO) was developed based on case method teaching literature and a dministered at mid semester and end of semester The instrument f the CMLO revealed Cronb lpha in dices of .94 at midsemester and.96 and end of semester. Results indicated that students believed that they had significantly increased their competence in a number of targeted skills, including skills related to problem solving decision maki ng and critical thinking as a result of participation in the course. ratings indicated statistically higher levels of self awareness, metacognition autonomy, and self efficacy. Although based on student self report, this research provides preli minary evidence of the efficacy of the case method in providing students with procedural as well as content knowledge. Jones (2005) evaluated the effectiveness of the case method to help MSW students understand and apply mezzo and macro practice dimensions Two cohorts of students (n=114) we re evaluated using pre and post test analysis on a 15 item measure report. Internal consistency estimates for the 15 Students repor ted statistically s ignificant increases in 13 of the 15 areas queried, including increases in their perceived ability to apply critical thinking skills to clinical

PAGE 65

53 situations, integration of first and second year content, and application of theory. Limitat ions include the fact that the findings are based upon student perception of their learning versus the actual demonstration of the learning through objective measures Alternate explanations for the learning outcomes are also possible, as students were als o enrolled in other courses and involved in the field as interns. An unpublished study by Reitmeier and Wolfer (2007) approached the subject of exercise designed to inductively generate ideas and facilitate meaning making, 149 discussing the cases in [the case method course ] ors identified more than 45 learning outcomes, which were clustered into six categories: basic professional skills, personal dispositions, using prior knowledge, gaining new knowledge, professional use of self, and a variety of problem solving skills. Of i mportance r elative to this study was that students believed they were better able to identify problems and r ecognize situational complexity, realized that there was not only one solution to problems and that they were more likely to consider various perspe ctives and a pproaches to problems. They also indicated that they were better able to reflect and learn from insights gained and that they were increasingly able to think critically and independently, observing, assessing, analyzing, and synthesizing data. An important contribution of this study is that students were allowed to generate their own answers, adding depth to the previous findings A limitation is that the results were based on self report, and that objective measures were not used to validate st

PAGE 66

54 Limitations of the Case Method A number of authors have cautioned that the claims of the case method may be overstated (Lundeberg, Levin, & Harrington, 1999 ) and that the case method may not be suited to all learners (Allen & Razvi, 2006; Cossom, 1991; Ertmer Newby, & MacDougall, 1996) The case method may challenge students excessively who are developmentally unable to accept ambigui ty and uncertainty beyond their level of comfort Cossom (1991) no ted that it does not appeal to all students and therefore will not draw neutral responses. His survey of student satisfaction with case teaching comp ared to other methods found that 58% reported the better much better other methods, while 27 % believed it w worse much worse and self regulati on may affect their ability to benefit from the method. Ertmer et al. (1996) observed that case persistence and self regulati methods study involving nine students, they found that students with high levels of self regulation (n=5) began with and maintained positive attitudes toward case based instruction, while students with low self regulation (n=4) questioned the value of cases and lacked confidence in their analys es. All students made gains, but those with high levels of self regulation demonstrated greater ability to consider multiple perspectives and adopt process goals (Ertmer et al., 1996). Results of research on the Reflective Judgment Model indicate a signifi cant amount of diversity in the way that students reason through ill structured problems. If

PAGE 67

55 and use appropriate teaching strategies to foster reflective thought, stude nts may become frustrated as demands exceed their current level of functioning Instructors utilizing case method approaches must be sensitive to the epistemological limitations of students who have not yet developed the level of cognitive flexibility and complexity required to engage in reflective thought and tailor course requirement to require tasks that will foster necessary growth. Summary The review of the selected literature supports this research, which will seek to evaluate the extent to which cas e method teaching influences the reflective thinking of MSW students. In addition, the literature reviewed provides the following evidence for the need and significan ce of the proposed research: 1) t he research reviewed corroborates the critical need for s ocial work educators to develop evidence based strategies to foster reflective thought in future practitioners ; 2) a limite d number of studies on the outcomes of the case method have reported positive effects on critical and reflective thinking; however, f ew have used objective measures of reflective thinking or comparison groups ; 3) s the nature, source, and ju stification of knowledge claims; 4) t he Reflective Judgment M odel is an empirically validated model of reflective thinking that is sensitive to changes in cognitive complexity and epistemic assumptions; 5) t he Reasoning about Current Issues Test is an adequate standardized measure for the a ssess ment of reflective th ought; 6) g en der, race and ethnicity are not consistently related to scores on the RCI; however,

PAGE 68

56 age and educational level have been consistently shown to be positively correlated with RCI scores; and 7) c ontent analysis of student case analyses is an effe ctive method for gauging changes in reflective thought as a result of an educational intervention. This study address es several gaps in social work educational literature To date, the Reflective Judgment Model has not been used to assess the reflective th inking of social work students. The fact that social work primarily involves students in addressing problems that can be characterized as ill structured problems makes this especially relevant Although schools of social work are mandated to prepare studen ts to apply critical thinking skills in the context of social work practice, empirical evidence to support the use of strategies for the achievement and assessment of th is aim is lacking. Finally, the claims regarding the case method as an effective strate gy for increasing reasoning skills related to social work practice merit investigation.

PAGE 69

57 C hapter III Methodology This chapter describes the research design and methodology used to assess the effect of a case method course on the development of reflectiv e judgment and refle ctive thinking skills in second year MSW students. The chapter will first describe the design. The following sections describe the participants, the proce dures used to recruit participants and collect data, and the instruments used to assess reflective judgment and reflective thinking skills. The final section addresses the data analysis strategies and the Hypotheses The hypotheses for this study were developed based on the preceding review of the literature, which provides preliminary support for the efficacy of the case method in fostering reflective judgment and reflective thinking skills, and the use of the Reflective Judgment Model (King & Kitchener, 1994) to assess developmental changes in the epistemology that supports reflective judgment. The hypotheses are also supported by literature regarding the use of the Reasoning about Current Issues (RCI) Test ( Wood et al., 2002) as a standardized measure to assess developmental growth in the levels of reflective judgment as defined by the Reflective Judgment Model. Finally, the literature

PAGE 70

58 supports the efficacy of content analysis as a method for gauging progress in the development of reflective thinking by examining the problem solving skills and hypotheses: Hypothesis 1.1 Gender will not significantly influence RCI scores at pre or posttest. Hypothesis 1.2 Race/ethnicity will not significantly influence RCI scores at pre or posttest. Hypothesis 1.3 Age will significantly influence RCI scores at pre and posttest. Hypot hesis 1.4 e will significantly influence RCI scores at pre and posttest. Hypothesis 2.1. MSW students engaged in a case method course will increase their reflective judgment scores on the RCI between pretes t and posttest. Hypo thesis 2.2. MSW students engaged in a case method course will demonstrate greater increases on RCI posttest scores than those who are not engaged in a case method course. Hypothesis 2.3 Gains in the reflective judgment scores of studen ts engaged in a case method course will exceed increases that can be attributed to maturation or other educational experience. Hypothesis 3.1. MSW students engaged in a case method course will demonstrate increased reflective thinking skills based on thei r scores on a customized rubric

PAGE 71

59 designed to assess problem solving skills related to the resolution of ill structured problems. Hypothesis 3 2 Students scores on the customized rubric will correlate positively with scores on the RCI. Variables Independe nt Variables The primary independent variable for this study was group membership as defined by participation in the case method course. In addition, this study examined the influence of gender, race/ethnicity, age, and years of experience in social work practice on reflective thinking. For the purposes of the current study, the case method was defined as a student open which] require students to formulate ( Wolfer, 2006 p. 3) A subsequent section describes the specific methods used in the case method course which is th e subject of the study. Dependent Variables This study assessed changes in reflective thinking following participation in a case method course. Two related aspects of reflective thinking are assessed in this study: reflective judgment and the reflective t hinking skills required to engage in reflective judgment. Reflective judgment (RJ) (King & Kitchener, 1994) is a term used by the authors of the Reflective Judgment Model (RJM) to describe the epistemic cognition th at supports the recognition of enduring uncertainty typified by

PAGE 72

60 ill structured problems, and the necessary development of strategies for evaluating potential solutions in light of available information ( Kitchener & King, 1990) The Reflective Judgment Model describes a developmental continuum based upon epistemic assumptions regarding the source of knowledge and the justification of knowledge claims. Optimal levels of RJ are associated with increased cognitive c omplexity and the effective justification of beliefs and conclusions. Assumptions that knowledge is actively constructed, understood in relationship to context, and that some knowledge claims are more credible than others, undergird the strategies adopted by reflective thinkers ( Wood 2000) In summary, R eflective J udgment requires: t he recognition of uncertainty or perplexity regarding the solution of a real problem; the assumption that such problems can be resolved by a process of reasonable inquiry for constructing a well informed understanding of the problem; the assumption that beliefs and conclusions are justified by using evidence and arguments which can be defended as representing the most complete, most comp elling, or most plausible understanding of an issue based on the current evidence; and the perspective that judgments must be grounded in relevant data, and evaluated by suitable criteria.

PAGE 73

61 The RJM postulates that reflective thinking can only be assessed a s it relates to the resolution of ill structured problems. Learners who hold epistemic assumptions consistent with R J will show evidence of problem resolution strategies that facilitate the consideration of knowledge claims in light of the evidence that su pport those claims to ( Wolcott & Lynch, 1997) to operationalize the thinking skills associated with reflective judgment strategies. Refl ective thinkers will develop strategies to construct knowledge, based on an objective process of critical inquiry. Skills that demonstrate evidence of this approach to problem resolution will include: 1 ) T he ability to identify and use relevant information wh ile acknowledging uncertainties; 2 ) T he ability to integrate multiple perspectives and clarify assumptions; 3 ) T he ability to qualitatively interpret information and create a meaningful organization; 4 ) T he ability to use guidelines or principles to judge obje ctively across the various options; 5 ) T he ability to implement and communicate conclusions for the setting and audience; 6 ) T he ability to use evidence/information effectively to justify conclusions and assumptions; 7 ) T he ability to acknowledge and monitor solut ion limitations through next steps.

PAGE 74

62 This study employed mixed methodology to assess the effect of a case method course on the reflective thinking of MSW students. A quasi experimental pre post nonequivalent control group desi gn was utilized to explore whether students who participated in a case method course demonstrated greater increases in reflective judgment over the course of a semester than those who did not. Both the intervention and comparison groups participated in a p re and posttest measure using the Reasoning about Current Issues Test (RCI), which is an online, standardized measure that has been widely used to assess reflective judgment ( Wood et al., 2002) Concurrently, studen ts enrolled in the case method course submitted their initial and final decision case papers to the researcher. Content analysis procedures facilitated the assessment of these primary products of the case method course for evidence of the skills associated with reflective judgment. Although content analysis is a qualitative method, it overlaps with quantitative methods in that it produces data that can be analyzed statistically (Schutt, 2004) The results of the con tent analysis were used for both quantitative and qualitative analysis. Figure 2 below describes the timeline for data collection of the qualitative and quantitative measures.

PAGE 75

63 Figure 2. Research Timeline Description of Setting This study was conduct ed at a large metropolitan university in the southeast, which has a well established school of social work with a national reputation for excellence in social work education. This setting was selected because it is one of the few Week s 1 4 Week s 10 14 Particip ation in Case Method Course or Traditional Course Group 1 Final Case Analysis Submitted RCI Posttest Group 1 O 2 Group 2 O 4 Group 1 First Case Analysis Submitted RCI Pretest Group 1 O 1 Group 2 O 3 Recruit Students Spring Semester 2008 Weeks 1 14

PAGE 76

64 schools of social work in the United States that is currently using a case method approach systematically in its MSW curriculum. In addition, faculty have contributed to the literature on the use of case method teaching i n social work ( Gray et al., 2006; Scales et al., 2002; Scales & Wolfer, 2006; Wolfer, 2006 ; Wolfer et al., 2001) During the final semester of their advanced year, social work stu dents are required to take a capstone course that utilizes a case method approach to facilitate the integration of micro and macro content, theory and practice, and all previous learning in the MSW curriculum. Although traditional social work education com monly utilizes illustrative or exemplar cases to meet curricular goals, this capstone course uses decision cases as the primary vehicle of instruction. In contrast to traditional cases, decision cases present open ended practice dilemmas to elicit problem formulation and problem solving The primary instructional method in the course is the discussion o f decision cases featuring real world problems that social workers face in the course of practice at the micro, mezzo and macro level s An additional factor in the selection of this setting was the availability of a large cohort of MSW students, providing a favorable opportunity to secure an adequate sample size for the methods proposed. The school of social work at the university has an average enrollment of 300 MSW students, including approximately 125 foundation year students and 200 advanced year students (Council on Social Work Education, 2007b) Description of Case Method Course The capstone c ourse was developed in response to a survey of social service executives that indicated that social service employers highly valued critical thinking

PAGE 77

65 skills, open mindedness, and skills related to research and evaluation (Dalton & Wright, 1999 as cited in Wolfer et al., 2001). Following careful consideration regarding potential teaching practices which would facilitate these types of outcomes, faculty decided to use ( Wolfer, 2006 p. 8) Faculty developed open ended decision cases describing actual social work practice situations in significant detail. The decision cases used were intentionally cting statements (by the various participants) Prior to launching the innovation, faculty were trained to facilitate discussion of the decision cases and challenge stu dents to think more deeply about relevant issues that emerged. Since the course innovation began nine years ago, approximately 200 graduating MSW students have en rolled in 10 to 12 sections of the capstone course each spring. Classes meet weekly for three hours over a 14 week semester during which they analyze twelve different decision cases. In the spring of 2008, eight instructors taught 11 on site sections of the case method course. Ten of these sections participated in the study. Instructors teaching t he course met once a week to plan collaboratively for weekly case discussions and instructional methods. The experience and familiarity of the individual instructors with case method teaching varied, with some having many years of experience and others hav ing limited to no exper ience with the method. However, collaboration among

PAGE 78

66 instructors on a weekly basis was intended to promote uniform delivery of instructional methods. All instructors used a common syllabus. The syllabus included a common schedule of d ecision cases and supplemental readings and required assignments. Required assignments included written case analyses and an annotated resume. However, instructors were free to modify the way they assessed student work and some of the guidelines for writin g case analyses. For example, one instructor required students to propose both long term and short term strategies for the resolution of the dilemma, while others did not. Another instructor encouraged students to describe how they would formally evaluate the success of the solution proposed. Students from micro and macro practice tracks are included in the sections in order to facilitate micro and macro content integration. Students prepare for instructor facilitated discussions by completing a written ex ecutive summary of the decision case. The summary must include problem identification, analysis of key internal and external issues, three or more possible alternative strategies with advantages and disadvantages, a recommendation for a specific strategy w ith justification, and the source of the rationale for the analysis and recommendation. Therefore, students must have wrestled with issues presented in the case prior to class participation. The ensuing case discussion s hey encounter the views of others and are challenged to defend or alter their own positions in light of the views presented by classmates. Seating is arranged in a semi circle to encourage small

PAGE 79

67 class session include d a two hour case discussion followed by a debriefing. During the remaining time, students may provide feedback to each other, and engage in discussion regarding topics such as improving writing assignments, st udents' own field dilemmas, supplemental readings, professional self care, or the annotated resume. Students are required to submit a written case analysis for seven of the twelve cases for assessment by their instructor All students submit an analysis of the first two cases for review in order to facilitate early feedback to students regarding their performance. After submitting the first two cases, students choose five of the remaining 10 cases to submit for a course grade. On alternate weeks, they provi de feedback to peers who are submitting their case analys e s for assessment. A Case Analysis Evaluation Matrix, which is included in the course syllabus, provides students with clear assessment criteria. Please refer to t he course syllabus in Appendix D for a complete description of the instructional methods, course calendar, assignment descriptions and the Case Analysis Evaluation Matrix. Participants Intervention Group Non probability purposive sampling was used to accomplish the goals of this study. Part icipants were recruited from the on site MSW students enrolled in the case method capstone course described above. Two off campus sections of the same course were not included because of a concern that there might be some unique differences between on camp us and off campus students. The Procedures section describes the methods used to recruit students.

PAGE 80

68 During the spring semester of 2008, 174 advanced year students enrolled in the capstone course including 54 advanced standing students. Advanced standing st udents proceed directly to the advanced year of graduate school based on the completion of a BSW degree from an accredited program and a minimum GPA of 3.0 in the 12 hours required during the summer session. Twenty three percent of the students enrolled i n the case method course participated in the study. The intervention group included 40 MSW students enrolled in 10 sections who completed at least one of the pre and posttest measures. Of the 40, 27 students participated in both the quantitative and quali tative aspects of the study, having valid RCI and case analysis scores at pre and posttest. Five students in the intervention group participated in the RCI pretest, submitted their initial and final case analyses but failed to participate in the RCI postt est. Eight participated in the RCI pre and posttests, but did not submit both of the required decision case analyses. The intervention group was predominantly female (97%), Caucasian (82%), traditionally aged (22 26 yrs) (67%), and had no professional soc ial work experience (56%). Table 2 depicts the demographic characteristics of the participants. Efforts were not made to stratify the sample with regard to gender or race based on the findings of previous studies that gender and race did not significantly influence Reflective Judgment scores (King & Kitchener, 2002) Comparison Group Foundation year students enrolled in a social work research methods course served as the comparison group. This course was selected as a comparison because it is required for all foundation year students, and is taught using

PAGE 81

69 traditional methods as opposed to the case method. In addition research courses have been associated with the development of critical thinking skills in the literatu re (Boyer Commission on Educating Undergraduates in the Research University, 1998; Gibbs, 2007; Kersting & Mumm, 2001; Lynch et al., 2001; Rowe, 2007a) The foundation year cohort is typically smaller than the advanced year cohort because it does not include advanced standing students. Students enrolled in the researc h methods course ( n= 84) were expected to be comparable to the intervention group in terms of demographics, although the lack of advanced standing students introduced the possibility of differences between groups in terms of age and social work experience. A further rationale for including these students as a comparison group is the fact that comparing pretest scores of foundation year students to the advanced year intervention group would provide a cross sectional comparison between first and second year st udents on RCI scores at pretest. The difference between the RCI mean scores would serve as an estimate of a maturation effect, ( i.e., the amount of change that could be attributed to graduate school experience prior to exposure to the case method course ) Eighteen students (21%) enrolled in the research methodology course, comprised the comparison group and participated in the RCI at both pre and posttest. Students who began the study but were excluded from the sample because they did not have both pre an d posttest scores included nine who did not complete the posttest, and two who took the posttest but had not taken the pretest. Like the intervention group, participants in the comparison group were predominantly female (89%), 22 to 26 years old (67%), Cau casian (79%), and had no

PAGE 82

70 social work experience (68%). The mean age ( M =29) was impacted by three outliers over the age of 40 as compared to only one in the intervention group ( M =27). Table 2 Demographic Characteristic s of Intervention and Control Groups Demographic Intervention Group Comparison Group Gender Female Male N % N % 39 97.4 16 88.9 1 2.6 2 11.1 Age 22 26 27 39 40 up 27 67.5 12 66.7 12 30.0 3 16 .6 1 2.5 3 16.6 SW Experience 0 yrs > 3 yrs 3 5 yrs 6 10 yrs 10+ yrs 23 57.5 14 77.8 11 27.5 2 11.1 2 5.0 0 3.7 4 10.0 2 11.1 0 0 0 0 Race/Ethnicity Minority White 7 17.5 1 5.6 33 82.5 17 77.8 Sample Size Estimates of effect si zes for short term educational interventions using the RCI are not available. Although there is considerable interest in developing educational interventions that will foster epistemological growth ( Wood & Kadrash,

PAGE 83

71 2002) to date research using the RCI to assess changes resulting from short term interventions has not been published. Effect sizes for between group differences on measures of epistemology observed across educational levels have been approximately one standar d deviation (King & Kitchener, 1994; Wood 1997) which is considered a large effect size (Cohen, 1988; Cohen, 1988) Because epistemolo gy develops slowly, studies involving shorter periods have been less successful in detecting differences between groups (King & Kitchener, 1994; Wood & Kadrash, 2002) Wood and Kadrash (2002) caut ioned that although there may be educational interventions that foster epistemological growth, the effect size is likely to be small, given that the changes between the freshman and senior year are modest (.51). Based on norming information for freshmen an d senior performance on the RCI, they observed that although a sample size of 21 freshmen and 21 seniors would yield sufficient statistical power to detect differences across educational levels, a similar study attempting to detect the small developmental changes that occur between the freshman and sophomore year would require 3,770 participants! A significant limitation of using the RCI to assess change in RJ over a semester is that even substantial growth may be undetected due to the lack of statistical power. Given the lack of accessibility to a large sample of MSW students enrolled in a case method course, it was not possible to increase power by substantially increasing the sample size. Nevertheless, a decision was made to continue with plans to use th e RCI, based on the lack of research using the RCI with MSW students, and the findings of one study with a

PAGE 84

72 relatively small sample size (n=80) that found modest differences in RJI scores as a result of a semester long educational intervention (Kronholm, 1996) Wood et al ., (2002) speculated that these findings (which have not been replicated) may have been influenced by inequalities between the two groups at the start of the stu dy, but concedes that the educational intervention might have been responsible for the observed increases in RJ levels. The use of qualitative methods in this study allays some concern regarding the case analyses has been used effectively to assess changes in the critical or reflective thinking of students enrolled in case method courses, in spite of small sample sizes (Allen, 1995; Ertmer et al., 1996; Harrington, 1999; Levin 1993; Lundeberg et al., 1999) Process and Procedures Recruitment of Participants Intervention Group In order to recruit part icipants, the researcher traveled to the host institution during the first week of the semester. On January 17 and 18, 2008, the researcher gave a 10 minute presentation in nine of the ten sections of the capstone course to describe the study and the propo sed methods for data collection. The researcher was unable to meet with the two sections that met on Saturday; however, these sections were provided with a pre recorded explanation of the research and invitation to participate. One of the professors in the se sections collected the names of interested students; the other chose not to participate.

PAGE 85

73 In a 10 minute presentation, students in the capstone course were informed of the importance and rationale for the study, the procedures involved in taking the RCI and the submission of their initial and final case analysis. Students were assured that their individual scores would not be given to the graduate program or their instructors and that their participation was voluntary and had no bearing on their grade. S tudents were also informed of modest incentives that included receipt of a $5.00 Amazon.com gift card each time they agreed to take the RCI and entry into a drawing for a $50.00 Amazon.com gift card each time they submitted their cases analysis papers. One hundred and ten students from the capstone course agreed to participate in the research project by signing a roster and providing the researcher with their email addresses. All one hundred and ten students received an email on January 23 or 24 2008 with an invitation to participate in the research and the following instructions: If you would like to participate in this study, just hit reply to this message. When I receive your email, I will send you a unique identifier, a link to the website to begin you r survey, and an Amazon.com gift card for $5.00, which you can use immediately. In addition, if you attach your initial case analysis with your reply, your number will be entered in a drawing for an additional $50. 00 Amazon.com Gift Certificate. ( See Appen dix E 1 ) Six email addresses were returned undeliverable. A reminder email was sent out on January 2 7 2 008 to all remaining students ( See Appendix E 2 ) Fifty nine (54%) of the original one hundred and ten students indicating a willingness to participate responded to the email invitation. Each of these students was

PAGE 86

74 assigned an identifier that they were to use for the RCI pretest and posttest and for identifying their cases analyses (See Appendix E3) Forty eight students (81%) completed the RCI pretest and 38 of the 59 (64%) submitted their initial case analysis papers. Winners of the drawings were notified by email and an announcement was made to all the participants regarding the winner at pre and posttest. Thirty (51%) submitted a usable final case anal ysis as well. One student resubmitted the initial case analysis rather than the final case analysis, and did not respond to email requests to send the final case instead. Papers were routed to a research assistant who made certain that they were completely de identified with regard to name, section, case number or date. All students who participated in the pretest automatically received an invitation to participate in the posttest with an Amazon.com gift card, a link to the website and a reminder of their unique identifier and the password for the RCI website on April 11 14, 2008 ( See Appendix E 4 ) R eminders were e mailed to the participants on April 23, 2008 and again on April 29, 2008 (Appendix E5) Additional drawings for gift cards were added as incenti ves to complete the posttest by the closing date (See Appendix E6) The RCI posttest was available from April 14, 2008 through May 1, 2008. Comparison Group. Because the majority of the Research Methodology sections met on Monday, and the researc her was on ly able to be at the u niversity on Wednesday through Friday, students in the comparison group were recruited via a pre recorded DVD. The DVD presentation, which included all of the key points presented to the intervention group, w as to be played in class d uring the second week of the semester. However, the

PAGE 87

75 Martin Luther King holiday caused the pre recorded presentation to be delayed until the week of January 28, 2008, which was the third week in the course. The pre recorded presentation was clearly not as effective as the personal appeal made in the intervention group as only 30 of the 84 students enrolled in the research course volunteered to participate. Professors of the research classes collected the names and email of the 30 students who agreed to part icipate. These students received an email invitation to participate in the study on February 5 2008 (See Appendix E 7 ) A reminder email was sent to comparison group participants on February 10 2008 with an added incentive (See Appendix E8 ) Students who completed the RCI by February 14 were also entered into a drawing for a $40 Amazon.com gift card. Twenty seven students (90%) responded to the email. Twenty of those responding completed the pretest and 18 (72%) completed the study by taking the RCI postte st. Two additional students took the posttest who had not taken the pretest. Attrition Several factors may have contributed to the significant attrition experienced in this study. Although more than half of all the students enrolled in the capstone course initially indicated their support and interest in the study, continued interest appeared to be mediated by the section the student was in. Students in Sections 001 and 003 were the most likely to participate and to follow through with taking the posttest a nd submitting the final case analysis. Students in Sections 002, 005, and 010 had only one student each who completed the study. Table 3 demonstrates the breakdown of student participation in the various aspects of the study based on their section of the c apstone course. Eight

PAGE 88

76 students who responded to the email did not participate in any of the measures and are not represented in the table below. One was in Section 001; four were in Section 002, and one each in Sections 003, 004, and 010. Table 3. Partic ipation of Intervention Group by Section Section RCI 1& 2 Case 1&2 RCI 1 & 2 Case 1 RCI1 & Case 1 & 2 RCI1 & 2 RCI 1 Case 1 RCI I Case 1 Total 001 6 0 0 1 1 0 0 9 002 1 0 0 0 1 0 0 2 003 7 0 1 0 0 0 1 9 004 1 1 2 1 0 0 0 5 005 1 0 0 0 1 2 1 5 006 3 0 0 3 0 1 0 7 007 3 0 1 1 0 0 0 5 008 3 0 0 0 1 1 0 5 009 1 1 0 0 1 0 1 4 010 1 1 0 0 0 0 0 2 *area in gray represents students that had at least one valid pre and posttest measure A seco nd factor contributing to student attrition was an unforeseen turn of events at the time of posttest. Within days of opening the survey for the posttest, the RCI password stopped working. It took the researcher over 24 hours to reach an administrator, and another day to resolve the problem. According to the administrator of the RCI, the organization had been undergoing some restructuring which resulted in the inaccessibility of the administrator as well as the need to reset the website password. Although al l participants were sent an email regarding the password failure as soon as the problem was

PAGE 89

77 identified, and again, as soon as the problem was resolved (See A ppendix E9 ) six students in the intervention group and four in the comparison group completed the demographic portion on Survey Monkey during this time but did not complete the RCI. Given the fact that the posttest occurred at the end of the semester, pressures related to final exams, assignments, and graduation were also a likely factor in student at trition. Vigorous efforts were made to encourage student participation at posttest. Between April 1 3 and April 15, 2008, all students in the intervention group received a personalized invitation to take the posttest and an Amazon.com gift card with their i dentifier and a link to the Survey Monkey website. Based on previous research (Dillman, 2000) indicating that it is more effective to include incentives with the invitation, rather than wait for participants to respo nd to an offered incentive, students were mailed the Amazon.com gift card with the invitation (Se e Appendix E10 ) The chair of the capstone course also posted a reminder on the Blackboard site used to communicate with students enrolled in the course. On Ap ril 19, students in the comparison group received a similar letter. On April 23, 2008 all students received a reminder to take the posttest if they had not already done so with a promise to enter all students who participated by April 28 in a drawing for a $50 Amazon.com gift card. Because of low student participation, fifteen students in the intervention group and six students in the comparison group also received personal appeals on April 27 or April 28 to participate in the standardized measure or submit their case study ( See Appendix E11 for sample letter )

PAGE 90

78 Statistical procedures were used to determine if there were any differences between those who completed the study and those who did not. Independent samples t tests indicated that there were no statis tically significant differences between groups with regard to age, t (73) = .693, p=. 491, or RCI scores, t (71) =1.187, p =. 239. Comparisons were made between participants and non participants on the remaining demographic variables using 2x2 Chi square pro cedures (on the dichotomous versions of the variables). No differences were found for gender, 2 (1, N =75) = .294, p =. 588, race, 2 (1 N =75) = 3.547, p =. 060, or experience, 2 (1 N =75) = 2.241, p =. 134 Permissions Permission was obtained on November 29, 2007 from the administrators of the RJM to utilize the RCI online assessment measure provided through the University of Denver. A nominal fee of one dollar per test is charged to graduate students utilizing the RCI, provided the data is added to their dat abase. Sheila Summers Thompson, the test administrator at the University of Denver provided the researcher with the URL for the website, along with a user name and password, which provides participants with access to the online test ( See Appendix F ) Permi ssion to conduct the research was received from The Internal Review Board (IRB) of the University of South Florida (USF) on December 21, 2008 IRB approval from the University of South Florida was forwarded to the chair of the capstone course at the host s etting. This approval was accepted by the IRB at the host university. Because

PAGE 91

79 the entire procedure was to be carried out over the internet, a request for waiver of documentation of informed consent was obtained from the IRB at USF. See Appendix G. Testing Procedure Students who replied to the invitational email were provided with a link to a Survey Monkey web page at pretest and posttest. Survey Monkey, a Web based survey builder, was used to provide participants with information regarding the testing proce dures, informed consent, and a demographic questionnaire before proceeding on to survey that contained a variety of question styles, including multiple choice, and short answer questi ons. Upon entering the site, participants were asked to read the consent form and indicate their consent by checking the appropriate box. A brief questionnaire requested information regarding their study identifier, age, gender, ethnicity, course section, and social work experience. All fields were required in order to advance to the next section. Once completed, students were redirected electronically to the RCI website. Participants could reenter the website if they were unable to complete the test. Stud ents had access to the site from their own computers at their convenience. The survey was available for a two week period for each testing. Each group was begin the pr etest until 10 days after the intervention group, the survey was still available for each group for a total of two weeks. At the end of the two week period, the survey was closed and no longer accessible to respondents The Survey Monkey questionn aire is i ncluded in Appendix H

PAGE 92

80 Collection of Case Analysis Papers Students in the intervention group who chose to participate by submitting their email address with their unique identifier and all personal information removed. The papers were automatically forwarded to a research assistant who opened them and removed all references to the case number, date of submission, and any remaining identifying information. The assist ant placed the electronic records in a folder contained in a flash drive and identified them by the number assigned to the participant. The researcher did not review student submissions until all initial and fin al case analysis papers were collected. Instr umentation There are two methods of assessing developmental changes in reflective thought. One involves production tasks requiring the student to generate solutions to problems p. 89). The other involves recognition tasks, which require the learner to choose the best response among a series of options. The review of the literature addresses the strengths and limitations of each. Utilizing each type of measure by using the Reas oning about Current Issues Test which require production tasks, allows for a richer assessment of reflective thought. Additionally, using different assessment measures, a llows triangulation, which has been

PAGE 93

81 shown to enhance internal validity when exploring complex and multifaceted constructs (Denzin, 1978) The RCI assesses the epistemic assumptions that students are currently using as they approach the resolution of ill structured problems. In contrast, case analysis papers instead provide opportunities to demonstrate a range of reasoning skills, whi ch are supported by the epistemic assumptions in use. Therefore, to complement the use of the RCI, a skill thinking skills in their written case analyses. The following section wi ll describe the RCI. The skill based rubric developed to assess reflective thinking skills evident in decision case analysis papers is described under content analysis procedures. Description of RCI The Reasoning about Current Issues Test (RCI) is an onli ne instrument developed by King and Kitchener (2002) to assess reflective thinking based on their Reflective Judgm ent Model (RJM) source, and certainty of knowledge claims. The RCI uses a Likert format to represent multiple stages of the RJM. Respondents are presented with three ill structured problems that represent contemporary issues about which there are multiple perspectives (See Appendix B). The three dilemmas on the current version of the RCI address questions regarding the causes of alcoholism, immigration policy, and the best methods for the preparation of the future workforce.

PAGE 94

82 The RCI requires respondents to write a short statement justifying their own position in response to each dilemm a in order to encourage them to think about their own views. They are then asked to rate a series of ten statements on a four point scale (from very similar to very dissimilar) to indicate how comparable they are to their own views. Each position represent s prototypical statements from respondents who participated in the Reflective Judgment Interview. To address concerns that students might endorse statements that sound impressive but do not realistically reflect their own positions, the test also contains nonsensical but grammatically correct statements. Students are instructed to expect such statements and given the option of rating them as Scores on the RCI range from 2 through 7 representing stages 2 through 7 of the Reflective Judgment Mo del. Answers to each dilemma are scored individually, and then averaged to determine the Reflective Judgment Score. The RCI is assumed to measure support than the RJI ( King & Kitchener, 2004). The RCI is available through the University of Denver in collaboration with the University of Michigan. Tests are scored at the University of Denver. Reliability. Internal consistency estimates for the RCI range from the mid .70s to low .80s depending on the sample (Wood et al., 2002). Wood found a coefficient alpha estimate of internal consistency of .83 based on a meta analysis of 6,101 individuals tested (Wood, 2004 as cited in Pittman, 2006). Owen (2005) reported an internal consistency alpha coefficient of .78. Coefficient alphas for global measures should

PAGE 95

83 ideally be over .80 (Royse, Thyer, Padgett, & Logan, 2006) however, coefficient alphas over 70 are considered adequate (Spicer, 2005) Because internal consistency rates for the RCI have been lower than those found using the RJI, which ranged between .55 and .99, the Reflective Judgment website suggests that it should only be used to make inferences about Reflective Judgment scores on a group level (King et al. ,n.d. ) Validity. Wood, Kitchener and Jensen (2002) suggest that one crit erion for judging the validity of the RCI is to determine whether it consistently reveals differences similar in magnitude to those found for the Reflective Judgment Interview, which was empirically validated over twenty years as a measure of reflective ju dgment. Data collected from over 8,000 undergraduate and graduate students indicates educational level differences of about one standard deviation, which is consistent with the results of the RJI ( W ood & Kadrash, 2002; Wood et al., 2002) The correlation between the two instruments is .40 indicating that it measures a construct related but not identical to R eflective J udgment (King et al., n.d.) Content An alysis Procedures papers for evidence of reflective thinking patterns as defined by the Reflective Judgment on of the principles of scientific research (objectivity, systemasticity, generalizability) to the analysis of (Silverman, 1993) describes the process as involving the esta blishment of categories, and the counting of the number of instances of those categories in a particular item of text.

PAGE 96

84 There are several critical elements involved in using content analysis to examine items of text. One is deciding whether one will use an inductive or deductive approach in the development of categories; another is ensuring that the categories are clearly defined so that other coders will come to the same conclusions when evaluating the same text (Hol sti, 1968) Finally, the unit of analysis and system of enumeration must be determined. These considerations are discussed in the following sections. Development of Content Analysis Themes A deductive approach, which relies on the use of a categorical sch eme suggested by a theoretical perspective (Berg, 1989) guided the development of the content analysis categories for the scaled rubric. However, an inductive approach was used to identify themes observed in the stu used to resolve the ill structured problem in the decision case. The Reflective Judgment Model served as the framework for the construction of rubric domains. Originally, an adaptation of Newma (1995) coding scheme for critical thinking indicators was proposed as a suitable coding scheme. However, in order to maintain theoretical integrity and continuity between the quantitative and qualitative aspects of the study, the researcher decided to construct the content analysis themes more intentionally around the stages of the Reflective Judgment Model Skills and problem solving approaches were identified for each stage of the RJM ba Because the RJM focuses on the epistemic assumptions that support problem solving it

PAGE 97

85 was necessary to adopt a skill based model using the RJM as a theoretical framework, rat her than using the RJ stages directly. Following consultation with the current administrator of the RCI online measure, a template developed by Susan Wolcott for creating a customized critical thinking rubric was adopted. customized developmental critical thinking rubric is available to educators at her website ( Wolcott 2006c) t heory ( Wolcott 2006c) necessary for the effective resolution of ill structured problems. Each step serves as a building block for the more advanced skills r equired in later steps. Although the model has not been validated by empirical research, it has been used by many colleges for critical thinking assessment ( Wolcott 2006a) and has face validity, as it is consisten t with the body of literature regarding the skills necessary for open ended problem solving. Please refer to Appendix I. An additional advantage of using The Steps for Better Thinking is that it parallels the requirements of the case analysis assignment R efer to the Table 4 for a comparison between the model and the assignment description.

PAGE 98

86 Table 4 Comparison between Steps for Better Thinking and Case Analysis Assignment. St eps for Better Thinking Required Components of Case Analyses Step One: Identify the Problem, Relevant Information, and Uncertainties Identify problem and acknowledge reasons for enduring uncertainty and absence of single Introduction Problem Statement : Give a specific and concisely written formulation of the problem to guide analysis and problem solving Step Two: Explore Interpretations and Connections: Interpret information and organize in meaningful ways that encompass problem com plexities Contextual Analysis : Summarize internal and external issues that created or sustain the problem. Step Three: Prioritize Alternatives and Implement Conclusions: After thorough analysis, develop and use reasonable guidelines for prioritizing f actors to consider and choose among solution options Efficiently implement conclusions, involving others as needed Alternative Strategies : Identify three or more possible solutions to the problem Recommendation : Justify your preferred strategy, explaini ng why you selected that particular one, how it best resolves the problem and how you will determine its effectiveness. Step Four: Envision and Direct Strategic Innovation : Acknowledge, explain, and monitor limitations of endorsed solution. Integrate sk ills into ongoing process for generating and using information to guide strategic innovation. [Evaluation :] Determine how you will determine its effectiveness. Based on this description of the case analysis assignment, a coding scheme was developed which included twelve of the twenty two competencies available for educators to choose from in the customized rubric template. The twelve categories comprise eleven app roach to problem solving. Skills that are related to reflective judgment but were not

PAGE 99

87 clearly required of students in the assignment description were not included. For example, identifying and controlling for own biases, and identifying and evaluating key assumptions were not included as they were not required and rarely occurred in student papers when pilot testing the rubric. Reflective Thinking Skills The following skills comprised the themes for the 1 ) I: Identifies and summarizes the problem/ question in the case 2 ) U: Identifies and addressed uncertainties (i.e. reasons why the problem is ill structured); 3 ) R: Identifies information/evidence that is relevant to the problem; 4 ) MP: Integrates multiple perspe ctives in the analysis; 5 ) IN: Qualitatively interprets information and creates a meaningful organization for the analysis; 6 ) E: Identifies and evaluates the implications and consequences of alternatives; 7 ) O: Uses guidelines or principles to judge objectively across the various options; 8 ) S: Clearly presents and supports own conclusions/positions; 9 ) J: Justifies positions with supportive evidence; 10 ) L: Identified the limitations of their position; 11 ) C: Identifies and considers the influence of the context on the issue /problem; 12 ) OA: The overall approach to problem solving.

PAGE 100

88 Rationale. they used for the strategies they chose to solve the dilemma presented by the decision case. Four of the nine them These included intuition (often referred to as instincts), personal/professional experience, personal/professional values, and research. Five other themes were identified based on the content o established belief, previous knowledge, unsupported opinion, and the utility of the solution. These themes were coded dichotomously based on their presence (1) or absence (0) in the stud strategy and each was included in the analysis. No attempts were made to prioritize the rationales that were given as there was rarely evidence that students listed their r easoning strategies in any given order. Unit of A nalysis According to Holsti (1968) the selection of recording and context units should be based upon two criteria: 1) the best fit for the requirements of the re search problem, and 2) efficiency, i.e. which units give satisfactory results with the least expenditure of resources. For example, a study comparing the coding of literature using paragraphs as the coding unit with assigning a single summary score to eac h category revealed little substantive difference in the two procedures (Schneider & Dornbush, 1958 as cited by Holsti, 1969). Based on the forgoing criteria, the unit of analysis for this study was based

PAGE 101

89 on the assignment structure, which included five se ctions. The process for coding these sections is discussed below. The Problem Statement was coded using I (Identification) and U (Uncertainty) to rate the skill used to clearly identified the primary issues in the case and acknowledge uncertainties in the situation. The Contextual Analysis section was coded using the following codes: U for identifying uncertainties, R (Relevance) for identifying relevant elements in the case, MP (Multiple Perspectives) for integrating multiple perspectives in the contextual analysis, and IN (Interpretation) for qualitatively interpreting information and creating a meaningful organization The Alternative Solutions and Recommendation sections were coded using E (Evaluation) for identifying and evaluating the implications and consequences of alternatives, O (Objectivity) for using guidelines or principles to judge objectively across the various options, S (Supports conclusions) for clearly presenting and supporting conclusions, and J (Justification) for justifying positions wi th supportive evidence. The Recommendation section was also coded with an L (Limitations) for identifying limitations in the proposed solution. Two global codes were used that applied to the overall analysis C (Context) was used to rate the students abili ty to integrate contextual elements throughout the analysis. OA (Overall Approach) was used to identify the students overall problem solving approach. System of Enumeration Perform ance Patterns, which range from 0 to 4 and correspond with stages 2 through 7 of reflective stages

PAGE 102

90 2 and 3. Performance Pattern 1 reflects the internal logic of Stage 4, which is the be ginning of quasi reflective thought. Performance Pattern 2 is based on the reasoning strategies typical of Stage 5 and Performance Pattern 3 is supported by the epistemic assumptions characteristic of Stage 6, which is the beginning of true reflective thin king. Pattern 4 corresponds with Stage 7, which is the final stage of the Reflective Judgment Model The characteristics of these performance patterns were defined in the rubric for each of the competencies measured. Please refer to the rubric in Appendix J to see how each was defined. Scoring. Student papers were assigned a score of 0 to 4 for each of the 12 on each of the twelve items. An overall score of 0 indicates the participant approached the problem based on the pre reflective assumptions of Stages 2 and 3. Students using this level of reasoning rely heavily on experts to provide answers, tend to view situations uthoritative sources. In the absence of a clear answer, they will base decisions on their own opinion. The relationship between assertions and evidence is not clear. A score of 1.0 indicates the student is able to acknowledge the existence of enduring unc ertainties, recognize the viability of multiple perspectives, and is beginning to use evidence logically to support conclusions. However, at this level, students tend to use evidence inconsistently, ignore disconfirming information in support of that which supports their own opinions, and have difficulty breaking problems down or understanding multiple perspectives.

PAGE 103

91 A score of 2.0 indicates the participant can logically and qualitatively evaluate evidence from different viewpoints, organize information wel l, identify issues, assumptions, and biases associated with multiple perspectives, and acknowledge and attempt to control personal biases. The primary weakness of thinkers at this level is that in their efforts to present a balanced description of the prob lem, they are unable to establish priorities, or select and defend a single overall solution. A score of 3.0 indicates the student evidenced the strengths of the previous performance patterns, but is also able to prioritize issues and information. After c onsidering all the options, the student is able to articulate well founded support for choosing one solution over other viable options. The conclusion is based on a qualitative evaluation of authoritative positions or situational pragmatics. A score of 4.0 indicates that the student has competencies in all of the previous of long conclusion ( Wolcott 2006a, p. 2 13) Internal Consistency Reliability analysis was used to determine if the 12 items on the rubric were measuring the same construc lpha on the 12 items was .918 for pretest scores indicating a high degree of internal consistency. The item means ranged from .37 to 1.30 with a scale mean of 12.30 ( SD =6.529). Each of the items contributed favorably to the scale mean. Cronb lpha for the items at posttest was .919. The items ranged

PAGE 104

92 from a low of .167 to a high of 1.233 with a scale mean of 11.47. Once again, each item contributed to the overall scale mean. Inter Rater R eliability An important aspect of content analysis is establishing intercoder reliability in order to validate the coding scheme (Lombard, Snyder Duch, & Bracken, 2005) Suggestions regarding sampling of content analysis units for reliability tests vary widely rangi ng from 5% to 25% (Lacy & Riffe, 1996) Based on the small sample, a decision was made to have 25% of the case analyses papers coded by an independent coder. An independent coder was hired with previous experience as a paid rater for the Cognitive Level and Quality of Writing Assessment (CLAQWA), which is an instrument developed to assess college level cognitive and writing skills across the curriculum. A small sample of de identified papers from a previous semester of the capstone course was obtained from the course chair for the purpose of training and pilot testing the instrument. Thinking, and several case analysis papers that had b een previously rated by the researcher Many texts on content analysis suggest that the minimal level of interc oder reliability should reflect the nature and difficulty of the categories and content (Lacy & Riffe, 1996) Based on their relative complexity and difficulty, schemes involving developmental cognitive stages often count ratings that are contiguous as agreement. For example, raters trained in the Cognitive Level and Quality of Writing Assessment (CLAQWA) devel oped by University of Florida to assess cognitive level and writing

PAGE 105

93 across the college curriculum (Flateby & Metzfer, 2005) follow this practice. Trained raters for the RJI counted ratings that were no more than t wo points apart across the three dilemmas as agreement ( Kitchener et al., 1993) Rather than using this practice, which would compromise the ability to analyze the data statistically, the acceptable inter coder reliability rate was set at 70% An initial pilot test of four cases that were not a part of the population to be studied yielded an inter rater reliability of 78%. Based on this result, the researcher proceeded with coding the entire sample. A subsample of 25% of the population was randomly selected (by choosing every fourth case) for the coder to rate. Student case studies. The researcher and the coder each coded stude nt papers independently and then met to discuss them. In order to avoid researcher bias, the researcher and the rater was blind to gender, ethnicity, race, age, experience or time of submission. Disagreements were discussed and codes were revised based on consensus between the two coders. Inter rater agreement was calculated based on the initial independent ratings. each group of case analysis papers. Values exceeding .75 indicat e strong agreement above chance, and values between .40 and .74 indicate moderate level of agreement above chance (Fleiss, 1981) Initial coding of 8 papers for the initial case yielded a Kappa coefficient of .73 a Coders met again to discuss the second case, which yielded considerably different results. Four papers were coded independently on the second set of 12 papers. This time

PAGE 106

94 percentage of agreement was only 54%. A clear pattern emerged however. Although the rater chosen had previous experience rating analytical content, she was not a social worker by profession and this particular case required that students make decisions th at required clinical judgment. Coder agreement on items that involved the problem identification, including uncertainties, context, and multiple perspectives ranged from .75 to 1.0, but those items that involved evaluation of the potential solutions ranged from .25 to .75. The coder had no social work background and therefore was unprepared to evaluate the plausibility of student solutions recalculated on these 5 papers and yielded a kappa of .76 and 91% agreement. Although there were two other decision cases, these represented only 11% of the sample, so it was decided to select all the cases from the three most populated decision cases to maximize all percent agreement on all cases coded before any were eliminated from the calculation, below the generally accepted standard of .70, because the rate of coder disagreement was significantly different on the one ca se that required clinical judgment, the overall level of agreement was closer to the agreement rate when those four cases were removed.

PAGE 107

95 Approach to Data Analysis Data were analyzed using a variety of statistical techniques. Descriptive statistics for RC I pretest scores and age, gender, race, and social work experience were utilized. Data were analyzed to determine whether or not the assumptions of normality, linearity and homeoscedacity were met using the EXPLORE procedure in SPSS. T tests were u sed to d etermine whether the intervention and comparison groups were equivalent at pretest on ratio level variables such as pretest scores and age. Nominal variables such as gender, race and years of experience (treated as categorical variable ) were compa red usin g Chi square procedur es small. The relationship between age and RCI scores was tested using a Pearson Product Moment Correlation. In addition, ANOVA procedures were utilized to assess the influence of categorical values for age, as well as years of experience. RCI pretest scores of foundation year and advanced year students were compared using independent samples t tests In addition to determining equivalency between the groups, t his comp arison was used to determine whether there was measureable change in reflective judgment between first and second year students that could be accounted for by graduate school expe rience and/or maturation. In order to assess whether students in the current sample were comparable to other graduate student populations, a n independent sample t test was used to compare the sample RCI pretest mean to a previously normed mean for graduate students Paired samples t tests were used to determine dif ferences between pre and post test scores on the RCI test for each of the groups. A change score was generated by subtracting pretest

PAGE 108

96 scores from posttest scores. Independent samples t test were used to assess whether there were significant differences between the groups in the amount of change over the course of the semester. Because of the small sample size, a d statistic, which is calculated by subtracting mean scores and dividing the value by the pooled standard deviation (C ohen, 1988) was used to determine effect sizes. A Pearson correlation was used to determine the correlation between RCI pretest scores and RCI change scores. Based on the results of previous tests, a stepwise multiple regression was also conducted to d etermine the best predictor of RCI change. which allows for on each of the six decision cases analyzed. One of the criticisms of content analysis is that conclusions drawn from the activities (Silverman, 2001, p. 123) Therefore, in the current study, extensive use of memos to record observations that supported categorizations or did not clearly fit into one of the categories available was utilized. These memos were used for qualita tive analysis. Although all the student papers were coded in ATLAS, because the content of memos is not viewable when printed, an example of a coded paper in WORD format is included in Appendix K Reliability analysis was used to assess the internal consis tency of the coding rubric. Inter rater agreement between the researcher and an independent

PAGE 109

97 Paired samples t tests were used to compare each of the subcategories as well as the initial and fi nal scores generated by the content analysis procedures. The relationship between rubric scores and the case analyzed as well as the section in which the student was enrolled was also examined using ANOVA procedures Differences on overall rubric scores based on the rationale used for the strategies chosen were assessed using independent samples t tests The Pearson correlation coefficient ( r) was used to measure the degree of association between the coding rubric scores and RCI test results.

PAGE 110

98 Chapte r IV Results The purpose of this study was to evaluate the influence of case method instruction on the reflective judgment of advanced year MSW students enrolled in a capstone course. Reflective judgment was assessed using quantitative and qualitative meth ods. A standardized online measure, The Reasoning about Current Issues Test was used to assess reflective judgment quantitatively In addition, a customized rubric based on the Reflective Judgment Model was used to assess the reflective thinking skills uti lized by students completing an initial and final decision case analysis. This chapter reports the results of the data analysis. Prescreening of Data Prior to data analysis, all data were screened for accuracy of data entry, missing values, and outliers by examining frequency distributions and descriptive statistics using SPSS. Box plots were examined to identify outliers for each variable. Given that parametric analytic techniques were to be used, data were also analyzed for adherence to the assumptions of normality as well as multivariate normality using the EXPLORE procedure in SPSS. K urtosis and ske wness values were within normal ranges for all variables with the exception of gender, which had a negative skew of 3.545, and age, which had a positive skew of 3.027. Kurtosis was significant for gender and age due to

PAGE 111

99 the homogeneity of the group on these variables. A few outliers were identified for age and RCI score. Violations of normality are addressed relative to the group and the specific statistical an alysis procedures conducted. Students were not randomly assigned to the intervention or comparison groups, therefore the demographic characteristics of students in each group were compared to determine whether any significant group differences existed at t he start of the study age (See Table 2 in Chapter 3 for frequencies on demographic variables). The next section describes the results of this analysis. The hypotheses relating to demographic va riables are addressed first, followed by the hypotheses that relate to the RCI, and finally the hypothesis that concerns the content analysis of decision case papers Equality of Groups on Demographic Factors Gender Females accounted for 95% of the overal l sample. Although the majority of MSW students are female (87%) (2006; Council on Social Work Education, 2007a) this sample still contained fewer male participants than wo uld have been expected. The comparison group had a slightly higher percentage of male participants ( n = 3; 5.3%) than the intervention group, which only had one male (2.6%) who completed the study. Both groups reflected a negative skew on gender with a va lue of 6.24 for the intervention group and 2.7 for the comparison group. Kurtosis for the intervention group (39.0) was considerably higher than the comparison group (5.97), reflecting the overwhelming

PAGE 112

100 percentage of female students in this group (97%). H owever, because assessment of skewness and kurtosis values has been reported to be less meaningful when applied to dichotomous variables (Morgan, Leech, Gloecknew, & Barrett, 2004) a 2x2 chi square procedure was p erformed to compare the two groups based on gender, 2 (1, N =58) = 1.804, p because 50% of cells had expected counts less than five. It confirmed that the difference was not signif icant and the effect size was small ( p = .232, V = .178). Race The race/ethnicity variable originally consisted of nine groups based on the categories used by Council on Social Work Education (Co uncil on Social Work Education, 2007b) This included, in alphabetical order: a) African American/Other Black (non Hispanic); b) American Indian/Native American/Alaskan Native; c) Asian American; d) Mexican American; e) Multiple Race/Ethnicity; f) Other Latino/Hispanic; g) Pacific Islander; h) Puerto Rican; and i) Caucasian. Although an Other category was added when a participant indicated that she did not fit into any of the categories created, it was not selected by any of the participants. Because 79% of the cells had expected counts of less than five, these categories were recoded into dichotomous groups, i.e., Minority and Caucasian. A 2x2 chi square indicated no significant differences between groups on race/ethnicity, 2 (1, N = 58) = .351, p = 1.56 8, V confirmed these results, p = .554.

PAGE 113

101 Social Work Experience Social work experience initially included the following five groups: No experience less than three years of experience, three to five years of experience, six to ten years of experience, and eleven years and above Frequencies for these categories are contained in Table 2. Because 63% of cells had expected counts of less than five, experience was recoded into the following three categories: no experience less tha n three years of experience and three or more years of experience Fifty eight percent of participants ( n = 23) in the intervention group had no experience, compared to 78% ( n = 14) in the comparison group. Twenty eight percent ( n = 11) of participants in the intervention group had less than three years of social work experience compared to 11% ( n = 2) in the comparison group. Finally, 15% ( n = 6) of participants in the intervention group had three or more years experience as compared to 11% ( n = 2) in th e comparison group. Contrary to expectations that the intervention group would be less experienced based on the presence of advanced standing students in the population, 44% had some social work experience compared to only 22% in the comparison group. A 2x 3 chi square experience were not statistically significant, although the effect size was moderate, 2 (2, N = 58) = 2.628, p = .269, V = .215. Age The range in age of the intervention group was 23 to 58 with a mean age of 26.8 ( SD = 5.88). The comparison group ranged from 22 to 61 with a mean age of 29.2 and a

PAGE 114

102 standard deviation of 11.52. Although the majority of students in the sample were under 26 years old (68%, n = 26) there was greater variability in the comparison group, resulting in moderate kurtosis in the comparison group (2.88) but substantial kurtosis (21.26 ) in the intervention group. A box test indicated that there were three outliers in the comparison g roup between 1.5 and 2.5 standard deviations above the mean age. The intervention group also had one outlier that was two standard deviations above the mean. Because both groups h ad similar leptokurtic curves and t tests are very robust to violations of no rmality (Montacalm & Royse, 2002; Thode, 2002) an independent samples t test was used to determine whether there were significant age differences between the two groups. ndicated that equal variances could not be assumed, p = .007; therefore, the p value for unequal groups was used. The t test results indicated that the age differences between the two groups were not statistically significant, t ( 21.24) = 1.07 p = 402. The effect size was moderate ( d = 0.27, r = .13 ). In summary, there were no significant differences between the intervention group and the comparison group on any of the demographic variables. This finding serves to diminish concerns regarding sampling er ror based on the lack of random assignment to the intervention and comparison groups, and supports the premise of Campbell and Stanley (Campbell & Stanley, 1963) that naturally occurring groups in educational set tings will be equivalent. The hypotheses and the procedures used to test them are addressed below. The alpha level for all parametric tests was set at .05. The hypotheses related to demographics will be considered first, followed by the hypotheses related to the quantitative and qualitative measures of reflective judgment.

PAGE 115

103 Hypothese s Related to Demographic Factors Based on the findings of previous studies that used the RCI to assess reflective judgment, it was predicted that there would be no relationship between gender or race and RCI scores, but that age and social work experience would have a positive impact. Each hypothesis and the statistical analysis used to test them are addressed in this section. Gender Hypothesis 1.1 Gender will not affect RCI s cores. The results of statistical testing for gender differences in RCI scores reflected a change when pretest scores were compared to the posttest scores. Gender differences observed at pretest were not observed at posttest. Of the 58 participants taking the RCI who completed at least one posttest measure in the study, 95% were female ( n = 55) and 5% were male ( n = 3). The mean score for male participants at pretest ( M = 6.26, SD = .61) was significantly higher than the mean score for female participants w ith a large effect size ( M = 5.22, SD = 66), t (55) = 2.66, p = .01, d = 1.627 ) Based on the imbalance in the number of participants in each group, Mann Whitney U was also used to examine gender differences between the means. The differences were signifi cant, U = 16, Z = 2.32, p = .02, confirming the results of the t test. Because previous studies have not found consistent differences on RCI scores based on gender (King & Kitchener, 2004) but have noted a relati onship between age and reflective judgment level (King & Kitchener, 2004) mean age by gender was

PAGE 116

104 examined to determine if this was a contributing factor to the differences in RCI scores between male and female parti cipants. The mean age for male participants was 38 ( SD = 20.07), while the mean age for female participants was 27 ( SD = 6.87). The differences were not significant based on the unequal variances in the two groups t (2.065) = 2.389, p = 444, but the effect size approached the .8 range which Cohen (1988) defined as large ( d = .73, r = .34 ). Based on r 2 age explained 12% of the variance in RCI scores between genders. At posttest, the mean RCI score for male participa nts regressed by nearly a stage ( .82). Although the mean for male participants was still higher ( M = 5.44, SD = .54) than the mean score females ( M = 5.07, SD = .74), an independent samples t test indicated that the difference was not statistically signif icant, t (51) =.844, p = .402. The effect size was moderate ( d = .57 r = .27 ) A Mann Whitney U confirmed that the age differences were not significant, U = 57.8, Z = .778, p = .467. Race/Ethnicity Hypothesis 1.2 Race/ethnicity will not significantly i nfluence reflective thinking levels at pre or posttest. RCI scores were significantly higher for minority students at pretest, but not at posttest. The dichotomous variable for race was used based on the small numbers of participants within each racial/eth nic category. Forty six Caucasian students and seven minority students had valid pre and posttest RCI scores. Among the minority students, one self identified as African American, one as Asian American, one as multiple race, and four as Hispanics from var ious nationalities. At pretest the mean for minority

PAGE 117

105 students ( M = 5.73, SD = .620) was significantly higher than the mean for Caucasian students ( M = 5.2, SD = .684), t (56) = 2.05, p = 0 d indicated a large effect size of .81. At posttest, t he mean RCI score for minority students decreased substantially ( M = 4.87, SD = 888) and was now lower than that of Caucasian students ( M = 5.12, SD = .710) which also decreased slightly. An independent samples t test indicated that differences between gr oups based on race/ethnicity were not statistically significant, t (51) = .844, p = .403, although minority scores were now one quarter of a stage lower than Caucasian student scores. There was a medium effect size ( d = .31). Because the racial/ethnic d ifferences observed at pretest were inconsistent with posttest results as well as with the findings of previous studies, the data were further examined using post hoc tests to determine if there were any confounding factors contributing to the variance. Th e EXPLORE feature in SPSS was used to determine if there were any violations of normality for race and RCI scores. Skewness and kurtosis values were within normal limits for both groups at pre and posttest. An examination of box plots for pretest scores re vealed that two outliers in the Caucasian group were between 1.5 and 2.5 standard deviations below the mean at pretest, possibly depressing scores. At posttest, there were no outliers in the Caucasian group, but one outlier in the minority group was two st regression. Crosstabs was used to determine whether there was a relationship between race and age categories. The results of a 2x3 Chi square in dicated that the relationship

PAGE 118

106 between race and th e categorical age variable was not significant and the effect size was small, 2 (2, N = 58 ) =.756, p = .685, V =.115 Race and social work experience was also examined. One hundred percent of the minority students had three or less years of social work ex perience compared to 69% of the Caucasian students. The results of a 2x3 Chi square indicated that there was not a significant relationship between race and social work experience, although there was a medium effect size, 2 (2, N = 58 ) = 4.578 p =. 101 V = 283. In summary, no confounding demographic factors were identified that accounted for the unusual differences between groups based on race at pretest, or the regression that occurred at posttest. Age Hypothesis 1.3. Age will significantly influence RCI scores at pre and posttest. a significant correlation between age and RCI scores at pretest, r ( 58) =.103, p = .44. Based on the coefficient of determination ( r 2 = .01), age accounted for 1% of the variance in RCI scores. However, at posttest there was a moderate correlation between age and RCI scores that approached significance at the .05 level, r ( 53) =.269, p = .052. Age accounted for 7% of the variance in posttest sc ores ( r 2 = .07). Because the sample was comprised predominantly of traditionally aged students (22 26), comparisons were also made between age categories using ANOVA procedures.

PAGE 119

107 that the differences between groups were non significant at pretest F (2, 55) =.394, p = .6 77 2 = .04, and posttest, F (2, 50) = 1.097, p = .342, 2 = .02 Table 3 displays the means of participants at pre and posttest by age category and by social work experience. An interesting observation was that students in the youngest age group regressed by one quarter of a stage, while the 27 to 40 and 41 and older groups each had differences of only .02 from their original score. Year s of Social Work Experience Hypothesis 1 .4 significantly influence their reflective thinking levels at pre and posttest. Th practice had less influence than was anticipated on reflective judgment. Because students in the sample were primarily inexperienced, resulting in small numbers in the other cate gories, the variable was recoded into three categories, no experience less than 3 years experience and 3 or more years of experience A one way ANOVA indicated that there were not significant differences on the pretest scores based on years of experience and the effect size was very small, F (2, 55) =.59, p = .558, 2 = .015. The results of an ANOVA for posttest scores indicated that the differences between experience level categories were also not significant, F (2, 50) =.670, p = .516, 2 = .026. Scor es at posttest were lower for all three groups ; however, those in the less than three years group regressed by more than a half a stage ( .585) Table 5 below

PAGE 120

108 depicts the number of participants at each level of experience and their mean scores at pre and p osttest Table 5 RCI Scores by Age and Experience Age Category RCI Pretest RCI Posttest N Mean SD Mean SD 22 26 34 5.27 .722 5.00 .722 27 40 15 5.20 .706 5.18 .765 41+ 4 5.55 .365 5.53 .649 Experience Level None 34 5.25 .692 5.14 .728 >3 years 12 5.43 .860 4.88 .764 <3 year s 7 5.17 .393 5.21 .731 Hypothes is Related to RCI Pr e and Posttest S cores Previous research on the Reflec tive Judgment Model using the Reflective Judgment Interview, which is a qualitative precursor to the standardized RCI, found that early level graduate students had a RJ mean score of 4.6 ( SD = .81) (King & Kitchener, 1994) indicating that they used epistemic assumptions consistent with Stages 4 and 5 of the RJ mode l. Scores on the RCI have been found to be consistently and systematically one stage above the RJ scores (Owen, 2005; Wood & Kadrash, 2002) To date, a mean RCI score for graduate students has not been published. However, an unpublished study by Kitchener and associates (Kitchener, Wood, & Jensen

PAGE 121

109 2002 as cited in Owen, 2005) from 5.5 at the 10 th percentile to 6.2 at the 90 th percentile with a mean of 5.86 across samples. The current sample of MSW students range d from 4.1 at the 10 th percentile to 6.1 at the 90 th percentile with a mean of 5.28. A one sample t test was conducted to compare the current sample to the reported graduate student mean. The results indicated that the mean pretest score (5.28, SD = .697) of students in the current study was significantly lower than the mean score reported above for graduate students, t (52) = 6.147, p = .000. Similar results were obtained when running a one sample t test using the RCI pretest scores for the intervention gr oup alone ( M = 5.26, SD = .742), t (34) = 4.879, p = .000. Although the disparity between the two means may be of some c oncern, a study that assessed the reflective judgment of graduate student counselors using the paper and pencil version of the RCI rep orted a mean score of 5.4, which is much closer to the mean for the current study (Owen, 2005) The sample for that study included 68 doctoral el students with 2.3 years of graduate education. Differences between scores on the paper and pencil version and the current online version have not been reported Nevertheless, the fact that the current study sample is comprised of younger, traditionally aged students including 43% who are advanced standing students may have contributed to the fact that the group mean was closer to the reported mean for college seniors (5.34) than to the graduate student mean (Kitchener, Wood, & Jensen, 2002 as cited in Owen, 2005)

PAGE 122

110 Hypothesis 2 .1 MSW students engaged in a case method course will show a significant increase at posttest on the Reasoning about Current Issues Test (RCI) as compared to their RCI pretest scores. The results of a paired samples t test failed to support this prediction, t (34) = .302, p = .764 d was .055 indicating an insignificant effect size. Thirty five students in the intervention group took both t he pre and posttest. Kurtosis and skewness values indicated that RCI scores at pre and posttest were normally distributed in the sample. The RCI mean score for the pretest ( M = 5.26, SD = .742) was slightly higher (.04) than the posttest ( M = 5.22, SD = .7 5), however the overall scores remained essentially unchanged. A change score was calculated by subtracting the posttest scores from the pretest scores. Although the mean change score was .04 ( SD = .813), there was considerable variability in the sample with a range of 3.16. The majority of participants ( n = 20) had positive change scores. Fifty and posttest with a range of .06 to 1.24. In comparison, 40% of participants had negative change score s ranging from 1.92 to .03. Twenty three percent had posttest scores that decreased more than half of a stage, which is considerable, as it parallels the reported differences between freshmen and senior scores ( Wood & K adrash, 2002) Because of the considerable variability, demographic variables were examined to determine if there were any factors that were related to the inconsistency in the change scores. Race and experience were significantly related to change score s. Minority students were significantly more likely to regress between pre and posttest, t (33) =

PAGE 123

111 3.385, p = .002, d = 1.44 Additionally, a one way ANOVA indicated that there were xperience, F (2, 32 ) = 5.323 p = .010 2 = .25 Scheffe post hoc tests indicated that participants with less than three years experience ( M = .67, SD = 835) regressed while those that had no experience ( M = .184, SD = 83) had positive change scores. Par ticipants with three years or more experience also had positive change scores ( M = .34, SD = 742 ) but the difference between this group and the others was not significant. Age was not significantly related to change scores and gender was essentially consta nt as there was only one male in the intervention group. Hypothesis 2 .2. MSW students engaged in a case method course will demonstrate greater increases in their reflective thinking level than those who are not engaged in a case method course based on the change between pre and post RCI test scores. This hypothesis was not supported. Both groups experienced mean decreases rather than increases in scores; however, participants in the comparison group decreased by nearly one half a stage, ( M = .475, SD =.94) while participants in the intervention group decreased four hundredths of a stage ( M = .042, SD =.81). The results of an independent samples t test based on change scores indicated that the difference between the groups was not statistically significant, t ( 51) =1.74, p = d indicated there was a medium effect size ( d = .49). A paired samples t indicated that post test scores ( M = 4.85, SD = .65) were significantly lower tha n pretest scores ( M = 5.32, SD =. 62), t (17) =.2.14, p d (.75) indicated the effect

PAGE 124

112 size was approaching the criteria for a large effect (.8) (1988) A Pearson correlation was run to determine the r elationship between pretest scores and change. The results indicated that pretest scores and change scores were significantly negatively correlated for all participants, r (52) = .587, p = .000. Students who began with higher pretest scores tended to regr ess at posttest, while those with lower pretest scores increased their scores at posttest. This pattern was also evident when isolating the intervention group r (34) = .539, p = .001. Figure 3 below graphs the mean pre and posttest scores of each group. Figure 3 Change in RCI Scores After examining the variables for assumptions of normality, a stepwise multiple regression was conducted to determine which of the following independent variables were predictors of change in RCI scores: RCI pretest scores age, group membership,

PAGE 125

113 and race (both recoded into dummy variables). Gender was excluded because it failed to meet the assumptions of normality. The regression results indicated that only pretest scores predicted RCI change; all other variables were remo ved from the model, F (1, 51) = 26.83, p = .000, R = .587, R 2 .587. Based on the adjusted R 2 RCI pretest scores accounted for 33% of the variance in RCI change. Hypothesis 2 .3. Gains in the reflective judgment scores of students engaged in a case method course will exceed increases that can be attribute d to maturation or educational experience. This hypothesis could not be tested because there were no significant differences between the RCI scores of foundation year and second year students. Comparison of the intervention group ( n = 39) and the comparis on group ( n = 18) pretest scores did not support the assumption that there would be a measurable maturation effect evidenced by higher RCI scores in the intervention group. In fact, the intervention group scores ( M = 5.25, SD = .74) were slightly lower tha n the comparison group ( M = 5.32, SD = .61). The results of an independent sample t test indicated that the difference was not significant and the effect size was negligible, t (55) = .381, p = .705, d = 0.01. Hypotheses R elated to Content Analysis Proc edures Content analysis procedures were utilized to explore whether reflective thinking analysis submitted. Analysis of the papers generated data that was analyzed us ing quantitative as well as qualitative methods. The quantitative analysis of the papers will be discussed first, followed by the qualitative analysis.

PAGE 126

114 g ( Wolcott 2007) The rubric contained twelve items, which describe a range of competencies related to solving ill structured problems. In order to facilitate the statistical analysis of the data, each of the 12 comp etencies on the rubric was scored on a scale of 0 to 4. Although not identical to the RJ stages, the scale corresponds with the reflective judgment stages in the sense that skills demonstrated on each level are characteristic of the related RJ Stage. Performance in the 0 column is related to stages 2 and 3 of the R eflective J udgment M odel. A score of 1.0 shows evidence of Stage 4 reasoning. A score of 2 .0 indicates the student is using skills characteristic of Stage 5 and a score of 3.0 corre sponds to Stage 6. Although a score of 4 was possible, all students scores in the sample ranged from 0 to 3.0. Mean scores were derived by averaging the scores on the twelve items. For further explanation of the meaning of the five performance levels, plea se refer to the descriptions in Chapter 3. The rubric is included in Appendix J Internal Consistency Reliability analysis was used to determine whether the 12 items on the rubric 85 for pretest scores indicating a high degree of internal consistency. The item means ranged from .47 on Limitations to 1.34 on Evaluation with a scale mean of 13.22 ( SD = 5.62). Each of the items contributed favorably to the scale mean. Although deleting Uncertainty respectively, the difference was not large enough to consider deleting them from the

PAGE 127

115 nged from a low of .19 on Limitations to a high of 1.34 on Justification with a scale mean of 13.03. Once again, each item contributed to the overall scale mean. With the exception of Limitations each item contributed to the Alpha score and deleting Limita tions would have only increased the Alpha to .930. Comparison of Initial and Final Rubric Scores Hypothesis 3.1 MSW students engaged in a case method course will demonstrate increased reflective thinking skills based on their scores on a customized rubric designed to assess problem solving skills related to the resolution of ill structured problems. This hypothesis was not supported by the data. The results of a paired samples t m ean scores on the initial ( M = 1.10, SD = .467) and final case analysis ( M = 1.08, SD = .486), t (31) = 1.16, p = .873, d = .04. A change score was computed by subtracting the initial case scores from the final case scores The mean change score ( .016, SD = .547 ) indicated that students scores remained essentially unchanged from pre to posttest. A categorical change variable was computed to determine the percentage of students whose scores increased versus those whose scores decreased between the initial an d final case analysis. Fourteen students (43.8%) had negative change scores, which ranged from a low of 1.17 to .08. Five students (15.6%) had a score of 0.0 indicating their initial and final scores were identical. Thirteen of the 32 students (40.1%) ha d positive change scores ranging from .08 to 1.0. These results indicate that although there were not significant differences

PAGE 128

116 between group means, there was considerable variability in student performance between pre and posttest. A second categorical cha nge variable was computed to differentiate between scores reflecting meaningful changes versus those that might be trivial. In keeping with patterns observed in empirical research of the Reflective Judgment Model, a change was considered meaningful if it w as one quarter of a stage higher or lower than the initial case score. This measure corresponds to approximately one half of the standard deviation for the change score ( SD = .54) and one half of the reported change between seniors and graduate students (. 52) (Kitchener, Wood, & Jensen, 2002 as cited in Owen, 2005) In addition, the only study to report growth in reflective judgment scores following a one semester educatio nal intervention, found a mean improvement of .296 (Kronholm, 1996) All scores were categorized by the following criteria: those that increased by at least .25 were counted as improved, those with change scores betw een .24 to .24 were counted as staying the same, and those whose scores declined by more than .25 were considered to (31.2%) remained constant, and 12 (37.5%) decreased. A Pearson correlation was computed between mean rubric scores on the initial case and change scores to determine if a pattern similar to the one observed for the RCI scores emerged. In fact, initial scores and change scores were significantly negatively c orrelated, r (31) = .551, p = .001. The results of a one way ANOVA between the categorical change variable for meaningful changes confirmed that students who started with higher scores regressed toward the mean, F (2, 29) = 6.026, p = .006. A Scheffe

PAGE 129

117 proc edure performed to assess pair wise differences between the change groups indicated that the initial scores of participants whose scores decreased by .25 or more were significantly higher ( M = 1.42) than those whose scores increased ( M = .90 ) or those whos e scores stayed the same ( M = .92). Scores on the rubric ranged from .42 to 2.16 on the initial case analysis and .42 to 2.66 on the final case analysis. Forty seven percent of students had ratings that were below 1.0 on the initial case analysis compared to 50% on the final case analysis, solving approaches related to epistemic assumptions consistent with Stage 3 and 4 of the Reflective Judgment Model. Fifty percent of students had scores between 1.0 and 2.0 at pretest compared with 47% at posttest. These scores reflect skills that are supported by the epistemic assumptions of the quasi reflective stages, 4 and 5. One student (3.2%) at pretest and one student (3.2 %) at posttest had scores a bove 2.0, indicating the beginning use of skills in the reflective thinking range. The 15 students (47%) who scored in the Pre Reflective Performance Pattern 0 range on the initial case showed significant increases in their final rubric score. The mean in itial case score for these students was .705 ( SD = .183) and the final was .938 ( SD = .354). The increase of .233 was statistically significant with a large effect size, t (14) = 2.333, p = 035, d =.82. In comparison, students ( n = 16) who scored in the Qu asi Reflective Performance Pattern 1 range regressed from an initial mean score of 1.4 ( SD = .572) to a final mean score of 1.2 ( SD = .299). Only one student scored in Quasi Reflective Performance

PAGE 130

118 f a stage to 1.41. Figure 4 below graphically displays the differences in the pattern of change between the three groups. Frequencies for performance pattern ratings on each of the rubric items are included in Appendi x L F igure 4 Mean Rubric Change by I nitial Case Performance Pattern Level Comparison by individual item scores Two tailed paired samples t tests were used to determine whether there were significant differences on each rubric item between initial case ratings and final case ratings. The results indicated that there were no was significantly lower on the final score. The final scores were slightly lower on 8 of the

PAGE 131

119 12 items for the final case analysis p aper. T able 6 displays the results of each of the t d) indicated that the associations ranged from insignificant to interesting (medium). Table 6 Paired Samples T Te sts on Rubric Items. Although the differences wer e not statistically significant, between pre and (I), identify relevant issues (R), offer inferences and interpret information (IN), and provide justification for their conclusions (J). The largest effect size was for the ability to identify relevant issues. The highest mean score obtained on the initial case was for Item Pre Post M SD t (31) p d Identification 1.22 1.31 .094 .818 .649 .521 .15 Uncertainty 1.06 1.03 .031 .695 .254 .801 .05 Relevance .91 1.22 .312 .965 1.83 .077 .46 Multiple Perspectives .94 .88 .062 1.014 .349 156 .09 Interpretation 1.12 1.19 .062 .759 .466 .645 .11 Evaluation 1.34 1.19 .156 .920 .961 .344 20 Objectivity 1.1 1.12 .031 .999 .177 .868 .04 Supports Conclusions 1.28 1.22 .062 .716 .494 .625 .09 Justification 1.22 1.34 .125 .976 .725 .167 .15 Limitations .47 .19 .281 .683 2.329 .027 40 Context 1.19 1.12 .062 .801 .442 .662 .11 Overall Approach 1.31 1.22 .094 .856 .619 .540 .12 Rubric Mean 1.10 1.09 .02 .548 .161 .873 .02

PAGE 132

120 Evaluation ( M = 1.34, SD = .701) which involved considering the implications and consequences of pro posed solutions. The highest mean score on the final case was for Justification ( M =1.34, SD = .653) which involved justifying positions with supportive evidence. The lowest mean scores on an individual item on both the initial and final case was for ackno wledging limitations ( M = .47, SD = .5; M = .19, SD =.47). The participants rarely used this skill, which demonstrates the ability to deal with and address ambiguity in an ill structured problem. This skill reflects the increased cognitive complexity that is decreased significantly in this category between pre and posttest Comparison by Case. All of the students analyzed the same initial case; however, because students c ould choose five of the last ten cases that they would submit for a grade, the last case submitted varied from Case #7 to Case #12. Case #7 and Case #9 were combined for analysis, because only one student submitted #7 and only two submitted Case #9. Studen ranged from a mean of 1.02 on Case #7 and #9 to 1.13 on Case #10. A one way ANOVA used to determine whether the differences between the scores on the final cases were significant, indicated that they were not and that t he effect size was very small, F (3, 28) =.036, p = .991, 2 = .004. Each of the final cases was also compared to the initial case using two tailed independent samples t tests. Although there wer e no significant differences effect sizes varied from very s mall to moderate. Students who completed case #7 and #9 ( n = 3) had the lowest mean score and the effect size for the decrease between pre and posttest was moderate. Students who submitted Case #11 as their final case ( n = 11), had only slight

PAGE 133

121 decreases in their scores and a very small effect size. In comparison, students submitting Case #10 and Case #12 ( n = 18) increased their scores from pre to posttest and the eff ect sizes were moderate. Table 7 below displays the results of each t test, mean scores, an d effect sizes for each of the final cases submitted. Table 7 Comparison of Initial and Final Scores by Final Case Case n PreM SD Post M SD t df p d #7 The Overcrowded Clinic & #9 Resp onsAbilities (comb) 3 1.11 .254 1.03 .046 .577 2 .622 .44 #10 Homeboy Industries 6 1.14 .515 1.45 .552 1.26 5 .261 .60 Entertainment 11 1.13 .606 1.08 .446 .290 10 .778 .04 #12 Seattle Community Association 12 .895 .431 1.08 .556 1.105 11 .293 .37 Comparison by Rationale Each of the nine rationale categories were coded as dummy variables, with 0 representing the absence of the rationale in the rationale statement and 1 indic ating its use. W hile codes for the rubric were based on the to rationale statements emerged from the data. The nine themes that were evident in n established belief, 3) intuition, 4) personal and /or professional experience, 5) personal values /beliefs 6) previous knowledge, 7) research, 8) an unsupported opinion, and 9) the utility of the solution. Students often cited more than one rationale for their pr oposed solutions making

PAGE 134

122 it impossible to isolate any of the rationale for analysis. Table 8 displays the frequencies of each of the rationale used on the initial and final case analysis. Table 8. Use of Rationale and Rubric Means Rationale Initial Case Final Case f M1 SD f M2 SD Authority 7 (22%) .642 .2 18 0 n/a n/a Facts that Fit Belief 3 (9%) .861 .4 27 4 (12%) 1.04 343 Intuition 3 (9%) 1.22 .673 3 (9%) 1.11 .173 Personal/Prof Exper 13 (41% ) 1. 3 463 12 (37%) .917 .317 Personal Values 12 (37%) 1. 13 509 7 (22%) 1.19 .5 Previous Knowledge 2 (6 %) .708 361 8 (25%) 1.32 .43 Research 8 (25%) 1. 16 342 5 (16%) 1.35 .757 Unsupported Opinion 1 ( 3 %) .666 n/a 1 ( 3 %) 1.25 n/a Utility 3 (9%) 1. 5 3 8 (25%) 1.30 .372 The most frequently used rationale for the recommendation made was personal experience. Forty one percent of students used it as a rationale on the initial case compared to 37% on the final case. This was followed by personal belie f s/values, which was cited by 37 % on the initial case and 22% on the final case. Twenty five percent of students on the initial case indicated that they based their conclusions on research, as compared to only 16% on the final case. Students were more likel y to base their conclusions on the utility of the solution on the final case (25%) than on the initial case (9%).

PAGE 135

123 Independent samples t tests were used to determine if there were significant solving approach (OA) a nd their final mean scores on both variables were compared for the initial case and the final case. Although there were differences observed in mean scores for sever al of the rationale that met the t tests indicated significant differences only for those who used authority or personal experience on the initial case. initial case had significantly lower mean rubric scores than those who did not, t (31) = 3.40, p d (1.7) indicated that students who used authority as a rationale scored in the 5 th percentile. Authority was assigned as a rationale code when students cited an authoritative source, be it a person, organization, or reference as the primary rationale for their proposed solution and did not clearly differentiate between the authority and their own position or indicated that the authoritative position settled the matter conclusively. While the mean score for those who SD = 218), the mean for those who did not was 1.23 ( SD = .438). Significant d ifferences were also noted on Overall Approach to Problem Solving, t (30) = 3.042, p = .005, d = 1.43. The mean OA score for those who used authority as a rational ( n = 7, SD = .535) was .57, while the mean OA score for those who did not ( n = 25, SD = .77) was 1.52.

PAGE 136

124 case, suggesting that students who were inclined to use authority on the initial case used rationales that were more complex on the final case. Although an ind ependent samples t test between initial and final case means for those who used authority on the initial case was not significant t (7) = 1.082, p d indicated the effect size w as approaching large, ( d = .68). Significant differences were observed between those who used p ersonal or professional experience as a rationale on the initial case and those who did not. On the initial case, the mean score for those who used personal expe rience was 1.30 ( SD = .463) compared to .96 ( SD =.427) for those who did not. The difference was significant and the effect size was large, t (30) = 2.183, p = .032, d = .76. Significant differences were also problem solving a pproach, t (30) = 2.313, p = .028, d = .84. In contrast, students who used personal experience on the second case had lower scores (.91, SD = .317) than those who did not (1.18, SD = .456). The difference was not statistically significant, but a moderate e ffect was observed, t (30) = 1.561, p = .129, d = .60. Students were significantly more likely to use previous knowledge as a rationale for their proposed solution on the final case t (31) = 2.252, p = .032, d = .53. While only two students indicated tha t previous knowledge had been the basis for their proposed solution on the initial case, eight used it on the final case. This code was applied when students indicated that theories, previous course work or other specific sources of information were used a s rationale for the methods proposed. This was differentiated

PAGE 137

125 from research based on students own statement that they used research as opposed to information they had previously learned, or by the use of specific citations in reference to their rationale. Although the difference between the mean scores of those who used previous knowledge and the mean score for those who did not was one third of a stage higher (.32) the difference was not significant, t (30) = 1.634, p = .113. The effect size approached Coh size ( d = .69). Students were also more likely to cite the utility of the solution on the final case analysis, although not significantly so. While 9% of students cited the utility of the solution in their rationale for the initial case, 26% used it on the final case. The mean score of those who used utility as a rationale on the final case was 1.35 ( SD = .757) as compared to 1.03 ( SD = .421) for those who did not. Comparison by Section. Differences in student es by section were also examined to determine if there was any variability in the ratings related to the section that that student was enrolled in Several sections had only one student completing the study, so these sections were combined into one The re sults of two one way ANOVAs indicated that there were significant differences on student scores based on section on the final case analysis F (7,24 ) = 2.424 p = .05, 2 = .414 but not on the initial case, F (7,24 ) = .146 p = .993 2 =.041 The effe ct size indicated that the relationship between section and final scores was strong. A Scheffe post hoc procedure was performed to assess pairwise differences among sections ( p =.05). The results indicated that differences between the individual sections were not significant at the .05 level. The inconsistency between the onminbus test and the pairwise comparisons is

PAGE 138

126 likely a result of sampling error (Nichols, 1998) A visual inspection of the section means reveal s that posttest scores for students in Section 10 were 1.06 above the second ranking section. Although the difference is substantial, there were only two students in that section. Crosstabs was also used to explore the relationship between section and mean ingful (=>.25) increases in rubric scores. While 67% of students in section 1 and 100% of students in section 10 experienced a positive change of at least .25, 67% or more of the students in the remaining sections did not increase their scores appreciably. A value of .40 for Lambda indicated evidence of a strong association between section and increased scores, p = .0 21, indicating that knowing the section in which the student was enrolled improved the chances of predicting improvement by 40%. Table 9 displ ays mean scores by section and the percentage of improvement Table 9 Initial and Final Rubric Scores by Section Section n M1 SD M 2 SD % Increase 1 6 1.05 .518 1. 15 .3 70 67% 3 8 1.17 .63 92 4 99 0% 4 4 .972 .673 1.03 .459 33% 5 2 1.0 .118 .9 2 .000 0% 7 5 1.07 .18 1.08 .25 20% 8 3 1.25 .440 83 .4 17 33% 10 2 1.0 .707 2.2 1 648 100% 2, 6 & 9 3 1.0 .440 1.0 8 463 33% Total 3 2 1.10 .467 1.09 486 38%

PAGE 139

127 Correlation between Rubric and RCI Sc ores Hypothesis 3.2 The content analysis ratings will correlate positively with the RCI scores. This hypothesis was not supported. The results of a Pearson correlation between RCI scores at pretest and the initial case analysis was not significant, r (3 1) = .103, p = .580. Similarly, RCI posttest scores and final case analysis ratings were not significant, r (26) = .006, p =.975. Although the scores were not correlated, the mean scores for both measures supported the premise that students were using e pistemic assumptions and skills related to Stage 4 of the Reflective Judgment Model. Based on the observation noted earlier that RCI scores tend to be one stage above RJI scores, a mean score of 5.2 on the RCI is equivalent to a 4.2 on the RJI. The adjuste d score indicates that the average student in the intervention group was functioning at Stage 4 of the RJ Model. Similarly, a mean score of 1.0 on the rubric indicates that student were primarily using problem solving skills related to the epistemic assump tions characteristic of Stage 4. Additionally, student scores between pre and posttest stayed essentially constant on both measures. A comparison of individual student scores adjusted for the purpose of comparison revealed that 44% of student scores on e ach measure were within half a sta ge of each other; 25% had RCI scores that were at least one half stage higher than the rubric; and 17% had rubric scores that were .5 or higher than the RCI scores Figure 5 depicts the relationship between the adjusted p re and posttest mean scores for each measure.

PAGE 140

128 Figure 5 Relationship between Adjusted RCI and Rubric Scores Qualitative Analysis This section will begin with a brief description of each of the decision cases in order to provide a context for the subsequent description of coding and student performance patterns. Following this section, Performance Patt erns 0 through 3 are described and e that correspond to each pattern are provided. The state m ents were chosen based on their representativeness. Because all but one of the participants were female pronouns Case Descriptions and Observations Students who participated in the study submi tted s ix different decision case analyses, which were analyzed using content analysis procedures. While all of the

PAGE 141

129 participants completed the initial case, the number of students completing each of the subsequent cases ranged from one to twelve students. T he decision cases are consistent in the review of the literature (Jones, 2003) Each narrative is rich with details, providing opportunities for students to determine w hat is most relevant, to acknowledge uncertainties, test assumptions, examine their own biases, and support conclusions with evidence within the case. The cases do not suggest an obvious solution to the prob lem and allow for multiple levels of analysis and inter pretation (Levin, 1995) Three of the decision cases raised issues regarding quality of life, the right to s elf determination, and the responsibility to protect vulnerable populations whose rights are limited Three others explored o rganizational lea dership issues related to the Decision Case #1. The initial case, Unusual Appeal by Rachel Parker and Terry A. Wolfer, 1 involves a social worker who is a mitigation investigator for a nonprofit la w firm that represents inmates on death row. The case concerns a Hispanic male diagnosed with paranoid schizophrenia, who does not wish to appeal his death sentence because of the quality of his life a s a prisoner on death row. The law firm does not believ e he is com petent to make the decision to refuse the appeal but given his apparent lucidity and quality of life, the social worker deliberates between her responsibility to uphold the 1 Parker, R. C., & Wolfer, T. A. (2008). Unusual appeal. In T. A. Wolfer & V. M. Runnion, Dying, death, and bereavement in social work practice: Decision ca ses for advanced practice (pp. 88 97). New York: Columbia University Press.

PAGE 142

130 determination and the mission of the agency to o ppose the death penalty. This case appears to be qualitatively different from subsequent cases in that it presents an obvious ethical dilemma, while the ethical dilemmas presented in each of the final cases is more subtle. It is a very compelling case, re quiring students to develop strategies to decide between the competing values of life and self determination in the context of pressing concerns. In addition, the case raises questions regarding adherence to social work values while working in secondary ho st settings. Although the mean score on the initial case was slightly higher than on the final cases, ( M = 1.0) the range of student scores suggests significant variability (.42 to 2.17) in performance. Students demonstrated evidence of a mixture of pre ref lective and quasi reflective epistemic assumptions. Forty seven percent of students ( n = 15) scored in the Performance Pattern 0 range (.42 to .91); 50% ( n = 16) scored in the Performance Pattern 1 range (1.0 to 1.91) and 3% ( n = 1) scored in Performance P attern 2 range (2.0 to 2.16). Students were more likely to use pre reflective thinking strategies to frame the problem when analyzing this case, such as framing the problem dichotomously, or missing the ethical dilemma presented by focusing primarily on th e inhumane treatment of the client ( n =10) rather than the ethical dilemma posed. Fifty eight percent of students framed the problem in terms of an interpersonal conflict between the characters in the case This approach is characteristic of Quasi Reflectiv e Judgment Stage 4, in which individuals are likely to perceive that differences of opinion results from the idiosyncrasies of the parties involved. Students were more likely to express perplexity,

PAGE 143

131 acknowledge uncertainties, and admit to limitations in the ir proposed solution on this case than on subsequent cases. Decision Case #7. The Overcrowded Clinic 2 involves a non profit family planning organization in a third world country with very limited resources, organizational planning problems leading to dimi nished service provision, and value issues related to cultural competency. This case, which was the earliest final case submitted, was only completed by one student, so no observations can be made regarding patterns in student performance Decision Case # 9 Responsibilities 3 focuses on end of life issues in the context of services to a terminally ill patient An initiative designed to extend hospice services to clients with no primary caregivers provokes anxiety amo ng staff members when a right t o self determination conflicts with staff concerns regarding her health and safety. The problem is posed from the perspective of a supervisor, who is struggling with how to deal with staff anxiety related to innovation, possible counter transference issues disagreement regarding priorities on a multidisciplinary team, and conflict between client and worker values. Only two of the thirty two students submitting a final case analysis chos e to submit this case Scores obtained on this case were the lowest of all the cases ( M =.1.0, 2 Strachan, D. (1977). The overcrowded clinic. Retrieved January 3, 2008, from The Electronic Hallway Web site: https://hallway.org/in dex.php 3 Cearley, S., & Runnion, V. M. (2008). ResponsAbilities. In T. A. Wolfer & V. M. Runnion, Dying, death, and bereavement in social work practice: Decision cases for advanced practice (pp. 40 48). New York: Columbia University Press.

PAGE 144

132 SD = 00) scoring in the pre reflective range of Performance Pattern 0. Although it is difficult to draw any conclusions based on the performance of two students, several factors may have had an impact on low scores. First, this case w as one of the earliest submitted. Secondly, the ethical dilemma reflects the perspective of the supervisor. Because students were primarily young and had no social work experience, they may have had difficulty relating to the supervisory dilemma, as oppose d to the issues faced by the young worker Decision Case #10 Homeboy Industries: An I ncubator of Hope and Business 4 concerns the conflicting values related to organizational and financial solvency vs. commitment to the dustries is an umbrella organization that has established a number of businesses in order to employ former gang members in East Los Angeles. In the face of its visio the g and limited reso urces, the operations director must make decisions regarding expansion opportunities that have potential to increase desperately need ed gang activity. The range of performance for studen ts who completed this case ( n = 6) was broad, from .42 at the bottom to 1.91 at the top. The mean score ( M = 1.14, SD =.515) reflected an increase between pre and posttest. Thirty three percent ( n = 2) of students scored in Performance Pattern 0 range (.42 to .83); 67% ( n = 4) scored in Performance Pattern 1 (1.0 to 1.91). 4 Choi, D. Y ., & Kiesner, F. (2007). Homeboy Industries: An incubator of hope and business. Entrepreneurship Theory and Practice 31(5), 1 22.

PAGE 145

133 The ethical dilemma in this case was more subtle, and s tudents completing this case most often framed it in terms of a simple management problem. Only two of the six students mentioned th e potential conflict between expansion opportunities and the Decision Case #11 I will not ntertainment 5 focuses on the challenge of making sound practice decisions in the face of incomplete information, with self rception of client issues. Students completing thi s case were required to make a decision re garding best practice with a depressed teenaged victim of Traumatic Brain Injury who is struggling to cope with the aftermath of a tragic accident, and its impact on his independence, family relationships, support systems, self concept and spirituality. The TBI, which creates temporary uncertainty. Enduring uncertainties depicted in this case call for students to examine their own assumptions and biases regarding spirituality, existential concerns, the limits of self determination when working with individuals whose freedom is constrained and the necessity of making practice decisions in the face of uncertainty Eleven students s ubmitted this case as their final case analysis. The range on rubric scores was broad, ranging from a low of 66 to a high of 2.66, with a mean of 1.08 Seventy three percent of students ( n = 8) scored below 1.0 on the final rubric score, 5 Sherr, M. E, & Wolfer, T. A. (2002). I will not be God's entertainment. In T. L. Scales, T A. Wolfer, D. A. Sherwood, D. R Garland, B. Hugen, & S. Pittman, S. (Eds.), Spirituality and religion in social work: A source book of decision cases (pp. 106 110). Alexandria, VA: Council on Social Work Education.

PAGE 146

134 compared to 18% ( n = 2) who scored between 1.0 and 2.0, and 9% ( n = 1) who scored above 2.0. Fifty percent ( n= 5) of the students analyzing this case focused on the enduring uncertainties pres ented in the case Decision Case #12 Seattle Community Association 6 was the final case assigned. Twelve participants completed this decision case, which involved the management of a large non profit organization experiencing low morale and conflict follo wing the establishment of an anti racism initiative designed to address institutional racism The executive director who is committed to the initiative, is considering how to respond to widespread frustration expressed by staff toward agency lea dership. Student performance on this case ranged from a low of .5 to 1.83, with a mean of 1.07 ( SD = .485). Fifty percent ( n = 6) of and 50% were in Performance Pattern 1. Students who had final rubric scores between 0 and 1 tended to over identify with staff concerns, framing the problem in terms of black and white with the supervisor as the antagonist. Students who attempted to consider the perspectives of each of the parties tended to use more complex reasoning str ategies resulting in higher scores. Reflective Thinking Performance Patterns The majority of students demonstrated skills consistent with Quasi Reflective Performance Pattern 1, with evidence of a variety of weaknesses and strengths leading to 6 Puckett, G., & Dobel, J. P. (n.d.). Seattle Community Association: U ndoing institutional racism. Retrieved January 3, 2008, from The Electronic Hallway Web site: https://hallway.org/index.php

PAGE 147

135 a mixture of scores on individual items ranging from 0 to 3. Fifty seven percent of ratings assigned on individual items were in the Quasi Reflective Performance Pattern 1 range, compared to 18% in Pre Reflective Performance Pattern 0, 22% in Quasi Reflective Performa nce Pattern 2, and only 3% in Reflective Performance Pattern 3. Performance Pattern 4 was not observed in the sample. This skill level, which is associated with Reflective Judgment Stage 7 has been rare in previous uses of the Step for Better Thinking rubr ic ( Wolcott 2006a) as well as the Reflective Judgment Interview and the RCI (King & Kitchener, 1994; King & Kitchener, 2004) It should be noted that R eflective Performance Pattern 4 involves preparing strategies for the ongoing construction of knowledge. Although students were required to include a method for evaluating their solutions, the assignment did not call for to students to address how they wou ld use evaluative measures to contribute to further knowledge about the issue (Refer to Table 4 ). The following section describes the characteristics of problem solving skills exemplified by Performance Pattern 0 through Performance Pattern 3. Each is org anized by the sections of the assignment, which was the unit of analysis used in the content analysis procedures. Student statements were selected based on their representativeness of the performance level and of other student responses at that level. Stud have not been edited for grammatical errors or inconsistencies. Some responses have been shortened in the interest of brevity (signified by the use of ellipsis points), but every effort has been made to maintain the integrity of the origin al statements.

PAGE 148

136 Pre Reflective Performance Pattern 0 The primary problem solving approach of individuals using Performance Pattern 0 skills to solve an ill ( Wolcott 2006a) of reflective judgment, this approach is based on a lack of sophistication in the ability to understand abstractions and deal with ambiguity ( Kitchener & Fischer, 1990) The underlying epistemic assumptions are that knowledge is certain, and is based upon the these individuals believe that although the answer may be temporarily uncertain, it will eventually be known by the experts. Until that time, individual conclusions must be reached based on personal beliefs. This failure to recognize the inherent uncertainty in the ill stru ctured problem or the need to evaluate evidence to resolve the uncertainty, results in inappropriate and overly simplistic problem solving strategies. Although this overall approach was relatively rare (9%, n = 6) and primarily occurred on the initial case ( n = 5) m any students used a variety of Pattern 0 and P attern 1 skills, indicating that they were functioning between the two stages. As a result, there is considerable overlap betwee n these adjacent levels evident in student statements. For example, stu dents may have identified the primary problem in the case (I1), but only acknowledged temporary uncertainty (U0). Each of the competencies assessed by the individual rubric items contributes to the overall problem solving approach. The codes that were use d at each performance pattern level for each aspect of the case analysis are listed. The percentage of cases which

PAGE 149

137 were assigned th e identified code on the initial (O 1 ) and final (O 2 ) case follow each code description. Problem Statement Two codes were assigned to this section of the assignment. I0: Does not identify the main pr (O 1 = 53%; O 2 = 62% ) U0: Ignores uncertainty, or attributes uncertainty to temporary lack of information or to own lack of knowledge (O 1 = 13% ; O 2 = 3% ) The majo rity of students were at Quasi Reflective Performance Pattern 1 or above on Problem Identification which is associated with the first level in the Steps for Better Thinking Model. The code I0 was most often assigned to case analyses in which the student seemed to miss the primary dilemma presented by the decision case For example, in Unusual Appeal, four students identified the primary problem as the inhumane treatment experienced by the prisoner, while ignoring the ethical dilemma regarding the death sentence appeal faced by the social worker and the question of his mental competence to exercise the right to refuse the appeal. One student identified the current conditions have caused him to believe that it would be better to die rather than appeal the This problem statement not only fails to identify the ethical dilemma faced by the professional social worker in the case, but also assu innocence are factual, although absolutely no evidence is presented to warrant the claim

PAGE 150

138 in the decision case. (A surprising, and disconcerting finding was that only one of thirty two students raised this as an issue in her analysis). Performance Pattern 0 skills used in problem identification were also characterized by presenting the problem as if it were well structured and could be resolved with certainty (U0) For example, one student framed the problem in the ini tial case in this way: competen t decision regarding his appeal (I0; U0). This student frames the problem from the perspective of the prisoner with an underlying assumption that once his quality of life improves the ethical dilemma will be resolved Contextual Analysis The f ailure to identify relevant contextual factors (R0), and to acknowledge the viability of multiple perspectives (MP0) as well as the tendency to simply describe the e lements of the case without offering any inferences regarding the meaning of the facts (IN0 ) resulted in the assignment of codes in the pre reflective pattern range The following t hree codes were applied to the contextual analysis when primarily used pre reflective strategies to address the context: R0: Identifies at least some information that is relevant to the problem (O 1 = 25%; O 2 = 16% ) MP 0 : Describes information without acknow ledging multiple perspectives or portrays perspecti ves and information dichotomously, e.g. good/bad, right/wrong (O 1 = 22%; O 2 = 28% ) IN0: Describes rather than interpret s information; or may use contradictory or illogical arguments; lacks organization (O 1 = 9%; O 2 = 13% )

PAGE 151

139 The following statement from an analysis of Homeboy Industries reveals this level of analysis. Father Boyle has helped so many youth who were either at risk or ex gang members In 2003, he was diagnosed with leukemia Many of the people he had helped came to visit, offering anything the y could do to help However, Father Boyle did not seem affected by the news He was still energetic and in shape and instead of worrying about his health, was worried about Homeboy Industries Homeboy Silkscreen was made by Ruben Rodriguez who felt that h e owed his changed life to the kindness of Father Boyle His wife h ad experience in silk However, finding a place for the silkscreen business would be tough because several of the sites were within gang territories Therefore, the former gang m embers would risk their lives just getting to work Also, one of the major customers of the silk screening was a radio station aimed at teenagers They ordered t shirts and advertised for free Therefore, the teenage population in LA was already targeted f or wearing Homeboy Industries gear Homeboy Industries has the support of many people including famous stars such as Martin Sheen, Angelica Huston, and Kirk Douglass (R0; IN0 :MP0 ). In this contextual analysis, the student reports extraneous details rela ted to the case without offering interpretations regarding the meaningfulness of the information as

PAGE 152

140 it relates to resolution of the pr oblem. She fails to address relevant issues, acknowledge multiple perspectives or link the facts presented to the proposed solutions Alternative Solutions and Recommendation. An expectation for this section of the assignment was that based on the contextual analysis, students would propose at least three distinct viable strategies for resolving the dilemma and discuss the p ros and cons of each before reaching a decision regarding the best alternative. The following codes in the Pre Reflective Performance Pattern 0 were used for this unit of analy sis : E0: prov ided or does not address implications or consequences beyond dichotomous characterizations (O 1 = 13%; O 2 = 16% ) O0: Fails to reason logically from pros/cons to recommendation or conclusion; relies primarily on unexamined prior beliefs (O 1 = 28%; O 2 = 22% ) S0: as conclusions instead of own conclusion (O 1 = 3%; O 2 = 0% ) J0: Based on authoritative source OR where absolute answers are not available on an unsupported opinion. (O 1 = 25%; O 2 = 6% ) L0: Does not acknowledge significant limitations beyond temporary uncertainty (O 1 = 53%; O 2 = 84% ) effectively in their presentation of the proposed solutions. S0 and J0 relate to the solution has limitations. Student papers coded with an E0 most commonly used faulty or

PAGE 153

141 inconsistent logic as they presented the disadvant ages of the va rious alternatives S tudents who were unable to realistically assess the disadvantages of the solutions they proposed, appeared to suggest disadvantages whimsically that were not clearly related to the proposal. In her analysis of the case I the following student suggests cons that do not logically follow from the implementation of her proposed solution One strategy is for Noah to continue the session by completing a full suicide assessment to determine the r isk of the Gregory harming his self or others. Throughout the rest of the session, Noah should establish rapport with Gregory so that he will be able to educate Noah on traumatic brain injury and the impact it has on his life. A pro of this alternative is that by will be better able to proceed with treatment. A con of this strategy is that if Noah does not examine his own spiritual beliefs, they may interfere with the most appropriate determination (E0). Note that the student does not explain why completing a suicide assessment or beliefs. Neithe r does she indicate how either of these strategies, which under normal self dete rmination.

PAGE 154

142 In the following narrative, the student uses faulty logic to conclude that treating the client who suffers from TBI and depression in the same way others are treated will have therapeutic effects on the client and convince the social worker tha t he has the knowledge necessary to proceed: based on the referral paperwork. By completing the assessment as usual, and can then decide if a depression screening is necessary. Gregory is accustomed to society treating him differently, if Noah can demonstrate e will build rapport and a healthy working relationship (L0) A drawback to this option is the time it takes Gregory to respond to ble disadvantage is because he is too consu med by his physical form (E0 ). a number of flawed assumptions. The student assumes that Gregory knows how Noah proceeds with other clients and will recognize th e similarity which will facilitate trust. She also assumes that following this strategy will assure Noah of his skills for dealing with the situation, in spite of the fact that Noah has no previou s knowledge or experience working with this population In

PAGE 155

143 addition, t he disadvantages she lists such as the challenge of working with a client who uses a voice machine are common to all the other strategies she proposes, and do not logically relate to this strategy in particular conclusion is evident as she adopts this strategy without warranting her claims, or addre ssing uncertainties or the limitations that she has suggested exist. Gregory needs Noah to treat him like any other client he would see. The rest of the world has always treated Gregory differently, and all he wants will feel understood and in control of the session (O0; L0: J0). The statement above is assigned a code of J0 because the student used an unsupported opinion to warrant her solution and L0 because she does not acknowledge any limitations. A code of J0 also applied when students justified their proposals based on unevaluated authoritative sources Students who justified their conclusions on the basis of authoritative sources if possible, and on their own unsupported opinions if not, used problem solving st rategies consistent with Stage 3 of the Reflective Judgment Model (King & Kitchener, 1994) A related code, S0, relates to the use of facts, he source from (on the initial case) as the majority of students who used authoritative s ources were able to differ e ntiate between their own opinion and the source.

PAGE 156

144 While many students cited t he NASW Code of Ethics as an authoritative source on the initial case, a code of J0 was assigned when students applied the Code of Ethics as justification simplistically, assuming that citing the code settled the matter conclusively. Students were most lik ely to use this strategy to support their conclusions in the analysis of Unusual Appeal. Twenty five percent of students analyzing this case supported their rationale with a simplistic interpretation of the NASW code of ethics, unequivocally equating advoc self determination with ethical practice. These students ignored the limitations to self determination su ggested by the code, as well as evidence within the case that suggested a need to determine whether the best interests of the cl ient were served by promoting his self determination, given the questions regarding his decision making competency. Only two analyses that used the code of ethics to support their proposals balanced the mandate to support self determination with the direc tive to limit those rights when a client poses a threat to themselves or others. Surprisingly, one student who cited this limitation in her contextual analysis reverted (without providing justification) to the more simplistic perspective when proposing the following solution: The first solution for Cynthia would be to follow the social work Code of Ethics By following the Code of Ethics, she is staying within professional guidelines and removing herself from criticism as to whether the decision she made wa s in the best interest of the client. If she does not follow the Code of Ethics, she could potentially harm the client, lose her licensure, and face professional and personal humiliation (J0; L0)

PAGE 157

145 In this example, the student uses the Code of Ethics as an authoritative source that eliminates all uncertainty regarding the right thing to do. Her interpretation of following the Code of Ethics demonstrates her belief that ther e is a single correct solution. Consequently, s he fails to consider the competing val ues articulated in the code and the need to determine which aspect of the code supports the best interest of the client A pervasive rating in Performance Pattern 0 was L0, which indicated a failure to acknowledge significant limitations beyond temporary uncertainty Although the assignment required students to address the pros and cons of their potential solutions, forcing them to think through the limitations of their proposals, they rarely addressed those limitations once they adopted the proposal as t they tended to focus on the advantages of the solution and ignore all evidence regarding its limitations In the selection below from a case analysis of Entertainment the student suggests that a draw back to a referral to a TBI support group is that Gregory might feel stigmatized: Another solution would be for Noah to recommend Gregory to a TBI support group This treatment approach will connect Gregory with people that are like him where he can build relationships This group could also provide the counseling and care that Gregory needs, as well as the chance to do something without his mother, creating independence A disadvantage of this is that Gregory might not like this group because it might make him feel like he is disabled and stigmatized like people with mental retardation ( E1)

PAGE 158

146 In adopting this proposal as her recommended solution, the student does not address this limitation. She appears to ignore any negative considerations, including the po ssibility pressing issues presented. My preferred strategy is for Noah to refer Gregory to a support group for people diagnosed with TBI A major theme throughout Grego issues is that he does not have support beyond his mother Not have support is difficult for everyone, especially a teenager boy who is trying to break away from his mother Gregory seems to be giving up on life as indicated through his refusal to eat his suicidal thoughts, and his withdrawal from church and his mother Gregory could benefit from finding support through a group of people that understand his feelings of anger and of all that he has been through The fact that a significant majority o f participants failed to acknowledge limitations, even among higher functioning students, may indicate that this is related in part to the academic culture, which discourages the acknowledgement of weaknesses or limitations in problem solving. Nevertheless the tendency to ignore disconfirming evidence was a common pattern observed in student proposals. It appeared that many students related to the listing of cons superficially, but did not consider their relevance when adopting solutions.

PAGE 159

147 Global Ratings T wo global ratings were assigned to each paper, one for incorporating important contextual considerations into the analysis, and another for the overall approach to problem resolution. C0: Does not address context beyond dichotomous characterizations such as right/wrong, good/bad, or smart/stupid (O 1 = 9%; O 2 = 9% ) OA0: answer to open ended questions/problems (O 1 = 5%; O 2 = 0% ) Because these codes were applied to the entire case analysis, examples are not provided. Stud ents wi th ratings of 0 for Context tended to describe contextual factors dichotomously or alternately, ignored the contextual factors listed when considering alternative solutions. A rating of OA0 indicated that the student approached the problem as if it were well structured and had one correct solution. For example, one student who used the Code of Ethics as her rationale in her analysis of Unusual Appeal, stated, determination is the only solution in which the best i The fact that this rating did not occur on the final papers supports the premise that students in the Pre Reflective Pattern 0 demonstrated improved problem solving strategies by the end of the semester. Quasi Reflective Performance Pattern 1 Based on the results of the content analysis, the majority of students were functioning in the Performance Pattern 1 range, which is associated with Stage 4 of the Reflective Judgment Model. Stage 4 re presents a significant progression from Stage 3 in the resolution of ill structured problems. At this stage, students understand uncertainty as

PAGE 160

148 a category of knowledge for which absolute answers do not exist, and begin to use evidence to justify conclusion s (B. K. Hofer & Pintrich, 2002; B. K. Hofer & Pintrich, 1997; King & Kitchener, 2004) However, because the relationship between evidence and justification remains a mbiguous, evidence is used inconsistently. Anecdotal evidence may be offered or the evidence may appear incomplete rather than linked to a coherent argument. Students using Quasi Reflective Performance Pattern 1 skills tended to choose evidence that confir med prior beliefs. This performance pattern was characterized by a problem solving (1997) Problem Stat ement Students who used Quasi Reflective Performance Pattern 1 in the problem statement were able to identify the primary issues in the case, and recognized that there was not an absolutely correct solution to the problem. The following codes were applied to the problem statement when students used identification strategies at Quasi Reflective Performance Pattern 1; I1 : Identifies the main problem (or what might reasonably be considered to be the main problem); but does not identify subsidiary, embedded, or implicit aspects of the problem (O 1 = 53% ; O 2 = 62 % ) U1 Identifies at least one reason for significant and permanent uncertainty, but does not integrate uncertainties into analysis (O 1 = 68%; O 2 = 68% ) The student statement below identifies the basic problem and also frames the problem as an ill structured rather than well structured problem (U1). For example:

PAGE 161

149 the decisions and opinions of her co workers, and superiors, Diane and Joe disagreement about the case, considering the best interest of the client and the fact that Diane has used the force of threat to make her comply (I1; U1). Although at Pre Refle ctive Performance Pattern 0 students assumed that one perspective was correct and the other incorrect, students who used Performance Pattern 1 skills acknowledged that there were multiple perspectives and that contextual factors must be considered in analy zing the dilemma. However, because diversity of perspectives was viewed as resulting from differences in the personal of professional characteristics of the various parties ( such as which perspective was most plausible was not objectively explored ( Wolcott 2006a) While students acknowledged that multiple perspectives existed regarding th e case dilemma, they focused on the perspective most similar to their ow n, rather than comparing and contrasting the evidence in support of each one. Contextual Analysis The following codes were assigned to student papers that demonstrated Quasi Reflective Performance Pattern 1 skills in their contextual analysis: R1:Identif ies most of the information that is relevant to the problem (O 1 = 59%; O 2 = 47% ) MP1: Acknowledges more than one potential viewpoint, approach or perspective (O 1 = 63%; O 2 = 5% ) ;

PAGE 162

150 IN1: Interprets information superficially as either supporting or not sup porting a point of view; ignores relevant information that disagrees with own position; fails to sufficiently break down the problem (O 1 = 69%; O 2 = 57% ) Students at Quasi Reflective Performance Pattern 1 identified most of the information that was relev example, in analyzing Unusual Appeal they were able to identify most of the following medication, inhumane treatment, an d unusual beliefs regarding the aftermath of his death ; as well as the Cynthia and her superior. While students at pre reflective level relayed facts rather than offering inferences o r interpretations, students at the first quasi reflective level began to offer some interpretations regarding the facts in the case. However, the tendency was to interpret information superficially as either supporting or not supporting a point of view (MP1) They often ignored relevant information that disagreed with their own position, or failed to break the problem down sufficiently (IN1). The contextual analysis below reflects many of these patterns. The student addresses a number of releva nt factors and offers a few interpretations of the issues. However, overall, the analysis is superficial and does not clearly link the relevant factors vulnerable mentally ill person whose competence to make life and death decisions is

PAGE 163

151 uncertain, or to support his right to self determination. Although the student acknowledges that there are multiple perspectives, she primarily focuses on the dete rmination is paramount and chooses facts from the case that support that point of view. Information that would support the premise that Jose may memos are included in bra ckets in the student statement below. One internal issue is between Diane Epps and Cynthia Sanders. Cynthia disagrees with Diane about not letting Jose Aranda waive his appeal. (MP1) She feels that even though he is diagnosed with paranoid schizophrenia he understands, during medicated and unmedicated states, that he is being treated inhumanely and wants to remain on death row [Alludes to schizophrenia, medication issues, inhumane treatment, and mental competence; claim regarding evidence] Diane let Cynthia know that she signs her pay checks and Cynthia needed to agree with her [power differential; no interpretation]. Another problem is that Jose struggles with himself knowing t hat his quality of life is poor. [quality of life] they will not pay for Jose to be medicated on a regular basis They say that it is too expensive; because of this Jose suffers with h allucinations and delusions Furthermore, right before the competency trial the prison guards did not adhere to

PAGE 164

152 issue is that the agency believes in advocating for prisoners to get off death row However, Jose does not want to appeal and the agency is saying that he is not competent to make that decision They want to offer their services against his will. (R1; IN1 ; MP1) Because students who use the epistemic assumptions of stage 4 have come to understand that there are areas in which knowledge is uncertain, they often argue that others have a right to their opinions, without regard to the plausibility or credibilit y of those beliefs (King et al., 1990; Perry, 1970) The internal logic is that because knowledge cannot be ascertained with certainty, any judgment regarding the evidence is peculiar to the in dividual. For example, in spite of the fact that Jose believed his execution to be an act of heroism that would usher in world peace and prosperity, and immortalize him as a Mayan rain god, a number of students defended his opinions as legitimate and ratio nal. We will often differ on religious and spiritual beliefs, but it is important to to his mental illn ess, but that does not mean that the belief itself is any less valid. No person, no matter how educated, knows definitely what happens to us when we die, so each and every perspective is equally valid (MP1:IN1). e inscrutable, some students argued that Rather than

PAGE 165

153 tudents who took this approach t ended to reframe his beliefs in more acceptabl e terms For example: sign of mental illness and incompetence when, in fact, it is what he believes as the after life and is, therefore, very rational and not substantial evidence against his right to waive his appeal. (MP1; IN1) Alternative Solutions and Recommendation The primary characteristic of case analysis at Performance Pattern 1 for this section was the tendency to limit the discussion spective. The following codes applied to this performance pattern level: E1: Considers implications and consequences only superficially; ignores negative consequences of own position (O 1 = 41%; O 2 = 53% ) O1: Provides arguments in favor of recommended opti on, and provides little or no opposing argument; uses superficially understood evidence and information in support of conclusions (O 1 = 48%; O 2 = 45% ) S1: Clearly states conclusions and reasons, but limited to supporting primarily one perspective (O 1 = 71 %; O 2 = 80% ) J1:Based on facts, evidence that fits an established belief or own perspective (O 1 = 53%; O 2 = 62% ) L1: Acknowledges at least one limitation or reason for significant and enduring uncertainty (O 1 = 53%; O 2 = 62% ) ;

PAGE 166

154 While the majority of stu determination in analyzing Unusual Appeal the following student provides an articulate, but one sided, to the contrary. Wh ile she provides alternatives and cites disadvantages, in each instance she provides the strongest support for her own perspective. The student appears to begin with the conclusion based on previous beliefs regarding capital punishment and stack up evidenc e to support it (OA1). Note that the language used to address disadvantages of the To resolve this impasse, [Cynthia] Sanders could concede her stand and align herself to [Diana] Epps to pr ovide a united front in an appeal for [Jose] life issues and euthanasia is still very illegal, there should be no double standards when it comes age of giving [the] individual virtually no freedom to volunteer for execution, this choice would sacrifice individual self determination for a higher ideal. Finally, a third solution would involve a compromise from both Sanders and Epps. This solution w ould involve getting a mental health professional or psychiatrist in to consult and potentially declare Aranda as mentally incompetent Hippocratic Oath of doing no harm, if the y decide that Aranda is indeed mentally competent. This middle of the road way would take the responsibility away from the main parties, but would create other ethical questions (E1, O1, and S1).

PAGE 167

155 In the first solution, the student supports the appeal by q uestioning the morality of capital punishment. The disadvantages of the solution, are worded in such a way that they continue to provide support for her preferred perspective. In the third solution, the student oint of view, while ignoring the possibility are merely subjective, (therefore biased) opinions viewed as a means to the desired end, rather than an objective evaluation based on relevant criteria. In her rationale for choosing the first solution, the student bases her opinion on her personal values Throughout the analysis, she focuses on the facts that support an established belief regarding the value of life regardless of the context: The preferred way to solve this problem would be the first strategy in aligning Epps and Sanders together in their fight towards better justice for inmates as well as another chance at life for Aranda, even though it might be one in degrading conditions Out of humanitarian reasons as well as religious reasons for some, life is worth fighting because no one really knows what the future will bring Even thoug h the individual wish would be squashed in the short run, it would serve a higher purpose in the long term (J1 ; L1 ). As compared to Pre Reflective Pattern 0, where significant limitations are personal wishes, are inherent limitations of this solution.

PAGE 168

156 Global Ratings The following codes were assigned to papers that demonstrated overall problem analysis and problem solving approaches consistent with Quasi Reflective Perfor mance Pattern 1 and Reflective Judgment Stage 4: C1: Acknowledges the existence of different contexts, but focuses on context in support of own opinion (O 1 = 66%; O 2 = 72% ) OA1: Appears to begin with conclusions and then stack up evidence/arguments to sup port it (O 1 = 53%; O 2 = 72% ) Students in the Quasi Reflective Performance Pattern 1 range acknowledged the significance of contextual factors in their analysis, but were selective in the issues that they addressed. While students in Pre Reflective Perform ance Pattern 0 tended to present the issues dichotomously, students at this level, acknowledged the existence and viability of other perspective, but only provided support for their own view. Their overall problem solving approach tended to reflect a lack of objectivity or acknowledgement of personal bias. The example provided above demonstrates this approach to problem resolution. Quasi Reflective Performance Pattern 2 Quasi Reflective Performance Pattern 2 is consistent with the epistemic assumptions cha racteristic of Reflective Judgment Stage 5. Students were most likely to score in Quasi Reflective Performance Pattern 2 on individual rubric items for Problem Identification Evaluation and Justificatio scored at Perf ormance Pattern 2 across the board. Because students exhibited a range of performance on the individual items, the majority of Performance Pattern 2 ratings were

PAGE 169

157 assigned to students whose final rubric score on the initial and final case was under 2.0 or w ithin the Performance Pattern 1 range. Only one student scored above 2.0 on the initial or final case. According to King & Kitchener (1994) assumptions related to the reflective jud gment model is not static, but may fluctuate between two stages, and occasionally three. Students who demonstrated Quasi Reflective Performance skills typically fluctuated between Performance Pattern 1 and 2 on the individual items, with an occasional r ati ng in Performance Pattern 3. This quasi reflective level is characterized by an overall approach to ill if the goal is to establish a detached, balanced view of evidence and information from ( Wolcott 2006a) Students who demonstrated Performance Pattern 2 skills were able to present a balanced description of the problem, identifying issues, assumptions and biases associated with various perspectives. T hey were able to organize material in a meaningful manner that allowed them to address complexities. However, they had difficulty prioritizing the issues in order to come to a well reasoned conclusion Problem Statement Case analyses that were c oded with an I2 were able to clearly identify the main problem as well as subsidiary, embedded, or implicit aspects of the problem. An important advance over Performance Pattern 1 was the ability to not only acknowledge uncertainty, but to also address the uncertai nties in the problem analysis. Relatively few students demonstrated this level of competency in dealing with ambiguities. This code was not assigned based on the problem statement alone, but in

PAGE 170

158 consideration of the student s ability to integrate the uncert ainties into the subsequent analysis. The following codes were assigned to problem statements at Quasi Reflective Performance Pattern 2: I2: Clearly identifies the main problem and subsidiary, embedded, or implicit aspects of the problem (O 1 = 34%; O 2 = 3 4% ) U2: Addresses significant and permanent uncertainties when interpreting and analyzing information a multifaceted problem definition, acknowledging uncertainty and including issues that are less obvious: (O 1 = 19%; O 2 = 12% ) The following student anal yzing concisely offers a multifaceted p roblem definition, acknowledges uncertainty, and includes issues that are less obvious: Psychiatric social worker, Noah Andrews is uncertain about how to proceed during his initial se ssion with new patient, Gregory Lange, a 15 frustration he naturally feels as a teenager attempting to gain freedom and independence from his parents. As a result, Gregory is experienci ng various personal, family, social, and spiritual issues. Due to the complexity of his issues and need for support, Noah must decide the best intervention to use while alone with Gregory and throughout the rest of the initial session (I2 ; U2 ). Contextual Analysis Quasi Reflective Stage 5 of the Reflective Model which undergirds the skills of Quasi Reflective Performance Pattern 2, is characterized by

PAGE 171

159 relativism, a belief that although knowledge is uncertain, individuals can make judgments about knowledge claims based on subjective interpretations of evidence which are bound by the context in which they occur (King & Kitchener, 1994; King & Kitchener, 1994; King & Kitchener, 1994) The following codes were assigned to contextual analyses that demonstrated Quasi Reflective Performance Pattern 2 skills: R2: Explores (considers from different perspectives) a wide range of relevant information (O 1 = 17%; O 2 = 38% ) MP2: Interpre ts information from multiple viewpoints; (O 1 = 16%; O 2 = 16% ) IN2: Objectively analyzes quality of information; Organizes information and concepts into viable framework for exploring realistic complexities of the problem (O 1 = 22%; O 2 = 25% ) Students in this range demonstrated the ability to objectively analyze the quality of information and organize it into a viable framework for exploring the realistic complexities of the problems (IN2). Rather than simply acknowledging the existence of mult iple perspec tives (MP1), students interpret ed information from multiple perspectives (MP2) In addition, they were able to identify less obvious issues that impacted the problem definition and analysis. The primary weakness of students using Performance Pattern 2 skil ls was the tendency to become overwhelmed by the amount of contextual considerations due to an inability to prioritize the issu es based on relevance or overarching criteria This often results in a thorough but excessively lengthy analysis, followed by wea k conclusions

PAGE 172

160 Consider the following contextual analysis of Seattle Community Association. In the interest of brevity, the selection is abbreviated. There are several issues in this case that Cheryl Cobbs must consider as she makes decisions about whether to continue to endorse cultural competency trainings or to nix them. As Executive Director of Seattle Community Association (SCA), Cheryl has several roles to fulfill in her job. She must act as a visionary for the agency and as a manager to her frustrations and possibly cause increased amounts of dissension and ront/cutting edge of cultural competency. Plus, what impact has her position had on how staff feels about the trainings? Even though the Cultural Competency trainings through the Minority Executive Directors Coalition were not mandatory, did staff feel pre ssure to attend workshops to please the boss?... Communication between top staff and lower staff seems to be one of the from top down staff, the other staff were probably already feel ing undervalued, and mandates may only make them feel even more undervalued. Another big issue to consider is the racial demographics of the agency.

PAGE 173

161 told that they needed to attend cu ltural sensitivity trainings from a boss s that raised their cultural awareness in ways that top management had not diversity on staff; with a more diverse staff, cultural competency trainings may not be needed if the staff hired is naturally competent. However, SCA may want to consider whether the new policies offend the current staff or possibly cause reverse discrimination, as in the case of Allan Bakke in 1978 (infoplease.com). Cheryl definitely needs to consider the funding s ources as she decides trainings, would any of the funding be in jeopardy? eliminated the cultural competency tr ainings for her staff?.... If the mission of the organization is to promote services that rid the community of poverty, prejudice, and neglect, would eliminating the program be a Cheryl also needs to take into consi deration the NASW Code 1.05 (a c) about cultural competence and social diversity (her legal dilemma).

PAGE 174

162 not have opportunities to be trained about different client cultures? However, Code 1.05c emphasizes that more than just client race should be Also, since the main idea of some of the anti racism classes was to awareness about a problem t hat was not previously perceived by the staff. The classes may have been effective, but also could have backfired against Cheryl if staff viewed her and top management as not being power sharers (IN2, MP2, R2). While seven out of twelve students analyzing this case identified with the staff perspective, and painted Cheryl in a decidedly negative light, this student attempted to describe the perspectives of each party in a balanced way. She addresses a wide range of relevant data and addressees enduring unce rtainties, such as the impact of the power differential on staff response, racial tensions, and the conflict between the roles of the visionary leader and the responsive manager. Furthermore, she considers the influence of contextual factors when analyzing the various perspectives. For example, she raises questions regarding the impact of staff discontent, the mission of the agency, need for leadership, the power differential between Cheryl and staff, budget cuts and layoffs, racial demographics of the orga nization, previous experience of employees, new hiring policies, funding sources, power sharing, the NASW code of ethics, and the impact of the decision on the community. However, because she is unable to prioritize the most

PAGE 175

163 relevant information, her conte xtual analysis is excessively lengthy and she has difficulty incorporating her analysis into the alternative solutions proposed. Alternative Solutions and Recommendation The problem solving approaches utilized by individuals at this stage might best be c the context, Stage 5 learners find it difficult to reach conclusions. Perhaps because the parameters of the problem were clearly defi ned in the decision case, this tendency to waffle between alternative solutions based on contextual factors was not clearly observed. Nevertheless, students using Quasi Reflective Pattern 2 skills appeared to be intentional about providing a more balanced approach to the problem analysis but tended to offer weak recommendations in comparison with the complexity of the analysis. The following codes applied to Quasi Reflective Pattern 2 for this unit of analysis: E2: Analyzes implications and consequences for multiple alternatives (O 1 = 47%; O 2 = 28% ) O2: Provides logical arguments for each option and either a) fails to provide an overall recommendation or b) offers a recommendation with little/no support (O 1 = 3 1 %; O 2 = 2 8 % ) S2: Reluctant to select and defe nd a single overall conclusion in light of viable alternative; may provide conclusions with inadequate support (O 1 = 0 %; O 2 = 9 % )

PAGE 176

164 J2: Based on interpretations of facts/evidence that are used to justify solutions within particular context. (Right solution depends on a variety of contextual factors) (O 1 = 3 4 %; O 2 = 34 % ) L2: Articulates connections among underlying contributors to limitations (O 1 = 0 %; O 2 = 3 % ) Although the students who used Quasi Reflective Performance Pattern 2 skills in the analysis w ere able to analyze the problem from multiple perspe ctives, they appeared to have difficulty es tablishing criteria that enabled them to choose between viable perspectives and options. While the student above provides a thorough problem analysis, and presen ts logical arguments for each alternative solution, she fails to adequately support her final recommendation in light of the issues she has identified in her analysis. She proposes three options: a) a mass email explaining the purpose of the trainings, b) holding small department meetings to reinforce the organizational mission, elicit feedback, and empower staff, or c) continuing with the status quo. Once again, in the interest of brevity, only the alternative that she chooses is represented. A second opti on is for Cheryl to hold small meetings in each department of the agency to serve several purposes: 1) ask staff to provide verbal and written feedback about specific things that top management can do to improve communication, 2) use this meeting time as a n opportunity to empower and recognize the staff members through verbal praise to let them know how valued they are, 3) reiterate the mission of the organization and explain why top management originally made a decision

PAGE 177

165 to offer the culture trainings. This approach addresses the problem by giving the staff a chance to express their concerns and giving top management a chance to explain why the trainings are needed. It does not address the problem of still having top down implemented trainings. There are man y positives to this approach since staff will be receiving attention on an intimate level that would allow for more discussion and would increase the likelihood of effective communication. Negatives to this approach is that scheduling meetings to involve a ll 400 staff will be nearly impossible and very time consuming; scheduled meetings may still feel like top down implementation; staff may not feel comfortable expressing their honest opinions verbally; and there is still a large possibility that frustratio ns and resentments are not resolved in one meeting (E2,O2 ) In this solution, the student makes an attempt to address the concerns of both staff and leadership, by providing a forum to address staff discontent, but continuing with the initiative She is ab le to analyze the implications and consequences of this approach including its limitations, which are significant. However, in adopting the strategy she fails to address the limitations or compare it to the other alternatives to reach a conclusion regardin g its superiority in spite of its shortcomings. The recommended strategy is for Cheryl to conduct meetings with all of the staff and invite staff to safely provide feedback, criticisms, and possibly even provide some ideas to Cheryl. Since there is an iss ue of the staff feeling undervalued, Cheryl could personally apologize and take the

PAGE 178

166 chance to affirm and empower her staff. She could model that she values their opinion by seeking their opinions. She does not necessarily need to abdicate her opinion of ne eding to have the culture trainings, but she can communicating the vision behind offering the trainings and explain the legal and financial mandates for the trainings. The most importa nt thing would be to address the original intentions of the trainings in the context of the mission while also addressing the misinterpretations/misconceptions about the purpose of the trainings (S2, J2, L0) er solution, in spite of her facility in analyzing the problem, and evaluating the implications and consequences of alternatives, may reflect the ambivalence of students at Stage 5 of the Reflective Judgment Model From this persp ective, endorsing one view point invalidates the legitimacy of the other. Having understood the complexity of the issues from each side, the student chooses an option that appears to meet the need to continue the initiative, but also addresses staff concerns Although she indicates and very time the concerns of each stakeholder As a result, she is unable to defend her position with the same rigor th at characterizes the rest of the paper. Global Ratings. The following two codes were assigned to papers that used Performance Pattern 2 strategies in their overall analysis and problem resolution approach:

PAGE 179

167 C2: Identifies and considers the influence of con text when analyzing perspectives and evidence (O 1 = 22%; O 2 = 16% ) OA2: Appears to perform comprehensive and objective analyses from different viewpoints, but unable to reach or strongly defend conclusions (O 1 = 25%; O 2 = 16% ) Surprisingly, these rating s were assigned more often to the initial than the final case analysis indicating that students who demonstrated the ability to perform a more complex analysis at the beginning of the semester, regressed on the final case. Nevertheless, these approaches o ccurred relatively infrequently throughout the sample as students were more likely to present a one sided analysis of the ill structured problem. Reflective Performance Pattern 3 Because Reflective Performance Pattern 4 skills are rarely observed, and the epistemic assumptions related to them have only been observed in advanced doctoral students (King & Kitchener, 1994) Performance Pattern 3 skills are arguably the goal of graduate education for MSW students. These s kills are related to the epistemic assumptions of RJM Stage 6, which marks the beginning of reflective thought and the related reflective thinking skills. The primary element of Reflective Performance Pattern 3 that differentiates it from the quasi refl ect ive skills of Pattern 1 and 2 is the ability to prioritize information and make comparisons across contexts by using general principles. Students using Reflective Performance Pattern 3 skills to resolve unstructured problems use a process for arriving at t he best conclusion, which involves considering multiple perspectives, evaluating information and evidence, comparing between options, and using

PAGE 180

168 come to a well founded conclusion based on objective comparisons of viable Performance Pattern 3 skills occurred on only 3% of the ratings assigned. Therefore, it is not possible to provide examples of Pattern 3 skills in each of the areas However, for the sake pattern 3 ratings is examined. Problem Statement The following codes were used to identify Reflective Performance Pattern 3 skills for the problem identification: I3: In addition to previo us level, emphasizes and states criteria for identifying the most important aspects of the problem (O 1 = 0%; O 2 = 0% ) U3: Identifies and discusses the significance of the most important uncertainties (O 1 = 0%; O 2 = 3% ) In order to receive a rating of I3 on the problem statement, students had to emphasize and state the criteria for identifying the most important aspects of the problem. This skill was not observed in any of the papers. A rating of U3, indicating that the student identified and discussed the significance of the most important uncertainties was assigned to the following case analysis of : This case raises the important question of what self determination means for a fifteen year old. Under the law, he is still considered a minor, but independence? How much of the presenting difficulties are attributable to

PAGE 181

169 social worker must determine how to proceed with Gregory's treatment, recognizing his client's unique limitations and need for supportive services while at the same time developing rapport and promoting his self determination to live as high a quality of life as possibl e (I2, U3). application of the right to self determination. She acknowledges the limitation as being of singular importance in the development of a treatment plan. (Of interes t is the fact that this factor is not considered in any of the other analyses.) In addition, she also acknowledges uncertainty regarding the problem definition based on the family Contextual Analysis The contextual ana lyses of students using Reflective Performance Pattern 3 skills are characterized by a balanced analysis organized on the basis of principles or criteria that apply across perspectives and contexts. The following codes applied to this performance level : R 3: Focuses on the most important relevant information able to prioritize (O 1 = 0%; O 2 = 3% ) MP3: Evaluates information using general principles that allow comparisons across viewpoints (O 1 = 0%; O 2 = 0% ) IN3: Focuses analysis on the most important inform ation based on reasonable assumptions about relative importance; organizes information using criteria that apply across different viewpoints (O 1 = 0%; O 2 = 3% )

PAGE 182

170 The following contextual analysis is a mixture of Quasi Reflective Performance P attern 2 and Reflective Performance P attern 3 skills. Gregory is fifteen years old, an age at which most youth begin to experience hormonal changes and a desire for greater independence Because of Gregory's physical limitations, however, he is dependent upon others f or care (chiefly his mother), including assistance with eating and mobility Gregory makes a powerful statement when he tells Noah that he thinks of ways to die because "I will not be God's entertainment" (Sherr & Wolfer, 2002, p. 108) This statement sugg ests that Gregory blames God for his misfortune, that he resents his condition, and that he is potentially as it relates to the problem assessment, she also assesses the impact of h is developmental stage on his ability to cope with his disability.] Noah appears to struggle with his feelings towards Gregory, first dismissing him as someone with MR or a disability and then feeling sympathetic towards his situation, and even charmed by his humor and intelligence He also admits that he understands why Gregory would want to die. Noah juggles three roles simultaneously -he is a social worker who must promote the self determination of clients, he is a spiritual individual whose beliefs impa ct his practice, and he is a grandson who was taught that "the Lord loves all of his creation" (Sherr & Wolfer, 2002, p. 106) With all of these roles and feelings minus an understanding of traumatic

PAGE 183

171 brain injury (TBI), Noah must think quickly on his feet and with relate the facts of the case, but links them together in a way that provides meaningful organization for problem resolution]. Despite the incredible tragedy that Gregory an d his family have endured, strengths are evident Noah observes that Mrs. Lange dearly loves her child and wants the best for him. She tolerates his abrasive words with the utmost of calm and patience She may be somewhat overprotective (perhaps due to the guilt she carries concerning his accident occurring in the first place), yet she does not allow him the opportunity to harm himself, and she brings him to the clinic because she acknowledges that she cannot help him entirely on her own. Gregory possesses a number of strengths, including intelligence, a good sense of humor, a desire to be independent, and the ability to adapt, as is shown through his learning to communicate with his voice machine. [Further interpretation of the family dynamics is offered fr om a strengths perspective that demonstrates an ability to empathize with each stakeholder]. problems are likely to persi st long after the injury occurs (Yeates & Taylor, 2006). The effects of TBI are unique to every individual due to the highly individualized nature of the lesions sustained during trauma. As a

PAGE 184

172 result, "successful treatment" can mean something different for every individual (Laatsch et al., 2007). Noah will clearly need to educate himself regarding TBI and its various effective treatments . Considering that in crisis, Noah will ne ed to obtain information concerning past treatment attempts before planning a treatment plan for the future (MP2; R3; IN2/3; U3). [The student uses research to support h er assessment of the problem]. This student grounds her interpretations on information presented in the case and the elements are linked to create a coherent and meaningful analysis. Although she does not clearly articulate it (IN2) she appears to use the strengths perspective as an organizing principle (IN3). Rather than focusing on the el ements in the case that support one perspective, the student is able to articulate the strengths of each individual involved in the case. Finally, the student appears to be able to prioritize the information based on its relative importance to the problem resolution (R3). Although she demonstrates an ability to interpret information from multiple perspectives (MP2), she stops short of making comparisons across viewpoints (MP3) For example, an analysis at Performance Pattern 3 might have included a realisti c comparison based on the facts in the case Alternative Solutions and Recommendation At Reflective Performance Pattern 3 students comparing alternative solutions consider the implications and consequences of each perspective and are able to articulate well founded support for one solution over other viable options. The following codes were used to identify this performance pattern:

PAGE 185

173 E3: In addition establishes criteria to prio ritize implications and consequences across alternatives (O 1 = 0%; O 2 = 3% ) O3: Provides well founded, overarching principles to objectively compare and choose among alternative solutions (O 1 = 3%; O 2 = 6% ) S3: Articulates criteria that apply across via ble alternatives to reach well founded conclusions (O 1 = 10%; O 2 = 18% ) J3: Based on Comparing evidence and opinion from different perspectives and constructing solutions that are evaluated by personally endorsed criteria, such tility, or need for action (O 1 = 6%; O 2 = 3% ) L3: Adequately describes relative importance of solution limitations when c ompared to other viable options (O 1 = 0%; O 2 = 0% ) In the selection below, the student uses two organizing principles that facilitat e the comparison of the solutions to each other. Each alternative solution addresses two determination. Noah could proceed with a depression/suicide screening due to his statements regarding stayi ng awake at night thinking of ways to die Because Gregory does appear to enjoy "getting a rise" out of his mother, this opportunity could allow Noah to develop a better understanding of his client and his true risk of suicide without Gregory's mother bei ng present.[Grounds solutions on information in the case] Gregory could open up, or he could feel that Noah is seeking to further control him through all of his questions and resist therapy all together By conducting

PAGE 186

174 a depression/suicide screening, Noah would address the immediate risk of self harm and potentially develop rapport By eliciting a commitment to not self harm and explaining that he cannot allow Gregory to harm himself, Noah could risk his new client feeling that his self determination is bei ng threatened and perhaps even that Noah is conspiring with his mother against him By encouraging the assessment, perhaps by first framing his questions around religion, however, Noah could provide a foundation upon which to develop a longer term treatmen t plan for his client. Noah could discuss with Gregory what he would like to accomplish in therapy This approach could provide an opportunity for Noah to better understand his client's needs, and it could contribute to the rapport building process. It co uld also have the effect of facilitating Gregory's continued negativity and ridicule of God, his mother, and life in general. This approach would not guarantee that Gregory would buy into the idea of improving his quality of life, but it would begin the di scussion and perhaps raise some possibilities (vocational training, supportive youth groups, etc.). This would address the problem by allowing Gregory to have some control over the topics discussed, thus encouraging his self determination On the other han d, if his suicidal behavior is not confronted directly, it may not get discussed at all, and Gregory could carry out a plan to harm himself before Noah gets a chance to help him.

PAGE 187

175 Noah could approach Gregory within the framework of his family's love and ded ication. In other words, he could approach his discussion with Gregory through the lens of the strengths perspective, highlighting those various "positives" listed in the contextual analysis above Because Noah has minimal knowledge of TBI, this approach c ould allow him to approach the situation with observations he has made thus far On the other hand, if Noah does most of the talking, Noah may not feel empowered to share his true thoughts and feelings This solution could address the problem by suggesting to Gregory that his life is worthwhile, not just for himself, but also for others, such as his mother and friends Gregory could feel antagonized, however, particularly due to his age and desire for independence now. This approach could have the effect of not promoting Gregory's self determination at all, but rather his mother's self determination and even Noah's, as a social worker seeking to "do best" for his client (E3, O3; S3). The student objectively considers the implications and consequences of each alternative and uses the dual criteria of safety and self determination to consider their plausibility. In the next section, although she fails to articulate her reasons based on a comparison of the two principles used to organize the evaluation of the al ternative for self determination. Therefore, although not clearly articulated, the first part of the solution appears to be based on a prioritization of the issues and the utility of the solution

PAGE 188

176 (J3). This is followed up with a long term strategy that addresses quality of life, the nature of the diagnosis (temporary uncertainty), and the family issues. However, the spirituality issues which seem to dominate the decision case narrative are not addressed. First and foremost, precautions must be taken to ensure that Gregory does not harm himself Addressing his lack of eating and sleeplessness are immediate concerns that will directly impact his upcoming surgery, which may directly improve his quality of life. After Noah completes some research, perhaps he will continue to provide family and individual therapy to address relationship issues and negative thinking processes CBT is an approach that has proven effective for som e TBI patients (Malec, et al., 2007) Web based family problem solving interventions The first alternative will be determined successfu l if Gregory does not harm himself and if he begins to engage in self care practices that allow him to undergo the scheduled surgery next month The family will participate in ongoing individual and family therapy to address communication and boundary issu es directed towards improving "Quality of life" is a very subjective concept for the TBI population (Souza, et al., 2007), and this will need to be clearly operationalized in future therapy sessions (J2/J3; L1)

PAGE 189

177 somewhere between Quasi Reflective Performance Pattern 2 and Reflective Performance Pattern 3 of the three alternatives she propos es, although it is most closely related to the first alternative regarding a suicide screening. Rather than justifying her decision based on a comparison of the three alternatives, she launches on a different track. The long term solutions regarding TBI an d family therapy almost appear to be added as an afterthought, and therefore are not included in the discussion of pros and cons In that sense, her paper seems to be more characteristic of Performance Pattern 2, which is typified by a balanced approach to examining all the important considerations, but a failure to adequately justify conclusions. Finally, the student is unable to appropriately address limitations raised in the ions, therefore the statement is coded L1. An example of Performance Pattern 3 Just ification skills is provided in the following recommendation from the initial case, Unusual Appeal. Rather than simply choosing one alternative and explaining its merits, the student compares it to the others, articulating criteria that apply across the al ternatives to reach a conclusion (S3, O3) While the comparison is somewhat superficial, it represents one of the few attempts to

PAGE 190

178 personal values, utility, or the need The third alternative is the preferred strategy because it most objectively treated more h survive and to possibly even be found innocent Cynthia assessment If, after treatment, the client still wants to die, this should be presented to the court If he renews his desire to live, this could fuel his appeal pro cess situation, leaving room for more reliable support to be gathered 003, quoted in Holdman, 2000) Making some compromises among highly skilled colleagues for the developing a holistic individual picture of the client is vital to accurately assess the According to Section

PAGE 191

179 responsibility is to promote the well Because determi nation. By stating his current desire to die and then recommending follow up services from the court, she is responsibly seeking additional information that will Note that the stu no single expert provides a conclusive solution because there are multiple factors that must be addressed. Additionally, there is a focus on seeking additional information, acknowledging tha t the construction of knowledge is ongoing and subject to evaluation. Another feature of this analysis is a thoughtful application of the NASW code of ethics. determination this is ba lanced against the need to assess his soundness to make such an important decision. Global Ratings Although a predominant use of Reflective Performance Pattern 3 skills was not observed, a few students who demonstrated an overall problem solving strategy consistent with Reflective Pattern 3 were assigned the following codes: C3: Analyzes the issue with a clear sense of scope and context sees the bigger picture (O 1 = 3%; O 2 = 3% ) OA3: Appears to develop well founded conclusions based on comprehensive and objective comparison of viable alternatives (O 1 = 9%; O 2 = 6%)

PAGE 192

180 Code Omissions Two significant skills that are expected to emerge in Performance Pattern 2 and continue in subsequent levels were omitted from the rubric because they were not observed in the papers. The ability to articulate assumptions and reasoning associated with various perspectives, and the ability to acknowledge and control for the effects of ills to be included with Performance Pattern 2 for Multiple Perspectives (MP2: Interprets information from multiple viewpoints; identifies and evaluates assumptions; attempts to control own biases). As a result, nearly all student papers were coded as MP1. However, because it became multiple perspectives and that demonstrat ed by students using primarily Quasi Reflective Performance Pattern 1 skills this qualifier wa s removed. The assumption was made that the skills were absent because they were not required in the assignment description. Nevertheless, the lack of critical appraisal of the quality of information presented is of concern. Although a few students challen ged the assumption in Unusual Appeal that acknowledgement or questioning of assumptions was very rare. As mentioned previously, innocence had not been warranted by any evidence in the case. Although a number of students referred to personal values as a rationale or justification for their positions, no one acknowledged having a personal bias that they

PAGE 193

181 intentionally controlled for in working through a solution. It can be argued, however, that although students did not articulate their biases, those who used at least Performance Pattern 2 skills showed evidence of an attempt to deal with personal biases by presenting a balanced anal ysis of the problem from multiple perspectives. Rationale Because the original case analysis assignment did not elicit information regarding students include a statemen t explaining the rationale for their problem resolution. Nine different themes were evident in these statements, however, most of the students used a variety of rationale, some of which were not evident in their analysis. For example, students may have ind icated that they used research to come to their conclusions, but did statements rarely fit in one category alone; therefore, they did not consistently facilitate a clear assessment regarding the epistemic assumptions in use. With the exception of authority, which appeared to be clearly related to the epistemic assumptions consistent with St age 3 of the Reflective Judgment Model, the connections between the other rationale and the underlying assumptions regarding how knowledge is ascertained were far more obscure However, the fact that personal experience was the most frequently cited ration ale, followed by personal beliefs and values, is consistent with the finding that the majority of students were functioning in the Quasi Reflective Perfor mance Pattern 1 range. This performance level, associated with

PAGE 194

182 RJM Stage 4, is consistent with the und erlying assumption that knowledge is (King & Kitchener, 1994, p. 58) According to King and a conclusion but use personal beliefs to choose the evidence used to support preconceived beliefs (p. 58). The rationale statement below typified many of the statements, in which personal beliefs intuition or both were used to guide the problem solving process: I based my decision on my intuition and my own beliefs. My job is to see the possibilities or look to find them. Also, I think this approach is best for someone [Intuit ion, Personal Beliefs, Previous Knowledge]. The following student bases her rationale on a combination of previous knowledge and personal beliefs: I based my decision on knowledge that I obtained through my cross over class regarding leadership styles and theories I practice the Power Principle, by Blaine Lee, and believe that people should lead by example and not be coercion I also based my decision on my own ethics and values in that it is important for people to be informed as a part of the decision ma king process [Previous Knowledge, Personal Beliefs/Values]. Although 10 students used previous kn owledge as a rationale, only t wo students alluded to a specific social work theory while the student above referred to the Power Principle. Most often student s made vague references to previous coursework without

PAGE 195

1 83 referring to specific theories or concepts One mentioned the strengths perspective and the other the ecological system model Basing the proposed solution on an unsupported opinion, a Stage 3 strateg y, was only observed in a few instances, however, the following statement provides an example of this rationale: Personally, I would fire Cheryl because she is not doing her job. But apparently she is the board so that is not possible. Therefore, it is imp ortant to limit her control over the agency and give some back to the employees. In addition, I believe in full disclosure, that when people know all the information, only then can they make informed decisions. I like giving handing over the decision makin g to the clients/employees [Unsupported opinion, Personal beliefs/values]. As previously discussed, the use of an authoritative source, such as the NASW Code of Ethics, a text, or a professor, was a rationale that supported Reflective Judgment Stage 3 prob lem solving approaches if it was used to validate the premise that the problem could be resolved with certainty. The following student uses both the Code of Ethics and a statement made by a professor to support her belief that there was only one correct ap proach to the problem. My decision for this case is based on Social Work Ethics and a statement a professor once made in class on ethics, than to lose my license for not upholding those ethics You can

PAGE 196

184 always f the case I remembered these words [Authority]. While some students cited research as part of their rationale, none indicated that they had compared competing views regarding the issues, an approach consistent with reflective judgment. Most often research was used to confirm previous beliefs. In many cases, specific citations were not offered In most instances where research was used as a rationale for the solution only one source was cited. The student below provided numerous APA references to support her analysis. The basis for the recommendation is based on empirical research of symptoms of schizophrenia, treatment of schizophrenia, mental health in the prison system, and research of the N ASW code of ethics It is also rather than find treatments for them to reduce recidivism [Research, Personal Beliefs/Values]. Using the utility of the solution has been associated with Stage 6 of the RJM in the literature. In the following statement, the student presents a coherent argument supported by research for the utility of requesting an official forensic psychological evaluation in Unusual Appeal. I chose this strategy because if Jose is found incompetent he would be moved to a psychiatric hospital. According to Goodnough (2006), there is a Florida state law that requires inmates to be moved from prisons to a psychiatric hospital within fifteen days of being found incompetent. A t a hospital his quality of life would

PAGE 197

185 eflection of his true self. Geiman (2007) states that inmates suffering from a mental illness often cannot behave, feel and think normally, therefore displaying improper behaviors within the system and ultimately violating the rules and norms of the jail. This belief takes me to believe that placement in a psychiatric facility would not be inappropriate for Jose because his recent violent streak might be a symptom of his environment and lack of care. I further based my strategy this way by reminding myself of the scope, mission and purpose of the agency and by asking the court to being able to place blame on the judge [Utility, Research]. The utility of the solution was utilized as a rationale more often on the final case than on the initial case. Although associated with higher levels of reflective judgment, this approach may have also been facilitated by the fact that the initial case represented an obvious ethical dilemma, which may have led students to rely more heavily on personal values rather than utilitarian concerns. concept of justification, they were too ambiguous to reveal clearly explicable p atterns in coupling their rationale and problem solving approaches, the statements did not address

PAGE 198

186 the question of uncertainty about their positions, an important key to understanding their epistemic assumptions.

PAGE 199

187 Chapter V Discussion and Implications This chapter will review the current study, including a summary of the findings, discussion, and implications for social work education. A summary of the study design an d theoretical framework is followed by the sample characteristics, and the quantitative and qualitative results of hypothesis testing. Finally, the chapter addresses the limitations of the study, implications for social work education and suggestions for f urther research. Summary of the Current Study The purpose of this study was to evaluate whether case method instruction had a associated with the ability to reason throug h ill structured problems. The development of this aspect of critical thinking is especially relevant in social work education because graduates will routinely confront complex, multifaceted problems in the course of their social work practice. Although th e case method has been endorsed within social work education as an instructional strategy with high utility for preparing students for the complex realities of the practice world, there has been little research that assesses outcomes (Cossom, 1991; Jones, 2003; LeCroy, 1999; Scales & Wolfer, 2006) This study utilized the Reflective Judgment Model as a theoretical framework to assess reflective thinking a t the beginning and end of a case method course. King and

PAGE 200

188 progression that occurs as individuals become better able to deal with ill structured problems by acknowledging uncert ainty, considering multiple perspectives, evaluating relevant evidence and defending their own points of view on controversial issues. A quasi experimental pre post nonequivalent control group design was used to explore whether students who participated in a case method course demonstrated greater increases in reflective judgment over the course of a semester than those who did not. At the beginning and end of the semester, the intervention and comparison groups completed the Reasoning about Current Issue s Test (RCI), which is an online, standardized measure based on the Reflective Judgment Model that has been widely used to assess reflective judgment ( Wood et al., 2002) Because of questions regarding the ability of the RCI to detect epistemological changes over short periods of time, qualitative methods were used to triangulate findings. Content analysis procedures were utilized to identify reflective thinking skills evident in the initial and final case analysis pa pers of participants enrolled in the case method course. The results of the content analysis were analyzed using quantitative as well as qualitative methods. The study also examined the influence of age, race, gender, and social work experience on RCI scor es S ample Characteristics Twenty three percent of the students enrolled in the advanced year case method course ( n = 40) completed the study, as compared to 21% ( n =18) of students enrolled in the foundation year research methodology course. The study was heavily impacted by

PAGE 201

189 attrition as only 53% of those who initially expressed interest in the study completed the pretest, and the sample was further reduced at posttest by 17% in the intervention group and 40% in the comparison group. No significant differe nces existed between those who dropped out and those who completed the study on pretest scores or any of the demographic variables. The level of participation differed by course section. Students enrolled in Sections 1 and 3 were the most likely to partic ipate in both the quantitative and qualitative aspects of the study, while sections 2, 6 & 9 each had only one student which completed the study. It is not known what may have contributed to greater participation in some sections than others; however, the level of support and encouragement of the professor is likely to have had an impact on student interest. The majority of participants in both groups were under 30 (85%), Caucasian (87%), female (95%), and had no previous social work experience (64%). Th ere were no significant differences between the groups on any of the demographic variables, which served to allay concerns regarding the lack of a randomized sample and the small sample size. The mean RCI pretest score for the overall sample was 5.28, with students in the intervention group having a mean score of 5.26 and those in the comparison group a mean of 5.32. The sample mean was significantly below a graduate student mean for the RCI reported in an unpublished report by Kitchener and colleagues (200 2). However, it should be noted that the RCI was only normed on 46 graduate students and that the paper and pencil precursor to the current online version was used. The current sample is

PAGE 202

190 comparable to samples from the only studies that have reported a mean for RCI scores of graduate students. One study involved 126 graduate students enrolled in an educational score of 5.3 ( MacDonald, 2003) Another study of 110 graduate counselors in a psychology program reported that 68% were doctoral interns and the mean RCI score was 5.4 (Owen, 2005) Because these more recent studies involved a muc h larger sample ( n = 236) than the previous normed mean of 46 students, the pooled mean RCI score of 5.35 was compared to the intervention group mean using a one sample t test. The results indicated that the differences were not significant, suggesting tha t the mean reflective judgment level of the current sample is comparable to other graduate student populations t (47) = 1.252, p = .217 Hypothesis Testing for Demographic Factors The results of hypothesis testing regarding the effects of demographic fac tors on reflective judgment were inconclusive, as significant differences indicated at pretest, were not evident at posttest. Based on the findings of previous studies, it was hypothesized that there would be differences in reflective judgment based on age and social work experience, but not based on gender or ethnicity. Although the most recent findings across studies using the RCI, slightly favored women rather than men, in the current study, the RCI mean for male participants was significantly higher th an the female mean at pretest ( p = .01). However, because there were only three male participants, and two scored in the 95 th percentile for pretest scores,

PAGE 203

191 these results cannot be generalized to other social work student populations. In addition, differen ces based on gender were not evident at posttest as the male mean regressed by nearly one stage ( .82) in comparison to pretest scores In a similar pattern, while RCI scores were significantly higher for minority students at pretest, they regressed by ne arly a stage ( .92), at posttest. No confounding factors were identified which would account for the differences at pretest; however, once again, the number of participants in the two groups were markedly different, with Caucasians outnumbering minority st udents nearly 7 to 1. The lack of consistency in the pre and posttest results for gender and race was apparently a function of the general regression at posttest. Because those with the highest scores regressed toward the mean, outliers in the minority and male groups regressed by nearly a stage, eliminating differences observed at pretest based on gender and race. Therefore, because of the small numbers in these groups and the regression at posttest, the pretest findings for gender and race are inconclusiv e. A hypothesis that age would positively impact RCI scores was not supported. The finding that RCI was not positively correlated with age, was likely impacted by the lack of variability in educational level in the sample. Although previous studies have re ported a positive correlation between age and RJ stage, age differences appear to be confounded with educational experience (King & Kitchener, 1994) The positive correlation between age and RCI scores is not evident in adult populations that have not completed college. A comparison of nonstudent adults across six studies concluded that adults who had not completed college had a mean RJ score of 3.6 compared to a mean of 4.29 for those with

PAGE 204

192 college degrees (King & Kitchener, 1994) Therefore, because all participants were at the same educational level, and the majority were under 30 there was not sufficient variability in the sample for differences based on age to be observed at pretest. A moderate correlation (.27) which approached significance was observed at posttest ( p=. 052). This may have been influenced by the finding that students in the youngest age group (under 26) regressed by an average of .25 while, the older two group s remained essentially constant ( .02). Previous social work experience did not influence RCI scores significantly. However, the majority of students in the sample were inexperienced. Because there are no previous studies that have assessed the reflective judgment level of social workers or social work students, these findings cannot be compared to others. In summary, the most likely explanation for the lack of consistency between the current study and previous studies on RCI scores and demographic variabl es is the small sample size and the homogeneity of the group. Because nearly 70% of the sample was under the age of 26, at the same educational level, and lacked previous social work practice experience, the sample lacked sufficient variability for differe ntial patterns to emerge. Likewise, because 87% of the sample was Caucasian and 95% was female, a realistic picture of the influence of gender or race on RCI scores could not be assessed. In a similar study of dental students, Boyd (2005) attributed the lack of differences on the RCI based on any of the demographic factors to the lack of variability in the sample and the small sample size. Kitchener (1994) notes that un balanced sample sizes and

PAGE 205

193 differential variability in performance reduces the statistical power of epistemological hypothesis testing Hypothesis Testing for RCI S cores This study predicted that students who participated in a case method course would inc rease their scores on the RCI by the end of the course and that increases would be greater than those experienced in the comparison group. The RCI scores of both the intervention and comparison group decreased at posttest. However, the scores of the compar ison group decreased significantly ( .474, p = .047), while the intervention group scores decreased very slightly, ( .041, p =. 764 ). Based on findings regarding the test retest reliability of the Reflective Judgment Interview (.87) King & Kitchener sugge st that regressions observed in RJ scores between testing over short intervals are likely to be a result of measurement error (1994) In a meta analysis of longitudinal studies, they found across studies that par primarily on the length of time between testing. However, one short term study with three months between testings reported reversals in 16% of the cases. Based on the consistency of all other find ings, King & Kitchener attributed these reversals to measurement error. T o date test retest reliability measures have not been reported for the RCI, so it is not possible to draw conclusions regarding the likelihood that the regression observed simply refl ects measurement error However, the differences between the nature of the tasks involved in the two assessment measures makes it improbable that test retest

PAGE 206

194 reliability measures for the two instruments are comparable. The ability of a personal interviewer to engage and sustain the interest of participants in questions regarding their epistemic assumptions is likely to be much greater than the level of interest and engagement generated by a retest of a computerized assessment measure The familiarity of the instrument and urgency of other demands are more likely to result in careless responses that do not reflect true scores Owen (2004, as cited in Owen, 2005) found that the internal consistency of the RCI increased when students took at least 35 minutes to complete the test. Therefore, it is possible that the regression in scores observed in this study may be attributed to haste and decreased interest at the end of the semester, when students wer e pressed by competing concerns. This trend has been observed by others completing posttest measures at the end of a semester (Allen & Razvi, 2006; C assarino, 2006; Hesterberg, 2005) Students in the intervention group may have sustained greater interest in the study than those in the comparison group because they were aware that the findings were related to relevant coursework. This possibility is su pported by the finding that RCI pretest and posttest scores were significantly correlated for intervention group participants r (34) =. 405 p = .016, but not for comparison group participants, r (17) = .105, p = .678 The most significant factor in predi decreased, or were constant was RCI pretest scores. Boyd (2005) reported a similar finding in a study of the effects of clinical journaling on the reflective judgment of 37 dental students who participated in an RCI pre and posttest at the beginning and end of the first year. In the current study, pretest scores were negatively correlated with change

PAGE 207

195 scores, indicating that participants who began with high scores regressed t oward the mean at posttest, while those with low pretest scores improved. A stepwise multiple regression conducted to determine the best predictor of change in RCI scores indicated that the mean RCI pretest score was the only factor that accounted for any of the variability in the change score. While group membership, age, and race were excluded from the model, pretest scores accounted for 33% of the variability. This finding suggests the possibility that variability in RCI scores between pre and posttest may have been a function of the principle of regression toward the mean, which is a concern in non equivalent quasi experimental designs due to the lack of random assignment to the groups (Shaughnessy & Zechmei ster, 1990) Although the distribution of RCI scores met the assumptions of normality, and there were no significant differences between groups on any of the demographic variables assessed, there is a possibility that high and low pretest scores were a fu nction of measurement error. A comparison between participants in both groups who scored below the mean on the RCI at pretest (and therefore showed the most improvement at posttest) indicated that although the intervention group participants ( M = .36) imp roved slightly more than the comparison group ( M = .30), the differences were not significant, t (14) = .149, p = .883. These results suggest that the increase in posttest scores observed among those who scored below the mean on the pretest cannot be attri buted to a treatment effect for participants in the intervention group. The study design called for a comparison of the pretest scores of both groups to determine the amount of change that could be attributed to simple maturation. It was

PAGE 208

196 predicted that increases in the intervention group would exceed the difference that existed between foundation year and advanced year students This hypothesis could not be tested because there were no significant differences between the RCI scores of foundation year and second year students. Comparison of the intervention group ( n = 35) and the comparison group ( n = 18) pretest scores did not support the assumption that there would be a measurable maturation effect evidenced by higher RCI scores in the intervention group Although the comparison was cross sectional rather than longitudinal, this finding further supports the premise that reflective judgment develops slowly. However, the similarity between pretest scores may have been impacted by the fact that the intervent ion group contained advanced standing students which were essentially in their first year of graduate school although completing advanced year courses. Although this information was not captured, 31% of all students enrolled in the case method course were advanced standing students. Because the RCI seeks to assess changes in the epistemological assumptions of respondents, the results of hypothesis testing must be interpreted within that context. Although theories regarding the relationship between epistemic assumptions and the ability to engage in complex problem solving have been well supported (M. M. K. Brabeck, 1980; B. K. Hofer & Pintrich, 1997; King & Kitchener, 1994; King & Kitchener, 2004) the methodological challenges of assessing the development of more complex epistemology are numerous. Wood and Kadrash (2002) noted that while studi es have been able to document substantial differences in epistemological assumptions across educational levels, they have been less successful in assessing the efficacy of educational

PAGE 209

197 interventions or in detecting patterns of differential growth. Because r eflective judgment changes slowly, studies with a short time between testing are less likely to show significant change. In a review of longitudinal studies using the Reflective Judgment Interview, King and Kitchener (1994) observed that only samples (N = 3) that were retested within less than a year failed to show significant change. Wood and Kadrash (2002) conclude that while educational interventions may have an effect on reflective judgment, the lack of sensitivity of measures of epistemology to short term changes requires substantially larger sample sizes in order to detect differences. As a result, most studies seeking to detect change as a result of educational interven tions are underpowered and prone to Type II error. Although the population of MSW students at the host university participating in the case method course was large enough to warrant an attempt to use the RCI, the researcher was unable to secure a high pe rcentage of participation in spite of incentives. Based on projections of the sample sizes required to assess changes in RCI scores for short term educational interventions, the study was critically underpowered ( Wood & K adrash, 2002) Because of these limitations, conclusions regarding the efficacy of case method teaching based on RCI pre and posttest scores alone would be premature. Hypothesis Testing for Content Analysis Procedures Content analysis was used to determin e whether students enrolled in the case method course demonstrated increased reflective thinking skills between the initial and final decision case analysis completed. A review of the literature indicates that it is a

PAGE 210

198 frequently used method for assessing e vidence of critical thinking in student communication (Corich & Kinshuk, 2006; Levin 1993; Lundeberg & Fawver, 1994; Newman et al., 1995) The content analysis themes were selected based on their (2006b) template for developing a critical thinking rubric which is based on the Reflective J udgment Model and the cognitive development theories of Fischer was adapted to correspond with the requirements of the decision case analysis assignment. Thirty two of the 40 students participating in the intervention group (80%) submitted an initial and final case analysis. This study predicted that students enrolled in the case method course would demonstrate changes in reflective thinking based on their scores on a customized rubric designed to assess problem solving skills related to the resolution of ill structured submitted. The mean for the initial case was 1.1, while the mean for the final case was 1.09, indicating that overall there were no group changes observed betw een the beginning and end of the semester. Paired samples t tests on individual rubric items, as well as the overall mean, indicated that student performance did not change significantly between pretest and posttest. ding rubric varied considerably, ranging from .42 to 2.2 on the initial case and .42 to 2.7 on the final case, indicating a developmental range of over two stages. Although the ranges almost completely overlapped from pre to posttest, there was substantial variability within the group. Ten student s scores increased by .25, which was the criteria established for meaningful

PAGE 211

199 change appreciably. Consistent with the findings fo r the RCI scores, students who began with lower scores tended to improve, while those with higher scores on the initial case were significantly more likely to regress. Although the principle of regression toward the mean may have accounted for this trend, patterns observed in the content analysis suggest an alternate explanation. Differential Performance Based on Beginning RJM Level Fifteen (47%) students scored in the Pre Reflective Performance Pattern 0 range on the initial case. This group made statistic ally significant progress moving from an initial mean score of .70 5 to a final mean score of .938, indicating greater reliance on q uasi reflective strategies at the end of the semester In comparison, 16 students (50%) scored in the Quasi Reflective Perfo rmance Pattern 1 range. Their initial mean score was relatively flat but moved in the opposite direction Only one student scored in Quasi Reflective Performance Pa ttern 2 range on the initial case regressed of a stage to 1.4. Pre Reflective Performance Pattern 0 Although it has been suggested that students with more compl ex epistemology may benefit the most from a case method course (Allen & Razvi, 2006; Ertmer et al., 1996) it is pla usibl e that students at the pre reflective stages were provided with the contextual support necessary to progress to the quasi reflecti ve levels, while those already in the quasi r eflective levels lacked the

PAGE 212

200 support necessary to facilitate the development of the underlying epistemic assumptions and skills characteristic of Stage 5 Previous research ( Kitchener et al., 1993) suggests that when individuals receive appropriate contextual support their performance will move level of reasoning, toward their optima l level, which is the level they are capable of when provided with appropriate support. participants functioning at the pre reflective level at the start of the study moved to a quasi reflective level in their approach to problem solving by the end of the course. Students in the pre reflective range at pretest adopted a problem solving approach at posttest that was more consistent with Stage 4 assumptions. Students function ing in the pre reflective stages view knowledge as certain and largely defined by authorities. Assignments that require that they make judgments may elicit expressions of confusion or suspicion that the professor is withholding information regarding the co rrect answer. When confronted with the uncertainties inherent in an ill structured problem, they may not recognize the ambiguity and attempt to find the right answer. Highly motivated students may research the issue in order find the correct answer. Altern ately, they may determine that the uncertainty is temporary and will be resolved when more information is available. Until then they are likely to draw their conclusion s by identifying a position that fits with previous beliefs or personal preference. Beca use the correct answer is unknown, they do not perceive a need to evaluate perspectives based on their plausibility.

PAGE 213

201 The primary developmental task for students in this range is to perceive the enduring uncertainty in an ill structured problem and begin t o acknowledge the viability of multiple perspectives. This change is grounded in transformative learning experiences that enable students to begin to perceive that what is known is frequently uncertain and that where there is uncertainty there is room for the consideration of alternate perspectives. The decision case analysis assignment and ensuing discussion provides contextual support for students with dualistic views of reality to move to a more multiplistic view as they are exposed to the various perspe ctives of their classmates and growth by giving students permission to entertain multi ple perspectives. In comparison with the initial papers, final papers were less likely to be characterized by dichotomous presentations of the issues. Eight of the 32 students completing the initial case analysis (25%) presented the issues as if the persp ective of one party was clearly wrong and the perspective of the other clearly right. By contrast, this tendency to present issues dichotomously only occurred on three (9%) of the final papers. While students writing the final paper continued to present fu ndamentally one sided positions, they acknowledged that there was more than one way of perceiving the issues based on the personal characteristics of the stakeholders. For example, students analyzing Seattle Community Association developed their recommenda tions largely based upon whether they identified with the positions of the director or the staff.

PAGE 214

202 Additionally, while seven students (22%) justified their positions on the initial case based on an authoritative source, no students used an authoritative sou rce as justification for their position on the final case. Those who justified their recommendations on the initial case by referring to the dictums of an authoritative source, were more likely to develop their own perspective and justify their solutions based on evidence that they believed supported their opinion on the final case. Quasi Reflective Performance Pattern 1. The mean RCI score of 5.2 and mean rubric score of 1.1 of the intervention group suggests that students in the sample were predominantly functioning in the lower range of Stage 4 of the Reflective Judgment Model, or Quasi Reflective Performance Pattern 1. Fifty percent of students scored between 1.0 and 1.9 on the initial case, the majority scoring between 1.0 and 1.5. Students who scored in this range on the initial case regressed slightly but not significantly, on the final case. This performance pattern and its corresponding developmental stage is characterized by the belief that because there are no absolutely certain ways to know the r ight solutions to an open ended problem, each person must decide what is right for themselves based on criteria that is idiosyncratic to the individual Students at this stage are more comfortable with making their own judgments in light of the realizati on that authorities cannot provide absolute answers to open ended problems. They recognize the viability of multiple perspectives, but differences are attributed to personal characteristics (Ki ng & Kitchener, 1994; Wolcott 2006a) For example, the majority of students analyzing the initial case framed the problem as an

PAGE 215

203 interpersonal conflict resulting from the differing value systems of attorneys and social workers (King & Kitchener, 1994; Wolcott 2006a) Consistent with the Reflective Judgment Model these students used evidence to support their position, but they tended to use evidence inconsistently and focus primarily on suppo rting their own positions. They tended to ignore p erspectives that differed from their own or evidence /information in the case that co ntradicted their conclusions This quasi reflective tendency toward confirmatory bias has been widely reported in the lite rature as a source of error in clinical decision making (Gambrill, 1990; Havercamp, 1993; Snyder & Swann, 1978; Spengler & Strohmer, 1994) A recent study to determine whether the reflective judgment level of graduate student counselors ( M =5.4) was related to their use of confirmatory bias, found that participants primarily relied on confirmatory clinical judgment strategies (Owen, 2005) when searching for information However, students with higher reflective judgment levels were more likely to use neutral strategies. The fact that the majority of students scoring in Quasi Reflective Performance Pattern 1 f ocused on evidence in the case that supported their own perspective and justified their solutions based on personal experience, personal values, or intuition provides evidence of this reflective judgment stage. While students using Pre Reflective strategie s relied on authoritative sources to justify their conclusions, students who primarily used Performance Pattern 1 skills exhibited a strong sense of personal ownership f or their decisions The most frequently cited rationale used to justify positions was p ersonal experience (40% on initial and 38% on final case). Students recommended solution s to the dilemma presented in the case based on previous experience and

PAGE 216

204 preconceived beliefs, a hallmark of Stage 4 thinking. Students commonly cited intuition, instinc ts, feelings, and personal opinions as their primary rationale, and secondarily other sources of knowledge. For example: The basis for my recommendation is my personal opinion and cla ss that mentioned how having a solid organizational structure helps a company function effectively and efficiently. previous class. Although the course syllabus states that students will be required to think multi was little evidence that students used research to consider alternate approaches. Only five students (16%) cited an outside source to support their conclusions on the final case. Four of the five made only a brief mention of one source to support their co nclusion and none indicated that they had referred to literature to consider alternate perspectives. Instead, research was used superficially to confirm or support the preferred view. The developmental challenge for students in the Quasi Reflective Perfo rmance Pattern 1 range is to begin to view open ended problems with a wider view of contextual factors, to learn to identify personal biases, to evaluate the quality of information and knowledge claims by using evidence, and to consider the various implica tions of different perspectives. However, this approach hinges upon a transition in epistemic assumptions

PAGE 217

205 that views knowledge as constructed but subject to evaluation. As a group, the students did not demonstrate improvement in these areas between pre an d posttest measures Quasi Reflective Performance Pattern 2 Only one student scored in this level on the initial case, however, seven students scored between 1.5 and 2.0, indicating that the skills they were using were more consistent with Quasi Reflectiv e Pattern 2 than Pattern 1. Students scoring in the upper range of Quasi Reflective Performance Pattern 1 showed beginning evidence of the ability to understand problems more complexly organize information more effectively, link ideas to form coherent arg uments, and use evidence more consistently. However, only one of the seven students that scored above 1.5 on the initial case improved their score on the final case. Given that the process of case analysis is expected to increase reflective thinking, this finding b students who showed promise at the start of the study fail to improve their reasoning ability after a semester of analyzing and discussing open ended problems? An examination of the course syllabus suggests the possib ility that the actual structure of the assignment did not lend itself to the further development of Reflective Judgment Stage 5 thinking and Pattern 2 performance. According to the syllabus: Case analyses should be written as executive summaries. Executive all of the analytic detail. In fact, executive summaries often represent the first few pages of a more comprehensive analysis. The executive summary format is not intended to be an exhau stive analysis of all possible issues and alternate strategies but rather a concise, focused summary with the

PAGE 218

206 issues and alternatives only mentioned to insure they receive consideration. Any situation, no matter how complex, can generally be summarized in no more than three pages if reduced to its most essential elements. Limit case analyses to 700 1,000 words. While this assignment description provides appropriate guidelines for students with well developed Quasi Reflective Performance Pattern 2 skills to move toward Reflective Performance Pattern 3, it may discourage the development of Pattern 2 skills for those in the upper ends of Quasi Reflective Pattern 1 and beginning levels of Pattern 2. While the strength of students in Pattern 2 is their ability to perform a thorough and complex analysis, their weakness is in prioritizing the issues and coming to strong conclusions (King & Kitchener, 1994; Wolcott 2006a) The sheer volume of information and contextual considerations tends to overwhelm learners who are using epistemic assumptions consistent with Stage 5 of the model. Research to validate the Reflective Judgment Model indicates that the ability to process and interpret information effectiv ely, establish criteria to prioritize relevant issues and to judge between competing options does not emerge until Stage 6, which rarely occurs in beginning graduate students (King et al., 1990; King & Kitchener, 1994) The requirement that students frame their analysis as an executive summary assumes that graduate level students will be capable of reducing a complex problem to its most essential elements. Research strongly suggests that the maj ority of graduate students are unable to demonstrate this level of sophistication in their problem solving approaches (Boost rom, 2005; Creamer & and Associates, 1990; King et al., 1990; King et al., 1990;

PAGE 219

207 Norris, 1985) Recent research using the RCI to assess the reflective judgment level of graduate students the majority of which were doctoral students, indicates that most a re functioning in the lower ranges of q uasi r eflective t hought (Boyd, 2005; MacDonald, 2003; Owen, 2005) Relatively few students in the sample demonstrated the a bility to conduct a thorough and objective analysis, which is the level of complexity required before students are able to progress to reflective thought. The research of Perry (1970) King and Kitchener (1994) Kuhn (Kuhn, Ho, & Adams, 1979) and other developmental cognitive theorists ( Hofer & Pintrich, 1997) indicate that each stage must be fully realized before individuals can progress to new stages. Although there are limits to the amount of time and energy that can be reasonably expended on a comprehensive analysis, these skills prepare students to engage in problem solving strategies a t the next level of cognitive complexity. Unless the course fosters this level of analysis, students in the beginning stages of Quasi Reflective Pattern 2 may regress to Pattern 1 when required to produce a summary analysis drawing conclusions based on pe rsonal opinion or previous experience rather than a studied approach to the case in hand Susan Wolcott, who has used case method instruction and created the Steps for Better Thinking to assess student progress, indicated in a personal email communication (January 18, 2009 ) that she suspects that most case method courses reinforce Reflective Judgment Stage 4 (Performance Pattern 1) thinking. Students are often rewarded for arguing their positions rather than for fully thinking through the problem. Although students are exposed to other

PAGE 220

208 primarily on how to get their own comments into the discussion to earn credit for participation. Although in t his course students were not awarded points for participation, t his observation may shed light on why students who showed evidence of epistemic assumptions consistent with Stage 3 improved, while higher functioning students regressed. Although anecdotal, this observation rings true in light of the experience with case method teaching.. A pervasive focus on what students think and feel about the case with an emphasis on respect and tolerance for the multiplicity of perspectives presented may dissuade Stage 3 thinkers from their autho rity based assumptions, but not encourage the level of analysis that provokes consideration of the credibility of arguments within the given context. Instead, t his level of discussion will reinforce Stage 4 epistemic assumptions that knowledge is uncertain and therefore each person makes decisions that are idiosyncratic to him or herself based on personal values and experiences. In contrast, a focus on the analysis assumptions and requires them to warrant claims, identify persona l biases and the limitations of their proposals may provide the scaffolding and contextual support that students at Quasi Reflective Performance Pattern 1 require in order to move toward Pattern 2. In comparison, students who are using well developed Refle ctive Judgment Stage 5 epistemic assumptions and Quasi Reflective Performance Pattern 2 skills will perceive the need to identify biases and complete a thorough analysis in order to consider all

PAGE 221

209 viable perspectives and to situate decision making within the context in which it occurs While these students may produce a skillful analysis, their tendency to become overwhelmed results in poor decision making. Luitgaarden (2009) notes that what may initially be considered strong critical thinking skills can dege nerate into decision making paralysis as a result of over analyzing the complexity and unpredictability of common social work problems. S tudents at this stage need assistance in identifying principles that can be applied across contexts in order to organiz e their analysis priori ti ze relevant elements, and arrive at a well supported conclusion. I n class discussions that focus on identifying criteria such as the credibility of the evidence, the utility of the solution, the pragmatic need for action, or the p rimacy of certain values over others, may provide the support these students need to become effective decision makers. Rationale rationale statements were generally consis tent with expectations regarding performance patterns and underlying epistemic assumptions. For example, students who used authority as a rationale, which is a pre reflective problem solving approach, scored in Pre Reflective Performance Pattern 0 and had the lowest mean scores of any other group. Students who cited facts that fit an established belief or who used an unsupported opinion also scored in the pre reflective pattern rage In comparison, student who used intuition, personal and professional exp erience or personal values scored in the bottom quarter of Quasi Reflective Performance Pattern 1. This is consistent with the epistemic assumptions of RJM Stage 4 that because

PAGE 222

210 knowledge is uncertain, conclusions about knowledge are determined by the perso nal values, experience or other idiosyncrati c characteristics of the individual. Although still within the bottom half of Quasi Reflective Performance Pattern 1, the highest mean scores were observed among those who used previous knowledge, research and u tility on the final case. Differential Performance by Section An additional, and possibly related, predictor of increased performance was the section of the case method course in which the student was enrolled. While improvement in eight of the sections ra nged from no improvement to 33% improving by .25 at posttest, 67% of students in Section 1( n = 6) and 100% of students in Section 10 ( n = 2) improved by at least .25. A value of .40 for Lambda indicated evidence of a strong association between section and meaningful growth, p = .021. That it, knowing the section that the student was enrolled in improved the chances of predicting whether they would increase their scores between pre and posttest by 40%. Unfortunately, no information was captured regarding fi delity to the case method experience with the case method ranged from no previous experience to many years of experience, the individual professors methods or expertise are unknown. In a similar study regarding the impact of Problem Based Learning on the critical thinking and self efficacy of students, the author concluded that lack of fidelity to the model among

PAGE 223

211 various instructors limited conclusions that could be drawn re garding outcomes (Hesterberg, 2005) Because most case method advocates argue that cognitive growth occurs primarily as result of class discussions (Gill, 2005; Harrington, 1999; Levin 1995; Lundeberg et al., 1999) the expertise of individual professors in facilitating case analysis discussions is relevant. Additionally, the ability to use q uestions to successfully educe ( Wood & Anderson, 2001) In an assessment of problem based learning in an undergraduate social work class, Coleman Collins and Baylis (2007) observed that the role of instructor s and the methods used to ask questions had a significant impact on student learning. Similarly, i n a qualitative study of case method teaching, Allen and Razvi (2006) f ound that the types of questions asked by the professor impacted the level of epistemological understanding elicited from students. Specifically, Evaluativist level questions asked by the instructor (roughly equivalent to RJ Level 5 and/or 6 s (2004) mod el were directly related to Ev a l uativist responses from students. The Socratic questioning encouraged by case method proponents is not an easily acquired skill (Burgoy ne & Mumford, 2001; Hesterberg, 2005) Professors more familiar with traditional instructional methods may be too quick to make their own assertions, They may fail to ask the kind of questions that stimul ate curiosity provoke exploration of alternate perspectives and facilitate the process of problem resolution. According to Boehrer and Linsky ( 1990)

PAGE 224

212 initiate, focus, and Similarly, while creating a learning environment of mutual empathy and respect for diverse perspectives is essential to foster the risk taki ng necessary for a lively and stimulating student centered discussion, a failure to utilize equally important challenging is not an absolutely correct answer. King and Kitchener (1994) and others argue that fostering reflective thinking requires that s tude nts experience challenge s to their current epistemic assumptions within an atmosphere of emotional support. The multiple and nuanced roles of the instructor as plan ner, facilitator, encourager, empathic responder, educators. Thus, s ome professors may have been more adept at providing the contextual support necessary to encourage ep istemological growth than others Correlati on between Rubric S cores and RCI This hypothesis scores on the RCI and th e content analysis rubric was not supported Because the rubric is based on th e Reflective Judgment Model this was an unexpected finding However, t he mean scores for both measures supported the premise that the majority of students in the sample were using epistemic assumptions and skills related to Stage 4 of the Reflective Judgm ent Model. A comparison of individual student scores adjusted to reflect the true

PAGE 225

213 of each other on the two measures suggesting that both measures were capturing the five percent had scores that were more than one half of a stage higher on the RCI than on the rubric; and 17% had higher scores on the rubric than on the RCI. The finding that 25% of stud ents had scores that were more than one half stage above the rubric score is consistent with the relationship between the RJI which is a production task, and the RCI which is a recognition task. However, the only explanation for the 17% who had higher scor es on the rubric than the RCI is either that the two instruments are assessing different kinds of information or that the differences are a result of measurement error. Both explanations are plausible. Interestingly, this percentage approximates a 16% esti mate of measurement error in RJI scores suggested by King and Kitchener (1994) based on a study in which 16% of cases experienced revers als in a retest taken after 4 months. The challenges of accurately assessing abstract reasoning skills are well supporte d in the literature (Blai, 1992; Boostrom, 2005; Brookfield, 1987; Ennis, 1993 ; Facione et al., 2000) ; therefore t he possibility of error in the coding of students papers is likely In reality, the lack of correlation between the two scores is apt to be the result of both measurement error and the different nature of the assessments. It is commonly recognized that ap titude and performance are not necessarily correlated. For example researchers using the Watson Glaser Critical Thinking Appraisal found that students who performed well on the standardized measure did not meet the expected performance level on an essay t est ( Browne, 1978) They argued that while the Watson Glaser measured their ability to recognize valid reasoning strategies, it did not

PAGE 226

214 test the ability of students to apply valid deductive and inductive reasoning to a problem. Similarly, w hile the RCI me asures the epistemic assumptions of respondents based on recognition rather than production, the Steps for Better Thinking rubric measures related problem solving skills via production tasks. Previous research indicates that students tend to perform in the ir functional level on production tasks, but at their optimal level on recognition tasks (King & Kitchener, 2004; Kitchener et al., 1993) Additionally, w hile apability to resolve ill structured problems based on the cognitive complexity of their epistemological perspective, the actual use of the problem solving skills may be based on many other variables These may include factors as diverse as the nature of t he task (decision case), the amount of time and energy available to devote to the task, individual student characteristics, previous feedback regarding the problem solving approaches, the amount of curiosity stimulated by the task, and th e degree to which the task is perc eived as familiar or perplexing. A frequent observation among those who are seeking to encourage critical or deal of persistence effort, and self motivation (Boostrom, 2005; Brookfield, 1987; Ertmer & Dillon, 1998; Paul & Elder, 2006) Based on qualitative research w ith veterinary students participating in a case based course, Ertmer and Dillon (1998) suggest that individual student characteristics such as self regulation, the value attributed to process as opposed to prod uct, and the ability to manage the anxiety of ambiguity and uncertainty, impact how much students gain from a case based course.

PAGE 227

215 Furthermore whether a student engages in the hard work required is likely to be affected by the amount of time and energy ava ilable and the perceived payoff Students who performed in the Performance Pattern 0 range on the initial case were likely to receive feedback from their professors that encouraged increased effort and attention on subsequent papers, while those who scored in the higher range may have received positive feedback, reducing the payoff for greater effo rt at the end of the course. Waning interest, and the work overload commonly experienced by students completing their final semester was also likely to impact opt imal performance. Although factors such as individual characteristics or the amount of time and energy students invest in the problem solving task are not subject to control, the influence of the nature of the task on the problem solving skills induced is of particular interest. According to Dewey (1933), the pivotal component of learning (and therefore thinking) is experience. In order to induce learning, students must encounter a situation that is new (and therefore uncertain and problematic ) and yet which can be sufficiently connected with existing knowledge as to provoke an effective response. Therefore, in order to call forth the problem solving skills that a student is capable of the task must be perceived as prob lematic but not capricious or completely unpredictable. For example, the fact that students cited research as well as the NASW Code of Ethics much more frequently when analyzing the initial case than the final case may indicate that the nature of the initi al case itself elicited more information seeking strategies based on the unfamiliarity of the terrain. In comparison, the fact that students most often cited previous experience and intuition on the final decision case may indicate

PAGE 228

216 that students more often perceived these cases as routine, resulting in the failure to explore the subtle complexities of the case or utilize more complex problem solving strategies. Gill (2008) suggests that the structural complexity of a problem is a property of (p. 254) He defines problem space as : a representation of the cognitive system that will be used to perform a task states of knowledge, (2) operators for changing one state into another, (3) constraints on applying operators, and (4) control knowledge for deciding what knowledge to apply next. He notes that the structural complexity (or the degree of uncertainty) of a task diminishes with ex perience or expertise. Consequently, what one student perceives as an ill structured problem eliciting complex problem solving strategies, may be perceived by another as familiar. It is possible that the more subtle nature of the dil emmas presented by the latter cases triggered the basic decision making strategy identified by Klein (1998, as cited by Luitgaarden, 2009) in which decision makers recognize a familiar situation and immediately take action based s, cues, expectancies and (p. 253 ). Using l, Luitgaarden (2009) explains that when faced w ith a situation that is novel (or ill structured) experts modify this decision making strategy by using mental simulations to evaluate the consequences of alternate actions until a course of action is discovered.

PAGE 229

217 Luitgaarden (2009) argues that this intuitive model is more suitable for soc ial work practice, which is characterized by a high degree of complexity, unpredictability, and the need to make quick decisions, than analytical decision making strategies. A significant concern however, is that novices who lack the experience or experti se to identify relevant cues, plausible goals, and expectancies in the same way that experts do, will tend to misinterpret cues based on superficial familiarity resulting in naive problem assessment and decision making errors. Arguably, the goal of using an experience based learning model, such as case method instruction, is to provide students with the necessary knowledge, skills, and cognitive scaffolding, to learn to utilize the analytical skills necessary for resolving ill structured problems while sti ll protected in a low risk environment. While expectations that practitioners engage in the problem solving strategies espoused by rational choice theory may prove impractical in real world settings, adopting a model for ideal practice that propels novices to make decisions based on intuition without the prerequisite experience will result in decision making errors at the expense of vulnerable populations. Social work educators then must model effective problem framing and decision making, provide opportuni ties for students to grapple with real world problems, and place a high premium on the value of objectivity, open mindedness, reflective thinking, and life long learning.

PAGE 230

218 Limitations Unbalanced Groups The challenges of ensuring internal and external va lidity of research conducted in educational settings has been well documented in the literature (Campbell & Stanley, 1963; Royse et al., 2006) Although ideal, randomized control groups are un usual in educational settings because either all the students are exposed to the educational intervention, or students select courses based on preference and convenience. At the institution from which the sample was drawn, the instructional methods used in the capstone course are a significant aspect of their educational philosophy; therefore, it was not possible to use an equivalent control group because all sections of the capstone course used a case method approach. Because random assignment was not pos sible, a quasi experimental design was used to attempt to eliminate alternative explanations (Royse et al., 2006) Although the statistical analysis indicated that the two groups were not significantly different on any of the measures, the difference in the size of the groups led to an unbalanced design with roughly twice as many students participating in the intervention group as in the comparison group. Additionally, the groups were unbalanced with regard to gender an d race. Although approximately 86% of MSW students are female, in the current study, 95% of the sample was female and only one male participated in the intervention group. While there were seven minority students, only one African American participated in the study. Therefore, although the statistical procedures used are generally robust to

PAGE 231

219 unbalanced designs (Montacalm & Royse, 2002) the findings cannot be said to be representative of the student population, or ge neralized to MSW students at other institutions. An additional concern regarding the variability between groups when (1994) observation that unbal anced sample sizes and differential variability in performance reduces the statistical power of epistemological hypothesis testing resulting in Type II error. Instrumentation One of the primary challenges faced when conducting research to assess changes in reflective judgment as a result of an educational intervention is the lack of measures with sufficient sensitivity to detect changes in epistemology, which research has shown develops slowly To date only one study has reported a significant change in reasoning on the Reflectiv e Judgment Interview following a semester long educational intervention (Kronholm, 1996) Wood and Kadrash (2002) questioned whether the changes detec ted could be attributed to the course, given that the intervention group had significantly lower scores at baseline. Because these results have not been duplicated, they concluded that uivalence of the two groups. While research indicates that reflective judgment is associated with educational experience, King and Kitchener (King & Kitchener, 1994) cautioned that short term interventions were unlik epistemology. Therefore, one significant limitation of the current study was the fact that it involved assessing changes over 12 weeks. Although the case method course may have

PAGE 232

220 fostered reflective thinking, the may have occurred in such a brief period were unlikely to be detected. Sample Size One method for increasing power is to substantially increase sample size. Given the fact that the case method is no t widely used in social work education, it was not possible to generate a larger sample. The setting used was selected because it had a large cohort of students and instructors who are invested in the case method. Although efforts were made to secure as mu ch participation as possible by personally recruiting students, offering incentives, and sending reminders, the sample was significantly smaller than anticipated and further impacted by attrit ion Therefore, because the sample size resulted in lack of suff icient power to detect differences that may have existed, the findings of the quantitative analysis should not be used to draw conclusions regarding the efficacy of the case method approach. Unequivalence of Qualitative Posttest Measure This study used a c oncurrent, mixed methods approach in order to strengthen anticipated weaknesses of the quantitative measure to detect changes in reflective judgment within a short time frame. However, an additional limitation that may have affected internal validity was f act that the initial and final measure for the qualitative analysis may not have been comparable. While all students completed the same initial case, students completed five different cases for the final assignment. Not only were these cases qualitatively different in that the original case presented an obvious ethical

PAGE 233

221 dilemma, while the others were more subtle (as previously discussed), but the time of measurement ranged anywhere from Week 7 to Week 12 on the course calendar. Therefore, some students submi tted their final case analysis when they were only half mean initial scores by case submitted indicates that students with higher pretest scores were more likely to subm it their cases early, while those with lower scores were more likely to submit the final case. Therefore, it is not known whether students who submitted early may have shown improvement had they been required to submit an analysis at the end of the course. The findings suggest that there may have been a correlation between time of submission and improvement. While ten students who submitted the last two only three who submi tted the earlier cases did, and only one was categorized as Fidelity to Case M ethod Model A significant limitation of the current study to assess the effect of a case method course on the reflective judgment of students is the lack of any measures regarding fidelity to the case method. Nine different instructors with varying levels of experience and knowledge regarding the case method approach taught the course. The researcher began with the assumption that t he history of the institution in using the case method, and the ongoing efforts made to collaborate on a weekly basis regarding instructional focus assured uniformity of delivery. However, the finding that students appeared to perform differently based on the section in which they were enrolled called that assumption into

PAGE 234

222 question. Although post hoc comparisons did not validate the omnibus test that indicated significant differences between sections, students in section 1 and 10 were more likely to improve than students in the other sections. Lambda indicated that knowing the section would improve by 40%. It is not clear whether there were differences in the way in which these instructor s applied the course method in their instruction. Differences in personal teaching style, grading, ability to use Socratic questioning, willingness to challenge students to move beyond their current comfort zones, and the ability to facilitate discussion w hich are appropriately challenging and supportive are only a few of the factors that may have impacted differential performance. This once again limits the conclusions that can be drawn from the findings. Lack of Empiri cal Validation of Steps for Better Th inking Rubric Although the Steps for Better Thinking have been used to train faculty across the nation to assess and foster the development of reflective thought in students, it has not been empirically tested. The current study is the first to attempt to correlate the rubric with the RCI. Although the rubric has face validity in that it appears to be clearly related to the Reflective Judgment Model, convergent validity was not established through correlation with the RCI in the current study. Although the findings indicate that it had adequate reliability and internally consistency, the lack of correlation suggests that the two measures are assessing different aspects of cognitive complexity. While the rubric to a range of cognitive complexity, the RCI

PAGE 235

223 assesses their ability to choose among options that most closely mirror their epistemic assumptions about knowledge. Match between Content A nalysis a nd Assignment D escription While the content analysis rubric s ought to examine patterns in students reasoning as they resolved the ill structured dilemma in the decision case, the requirement that they write an executive summary may have limited the evidence of the processes by which students arrived at their conclu sions. Because some students may have failed to articulate important elements of their analysis, their reasoning may have appeared to be more whimsical or superficial than it actually was, had more of their reasoning been apparent. Implications for Social Work Education Importance of Assessing Reflective J udgment The most important finding of this study is that MSW students in the sample were functioning substantially below the level of cognitive complexity that cognitive theorists argue is necessary to ma ke well informed decisions when faced with complex problems which cannot be defined or resolved with certainty (Dewey, 1933; King & Kitchener, 1994; Kuhn et al., 1979) Social w ork educators have cited the many risks associated with reasoning errors in terms of lost potential, human suffering, cultural incompetence, unethical practice, inadequate intervention, and lack of service provision as grounds for the need to prepare gradu ates who can apply critical thinking skills to practice (Gambrill,

PAGE 236

224 1990; Gibbs, 1991; Gibbs, 2007) However, i n spite of concern regarding potential harm to vulnerable p opulations as well as legal risks to social work practitioners, the skills and aptitudes required to engage in effective problem solving and avoid errors in decision making have been little studied in social work literature when compared to other helping professions (Murdach, 1994) Although evidence based practiced (EBP) has been offered as a primary solution to these concerns (Blythe & Witkin, 1992; Gambrill, 1999; Gibbs, 2007) the role of practitioner cognitive complexity and thinking processes required i n the applic ation of EBP has not been studied (McCracken & Marsh, 2008) McCr acken and Marsh (2008) argue that effective use of EBP requires reflective thinking skills in order to interpret and apply evidence appropriately to client concerns within the context of social work practice. Recognition of the highly ambiguous, contextual ized, and multi faceted nature of social work practice (Gambrill, 1990; Gi bbons & Gray, 2004; Murdach, 1994; Sung Chan & Yuen Tsang, 2008; van de Luitgaarden, G. M. J., 2009; Wright & Michaud, 2002) obliges educators to seek methods which will enhance the reasoning skills required to make effective decisions regarding ill struc tured problems as opposed to well structured problems. Based on the extensive research that supports the relationship between reflective judgment and the ability to reason effectively when confronted with ill structured problems, the potential of the Refl ective Judgment Model within social work education to assess and target the reasoning skills of social work students is significant. Research suggests that developmental gaps in the reasoning of those with lower levels of reflective

PAGE 237

225 judgment may restrict t heir ability to engage in best practice. Owen (2005) found that graduate student counselors who had lower scores on the RCI were more likely to use confirmatory bias when searching for or attending to information regarding a client problem than those with higher scores. In the current study, students often looked for facts and information that supported their opinion, while ignoring relevant and potentially hazardous information. For example, on the initial case, the majority of students argued in favor of determination, while ignoring or discounting evidence that called to question his mental competence to make a decision regarding life and death. Similarly only four of eleven students analyzing a final case regarding a teenager refer red for treatment because of depression, inability to eat or sleep and suicidal ideation appraised the potential severity of these symptoms or included a suicide risk assessment in their recommendation. Owen (2 005) also observed the proclivity of students using lower levels of reflective judgment to focus on deficits rather than strengths when assessing client problems G raduate student counselors who scored in the lower levels of the quasi reflective stages ra ted client problems as more severe than those in the higher ranges He attributed this to the inability of indivi duals at the lower levels of RJ to develop a more balanced assessment that This pattern was also observed in the current study, as only students using Quasi Reflective Performance Pattern 2 skills demonstrated the ability to present characters on both sides of an issue in a fair and balanced way. The tendency was to focus on the deficits of those with whom the y did not agree, while ignoring the weaknesses of those with whom they identified.

PAGE 238

226 An additional concern is preliminary evidence that students who demonstrate lower levels of reflective judgment are less likely to practice without discrim in ation with regard to client differences. In a mixed methods study, Guthrie (1996) found a significant correlation between reflective judgment levels as measured by the Reflective Judgment Appraisal (a paper and pencil pre cursor to the RCI) Reflective Judgment Interviews, and Toleran ce for diversity was measured using the New Racism Scale, the Heterosexuals Attitudes towards Lesbians and Gay Men Scale, a nd individual inter views Guthrie concluded that truly tolerant responses to diversity required reasoning that was at least at or above the quasi reflective thinking stage 4 of the Reflective Judgment Model. Ev idence Based Practice and Case Method Instruc tion Several decades of research support the finding that the majority of college seniors and beginning graduate students are functioning within Stage 4 of the Reflective Judgment Model { {462 Hofer,Barbara K. 1997; 443 King, P. M. 1990; 465 Perry, W. G. 19 70}}. This means that students entering graduate programs are likely to believe that knowledge is so uncertain that research and theories have little more value than their own opinion. They are unlikely to support their decisions with research, unless prom pted, and then will use evidence primarily to support their own opinions. The observation in the current study that very few students used research to support their proposed solutions, and that when used, research primarily served to confirm preconceived ideas rather than to explore options, is consistent with previous studies (Havercamp, 1993; King et al., 1990; Owen, 2005; Spengler & Strohmer, 1994) in the

PAGE 239

227 current study to refer vaguely to previous coursework rather than to specific theoretical perspectives is c onsistent with previous research indicating the existence of a significant gap between theory and practi ce in helping professions (Freshwater, 2007; Osmond & O'Connor, 2006; Rosen, Proctor, Morrow Howell, & Staudt, 1995; S chn, 1983; Wilson, 2008) Over the last three decades, numerous studies have concluded that social work practitioners rarely support their clinical decisions with empirical evidence or theory (Gambrill, 1990; Osmond & O'Connor, 2006; Rosen et al., 1995) Following a study that indicated that novice workers were even less likely than more experienced workers to support clinical decisions with theory or research, Rose n (1995) concluded that social work education must prepare graduates to equate social work activity with critical evaluation of available knowledge. Rosen and colleagues concluded that schools of social work neede decisions, explicating the knowledge base and subjecting the decisions to critical scrutiny (1995, p. 521). Although a criticism of evidence based practice is its incongruence wi th the way that experts make practice decisions (Sung Chan & Yuen Tsang, 2008; van de Luitgaarden, G. M. J., 2009) the need to provide novice workers with the missing co nnections between theory and practice that experience will eventually provide cannot be overstated. Until graduates can begin to draw upon the wealth of their own practice experience to make decisions, they must not be lulled into believing that their limi ted life experiences provide them with the necessary expertise to accurately assess and resolve

PAGE 240

228 complex problems. Requiring students to support their positions with evidence and to search for information that contradicts their own points of view is a neces sary aspect of graduate education if students are to become practitioners who will assume responsibility for becoming critical consumers of knowledge as a life long learning strategy. Active learning strategies that focus on collaboration and encourage st udents to assume responsibility for their own learning continue to hold the best promise for fostering critical and reflective thinking skills in students (Boyer Commission on Educating Undergraduates in the Research University, 1998; Brookfield, 1987; Browne & Freeman, 2000; Coleman et al., 2007; Holten & Baldwin, 2000; King & Kitchener, 1994; Steiner, Stromwall, Brzuzy, & Gerdes, 1999) The first recommendation for transforming undergraduate education in answer to the finding of the Boyer Commission (1998) that most graduates were unable to integrate course learning with practice was to based learning appro ach in which students work collaboratively to seek the necessary information to solve open ended problems was one option recommended by the Boyer Commission (1998) for engaging students in the process of inquiry. This strategy is closely related to case use research to fill the gaps in their existing knowledge (Altshuler & Bosch, 2003; Gibbons & Gray, 2002) Rowe (2007b) suggests that experience based research is not only engaging and enjoyable but also involves analysis and creative thinking. Adding a collaborative research component to the use of decision case s may increase

PAGE 241

229 value of research in assessing problems and formulating action plans as well as the likelihood that student s will use more objective information search strategies. While searching for research that encompasses all potential perspectives or addresses all relevant contextual factors in a decision case may be implausible as an individual undertaking, having studen ts divide the work in small groups after identifying relevant issues encourages an interchange of ideas and the appraisal of previously unconsidered alternatives Tailoring Assignments to the Development Level of Students In order to facilitate reflectiv e thought, educators must be aware of the epistemological beliefs and related problem solving approaches that may hinder student progress. An important observation made in this study is that it may be possible to actually reinforce lower levels of reflecti ve thought by failing to design assignments that take in to account the actual cognitive developmental level of students in the course. While students in the course were at the lower levels of quasi reflective judgment, the course assignment was designed t o promote movement from Quasi Reflective Pattern 5 to Reflective Pattern 6. Although this level of reflective judgment is cl early the goal of graduate education, the developmental nature of cognitive complexity has been well established by numerous cognit ive theorists (Baxter Magolda,1990; Hofer, 1997; King, 1994; Kuhn, 1979; Perry, 1970; Piaget, 1964) Proponents of the various stage models of cognitive complexity argue that movement through each of the stages is imperative before

PAGE 242

230 progression to the nex t stage is possible. Movement to each new stage challenges h greater levels of uncertainty and responsibility. Because students often resist the transition, a mixture of support and challenge is required to facilitate movement from one stage to the next (Dewey, 1933; Boostrom, 2005; Perry, 1970; Piaget, 1964) MSW curriculums should be designed with the assumption that entering students will be functioning primarily in Reflective Judgment Model St age 4 and that the majority of students do not have the skills to reason effectively through ill structured problems. Several studies have reported positive outcomes from intentionally including content on critical thinking (Mumm & Kersting, 1997; Plath et al., 1999) however, as a stand alone approach it has fallen short of the expected outcomes (Kersting & Mumm, 2001) The integration of both acti ve learning strategies such as case method or problem based learning with specific content on effective reasoning and decision making may be a more effective approach (Bellefeuille, 200 6; Plath et al., 1999) Assignments in the first semester should provide the contextual support necessary for these students to examine different points of view on various topics reflectively. King and Kitchener (1994) note that the type of synthesized conclusions represented in most textbooks will not serve this purpose. They suggest that students read widely on topics, including discussion of alternate views, before an educator offers their own interpretation. The use of evidence to support personal opinions must be strongly encouraged without shaming students into personal retreat and withdrawal.

PAGE 243

231 As students move into Stage 5 of the RJM they must be encouraged to use analytic al skills to view problems more syst emically and with a more balanced and neutral perspective. Educators can facilitate growth by understand ing relative weakness in coming to conclusions as a function of their cognitive developmental level and avoid ing harsh penalties for overly le ngthy analyses and lack of strong conclusions. Each semester should intentionally include assignments that gradually increase the level of cognitive complexity required while creating environments in which students can experience a measure of cognitive dis sonance within a supportive enviro nment. In addition, efforts should include address ing the unique developmental needs of students who are at the low end by pairing them with higher functioning students, and at the high end by calling on them to demonstrat e higher level skills in class discussion Social Work Values and Ethical Decision Making A major aspect of social work education is the socialization of students into the erous thing. Content analysis of the decision case papers indicated that students who had not developed the cognitive schemas for understanding issues contextually were likely to apply the code of ethics simplistically in a way that puts their clients at r isk. Dewey (1910) made the following observation: Genuine ignorance is profitable because it is likely to be accompanied by humility, curiosity, and open mindedness; whereas ability to repeat catch

PAGE 244

232 phrases, cant te rms, familiar propositions, gives the conceit of learning and coats the mind with a varnish, waterproof to new ideas. (p. 177). Social work educators must guard against arming students with a superficial understanding of the social work values that preclud es a thorough analysis of the issues. For example, prized social work values such as self determination and confidentiality must be balanced against the duty to protect those who may be unable to protect themselves. The fact that students did not use autho ritative approaches to resolve the final decision case analyses provides preliminary support for the effectiveness of case method teaching to facilitate the transition from viewing social work values as absolutes to guidelines that must be placed within th e contextual realities in which graduates will be required to apply them. Decision cases can provide pre practice learning opportunities to practice ethical decision making in the face of competing values and complex realities. Recommendations for Targeti ng Reflective Judgment through Case Method Instruction Based on the observations of this study, the following specific recommendations may facilitate the fostering of reflective thought using case method instruction. Determine the baseline level of reflec tive judgment of entering MSW students. Based on extensive research on the reflective judgment model, social work educators should assume that beginning graduate students are functioning at the lower levels of quasi reflective thought and that some are sti ll functioning in the pre reflective levels. deal with uncertainty, the assumptions t hey make about source s of knowledge, and the

PAGE 245

233 methods they use to justify their dec isions in early assignments (King & Kitchener, 1994) 1 Develop a case based curriculum that is developmental and spans at least two semesters and preferably four. Current research on reflective judgment strongly sug gests influenced by education, it develops slowly An educational intervention that targets reasoning skills over one semester is not likely to yield measurable results (King & Kitchener, 1994; Wolcott 2006a) Wolcott (2006a) suggests integrating critical thinking across the curriculum and expecting that it may take as long as two years to develop strong Stage 5 skills for underg raduates. Graduate programs aspiring to develop strong Stage 6 skills might expect a similar trajectory. 2 Focus initially on identifying uncertainty, and f raming problems to foster movement from RJM Stage 3 to RJM Stage 4 Before students can effectively an alyze problems and make decisions, they must be able to perceive the inherent ambigu ity of ill structured problems and frame problems accurately While the majority of graduate students will be able to distinguish a well structured problem from an ill s tru ctured problem, some will need help dist inguishing between the problem solving strategies that are appr opriate for each 3 Because research suggests that beginning graduate students are likely to use RJM Stage 4 epistemic assumptions, resulting in predomina ntly one sided

PAGE 246

234 approaches to problem solving, a significant emphasis should be placed on encouraging students to identify and control biases. Embedding questions in the case analysis that require s students to identify biases is one possibility. R equiring s tudents to take a position on an issue and then write an objective p aper taking an opposing view is one suggestion offered by Wolcott (2006a) in her Faculty Handbook. 4 Require students to support their positions and claims with evidence, and to support thei r recommendations based on applicable theory and empirical evidence (Gibbs, 2007; Rowe, 2007b) Although open ended problems require students to make judgments rather than find correct solutio ns, in order to foster reflective judgment, students must understand the relationship between evidence and justification of their positions. While students using Reflective Judgment Stage 4 skills are comfortable making judgments about problems, they do no t see the need to warrant their claims or to evaluate the relative strength or weakness of the evidence used. Requiring them to support their claims and to evaluate the strength of the evidence they use augments their understanding of the relationship betw een knowledge claims and sound evidence. This may be encouraged within small or large class discussions as well as in written assignments. 5 In order to foster Stage 5 analysis skills, require students to complete a comprehensive, objective analysis and to search for disconfirming evidence as well as confirming evidence ( Kitc hener & Fischer, 1990; Kitchener, 1994)

PAGE 247

235 While requiring frequent comprehensive analyses can be daunting both for students and instructors who must assess the work, spending more time on fewer cases in earlier course s in the curriculum can facilitate this level of skill development. Lundeberg (1999) reported that repeate d exposures to a case strengthened theoretical and practical knowledge as well as reasoning and metacognition. Requiring that students complete less frequent case analyses after the class has had time to process the issues may be beneficial as well ( Levin 1995) 6 In order to facilitate movement from Stage 5 ambivalence to more effective decision making, educators may focus on identify ing and prioritizing values that can be used to judge across alternatives in spite of contextual considerations. Once students have become adept at performing a comprehensive analysis, class discussions can be used to help students identify principles, or values that can be applied across contexts to choose among viable alternatives ( Wolcott 2006a) At this stage, more of class discussion time should focus on problem resolution than problem analysis. 7 Consider pairing higher functioning students with lower functioning students for collaborative smal l group work. Social learning theory (Bandura & Walters, 1963) provides support for the powerful role of modeling in the development of new skills. Bidel and Fischer (1992 as cited in King & Kitchener, 1994 ) suggest that students may be able to function at an even higher level than their optimal level when a coach models or assists the

PAGE 248

236 learner with the new behavior. Requiring higher skills such as identifying and evaluating assumptions, limitations of proposed soluti ons, and warranting claims with evidence may inhibit the likelihood that stronger students will regress to Stage 4 reasoning 8 Require students to engage in meta analysis reflecting on their own thinking Questions regarding the processes that students u sed to arrive at their conclusions can b e embedded in the case analysis. A lternately, asking students to reflect on how the case discussion may or may not have affected their thinking about the case provides insight into the students own thinking processes as well as providing feedback to the professor regarding the influence of class discussions. 9 discussion positions reflect enhanced ability to consider the viability of multiple pers pectives, and reflective thinking attitudes such as humility, open mindedness, objectivity, and self awareness. Moje, Remillard, and Southerland (1999) repor ted that while students indicated that they enjoyed the case discussions and instructors assumed th at rich discussion indicated meaningful learning was taking place, an analysis of the interactions during the case discussion told a different story. These researchers concluded that much as developed assumption s

PAGE 249

237 case discussions to determine whether they are meeting curricular goals. Instructors who find tha t the discussions are reinforcing existing beliefs and problem solving strategies may need to make adjustments in the type of Socratic questions used to facilitate discussion based on the RJ levels demonstrated by students. 10 Provide instructors with trainin g in case method teaching, Socratic questioning, and assessment of the development of reflective judgment (Burgoyne & Mumford, 2001; Wood & Anderson, 2001) A frequent observation in academia is that although instructors of higher learning are often experts in their respect ive disciplines, they often have no formal training in pedagogy. The unique challenges of case method instruction discussed previously make it imperative that deans and directo rs of social work programs provide instructors with training rather than assuming that the process is intuitive. While requiring reading and preparation may be beneficial, encouraging instructors to participate in a case method course facilitated by an exp erienced and effective case method teacher is ideal. Recommendations for Further R esearch This study was the first to apply the Reflective Judgment Model to social work students. While the results of this study do not indicate that the RCI is an effective measure for assessing the effect of short term educational interventions, it is recommended as a measure for assessing the baseline level of reflective judgment. In

PAGE 250

238 addition, a follow up longitudinal study, which uses the RCI at the beginning and end of th e MSW curriculum, is likely to provide a more accurate representation of reflective judgment growth. Efforts to assess the effects of case method teaching objectively should continue, ity to think critically may not match up with their perform ance on objective measures ( Hesterberg, 2005; Owen, 2005) A replication should ensure that pre and posttest decision cases are comparable, and that all part icipants submit their analyses at the same point in the course. Additionally, a replication of this study should include a measure of fidelity to case method teaching with a focus on skills that are related to reflective thinking outcomes. Efforts should be made to identify differences in style, experience, use of questions, and assessment of student work. Because the group dynamics and role of the instructor are important factors in student learning, an analysis of the actual class discussions may also yi eld rich information regarding instructor practices that foster ref lective judgment through student centered case discussions. B epistemic assumptions, follow up interviews or surveys should be utilized to capture a more accurate perception of the underlying epistemological framework that students are using. Questions should address students acknowle dgement of uncertainty, their explanations for disagreement between experts abo ut the issues addressed in the decision cases and their approach es to resolving competing knowledge claims (King & Kitchener, 1994)

PAGE 251

239 The following table addresses the limitations of the curre nt study and specific mo difications that may be used to further inquiry regarding the ability of case method instruction to foster the ability to use reflective judgment in social work students. Table 10. Recommendations for Future Research Limitation Recommended Modification Small sample size Incorporate assessment measures into course design so that all students participate in pre post measures ; If it is not possible to substantially increase sample size, increase time bet ween testing Unbalanced design Consider a stratified sample, oversampling males and minorities Use same strategies to recruit comparison group as intervention group; If possible, random assignment to case based course or traditional course. Time Frame T oo Short Extend case method instruction over two semesters with assessment measures at beginning of each semester and end of second semester Unequivalence of Qualitative Post Test Measures Have all students complete the same case at the same point in the semester. Ensure that the case is similar in terms of level of uncertainty and the perplexity it is likely to illicit from graduate students Lack of measure regarding fidelity to the case method Include a measure regarding instructor experience and adher ence to the principles of case method teaching Consider adding a content analysis of case discussions to explore whether characteristics of the in class discussion influence learning outcomes and reflective thinking levels. Lack of mat ch between content a nalysis and assignment description Include more comprehensive analysis which will yield richer information regarding students problem solving strategies Lack of evidence regarding meta cognitive analysis Require students to reflect on own thinking includi ng reflecting on process, rationale, biases and limitations of their proposed solutions. Lack of measures regarding impact of discussion on students processes Include a measure that requires students to periodically reflect on specific changes in their th inking regarding a case following case discussion Lack of qualitative measure regarding epistemic assumptions Include a questionnaire or personal interview that captures through pre, mid, and post case analyses

PAGE 252

240 As the gatekeepers of the profession, social work educators are responsible for developing strategies that will prepare graduates to think reflectively when faced with complex, multi faceted problems. The purpose of this study was to evaluate the efficacy of th e case method of instruction in fostering the development of reflective judgment skills. While this study was unable to clearly validate the efficacy of the case method as a teaching strategy that accomplishes this important goal, the lessons learned can be used to better assess course outcomes in the future. Additionally, this study sheds light on the cognitive skills and thinking processes that graduate students in social work are likely to employ as they enter the world of practice. In light of professional values such as respect for diversity, strengths based practice, and competence, findings that lower levels of reflective judgment are associated with intolerance (Guthrie, 1996) the inability to use evidence consistently to justify conclusions (King & Kitchener, 2002) the tendency to focus on client deficits rather than strengths,and engage in confirmatory bias (Owen, 2005) warrant concern. The observations made in this study highlight the importance of assessing and fostering the reflective judgment of MSW students and providing them with guided practice in the decision making s kills that are vital to effective and ethical practice.

PAGE 253

241 References Allegretti, C. L., & Frederick, J. N. (1995). A model for thinking critically about ethical issues. Teaching of Psychology, 22 (1), 46 48. Allen, J. D. (1995). A researc h study to investigate the development of reflective thought processes of preservice teachers through the use of case studies in educational psychology courses. In H. E. Klein (Ed.), Case method research and application: The art of interactive teaching wit h cases, simulations, games and other interactive methods (pp. 3 13). Boston, MA: World Association for Case Method Research and Application. Allen, J. D., & Razvi, S. (2006). Students' perspectives, levels of epistemological understanding, and critical thinking dispositions related to the use of case studies in an educational psychology course. Annual Meeting of the American Educational Research Association, San Francisco. Alter, C., & Egan, M. (1997). Logic modeling: A tool for teaching critical thinki ng in social work practice. Journal of Social Work Education, 33 (1), 85 102. Altshuler, S. J., & Bosch, L. A. (2003). Problem based learning in social work education. Journal of Teaching in Social Work, 23 (1/2), 201 215. Bailey, R. W. (2000). Visiting th e music classroom through the use of the case method in the preparation of senior level music teacher education students: A study of reflective judgment. (D.M.E., University of Cincinnati). Retrieved from

PAGE 254

242 http://proquest.umi.com.ezproxy.lib.usf.edu/pqdweb?did=728323971&Fmt=7&clie ntId=65345&RQT=309&VName=PQD Bandura, A., & Walters, R. H. (1963). Social learning and personality de velopment New York: Holt, Rinehart and Winston. Barnes, L. B., Christensen, C. R., & Hansen, A. J. (1994). Teaching and the case method Boston, MA: Harvard Business School Press. Barnett, C. S., Tyson, P. A., & San Francisco, W. (1999). Case methods an d teacher change: Shifting authority to build autonomy. In M. A. Lundeberg, B. B. Levin & H. L. Harrington (Eds.), Who learns what from cases and how? The research base for teaching with cases (pp. 53 71). Mahwah, NJ: Lawrence Erlbaum. Baxter Magolda, M. B. (1990). The impact of the freshman year on epistemological development: Gender differences. Review of Higher Education, 13 259 284. Bellefeuille, G. L. (2006). Rethinking reflective practice education in social work education: A blended constructivist and objectivist instructional design strategy for a web based child welfare practice course. Journal of Social Work Education, 42 (1), 85 103. Berg, B. L. (1989). Qualitative research methods for the social sciences. Needham Heights, MA: Allyn & Bacon. B issell, A. N., & Lemons, P. P. (2006). A new method for assessing critical thinking in the classroom. Bioscience, 56 66 72. Blai, B., Jr. (1992). Assessment of critical thinking in postsecondary education No. ED 351 954 ) ERIC

PAGE 255

243 Blythe, B., & Witkin, S. L (1992). Should undergraduate and graduate social work students be taught to conduct empirically based practice? Yes Journal of Social Work Education, 28 (3), 260 263. Boehrer, J., & Linsky, M. (1990). Teaching with cases: Learning to question. In M. D. Svinicki (Ed.), The changing face of college teaching (pp. 41 57). San Francisco, CA: Jossey Bass. Boostrom, R. (2005). Thinking: The foundation of critical and creative learning in the classroom New York: Teachers College Press. Boyd, L. (2005). Develo pment of reflective judgment in the health professions clinical curriculum. (Ed.D., Portland State University). Boyer Commission on Educating Undergraduates in the Research University. (1998). Reinventing undergraduate education: A blueprint for research university Stony Brook, NY: State University of N ew York at Stony Brook. Brabeck, M. M., & Welfel, E. R. (1985). Counseling theory: Understanding the trend toward eclecticism from a developmental perspective. Journal of Counseling & Development 63 (6), 343. Brabeck, M. M. K. (1980). The relationship between critical thinking skills and development of reflective judgment among adolescent and adult women. (Ph.D., University of Minnesota). Brookfield, S. D. (1987). Developing critical thinkers. c hallenging adults to explore alternative ways of thinking and acting Milton Keynes: Open University Press.

PAGE 256

244 Browne, M. N., & Freeman, K. (2000). Distinguishing features of critical thinking classrooms. Teaching in Higher Education, 5 (3), 301 309. Burgoyn e, J., & Mumford, A. (2001). Learning from the case method: A report to the European case clearing house. No. RP0301)The European Case Clearing House. Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi experimental designs for research. Chic ago: Rand McNally. Cassarino, C. A. (2006). The impact of problem based learning on critical thinking and problem solving skills. (Ed.D., Nova Southeastern University). Cearley, S., & Runnion, V. M. (2008). ResponsAbilities. In T. A. Wolfer & V. M. Runni on, Dying, death, and bereavement in social work practice: Decision cases for advanced practice (pp. 40 48). New York: Columbia University Press. Choi, D. Y., & Kiesner, F. (2007). Homeboy Industries: An incubator of hope and business. Entrepreneurship The ory and Practice, 31(5), 1 22. Cohen, J. (1988). Statistical power analysis for the behavioral sciences Hillsdale, NJ: Lawrence Earlbaum Associates. Coleman, H., Collins, D., & Baylis, P. (2007). "You didn't throw us to the wolves": Problem based learnin g in a social work family class. The Journal of Baccalaureate Social Work, 12 (2), 98 113. Coleman, H., Rogers, G., & King, J. (2002). Using portfolios to stimulate critical thinking in social work education. Social Work Education, 21 (5), 583 595. Corich, S., & Kinshuk, H., L.M. (2006). Measuring critical thinking within discussion forums using a computerised content analysis tool. Paper presented at the

PAGE 257

245 International Research Conference, Lancaster, UK. Retrieved from http://www.networkedlearningconference.org.uk/past/nlc2006/info/confpapers.htm Cossom, J. (1991). Teaching from cases: Education for critical thinking. Journal of Teaching in Social Work, 5 (1), 139 155. Council on Social Work Education. (2007a). 2006 S tatistics on social work education in the United States : A summary Alexandria, VA: Council on Social Work Education. Council on Social Work Education. (2007b). Statistics o n social work edu cation in the United S tates: 2004 Alexandria, VA: Council on Social Work Education. Creamer, D. G., & and Associates (Eds.). (1990). College student development: Theory and practice for the 1990s Alexandria, VA: American College Personnel Association. Crook, J. A. (2001). How do expert mental health nurses make on the spot clinical decisions? A review of the literature. Journal of Psychiatric and Mental Health Nursing, 8 (1), 1 6. Denzin, N. K. (1978). The research act New York: McGraw Hill. Dewey, J. (1910). How we think Lexington, MA: Heath. Dewey, J. (1933). How we think: A restatement of the relation of reflective thinking to the educational process Lexington, MA: Heath. Dillman, D. (2000). Mail and internet surveys: The tailored design method. New York: John Wiley & Sons. Ennis, R. (1991). Critical thinking: A streamlined conception. Teaching Philosophy, 14 Ennis, R. H. (1993). Critical thinking assessment. Theory into Practice, 32 (3), 179 186.

PAGE 258

246 Ertmer, P. A., & Dillon, D. R. (1998). "Shootin g in the dark versus "breaking it down": Understanding students' approaches to case based instruction. Qualitative Studies in Education, 11 (4), 605 622. Ertmer, P. A., Newby, T. J., & MacDougall, M. (1996). Students' responses and approaches to case base d instruction: The role of reflective self regulation. American Educational Research Journal, 33 (3), 719 752. Facione, P. A. (1998). "The Delphi report" critical thinking: A statement of expert consensus for purposes of educational assessment and instruct ion California Academic Press. Retrieved from www.insightassessment.com/pdf_files/DEXadobe.PDF Facione, P. A., Facione, N. C., & Giancarlo, C. A. (2000). The disposition to ward critical thinking: Its character, measurement, and relationship to critical thinking skill. Informal Logic, 20 (1), 61 84. Retrieved from JSTOR database. Fauri, D. P., Wernet, S. P., & Netting, F. E. (Eds.). (2000). Cases in macro social work practice Needham Heights, MA: Allyn & Bacon. Fischer, K. W., & Pipp, S. L. (1984). Processes of cognitive development: Optimal level and skill acquisition. In R. J. Sternberg (Ed.), Mechanisms of cognitive development (pp. 45 80). New York: Freeman. Fischer, K. W., & Pruyne, E. (2002). Reflective thinking in adulthood: Emergence, development, and variation. In J. Demick, & C. Andreoletti (Eds.), Handbook of Adult D evelopment (pp. 169 198). New York: Plenum.

PAGE 259

247 Flateby, T. L., & Metzfer, E. A. (2005). Cognitive lev el and quality of writing assessment: Building better thought through better writing Tampa, FL: University of South Florida. Fleiss, J. L. (1981). Statistical methods for rates and proportions. (2nd Ed. ed.). New York: John Wiley and Sons. Freshwater, D (2007). Teaching and learning reflective practice. In T. Stickley, T. Basset, T. Stickley & T. Basset (Eds.), Teaching Mental H ealth. (pp. 265 274). New York, NY US: John Wiley & Sons Ltd. Gambrill, E. (1990). Critical thinking in clinical practice: Imp roving the accuracy of judgments and decisions about clients San Francisco: Jossey Bass Publishers. Gambrill, E. (1997). Social work practice: A critical thinker's guide New York: Oxford University Press. Gambrill, E. (1999). Evidence based practice: A n alternative to authority based practice. Families in Society, 80 (4), 341 350. Gambrill, E. (2006). Social work practice: A critical thinker's guide (2nd ed.). New York: Oxford University Press. Garvin, D. A. (2003). Making the case: Professional educat ion for the world of practice. Harvard Magazine, 106 (1), 56 66. Retrieved from http://harvardmagazine.com/2003/09/making the case.html Gibbons, J., & Gray, M. (2002). An int egrated and experience based approach to social work education: The Newcastle model. Social Work Education, 21 (5), 529 549.

PAGE 260

248 Gibbons, J., & Gray, M. (2004). Critical thinking as integral to social work practice. Journal of Teaching in Social Work, 24 (1/2), 19 38. Gibbs, L. E. (1991). Scientific reasoning for social workers: Bridging the gap between research and practice. New York: MacMillan. Gibbs, L. E. (2007). Applying research to making life affecting judgments and decisions. Research on Social Work Pr actice, 17 143 150. doi:10.1177/1049731506294802 Gibbs, L. E., & Gambrill, E. (1999). Critical thinking for social workers: Exercises for the helping professions Thousand Oaks, CA: Pine Forge Press. Gibbs, L. E., Gambrill, E. D., Keniston, A., Peden, B ., Blakemore, J., Begun, A., et al. (1995). A measure of critical thinking about practice. Research on Social Work Practice, 5 (2), 193 204. Gill, T. G. (2005). Teaching brief: A protocol for online case discussions. Decision Sciences Journal of Innovative Education, 3 (1), 143. Gill, T. G. (2006). Why should I participate in a case study: Answers to ten frequently asked questions. Unpublished manuscript. Gill, T. G. & Hicks,R. (2006). Task complexity and informing science: A synthesis. Informing Science: the International Journal of an Emerging Transdiscipline, 9, 1 30. Retrieved from http://inform.nu/Articles/Vol19/v9p001 030Gill46.pdf. Gill, T. G. (2008). Structural complexity and effective informing. Informing Science: The International Journal of an Eme rging Transdiscipline, 11 doi: http://inform.nu/Articles/Vol11/ISJv11p253 279Gill223.pdf

PAGE 261

249 Glaser, R. (1984). Education and thinking: The role of knowledge. American Psycholo gist, 39 (1), 83 104. Gould, N. (1996). Introduction: Social work education and the "crisis of the professions". In N. Gould, & I. Taylor (Eds.), Reflective learning for social work: Research, theory and practice (pp. 1 10). Aldershot, England: Arena Ashga te Publishing Limited. Graham, P. T., & Cline, P. C. (1980). The case method: A basic teaching approach. Theory into Practice, 19 (2 ) Teaching Methods: Learning Applications), 112 116. Gray, K. A., Wolfer, T. A., & Maas, C. (2006). The decision case meth od: Teaching and training for grassroots community organizing. Journal of Community Practice, 14 (4), 91 112. Gray, M., & Gibbons, J. (2007). There are no answers, only choices: Teaching ethical decision making in social work. Australian Social Work, 60 (2) 222 238. Grossman, R. (1994). Encouraging critical thinking using the case study method and cooperative learning techniques. Journal on Excellence in College Teaching, 5 (1), 7 20. Gullahorn, J. T. (1959). Teaching by the case method. The School Review, 67 (4), 448 460. Guthrie, V. L. (1996). The relationship of levels of intellectual development and levels of tolerance for diversity among college students. (Ph.D., Bowling Green State University).

PAGE 262

250 Halonen, J. S. (1995). Demystifying critical thinking. T eaching of Psychology, 22 (1), 75 81. Halx, M. D., & Reybold, L. E. (2005). A pedagogy of force: Faculty perspectives of critical thinking capacity in undergraduate students. JGE: The Journal of General Education, 54 (4), 293 315. Harrington, H. L. (1995). Fostering reasoned decisions: Cased based pedagogy and the professional development of teachers. Teaching and Teacher Education, 11 (3), 203 214. Harrington, H. L. (1999). Case analyses as a performance of thought. In M. A. Lundeberg, B. B. Levin & H. L. Harrington (Eds.), Who learns what from cases and how? the research base for teaching and learning with cases. (pp. 29 47). Mahwah, NJ: Laurence Earlbaum Associates. Harrington, H. L., Quinn Leering, K., & Hodson, L. (1996). Written case analyses and crit ical reflection. Teaching and Teacher Education, 12 (1), 25 37. Haulotte, S. M., & Kretzschmar, J. A. (Eds.). (2001). Case scenarios for teaching and learning social work practice Alexandria, VA: Council on Social Work Education, Inc. Havercamp, B. E. (1 993). Confirmatory bias in hypothesis testing for client identified and counselor self generated hypothesis testing for client identified and counselor self generated hyptheses. Journal of Counseling Psychology, 40 (3), 303 315.

PAGE 263

251 Hesterberg, L. J. (2005). E valuation of a problem based learning practice course: Do self efficacy, critical thinking, and assessment skills improve? (Ph.D., University of Kentucky). Hofer, B. K., & Pintrich, P. R. (Eds.). (2002). Personal epistemology: The psychology of beliefs ab out knowledge and knowing Mahwah, NJ: Lawrence Erlbaum Associates. Hofer, B. K., & Pintrich, P. R. (1997). The development of epistemological theories: Beliefs about knowledge and knowing and their relation to learning. Review of Educational Research, 67 (1), 88 140. Holsti, O. R. (1968). Content analysis. In E. A. G. Lindzey (Ed.), The handbook of social psychology (Second ed., pp. 596 692). Reading, MA: Addison Wesley. Holten, E. F., & Baldwin, T. T. (2000). Making transfer happen: An action perspectiv e on learning transfer systems. In E. F. Holten, T. T. Baldwin & S. S. Nacquin (Eds.), Monograph of the academy of human resources development Jones, K. (2003). Making the case for the case method in graduate social work education. Journal of Teaching i n Social Work, 23 (1/2), 183 200. Jones, K. (2005). Widening the lens: The efficacy of the case method in helping direct practice MSW students understand and apply mezzo and macro dimensions of practice. Social Work Education, 24 (2), 197 211. Kersting, R. C., & Mumm, A. M. (2001). Are we teaching critical thinking in the classroom? The Journal of Baccalaureate Social Work, 7 (1), 53 67.

PAGE 264

252 King, P. M., & Kitchener, K. S. (1994). Developing reflective judgment: Understanding and promoting intellectual growth a nd critical thinking in adolescents and adults San Francisco: Jossey Bass Inc., Publishers. King, P. M., & Kitchener, K. S. (2002). In Hofer B. K., Pintrich P. R. (Eds.), The reflective judgment model: Twenty years of research on epistemic cognition Mah wah, NJ, US: Lawrence Erlbaum Associates Publishers. King, P. M., & Kitchener, K. S. (2004). Reflective judgment: Theory and research on the development of epistemic assumptions through adulthood. Educational Psychologist, 39 (1), 5 18. King, P. M., Linds ay, N. K. & Brown, M. K. Reflective judgment. Retrieved August 18, 2007, from http://www.umich.edu/~refjudg/reflectivejudgmentmodel.html King, P. M., Wood, P. K., & Mines R. A. (1990). Critical thinking among college and graduate students. Review of Higher Education, 13 (2), 168 183. Kitchener, K. S., & Fischer, K. W. (1990). A skill approach to the development of reflective thinking. Contributions to Human Development, 2 1 48 62. Kitchener, K. S., & King, P. M. (1990). In Commons M. L., Armon C., Kohlberg L., Richards F. A. and Grotzer T. A. (Eds.), The reflective judgment model: Ten years of research New York, NY, England: Praeger Publishers. Kitchener, K. S., Lynch, C. L., Fischer, K. W., & Wood, P. K. (1993). Developmental range of reflective judgment: The effect of contextual support and practice on developmental stage. Developmental Psychology, 29 (5), 893 906.

PAGE 265

253 Kitchener, K. S. (1994). Assessing reflective thinking within curricular contexts (ERIC No. ED415751). Washington, D.C.: Fund for the Improvement of Postsecondary Education. Kohlberg, L. (1969). Stages in the development of moral thought and action New York: Holt, Rineheart & Winston. Kronholm, M. (1996). The impact of developmental instruction on reflective judgment. Review of Higher Education, 19 199 225. Kuhn, D., & Dean, J., D. (2004). Metacognition: A bridge between cognitive psychology and educational practice. Theory into Practice, 43 (4), 268 273. Kuhn, D., Ho, V., & Adams, C. (1979). Formal reasoning among pre and late adolescents. Child Development, 50 (4), 1128 1135. Lacy, S., & Riffe, D. (1996). Sampling error and selecting intercoder reliability samples for nominal content categories. Journal ism and Mass Communication Quarterly, 73 (4), 963 973. Laird, J. (1993). Revisioning social work education: A social constructionist approach. Journal of Teaching in Social Work (Entire Issue), 8 (1/2), 1 10. LeCroy, C. W. (Ed.). (1999). Case studies in so cial work practice (2nd ed.). Pacific Grove, CA: Brooks/Cole Publishing Co. Levin, B. B. (1995). Using the case method in teacher education: The role of discussion and experience in about cases. Teaching and Teacher Education, 11 (1), 63 79. Retrieved from Science Direct database.

PAGE 266

254 Levin, B. B. (1993). Using the case method in teacher education: The role of discussion and experience in teachers' thinking about cases. (Ph.D., University of California, Berkeley). Lister, P. G. (2004). Deve loping students' ability to critically analyze social work theory Glasgow: Social Policy and Social Work. Retrieved from http://www.swap.ac.uk/about/miniproject2.asp Lombard, M., Snyd er Duch, J. & Bracken, C. C. (2005). Practical resources for assessing and reporting intercoder reliability in content analysis research projects. Retrieved July 7, 2008, from http://www.t emple.edu/sct/mmc/reliability/ Lundeberg, M. A. (1999). Discovering teaching and learning through cases. In M. A. Lundeberg, B. B. Levin & H. L. Harrington (Eds.), Who learns what from cases and how? the research base for teaching and learning with cases (pp. 3 27). Mahwah, NJ: Lawrence Erlbaum Associates. Lundeberg, M. A., & Fawver, J. E. (1994). Thinking like a teacher: Encouraging cognitive growth in case analysis. Journal of Teacher Education, 45 (5), 289 297. Lundeberg, M. A., Levin, B. B., & Harri ngton, H. L. (Eds.). (1999). Who learns what from cases and how: The research base for teaching and learning with cases Mahwah, N.J.: Earlbaum Associates. Lynch, D., Vernon, R. F., & Smith, M. L. (2001). Critical thinking and the web. Journal of Social W ork Education, 37 (2), 381 386. Lynn, L. E. (1999). Teaching and learning with cases: A guidebook U.S.; New York: Seven Bridges Press.

PAGE 267

255 Macaulay, C., & Cree, V. E. (1999). Transfer of learning: Concept and process. Social Work Education, 18 (2), 183 194. MacDonald, M. R. (2003). The contribution of education, experience, and personal characteristics on the reflective judgment of students preparing for school administration. (Ph.D., Andrews University). McBride, R. E., Xiang, P., & Wittenburg, D. (2002). D ispositions toward critical thinking: The preservice teacher's perspective. Teachers & Teaching, 8 (1), 29 40. McCracken, S. G., & Marsh, J. C. (2008). Practitioner expertise in evidence based practice decision making. Research on Social Work Practice, 18 ( 4), 301 310. McDade, S. A. (1995). Case study pedagogy to advance critical thinking. Teaching of Psychology, 22 (1), 9 10. McPeck, J. E. (1981). Critical thinking and education New York: St. Martin's Press. Merseth, K. K. (1996). Cases and case methods in teacher education. In J. Sikula & T. J. Buttery & E. Guyton (Eds.). Handbook of research on teacher education (2 nd ed., pp. 722 744). New York: Simon & Schuster Macmillan. Mezirow, J. (1998). On critical reflection. Adult Education Quarterly, 48 (3), 185 Mojo, E. B.; Remillard, J.T.; Southerland, S.; Wade, S.E. (1999) Researching case pedagogies to inform our teaching. In M. A. Lundeberg, B. B. Levin & H. L. Harrington (Eds.), Who learns what from cases and how? the research base for teaching and learni ng with cases. (pp. 73 94 ). Mahwah, NJ: Lawrence Erlbaum Associates.

PAGE 268

256 Montacalm, D., & Royse, D. (2002). Data analysis for social workers Boston: Allyn & Bacon. Morgan, G. A., Leech, N. L., Gloecknew, G. W., & Barrett, K. C. (2004). SPSS for introductory statistics: Use and interpretation Philadephia, PA: Lawrence Erlbaum Associates. Mumm, A. M., & Kersting, R. C. (1997). Teaching critical thinking in social work practice courses. Journal of Social Work Education, 33 (1), 75 84. Murdach, A. D. (1994). A voiding errors in clinical prediction. Social Work, 39 (4), 381 386. Nesoff, I. (2004). Student journals: A tool for encouraging self reflection and critical thought. Journal of Baccalaureate Social Work, 10 (1), 46 60. Newman, D. R., Webb, B., & Cochrane, C. (1995). A content analysis method to measure critical thinking in face to face and computer supported group learning. Interpersonal Computing and Technology: An Electronic Journal for the 21st Century, 3 (2), 56 77. Nichols, D. P. (1998). My tests don' t agree. from SPSS keywords, number 66. Norris, S. P. (1985). Synthesis of research on critical thinking. Educational Leadership, 42 (8), 40 45. Osmond, J., & O'Connor, I. (2006). Use of theory and research in social work practice: Implications for knowle dge based practice. Australian Social Work, 59 (1), 5 19. Owen, J. (2005). Counselors' reflective judgment: An examination of clinical judgment strategies. (Ph.D., University of Denver).

PAGE 269

257 Parker, R. C., & Wolfer, T. A. (2008). Unusual appeal. In T. A. Wolf er & V. M. Runnion, Dying, death, and bereavement in social work practice: Decision cases for advanced practice (pp. 88 97). New York: Columbia University Press. Pascarella, E. T., & Terenzi, P. T. (1991). How college affects students San Francisco: Josse y Bass. Paul, R. W., Elder, P., & Bartell, T. (1997). California teacher preparation for ins truction in critical thinking: R esearch findings and policy recommendations California commission on teacher credentialing Sacramento, CA: Foundation for Critical Thinking. Paul, R., & Elde r, L. (2006). Critical thinking: The nature of critical and creative thought. Journal of Developmental Education, 30 (2), 34 35. Perry, W. G. (1970). Forms of intellectual and ethical development in the college years: A scheme. New York: Holt, Rinehart and Winston. Phillips, V., & Bond, C. (2004). Undergraduates' experiences of critical thinking. Higher Education Research & Development, 23 (3), 277 294. Piaget, J. (1964). Development and learning. In R. Ripple, & V. Rockcastle ( Eds.), Piaget rediscovered Ithaca, NY: Cornell University Press. Pittman, D. M. (2006). Applying the reflective judgment model to nursing students. (Ph.D., The University of Kansas). Plath, D., English, B., Connors, L., & Beveridge, A. (1999). Evaluatin g the outcomes of intensive critical thinking instruction for social work students. Social Work Education, 18 (2), 207.

PAGE 270

258 Puckett, G., & Dobel, J. P. (n.d.). Seattle Community Association: Undoing institutional racism. Retrieved January 3, 2008, from The Ele ctronic Hallway Web site: https://hallway.org/index.php Reittmeier, M. C., & Wolfer, T. A. (2007). Identifying learning outcomes in a social work case method course: Graduate student perspectives. Unpublished m anuscript. Rest, J. (1979). Development in judging moral issues Minneapolis: University of Minnesota Press. Ringel, S. (2003). The reflective self: A path to creativity and intuitive knowledge in social work practice education. Journal of Teaching in So cial Work, 23 (3/4), 15 28. Rivas, R. F., & Hull, G. H., Jr. (Eds.). (2004). Case studies in generalist practice (3rd ed.). Belmont, CA: Brooks/Cole Thomson Learning. Rosen, A., Proctor, E. E., Morrow Howell, N., & Staudt, M. (1995). Rationales for prac tice decisions: Variations in knowledge use by decision task and social work service. Research on Social Work Practice, 5 (4), 501 523. Rowe, W. (2007a). The revitalization of traditional university values leading to undergraduate research. In C. N. Dulmas & K. M. Sowers (Eds.), Building excellence: The rewards and challenges of integrating research into the undergraduate curriculum (pp. 1 9). Birmingham, NY: The Haworth Press. Rowe, W. (2007b). The revitalization of traditional university values leading to undergraduate research. Journal of Evidence Based Social Work, 4 (1), 1 9. Royse, D., Thyer, B. A., Padgett, D. K., & Logan, T. (2006). Program evaluation: An introduction (4th Ed. ed.). Belmont, CA: Brooks/Cole.

PAGE 271

259 Samson, A. W. (2000). Latino college st udents and reflective judgment. ( Ph. D., University of Denver ). Retrieved from Proquest Dissertations database. Scales, T. L., & Wolfer, T. A. (Eds.). (2006). Decision cases for generalist social work practice: Thinking like a social worker (1st ed.). Pa cific Grove, CA: Brooks/Cole. Scales, T. L., Wolfer, T. A., Sherwood, D. A., Garland, D. R., Hugen, B., & Pittman, S. W. (Eds.). (2002). Spirituality and religion in social work practice: Decision cases with teaching notes Alexandria, VA: Council on Soci al work Education. Schn, D. A. (1983). The reflective practitioner: How professionals think in action New York: Basic Books, Inc. Schn, D. A. (1987). Educating the reflective practitioners: Toward a new design for teaching and learning in the professi ons San Francisco: Jossey Bass. Schutt, R. K. (2004). Investigating the social world: The process and practice of research (4th ed.). Thousand Oaks, CA: Pine Forge Press. Seelig, J. M. (1991). Social work and the critical thinking movement. Journal of T eaching in Social Work, 5 (1), 21 34. Shaughnessy, J. J., & Zechmeister, E. B. (1990). Research methods in psychology McGraw Hill: New York. Shermis, S. S. (1992). Critical thinking: Helping students learn reflectively Bloomington, IN: ERIC Clearinghouse on Reading and Communication Skills.

PAGE 272

260 Sherr, M. E, & Wolfer, T. A. (2002). I will not be God's entertainment. In T. L. Scales, T A. Wolfer, D. A. Sherwood, D. R. Garland, B. Hugen, & S. Pittman, S. (Eds.), Spirituality and religion in social work: A sou rce book of decision cases (pp. 106 110). Alexandria, VA: Council on Social Work Education. Silverman, D. (1993). Interpreting qualitative data: Methods for analyzing talk, text and interaction. London: Sage. Silverman, D. (2001). Interpreting qualitative data: Methods for analyzing talk, text and interaction. (2nd ed.). London: Sage. Smith, G. (2002). Are there Domain Specific thinking skills? Journal of Philosophy of Education, 36 (2), 207. Snyder, M., & Swann, W. (1978). Hypothesis testing processes in social interaction. Journal of Personality and Social Psychology, 36 1202 1212. South Carolina Higher Education Assessment Network. (1996). Critical thinking assessment: Measuring a moving target. report and recommendations of the south Carolina higher education assessment network critical thinking task force No. ED 413808 ) ERIC Spengler, P. M., & Strohmer, D. C. (1994). Clinical judgmental biases; the moderating roles of counselor cognitive complexity and counselor client preferences. Journal of Couns eling Psychology, 41 (1), 8 17. Spicer, J. (2005). Making sense of multivariate data analysis Thousand Oaks, CA: Sage Publications.

PAGE 273

261 Steiner, S., Stromwall, L. K., Brzuzy, S., & Gerdes, K. (1999). Using cooperative learning strategies in social work educa tion. Journal of Social Work Education, 35 (2), 253 264. Strachan, D. (1977). The overcrowded clinic. Retrieved January 3, 2008, from The Electronic Hallway Web site: https://hallway.org/index.php Sung Chan, P. & Yuen Tsang, A. (2008). Bridging the theory practice gap in social work education: A refle ction on an action research in C hina. Social Work Education, 27 (1), 51 69. Teare, R. J., & S heafor, B. W. (1995). Practice sensitive social work education: An emp irical analysis of social work practice and practitioners. Alexandria, VA: Council on Social Work Education. Terenzi, P. T., Springer, L., Pascarella, E. T., & Noram, A. (1995). Influences affecting the development of students' critical thinking skills. R esearch in Higher Education, 36 (1), 23 38. Thode, H. C., Jr. (2002). Testing for normality New York: Marcel Dekker. Thompson, S. S. (1995). Techniques for assessing general education outcomes: The natural science core program at the university of denver (Ph.D., University of Denver). Towle, C. (1954). The learner in education for the professions. Chicago, IL: University of Chicago Press. Towle, C. (1958). The case method in teaching social work. New York: National Association of Social Workers.

PAGE 274

262 U. S. Department of Education, National Center for Education Statistics. (2000). The NPEC sourcebook on assessment, volume I: Definitions and assessment methods for critical thinking, problem solving and writing Washington, D. C.: U.S. Government Printing Off ice. van de Luitgaarden, G. M. J. (2009). Evidence based practice in social work: Lessons from judgment and decision making theory. British Journal of Social Work, 39 243 260. Vandsburger, E. (2004). A critical thinking model for teaching human behavior and the social environment. Journal of Baccalaureate Social Work, 10 (1), 1 11. Wassermann, S. (1994). Introduction to case method teaching. A guide to the galaxy U.S.; New York: C loth. Webb, H. W., Gill, G., & Poe, G. (2005). Teaching with the case met hod online: Pure versus hybrid approaches. Decision Sciences Journal of Innovative Education, 3 (2), 223 250. Wells, C. C. (Ed.). (1998). Social work day to day (3rd ed.). New York: Longman. Wilson, J. (2008). Bridging the theory practice gap Australian N ursing Federation. Wingo, R., Perry J., & Orton, D. (2003). Congruence between curriculum content and employer expectations. 21st Annual Conference of the Baccalaureate Social Work Program Directors, Reno, Nevada. Wolcott, S. K. (2007). Master for deve loping a customized critical thinking rubric. Retrieved July 15, 2008, from http://www.wolcottlynch.com/EducatorResources.html

PAGE 275

263 Wolcott, S. K. (2006a). College faculty handbook: S teps for better thinking. Unpublished manuscript. Wolcott, S. K. (2006b). Educator resources. Retrieved September 8, 2008, 2008, from http://www.wolcottlynch.com/EducatorResources .html Wolcott, S. K. (2006c). Steps for better thinking rubric: A developmental problem solving process. Retrieved July 10, 2008, from http://www.wolcottlynch.com/D ownloadable_Files/Rubric_060209.doc Wolcott, S. K., & Lynch, C. L. (1997). Critical thinking in the accounting classroom: A reflective judgment developmental process perspective. Accounting Education: A Journal of Theory, Practice and Research, 2 (1), 59 78. Wolfer, T. A. (2006 ). An introduction to decision cases and case method learning. In T. L. Scales, & T. A. Wolfer (Eds.), Decision cases for generalist social work practice: Thinking like a social worker (pp. 3 14). Belmont CA: Brooks/Cole. Wolfer, T. A., & Miller Cribbs, J. (2005). Assessing outcomes of decision case learning. Unpublished manuscript. Wolfer, T. A., Freeman, M. L., & Rhodes, R. (2001). Developing and teaching an MSW capstone course using case methods of instruction. Advances in Soci al Work, 2 (2), 156 170. Wolfer, T. A., & Gray, K. A. (2007). Using the decision case method to teach legislative policy advocacy. Journal of Teaching in Social Work, 27 (1), 37 59.

PAGE 276

264 Wood, A. T., & Anderson, C. H. (2001). The case study method: Critical thi nking enhanced by effective teacher questioning skills No. ED 455 221; SP 040 149)U.S. Dept. of Education ERIC. Wood, P. K. (1997). A secondary analysis of claims regarding the reflective judgment interview: Internal consistency, sequentiality and intra i ndividual differences in ill structured problem solving In J. Smart (Ed.), Higher education: Handbook of T heory and theory (pp. 245 314) Wood, P. K. (2000). Reflective judgment. Retrieved August 18, 2007, from http://web.missouri.edu/~woodph/html/refl.judg..html Wood, P., & Kadrash, C. A. (2002). Critical elements in the design and analysis of studies of epistemology. In Hofer, B. K.: Pintrich, P.R. (Ed.), Personal epistemolog y: The psychology of beliefs about knowledge and knowing (pp. 321 258). Mahwah, NJ: Lawrence Erlbaum Associates. Wood, P. K., Kitchener, K., & Jensen, L. (2002). Considerations in the design and evaluation of a paper and pencil measure of epistemic cognit ion. In B. K. Hofer, & P. R. Pintrich (Eds.), Personal epistemology: The psychology of beliefs about knowledge and knowing (pp. 277 294). Mahwah, NJ: Lawrence Erlbaum Associates. Wright, J. M., & Michaud, S. (2002). Teaching BSW students to work with comp lex families. Journal of Baccalaureate Social Work, 8 (1), 145 158.

PAGE 277

265 Appendices

PAGE 278

266 Appendix A: Reflective Judgment Interview (RJI)

PAGE 279

267 Appendix A (Continued)

PAGE 280

268 Appendix B : Longitudinal Studies of Reflective Judgment

PAGE 281

269 App endix B (Continued)

PAGE 282

270 A ppendix C: RCI Sample

PAGE 283

271 Appendix C (Continued)

PAGE 284

272 Appendix C (Continued)

PAGE 285

273 Appendix D: Course Syllabus SOWK 718: Systems Analysis of Social Work Practice Spring 2008 Course Syllabus Purpose / Rationale of t he Course Competent social work practice demands integration of a wide range of theories, knowledge, skills, and values. This course is designed to draw on all previous courses and to engage students in integrating and applying all that they have learned. It promotes holistic practice by supporting shared learning among advanced students in both concentrations (Social Work Practice with Individuals, Families, and Groups, and Social Work Practice with Organizations and Communities) and helps students gain competence and confidence as practitioners in accordance with specific College defined objectives. Content exposed during their MSW courses and field practica. In addition, it w ill include new content as students address a social problem or current issue of concern to the profession; they will be expected to use the library, Internet, and personal contacts with other professionals to survey legislation, policies, theories, resear ch, programs, services, practice models, and interventions. As students from the two concentrations interact, they will bring new material to one another. There are twelve decision cases each year, and each year most of the cases are new to this course. C ases contain practice dilemmas concerning social work values and ethics, social justice, and diversity. Course Objectives Students who successfully complete this course will be able to: 1 articulate their integration of theories, knowledge, skills, and val ues developed across the curriculum, including field, in approaching practice situations from an eco systems perspective; 2 critically analyze: a social problems and cases at all systems levels, b relevant human behavior and practice theories and social welfare policies, c research findings reported in the professional literature and other media, d current practice and intervention alternatives, e issues associated with evaluation of practice;

PAGE 286

274 Appendix D (Continued) 3 collaborate effectively with others to explore iss ues and develop intervention plans; 4 defend practice decisions based on current theory and knowledge, and the values and ethics of the profession; 5 link practice decisions to appropriate outcomes and methods for evaluation of practice; 6 apply their understand ing of, and commitment to, the promotion of social and economic justice for populations at risk, and their recognition of and respect for diversity, as they respond to a variety of specific cases. Linkages to Other Courses t in the final semester of the program, and is intended to help students integrate all their coursework in preparation for graduation and professional practice through the use of decision cases and additional written assignments. Most students will be enr olled in this course concurrently with their final field placement. Ideally, students from both concentrations Social Work Practice with Individuals, Families and Groups and Social Work Practice with Organizations and Communities will be enrolled in each s ection and will facilitate the learning of their colleagues by sharing the knowledge and experiences specific to each concentration. Methods of Instruction The course will use discussion on specific decision cases that reflect human services issues, prob lems, and challenges. Students will be required to think multi systemically, as they: 1 analyze the context and meaning of the situations for individuals, families, groups, organizations, and communities; 2 consider various approaches suggested in the literatu re and the resources offered in a wide variety of human service settings; and 3 make recommendations for social work intervention and evaluation of practice and discuss the rationale for choices. ssignments, point students toward resources, assist with group process, facilitate periodic checks on student attainment of objectives, and evaluate performance through assignment of grades.

PAGE 287

275 Appendix D (Continued) Any student who because of a disabilit y may need special arrangements or accommodations to meet the requirements of this course should consult with the instructor as soon as possible. The office of Disability Services provides an array of services to meet the needs of students with disabilit ies, according to Section 504 of the Rehabilitation Act of 1973 and the Americans with Disabilities Act of 1990. Attendance Policy Students are expected to attend all class meetings as scheduled and all meetings scheduled by their work group. Informed p articipation in all class sessions is expected. Absence from class meetings or from group meetings may result in a lowered course Academic Responsibility (omitted in order to maintain the anonymity of the host institution). Calendar and Required Readings There is no text book for this course. All required readings will be available via Blackboard. All course sections will follow the decision case schedule below. However, some instruc tors may vary the order and timing of other required readings.

PAGE 288

276 Appendix D (Continued) January 16 19 Introduction to course, objectives, decision case method, and assignments January 23 26 Wolfer, T. A. (2006). An introduction to decision cases and case method learning. In T. A. Wolfer & T. L. Scales (Eds.), Decision cases for advanced social work practice: Thinking like a social worker (pp. 3 16). Pacific Grove, CA: Brooks/Cole Thomson Learning. Wolfer, T. A., & Scales, T. L. (2006). Tips for discussing decision cases. In T. A. Wolfer & T. L. Scales (Eds.), Decision cases for advanced social work practice: Thinking like a social worker (pp. 17 25). Pacific Grove, CA: Brooks/Cole Thomson Learning. Parker, R., & Wolfer, T. A (2007 ). Unusual appeal (Decis ion case #1) January 30 February 2 Gambrill, E. (1997). A problem focused model based on critical inquiry. In Social work (pp. 96 124). New York: Oxford University Press. Stivers, J., & Kent, J. (2004). Who speaks for us? [Electronic Hallway] (Decision case #2) February 6 9 match: Problems versus strengths. Families in Society: The Journal of Contemporary Social Services, 85 (3), 317 325. U cci, J., & Wolfer, T. A. (2007). Suicidal co ed (Decision case #3) February 13 16 Generalist social work practice: An empowering approach (pp. 22 49). Boston: Allyn & Bacon. Golensky, M. (2001). Hope Network: Where do we go from here? [Program on Nonprofit Organizations, Yale University] (Decision case #4)

PAGE 289

277 Appendix D (Continued) February 20 23 Congress, E. P. (2000). What social workers should know about ethics: Understa nding and resolving practice dilemmas. Advances in Social Work, 1 1 22. Wolfer, T. A. (200 2 ). Drinking social worker (Decision case #5) February 27 March 1 Clinical S ocial Work Journal, 4 (2), 110 120. Barsade, S. G., Frank, C., Kim, P. Landsberg, R., Shiba, A, & Su, C. (2001). ABC Childcare: My hands are tied [Program on Nonprofit Organizations, Yale University] (Decision case #6) March 5 8 Hardcastle, D. A., Wenocur, S., & Powers, P. R. (1997). Using self in community practice: Assertiveness. In Community practice: Theories and skills for social workers (pp. 196 232). New York: Oxford University Press. Strachan, D. (1977). The overcrowded clinic [Electronic Hallway] (Decision case #7) March 12 15 No class Spring break March 19 22 Fleck Henderson, A., & Melendez, M. P. (2002). A cursed child? (Decision case #8) March 26 29 Cearley, S. & Runnion, V. M. (1999). ResponsAbilities. (Decision case #9) April 2 5 Choi, D. Y., & Kiesner, F. (2007). Homeboy Industries: An incubator of hope and business. Entrepreneurship Theory and Practice 31 (5), 1 22. (Decision case #10) April 9 12 #11 )

PAGE 290

278 Appendix D (Continued) April 16 19 Puckett, G., & Dobel, J. P. (n.d.). Seattle Community Association: Undoing institutional racism [Electronic Hallway]. (Decision case #12) April 23 26 Beyond burnout: Helping teachers, nurses, therapists, and lawyers recover from stress and disillusionment (pp. 17 36). New York: Routledge. Required Assignments The major classroom activity in this course will be in depth discussion of ass orted decision cases. These discussions will be facilitated using the case method of teaching. The case method of teaching begins with the twin ideas that working to understand and resolve challenging puzzles or problems will stimulate learning and that su ch efforts closely resemble the assessment and decision making processes needed in professional practice. This educational strategy will be further explained in class. In addition, two required readings briefly describe the case method and suggest ways to prepare for case discussions (Wolfer, 2006; Wolfer & Scales, 2006). Because the course is highly experiential, students must attend class consistently and participate actively to maximize their learning There are three types of required assignments for t his course. As explained in more detail analyses, and 3) write an annotated resume/statement of qualifications. Individual instructors may require additional assignments Instructors will provide further information about the point distribution for the required assignments and their grading scale.

PAGE 291

279 Appendix D (Continued) WRITTEN CASE ANALYSES AND FEEDBACK There are a total of twelve decision cases for in depth analysis and discussion. The cases involve a variety of problems and dilemmas at various system levels. Each case reports the actual experience of a social work professional, sometimes one who is relatively new to profession practice Each week, students will eith er write and submit case analyses or provide feedback on case analyses written by members of their small group. A ll case analyses must include the six sections in the table below although some instructors may require additio nal elements Use headings to i dentify these sections in the case analyses. Required Components of Case Analyses Introduction Briefly identify the major elements (i.e., people, settings) of the case. Problem Statement Give a specific and concisely written formulation of the problem to guide analysis and problem solving. Not a question but a statement of the problem. Usually no more than two sentences. Contextual Analysis Summarize internal and external issues that created or sustain the problem. Depending on the system level, these may include: cultural, economic/resource, political/legal, organizational, social, and ethical issues, interpersonal relationships, and intrapsychic and biological conditions. Alternative Strategies Identify three or more possible solutions to the proble m. These solutions should be plausible, distinct and non contingent (i.e., not interdependent). Briefly note advantages and disadvantages of each possible solution. Recommendation Justify your preferred strategy, explaining why you selected that particula r one, how it best resolves the problem, and how you will determine its effectiveness. Be sure your recommended strategy can be supported by resources available in the context. Rationale Identify the actual basis for your analysis and recommendation. For example, did you base it on previous experience, intuition, specific theories, personal values, empirical research, previous discussion of similar problems, or something else?

PAGE 292

280 Appendix D (Continued) Case analyses should be written as executive summari es Executive summaries are designed to aid decision makers who need understanding of and advice for dealing with a problematic situation. They provide a concise analysis and recommendation but without all of the analytic detail. In fact, executive summari es often represent the first few pages of a more comprehensive analysis. The executive summary format is not intended to be an exhaustive analysis of all possible issues and alternate strategies but rather a concise, focused summary with the issues and al ternatives only mentioned to insure they receive consideration. Any situation, no matter how complex, can generally be summarized in no more than three pages if reduced to its most essential elements. Limit case analyses to 700 1,000 words. In addition to providing edits and comments with Track Changes, professors will rate written case analyses using the following or a similar matrix: C ase Analysis Evaluation Matrix Problem Formulation Contextual Analysis Alternative Strategies Recommend ed Strategy inking like a Writing Style Accurate, clear, specific, concise, and useful Adequately addresses all important issues Several distinct and appropriate strategies, with well developed pros/cons for each Explicitly resolves the entire problem Reflects thorough problem solving Compellin g, clear and interesting, with no errors Mostly accurate but not clear, specific, and/or concise Adequately addresses most of the important issues Several distinct and appropriate strategies, but pros/cons not well develo ped Resolves most of the problem Reflects good problem solving Clear and interesting, with few errors Part of the problem not incorporated Inadequately addresses some important issues Several strategies, but they are not distinct and appropriate, and/or p ros/cons not well developed Resolves only part of the problem Reflects adequate problem solving Good, with few errors Vague and not useful Omits some of the important issues Strategies would only partly resolve the problem Vaguely resolves problem Reflec ts faulty problem solving Difficult to follow, and/or many errors Misleading Omits most of the important issues Strategies would not resolve the problem, and/or no pros/cons Does not resolve the problem at all Reflects poor problem solving Confusing, and/ or excessive errors

PAGE 293

281 Appendix D (Continued) Guidelines for Submitting Written Case Analyses The process for writing and submitting case analyses is as follows: 1 The class will be divided into small groups. 2 All students write and submit an analysis of the f irst two case s ; instructors provide detailed feedback to each student. 3 After the first two cases, students will write and submit case analyses for half of the cases (i.e., five of the ten remaining cases); instructors provide detailed feedback 4 On weeks s tudents do not write and submit a case analysis, they will be responsible for providing feedback to members of their small group who do write. 5 Each small group is responsible for determining a schedule for writing and submitting case analyses and feedback for the second through twelfth cases. For each of these cases, the schedule should indicate which group members will write and submit case analyses and which members will provide feedback. 6 The schedule should be established in such a manner that feedback i s alternated between group members (i.e., not the same two people for each case) 7 Each small group should submit a final written schedule to the instructor by the second week of class. All group members should sign this schedule. 8 Instructors may require t hat students submit case analyses via Safe Assignment on Blackboard. 9 All case analyses must be submitted each week by 1 p.m. on Wednesday ( students from Thursday and Saturday sections must also submit their analyses on Wednesday). No late case analyses wil l be accepted! Guidelines for Providing Feedback The process for providing feedback is as follows: 1 In addition to writing case analyses, s tudents will provid e feedback to their group members Providing feedback will help writers to improve their case an alyses and provide incentive for them to reciprocate. 2 T o provide beneficial feedback (and also participate effectively in class discussion s), students must study the cases carefully every week, including weeks when they provide feedback to their small gro up members. Read and a nalyze case s before giving feedback ( i.e. develop your own judgments and conclusions)

PAGE 294

282 Appendix D (Continued) 3 T o get or receive feedback for revising case analyses prior to submission students must establish and adhere to a system for timely transfer of draft case analyses and feedback. For example, students scheduled to write and submit case analyses must provide draft case analyses to group members responsible for feedback no less than three days before the analyses are due (i.e., the preceding Sunday) Students who provide feedback must respond to writers with feedback no less than two days before the analyses are due (i.e., the preceding Monday) 4 Beneficial feedback consists of: a Concrete, usable suggestions (avoid vague statement s about quality) b Information regarding gaps that authors may have overlooked c Suggestions regarding the content and flow of the paper: Does it make sense? Is the problem formulation accurate and helpful? Are the internal and external issues adequately addre ssed? Has the author considered an adequate range of strategies? Does the recommendation fit the original problem formulation? Does it seem reasonable? d General assistance with writing (e.g., grammar, spelling, sentence structure). 5 F eedback provided by inst ructor s during the initial weeks of the course will serve as a model for students to follow. Students may also use the matrix for providing feedback. 6 It is recommended that students utilize e mail for the transmission of analysis drafts and feedback. This can be accomplished by attaching documents to e mail. Alternately, instructors may set up discussion groups on Blackboard. 7 When writers send case analysis drafts to group members for feedback, they should simultaneously send drafts to the instructor for co nfirmation. Likewise, when group members provide feedback, they should send the feedback to both the writer and instructor (for grading). 8 Case analysis drafts and feedback must be sent by the Sunday and Monday deadlines respectively Late feedback may rec eive no credit. In sum, students will read and analyze a total of twelve cases for this course. Every student will write and submit a case analysis of the first two case s and five of the remaining ten cases. When not writing case analyses, students will p rovide written feedback to their group members for five cases.

PAGE 295

283 Appendix D (Continued) RESUME/STATEMENT OF QUALIFICATIONS AND ANNOTATION This assignment combines reflection on past experience, cumulative learning, areas of strength and areas for continu ed development, with planning for next steps as an emerging social work professional. The assignment is intended to prepare you for the job search process, aid your transition from graduate school to advanced social work practice, or both. The final prod uct will have two parts. The first part is a resume /statement of qualifications that can be used in the job search process in presenting yourself for review at work, or in evaluating your current professional skill set to guide thinking about next steps in your career. The second part is a document that expands on each component of this resume/statement of qualifications explaining in greater depth the goals, knowledge, theoretical orientation to practice skills, experiences, and professional relations hips that shape who you are as a social worker at this moment in your development. Part 1: The Resume /Statement of Qualifications This overview of your current qualifications and goals should be no more than 2 pages long, and should contain the followin g information: Objective. Your immediate career objective: what type of position are you seeking at this moment in your career? What kind of career change are you considering now that you about to have an MSW? What kind of assessment/feedback do you hope to receive from your supervisor (if you are employed in a social work type position that you intend to keep for the foreseeable future). Summary. Include four bulleted points that highlight : your theoretical orientation to practice your substantive area of expertise the population(s) with which you hope to work your particular professional strengths Education. List your degrees (degree, institution and location, major or concentration, any honors (e.g. cum laude)

PAGE 296

284 Appendix D (Continued) Experience For each social work related position (paid and practicum), include: the agency/organization for which you worked and its location your job title dates of employment bulleted list of major job responsibilities Additional diversity experience. List any experiences you have had volunteer, classroom based, etc., that involved work with members of diverse populations. For each, include: the agency, organization or activity in which you were involved and its location your role and dates of involvement bu lleted list of major activities and responsibilities Community Involvement List any additional community volunteer work you have participated in. This could include fundraising, board membership, advocacy efforts, major political participation, volunteer ing with community and/or faith community service projects, etc. Memberships List any professional associations in which you are a member (i.e. NASW) Seminars and workshops List any specific trainings you have attended, either on your own or through w ork/practicum experience. References List three people who have agreed to serve as professional references for you. Keep in mind that a resume is a tool for selling yourself to a potential employer. The visual presentation, wording, organization, and a ccuracy (both grammatical/spelling accuracy and accuracy of information) are critical elements in an effective resume.

PAGE 297

285 Appendix D (Continued) Part 2: The Annotation. This document should be approximately 10 12 pages long (double spaced, 12 poi nt font, margins of 1 inch). In general, you will be explaining in greater depth each piece of information on your resume previewing the type of honest and strengths focused discussion you might have during a job interview. These explanations should be f ocused and concise, stressing the ways in which each resume entry is a meaningful reflection of who you are, at this moment, as an emerging social work professional. These explanations should also be specific, and include appropriate citations on theories, approaches and skills that you claim as areas of expertise. Below are listed some questions that should help guide you in your written reflections. You do not need to address each question for each entry, and not all entries should receive equal attenti on. This is not intended as an exhaustive set of questions, but rather as a starting point to help stimulate your thinking. Objective: Why are you seeking this type of position at this point in your career? What other directions have you considered, and h ow have you decided on this one? When you began your MSW, what did you intend to do upon graduation? If your plans have changed, how and why? If not, what experiences or learnings have sustained you in your plans? Summary: First, discuss in some depth yo ur theoretical orientation to practice. You MUST identify at least one theory at the macro level (those that help explain how the social world works), one at the mezzo level (those that explain particular issues/challenges facing the client population wit h whom you wish to work), and one micro (practice) theory (those that guide intervention). For each, clearly discuss the major concepts, how/why you find this compelling and helpful given your practice interests, and what experience you have in applying th is theory to practice. Second, for your substantive area, please explain the nature, extent and severity of the problem(s), and discuss your experience/motivation for working in this area. Third, for your population areas, please describe how this popula tion is affected by the substantive problem(s) you have identified, discuss anything unique to this population group in terms of appropriate practice, and describe your experience working with this population group. Fourth, for your list of professional s trengths, discuss: w hat is the evidence that you actually have these strengths and skills? Why have you focused your learning and practice experience in these particular areas? Finally, considering your bulleted points all together, w hy do you feel that these characteristics will help a potential employer to understand what you, uniquely, have to offer? (Please give citat ions throughout this discussion)

PAGE 298

286 Appendix D (Continued) Education: For each degree you have obtained (including the MSW you are about to be awarded), what learnings are most significant in shaping who you are as an emerging social work professional? What learnings best prepare you for the type of job you are seeking? What are the gaps in your professional knowledge that you will need to address in your ongoing development while in the work place? Experience: For each position you have held, what aspects of your work have helped you grow and develop as a social worker? What social work knowledge, theories, skills and approaches have info rmed your work (please give citations in this discussion)? What have you particularly enjoyed or found meaningful? What have you learned about your strengths and limitations? What have you learned about how your own background, beliefs and values influence who you are as a social worker? What has been most challenging? After going through each social work practice. How have you, and how do you plan to use your self within your professional practice? What lessons have you learned about what is appropriate? What does not work? How to make decisions about self disclosure, etc.? Finally, please discuss your plans for self care. Given the often stressful demands of social w ork practice, what strategies do you have for keeping yourself healthy? For avoiding burn out? For maintaining balance between work and personal life? Additional diversity experience: How have your experiences with diverse populations influenced your pr ofessional development? What have you learned about particular, vulnerable populations? What have you learned about yourself? What experiences and exposure do you feel you are lacking at this moment in your professional development? How might the job you a re seeking make use of these experiences? How might it enhance these experiences? What support or additional training might you need? Community involvement: How have your experiences in community work influenced your professional development? What have y ou learned about social structural conditions? What have you learned about the service system? What has been most challenging for you, and what have you most enjoyed? What experiences and exposure do you feel you are lacking at this moment in your professi onal development? How might the job you are seeking make use of these experiences? How might it enhance these experiences? What support or additional training might you need?

PAGE 299

287 Appendix D (Continued) Memberships: What professional affiliations have you fo rmed and why? What type of support do you hope to gain from these affiliations? What will you aim to contribute? What needs for support do you anticipate having to meet outside of professional organizations, and how do you plan to go about meeting them? S eminars and workshops: What, specifically, did you learn from each? How will this learning enhance your social work practice in the type of job you are seeking? References: Why have you chosen each of these people? In what ways are they familiar with you as a social worker? What would each say to a potential employer about your strengths and areas for development? The annotation requires that you reflect upon the experiences and strengths presented in various sections of the resume. Doing so can help you to discuss this information in hiring interviews. The resume should be word processed and formatted to create an attractive professional presentation. The annotation should be double spaced and word processed, with a reference list for literature cited i n the paper (APA style). Headings and sub headings should be used to identify the above components of the resume. The resume should be no more than 2 pages long, while the annotation should be about 10 12 pages long. As appropriate, the following criteria will be used to evaluate the resume/statement of qualifications and the accompanying annotation: Ability to identify, summarize and present your relevant practice experience and strengths T horoughness in addressing all components of the assignment S pecifi city of discussion and analysis D epth of discussion and analysis P rofessional writing and presentation skills

PAGE 300

288 Appendix E1: Initial Intervention Group Invitation

PAGE 301

289 Appendix E2: Intervention Group Reminder

PAGE 302

290 Appendix E3: Intervention Group Identifier & Link

PAGE 303

291 Appendix E4: Intervention Group Posttest Invitation

PAGE 304

292 Appendix E5: Intervention Group Reminder 1

PAGE 305

293 Appendix E6 : Intervention Group Reminder 2

PAGE 306

294 Appendix E7: Comparison Group Invitation

PAGE 307

295 Appendix E8: Comparison Group Reminder

PAGE 308

296 Appendix E9: W ebsite Problem Notice

PAGE 309

297 Appendix E10 : Personal Reminder

PAGE 310

298 Appendix E11 : Personal Reminder re: Case Analysis

PAGE 311

299 Appendix F: Permission to use RCI

PAGE 312

300 Appendix F (Continued)

PAGE 313

301 Appendix G : Permission from IRB

PAGE 314

302 Appendi x G (Continued)

PAGE 315

303 A pp endix G (Continued)

PAGE 316

304 Appendix G (Continued)

PAGE 317

305 Appendix H: Reflective Judgment Study Survey

PAGE 318

306 Appendix H (Continued)

PAGE 319

307 Appendix H (Continued)

PAGE 320

308 Appendix I: Steps for Better Thinking Rubric 7 7 Wolcott, S. K. (February 9, 2006). Steps for Better Thinking Rubric [On line]. Av ailable: http://www.WolcottLynch.com

PAGE 321

309 Appendix J: Coding Rubric Code Skills Pre Reflective Performanc e Pattern 0 Quasi reflective Performance Pattern 1 Quasi reflective Performance Pattern 2 Reflective Performance Pattern 3 Reflective Performance Pattern 4 I (Identific ation) Identifies and summarizes the problem/question in case Does not identify the m ain Identifies the main problem (or what might reasonably be considered to be the main problem); but does not identify subsidiary, embedded, or implicit aspects of the problem Clearly identifies the main problem and subs idiary, embedded, or implicit aspects of the problem In addition to previous level, emphasizes and states criteria for identifying the most important aspects of the problem In addition to previous level, anticipates future problems and identifies issues ar ising from current limitations U (Uncertai nty) Identifies and addresses uncertainties (i.e., reasons why the problem is open ended) Ignores uncertainty, or attributes uncertainty to temporary lack of information or to own lack of knowledge Identifies at least one reason for significant and permanent uncertainty, but does not integrate uncertainties into analysis Addresses significant and permanent uncertainties when interpreting and analyzing information Identifies and discusses the significance of t he most important uncertainties Develops viable strategies for minimizing the most important uncertainties over time R (Relevanc e) Identifies information/ evidence that is relevant to the problem Identifies at least some information that is relevant to the problem Identifies most of the information that is relevant to the problem Explores (considers from different perspectives) a wide range of relevant information Focuses on the most important relevant information able to prioritize Develops viable s trategies for generating important relevant information MP (Multiple Perspecti ves) Integrates multiple perspectives Describes information without acknowledging multiple perspectives OR portrays perspectives and information dichotomously, e.g. good/bad, right/wrong Acknowledges more than one potential viewpoint, approach or perspective; Interprets information from multiple viewpoints; Evaluates information using general principles that allow comparisons across viewpoints; Same as 3 PLUS argues convinc ingly using a complex, coherent discussion of own perspective, including strengths and limitations. IN (Interpret ation) Qualitatively interprets information and creates a meaningful organization Describes rather than interpreting information; or may u se contradictory or illogical arguments; lacks organization Interprets information superficially as either supporting or not supporting a point of view; ignores relevant information that disagrees with own position; fails to sufficiently break down the pro blem Objectively analyzes quality of information; Organizes information and concepts into viable framework for exploring realistic complexities of the problem Focuses analysis on the most important information based on reasonable assumptions about relative importance; organizes information using criteria that apply across different viewpoints Same as 3 PLUS systematically reinterprets evidence as new information is generated over time OR describes process that could be used to systematically reinterpret ev idence.

PAGE 322

310 Appendix J (Continued) E (Evaluati on) Identifies and evaluates implications and consequences of alternatives make sense based on the information provided or does not address implications or consequences beyond dicho tomous characterizations Considers implications and consequences only superficially; ignores negative consequences of own position Analyzes implications and consequences for multiple alternatives In addition establishes criteria to prioritize implications and consequences across alternatives In addition to previous level, identifies processes for addressing implications and consequences over time. S (Supports Conclusio ns) Clearly presents and supports conclusions Provides fact, definitions, or information that mask as conclusions instead of own conclusion Clearly states conclusions and reasons, but limited to supporting primarily one perspective Reluctant to select and defend a single overall conclusion in light of viable alternative ; may provide conclusions with inadequate support Articulates criteria that apply across viable alternatives to reach well founded conclusions In addition to previous level, articulates how problem solving approach and criteria can be refined, leading to b etter solutions or greater confidence over time. J (Justificat ion) Justifies positions with supportive evidence Based on authoritative source OR where absolute answers are not available on an unsupported opinion. Based on facts, evidence that fits an e stablished belief or own perspective Based on interpretations of facts/evidence that are used to justify solutions within particular context. (Right solution depends on a variety of contextual factors). Based on Comparing evidence and opinion from differen t perspectives and constructing solutions that are evaluated by personally endorsed personal values, utility, or need for action Justified probabilistically on the basis of a variety of interpretive considerations, such as the weigh t of evidence, explanatory value of the interpretations, the risk of erroneous conclusions, the consequences of alternative judgments, and the interrelationships of these factors

PAGE 323

3 11 Appendix J (Continued) L (Limitatio ns) Identifies limitations of posi tion/thesis and establishes plans for addressing those limitations Does not acknowledge significant limitations beyond temporary uncertainty Acknowledges at least one limitation or reason for significant and enduring uncertainty; Articulates connections a mong underlying contributors to limitations Adequately describes relative importance of solution limitations when compared to other viable options; In addition to 3, identifies viable processes for strategically generating new information/knowled ge to a id in addressing significant limitations over time C (Context) Same Identifies and considers the influence of the context on the issue Does not address context beyond dichotomous characterizations such as right/wrong, good/bad, smart/stupid Acknowled ges the existence of different contexts, but focuses on context in support of own opinion Identifies and considers the influence of context when analyzing perspectives and evidence Analyzes the issue with a clear sense of scope and context sees the bigge r picture Identifies and addresses long term considerations related to the scope and context OA Overall Approach to Problem Attempts to find single ended questions/problems Appears to begin with conclusions and then stack up eviden ce/arguments to support it Appears to perform comprehensive and objective analyses from different viewpoints, but unable to reach or strongly defend conclusions Appears to develop well founded conclusions based on comprehensive and objective comparison of viable alternatives. Proceeds as if goal is to construct knowledge, to move toward better conclusions or greater confidence in conclusions as the problem is addressed over time.

PAGE 324

312 Appendix K: Coded Paper

PAGE 325

313 Appendix K (C ontinued)

PAGE 326

314 Appendix K (Continued)

PAGE 327

315 Appendix K (Continued)

PAGE 328

316 Appendix L: Performance Pattern Frequencies Item Pre Post f % f % Identification (I0) Seems to miss the point 4 12.5 1 3.1 (I1) Identifies main problem 17 53.1 20 62.5 (I2) Identi fies primary issues and subsidiary, embedded, or implicit aspects of the problem 11 34.4 11 34.4 Uncertainty (U0) Ignores uncertainty, or considers it temporary 4 12.5 5 15.6 (U1) Identifies at least one reason for uncertainty 22 68.8 22 68.8 (U2) Addresses significant uncertainties in analysis 6 18.8 4 12.5 (U3) Discusses the significance of the most important uncertainties 0 0 1 3.1

PAGE 329

317 Appendix L (Continued) Item Pre Post f % f % Multiple Perspectives (MP0) Does not acknowledge multiple perspectives 7 21.9 9 28.1 (MP1) Acknowledges more than one perspectives 20 62.5 18 56.2 (MP2) Interprets information from multiple perspectives 5 15.6 10 15.6 Interpretation (IN0) Describes rather than inte rprets 3 9.4 4 (IN1) Interprets information superficially as supporting one position 22 68.8 19 (IN2) Interprets information; objectively analyzes quality of information; organizes information into viable framework for exploring complexities of probl em. 7 21.9 8 Evaluation 4 12.5 5 15.6 (E1) Considers implication and consequences only superficially 13 40.6 17 53.1 (E2) Analyzes implications and consequences of various alternatives 5 46.9 9 28 .1 (E3) In addition establishes criteria to prioritize implications and consequences across alternatives 0 0 1 3.1

PAGE 330

318 Appendix L (Continued) Item Pre Post f % f % Objectivity (O0) Fails to reason logically from pros/cons to conclusions 9 28.1 7 21.9 (O1) Provides arguments in favor of recommended option and provides little or no opposing argument; uses superficially understood evidence in support of conclusions. 11 34.4 15 46.9 (O2) Provides logical arguments for each option and eithe r a)fails to provide an overall recommendation or b) offers a recommendation with little/no support 10 31.2 9 28.1 (O3) Provides well founded, overarching principles to objectively compare and choose among alternative solutions 2 6.2 1 3.1 Supports Co nclusions (S0) information that mask as conclusions instead of own conclusion 1 3.1 0 0 (S1) Clearly states conclusions and reasons, but limited to supporting primarily one perspective 26 81.2 27 84.4 (S2) Reluctant to select and defend a single overall conclusion in light of viable alternative; may provide conclusions with inadequate support 0 0 3 9.4 (S3) Articulates criteria that apply across viable alternatives to reach well founded co nclusions 5 10.4 2 18.8

PAGE 331

319 Appendix L (Continued) Item Pre Post f % f % Justification (J0) Based on authoritative source OR where absolute answers are not available on an unsupported opinion. 8 25.0 2 6.2 (J1) Based on facts, evidence that fits an established belief or own perspective 11 34.4 18 56.2 (J2) Based on interpretations of facts/evidence that are used to justify solutions within particular context. (Right solution depends on a variety of contextual factors). 11 34.4 11 34.4 (J3) Based on Comparing evidence and opinion from different perspectives and constructing solutions that are evaluated by personally utility, or need for action 2 6.2 1 3.1 L imitati ons (L0) Does not acknowledge significant limitations beyond temporary uncertainty 17 53.1 27 84.4 (L1) Acknowledges at least one limitation or reason for significant and enduring uncertainty; 15 46.9 4 12.5 (L2) Articulates connecti ons among underlying contributors to limitations 0 0 1 3.1

PAGE 332

320 Appendix L (Continued) Item Pre Post f % f % Context (C0) Does not address context beyond dichotomous characterizations such as right/wrong, good/bad, smart/stupid 3 9.4 3 19.4 ( C1) Acknowledges the existence of different contexts, but focuses on context in support of own opinion 21 65.6 23 71.9 (C2) Identifies and considers the influence of context when analyzing perspectives and evidence 7 21.9 5 15.6 (C3) Analyzes the is sue with a clear sense of scope and context sees the bigger picture 1 3.1 1 3.1 Overall Approach (OA0) answer to open ended questions/problems 4 12.5 2 6.2 (OA1) Appears to begin with conclusions and then sta ck up evidence/arguments to support it 17 53.1 23 71.9 (OA2) Appears to perform comprehensive and objective analyses from different viewpoints, but unable to reach or strongly defend conclusio ns 2 25.0 5 15.6 (OA3) Appears to develop well founded co nclusions based on comprehensive and objective comparison of viable alternatives. 3 9.4 1 6.2

PAGE 333

About the Author Marleen Milner received her BSW from the University of Texas at El Paso and her MSSW from the Uni versity of Texas at Arlington. She has pra cticed social work in the fields of child welfare, adoptions, substance abuse, aging, and mental health. She is currently the Director of the Social Work Program at Southeastern University in Lakeland, Florida, where she has taught for eight years. Her res earch interests are in the areas of social work education, spirituality, and cultural competence.


xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam 2200397Ka 4500
controlfield tag 001 002063941
005 20100318120321.0
007 cr bnu|||uuuuu
008 100318s2009 flu s 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0003025
035
(OCoLC)558634977
040
FHM
c FHM
049
FHMM
090
HV40 (Online)
1 100
Milner, Marleen.
3 245
An evaluation of the influence of case-method instruction on the reflective thinking of MSW students
h [electronic resource] /
by Marleen Milner.
260
[Tampa, Fla] :
b University of South Florida,
2009.
500
Title from PDF of title page.
Document formatted into pages; contains 320 pages.
Includes vita.
502
Dissertation (Ph.D.)--University of South Florida, 2009.
504
Includes bibliographical references.
516
Text (Electronic dissertation) in PDF format.
520
ABSTRACT: Social work practice requires that graduates be prepared to deal with complex, multifaceted problems which cannot be defined completely, do not have absolute, correct answers and can be approached from multiple perspectives. This study evaluated the influence of case-based instruction on MSW students' reflective judgment, an aspect of critical thinking associated with the ability to reason through ill-structured problems. (King, Wood, & Mines, 1990). The Reflective Judgment Model, which describes a developmental continuum based upon epistemic assumptions regarding the source and justification of knowledge claims, served as the theoretical framework for the assessment of reflective thinking in this mixed methods study. A quasi-experimental pre-post nonequivalent control group design was utilized to explore whether students who participated in a case method course demonstrated greater increases in reflective judgment than those who did not.MSW students enrolled in a case-based capstone course at a major metropolitan university in the southeast served as the intervention group, while foundation year students enrolled in a research methodology course served as the comparison group. Both groups completed the Reasoning about Current Issues Test (RCI), which is an online, standardized measure that has been widely used to assess reflective judgment (Wood, Kitchener, & Jensen, 2002) at pre and posttest. Content analysis procedures were used to facilitate assessment of students' initial and final case analysis papers for evidence of changes in the reflective thinking skills and problem-solving approaches utilized on initial and final case analysis papers. The case method participants' mean RCI scores remained unchanged between pre and posttest, while RCI posttest scores of participants in the control group decreased significantly.Pre and posttest comparison of students' case analysis papers using a customized rubric based on Wolcott's Steps for Better Thinking (2006) similarly indicated no mean changes in problem-solving approaches between pre and posttest. However, students who began the course using strategies associated with pre-reflective judgment increased their scores on the rubric significantly while those who exhibited higher levels of quasi-reflective judgment at pretest decreased at posttest. Strategies for designing a developmental curriculum to target the reflective judgment levels of MSW students are proposed.
538
Mode of access: World Wide Web.
System requirements: World Wide Web browser and PDF reader.
590
Advisor: William Rowe, D.S.W.
653
Decision cases
Reflective Judgment Model
Critical thinking
Social work education
Learning outcomes
0 690
Dissertations, Academic
z USF
x Social Work
Doctoral.
773
t USF Electronic Theses and Dissertations.
4 856
u http://digital.lib.usf.edu/?e14.3025