USF Libraries
USF Digital Collections

Affective dispositions and cognitive skills in critical thinking

MISSING IMAGE

Material Information

Title:
Affective dispositions and cognitive skills in critical thinking implications for measurement, training, and team performance
Physical Description:
Book
Language:
English
Creator:
Willis, Timothy John
Publisher:
University of South Florida
Place of Publication:
Tampa, Fla.
Publication Date:

Subjects

Subjects / Keywords:
Behavioral ratings
Expert ratings
Collaboration
Problem-solving
MANCOVA
Mental models
Delphi technique
Intelligence
Dissertations, Academic -- Psychology -- Masters -- USF   ( lcsh )
Genre:
government publication (state, provincial, terriorial, dependent)   ( marcgt )
bibliography   ( marcgt )
theses   ( marcgt )
non-fiction   ( marcgt )

Notes

Abstract:
ABSTRACT: This study attempts to increase critical thinking among teams by making them cognizant of seven critical thinking dimensions. Forty three-person teams of undergraduates worked together on a complex decision-making task. Each team received training in and was asked to ensure their group exhibited the characteristics of cognitive skills, affective dispositions, both of these or neither. Critical thinking was assessed using self-report, behavioral observation rating, and expert outcome analysis. The findings suggest training in affective dispositions increases the exhibition of at least one dimensions of critical thinking. A behavioral measurement is presented and evaluated with respect to established critical thinking methods.
Thesis:
Thesis (M.A.)--University of South Florida, 2004.
Bibliography:
Includes bibliographical references.
System Details:
System requirements: World Wide Web browser and PDF reader.
System Details:
Mode of access: World Wide Web.
Statement of Responsibility:
by Timothy John Willis.
General Note:
Title from PDF of title page.
General Note:
Document formatted into pages; contains 53 pages.

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
aleph - 001711507
oclc - 69659756
usfldc doi - E14-SFE0001335
usfldc handle - e14.1335
System ID:
SFS0025656:00001


This item is only available as the following downloads:


Full Text

PAGE 1

Affective Dispositions and Cognitive Skills in Critical Thinking: Implications for Measurement, Training, and Team Performance by Timothy John Willis A thesis submitted in partial fulfillment of the requirements for the degree of Master of Arts Department of Psychology College of Arts and Sciences University of South Florida Major Professor: Michael D. Coovert, Ph.D. Carnot E. Nelson, Ph.D. Doug Rohrer, Ph.D. Date of Approval: August 26, 2004 Keywords: behavioral ratings, expert ratings, collaboration, problem-solving, MANCOVA, mental models, Delphi technique, intelligence. Copyright 2005, Timothy J. Willis

PAGE 2

i Table of Contents List of Tables................................................................................................................. ...ii List of Figures..................................................................................................................iii Abstract....................................................................................................................... ......iv Chapter One: Introduction.................................................................................................1 Mental Models.......................................................................................................3 Critical Thinking....................................................................................................7 Critical Thinking In Industrial and Organizational Psychology..........................10 Cognitive Skills and Critical Thinking................................................................11 Affective Critical Thinking..................................................................................13 Training................................................................................................................15 Training Critical Thinking...................................................................................16 Hypotheses...........................................................................................................17 Chapter Two: Research Design.......................................................................................19 Participants and Materials....................................................................................19 Procedure.............................................................................................................20 Scoring.................................................................................................................22 Chapter Three: Results.....................................................................................................22 Chapter Four: Discussion.................................................................................................28 Ramifications For Training and Development Programs....................................29 Limitations of Current Study...............................................................................30 Future Research...................................................................................................30 References........................................................................................................................31 Appendices.......................................................................................................................36 Appendix A: Command and Control Fog Tactical Decision Game....................37 Appendix B: Ambush at Dusk Tactical Decision Game.....................................40 Appendix C: Personality Measure.......................................................................43 Appendix D: Exit Questionnaire..........................................................................45 Appendix E: Behavioral Rating Form.................................................................50

PAGE 3

ii List of Tables Table 1 Cognitive skills of critical thinking..........................................................12 Table 2 Affective Dimensions of Critical Thinking.............................................14 Table 3 Variables used in multiple analysis of covariance...................................26

PAGE 4

iii List of Figures Figure 1 Graph of Subject Matter Expert Ratings by Training Condition.............11 Figure 2 Graph of Behavi oral Observation of Truthseeking Dimension by Training Condition.....................................13 Figure 3 Graphical Representation of Variables used in the Analysis...................26

PAGE 5

iv Affective Dispositions And Cognitiv e Skills In Critical Thinking: Implications For Measurement, Training, And Team Performance Timothy John Willis ABSTRACT This study attempts to increase criti cal thinking among teams by making them cognizant of seven critical thinking dimensions. Forty three-person teams of undergraduates worked together on a complex d ecision-making task. Each team received training in and was asked to ensure their gr oup exhibited the characte ristics of cognitive skills, affective dispositions, both of these or neither. Critical thi nking was assessed using self-report, behavioral observation rating, a nd expert outcome analysis. The findings suggest training in affective dispositions increases the exhibiti on of at least one dimensions of critical thinki ng. A behavioral measurement is presented and evaluated with respect to established critical thinking methods.

PAGE 6

1 The teaching techniques of Socrates, whereby probing questions are asked of students to lead them toward knowledge, are ofte n cited as the first recorded instance of critical thinking (Fasko, 2003). Descartes subsequently he lped move the study of the mind from the realm of theology to the realm of pre-science by sugge sting thinking is the property of the mind, and as such it is endow ed with certain innate knowledge (Schultz & Schultz, 1996). Two hundred years later, how ever, John Locke proposed the notion that the mind without experience was a tabula rasa, or blank slate. He believed ideas were a result of experience and complex ideas were simply many simple ideas joined together (Fasko, 2003). Regardless of with whom you agr eed, it was clear the scientific study of thought had begun. The nineteenth century saw rise to modern psychology and with it new ways of explaining thinking. Beginning with Wilh elm Wundt, and continuing with Edward Titchener, thinking was assessed and explored using introspection. Titchener proposed an elaborate structure of consciousness comp rising three states: sensations, images, and affective states (Schultz & Schultz, 1996) John B. Watson later denied even the existence of a consciousness; he believed thinking was nothing more than sub-vocal speech (Fasko, 2003). With the very existen ce of thinking in question, it seemed a complete change in perspective was needed. And so it seemed everything from physic s to psychology was beginning to shift from the reductionistic introspection associated with Wundt and Titchener toward the more integrative school of gestalt. Led by Wertheimer, Kohler, and Koffka, the gestalt school viewed thinking as an active, construc tive process. This active view of thought

PAGE 7

2 was later influenced by the burgeoning interest in computer technology, and led naturally to the information-processing model of thinki ng that was prevalent in the 1960s. This model provides a functional, organized fr amework for understanding mental processes using the processes of attention, encoding and retrieval, and is evocative of the model used to understand the way computers proces ses information. In the early 1980s, the information-processing model evolved into a more involved, more active thought processa framework ideally suited to the investigation of critical thinking (Fasko, 2003). In part in response to the newly active, involved thought process models that were being proposed during this time, many researchers became increasingly aware of the automatic aspects of human thinking. In order to survive in an increasingly hostile world, humans have developed the ability to simplify their existe nce somewhat through automation. Several theories of the process by which this occurs were developed in part to help explain why some situations elicit a reasoned response from someone while other situations are responded to au tomatically. One automaticity theory suggests that as a person begins to learn a particular skill, he or she works through mental processing algorithms (Logan, 1988). For example, during the beginning of her stay in a foreign land, a traveler might perform a mental conversion of the price of items in a store to her home currency each time she makes a purchase. Eventually she will recall from memory rather than calculate prices for items frequently purchased. In this way, active computation has been reduced or eliminated and the much less effortful process of memory retrieval provides the information previously acquired through computation.

PAGE 8

3 Another form of automa ticity involves the caching of solutions to common problems. In much the same way peopl e use shortcuts and automatic means of information processing, so too can they cache solutions to common problems for use at a later time. In fact, we can use the knowle dge of a persons tendencies to use the same solutions over and over in similar situati ons to predict their behavior (Markman & Gentner, 2001). Through the process of categorization peopl e classify things into categories and draw inferences from those categories. Categorization can be divided into domaingeneral processes such as the role of similarity, or domainspecific processes such as in the study of cross-cultural differences in categorization (Markman & Gentner, 2001). The concept of similarity refers to the co mparing of stimulus objects with prototype objects in order to categorize new objects. These stored category prototypes are comprised of the most typical features of the members of the category it represents; new objects are placed into categories according to their similarity to the prototype. Mental Models In an attempt to understand how humans perceive and interact with their environment, scientists rely heavily on the concept of the mental model as a framework for understanding certain cognitive processe s. Markman and Gentner (2001) broadly define mental models as representation[s] of some domain or situation that supports understanding, reasoning, or prediction (p.229), yet this definition is far from incontrovertible, and has been criticized as being so general that it does not accurately

PAGE 9

4 differentiate mental models from knowledge systems. Rouse and Morris (1986) review several alternative definitions of mental models, recognizing that most theories are developed to explain a specific situati on or phenomenon, and provide a general definition: a mental model is a mechanism whereby humans generate descriptions of system purpose and form, explanations of system functioning and observed system states, and predictions of future system states (p. 360) Mental models have been theorized to help individuals predict and explain behavior, draw infere nces and make predictions, determine appropriate actions, recognize and remember relationships among system variables, understand expecta tions, and organize and proce ss new information in a rapid and flexible manner (Cannon-Bowe rs, Salas, & Converse, 1993). Two types of mental models are generall y recognized: logical mental models and causal mental models. People form logical mental models in order to organize complex series of items and relationships when engagi ng in logical reasoning tasks. These models are developed as a way of organizing and cat egorizing information for a specific task, and are discarded after the task has been accomplished. Causal mental models, on the other hand, are mental representations used in reasoning that are based on long-term knowledge or theories. An example of this t ype of mental model is a mental simulation of events yet to occur. These simulations ar e qualitative rather than quantitative, in that they use imagery rather than calculations. Hegarty and Just (1993) st udied the processes by which people form mental models. They measured the time it took for individuals to solve a mechanical comprehension test when given a textual desc ription, a diagrammatic description, or both.

PAGE 10

5 They found evidence that subjects in all cond itions were constructing mental models of the mechanical system they were asked to explain, but the group that was given text and diagrams performed better than those given one or the other. S ubsequent eye-fixation analysis showed the readers were constructing mental models incrementally by reading and re-reading a line of text be fore integrating that information with the diagram. These diagram consultations are thus seen as elabor ative--an aid in the formation of a mental model rather than as an aid in understating the text. Although mental models are generally t hought to be cognitive phenomena, there is some evidence that they have a motor component as some mental simulations have been linked to bodily movement. Schwartz and Black (1999) asked people to mentally simulate tilting either a wide or a narrow gl ass as far as they c ould without spilling the water it contained. They found people were usually quite inaccurate when asked to visualize and report how far they could tilt each glass, but when they closed their eyes and tilted empty glasses they were much more lik ely to report the correct angle. Kathleen Metz (1985) found children often used hand moti ons to simulate the motion of a turning gear when asked to predict in which direction a specific gear will rotate. Eventually the children developed decision rules (e.g. each gear turns the opposite direction from the one adjacent), and made decisions based on ru les rather than mental simulations. These decision rules reduce or eliminate the need to engage in mental simulation by utilizing previously stored information, thereby re ducing cognitive load. Gentner and Medina (1998) argue that the comparis on of two or more mental re presentations allows for the extension and application of abstract knowledge from one setting to another. The more

PAGE 11

6 similar the mental representations are to one another the more likely it is that knowledge will be extended and connections between concepts strengthened by the formation of a new decision rule. If two mental representati ons are very dissimilar it is likely that a new decision rule will not be formed, knowledge will not transfer from one model to the other, and there will be no decrease in cognitive load. Mental models are also studied at the t eam level, though not without controversy. Klimoski and Mohammed (1994) provide a re view of studies supporting and refuting the idea that teams can share a mental model or form a shared cognition (p.406). Klimoski and Mohammed also discuss the relative lack of empirical evidence with regard to group cognition, and cite as cause the difficulties in collecting such data. Endsley (1995), however, provides evidence suggesting shared mental models can be accurately assessed using self-report and behavioral observation methods. Training is also an integral factor in the relationship between mental models and team performance. Rouse, CannonBowers, and Salas (1992) provide evidence that training teams to develop appropriate expectations of likely team behaviors and explanations of observed team behaviors enhances mental models and therefore improves performance. Cannon-Bowers, Salas, and Converse (1993) suggest cr oss-training team members ma y help them understand the roles and responsibilities of their teammates, and training team leaders to share their views of the task with their team may help standardize the mental model for the group. In summary, though the definition of ment al models may be in flux, there is no question as to their usefulness in understan ding and explaining the way individuals and groups organize and retrieve information. Ther e is clear evidence that mental models are

PAGE 12

7 formed spontaneously and rapidly, and ar e revised as new information becomes available. There is evidence that suggest s mental models may have a motor component, and that physical movement can aid in the form ation of models. Mental models assist in the development of decision rules that guide future decisions and reduce cognitive load. Though the study of team mental models has be en stymied by the relative difficulties in measurement, familiar methods are gaining credence from empirical studies and new techniques are being develope d to address this issue. Critical Thinking The study of critical thinking has been plagued by the lack of a unanimously agreed-upon conception of the te rm. In fact, it has been said that there are as many definitions of critical thinking as there are experts in the field (Benderson, 1990). Fasko (2003) reviews nearly two dozen such definitio ns coming from the fields of education, philosophy, and psychology. Each of these fiel ds includes in thei r list of crucial characteristics for critical thinking a slight ly different set of be haviors, thoughts, and qualities, and there is a great deal of disagreement regardi ng the underlying processes and key requirements for something to be c onsidered critical thinking. John Dewey developed the first universal definition of what we now ca ll critical thinking when he proposed his notion of reflective thinking, whic h is the active, persistent, and careful consideration of any belief or supposed form of knowledge in the light of the grounds that support it and the further conclusions to which it tends [It] includes conscious and voluntary effort to establish belief upon a firm basis of evidence a nd rationality (Dewey,

PAGE 13

8 1933, p. 9). Dewey further restrict ed his definition to say that efforts to establish beliefs based on evidence and rationality (p. 9) could only be considered critical thinking if they were both conscious and voluntary. In what may be the most thorough attempt to obtain consensus to date, a panel of experts in critical thinking instruction, assessment, and theory was convened in February of 1988 to systematically study the nature a nd assessment of critic al thinking (Facione, 1990). Over the next seventeen months, these forty-six experts used the Delphi technique (Linstone & Turoff, 1975), an iterative, co llaborative process often used to reach consensus on conceptual definition or other problems not amenable to more quantitative inquiry, to define the criteria of elements comprising critical th inking. Their findings resulted in the identification of an affective disposition component to critical thinking and a cognitive skill component (Facione, 1990). This two-factor model of critical thinking that was developed through expert consensus was later buttressed by empirical research. Taube (1997) tested college students educational values, need for cognition, ambiguity tolerance, and critical thinking ability (as measured by the Watson-Glaser Critical Thinking Appraisal and the Ennis-Weir Criti cal Thinking Essay Test). They also collected college GPA and SAT verbal and math scores. Confirmatory factor analysis showed the two-factor (ability and disposit ion) model provides a better fit with the obtained data than a one-factor model (GFI = .955 versus .869, respectively). Taube cautions that general mental ability may be partly or largely infl uencing the critical thinking ability score. In fact, the factor loadings of SAT-math and SAT-verbal on the critical thinking dimension were both higher than the Watson-Glaser measure, and the

PAGE 14

9 Ennis-Weir tests loading on th e ability factor of the two-factor was more than double its loading on the disposition f actor, though it was assumed it would measure both critical thinking ability and disposition. The study of critical thinki ng is unfortunately somewhat intertwined with the concept of intelligence. We cannot in good c onscious discuss critical thinking without at least acknowledging its relationship with g. g is defined as the ability to deal with complexity (Gottfredson, p.29), and as such must be included in the discussion of critical thinking at least as it is measured by complex tasks. There is evidence, however, that other forms of intelligence such as ta cit knowledge or pract ical intelligence may predict performance on situational judgment tests or simulation exercises (a common method for assessing critical thinking) better than g (Ste rnberg & Hedlund, 2002). The first major documented case of the a ssessment of critical thinking resulted from an experiment Edward Glaser conducted to assess the thinking skills of high school students (Glaser, 1941). He iden tified three components of crit ical thinking: a disposition to carefully consider the problems one encounters, knowledge of logical reasoning methods, and the ability to apply those met hods. These findings became the beginnings of a stream of research that would lead to the eventual development of the Watson-Glaser Critical Thinking Appraisal (Watson & Glaser, 1980). Glaser, now armed with a tool to assess his success, set out to teach critical thinking. His training program consisted of eight lessons related to critical thinking, su ch as evidence, attitude, prejudice, and the scientific method. He obtained evidence suggesting the successful teaching of critical thinking in that the group that had receive d his training performed significantly better

PAGE 15

10 than the group that did not, but his findings were tempered by the subsequent realization that the obtained critical thinking scores we re significantly correlated with intelligence measures (.46) and reading comp rehension (.77) (Facione, 1984). Since that time critical thinking has been studied in nearly every academic discipline, particularly nursing (see Simpson & Courtney, 2002 for a review) and education (e.g. Meyers 1996). Critical Thinking in Industrial and Organizational Psychology One of the ways critical thinking is linke d directly to industrial and organizational psychology is through task complexity. Task co mplexity is invariably linked to critical thinking because the ability to successfully e ngage a complex task often requires a degree of intelligence and the ability to think cri tically. The impact that general mental processing ability (g) has on performance is evid ent in everyday tasks but is most evident in higher order thinki ng skills such as reasoning, abstra ct thinking, and problem solving (Gottfredson, 2002, p.27). These higher order thinking skills are among those generally considered requisite of critical th inking (Fasko, 2003; Facione, 1990). Performance management and assessment are often troublesome in even simple jobs, but the task is exponentially more di fficult when the performance to be assessed involves critical thinking. Molleman and Timmerman (2003) found it difficult to assess the knowledge and skills requisite of some of the complex jobs they assessed, in part due to the high degree of creativity and specializ ation. They also found a great deal of teamwork among those workers who thought critica lly as part of their jobs, and therefore

PAGE 16

11 stress the importance of bei ng aware of the influence of team members when assessing the performance of those workers who are likely to think critically. Cognitive Skills and Critical Thinking Six cognitive skills are thought to be em ployed by good critical thinkers: analysis, evaluation, explanation, inference, interpre tation, and self-regulat ion (Facione, 1990, see Table 1). These six skills are thought to be as trainable and teachable as other skills. In fact, research has long been conducted in th e fields of education and nursing regarding training and evaluating critical thinking skills in students and nurses (f or a review of the education literature see Meyers 1986; for a review of the nursing literature see Simpson & Courtney, 2002).

PAGE 17

12 Table 1 Cognitive skills of critical thinkers (Facione, 1990). Dimension Description Analysis Identifying the intended and actual inferential relationships among statements intended to express belief, judgment, experiences, reasons, information, or opinions Evaluation Assessing the cred ibility of statements or other representations, and to assess the logical stre ngth of the actual or intended inferential relationships among those statements. Explanation Justifying and explaining ones reasoning in terms of the evidential, conceptual, met hodological, criteriological, and contextual considerations upon which ones results were based. Inference Identifying and securing elements needed to draw reasonable conclusions. Forming conjectur es and hypotheses, considering relevant information and resulting consequences. Interpretation Comprehending and expre ssing the meaning or significance of a wide variety of experiences, situ ations, data, events, judgments, conventions, beliefs, rules, procedures, or criteria. Self-regulation Monitoring the elements and the results of ones cognitive activities. Applying the skills of analysis and evaluation to ones own inferential judgments with a view toward questioning, confirming, valid ating, or correcting ones reasoning or results.

PAGE 18

13 Affective Dispositions and Critical Thinking Although cognitive skills and situational f actors influence the ability of someone to think critically, it does not explain why some people seem to think critically often in a given situation while others rarely do. If someone has all the requisite skills to think critically but fails to do so, and a situational explanati on cannot be found, it stands to reason that there is a dispositional explanation for the lack of critical thinking. Based on this observation, scientists and educators be gan looking for those in ternal factors which cause an otherwise capable individual to engage in critical thinking or not. They were looking for the critical spirit of the critical th inker, that way of livi ng that is consistent with critical thinking (Facione, 1998). It was agreed that cri tical thinkers are analytical, inquisitive, judicious, open-minded, systema tic, truth-seeking and possess cognitive maturity (see table 2).

PAGE 19

14 Table 2 Affective dimensions of cr itical thinking (Facione, Sanc hez, & Facione, 1994). Dimension Description Analyticity Prizing the application of reasoning and the use of evidence to resolve problems, anticipating pote ntial conceptual or practical difficulties and consistently being alert to the need to intervene Inquisitiveness Being intell ectually curious and posse ssing a desire to learn even when the application of the knowledge is not readily apparent Open-mindedness Being tolerant of divergent views and sensitive to the possibility of ones own bias Systematicity Being organized, orderly, focused and diligent in inquiry Truth-seeking Being eager to seek th e best knowledge in a given context, courageous about asking ques tions, and honest and objective about pursuing inquiry even if the findings do not support ones self-interests or preconceived notions. Cognitive Maturity Trusting ones own reason ing processes and the ability to lead others in the rational resolution of problems Judiciousness Approaching problems, inquiry, and decision-making with a sense that some problems are ne cessarily ill-structured, and many times judgments must be made on standards, contexts and evidence which preclude certainty.

PAGE 20

15 Training Though there have been many advances in the study of training throughout the last twenty years, three factors consis tently emerge: needs assessment, training methodology, and training evaluation. Accord ing to Wexley (1984), needs assessment answers three questions: where should training be placed, what needs to be trained, and who needs the training? Latham (1988) provi des four methods for identifying training needs: organization analysis, task analysis, pe rson analysis, and demographic analysis. In conducting an organization analys is the training program is linked to the goals of the organization and is designed to minimize the technical obsolescence of the organizations members. The task analysis evaluates the task s that the recipients of the training will be required to do, taking care to anticipate the requirements of the job likely in the near future. A person analysis is conducted to de termine who needs training. Interviews with trainers and managers, surveys, and selfnomination are all methods of conducting person analyses. A demographic analysis is conducte d to determine the training needs of entire populations of workers. These studies are often inter-organizationa l and focus on groups such as workers over fifty or on female managers (Latham, 1988). A successful training method is designed to achieve four goals: presentation of relevant information or concepts to be le arned, demonstration of the knowledge, skills, and abilities (KSAs) to be le arned, create opportunities for trai nees to practice the skills, and provide feedback to trainees during a nd after practice. (Salas & Canon-Bowers, 2001). Training methodology is constantly evol ving. Computer-based training has been used for approximately twenty years, and has been consistently successful (Wexley,

PAGE 21

16 1984), though now when we talk of computer-b ased training we are usually speaking of distributed or distance lear ning. The convenience and savings of time and money that Web-based training programs offer make th em an attractive a lternative to more traditional methods. These new tools have greatly impacted the way training is conducted, but we dont know as much about th e learning processes that occur with the use of these media as we do about face-to-f ace classroom training (i.e., what level of interaction between trainer and student is needed for effective knowledge transfer and retention?) (Salas and Canon-Bowers, 2001). In addition to com puter-based training, simulators and simulation games are very popular training methods. Though the cognitive processes and trai ning design of these methods have been largely ignored, simulation and behavior role modeling ar e usually effective in producing immediate training effects as well as long-term re tention (Salas and Canon-Bowers, 2001). Training Critical Thinking There is a great deal of in terest in training people to be good critical thinkers. Nearly everyone, from teachers to employers to military leaders has an interest in developing the critical thinking skills of their charges. La tham (1988) predicted jobs of the future will require less memorizing of f acts and procedures, fewer physical skills, and far more conceptual ability (p.550). There is little disagreement th at cognitive skills can be trained; these are seen as developabl e skills in much the same way artistic or leadership skills can be developed (Facione, Facione and Giancarlo, 2000). Where there is disagreement, however, is in whether affectiv e dispositions can be trained. That is, can

PAGE 22

17 we train someone to behave in a manner that is consistent with a disposition toward critical thinking? Perkins and Tishman (1993) argue th at the engagement in critical thinking is indicative of a disposition toward cr itical thinking, but th eir argument requires the ability for people to choose their activit y. We hope to determine whether people, given the knowledge of critical thinking dispositions, can perform the behaviors associated with these dispositions in the context of thinking critically. Hypotheses 1. Teams receiving instruction in the use of affective dispositions and cognitive skills will exhibit these characteristics mo re during a critical-thinking task. a. Teams in the cognitive skills condition will use cognitive sk ills to a greater extent than teams in the control or affective dispositions condition, as measured by both the behavioral and self-report ratings. b. Teams receiving training in affectiv e dispositions will use affective dispositions to a greater extent than teams in the control or cognitive skills condition, as measured by both the beha vioral and self-report ratings. c. Teams receiving both cognitive skills training and affective dispositions training will use both affective dispositions and cognitive skills to a greater extent than teams in the control cond ition, as measured by both the behavioral and self-report ratings. 2. Behavioral measures of cognitive skills will correlate with the se lf-report measure of cognitive skills.

PAGE 23

18 3. Behavioral measures of affective disposition correlate with the self-report measure of affective disposition 4. Teams displaying evidence of affective dispositions and cognitive skills perform better on critical-thinking tasks. a. The expert performance rating will be significantly and positively correlated with the affective dispositions self-report score. b. The expert performance rating will be significantly and positively correlated with the affective dispositions behavioral observation score. c. The expert performance rating will be significantly and positively correlated with the cognitive skills self-report score. d. The expert performance rating will be significantly and positively correlated with the cognitive skills behavioral observation score. 5. Affective dispositions are closely related to the personality variable openness to experience. a. The behavioral measure of affective disposition will be highly correlated with the personality measure of openness to experience. b. The self-report measure of affective disposition will be highly correlated with the personality measure for openness to experience.

PAGE 24

19 Method Participants One hundred-twenty students self-sel ected into the study from the USF Psychology Subject Pool. The age of the par ticipants ranged from 18 to 50, averaged 21.9 years (SD=5.13), with a median of 20. Each subject had worked an average of 4.74 years (SD=3.83), ranging from zero to twenty years. The median number of years of work experience was four. Participants worked in three-person teams and were given extra-credit in their classes in c onsideration of th eir participation. Materials Tactical Decision Games We used a modified version of two Tactic al Decision Games (TDGs) reprinted from The Marine Corps Gazette (Schmitt, 1994). These paper-based scenarios are used by military leaders as instructional aids to teach tactics, operations, and strategy. The TDGs were modified to remove military terms that w ould hinder their understanding by non-military undergraduate students and then used to s timulate and measure critical thinking. Exit Questionnaire This is a 26-item questionnaire developed by the researchers. It asked participants to rate the performance of their group in effec tive team functioning, cognitive skills, and affective dispositions. Participants indicated their agreement to questions such as My team had clear agreement on priorities on a fi ve-point Likert scale anchored at strongly disagree and strongly agree.

PAGE 25

20 Personality Measure This 60-item measure was developed w ith questions from the international personality item pool. It is comprised of the 50-item Big Five Domain scale and the 10-item Openness to Experience scale (Goldberg, 1999). TinkerToy project TinkerToys are a childrens toy made of wooden rods and connecting pieces. Three 66-piece junior builder TinkerToy sets were combined, then divided into groups of like shape and function and placed into new bins. One bin contained all connector pieces, another all long rods, and the last bi n contained medium-sized rods and wheels. Procedure Each team was welcomed and told they will be studying team dynamics during problemsolving tasks. After informed consent, consent to videotape, and demographic information was gathered, the participants we re asked to read along while listening to an informative audio recording. Depending on cond ition, participants rece ived instruction in either cognitive skills used in critical thi nking, affective dispositions of critical thinkers, both of these, or neither of these. Thos e groups that did not receive training in both cognitive skills and affective dispositions li stened to training tapes on choosing college classes or conducting a job search, such that they will have listened to a total of two training tapes. Assignment cards were then distributed to those groups who had listened to the cognitive skills tape and/or the affective dis positions tape. These cards list one of the

PAGE 26

21 cognitive skills or affective dispositions and its definition. Participants were instructed to ensure the group engages in the skill or dis position they had been given. Teams received a short training on effective teams and team building. Each team was then asked to complete an icebreaker exercise, designed to enable communication and cooperation, in which they built a tower of TinkerToys The participants were reminded of the qualities of effective teams before the task began. Teams were given one of the bins of TinkerToys and were instructed to build a tower as tall as possible within fifteen minutes. They were limited by the inability to touch only the pieces that came from their bin; they were not allowed to touch their teammates pieces. A discussion followed during which th e team members examined their team and individual performance and the degree to wh ich they used their assignment cards and reminded the group to exhibit the listed charact eristics. They then discussed ways to improve in future tasks, and were reminde d of their assignment to ensure the group exhibits the characteristics on their cards. Next, participants were given their fi rst Tactical Decision Game (TDG). The group was again reminded of effective habits of teams, and was then presented with the scenario and answer sheets. The group was asked to determine and prioritize their goals, issue orders and justifications to the pe ople under their command, and to use a map to provide a visual aid to assist the researchers in understanding their solution. Teams engaged in a discussion after the scenario in which they evaluated their performance and reminded to use their assignment cards in the next task.

PAGE 27

22 The second TDG was given in the same ma nner as the first, with a reminder of the qualities of effective teams. This scenario lasts up to an hour and is videotaped. The participants then each comple ted a personality measure, a nd an exit questionnaire. Scoring SME Rating A military expert used a five-point scale to rate the military effectiveness of the orders given in each teams solution to the second TDG. This score serves as an objective measure of task performance base d on performance-relevant criteria. Behavioral Rating After training to an acceptable level of agreement, three graduate students individually viewed and rated the videot ape of the second TDG of each group. The frequency of exhibition of each of the six cogni tive skills and seven affective dispositions were made, as well as an overall rating (one to five) on each of these 13 dimensions. Self-report Team members were asked to complete a 24-item questionn aire assessing the degree to which their team engaged in each of the thirteen affective dispositions and cognitive skills, and assessed the degree to wh ich the team members engaged in effective team processes. Results The ratings of the three judges completi ng the behavioral ratings of cognitive skills and affective dispositions evidence a hi gh level of inter-judge agreement (type C

PAGE 28

23 intraclass correlation coefficient = 0.87). A one-way between-subjects multivariate analysis of covariance (MANCOVA) including the TDG expert rating, all the 34 dependent variables and three covariates listed in table 3 achieved multivariate significance (F (90,246) = 1.647, p=0.001). The covariates of age, gender, and work experience, however, were not significant (F (30,80) = 0.603, p>0.05; F (30,80) = 0.792, p>0.05; F (30,80) = 1.116, p>0.05, respectively), and were thus removed from future analysis. A one-way between-subjects multiple analysis of variance (MANOVA) was run on the remaining variables. This resulted in an overall significant main effect for experimental condition (F (90,252) = 1.551, p<0.01). Subsequent univariate analysis revealed a significant main effect for condition on subject matter expert (SME) ratings. (F (3,113) =5.784, p<.001). A Games-Howell post-hoc an alysis of the SME ratings reveals the cognitive condition and the control condition scored higher than the affective group condition (mean difference=0.7761, p<0.001 and 0.3937, p<0.05, respectively). The overall behavioral ratings of each team reveal a difference among teams with regard to the affective disposition of truthseeking. Those teams that received affective dispositions training (M=3.62, SD=0.846) received a higher overall rating on the truthseeking dimension of the behavioral rating than did the control group (M=2.88, SD=0.878). The Games-Howell test shows th is difference is significant at the p<.01 level. The behavioral ratings show a strong correspondence with the subject matter expert ratings (r=0.80, p<.01), such that more behavioral expression of cognitive skills and affective dispositions was associated with better solutions to the problem scenario as judged by the subject matter expert.

PAGE 29

Figure 1 Subject matter expert ratings by training condition. 24

PAGE 30

Figure 2 Behavioral observation of Truthseeking dimension by training condition. 25

PAGE 31

26 Table 3 Variables used in multivaria te analysis of covariance. Independent Variable (1) Experimental condition (training in cognitive skills, affec tive dispositions, both, or neither) Dependent Variables (37) TDG expert rating (1) Behavioral ratings (30) a. Frequencies of affective dispositions exhibited, by dimension (7) b. Frequencies of cognitive skills exhibited, by dimension (6) c. Total affective dispositions frequency (1) d. Total cognitive skills frequency (1) e. Global rating (1-5) for each category (13) f. Total global rating for cognitive skills (1) g. Total global rating for affective dispositions (1) Self-report measures (6) a. Affective disposition utilization self-report individual score (1) b. Affective disposition utilizati on self-report team score (1) c. Cognitive skills utilization self-report individual score (1) d. Cognitive skills utilization self-report team score (1) e. Team processes self repo rt individual score (1) f. Team processes self report team score (1) Covariates (8) Personality (5) Age (1) Sex (1) Work experience (1)

PAGE 32

Figure 3 Variables analyzed. 27

PAGE 33

28 Discussion This study sought to shed light on the in fluence of cognitive skills and affective dispositions on team processes and outcomes on a critical-thinking task, to determine which if any of the dimensions of critical thinking are trainable, and to do it using objective measures when possible. Since our subject matter expert was unknown to both participants and researchers a nd saw only the participants solu tions, his or her ratings are very objective ratings of critical thinking from an outcome perspective. A larger challenge is developing objective process measures. The lack of a generally agreed-upon taxonomy of critical thinking made the deve lopment of behavior-based scoring much more difficult. This study provides eviden ce of the potential value of behavioral observation in the study of cr itical thinking in teams. Based on our subject matter expert outcome ratings, we found the cognitive and control groups provided better solutions to the scenario than did the affective group. Although not a significant difference, the bot h group tended toward the affective group. This may indicate a deleterious effect of th e affective training (as both groups received it), or of the instruction to exhibit the char acteristics described on their reminder cards. Affective dispositions are ge nerally harder to understand and integrate than cognitive skills, and the additional attention required to implement the requested behavioral changes could have detracted from their ability to complete the task assigned. In behavioral ratings, we found the a ffective dispositions group exhibited an overall score on truthseeking that was hi gher than the control group. This finding suggests the affective dispositions training work ed, that it increased the extent to which

PAGE 34

29 the teams engaged in truthseeking. If this difference was based solely on the affective dispositions training we would expect to see a significant difference between the affective dispositions group and the cognitive skills group, as they received no affective dispositions training. A possibl e explanation for the lack of a significant difference is the high degree of overlap between the six cognitive skills dime nsions and the truthseeking affective dispositions dimension. The corr elations range from .32 to .45 and are all significant at p=.05. In addition to the findings re lating to the trainability a nd performance influence of cognitive skills and affective dispositions this study developed a unique behavioral analysis technique in order to objectively measure the expression of cognitive skills and affective dispositions in group interactions. In the course of conducting this experiment several researchers noted that although the training sessions were somewhat tedious for all pa rticipants, they seemed to be especially arduous for those in the both condition. Indeed, these participants were trained in cognitive skills followed immediat ely by training in affective dispositions whereas those in the cognitive skills or a ffective dispositions condition received one or the other, followed by traini ng in conducting a job search or choosing college classes. We believe the different traini ng programs were differentially relevant to the participants lives, and that this difference may have affect ed our ability to draw conclusions regarding the efficacy of our training program. This di fference can be seen in the responses to question 23 of the exit questionnai re (see appendix D), which asks participants to rate the degree to which they enjoyed the exercise. There is a small but compelling trend, in

PAGE 35

30 which the groups that received the most life -relevant training (the control group) enjoyed the experiment more than the groups that rece ived no life-relevant tr aining (the both condition), and the two groups that received one life-relevant training each were in between. This effect did not achieve signifi cance in our study but we believe a larger sample would find significant differences. Futu re studies should take this into account when designing training for control groups. The data from behavioral analysis ar e rich and can provide information that cannot be obtained through other means, pa rticularly in the re alms of collaborative critical thinking and team processes. Though th ere is inherent value in the collection and analysis of critical thinking data independent of outcome measures, it is heartening to see such a high correlation (.80) with established me asures of critical th inking. This lends further credence to the behavioral method de veloped in this study, and to the taxonomy that underlies it.

PAGE 36

31 References Cannon-Bowers, J.A., Salas, E. & Converse, S. (1993). Shared mental models in expert team decision making. In N. John Castellan, Jr. (Ed.), Individual and Group Decision Making (pp. 221-246). Hillsdale, NJ: Lawrence Erlbaum Associates. Dewey, J. (1933). How We Think: A Restatement of the Relation of Reflective Thinking to the Educative Process Oxford, England: Heath. Endsley, M.R. (1995). Measurement of s ituation awareness in dynamic systems. Human Factors 37, 65-84. Ennis, R.H. (1996). Critical thinking dispositions: Their nature and assessability. Informal Logic 18 (2&3), 165-182. Facione, P.A. (1984). Toward a th eory of critical thinking. Liberal Education 70 (3), 253-261 Facione, P.A. (1990). Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Assessment and Instruction. Millbrae, CA: California Academic Press. (ERIC Document Reproduction Service No. ED 315 423). Facione, P.A. (1998). Critical Thinking: What it is and why it counts Millbrae, CA: California Academic Press. Facione, P.A., Facione, N.C., & Giancarlo, C.A. (1997). Professional Judgment and the Disposition Toward Critical Thinking Millbrae, CA: California Academic Press. Facione, P.A., Facione, N.C., & Giancarlo, C.A. (2000). The disposition toward critical thinking: Its character, measurement, and relationship to critical thinking skill. Informal Logic 20 (1). 61-84.

PAGE 37

32 Facione, P.A., Sanchez (Giancarlo), C.A., Facione, N.C. & Gainen, J. (1995). The disposition toward critical thinking. Journal of General Education, 44 1-25. Facione, P.A., Sanchez, C.A., & Facione, N.C. (1994). Are college students disposed to think? Millbrae, CA: California Academic Press. (ERIC Document Reproduction Service No. ED 368 311). Fasko, D., Jr. (2003). Critical thinking: Or igins, historical development, future directions. In D. Fasko Jr. (Ed.), Critical Thinking and Reasoning: Current Research, Theory, and Practice (pp. 3-18). Cresskill, NJ: Hampton Press. Gentner, D. & Medina, J. (1998). Similarity and the development of rules. Cognition 65 263.297. Giancarlo, C.A. & Facione, P.A. (2001). A look across four years at the disposition toward critical thinking among undergraduate students. The Journal of General Education 50 (1), 29-55. Glaser, E.M. (1941). An Experiment in the Deve lopment of Critical Thinking New York: Teachers College of Columbia Un iversity, Bureau of Publications. Goldberg, L. R. (1999). A broad-bandwi dth, public domain, personality inventory measuring the lower-level facets of several five-factor models. In I. Mervielde, I. Deary, F. De Fruyt, & F. Ostendorf (Eds.), Personality Psychology in Europe 7, 7-28. Tilburg, The Netherlands: Tilburg University Press. Gottfredson, L.S. (2002). Where and why g matters: Not a mystery. Human Performance 15 (1/2), 25-46.

PAGE 38

33 Gray, P. (1993). Engaging students intellects: The immersion approach to CT in psychology instruction. Teaching of Psychology, 20 (2), 68-74. Holyoak, K.J. & Spellman, B.A. (1993). Thinking. Annual Review of Psychology 44 265-315. Klayman, J. & Ha, Y.-W. (1987). Confirma tion, disconfirmation, and information in hypothesis testing. Psychological Review 94 221-228. Latham, G.P. (1988). Human resource training and development. Annual Review of Psychology39 545-582. Linstone, H.A. & Turoff, M. (1975). The Delphi Method Reading, MA: AddisonWesley. Linstone, H.A. (1999). Decision Making for Technology Executives: Usi ng Multiple Perspectives to Im prove Performance Boston: Artech House. Logan, G.D. (1988). Toward an in stance theory of automaticity. Psychological Review 95, 492-527. Markman, A.B. & Gentner, D. (2001). Thinking. Annual Review of Psychology 52, 223247. Metz, K.E. (1985). The development of child rens problem solving in a gears task: A problem space perspective. Cognitive Science 9 431-471. Meyers, C. (1986). Teaching Students to Think Critically: A Guide for Faculty in All Disciplines. San Francisco: Jossey-Bass.

PAGE 39

34 Molleman, E. & Timmerman, H. (2003). Perf ormance management when innovation and learning become critical performance indicators. Personnel Review 32 (1), 93113. Norris, S.P. (1989). Verbal Reports of Thinking and Mu ltiple-choice Critical Thinking Test Design (Technical Report No. 447). Champa ign, Illinois: University of Illinois at Urbana-Champaign, Cent er for the Study of Reading. Perkins, D.N., Jay, E. & Tishman, S. (1993). Beyond abilities: A dispositional theory of thinking. Merrill-Palmer Quarterly 39 (1). 1-21. Rapps, J., Riegel, B., & Glaser, D. (2001). Te sting a predictive model of what makes a critical thinker. Western Journal of Nursing Research 23(6), 610-626. Rouse, W.B. Cannon-Bowers, J., & and Salas, E. (1992). The role of mental models in team performance in complex systems. IEEE Transactions on Systems, Man, and Cybernetics 22 1296-1308. Rouse, W.B. & Morris, N.M. (1986). On looki ng into the black box: Prospects and limits in the search for mental modals. Psychological Bulletin 100 349-363. Salas, E. & Cannon-Bowers, J.A. (2001). The sc ience of training: A decade of progress. Annual Review of Psychology 52 471-499. Schmitt, J.F. (1994). Mastering Tactics: A Tactical Decision Game Workbook. Quantico, VA: Marine Corps Association Schultz, D.P. & Schultz, S.E. (1996). A History of Modern Psychology (6 th ed.). Orlando, FL: Harcourt Brace.

PAGE 40

35 Schwartz, D.L. & Black, T. (1999). Infere nces through imagined actions: Knowing by simulating doing. Journal of Experimental Psychology: Learning, Memory, and Cognition 25 116-136 Simpson, E. & Courtney, M. (2002). Critical thinking in nursing edu cation: Literature review. International Journal of Nursing Practice, 8 89-98. Sternberg, R.J. & Hedlund, J. (2002). Pract ical intelligence, g, and work psychology. Human Performance 15 (1/2), 143-160. Taube, K.T. (1997). Critical thinking ability and disposition as factors of performance on a written critical thinking test. The Journal of General Education 46 (2), 129-164. Wexley, K.N. (1984). Personnel training. Annual Review of Psychology 35, 519-551.

PAGE 41

36 Appendices

PAGE 42

Appendix A: Command and Control Fog Tactical Decision Game 37

PAGE 43

38

PAGE 44

39

PAGE 45

Appendix B: Ambush at Dusk Tactical Decision Game 40

PAGE 46

41

PAGE 47

42

PAGE 48

Appendix C: Personality Measure 43

PAGE 49

Appendix D: Exit Questionnaire 44

PAGE 50

45

PAGE 51

46

PAGE 52

47

PAGE 53

Appendix E: Behavioral Rating Form 48


xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam Ka
controlfield tag 001 001711507
003 fts
005 20060614135556.0
006 m||||e|||d||||||||
007 cr mnu|||uuuuu
008 060531s2004 flua sbm s000 0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0001335
035
(OCoLC)69659756
SFE0001335
040
FHM
c FHM
049
FHMM
090
BF121 (Online)
1 100
Willis, Timothy John.
0 245
Affective dispositions and cognitive skills in critical thinking
h [electronic resource] :
b implications for measurement, training, and team performance /
by Timothy John Willis.
260
[Tampa, Fla.] :
University of South Florida,
2004.
502
Thesis (M.A.)--University of South Florida, 2004.
504
Includes bibliographical references.
516
Text (Electronic thesis) in PDF format.
538
System requirements: World Wide Web browser and PDF reader.
Mode of access: World Wide Web.
500
Title from PDF of title page.
Document formatted into pages; contains 53 pages.
3 520
ABSTRACT: This study attempts to increase critical thinking among teams by making them cognizant of seven critical thinking dimensions. Forty three-person teams of undergraduates worked together on a complex decision-making task. Each team received training in and was asked to ensure their group exhibited the characteristics of cognitive skills, affective dispositions, both of these or neither. Critical thinking was assessed using self-report, behavioral observation rating, and expert outcome analysis. The findings suggest training in affective dispositions increases the exhibition of at least one dimensions of critical thinking. A behavioral measurement is presented and evaluated with respect to established critical thinking methods.
590
Adviser: Michael D. Coovert.
653
Behavioral ratings.
Expert ratings.
Collaboration.
Problem-solving.
MANCOVA.
Mental models.
Delphi technique.
Intelligence.
690
Dissertations, Academic
z USF
x Psychology
Masters.
773
t USF Electronic Theses and Dissertations.
4 856
u http://digital.lib.usf.edu/?e14.1335