USF Libraries
USF Digital Collections

Cognitive load theory and programmed instruction

MISSING IMAGE

Material Information

Title:
Cognitive load theory and programmed instruction
Physical Description:
Book
Language:
English
Creator:
Brazas, Michael L
Publisher:
University of South Florida
Place of Publication:
Tampa, Fla.
Publication Date:

Subjects

Subjects / Keywords:
Working memory
Learning theory
Linear programmed instruction
Intrinsic programmed instruction
Dissertations, Academic -- Psychology -- Doctoral -- USF   ( lcsh )
Genre:
government publication (state, provincial, terriorial, dependent)   ( marcgt )
bibliography   ( marcgt )
theses   ( marcgt )
non-fiction   ( marcgt )

Notes

Summary:
ABSTRACT: Programmed Instruction was a dominant force in educational research during the 1960's and early 1970's. During this period a new cognitively oriented psychology of education arose that eventually replaced the behaviorally driven research into programmed instruction. For some reason little work was done on integrating these two approaches. This study was designed to test a programmed instructional system based on the intrinsic programmed instructional technology of Crowder (1961) but incorporating design elements derived from cognitive load theory (Sweller, 1988). Specifically, this study tested a multimedia programmed instructional system (CLT-PI) based on work by Mayer (2001) against a text based system (TXT), a traditional linear programmed instructional system based on Skinner (1954, 1958)(LPI), and an intrinsic programmed instructional system based on the work of Crowder (1960, 1961)(IPI).Three hypotheses were tested; 1) Programmed instruction would produce higher long-term retention and transfer scores than TXT, 2) IPI would produced higher long-term retention and transfer scores than LPI, and 3) CLT-PI would produce higher long-term retention and transfer scores than LPI or IPI. 115 undergraduate university students were randomly assigned to one of four conditions where they studied a chapter on human sleep. Each condition was presented on a computer with a test following study. A long-term test was given 4 weeks later. Both short- and long-term tests contained retention and transfer questions. Analyses were conducted using repeated measures MANOVA. A series of ANOVA tests were conducted to determine specific effects and interactions. The first hypothesis was partly supported in that CLT-PI and IPI produced higher long-term retention scores than TXT. LPI, however, did not.
Thesis:
Thesis (Ph.D.)--University of South Florida, 2005.
Bibliography:
Includes bibliographical references.
System Details:
System requirements: World Wide Web browser and PDF reader.
System Details:
Mode of access: World Wide Web.
Statement of Responsibility:
by Michael L. Brazas.
General Note:
Title from PDF of title page.
General Note:
Document formatted into pages; contains 160 pages.
General Note:
Includes vita.

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
aleph - 001681103
oclc - 62774284
usfldc doi - E14-SFE0001011
usfldc handle - e14.1011
System ID:
SFS0025332:00001


This item is only available as the following downloads:


Full Text

PAGE 1

Cognitive Load Theory and Programmed Instruction by Michael L. Brazas A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy Department of Psychology College of Arts and Sciences University of South Co-Major Professor: Toru Shimizu, Ph.D. Co-Major Professor: Mari a dePerczel Goodwin, Ph.D. Judith Becker Bryant, Ph.D. Doug Rohrer, Ph.D. James A. White, Ph.D. Date of Approval: April 5 2005 Keywords: working memory, learning theory, linear programmed in struction, intrinsic programmed instruction Copyright 2005, Michael L. Brazas

PAGE 2

DEDICATION To Jennefer. My best friend, my anchor, my wife.

PAGE 3

ACKNOWLEDGMENTS It is difficult to list everyone who helped, prodded, encourage d, cajoled and generally kept me on track during the years it took to comp lete this project. Mo st of all I need to acknowledge and thank my family. My wife, Jennefer, whose unwavering faith in me gave me the strength to keep going when th e going got really tough. To my sons, Sean, Carson, and Christopher, who gave up a lot growing up with a professional student for a father. Without your unconditional love and s upport I would never have finished this project. I would like to thank the members of my dissertation committee, Toru Shimizu, Maria dePerczel Goodwin, Judith Becker Bryant, Doug Rohrer, and James White for your flexibility, your honest criticism, and your helpful suggestions. A special thanks to Toru for welcoming me into his lab, for the endless hours of editing on a number of projects, and for helping me learn to think like a scientist. Thank you to Maria for introducing me to a whole other psychological world and for giving me a set of tools I intend to use for the rest of my life. To Judy, the one we all go to when we just dont know where else to go, or what to do, your support over the last decade is priceless. I would also like to acknowledge and thank the staff of the Psychology Department, who are always there to help me find my wa y when things got really odd and confusing.

PAGE 4

TABLE OF CONTENTS LIST OF TABLES iii LIST OF FIGURES iv ABSTRACT v CHAPTER ONE INTRODUCTION 1 Goals of the Study 1 Measurement 2 Significance 3 Background of the Four Groups 4 1. TXT 4 2. Linear Programmed Instruction 7 3. Intrinsic Programmed Instruction 18 4. CLT-PI 28 Experiment 36 1. Rationale for this study 36 2. Hypotheses 36 CHAPTER TWO METHODS 37 Participants 37 Materials 38 1. Instructional Materials 38 2. Program Development 41 Procedure 44 1. Data Collection 44 a. Training and Short-Term Assessment 44 b. Long-Term Assessment 45 2. Analysis 45 CHAPTER THREE RESULTS 47 Summary 47 Overall Effects 47 Analysis by Hypothesis 54 CHAPTER FOUR DISCUSSION 59 Hypothesis One 59 Hypothesis Two 61 Hypothesis Three 62 Short-Term Retention and Transfer Data 64 Study Time 65 Limitations of the Study 66 Implications 68 i

PAGE 5

Future Research 69 Conclusions 71 REFERENCES 74 APPENDICES 86 Appendix A: Terminal Behaviors 87 Appendix B: Chi Square Output Pilot Study 92 Appendix C: Chi Square Output Results of Pilot Study Modifications 101 Appendix D: Retention/Transfer Question Matrix 109 Appendix E: Short-Term Assessment 110 Appendix F: Long-Term Assessment 124 Appendix G: Five Page Extract of TXT Program 138 Appendix H: Five Frame Extract of LPI Program 141 Appendix I: Five Frame Extract of IPI Program 144 Appendix J: Five Frame Extract of CLT-PI Program 147 Appendix K: CD Containing Instructional Programs in Windows Executable Format 150 ABOUT THE AUTHOR End Page ii

PAGE 6

LIST OF TABLES Table 1 Demographic Information. 37 Table 2 Results of Study Time Post Hoc Analyses 49 Table 3 Study Time and Dependent Variable Correlations 50 Table 4 T-Test Results Comparing Shortto Long-Term Retention 50 Table 5 T-Test Results Comparing Shortto Long-Term Transfer 51 Table 6 Repeated Measures MANOVA Output 53 Table 7 Post Hoc Analyses of LTR Data H1 54 Table 8 Post Hoc Analyses of STT Data H1 55 Table 9 Post Hoc Analyses of LTR Data H2 56 Table 10 Post Hoc Analyses of STT Data H2 56 Table 11 Post Hoc Analyses of LTR Data H3 57 Table 12 Post Hoc Analyses of STT Data H3 58 iii

PAGE 7

LIST OF FIGURES Figure 1 Sample frame from the TXT Program. 5 Figure 2 A block diagram illustrating frame arrangement in a typical LPI segment. 9 Figure 3 An LPI sequence. 10 Figure 4 A simple branching program with one correct response path and two single frame remedial branches per unit. 19 Figure 5 An IPI segment demonstrating a more sophisticated remedial branch involving a sub-branch. 20 Figure 6 Pass Forward Branching 20 Figure 7 This figure is an example of a much more complicated sequence involving remedial branching and a number of sub-branches. 21 Figure 8 An IPI sequence with three possible responses. 23 Figure 9 The current version of the multi-component working memory model (Baddeley, 2000). 30 Figure 10 The cognitive theory of multimedia learning. 32 Figure 11 A CLT-PI frame. 33 Figure 12 Mean Study Times 48 Figure 13 Mean Retention Scores 51 Figure 14 Mean Transfer Scores 52 iv

PAGE 8

Cognitive Load Theory and Programmed Instruction Michael L Brazas ABSTRACT Programmed Instruction was a dominant force in educational research during the 1960s and early 1970s. During this period a new cognitively oriented psychology of education arose that eventually replaced the behaviorally driven research into programmed instruction. For some reason little work was done on integrating these two approaches. This study was designed to test a programmed instructional system based on the intrinsic programmed instructional technology of Crowder (1961) but incorporating design elements derived from cognitive load theory (Sweller, 1988). Specifically, this study tested a multimedia programmed instructional system (CLT-PI) based on work by Mayer (2001) against a text based system (TXT), a traditional linear programmed instructional system based on Skinner (1954, 1958)(LPI), and an intrinsic programmed instructional system based on the work of Crowder (1960, 1961)(IPI). Three hypotheses were tested; 1) Programmed instruction would produce higher long-term retention and transfer scores than TXT, 2) IPI would produced higher long-term retention and transfer scores than LPI, and 3) CLT-PI would produce higher long-term retention and transfer scores than LPI or IPI. 115 undergraduate university students were randomly assigned to one of four conditions where they studied a chapter on human sleep. Each condition was presented on a computer with a test following study. A long-term test was given 4 weeks later. v

PAGE 9

Both shortand long-term tests contained retention and transfer questions. Analyses were conducted using repeated measures MANOVA. A series of ANOVA tests were conducted to determine specific effects and interactions. The first hypothesis was partly supported in that CLT-PI and IPI produced higher long-term retention scores than TXT. LPI, however, did not. Tests for hypothesis 2 found IPI better than LPI, but only for long-term retention. Tests for the third hypothesis found that CLT-PI produced higher long-term retention scores than TXT or LPI. Long-term retention scores for the two intrinsically based programs, CLT-PI and IPI, were equivalent. No significant differences long-term transfer effects were seen. The retention tests support past research on multimedia instruction but the lack of significant differences for transfer do not. Possible effects related to study time and an effect for auditory narration were discussed. vi

PAGE 10

CHAPTER ONE INTRODUCTION GOALS OF THE STUDY Education is undergoing a period of transition with efforts occurring at both the state and national levels aimed at 'improving education.' This is not a phenomenon unique to the present. In the late 1950s the United States educational system was carefully scrutinized in response to the Soviet Union's launch of Sputnik. One of the more successful technological solutions came from the field of behavioral psychology. B.F. Skinner introduced a behaviorally based technology of instruction that came to be known as programmed instruction (Skinner, 1954). By the early 1960's Programmed Instruction (PI) was being touted as the most important innovation in the history of education (Lysaught & Williams, 1963). Coincidently, the cognitive revolution was sweeping psychology at the same time. Over the next 15 years as cognitive psychology grew in popularity, behaviorism's popularity diminished. There is scant evidence of systematic attempts to combine elements of both psychological approaches, to devise a system utilizing the empirically supported methodology of behaviorism and the newly discovered principles of human memory and learning derived from research in cognitive psychology. The goals of the present project were two-fold. The first goal was to develop a new, programmed instructional system that would incorporate the Cognitive Load Theory (CLT-PI). The second goal was to evaluate different forms of self-study methods, including CLT-PI, in terms of long-term retention and transfer of knowledge. The CLT-PI system was compared to three other self-study methods: Traditional text-based instruction (TXT), Skinner's Linear Programmed Instruction (LPI), and Crowder's Intrinsic Programmed Instruction (IPI). My hypotheses were that 1) programmed instructional methods (i.e., CLP-PI, LPI, and IPI) would prove superior to the TXT system in terms of long-term retention and transfer of knowledge and that 2) CLT-PI 1

PAGE 11

would be the most effective of all the programmed instructional methods. This was an examination of non-programmed learning vs. 3 variations of programmed instruction. MEASUREMENT Various measures can be used to study the effectiveness of different instructional strategies. In this study, long-term retention and transfer of knowledge were selected as the dependent measures for the following reasons. Long-term retention is a vital measure of learning when considering the practical application of learning. Unlike short-term retention, long-term retention concerns the aspects of learning that are retained and can be applied outside the training environment. In both educational and professional settings, there are varying time intervals between the completion of training and the time people need to use what they learned (Druckman & Bjork, 1991; Healy, Wohldmann, & Bourne, 2005). Therefore, training should provide for sufficient retention so as to extract the full value of the training investment. Transfer of knowledge is also important. Commonly, learners acquire the knowledge and skills that will be needed in their professions in environments that are radically different from those of the workplace. Training situations can vary significantly in environment, the tools used, and the location of the workplace. Furthermore, in the workplace, the skills learned are not always utilized exactly as they were in the training setting (Druckman & Bjork, 1991; Healy et al., 2005). Finally, the rate of technological change seems to be accelerating. This creates an environment that requires workers who can upgrade their skills, integrate new knowledge, and flexibly adapt to knowledge-induced changes in the workplace (Healy et al., 2005; Lermack, 2003; Wagner, 2003). This ability to adapt relies on the ability to transfer previously learned knowledge to new situations (Druckman & Bjork, 1991). 2

PAGE 12

SIGNIFICANCE a. Why do different types of programmed instructional systems need to be studied? There is an urgent need for reliable forms of self-study because the rate of technological change has created a work environment that requires continuing education (Lermack, 2003). Workers must be able to upgrade skills, integrate new knowledge, and flexibly adapt to knowledge-induced changes in the workplace on short notice and with little supervision (Globerson & Korman, 2001; Wagner, 2003). Both industry and government need training systems that can be easily accessed by distance workers as well as centrally located employees (Isaacs, 2003). Computer-based self-study programs with a programmed instructional format may be one effective answer to these demands. Programmed instruction can be defined as a method of instruction in which new subject matter is presented to learners in a graded sequence of controlled steps. Learners work at their own speed, responding frequently in interaction with the program, and demonstrating comprehension of the subject matter at each step before proceeding in the program (Lumsdaine, cited in Hughes, 1962; Tennyson & Schott, 1997). Although a number of programmed instructional systems have been developed, no systematic comparisons have been made between the various programmed instructional methods and the text-based instructional method. It is important to clarify the nature of each of these instructional systems and to further identify the most effective educational method. b. Why does a new, programmed instructional system (CLT-PI) need to be developed? Previous programmed instructional systems (e.g., LPI and IPI) have been behavioral in orientation (Saettler, 1990; Tennyson & Elmore, 1997). These behaviorally oriented programmed instructional systems have been proven to be effective instructional methods, but have limitations that are directly associated with the fundamental principles of behaviorism (see discussion below). 3

PAGE 13

In the late 1950's the study of human learning began shifting to a cognitive viewpoint. This change in philosophical orientation involved a shift from the behaviorally dominant study of procedures for manipulating instructional materials to one of facilitation of learner processing and interaction (Saettler, 1990). In particular, research on Cognitive Load Theory (CLT) has shown how manipulation of learning materials with the goal of controlling working memory load can result in improved learning (Mayer & Moreno, 1998, 1999, 2002a, 2002b, 2002c, 2003; Pollock, Chandler, & Sweller, 2002; Sweller, 1988, 1994; Van Gerven, Paas, van Merrienboer, & Schmidt, 2002; van Merrienboer, Schuurman, de Croock, and Paas, 2002;). This study was an attempt to develop a programmed learning system that utilizes recent advances in cognitive learning theories while incorporating proven behavioral strategies. BACKGROUND OF THE FOUR GROUPS 1. TXT a. What is TXT? The text-based instruction (TXT) is defined as non-programmed instructional messages (Mayer, 2001). It can consist of text information, as well as photographs, tables, and other static graphic illustrations. Materials used for TXT are the predominant form of educational technology in use in the world today. TXT can be delivered using a number of technologies. The most common format is printed on paper, but computer presented TXT is rapidly becoming a standard. b. Operational definition of TXT: In the present study, TXT was presented in the form of an electronic book (eBook), which is textual material presented on a computer monitor. The Open eBook forum, an international trade organization, offers this definition: An eBook is a literary work in the form of a digital object, consisting of one or more standard unique identifiers, metadata, and a monographic body of content, intended to be published and accessed electronically (Open eBook Forum, 2000, pg. 6). Figure 1 is a sample frame from the TXT material that includes both text and a graphic with a caption. In the TXT program, a frame consisted of a full screen on the computer monitor. 4

PAGE 14

Night Night Night Day Day Day Boston Boston Boston Paris Paris Paris Figure 1: Sample frame from TXT program. Figure 3: Jet Lag. When an airplane leaves Boston at 9:00 p.m., it is already 3:00 a.m. the next day at its Paris destination. As a result of flying into earlier time zones, sunrise occurs only 3 or 4 hours later, when the passenger has just begun to sleep. On landing in Paris at 8:00 a.m. local time, it is only 3:00 a.m. in the passengers brain, which is not only still sleepy but also sleep-deprived. The result is jet lag. Based on Hobson, J. A. (1989), Sleep. Jet Lag and Shift Work Jet lag is a modern condition, a direct result of modern transportation systems that allow us to cross multiple time zones in short periods of time. Jet lag is the desynchrony between your internal clock and the external environment that occurs as a result of this rapid crossing of time zones. The symptoms of jet lag, which include trouble sleeping at appropriate times, daytime sleepiness, headaches, stomachaches, irritability and decreased alertness, vary among individuals and are related to the number of time zones crossed. This is an important point. Distance traveled is not the critical factor in jet lag. Travel across meridians, the north-south lines that are used to define time zones, can result in jet lag, North or south travel, even great distances, if done within a single time zone, will not result in jet lag. Jet lag is when your internal clock is out of sync with the local time. c. What are the advantages of TXT? The main advantages of using texts as learning delivery instruments are logistic and economic in nature (Misanchuk, 1992; University of Idaho, Engineering Outreach, 2003; Verduin & Clark, 1991). There is minimal need for specialized skills in accessing textual materials; the ability to read at an appropriate level is all that is really needed. Such materials are the most cost-effective educational materials available, requiring common forms of technology. 5

PAGE 15

d. Why is this important to my study? The TXT system needs to be systematically compared with programmed instruction for the following three reasons. First, although there is a body of research comparing TXT with programmed instruction (Buckland, 1967; Goldbeck & Campbell, 1962; Jamieson, James, Leytham, & Tozer, 1969; Lysaught & Williams, 1963; Pikas, 1969), many of the studies were conducted over 30 years ago, when modern computer technology was not available. Because programmed instructional methods today are almost exclusively presented using computer systems, comparisons among different instructional methods must be made utilizing comparable technology. For this reason, the TXT material was updated to evaluate its effectiveness using a computer presentation system. Another problem is that previous studies used a variety of independent and dependent measures so that it is impossible to draw precise conclusions as to the exact relationship between measures. For example, long-term retention has been loosely defined with varying delay periods from training. Buckland (1967) used a 27-day delay from training, Buckle (1967) assessed LTR at 6 weeks, Berglund (1969) retested at 3 months, and Jamieson et al. (1969) tested at 5 months. In a large review of factors affecting student performance in programmed learning, Hartley (1968) reports retest (LTR) periods of as short as 2 days to as long as 1 year. Finally, many researchers failed to include a non-programmed control group, concentrating instead on testing new manipulations against earlier programmed instructional designs. Buckland (1968) used a previously studied program as a control in an investigation of the ordering of frames in a linear program. Berglund (1969) used a previously constructed but unreported program in a study of partial reinforcement. Even when a non-programmed control group was used, this lack of control was common (e.g., Balson [1969] used an untested text version of Holland and Skinner's The analysis of behavior program [Holland & Skinner, 1961] as a control). 6

PAGE 16

2. LINEAR PROGRAMMED INSTRUCTION a. What is Programmed Instruction? Programmed instruction is derived from behavioral theories of learning with the goals of shaping behavior into predetermined patterns by strengthening stimulus-response bonds (Entwistle, 1994). Materials used for programmed instruction require precise learning objectives (terminal behaviors) that are specified before the writing of the program begins. Learners are required to complete a sequence of tasks designed to achieve these objectives. The origin of programmed instruction can be traced back to the early 20th century. Based on behavioral learning principles evolving out of the work of Thorndike (1911) and Watson (1913), Pressey (1926) developed the first device for teaching based on the laws of learning (cited in Lumsdaine & Glaser, 1960). This machine gave immediate feedback as to the correctness of responses. Pressey found that students who used the testing machine actually learned from the machine, correcting their errors and achieving higher retention levels compared to students studying the same material without use of the machine (Angell & Troyer, 1948; Pressey, 1926; 1927; 1932). Since then, a number of different programmed instructional systems have been developed. Pressey continued to refine his ideas, creating a primitive branching device called the punchboard (Pressey, 1950). This device was followed by the Drum Tutor, which was further researched and reported by Stephens (1953) who compared both devices to control groups using non-programmed materials, finding higher exam scores for the machine taught students. In 1954 Skinner introduced the operantly based LPI technology that came to dominate programmed instruction for several decades. Based primarily on the work of Skinner, a set of characteristics common to all programmed instructional systems evolved. These include: Each learner works independently at his/her own pace. Although this can be true in any non-classroom study system by definition, programmed instruction is specifically designed to make this possible. 7

PAGE 17

The material to be learned is broken down into logical units. The presentation sequence of these units is planned and tested for optimal learning. The logical units of information (stimuli) are called frames. The learner is required to respond to each frame before being allowed to advance to the next frame within the program. The learner is given feedback as to the accuracy of his/her response. This cycle of presentation-answer-feedback (a stimulus-response-reinforcement sequence) is repeated with frames designed for introduction of new information, practice and review, and assessment of learning. b. What is Linear Programmed Instruction? It is generally agreed that the different types of programmed instructional systems can be divided into two schools; Linear Programmed Instruction (LPI) and Intrinsic Programmed Instruction (IPI), also known as the branching technique (Cram, 1961; Leith, 1967; Markle, 1969). An LPI program is based on a set of principles outlined by Skinner (1954, 1958). It consists of a sequence of small units of information presented sequentially to the learner (Deterline, 1962; Skinner, 1961). In each unit, called a frame, the learner must generate a response to some prompting stimuli, which are typically in the form of a question, or blank spots in a sentence that must be filled in. LPI has the following characteristics (Hughes, 1962; Lysaught & Williams, 1963; O'Day, Kulhavy, Anderson, & Malczynski, 1971; Skinner, 1954, 1958) All instruction is goal oriented. Each program step should help the learner to acquire the specific knowledge or skills outlined in the terminal behaviors list and to avoid superfluous material. Instructional items should be ordered into an effective sequence. Skinner believed that an LPI program should be designed to essentially eliminate the probability of the learner making errors (errorless learning). Skinner designed programs using text only to shape verbal behavior. Graphic material was presented in supplementary materials. 8

PAGE 18

Present only one point at a time. The presentation of small, self-contained units of information focuses the students attention. The learner proceeds through the program one frame at a time. Overt responding. The learner overtly responds to each frame of the program. In LPI, the learner must type in the answer to a question or the word that completes a sentence (a constructed response). Immediate reinforcement. The learner is given knowledge of the accuracy of every response in order to strengthen the stimulus-response bond. Figure 2 is a block diagram illustrating the flow of a typical linear program. The instructional frame provides the learner with a small unit of information and prompts for a response (either answering a question or completing a sentence by filling in a blank). The learner is then given feedback as to the correctness of the response. The program will then advance to the next frame. There is no provision for correcting errors or for giving remedial training. Figure 2: A block diagram illustrating frame arrangement in a typical LPI segment. InstructionalFrameInstructionalFrame ResultsFrame(Reinforcement)ResultsFrame(Reinforcement) The learner must complete the program with a score that meets a preset criterion level. Scores below the criterion level result in a repeated pass through the program. Some programs will shift frames that have elicited errors to the end of the program. In either case, the learner must reach criterion in each unit before advancing (Lysaught & Williams, 1963; Markle, 1969). 9

PAGE 19

Time Correct! FRAME 3 FRAME 2 Jet lag is a modern condition, a direct result of modern transportation systems that allow us to cross multiple _____ zones in short periods of time. Press "ENTER" to advance to the next frame. FRAME 1 Jet lag is a modern condition, a direct result of modern transportation systems that allow us to cross multiple _____ zones in short periods of time. Figure 3: An LPI sequence. In frame 1, the learner is shown a frame containing information with a formal prompt. The blank in the sentence prompts the learner to respond with the word "time." Frame 2 shows the constructed response. Frame 3 demonstrates the presentation of a reinforcer, knowledge of a correct response. An LPI program generally contains a number of frames covering a single topic. Figure 3 illustrates this. This sequence of frames covers the same material presented in a single TXT frame (See Figure 1). c. The origins and evolution of LPI. LPI was strongly influenced by the work of B.F. Skinner. By 1960, LPI techniques were used extensively in every level of education in a number of countries. Researchers 10

PAGE 20

working under contract for the U.S. government began extensive research on training programs using LPI technologies as early as 1954 (Lumsdaine & Glaser, 1960). LPI was used to teach topics as varied as physics (Skinner, 1958), foreign language learning (Ferster & Sapon, 1958), mathematics (Keislar, 1959), and visual discrimination (Hively, 1962). In 1964 Fred Keller began work in Brazil on what came to be known as the Personalized System of Instruction (PSI). In 1968, Keller published Goodbye teacher outlining a system of instruction based on Skinners LPI. A unique aspect of PSI is the use of proctors. Proctors provide assistance when learners cannot advance through the programmed material on their own. Proctors assist in the revision of course programs, providing feedback to the instructor on problem areas they encounter in their interactions with students. This system has proven to be quite powerful and persists in a large number of educational settings to this date (Brothen, Wambach, & Hansen, 2002; Buzhardt & Semb, 2002; Eppler & Ironsmith, 2004; Martin, Pear, & Martin, 2002). Although the use of a proctor is ideal, a competent proctor may not always be available, especially in the workplace. A number of guides to program writing have been published over the years. Most tend to elaborate on the work of Evans et al. (1960). This guide, The RULEG system for the construction of programmed verbal learning sequences," outlines a system based on the premise that verbal material can be classified into two classes of statements, RUs (rules) and EGs (examples). Accordingly, the RULEG system includes a series of rules (RU), or semantic concepts, and a series of examples (EG's) that illustrate and give the learner practice working with the RU (Evans et al., 1960; Markle, 1969). Because the RULEG system is simple to learn and implement it has found wide acceptance among programmed instructional writers (Lysaught & Williams, 1963; Markle, 1969). The RULEG system is still taught in programmed instructional courses such as the Principles of Learning course (EDF 6215) offered by USFs College of Education. 11

PAGE 21

d. Research comparing LPI with conventional instruction. Are programmed instructional methods more effective than traditional instructional methods using text? There have been a number of studies on this issue. In at least three aspects of learning (i.e., short-term retention, long-term retention, and satisfaction), the results tend to support the assertion that LPI is more effective and useful than traditional instructional methods. Critical questions have been left unanswered in the aspect of transfer of knowledge. Short-term retention: The majority of investigations into LPI used gain (preto post-study improvement) or immediate post-study test (short-term retention) scores as a dependent measure. Senter, Abma, Johnson, and Morgan (1966) compared LPI to narrative text in a study teaching binary and octal arithmetic to Air Force trainees. They found significantly higher gain scores on a criterion test with the LPI group. In a study comparing LPI and conventional textbook instruction in a college psychology course Daniel and Murdoch (1968) found LPI to produce higher scores on a short-term retention test. Research has been conducted comparing LPI with various forms of lecture. In a study of educational psychology students comparing LPI with augmented (audio-visual) lecture and straight lectures gain scores were found to be highest for the LPI group followed by the augmented lectures, then the straight lecture. Studying response type, Pikas (1969) found college students learning from a programmed text to have higher short-term retention scores compared to control groups studying from a conventional text or listening to a lecture. This research demonstrates the superiority of LPI in short-term retention over conventional instructional methods. Long-term retention (LTR): Far less work has been published investigating long-term retention with LPI. However, the work that has been done showed an advantage of LPI over conventional instructional methods for long-term retention. Goldbeck and Campbell (1962) found no significant differences between an LPI group and a text-based control group on short-term retention scores but significantly higher scores in a 10-week follow up. Buckland (1967) showed significant advantages for students studying mathematics using LPI over conventional instruction in terms of LTR, assessed 27 days post-study. 12

PAGE 22

Satisfaction: Students who compared LPI to conventional instruction tend to report preferring LPI over conventional instructional methods (Holland, 1959). In 10 of 11 studies using Keller's PSI with LPI as a principle instructional system, the student satisfaction ratings were higher for PSI than for conventional instructional systems (Kulik, Kulik, & Cohen, 1979). Fleischman (1968) assessed learners attitudes toward programmed instruction in military training programs, finding that naval and marine enlisted students reported high levels of satisfaction, favoring LPI over traditional training methods. Transfer of Knowledge (TOK): As with investigations into long-term retention, transfer of knowledge research with LPI is limited. This small body of research has failed to reliably determine the effect LPI has on transfer of knowledge. Davis, Marzocco, and Denny (1970) in a study teaching algebra using an LPI system compared to conventional instruction found higher levels of short-term retention for the LPI group but slightly higher levels of transfer of knowledge for the conventional group. Other research has failed to demonstrate any significant differences in transfer of knowledge compared to conventional instruction. Balson (1969) compared the transfer value (a measure of transfer of knowledge) of LPI and traditional lecture in the training of student teachers. This study failed to find any significant difference in transfer of knowledge between groups, however the LPI group did have superior short-term retention scores. e. What are the advantages of LPI? Why is LPI a more efficient method than traditional instructional methods? Some argue that at least two important characteristics, response contingency and overt responding, account for the superiority of LPI. Researchers have suggested that a major factor contributing to the superiority of LPI over conventional study is the fact that LPI is based on response-contingency (Berman, 1970; Sime & Boyce, 1969). These studies have shown that response-contingent (programmed) materials result in higher post-test scores than educational materials without response contingencies (Berman, 1970; Sime & Boyce, 1969). An overt response in programmed instruction is simply any response made by the learner in response to a program prompt that can be measured. An overt response can be 13

PAGE 23

as simple as the requirement that the learner press the enter key to advance to the next frame, or it can be as complicated as the requiring the learner to recall some facts from memory, make calculations using those facts and to construct an answer by typing it into the computer. Although it has been argued that overt responding is not necessary for learning to occur (Reid & Taylor, 1965; Valverde, 1968; Yarmey, 1964), it is an essential element of the automated learning process provided by programmed instruction (Skinner, 1954; 1958). In programmed instruction, regardless of step size or response type, the program must be informed when the learner is ready to advance to a new frame. f. What are the limitations/problems of LPI? Previous studies also revealed disadvantages of LPI when compared to traditional and other programmed instructional methods. Many of the foundational principles of LPI, when strictly applied, limit the effectiveness of the program. The limitations of LPI can be divided into four categories; issues related to 1) step size, 2) immediate reinforcement, 3) constructedresponses, and 4) a lack of corrective programming. Small steps: In LPI, a learner can see and respond to only very limited information at one time. This small step system is essential to LPI for conceptual and practical reasons. Skinner believed that a gradual shaping of behavior was necessary to ensure that learning would occur (Skinner, 1954, 1958, 1968). Therefore, a single frame contained at most two or three sentences of text. Supplementary graphic material was provided in a workbook. It has been speculated that the idea of small step is actually an artifact of the limitations of the technology used in the first teaching machines (Leith, 1967). For practical reasons, early teaching machines used paper rolls or disks encased in devices that only allowed a single frame to be viewed at any given time (Lumsdaine, 1959). The limitations created by this technology only allowed for a small window or frame of information to be seen. The physical window size may have been instrumental in the small step size principle. A number of studies investigated the small step principle and found that this small step size does not positively influence learning outcomes (Jones & Hick, 1969; Klaus, 1964; Krieger & Dechert, 1974; Smith & Moore, 1962). However, the use of small step size in every frame places unnecessary limitations on the programmer. Because some 14

PAGE 24

concepts are difficult to break into the small frames that LPI calls for, frame size should be dictated by the content of the material being taught in a particular sequence. Also, the use of small frames increases the total number of frames needed in a sequence, adding to the overall size of a program and increasing the time needed to complete a learning program. Immediate reinforcement: In strict LPI, after a learner responds, he/she is immediately informed as to the correctness of the response. This is referred to as knowledge of results or knowledge of correct response, which is thought to be reinforcing and necessary for learning (Skinner, 1954; Stolurow, 1961). Although there are studies supporting this premise (Anderson, Kulhavy, & Andre, 1971), other studies suggest that reinforcement does not have to be delivered immediately after a response. For instance, Jacobs and Kulkarni (1966) investigated the effects of knowledge of results by comparing learners using different versions of an LPI program. One group received knowledge of results on each frame, and the second group received no knowledge of results. The group that received knowledge of results on each frame actually had lower post-study retention scores than the group receiving no knowledge of results. Merrill, Yaryan, and Musser (1969) manipulated reinforcement in two studies on medical and dental students in a programmed learning environment. The authors found no differences between conditions where reinforcement schedules (fixed and variable) were manipulated. As long as reinforcement was provided during the course of the teaching program, learning occurred. Therefore, it appears that knowledge of a correct response can serve as a reinforcer in programmed instruction. This knowledge must be provided to the learner when an appropriate response is made, but the timing of the delivery of the reinforcer is not as critical as Skinner believed. Constructed responses: Skinner (1954;1958) believed that an indispensable element of programmed instruction is constructed responding, which forces the learner to recall, not just recognize, a correct response. However, the idea that the constructed response is superior to other forms of responding was controversial. There are studies showing little 15

PAGE 25

support for this contention (Cauzinille & Weil-Barais, 1974; Davis et al., 1970; Shah, 1971). In these studies, researchers often compared the effects of different types of response format (e.g., constructed responses, partial constructed responses, click to advance, and multiple-choice responses) and found no clear advantages of constructed responses. Other studies, however, suggest constructed responding has a place in programmed instruction. Tudor and Bostow (1991) conducted an investigation into this question, where students learned programmed instruction writing techniques in a linear program. This study was replicated and extended by Kritch, Bostow, and Dedrick (1995) and Kritch (1995). In comparisons of LPI with constructed responding vs. LPI with other overt and covert response types (narrative text, click to advance with correct response supplied) learners in the constructed response group outperformed the other groups on gain scores and transfer in a program writing condition. These data generate the question: Are constructed responses required and, if so, under what conditions? The answer may lie in a series of experiments that suggest that the type of material being taught may influence the effectiveness of constructed responses. A study using standard constructed responses, multiple-choice responses, and a combination format where constructed responses were used on frames that required technical terminology as responses and multiple-choice where general vocabulary was required found a significant advantage for the combination format (Williams, 1966). Similar results were seen in two other studies. Tobias (1969) compared results in terms of the technical difficultly of the material being taught and found that constructed response items result in higher scores for more technical material. An elaboration of this study (Tobias & Abramson, 1971) examined how reinforcement affects outcome in a constructed response programmed learning environment. In a comparison of reinforced constructed response, non-reinforced constructed response, or a reading condition the reinforced constructed response group achieved higher retention scores. Therefore when the material to be learned has a higher degree of difficulty (as in technical terminology) the use of constructed responses may increase the amount of retention, especially long-term retention. 16

PAGE 26

The use of constructed responding for all frames is contraindicated, however. Constructed responding places limits on the programmer that multiple-choice or covert responding does not (Markle, 1969). In a constructed response frame the programmer must anticipate every possible response the learner can make to the question in that frame. Each of these possible responses must be factored into the program. This restriction requires an intimate knowledge of the productive vocabulary of every learner that will use the program, a task that may be impossible. Using constructed responding only to teach specific, technical terminology would avoid potential problems of this nature. Corrective programming: LPI is designed to maximize the probability of reinforcement while minimizing the probability of error (Skinner, 1954, 1958). Thus, when a learner does make an error in an LPI sequence, the program responds by indicating that an error was made, and in some cases providing the correct answer. However, no provision for determining why the error was made or for correcting the thinking of the learner is provided. Cognitive learning theories emphasize the role of learner in the learning process (Reigeluth & Moore, 1999; Saettler, 1990). Intrinsic Programmed Instructional systems are based on the assumption that we learn from our mistakes (Crowder, 1961). Corrective branching allows the learner to see where an error was made and gives remedial training that allows the learner to correct this error in a guided manner. Two basic assumptions about the way humans learn influence the way learning programs are constructed. The first assumption is that "the learner is a receptive mechanism for whom associative connections become formed so as to mirror experience (Stolurow, 1961, p. 51). The second assumption is that "the learner is a selective, self-organizing mechanism who selects and extracts information from the environment (Stolurow, 1961, p. 51) (The difference is between a strict behaviorist orientation and that of early cognitive psychology [Saettler, 1990]). It is also the defining critical difference between LPI and IPI (Crowder, 1960, 1961). Because the CLT-PI program was constructed from a constructivist/cognitive orientation, corrective branching was an integral part of its design. 17

PAGE 27

g. Why is this important to my study? As discussed above, a number of the initial principles of LPI as described by Skinner have been challenged. Nevertheless, LPI is the most widely studied form of PI, with almost 50 years of empirical research into the various aspect of programmed learning (Buckland, 1967; Crosbie & Kelly, 1994; Davis et al., 1970; Holland & Skinner, 1961; Jamieson et al., 1969; Kritch & Bostow, 1998; Kulhavy, Yekovich, & Dyer, 1979; Little, 1934; Pressey, 1926; Skinner, 1954, 1958). Thus, it is important to evaluate the effectiveness of LPI in comparison to the other instructional methods used in the present study. 3. INTRINSIC PROGRAMMED INSTRUCTION a. What is Intrinsic Programmed Instruction? Intrinsic Programmed Instruction (IPI) is a behaviorally based system intended to automate the instructional process (Crowder, 1960). As in LPI, there are certain characteristics that IPI shares with other programmed instruction systems including knowledge of response and overt responding. However, intrinsic programming can be differentiated from other forms of programmed instruction in its use of branching. In IPI the program adapts to the responses of the learner. When the learner correctly answers a query he/she is advanced in the program sequence. When the learner responds erroneously, the program responds by routing to a remedial branch. The learner only advances to a new topic when he/she has demonstrated understanding by responding correctly to a diagnostic probe question. Linear programs, on the other hand, are not constructed with such error correction in mind. An LPI program is designed to teach with minimal chance of error on the learner's part. In strict LPI programming, the learner will advance through the sequence of frames, responding appropriately to every prompt with a maximum number of reinforcers provided. In IPI, the learner is first presented with a block of information to be learned. Immediately after reading this block of information the learner is given a short test, typically a single multiple choice question, to assess his/her understanding of the 18

PAGE 28

material. The results of the test question are used to determine the learner's progression through the program. If the response is correct, the program advances to the next topic. If incorrect, the program routes the learner to a frame or series of frames for remediation before returning to the original assessment frame (Crowder, 1960). FRAME 1InstructionalFrame w/question. FRAME 3RemedialFrameFRAME 5RemedialFrameFRAME 6RemedialFrameFRAME 4RemedialFrameFRAME 2InstructionalFrame w/question. FRAME 5RemedialFrameFRAME 6RemedialFrame Correct Response Correct Response Incorrect Response Incorrect Response Figure 4: A Simple Branching Program. A simple branching p rogram with one correct response path and two single frame remedial branches per unit. IPI programs can be constructed with various levels of complexity. A simple program will contain a series of sequential frames with remedial branches that a learner can be diverted into in the case of an error in a frame. Figure 4 illustrates a simple branching program segment; Figure 5 demonstrates a more sophisticated program that contains a remedial branch with sub-branches. 19

PAGE 29

FRAME 1InstructionalFrame w/question. FRAME 1ARemedialFrameFRAME 2ARemedialFrameFRAME 5RemedialFrameFRAME 6RemedialFrameFRAME 2InstructionalFrame w/question. FRAME 1CRemedialFrame FRAME 1DRemedialFrameFRAME 1DRemedialFrameFRAME 2BRemedialFrameFRAME 2ARemedialFrame Incorrect Response Incorrect Response Branch Correct Response Figure 5: An IPI segment demonstrating a more sophisticated remedial branching system involving a sub-branch. Figure 6 diagrams a branching technique called a pass forward. In a pass forward the answer to a question (Frame 1) determines if the learner has grasped a concept early and is ready to move on to the next concept. A correct response on Frame 1 allows the learner to skip several frames. An incorrect response on Frame 1 forces the learner to complete the standard sequence. Figure 6: An IPI segment demonstrating how the program can be used to determine if a learner can pass forward or must follow an alternative, more intensive, path FRAME 1InstructionalFrame w/question. FRAME 5AlternatePathFRAME 6AlternatePathFRAME 7AlternatePath FRAME 2InstructionalFrame w/question.FRAME 4InstructionalFrame w/question.FRAME 3InstructionalFrame w/question.FRAME 3InstructionalFrame w/question. 20

PAGE 30

Figure 7 shows a short intrinsic sequence that contains a relatively complicated set of remedial branches and sub-branches. The imagination and skill of the instructional designer are the only limitations to the design of a remedial branching sequence. FRAME 1InstructionalFrame w/question. RemedialBranch 1Frame 1 Correct response RemedialBranch 1Frame 2RemedialBranch 2Frame 1RemedialBranch 3Frame 1RemedialBranch 2Frame 2FRAME 2InstructionalFrame w/question.RemedialBranch 1Frame 1RemedialBranch 2Frame 1RemedialBranch 1Frame 1FRAME 3InstructionalFrame w/question.RemedialBranch 2Frame 1RemedialBranch 3Frame 1FRAME 4InstructionalFrame w/question.RemedialBranch 1Frame 1RemedialBranch 2Frame 1 Incorrect response Correct responseCorrect respons e Figure 7: An example of a complicated branching sequence that contains remedial branching and a number of sub-branches. b. Conceptual differences between IPI and LPI. Crowder (1961) classified IPI as a 'responsive' form of programmed instruction and LPI as 'non-responsive,' based on the structural differences between these programs. An LPI program does not change (adapt) depending on the responses of the learner, while an IPI program does. Therefore, all learners progress through an LPI program in the same way. In LPI, the only differences encountered are in the number of 'correct' or 'incorrect' response frames different learners see. However, if the complexity of an IPI program reaches a certain level, it is possible for hundreds of learners to go through the same program without two seeing the same sequence of frames. 21

PAGE 31

These two systems also differ in terms of their basic views regarding how learning is accomplished. In LPI, a learner is prompted to emit a response in order to receive a reinforcer (Hughes, 1962). Skinner believed that this sequence of stimulus response reinforcement is instrumental in the learning process (Skinner, 1954; 1958). On the other hand, in IPI, responses of the student are used to diagnostically monitor the effectiveness of the teaching program and to alter the path through the program to maximize learning efficiency (Crowder, 1961; Hughes, 1962). Crowder (1961, pg. 60) described the IPI programmer's goals with regards to questions within a program, stating that a routine question should serve "1. to determine whether the student has learned the material just learned; 2. to select appropriate corrective material if the student has not learned; 3. to provide desirable practice with the concept involved; 4. to keep the student actively working at the material; and, 5. to fill a desirable motivational purpose if the student answers correctly." Figure 8 contains a sample of a computer presented IPI sequence. This sequence consists of an information frame with a multiple-choice question and 3 possible branch responses. 22

PAGE 32

xt e 2). Three individual fly out of Los Angeles, one to Chicago, one to New York, and one to Anchorage. Travel to which destination will most likely result in jet lag? a ) Lo s An g eles b ) Chica g o c ) Ne w York FRAME 4 Your response was Anchorage. This is incorrect, while Alaska may be further from Los Angeles than the other two cities; it is not as many time zones different. Remember, the distance traveled is not as critical with jet lag as the number of time zones crossed. Try Again! Three individual fly out of Los Angeles, one to Chicago, one to New York, and one to Anchorage. Travel to which destination will most likely result in jet lag? a ) Lo s An g eles b ) Chica g o c ) Ne w York FRAME 1 Three individual fly out of Los Angeles, one to Chicago, one to New York, and one to Anchorage. Travel to which destination will most likely result in jet lag? a) Los Angeles b) Chicago c) New York FRAME 2 Figure 8: An IPI sequence with three possible responses. In Frame 1, the learner is presented with a paragraph of material on one topic and an assessment question is given. Clicking on one of the answers routes the learner to one of the next 3 frames. If the response given is correct, the learner is given reinforcement and allowed to proceed to the netopic (Fram If the response is incorrect, remediation is provided and the learner is given the opportunity to repeat the assessment question (Frames 3 and 4). Your response was Chicago. This is incorrect; Chicago is only 2 times zones east of Los Angeles, 1 less than New York. Remember, the severity of jet lag is a function of the number of time zones shifted away from the home time zone. A two-hour shift is not as severe as a three hour shift. Try Again! FRAME 3 Correct! New York is three times zones east of Los Angeles. Jet lag is a modern condition, a direct result of modern transportation systems that allow us to cross multiple time zones in short periods of time. Jet lag is the desynchrony between your internal clock and the external environment that occurs as a result of this rapid crossing of time zones. The symptoms of jet lag, which include trouble sleeping at appropriate times, daytime sleepiness, headaches, stomachaches, irritability and decreased alertness, vary among individuals and are related to the number of time zones crossed. This is an important point, distance traveled is not the critical factor in jet lag. Travel across meridians, the north-south lines that are used to define time zones can result in jet lag, North or south travel, even great distances, if done within a sgle time zone will not result in jet lag. Jet lag is when your internal clok is out of sync with the local time. inc 23

PAGE 33

c. Research comparing IPI with other instructional methods. Research origins: No systematic research comparing the various elements of LPI to IPI has been published. A number of studies examining various elements of programmed instruction from an IPI perspective have been done, but these are not as numerous as the studies comparing LPI to conventional instruction. A possible reason for the lack of publications on IPI research comes from its origins. LPI and IPI systems have different origins. LPI has its roots in academic research into learning. Pressey and Skinner were both university professors interested in the learning process. LPI is a direct product of this approach. IPI systems have their roots in U.S. military research and development programs. This research was intended to find the most efficient and cost effective way to teach military recruits. Where LPI researchers were seeking to answer specific questions regarding learning in programmed environments, IPI researchers were working to advance the efforts of the United States Military during the cold war environment (Lumsdaine & Glaser, 1960; Saettler, 1990). Crowder (1961, pp. 58-59) describes a difference in design philosophy that comes from separate theoretical backgrounds. LPI is the product of classical experimental psychology and views the learners response as an integral part of the learning process and therefore as a legitimate end in itself. The goal of the instructional program is to produce a specific response or set of responses from a learner in the presence of specific stimuli. IPI, on the other hand, is the product of what Crowder calls differential psychology and is primarily interested in using the students responses to control the course of the programmed material presented to that particular student. This is a feedback approach to programming. The learners response provides feedback to the program about the level of understanding at that particular point in the program. The goal of the instructional program is to facilitate learning, not to condition the production of specific responses to specific stimuli. Military origins of IPI: LPI based teaching machines were developed to implement behaviorally based learning theories (Skinner, 1954). IPI systems were the result of adding intrinsic programming theory to existing teaching technology (Lumsdaine and 24

PAGE 34

Glaser, 1960). The AutoTutor was an IPI based teaching machine developed by Crowder for the military and later made commercially available (Crowder, 1961; Lumsdaine & Glaser, 1960). This device was the result of a system developed by French, Crowder, and Turner (1956). An early prototype system was tested against the training programs using existing simulator devices and found to produce learning that compared favorably in teaching advanced technical troubleshooting skills to Air Force recruits. This system was found to be more cost effective with the same level of proficiency from training. Further development led to a series of specialized training systems based on Crowder's intrinsic programming techniques (Crowder, 1957; Kopstein & Cave, 1962; Kopstein, Cave, & Zachert, 1962). These systems were found to produce training results comparable to traditional military training programs but at lower cost per student (Kopstein & Cave, 1962). Migration of IPI to the private sector: Early in the 1960's researchers began adapting the military IPI system for use in civilian classrooms. Coulson, Estavan, Melaragon, and Silberman (1962) tested an auto-instructional system using IPI to teach a high school logic course. Compared to students taught conventionally (using lecture and text reading), the IPI students produced significantly higher post-study exam scores. A study investigating the differences between branching and linear programmed instruction in teaching logarithms to secondary school students found superiority for branching on short-term retention but failed to see these results on a two-week follow up test (Hartley, 1965). IPI technology has been adapted for use with differentially enabled populations. Jahoda and Jahoda (1986) created a branching program recorded on audiocassette designed to teach visually disabled colleges students. This program taught students how to voice-index recorded lecture notes, textbooks, and other assigned readings in college classes. A follow up study 22 months post-training found that 86% of these students still voice-indexed some of their recorded material. Snelbecker, Sherman, Rothstein, and Cramer (1963) conducted a study that investigated the effectiveness of a branching IPI program across different populations. A group of psychiatric inpatients was compared 25

PAGE 35

to a control group from the general population. Both groups learned from a branching IPI program "Taking care of diabetes." The major difference between the groups found was in time to complete the program, with the experimental group tending to make more errors within the program and taking longer to complete to mastery (Snelbecker et al., 1963). Types of IPI systems: The IPI systems that evolved out of military research were of a machine presentation type (Lumsdaine & Glaser, 1960). However, Crowder (1960) also devised a printed version of IPI commonly called a scrambled book. In a scrambled book the learner is routed to 'branches' that are randomly placed throughout the book. Other than its physical structure, a scrambled book works like machine based IPI. There are mixed results using the scrambled book. Senter, Nieberg Abma, and Morgan (1963) generated three versions of an IPI program; 1) unmodified; 2) removed motivational phrasing, and; 3) a straight line version with no motivation and no branching but not constructed to strict linear methodology. They found no differences between conditions in post-study scores for Air Force recruits in technical training programs. However, Senter et al. (1966) found IPI to be superior to narrative text in teaching binary and octal arithmetic. Similar results were seen by Ter Keurst (1965) in a study that tested the ability of IPI programs to enhance concept learning in less structured subjects like psychology. This study found that students learning from an IPI program scored higher on tests of conceptual knowledge compared to conventionally taught students. Some learners find LPI programming to be repetitious and report low levels of motivation to continue once the novelty has worn off (Saettler, 1990). With IPI technology this is not seen, with students reporting high levels of satisfaction with IPI programs. The structure of IPI programs is generally considered more challenging and therefore more interesting (Espich & Williams, 1967). d. What are the advantages of IPI? Due to IPI's origins in military R&D, published accounts of research are scant compared to those available for LPI. However, a review of what is available finds the common element in all IPI programs to be corrective branching in a system designed to teach to mastery before completion of the program. 26

PAGE 36

IPI has been used with a wide range of populations, from military recruits to students at the high school, college, and graduate school levels. Psychiatric and vision impaired learners have benefited from IPI training. In teaching highly technical topics like electronic device troubleshooting and repair, IPI has proven to be more efficient in terms of time and expense. IPI, when machine (computer) presented, has a consistent record of efficient teaching that results in high degrees of learner satisfaction compared to LPI systems. Due to the more challenging nature of IPI programs, they are found to be more interesting than LPI. e. What are the limitations of IPI? Like LPI, IPI teaching programs are behaviorally based and fail to take cognitive factors like the limitations of working memory into consideration (Reigeluth, 1999; Saettler, 1990). Whereas LPI may over limit the size of frames, IPI makes no procedural limits to frame size and may present too much information for effective learning. Branching programs may be presented as computer programs or in the form of "scrambled books," where the sequence of branches is randomly placed throughout the book (Crowder, 1961; Hughes, 1962). Biran (1967) compared two forms of IPI, the "scrambled-book" and a "sequential branching" format. This study found that students learning from the sequential format material had greater gain scores (pre-test to post-test comparisons. These scrambled books are confusing for many learners (Stolurow, 1961) and difficult to produce (Markle, 1969). Computer-based IPI involves quite large programs may require longer development cycles than LPI programs. IPI generally uses multiple-choice questions to assess the learner's level of understanding. However, the multiple-choice format only gives the programmer knowledge of the learner's ability to recognize (to discriminate) the correct answer when it is present among distracters. The ability to discriminate may not always be sufficient to demonstrate the depth of knowledge needed in real world situations. Therefore, some have argued that a means of assessing recall is needed (Deterline, 1962). Deterline (1962) argues that in the work environment the learner will seldom be given a choice of solutions to select from, and must rely on recall of previously learned material. 27

PAGE 37

f. Why is this important to my study? IPI was LPIs major competitive system. As discussed above, although IPI has been shown to be an effective system of self-instruction, research looking at how IPI affects long-term retention and transfer of knowledge has not been systematically examined. Also, the research examining the differences between IPI, LPI, and TXT was conducted using printed materials in some, or all, of the conditions. The present study attempted to eliminate this possible confound by presenting all four conditions using the same presentation technology. 4. CLT-PI a. What is CLT-PI? Although LPI and IPI systems have been found to be more effective than traditional text based instructional methods, there are significant limitations with both LPI and IPI, as described above. Therefore, the programmed instructional system produced in this study incorporates some of the successful components of the LPI and IPI systems. In particular, this system utilizes overt responding and response contingency from LPI and remedial branching from IPI. Furthermore, both IPI and LPI do not consider the limitations of the working memory capacity. Incorporating some of the instructional principles derived from cognitive load theory may solve this problem. b. What is CLT? The Cognitive Load Theory (CLT) (Sweller, 1988, 1994) is an instructional theory coming out of constructivist research into learning, memory, and problem solving. Constructivism approaches knowledge as an entity that is produced by the experiences of the individual learner. Therefore knowledge is actively constructed as a result of the interaction of the learner with the learning environment (Tenenbaum, Naidu, Jegede, & Austin, 2001). The cognitive load theory describes learning in terms of information construction and processing in a system of memories (sensory, short-term or working memory [WM] and long-term memory [LTM]). All information from the sensory system 28

PAGE 38

and from LTM is processed in WM. WM contains specialized subsystems for processing information based on modality (auditory and visual information are handled by separate subsystems). WM has capacity limitations, limiting the number of elements to about seven (Miller, 1956). This capacity is reduced when processing must take place. The load placed on WM due to processing demands and element interactivity is called cognitive load. Learning technologies derived from CLT involve systems and processes designed to control the load placed on WM during training. When total CL is properly managed mental effort during the learning process is reduced, the amount of information capable of being processed increases, and cognitive schema development is enhanced leading to greater ability to generalize learned information to novel situations (Sweller, van Merrienboer, & Paas, 1998). c. What are the characteristics of CLT? The Cognitive Load Theory is based on the idea of an unlimited capacity LTM and a limited capacity WM (Baddeley & Hitch, 1974; Miller, 1956). Specifically, CLT focuses on WM limitations that influence learning (which is defined in terms of schema construction) (Bartlett, 1932; Derry, 1996). Baddeley and Hitch (1974) introduced a model that divides WM into a "visual-spatial scratch pad" that deals with visually based information, a "phonological loop" that deals with auditory (primarily speech) information, and a "central executive" an attentional control system that governs the action of the entire system. Baddeley (2000) has expanded his model, adding an "episodic buffer" to handle multi-dimensional information (Figure 9 shows the current WM model with the addition of the episodic buffer). 29

PAGE 39

Figure 9: The current version of the multi-component working memory model (Baddeley, 2000). CentralExecutive VisuospatialSketchpad PhonologicalLoop EpisodicBuffer VisualSemanticsEpisodicLTMLanguage In the learning process, appropriate information must be extracted from LTM, transferred to WM, manipulated (processed or integrated with new information from the sensory system), and returned to LTM. The ease or difficulty of this processing in WM is the primary concern of cognitive load theory. WM load is influenced by several factors, the intrinsic nature of the material under study Intrinsic Cognitive Load (ICL), the way in which the material is presented or the processing required of the learner on the material Extraneous Cognitive Load (ECL), and the effort required in construction of schemas (Germane Cognitive Load GCL) (Sweller, 1988; Sweller, van Merrienboer, & Paas, 1998). ICL is a factor of the material under study and is not altered by manipulation of the instructional process or by learning technique. Some things are just harder to learn than others. ECL is determined by the design of the instruction administered. GCL is also influenced by instructional design. Good instructional design is hypothesized to decrease ECL and increase GCL (Sweller et al., 1998). The Cognitive load theory has led to several principles of instructional design, three of which are proven to be effective for learning and are directly related to the programmed instructional system to be developed in this study. 30

PAGE 40

Redundancy: In situations where either the textual passages or the illustration convey sufficient information to explain a topic fully, the use of both results in excessive cognitive load and reduced learning (Sweller et al., 1998). A requirement of attention to both graphical and textual sources of information uses more of WM capacity than either of the sources alone. The second channel of information is not providing anything new, which reduces WM capacity that could be used for schema construction and assimilation (Cooper, 2000). Split attention: Certain instructional situations require the use of illustrations but frequently force the learner to split his/her attention. The conventional method of including a textual explanation of the information, an illustration, and a caption in the form of a text box containing information relevant to the illustrations causes unnecessary cognitive load (Sweller, 1988). By incorporating the text information into the illustration the learner is no longer forced to shift attention between physically separated locations or to try to hold all relevant information in WM (Mayer, 2001, 2002a) Mode of presentation: The modality effect states that providing related information along two channels simultaneously increases germane cognitive load and facilitates learning (Mayer, Heiser, & Lonn, 2001; Sweller et al., 1998). Figure 10 is an illustration of the Cognitive Theory of Multimedia Learning. In studies where learners were presented complementary (but not redundant) information via an auditory and a visual channel resulted in higher retention and transfer scores (Mayer et al., 2001). This work supports findings by Brunken, Plass, and Leutner (2003) and Verhaeghen, Cerella, & Basak (2004) that show that, under certain conditions, the capacity of WM can be expanded. By processing language related information using an auditory channel via the phonological loop and related visual imagery via the visuospatial scratchpad, the capacity of WM is effectively expanded. 31

PAGE 41

PriorKnowledge Words Pictures Ears Eyes SoundsImagesVerbalModelPictorialModel WORKING MEMORY MULTIMEDIAPRESENTATIONSENSORYMEMORYLONG-TERMMEMORY organizingwords organizingimages selectingwordsselectingimages integrating Figure 10: The cognitive theory of multimedia learning (Mayer et al., 2001). d. How are these characteristics incorporated in the new CLTPI? Redundancy: The redundancy effect is controlled for in CLT-PI by elimination of duplicate sources of information within the TXT material. Where illustrations are used, descriptions have been removed from the body of the text and the illustration is narrated. Split Attention: By eliminating redundant information such as text and graphics describing the same concept, the need for the learner to split attention between to different locations has been removed. Frames are constructed so that all relevant information needed for cognitive processing is contained within that frame. The need to shift back and forth between frames is therefore eliminated. Mode of Presentation: The CLT-PI program employs multimedia (auditory narrative explanation of graphics) in order to reduce extrinsic cognitive load. Graphics included in the CLT-PI program have been designed according to the guidelines outlined in Mayer (2002a). Specifically, all textual information critical to the understanding of a graphic is contained within the graphic element. There will are no text boxes providing explanation or supplementary information regarding a graphic element. Figure 11 is a sample of a CLT-PI frame. This frame consists of an informational paragraph, an interactive multimedia illustration, and a branching assessment item. 32

PAGE 42

Student hears: When an airplane leaves Boston at 9PM, it is already 3AM the next day at its Paris destination. Student hears: As a result of flying into earlier time zones, sunrise occurs only three or four hours later when a passenger has just begun to sleep. Student hears: On landing in Paris at 8AM, local time, it is only 3AM in the passengers brain. He is not only still sleepy, but sleep deprived. Image 2 Image 3 Image 1 Figure 11: A CLT-PI graphics frame. In this frame the learner is presented a series of narrated graphic images. When the learner advances to this frame the narrated sequence begins automatically, first the learner will see Image 1 and hear the associated text. The program will automatically sequence to Image 2, while leaving Image 1 on screen, and present the second narrated text. Finally, Image 3 will appear and the learner will hear the narrated text associated with Image 3. At the end of the sequence the learner has the option of repeating the sequence or advancing in the program. e. What are the characteristics of CLT-PI? As described, the CLT-PI program incorporates contingent-responding, overt responses, and corrective branching from LPI and IPI. In addition, CLT-PI uses the instructional design principles described in Sweller et al. (1998). In particular, illustrations in the CLT-PI program have been constructed in a multimedia format as 33

PAGE 43

described by Mayer (2001). This consists of auditory narration accompanying static graphic illustrations. This new, programmed instructional system (CLT-PI) has the following characteristics: Overt responding. The student must overtly respond to each frame of the program. By requiring an active response by the learner at each step of the learning process, the programmer is assured that the learner is actively attending to the program. Although there is evidence to suggest that the use of constructed responses will benefit learning of technical vocabulary there is a possibility of a confound that could arise from using question formats during training that differ from the format used during final assessment. As this study uses multiple choice questions for all phases of final assessment multiple choice questions have been used within the training programs for all three programmed instructional sets. All assessment items in the CLT-PI program are in the form of branching frames so as to provide remediation if necessary. Response Contingency. Behavioral theories of learning are based on a three-term contingency model. Learning occurs when the learner's measurable behavior changes reliably in the presence of an eliciting stimulus and this behavior is maintained by the occurrence of a contingent reinforcer (Skinner, 1954; 1958). Research has shown that response-contingent learning materials produce superior learning results in terms of retention (Holland, 1965; Kritch & Bostow, 1998; Sime & Boyce, 1969). For this reason the CLT-PI program utilizes teaching frames that are response-contingent. As discussed above, the learner is required to overtly respond to each instructional element. These responses result in feedback as to the correctness of the response. Correct responses confirm the correctness of the response and congratulate the learner. Incorrect responses result in transfer to remedial training frames and an opportunity 34

PAGE 44

to 're-test' on the learning element. A correct response then results in a reinforcement frame. Remedial branching. Every response generated by the learner is routed in one of two ways, to a reinforcement frame if correct, or to a remedial frame if the response is incorrect. Remedial branches vary depending on the level of complexity of the learning element. A simple remedial branch (see Figure 4) simply restates the learning element and redirects the learner back to the original assessment frame. More complicated remedial branches, such as those illustrated in Figure 7, provide greater background information on a topic on the assumption that the learner's erroneous response may have been the result of insufficient prerequisite knowledge. Failure to emit a correct response after remediation results in the learner being returned to the beginning of the learning sub-section and an opportunity to run through the entire learning element again. Cognitive load reduction techniques. The CLT-PI program incorporates several instructional design techniques intended to reduce WM load during instruction. Specifically, this program has been designed to regulate two of the three previously describe types of cognitive load (ICL, ECL, and GCL). ICL, which is inherent in the difficulty of a topic, is not generally subject to manipulation. By utilizing the instructional design techniques created by a number of cognitive load researchers (Mayer, 1990, 1993, 2001, 2002a, 2002b, 2002c, 2003a, 2003b; Mayer & Moreno, 1999, 2002a, 2002b, 2002c; Sweller, 1988, 1994; Sweller, van Merrienboer & Paas, 1998; van Merrienboer et al., 2002) this program focuses on the reduction of ICL with the goal of increasing GCL. Techniques related to the redundancy, split attention, and modality effects are utilized. 35

PAGE 45

EXPERIMENT a. Rationale for this study. In the more than 50 years since the first programmed instructional system was introduced, a large body of evidence accumulated demonstrating its usefulness as an instructional tool. Several factors, including a strong behaviorist orientation in early programmed instruction, the rise of cognitive science, and advancement in information technology, led to its gradual loss of use (Saettler, 1990). The societal issues that led to the need for technologies such as programmed instruction still exist, possibly with greater strength. A technology with proven effectiveness should be incorporated into modern learning systems, not discarded. The proven elements of PI can be coupled with instructional technologies deriving from cognitive research in learning. This study represent an attempt at developing a programmed instructional system utilizing certain elements of traditional PI (LPI and IPI) as a foundation for a cognitively based, programmed instructional system (CLT-PI). b. Hypotheses The hypotheses formulated based on previous studies were: 1. Programmed instruction will result in greater long-term retention and transfer of knowledge than non-programmed learning materials. 2. IPI will result in greater long-term retention and transfer of knowledge than LPI. 3. CLT-PI will result in greater long-term retention and transfer of knowledge than LPI and IPI. 36

PAGE 46

CHAPTER TWO METHODS 1. PARTICIPANTS One hundred sixty eight undergraduate students at the University of South Florida (USF) were recruited for this study. In order to control for variance in knowledge of psychology and past exposure to the subject to be taught (human sleep), a sample was drawn from courses typically taken by freshmen or non-psychology majors. Participants were students enrolled in either Psychological Science I (PSY 2012), Child Psychology (DEP 3103), or Social Science Statistics (STA 2122). Participants were randomly assigned to one of four learning conditions, TXT, LPI, IPI, or CLT-PI. Data from 115 participants was used in this study: 29 in TXT, 27 in LPI, 32 in IPI, and 27 in CLT-PI. Of the original 168 participants, data from 36 participants who failed to complete all phases of the study on time were excluded from analysis. Data from 12 participants who were not native English speakers were also excluded from analysis. Scores from seven were unusable due to some form of data corruption resulting in the recording of null values for all responses. Table 1 shows the demographic information collected from the 115 participants in the four groups. Gender Year in school GROUP Male Female Mean Age Age Range Age SD Freshmen Sophomores Juniors Seniors TXT 5 24 21.24 18 42 5.03 8 9 7 5 LPI 4 23 19.96 18 30 2.75 13 6 4 4 IPI 2 30 18.78 18 25 1.48 21 7 2 1 CLT-PI 2 25 20.07 18 30 3.58 17 2 3 4 Table 1. Participants in this study were awarded Psychology Department Experimental Points, which could be used for extra credit in a number of classes, including the class from 37

PAGE 47

which the participants were drawn. Because this study involved a long-term follow up phase and retaining participants in protracted studies is often difficult, students who completed all phases of the study were entered in a lottery with three students winning a $75 cash prize as further incentive. Participants were informed that this study involved two parts and would take 4 weeks to complete at the time of recruitment. Experimental points were awarded only to those participants who completed all phases of the study. 2. MATERIALS a. Instructional Materials The subject matter used for the study covered Human Sleep and Sleep Staging (Hobson, 1995; Kryger, Roth, & Dement, 2000). A chapter introducing consciousness and human sleep was adapted for this project (Brazas, 2004). This text was modeled on typical chapters on sleep and consciousness found in currently available general psychology texts (Bernstein, Penner, Clark-Stewart, & Roy, 2003; Davis & Palladino, 2000; Morris & Maisto, 2002; Myers, 1998; Nairne, 2003). The text developed for this study expanded on the material covered on sleep stages and the study of sleep and introduced information on the technology involved in measuring sleep. Topics covered in the text included: consciousness, sleep terminology (including technical terminology associated with measurement of human sleep), sleep staging, sleep deprivation, and sleep debt. a. Instructional Units Development: Four separate programs were used in this study. A text (control) condition (TXT), a linear programmed instructional program (LPI), an intrinsic (branching) programmed instruction program (IPI), and an instructional program using certain elements of traditional programmed instruction and integrating design elements based on the cognitive load theory of learning (CLT-PI). Several factors were taken into consideration before construction of the instructional programs. One of these factors concerned the amount of time a learner would spend in the various phases of the study. The chapter on sleep was first converted to an eBook format and read by a number of people of varying ability. This chapter was modified so that the time needed to read it carefully was approximately 50 minutes. This time 38

PAGE 48

interval was selected because it is comparable to the amount of independent study required for a single class session. The resulting text became the first instructional program, a text-based control labeled TXT. Before an instructional program can be written, a list of instructional objectives (terminal behaviors) must be made (Mager, 1963). The TXT eBook chapter was analyzed to identify key instructional points and to develop a list of instructional objectives. In this study, terminal behaviors were classified as discriminatory or transfer (generalization). Most discrimination items required that the subject be able to identify the definition of a term from the text. Transfer items were designed to assess the subjects understanding of concepts and an ability to apply this learned material to unique or novel problems. A copy of this terminal behaviors list is included as Appendix A. With the terminal behaviors list as a guide, a set of assessment items was then developed. Two assessment tests were developed: one for administration immediately following the learning phase (short-term assessment), and one to be administered 4 weeks later (long-term assessment). Each test consisted of 43 multiple-choice question items, in which students were required to select one correct answer of three to five possible choices. Before these tests were used in the study, the assessment items were tested for validity. Validity was tested in a pilot study using 42 undergraduate students recruited from three different summer session classes. These students had not completed advanced psychology coursework. The goal of this pilot study was to determine whether all of the possible answer choices for each question had an equal probability of being chosen by a nave participant. This study was run as two phases. In the first phase 27 students completed a test that included all of the questions in both the short-term and long-term assessments. The order of presentation of short-term/long-term questions was counterbalanced between subjects. Each assessment item was tested using Chi Square analysis to show that each possible answer had an equal probability of being chosen by a nave participant. The results of this Chi Square analysis are reported in Appendix B. Nineteen questions were found to have significant cell differences. The 19 questions with significant cell differences were modified and retested in a second phase, using the 39

PAGE 49

remaining 15 students. The data from these 15 students was tested using Chi Square analysis, which showed that all of the questions now had equivalent cell values, meaning that there was no significant pattern of responses that differed from chance. The results of this second phase Chi Square analysis are contained in Appendix C. This resulted in two 43 items tests with items covering the same material, a short-term assessment (Appendix E) and a long-term assessment (Appendix F). Appendix D is a list identifying the retention and transfer questions for both Short-Term and Long-Term tests. These assessment tests were designed to test for comprehension of the learned material (retention items) and the ability to generalize or apply the newly learned material to a novel problem (transfer). Retention was operationally defined as the ability to recognize the correct answer to a question from a field of two to four distracters. Transfer was operationally defined as the ability apply learned knowledge to the solution to a novel problem. Transfer was assessed by determining the learners ability to identify the correct response in a multiple choice question with a field of two to four distracters. A Each test consisted of 29 retention questions and 14 transfer questions. In the main study, these assessment tests were given to all four groups of participants. On completion of the learning phase all participants were given the short-term assessment test. This short-term test was administered on the same computer that the participant had used to study presented material. Four weeks after training and the short-term assessment test participants were given the long-term retention and transfer test. The long-term retention test was administered using the Universitys Blackboard Learning Management System (LMS). The Blackboard LMS is a Networked Learning Environment and is described as a system where student and teacher can view instructional content, collaborate with educators, evaluate academic performance and access learning resources at any time to achieve their educational objectives (Blackboard Academic Suite Brochure, n.d.). 40

PAGE 50

2. PROGRAM DEVELOPMENT a. Development of the TXT program. The sleep and consciousness text was converted to an eBook format. The content of the text was not altered, but the arrangement of certain elements was changed to accommodate frame size restrictions. Illustrations included standard captions and labels. Descriptions of the illustrations were included within the body of the text. This eBook was constructed using the Macromedia Authorware authoring program. The final eBook contains 43 pages. A five-page extract of this text is included as Appendix G. A CD containing the entire TXT program as a Windows executable file is included as Appendix K. b. Development of Linear Program Materials (LPI). A linear instructional program was developed based on the terminal behavior list. This program was designed to elicit the behaviors necessary to answer the questions in the assessment set. This programmed instructional set is based on the principles outlined by Holland and Skinner (1961). This material was delivered using a computer-based tutorial program constructed using Authorware 6.5 (Macromedia, Inc). Text from the original TXT program was incorporated into this LPI program as much as possible to provide similar content. This LPI program contains redundant and split-attention features from the TXT program, modified only to meet the programming requirements of a linear program. Graphic illustrations were presented as supplementary information on the computer screen, but in keeping with linear programming tradition, were not integrated into the text of the program. This linear program was written using the methods described in the Ruleg System for the Construction of Programmed Verbal Learning Sequences (Evans, Homme, & Glaser, 1962), which had been further refined in the primer Good Frames and Bad (Markle, 1969). An assessment item followed each instructional sequence. These assessment items were in the form of multiple-choice questions. The learner clicked on one of the choices. If a correct response was made, the learner advanced in the program to the next instructional sequence. Reinforcement was provided for every correct response. Reinforcement was in the form of a congratulatory statement such as Good Job! or 41

PAGE 51

Correct! When an error was made the learner was routed back to the instruction related to the question and allowed to repeat the sequence. There were no limits to the number of times a learner could repeat a sequence. No advancement was possible without correctly answering an assessment item. The linear program developed for this study consists of 98 frames. Appendix H contains a 5-frame extract from this program. A CD containing the entire LPI program as a Windows executable file is included as Appendix K. c. Development of Intrinsic (Branching) Program Materials (IPI). The intrinsic program was written using the design guidelines described by Rowntree (1966). The techniques described in this handbook were supplemented by the methods described in the Ruleg System for the Construction of Programmed Verbal Learning Sequences (Evans et al., 1962) and the primer Good Frames and Bad (Markle, 1969). To avoid varying writing styles, this program incorporated blocks of text directly copied from the TXT program. Graphic information was incorporated into the body of the text in a format similar to that of the TXT program. An assessment item or assessment items followed each instructional sequence. These items were in the form of multiple-choice questions and were often in the exact format as used in the LPI program. If a correct response was made, the learner advanced in the program to the next instructional sequence. After every correct response, reinforcement was provided in the form of a congratulatory statement such as Good Job! or Correct! If the learner made an incorrect response to an assessment item, the program routed the learner to a remedial instructional loop where the instructional sequence was presented again using different form or wording. Each assessment item (a multiple-choice question or an illustration with items to click on) consisted of a correct response and a set of incorrect responses (distracters). Remedial branches were designed to address the particular error response made to that assessment item. The learner was shown what error was made and the concept was explained again. The learner was then routed to an assessment item covering this learning concept. In some cases the learner was routed back to the original assessment item, but in most cases was presented a new, reworded, item that addressed the same material. In 42

PAGE 52

situations where the instructional material was more complex, the remedial sequence could consist of several frames. The intrinsic program developed for this study consisted of 105 primary frames with multiple corrective branches. Appendix I contains a 5-frame extract of this program. A CD containing the entire IPI program as a Windows executable file is included as Appendix K. d. Development of Cognitive Load Theory-based Program Materials (CLT-PI). The CLT-PI program was essentially a modification of the IPI program. It contained the same text and assessment items as the IPI program with the exception of deletion of any text describing included graphics. Materials presented in this program were constructed in accordance with the instructional design guidelines derived from cognitive load theory (Sweller, 1988) and further refined in the Cognitive theory of multimedia Learning (Mayer et al., 2001). Three experts in the field of cognitive load theory and the cognitive theory of multimedia learning were consulted before program construction began. Richard Mayer, John Sweller, and J.J.G. van Merrienboer were consulted on the proposed methods and the design of the CLT-PI program. Suggestions were incorporated into the design of this instructional program. All of the graphic illustrations used in the IPI program were included in the CLT-PI program, but all graphic illustrations had text other than titles and labels removed and an auditory narration was mated to these frames. In cases where illustrations originally contained large amounts of textual information, multiple frames were used with indicators showing what portion of the text was being described by the auditory narration. The primary investigators (male) voice was used to narrate all illustration frames. The narrated text was identical to that presented in textual form in the IPI program. Assessment items that required the student to interact with an illustration and all remedial frames that included illustrations were constructed using this format. The CLT-PI program developed for this study consisted of 105 primary frames with multiple corrective branches from each frame that contains an assessment item. A 6-frame extract of the CLT-PI program is shown in Appendix J. A CD containing the entire CLT-PI program as a Windows executable file is included as Appendix K. 43

PAGE 53

PROCEDURE 1. DATA COLLECTION a. Training Session and Short-Term Assessment. Data were collected over the course of six months. There were two phases of data collection, a training phase in which short-term retention and transfer were assessed, and a long-term retention and transfer phase. The training and short-term assessment phase was conducted in laboratory space provided by the Psychology Department at USF. This laboratory consisted of a number of computers set up on tables so that individual learners did not face other people in the room. Four learners could be trained and assessed at one time. Appointments were made using the Psychology Departments online, computerized experiment management system. Appointments were made two hours apart, and on average 12 appointment slots were available on each training day. Appointments were typically available from 8:30am to 6pm five days per week. All procedures were conducted in accord with the university Institutional Review Board. The procedure for a training session began at the scheduled appointment time, and participants waited outside the lab until the scheduled time. All participants completed a contact information sheet, a demographics form, and read and signed an informed consent form (a copy of this informed consent form was given to each participant). All participants generated a unique identification number at this time. This number was used on all experimental data sheets, programs, and in all analyses. The contact information sheet was used to enroll each participant into a dedicated test group on the Universitys Blackboard LMS. Contact information sheets were filed separately from all paperwork containing experimental information. All paperwork related to group membership or assessments was coded using the ID number generated by the participant. Demographic information collected was identified only by ID number and consisted of the participants age, gender, year in school, native language, and course work in psychology. Participants were randomly assigned to one of the four learning conditions at the time of their appointment. Before the learning sessions began all participants were told how they would be contacted four weeks later to complete the second, long-term, phase of the 44

PAGE 54

study. The assignment of experimental points and entry into the lottery for the cash prize was also explained. Participants were then given an opportunity to ask questions and have them answered. Learning sessions were timed and these data recorded for each participant. A learning session began when the experimenter said to begin. When a participant completed the study the program instructed them to notify the experimenter who then logged the completion time. The participant was then logged into the short-term assessment program. The short-term assessment program recorded the scores and amount of time required to complete the test for each participant identified by ID number. All data from these learning sessions were transferred to a spreadsheet for collation and analysis. b. Long-Term Assessment. A dedicated organization was created on USFs Blackboard LMS titled Cognitive Load Theory and Instruction. All participants were registered into this organization at the time of their initial training session. This LMS system allowed the experimenter to control access to the long-term assessment test by use of a password system. Three days before the completion of the fourth week after their learning session each participant was contacted, either by email or by telephone, given his or her password, and instructed when and how to log onto the Blackboard system to take the long-term assessment. Participants who failed to complete this assessment by the requested time were contacted again and asked to take the assessment. If necessary a third request was made. Participants who failed to complete the second phase were excluded from the data analysis and their experimental points were withdrawn. Data collected by the Blackboard LMS were transferred to a spreadsheet for collation and analysis. 2. ANALYSIS Each assessment test consisted of 43 multiple-choice questions. Twenty-nine questions on each test assessed the participants retention of factual information learned during the training session. Fourteen questions assessed the participants ability to generalize, or to transfer, this information to solve a problem or to use the learned material in a unique or novel situation. Therefore a separation of responses into 45

PAGE 55

Retention or Transfer on both shortand long-term tests required a repeated measures multiple analysis of variance (MANOVA) test. This test was conducted using the SPSS General Linear Model program. To determine specific differences between the various elements of this study, a series of analyses of variance (ANOVA) tests was also conducted. 46

PAGE 56

CHAPTER THREE RESULTS 1. SUMMARY The present study revealed the following results for each of the three hypotheses. The first hypothesis states that programmed instruction results in greater long-term retention and transfer of knowledge than non-programmed learning materials. This hypothesis was partially supported. Although LPI did not produce significantly greater retention or transfer scores than TXT, the branching programs, both IPI and CLT-PI, did show beneficial differences in long-term retention scores from TXT. However, there were no significant differences between any programmed instruction conditions and TXT on short-term retention, short-term transfer, or long-term transfer. The second hypothesis states that IPI will result in greater long-term retention and transfer of knowledge than LPI. This hypothesis was also partially supported. There was a significant difference between IPI and LPI for long-term retention. However, there was no significant advantage for IPI over LPI for short-term retention, short-term transfer, or long-term transfer. The third hypothesis states that CLT-PI would result in greater long-term retention and transfer of knowledge than LPI and IPI. This hypothesis is also partially supported in that CLT-PI produced significantly higher long-term retention scores than LPI. However, CLT-PI was not superior to IPI on long-term retention, long-term transfer or shortterm retention. CLT-PI did, however, produce better short-term transfer scores than IPI and LPI. 2. OVERALL EFFECTS a. Effect of time spent studying. First, analyses were conducted to assess whether the amount of time spent in study varied across programs and to determine whether any variation was related to score 47

PAGE 57

differences. Figure 12 shows the mean study times for the four conditions. An ANOVA using study time as a dependent variable found significant differences between groups (F (1,111) = 3.279, p = .024). Post Hoc tests revealed that CLT-PI took significantly longer to study than both LPI and TXT but not longer than IPI. IPI took longer to study than TXT but not longer than LPI. Significant findings are highlighted in Table 2, which shows the results of the Post Hoc analyses. Figure 12: Mean Study Times. This graph shows the means study times for each group with standard deviation bars. The red lines indicate Tukey HSD and LSD significant differences. 47.590850.110455.199756.924401020304050607080Study TimeMean Study Time (Minutes) TXT LPI IPI CLT-PI HSD & LSD LSD LSD 48

PAGE 58

Multiple Comparisons Dependent Variable: Study Time (I) Group (J) Group Mean Difference (I-J) Std. Error Sig. IPI 1.7248 3.2333 0.9507 LPI 6.8141 3.3675 0.1856 CLT-PI TXT 8.9379 3.3089 0.0394 CLT-PI -1.7248 3.2333 0.9507 LPI 5.0893 3.2333 0.3976 IPI TXT 7.2131 3.1722 0.1104 CLT-PI -6.8141 3.3675 0.1856 IPI -5.0893 3.2333 0.3976 LPI TXT 2.1238 3.3089 0.9182 CLT-PI -8.9379 3.3089 0.0394 IPI -7.2131 3.1722 0.1104 Tukey HSD TXT LPI -2.1238 3.3089 0.9182 IPI 1.7248 3.2333 0.5948 LPI 6.8141 3.3675 0.0454 CLT-PI TXT 8.9379 3.3089 0.0080 CLT-PI -1.7248 3.2333 0.5948 LPI 5.0893 3.2333 0.1183 IPI TXT 7.2131 3.1722 0.0249 CLT-PI -6.8141 3.3675 0.0454 IPI -5.0893 3.2333 0.1183 LPI TXT 2.1238 3.3089 0.5223 CLT-PI -8.9379 3.3089 0.0080 IPI -7.2131 3.1722 0.0249 LSD TXT LPI -2.1238 3.3089 0.5223 Based on observed means. The mean difference is significant at the .05 level. Table 2: Results of Study Time Post Hoc Analyses. Tukey HSD and LSD analyses of study time ANOVA results. Despite the differences in time spent studying, no significant correlation between study time and the four dependent variables (short-term retention, long-term retention, 49

PAGE 59

short-term transfer, and long-term transfer) was found. Table 3 shows the results of this analysis with the 2-tailed significances highlighted. Correlations STR STT LTR LTT Study Time Pearson Correlation 0.1115 0.1669 0.1682 0.0138 1 Sig. (2-tailed) 0.2357 0.0745 0.0723 0.8838 Sum of Squares and Cross-products 555.8962 612.6224 1053.6190 45.4944 18498.7662 Covariance 4.8763 5.3739 9.2423 0.3991 162.2699 Study Time N 115 115 115 115 115 Table 3: Study Time and Dependent Variables Correlations. b. Scores for the four tests (Shortand Long-term Retention and Transfer). Figure 13 shows the mean Short-term and Long-term Retention scores for all four groups. The mean scores for short-term retention were higher than long-term retention for all four groups. A series of t-tests comparing short-term retention to long-term retention scores found significant differences for all four conditions (Table 4). Table 4: T-test results comparing shortto long-term retention. t df Sig. (2-tailed) TXT 8.903 28 < 0.001 LPI 7.473 26 < 0.001 IPI 6.047 31 < 0.001 CLT-PI 6.434 26 < 0.001 50

PAGE 60

63.021%63.728%61.831%65.262%38.645%40.486%48.990%52.617%0.000%10.000%20.000%30.000%40.000%50.000%60.000%70.000%80.000%TXTLPIIPICLT-PIPercent Correct STR LTR Figure 13: Mean Retention Scores. This chart shows the mean shortand long-term retention scores for all four groups expressed as the percentage correct. Y-error bars indicate standard error. Figure 14 shows the mean Short-term and Long-term Transfer scores for all four groups. The difference between short-term and long-term transfer was not clearly delineated. A series of t-tests comparing short-term transfer to long-term transfer scores found only one significant difference, that for CLT-PI (Table 5). Table 5: T-test results comparing shortto long-term transfer. t df Sig. (2-tailed) TXT 0.83 28 0.413 LPI 2.043 26 0.051 IPI -0.539 31 0.594 CLT-PI 3.793 26 0.001 51

PAGE 61

21.879%20.690%20.334%26.310%20.097%17.114%20.807%19.159%0.000%5.000%10.000%15.000%20.000%25.000%30.000%35.000%TXTLPIIPICLT-PIPercent Correct STT LTT Figure 14: Mean Transfer Scores. This chart shows the mean shortand long-term transfer scores for all four groups expressed as the percentage correct. Y-error bars indicate standard error. To determine whether there were significant differences between groups for the dependent variables of retention and transfer, a repeated-measures MANOVA was conducted. Table 6 shows the results of this MANOVA. As can be seen in the highlighted cells, there are significant effects for Retention and Transfer as well as significant interactions between the dependent variables and Group. This indicates that there are significant differences between groups on the repeated measures of Retention and Transfer. 52

PAGE 62

Table 6: Repeated Measures MANOVA Output. Multivariate Tests Effect Value F df Error df Sig. p 2 Observed Power RETENT Wilks' Lambda 0.0648 1602.46 1 111 0.000 0.935 1.000 RETENT GROUP Wilks' Lambda 0.925 3 3 111 0.034 0.075 0.694 TRANSFER Wilks' Lambda 0.4269 149.02 1 111 0.000 0.573 1.000 TRANSFER GROUP Wilks' Lambda 0.9169 3.36 3 111 0.022 0.083 0.748 RETENT TRANSFER Wilks' Lambda 0.4521 134.54 1 111 0.000 0.548 1.000 RETENT TRANSFER GROUP Wilks' Lambda 0.833 7.42 3 111 0.000 0.167 0.983 Computed using alpha = .05 Design: Intercept GROUP Within Subjects Design: RETENT + GROUP + RETENT TRANSFER One-way ANOVA tests were conducted to identify significant differences between groups on the dependent variables. According to the ANOVA, there were significant differences between groups for long-term retention (F (3,111) = 6.569, p < .001, p = 0.151), short-term transfer (F (3,111)= 3.098, p = 0.030, p = 0.077). However, no differences were found for long-term transfer (F (3,111) = 1.195, p = 0.315, p = 0.151) or short-term retention (F (3,111)= .207, p = .892, p = 0.006). 53

PAGE 63

In order to determine which means differed for long-term retention and short-term transfer, Post Hoc analyses were then conducted. The results of the analyses were organized for each of the three hypotheses, as follows: 3. ANALYSIS BY HYPOTHESIS a. Analyses on Hypothesis 1 Comparison between programmed instruction and TXT Long-term retention: Post Hoc analyses showed that there were no differences between TXT and LPI. IPI and CLT-PI, however, did produce significantly greater long-term retention scores than did TXT. Table 7 shows the results of the Post Hoc analyses with significant findings highlighted. Multiple Comparisons Dependent Variable: LTR (I) GROUP (J) GROUP Mean Difference (I-J) Std. Error Sig. CLT-PI TXT 4.052 1.077 0.002 IPI TXT 3.012 1.033 0.022 Tukey HSD LPI TXT 0.534 1.077 0.960 CLT-PI TXT 4.052 1.077 0.000 IPI TXT 3.012 1.033 0.004 LSD LPI TXT 0.534 1.077 0.621 Based on observed means. The mean difference is significant at the .05 level. Table 7: Post Hoc Analyses of LTR Data H1 Short-term transfer: Post Hoc tests failed to find a significant advantage for programmed instruction over text. Table 8 shows the results of these Post Hoc tests. The results of two Post Hoc tests, the Tukey HSD and the LSD, are reported. Some of the results of the conservative Tukey HSD test, while not significant, approached 54

PAGE 64

significance. The LSD test is a much more liberal test and show significance for some of the findings to follow. Multiple Comparisons Dependent Variable: STT (I) Group (J) Group Mean Difference (I-J) Std. Error Sig. CLT-PI TXT 1.285 0.658 0.212 IPI TXT -0.532 0.631 0.833 Tukey HSD LPI TXT -0.345 0.658 0.953 CLT-PI TXT 1.285 0.658 0.053 IPI TXT -0.532 0.631 0.400 LSD LPI TXT -0.345 0.658 0.601 Based on observed means. Table 8: Post Hoc Analyses of STT Data H1 b. Analyses on Hypothesis 2 Comparisons between branching and linear programmed instructions. Long-term retention: Post Hoc analyses showed that the difference between IPI and LPI is significant at the .05 level when using the LSD test. The results of the Post Hoc analyses for Hypothesis 2 are shown in Table 9, with significant findings are highlighted. 55

PAGE 65

Multiple Comparisons Dependent Variable: LTR (I) GROUP (J) GROUP Mean Difference (I-J) Std. Error Sig. Tukey HSD IPI LPI 2.478 1.052 0.092 LSD IPI LPI 2.478 1.052 0.020 Based on observed means. The mean difference is significant at the .05 level. Table 9: Post Hoc Analyses of LTR Data H2 Short-term transfer: Post Hoc tests showed that the results for IPI were not different from those of LPI for short-term transfer. These Post Hoc analysis results are shown in Table 10. Table 10: Post Hoc Analyses of STT Data H2 Multiple Comparisons Dependent Variable: STT (I) GROUP (J) GROUP Mean Difference (I-J) Std. Error Sig. Tukey HSD IPI LPI -0.187 0.643 0.991 LSD IPI LPI -0.187 0.643 0.771 Based on observed means. c. Analyses on Hypothesis 3: Comparisons among CLT-PI, IPI, and LPI. Long-term retention: Post Hoc analyses were conducted to test this hypothesis. CLT-PI was found to produce greater long-term retention scores than LPI, but there was 56

PAGE 66

no evidence of an advantage over IPI. Table 11 shows the results of the Post Hoc analyses for long-term retention with the significant cells highlighted. Table 11: Post Hoc Analyses of LTR Data H3 Multiple Comparisons Dependent Variable: LTR (I) GROUP (J) GROUP Mean Difference (I-J) Std. Error Sig. IPI 1.041 1.052 0.756 Tukey HSD CLT-PI LPI 3.519 1.096 0.009 IPI 1.041 1.052 0.325 LSD CLT-PI LPI 3.519 1.096 0.002 Based on observed means. The mean difference is significant at the .05 level. Short-term transfer: Post Hoc analyses showed that CLT-PI produced greater short-term transfer scores than IPI and LPI using the liberal LSD test. Table 12 shows the results of the Post Hoc analyses on short-term transfer with the significant cells highlighted. 57

PAGE 67

Table 12: Post Hoc Analyses of STT Data H3 Multiple Comparisons Dependent Variable: STT (I) Group (J) Group Mean Difference (I-J) Std. Error Sig. IPI 1.817 0.643 0.028 Tukey HSD CLT-PI LPI 1.630 0.670 0.077 IPI 1.817 0.643 0.006 LSD CLT-PI LPI 1.630 0.670 0.017 Based on observed means. The mean difference is significant at the .05 level. 58

PAGE 68

CHAPTER FOUR DISCUSSION The present study proposed three hypotheses regarding the effectiveness of different types of learning systems. The three hypotheses tested in this study were: Hypothesis 1: Programmed instruction results in greater long-term retention and transfer of knowledge than non-programmed learning materials. Hypothesis 2: IPI will result in greater long-term retention and transfer of knowledge than LPI. Hypothesis 3: CLT-PI will result in greater long-term retention and transfer of knowledge than LPI and IPI. In some aspects, the present data were consistent with the contention that CLT-PI is a superior programmed instructional system compared to both text-based instruction and traditional programmed instruction. However, there were also data contradictory to all three hypotheses. Below, these results and their significance for each hypothesis are discussed. 1. HYPOTHESIS 1: Comparison between programmed and non-programmed instruction. For long-term retention, intrinsically based programmed instruction (IPI and CLT-PI) showed significantly better results compared to TXT. These results are consistent with previous data (Coulson et al., 1962; French, Crowder, & Turner, 1956; Hartley, 1965; Kopstein & Cave, 1962; Kopstein et al., 1962; Senter et al., 1966; Ter Keurst, 1965). The results of comparisons between LPI and TXT, however, do not support the hypothesis that there were significant differences between these two conditions for long59

PAGE 69

term retention. These data are contrary to the large body of research into linear programming (Buckland, 1967; Goldbeck & Campbell, 1962; Sime & Boyce, 1969). One possible explanation for why LPI and TXT were not significantly different for long-term retention may be in the way the TXT program was delivered in the present study. Previous studies for the assessment of text-based learning used hardcopies of real textbooks (Daniel & Murdoch, 1968; Pikas, 1969). With real textbooks there was no way to insure that an individual student was going to read a chapter exactly the way the author had intended. The student might skip around looking at the illustrations, reading section headings, or jumping to the end to review the study questions. In contrast, the TXT program used in this study was presented on a computer and the material presented was divided into screen-sized units of information. This division of the text into smaller units of information created a visual learning environment similar to that provided by the programmed instructional programs. Although the learners were able to page forward and backward through the TXT program, they were required to page through in the order designed by the experimenter. The design of the TXT program reduced the probability of this non-sequential type of reading behavior due to the necessity of waiting for each page to load, locating the next page prompt, and repeating this process to advance or move backward in the program. In addition, the learner knew that a quiz would immediately follow the chapter. Knowing that there would be a quiz at the end of the chapter might motivate the learner to read the material on each screen more carefully rather than skipping to the next page. It is possible that the TXT format in the present study resulted in higher scores for the TXT group than would have been expected based on past research. The design may have allowed for better concentration and the expected quiz may have increased motivation. For long-term transfer, all four groups performed equally well with no group showing significantly different scores. This may be due to a type of ceiling effect. Because this study only provided a single opportunity to study the material, the scores seen for long-term transfer may represent the maximum amount of information that could be learned in a single, massed practice, session. Research has shown that distributed practice is superior in terms of task performance (Donovan & Radosevich, 1999) compared to 60

PAGE 70

massed practice. Thus, distributed practice sessions may be needed for an instructional program to produce transfer scores higher than those seen in this study and thereby reveal potential differences between conditions. 2. HYPOTHESIS 2: Comparisons between IPI and LPI. The results showed that the IPI condition resulted in higher scores than did LPI for long-term retention. This is consistent with the prediction based on the second hypothesis. The results are significant because they support Crowders assertions that intrinsic programming is superior to linear programming (Crowder, 1960, 1961, 1964). As described earlier in this paper, Crowder hypothesized that intrinsic programming, with its system of remedial branching, was superior to linear programs (like LPI) because the learner was provided corrective education at the time of the error. A properly designed intrinsic program consists of blocks of educational content followed by assessment items designed to determine the level of understanding acquired by the learner. These assessment items are typically multiple-choice questions where the distracter answers are based on common misunderstandings about the content. When a learner selects one of these incorrect answers, it indicates that he has misunderstood the content and is directed to a set of remedial frames designed to correct this specific error. Linear programming, on the other hand, merely indicates that the learner has made an error. When a learner makes an error on a linear program one of two consequences are typically presented; the learner may redirected back to the assessment frame and allowed to try again, or; the learner may be informed that an incorrect response has been made and allowed to proceed with the program. If the criterion level is not achieved on the program, the learner must repeat the entire program. The LPI program utilized the first alternative; an incorrect response led caused the program to redirect back to the question as many times as it took for the learner to respond correctly. This difference in program response to incorrect answers is a fundamental difference between linear and intrinsic programming. This difference, which gives learners immediate remediation of errors may lead to the higher long-term retention scores seen for IPI. 61

PAGE 71

Furthermore, both the IPI and CLT-PI programs produced higher long-term retention scores than did LPI. Because CLT-PI is a modification of the IPI branching program, the results are consistent with the notion that branching programs are superior, in terms of long-term retention, to linear programmed instruction. In terms of long-term transfer, the results did not show significant differences between IPI and LPI. Given the results for long-term retention, this was suprising. It indicates that there may be a fundamental difference between the retention and transfer processes. Retention questions tested the learners ability to recognize the correct answer from a group of distractors. These questions were designed to assess the ability of the learner to remember factual information (declarative information), generally information that was prestented in the form of text. The transfer questions, on the other hand, generally involved extracting information from illustrations and relating this information to the declarative information presented in the text. A deeper level of processing may have been involved in the study of this information, leading to a more consistent ability to answer transfer questions across groups. 3. HYPOTHESIS 3: Comparison between CLT-PI and IPI/LPI. The results showed superiority of CLT-PI over LPI for long-term retention. These results are consistent with the large body of research on multimedia instructional programs (Mayer, 2001, 2002a, 2002b, 2002c, 2003a, 2003b; Mayer & Moreno, 1998, 2002b, 2003), which showed that multimedia programs, such as CLT-PI, resulted in better retention compared to instructional programs that utilize a single instructional channel (text and printed graphics), such as LPI. In contrast to LPI, CLT-PI was not superior to IPI for long-term retention. This is inconsistent with the research on multimedia instruction (Mayer, 2001, 2002a, 2002b, 2002c, 2003a, 2003b; Mayer & Moreno, 1998, 2003), which showed a clear superiority for multimedia instruction over other instructional programs, such as IPI and LPI. The fact that there were no differences between the scores of CLT-PI and IPI suggests that the branching technology which CLT-PI and IPI shared was important for long-term 62

PAGE 72

retention. The multimedia instructional system, which characterized CLT-PI, was not a significant factor for long-term retention in this study. Hypothesis 3 also predicted that CLT-PI would produce higher long-term transfer scores than the other two programmed instructional conditions. What was found, however, is that there were no significant differences between CLT-PI and IPI/LPI for long-term transfer. The results are in contrast to previous studies of multimedia instruction (Mayer, 2001, 2002a, 2002b, 2002c, 2003a, 2003b; Mayer & Moreno, 1998, 2003). The finding of no differences between conditions for long-term transfer is possibly due to a spoken language effect, which was recently described by Tabbers, Martens, and van Merrienboer (2004). They found that a multimedia instructional system produced lower transfer scores than would be expected based on previous work. Tabbers et al. (2004) tested four instructional paradigms for retention and transfer, including two conditions that are functionally similar to the CLT-PI and IPI conditions in the present study. A condition that contained text and cue enhanced diagrams (VC condition) is comparable to the IPI condition in the current study. Another condition, labeled the AN condition, used multimedia presentation of graphic information in a manner similar to that used in the CLT-PI condition. The VC (IPI) condition resulted in higher transfer scores than the AN (CLT-PI) condition. The authors speculated that the use of narrated instruction in the AN (CLT-PI) condition was the cause of the lower transfer scores. They suggested that because narration was used in the AN condition, time spent studying became longer and the pacing of study became more controlled than in the VC condition. Study time and pacing were substantially different in the Tabbers et al. (2004) study and in the present study from that used in previous multimedia studies (Mayer, 2001, 2002a, 2002b, 2003; Mayer & Moreno, 1998; Tabbers et al., 2004). The longer amount of time allowed for study in the Tabbers et al. (2004) study matches that of the current project (i.e., just over 50 minutes average) and may have led to a reduction in motivation on the part of the learners. It may be that listening to a narrated text for long periods is less interesting than reading the same material. 63

PAGE 73

A possible interaction between study time and the way in which pacing was controlled in these two studies may also contribute to reduced motivation. Past research on multimedia instruction (Mayer & Moreno, 1998, 1999) was designed with system-paced textual presentation. In the CLT-PI condition the learner controlled how long to spend in study on any section except the narrated sections. Textually presented information could be studied as long as needed and the narrated sections could be repeated as often as needed for understanding. The difference was that a learner could focus attention on a poorly understood section of text but was required to listen to an entire narrated passage when repeated. Learners may have avoided repeating narrated passages as often as needed for full understanding because of the amount of time it took and the requirement of re-listening to some material already understood. because transfer material was concentrated in the narrated sections of the program this could account for the lower than expected transfer scores seen in CLT-PI. 4. SHORT-TERM RETENTION AND TRANSFER DATA. Although short-term predictions for retention and transfer were not included in the hypotheses, these data were collected. Having short-term retention and transfer data allows comparisons to be made to previous research and allows for analysis of change across time. The data for short-term retention and short-term transfer were inconsistent with each other, with previous research, and with the long-term data. There was no significant difference between any of the four groups for short-term retention and yet long-term retention shows an advantage for intrinsic programming. The short-term transfer data showed an advantage for CLT-PI over the other programmed instructional systems but not for TXT. In contrast, the long-term transfer data showed no significant difference between groups. It would appear that there was something common to CLT-PI and TXT in terms of short-term transfer, a commonality that did not show in the short-term retention data. A possible common feature might be related to the presentation of illustrations. Thus, it might be that the TXT and CLT-PI conditions allowed the learner to better focus attention on an illustration and its 64

PAGE 74

explanation than did the LPI and IPI conditions. The illustrations in the TXT condition were always presented imbedded in a block of text that referenced the illustration. The CLT-PI program generally presented illustrations as separate frames with all of the explanation presented by auditory narration. In contrast, the other two conditions, LPI and IPI, were less consistent in the presentation of illustrations. In the LPI condition frame size was kept small and material presented was limited to a single concept within a frame. In the IPI condition, the program was designed to present larger conceptual blocks of information that may have contained several illustrations. Another possible explanation for the high short-term retention scores of TXT is that the data from the TXT group were not truly representative of the population at large. The amount of variance in the TXT data for short-term transfer was 67.9% higher than the highest variance for a programmed instructional condition. This high amount of variance in TXT scores for short-term retention could indicate that these data were less reliable than that for the other conditions. Although this study had sufficient power, it is possible that replication of this study larger with groups would eliminate this possibility. 5. STUDY TIME The amount of time spent in study differed significantly between groups. And, it seems logical to conclude that increasing the amount of time in study would lead to higher scores. Although there was no significant correlation between scores on the dependent measures and study time, this factor deserves consideration. It is only to be expected that we would see significant differences in how long it took to complete the different programs. The time spent in study was related to the amount of interaction between the learner. The use of multimedia also evokes a time cost. The shortest mean study time was for the TXT group who only had to read the material. There were no questions asked, no remedial branches to view, and no narrated illustrations to study. The LPI program required considerable interaction between the learner and the program, but these interactions were generally very short. Each interaction did not increase study time, however the cumulative effect of the large number of interactions did cause LPI to a higher mean study time than TXT. 65

PAGE 75

The study time for IPI was significantly higher than for TXT. This probably due to the time required to study remedial branches. The longest study time of the four groups was for CLT-PI. This program, like IPI, was an intrinsic program and contained remedial branches. In addition, this program contained auditory narration associated with illustrations. The time required to view a narrated section of the program was fixed and was controlled by the program. The rate of presentation depended on the amount of narration contained within the segment. The differences in study time between conditions, is therefore, a factor of the design of the individual programs. There may not be a way to completely control for differences in study time when comparing programs as different in structure as seen in this study. This possible confound must be taken into consideration when designing a comparative study such as this. 6. LIMITATIONS OF THE STUDY. The limitations to this study fall into two general categories. First, there were technical problems related to the use of the Blackboard LMS system. Second, there were several problems regarding the participants involved in the study. There were two issues related to the use of USFs Blackboard LMS for administering the long-term assessment used in this study. The first issue was a problem with data loss. One advantage of this LMS was that it could be accessed via the Internet. However this feature resulted in problems with data acquisition. At least seven participants experienced severe problems accessing the LMS system from their homes using dial-up modems. Data transfer rates by modem were too slow, causing the LMS system to freeze mid-test. The participants were forced to contact the experimenter to have the system reset. This allowed these participants to view part of the test more than one time. It cannot be determined how many participants gave up trying to get the system to work from home and dropped from the study. The second issue in using the LMS was the lack of control over the testing environment. There was no way to ensure that individual participants followed all of the test requirements (e.g., working alone). 66

PAGE 76

Several issues related to the participant pool surfaced in this study. The data from 31.5% of the subjects enrolled in this study could not be used in the final analyses for a number of reasons. First, there were seven cases of data corruption. A review of the data did not point to a specific reason for these rare cases. While it is possible that there was equipment malfunction, it is also possible that participant error could be to blame. Second was a problem with participant dropout. The most prevalent cause for dropping a participant from the study was failure to complete the second (long-term) assessment. Participants were notified by e-mail and attempts at phone contact were made in each case. It is possible that these students had dropped the course for which extra credit was earned and were no longer motivated to participate or some may have dropped out of school completely. Third, the question of participant motivation arises in any study that relies on volunteer participants. The participants in this study were rewarded with extra credit and entry into a lottery for a cash prize if they completed the full study. The reward was earned merely by participating, not based on performance, so there were no external motivating factors that drove performance. The question of what results would be seen in a situation where learners were performing for more salient rewards must be asked. Would the use of these programs in a class where grades and course credit depended on performance result in different results? It is not clear at this point how a reward system more motivating than that used in this study could be implemented in a public university setting, but methods intended to improve the level of motivation should be investigated. Finally, with any study utilizing university students as a participant population one must ask the question How well do the results of this study generalize to the population as a whole? This study was designed to assess several instructional systems in the teaching of academic material. As such, the intended population should be quite similar to the students who participated in this study. Issues related to demographics aside, university students hopefully are a fairly homogeneous group. As further investigation into cognitive load based instructional systems progresses and its application to more 67

PAGE 77

diverse student populations becomes an issue testing with a more heterogeneous population should be performed. 7. IMPLICATIONS The recent work by Tabbers et al. (2004) brings into question a significant amount of the research regarding the use of cognitive load theory in guiding instructional design. A large body of work has shown significant advantages in both retention and transfer for multimedia instructional systems (Mayer, 2001, 2002a, 2002b, 2003; Mayer & Moreno, 1998, 1999, 2002a, 2002b, 2002c, 2003). On one the hand, the findings of the present study showing a long-term retention advantage for CLT-PI support previous multimedia research. The CLT-PI results for long-term transfer, on the other hand, lend support to the findings of Tabbers et al. (2004), which showed that auditory presentation of instruction within an instructional program could actually reduce learning. There is a possibility that multimedia instruction using an instructional system similar to that used by Mayer and others is only effective in short instructional sequences. The present study produced conflicting long-term results for CLT-PI, with its significant advantage in retention but lack of significant difference between conditions for transfer. This is important because this study, unlike most of the other work on cognitive load theory, measures long-term effects. There may be a difference in how transfer material is processed long-term. One possible explanation may come from research on memory consolidation. Memory consolidation is defined as the time dependent process that converts labile memory traces into more permanent and/or enhanced forms. (Peigneux, Laureys, Delbeuck, & Maquet, 2001). There is a growing body of research that shows that recently learned material changes after sleep. The exact nature of this change is still debated, but there is general agreement that a combination of slow-wave sleep (SWS), or deep sleep, and rapid eye movement sleep (REM) occurring in a normal circadian sequence is necessary for memory consolidation of both recently learned retention and transfer information (Peigneux et al, 2001). REM sleep has been shown to be instrumental in the consolidation of spatial information as well as episodic memory consolidation (Rauchs et al., 2004). SWS has been shown to be involved in the 68

PAGE 78

consolidation of declarative memory and REM in the consolidation of procedural memory (Born & Wagner, 2004). Recently learned complex, generalized language appears to require both SWS and REM occurring in the proper order (extended SWS in the early night followed by progressively longer REM episodes in the later night) (Fenn, Musbaum, & Margoliash, 2003). Future research may need to control for the type of sleep learners get during a study. The advantage seen for long-term retention coupled with the large body of positive short-term retention and transfer findings by researchers such as Mayer (2001, 2002a, 2002b, 2003) and Mayer and Moreno (1998, 1999, 2002a, 2002b, 2002c, 2003) indicate that multimedia instructional technology has a potential for use in applied settings however it remains to be seen if this methodology will translate into longer instructional programs. This study has implications for those designing instruction in both the academic and commercial settings. Because this study used training sessions that were more in line with those seen in a college course (approximately 60 minutes) the results showing strong scores for the IPI program suggests that instructors seeking self-instructional systems should investigate intrinsic programmed instruction. The conflicting results for the multimedia CLT-PI program indicate that multimedia applications should be limited to short instructional sequences, at least until a better understanding of dual channel processing in longer sessions is available. In industry, where on demand instruction is frequently needed, both intrinsic and multimedia programmed instructional systems may be indicated. For very short self-study instruction particularly that involving graphic intensive materials, multimedia programmed instruction may be indicated. 8. FUTURE RESEARCH The results of this study generate a number of questions, covering several broad categories which future research should address. These broad areas include transfer vs. retention; short vs. long-term learning; and the role of auditory narration in instruction. 69

PAGE 79

Transfer vs. Retention: The data lead to the questions regarding why the CLT-PI program produced superior long-term retention scores but failed to outperform the other instructional systems. Are there differences in the way we learn material for retention as opposed to for transfer? What factors in an instructional program, and specifically a multimedia program, influence transfer learning? What factors influence retention learning? And, how do we use these factors to maximize the instructional power of a program? As previously discussed, the role of context between text and illustrations may play a role in transfer. Issues related to memory consolidation also need investigation in terms of retention vs. transfer learning. Before any future study on the differences between retention and transfer are done the issue of defining transfer should be resolved. Transfer is generally discussed in terms of practical application, typically in the workplace. A recent paper defined transfer as the extent of retention and application of the knowledge, skills and attitudes from the training environment to the workplace environment. In other words, transfer of training is the degree to which trainees effectively apply the learning from a training context to the job. (Subedi, 2004). Since transfer refers to the application of knowledge it may be that we need to rethink how we design experiments for transfer. All material learned in a classroom setting, or from self-study goes to form a knowledge base that the learner can then apply. Therefore, retention material is potentially also transfer material. It all depends on whether it is used to solve a problem or complete a task. This leads to a conclusion that transfer is a long-term process and attempts to access short-term transfer are unnecessary. This also indicates that long-term retention should be the focus of study as this is the knowledge the learner will transfer. Future research should focus on programmed instructional designs that lead to long-term retention and investigate ways to improve instructional design so that it enhances the students ability to transfer learning. Short vs. Long-Term: Differences in short-term vs. long-term learning are seen in this study. Why didnt CLT-PI produce short-term retention results that were a strong as 70

PAGE 80

the long-term retention results? Are different processes involved in short-term as opposed to long-term learning? If there are differences in short-term and long-term processes in learning, as is suggested by the results of this study, then further work should be done to explain these differences. A study that assesses retention and transfer items at varying times after study could show at what point learning changes for differing instructional conditions. Auditory Narration: Tabbers et al. (2004) suggested the idea that auditory narration loses effectiveness in longer multimedia programs. This issue needs investigation. If there are limits to how long a student can be expected to attend to narrated instruction, then this knowledge needs to be fully understood as it greatly impacts future programs. This study shows that intrinsic (branching) programmed instruction can be an effective method of self-study. Both IPI and CLT-PI produced superior long-term results, however CLT-PI did outperform the traditional IPI for long-term retention. As this multimedia instructional program incorporates several cognitive load theory design elements including auditory narration. It is possible to retain some cognitive load features, such as control for split attention and redundancy, without using auditory narration. A study comparing a CLT-PI type program with, and without, auditory narration could help to identify the most beneficial cognitive load theory design elements. 9. CONCLUSIONS Forty-four years ago Lawrence Stolurow opened his monograph Teaching by Machine (1961) with a description of a major problem with the United States educational system. He described how the shortage of teachers and classrooms caused by a population boom necessitates the development and implementation of an automated technology of instruction. His concerns are still relevant today. Twenty-two years later the Reagan administration released A Nation at Risk detailing a declining state of affairs in US education. The need for a reliable automated system of self-study is now needed more than ever. In 2001 the No Child Left Behind act was passed outlining another set of plans to improve our educational system. 71

PAGE 81

The problems in education continue, perhaps compounded, yet we have not really changed the way we educate. Stolurow (1961, pgs. 2-3) suggested that we need to change the way we approach this problem, with less emphasis on teachers and more emphasis on learning, specifically on systems of learning. For almost 80 years researchers have been showing the world an instructional process, based on the science of learning, that addresses the problems in our educational system. Programmed instruction solves the problems of overcrowded classrooms and the resulting lack of sufficient teacher-student contact by providing an efficient means of self-study. One of the issues that may have led to the reduction in use of programmed instruction over the years is that it is time consuming to produce and are therefore costly. Computer-based programming requires several specialized skills in addition to knowledge of the topic being taught. These skills include some degree of software or internet programming, the ability to produce and work with digital illustrations and sound files, and the ability to generate the programmed instructional sequences. It is rare that a single individual possesses all of these skills, thus necessitating an instructional team. This, on the face, would appear to make programmed instruction more costly than hiring a certified teacher. However, a single teacher can only instruct the students present in the classroom, and then the teachers attention must be spread among an entire class. No one student gets undivided attention from the source of knowledge, the teacher. While developing an effective instructional program may incur more up front costs, the long-term benefits outweigh them. Once developed, an instructional program can be duplicated an unlimited number of times, and at low cost. An educational setting that relies on technology to provide the bulk of instruction would allow a teacher to supervise a larger number of students, thereby saving money in the long run. The results of this study show that a programmed instructional system can be as effective, if not more effective, as traditional texts. This coupled with the economic benefits of automated instruction show that it is time to integrate this decades old technology of programmed instruction with the current knowledge of human learning derived from cognitive science. This project demonstrated an instructional system that does this, integrates intrinsic programmed instruction and a current cognitive theory of 72

PAGE 82

memory and learning. The resulting CLT-PI program produces long-term learning that is superior to both traditional text based instruction and the purely behavioral programmed instructional systems of LPI and IPI and deserves continued investigation. 73

PAGE 83

REFERENCES Anderson, R.C., Kulhavy, R.W., and Andre, T. (1971). Feedback procedures in programmed instruction. Journal of Educational Psychology, 62(1), p. 148-156. Angell, G.W. & Troyer, M.E. (1948). A new self-scoring test device for improving instruction. School & Society, 67, p. 84-85. Baddeley, A.D. and Hitch, G. (1974). Working memory. In Bower, G.A. (Ed.) The Psychology of Learning and Motivation. pp. 48-79, Academic Press Baddeley, A. (2000). The episodic buffer: a new component of working memory? Trends in Cognitive Sciences, 4(11), p. 417-423. Balson, M. (1969). The effectiveness of programmed learning in promoting transfer in a training situation. Programmed Learning, 6 (3), pp. 159-163. Bartlett, F.C. (1932). Remembering: An experimental and social study. Cambridge: Cambridge University Press. Berglund, G. W. (1969). The effect of partial reinforcement in programmed instruction. Programmed Learning, 6 (2), pp 102-108. Berman, M.L. (1970). Effects of response cost and reinforcement magnitude on performance on programmed materials. Psychological Reports, 27(1), p. 17-18. Bernstein, D.A., Penner, L.A., Clarke-Stewart, A., and Roy, E.J. (2003). Psychology, 6th Edition. Boston, MA: Houghton Mifflin Company. Biran, L.A. (1967). A comparison of a scrambled and a sequential presentation of a branching program. Programmed Learning, 4(4), p. 290-295. Blackboard Academic Suite Brochure (n.d.). Retrieved March 7, 2005, from http://www.blackboard.com/docs/AS/Bb_Academic_Suite_Brochure_single.pdf Born, J. and Wagner, U. (2004). Awareness in memory: being explicit about the role of sleep. Trends in Cognitive Sciences, 8(6), p. 242-244. Brazas, M.L. (2004). Human Sleep Stages. Unpublished manuscript. 74

PAGE 84

Brothen, T., Wambach, C. & Hansen, G. (2002). Accommodating students with disabilities: PSI as an example of universal instructional design. Teaching of Psychology, 29(3), p. 239-240. Brunken, R., Plass, J.L., & Leutner, D. (2003). Direct measurement of cognitive load in multimedia learning. Educational Psychologist, 38(1), p. 53-61. Buckland, P.R. (1967). The response in a linear program: Its mode and importance. Programmed Learning, 4(1), p. 47-51. Buckland, P.R. (1968). The ordering of frames in a linear program. Programmed Learning, 5 (3), pp. 197-205. Buckle, C.F. (1967). Relations between attainment of component skills, retention, and transfer. Programmed Learning, 4(3), p. 179-184. Buzhardt, J. & Semb, G.B. (2002). Item-by-item versus end-of-test feedback in a computer-based PSI course. Journal of Behavioral Education, 11(2), p. 89-104. Cauzinille, E. & Weil-Barais, A. (1974-1975). Evaluation of different modalities of multiple choice utilized during learning in school: Programmed instruction of a grammatical concept in the 6 th and 5 th forms. Bulletin de Psychologie, 28(1-6), p. 236245. Cooper, G. (2000). Research into Cognitive Load Theory and Instructional Design at UNSW. http://education.arts.unsw.edu.au/CLT_NET_Aug_97.HTML Coulson, J.E., Estavan, D.P., Melaragno, R.J., and Silberman, H.F. (1962). Effects of branching in a computer controlled autoinstructional device. Journal of Applied Psychology, 46(6), p. 389-392. Cram, D. (1961). Explaining Teaching Machines and Programming. San Francisco, CA: Fearon Publishers. Crosbie, J. and Kelly, G. (1994). Effects of imposed postfeedback delays in programmed instruction. Journal of Applied Behavior Analysis, 27(3), p. 483-491. Crowder, N.A. (1957). A part-task trainer for troubleshooting. USAF Personnel & Training Research Center Development Report 1957, No. 57-71. Crowder, N.A. (1960). Automatic Tutoring by Intrinsic Programming. In Teaching Machines and Programmed Learning, Lumsdaine and Glaser (Eds.). Washington, DC: Department of Audio-Visual Instruction, National Education Association. 75

PAGE 85

Crowder, N.A. (1961). Intrinsic and Extrinsic Programming. In Programmed Learning and Computer-Based Instruction. John E. Coulson, (Ed.). New York, NY: John Wiley and Sons, Inc. Crowder, N. (1964). Programmed Instruction Compared with Automated Instruction. In Ofiesh, G.D. and Meierhenry, W.C. (Eds.), Trends in Programmed Instruction, Washington, DC: Department of Audio-Visual Instruction, National Education Association. Daniel, W.J. & Murdoch, P. (1968). Effectiveness of learning from a programmed text compared with a conventional text covering the same material. Journal of Educational Psychology, 59(6), p. 425-431. Davis, R.H., Marzocco, F.N., and Denny, M. R. (1970). Interaction of individual differences with modes of presenting programmed instruction. Journal of Educational Psychology, 61 (3), p. 198-204. Davis, S.F. and Palladino, J.J. (2000). Psychology, 3rd Edition. Upper Saddle River, NJ: Prentice Hall. Derry, S.J. (1996). Cognitive schema theory in the constructivist debate. Educational Psychologist, 31(3-4), p.163-174. Deterline, W.A. (1962). An Introduction to Programmed Instruction. Englewood Cliffs, NJ: Prentice-Hall, Inc. Donovan, J.J. & Radosevich, D.J. (1999). A Meta-Analytic Review of the Distribution of Practice Effect: Now You See It, Now You Dont. Journal of Applied Psychology, 84(5), p. 795-805. Druckman, D. and Bjork, R.A. (eds.), In the Minds Eye: Enhancing Human Performance, Washington, DC: National Academy Press, 1991. Entwistle, N. (1994). Styles in learning and teaching: An integrated outline of educational psychology for students, teachers and lecturers. London: David Fulton Publishers. Eppler, M.A. & Ironsmith, M. (2004). PSI and distance learning in a developmental psychology course. Teaching of Psychology, 31(2), p. 131-134. Espich, J.E. and Williams, B. (1967). Developing programmed instructional materials: A handbook for program writers. Palo Alto, CA: Fearon Publishers. 76

PAGE 86

Evans, J.L., Homme, L.E., & Glaser, R. (1962). The RULEG system for the construction of programmed verbal learning sequences. Journal of Educational Research, 55(9), p. 13-518. Fenn, K.M., Nusbaum, H.C., and Margoliash, D. (2003). Consolidation during sleep of perceptual learning of spoken language. Nature, 425(6958), p. 614-616. Ferster, C.B. & Sapon, S.M. (1958). An application of recent developments in psychology to the teaching of German. Harvard Educational Review, 28, p.58-69. Fleischman, Howard (1968). Consumer acceptance of programmed instruction. Washington University Department of Psychology Technical Report 1968, No. 15, p.60-65. [Abstract obtained from PsychINFO Database Record (c) 2000 APA.] French, R.S., Crowder, N.A., & Turner, J.S. Jr. (1956). The K-System MAC-1 TroubleShooting Trainer: II. Effectiveness in an experimental training course. USAF Personnel & Training Research Center Report 1956, No. 56-120. Globerson, S. & Korman, A. (2001). The use of just-in-time training in a project environment. International Journal of Project Management, 19(5), p. 279. Goldbeck, R.A. & Campbell, V.N. (1962). The effects of response mode and response difficulty on programmed instruction. Journal of Educational Psychology, 53(3), p. 110-118. Hartley, J. (1965). Linear and skip-branching programmes: A comparison study. British Journal of Educational Psychology, 35(3), p. 320-328. Hartley, J. (1968). Some factors affecting student performance in programmed learning. Programmed Learning, 5 (3), pp 206-218. Healy, A.F., Wohldmann, E.L., & Bourne, L.E. (2005). The Procedural Reinstatement Principle: Studies on Training, Retention, and Transfer. Found in Healy, Alice F. (Ed.) (2005). Experimental Cognitive Psychology and Its Applications. Washington, DC: American Psychological Association. Hively, W. (1962). Programming stimuli in matching to sample. Journal of the Experimental Analysis of Behavior, 5(3), p. 279-298. Holland, J.G. (1959). Teaching psychology by a teaching machine program. Paper read at the American Psychological Association, Cincinnati, 1959. Found in Derterline, William A. (1962). An introduction to programmed instruction. Englewood Cliffs, NJ: Prentice-Hall, Inc. 77

PAGE 87

Holland, J.G. (1965). Response contingencies in teaching-machine programs. Journal of Programmed Instruction, 3(1), p. 1-8. Holland, J.G. & Skinner, B.F. (1961). The analysis of behavior: A program for selfinstruction. New York, NY: McGraw-Hill. Hobson, J. (1995). Sleep. New York, NY: W.H. Freeman and Company. Hughes, J.L. (1962). Programmed Instruction for Schools and Industry. Chicago, Il: Science Research Associates, Inc. Isaacs, L. (2003) Online Lessons. The American City and County, 18(3), p.40. Jacobs, P.I. and Kulkarni, S. (1966). A test of some assumptions underlying programmed instruction. Psychological Reports, 18(1), p. 103-110. Jamieson, G.H., James, P.E., & Leytham, G.W.H. (1969). Comparisons between teaching methods at the post-graduate level. Programmed Learning, 6(6), p.243-249. Johoda, G. & Johoda, P. (1986). A programmed text in recorded form for teaching voiceindexing to visually disabled students. Journal of Visual Impairment & Blindness, 80(6), p.799-800. Jones, R.M. & Hick, T.L. (1969). Two sequence factors in programmed instruction. Journal of Experimental Education. 38(2), p.66-69. Keisler, E.R. (1959). The development of understanding in arithmetic by a teaching machine. In Lumsdaine and Glaser (Eds.) Teaching Machines and Programmed Learning. National Education Association of the United States. Keller, F.S. (1968). Good-bye, teacher Journal of Applied Behavior Analysis, 1(1), p.79-89. Klaus, D.J. (1964). The investigation of step size and error rate in programmed instruction. USN Training Devices Center Technical Reports 1964, No. 1208-1 Kopstein, F.F., Cave, R.T., and Zachert, V. (1962). Preliminary evaluation of a prototype automated technical training course. USAF Medical Research Laboratory Technical Documentary Report 1962, No. 62-78. [Abstract obtained from PsychINFO Database Record (c) 2000 APA]. 78

PAGE 88

Kopstein, F.F. & Cave, R.T. (1962). Preliminary cost comparison of technical training by conventional and programmed learning methods. USAF Medical Research Laboratory Technical Documentary Report 1962, No. 62-79. [Abstract obtained from PsychINFO Database Record (c) 2000 APA]. Krieger, R. & Dechert, H.W. (1974). Structured texts: An alternative to the method of small steps in programmed instruction? Psychologie in Erziehung und Unterricht, 21(1), p.22-34. Kritch, K.M. (1995). Density of constructed-response contingencies and its relation to the acquisition of verbal rules in computer-based programmed instruction. Dissertation Abstracts International Section A: Humanities & Social Sciences. University Microfilms International, US Oct 1995, Vol. 56(4-A), p. 1291. Kritch, K.M., Bostow, D.E., & Dedrick, R.F. (1995). Level of interactivity of videodisc instruction on college students recall of AIDS information. Journal of Applied Behavior Analysis, 28(1), p 85-86. Kritch, Kale M. and Bostow, Darrel E. (1998). Degree of constructed-response interaction in computer-based programmed instruction. Journal of Applied Behavior Analysis, 31(3), p. 387-398. Kryger, Roth, and Dement (Eds.) 2000. Principles and Practice of Sleep Medicine, 3rd Edition, Philadelphia, PA: Saunders. Kulhavy, R.W., Yekovich, F.R., and Dyer, J.W. (1979). Feedback and content review in programmed instruction. Contemporary Educational Psychology, 4(2), p. 91-98. Kulik, J.A., Kulik, C.C., & Cohen, P.A. (1979). A meta-analysis of outcome studies of Kellers personalized system of instruction. American Psychologist, 34(4), p. 307-318. Leith, G.O.M. (1967). Developments in Programmed Learning. In New Media and Methods in Industrial Training, Robinson, J. and Barnes, N. (Eds.). pp. 47-49. British Broadcasting Corporation. Reprinted in: Richmond, W. Kenneth (1970). The concept of educational technology: A dialog with yourself. London: Weidenfeld and Nicolson. Lermack, H. (2003). Using Risk Management to Enhance Employee Training. Risk Management, 50(6) p.34. Little, J.K. (1934). Results of use of machines for testing and for drill upon learning in educational psychology. Journal of Experimental Education, 3. Reprinted in: A.A. Lumsdaine and Robert Glaser (Eds.) 1960. Teaching Machines and Programmed Learning, National Education Association of the United States. 79

PAGE 89

Lumsdaine, A.A. (1959). Some issues concerning devices and programs for automated learning. Reprinted in: A.A. Lumsdaine and Robert Glaser (Eds.) 1960. Teaching Machines and Programmed Learning, National Education Association of the United States. Lumsdaine, A.A. & Glaser, R. (1960). Teaching Machines and Programmed Learning: A Source Book. Oxford, England: National Education Association. Lysaught, J.P. and Williams, C.M. (1963). A Guide to Programmed Instruction. New York, NY: John Wiley and Sons, Inc. Mager, R.F. (1963). The psychologists role in automated instruction. Psychological Reports, 12(3), p.743-747. Markle, S.M. (1969). Good Frames and Bad. New York, NY: John Wiley and Sons, Inc. Martin, T.L., Pear, J.J., & Martin, G.L. (2002). Feedback and its effectiveness in a computer-aided personalized system of instruction course. Journal of Applied Behavior Anlsysis, 35(4), p. 427-430. Mayer, R.E. & Gallini, J.K. (1990). When Is an Illustration Worth Ten Thousand Words? Journal of Educational Psychology, 82(4), p. 715-726. Mayer, R.E. (1993). Illustrations that instruct. Found in Glaser, Robert (Ed.), Advances in Instructional Psychology, Vol. 4; pp. 253-284. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Mayer, R.E. (2001). Multimedia Learning. Cambridge, UK: Cambridge University Press. p.3. Mayer, R.E. (2002a). Multimedia Learning. In Ross, Brian H. (Ed.), Psychology of learning and motivation: Advances in research and theory, Vol. 41. San Diego, CA: Academic Press, pp. 85-139. Mayer, R.E. (2002b). Using illustrations to promote constructivist learning from science text. In J. Otero & J. Leon (Eds.). The psychology of science text comprehension. Mahwah, NJ: Lawrence Erlbaum Associates, p333-356. Mayer, R.E. (2003a) Theories of Learning and Their Application to Technology. In H.F. O'Neil, Jr. & R.S. Perez (Eds.). Technology Applications in Education. Mahwah, NJ: Lawrence Erlbaum Associates, Publishers. Mayer, R.E. (2003b). The promise of multimedia learning: Using the same instructional design methods across different media. Learning and Instruction, 13(2), 125-139. 80

PAGE 90

Mayer, R.E., Heiser, J., & Lonn, S. (2001). Cognitive constraints on multimedia learning: When presenting more material results in less understanding. Journal of Educational Psychology, 93(1), p. 187-198. Mayer, R.E. & Moreno, R. (1998). A split-attention effect in multimedia learning: Evidence for dual processing systems in working memory. Journal of Educational Psychology, 90(2), p.312-320. Mayer, R.E. & Moreno, R. (1999). Cognitive principles of multimedia learning: The role of modality and contiguity. Journal of Educational Psychology, 91(2), p. 358-368. Mayer, R.E. & Moreno, R. (2002a). Learning science in virtual reality multimedia environments: Role of methods and media. Journal of Educational Psychology, 94(3), p.598-610. Mayer, R.E. & Moreno, R. (2002b). Verbal redundancy in multimedia learning: When reading helps listening. Journal of Educational Psychology, 94(1), p. 156-163. Mayer, R.E. & Moreno, R. (2002c). Aids to computer-based multimedia learning. Learning & Instruction, 12(1), p.107-119. Mayer, R.E. & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38(1), 43-52. Merrill, I.R., Yaryan, R.B., and Musser, T.S. (1969). The effects of effort and reinforcement on retention and enjoyment of programmed instruction. Journal of Medical Education, 44(3), p. 184-192. Miller, G.A. (1956). The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychological Review, 63, pp. 81-97. Misanchuk, E.R. (1992). Preparing instructional text: Document design using desktop publishing. Englewood Cliffs, NJ: Educational Technology Publications. Morris, C.G. and Maisto, A.A. (2002). Psychology: An Introduction. Upper Saddle River, NJ: Prentice Hall. Myers, D.G. (1998). Psychology, 5th Edition. New York, NY: Worth Publishers. Nairne, J.S. (2003). Psychology: The Adaptive Mind, 3rd Edition. Belmont, CA: Wadsworth/Thomson Learning. O'Day, E.F., Kulhavy, R.W., Anderson, W., and Malczynski, R.J. (1971). Programmed Instruction: Techniques and Trends. New York: Apleton-Century-Crofts. 81

PAGE 91

Open eBook Forum (2000). A framework for the epublishing ecology: Public comment draft. Version 0.78 September 25, 2000. International Trade and Standards Organization for the eBook Industry. http://www.openebook.org/default.htm Peigneux, P., Laureys, S., Delbeuck, X., and Maquet, P. (2001). Sleeping brain, learning brain. The role of sleep for memory systems. NeuroReport, 12(18), p. A111-A124. Pikas, A. (1969). Comparisons between traditional programmed learning as a function of passive performance and active application and time till application. Programmed Learning, 6(1), p.20-25. Pollock, E., Chandler, P. & Sweller, J. (2002). Assimilating complex information. Learning & Instruction, 12(1), p. 61-86. Pressey, S.L. (1926). A simple apparatus which gives tests and scores and teaches. School and Society, 23, p. 373-376. Pressey, S.L. (1927). A machine for automatic teaching of drill material. School and Society, 25, p. 549-552. Pressey, S.L. (1932). A third and fourth contribution toward the coming industrial revolution in education. School and Society, 34, p668-672. Pressey, S.L. (1950). Development and appraisal of devices providing immediate automatic scoring of objective tests and concomitant self-instruction. Journal of Psychology, 29, p. 417-447. Rauchs, G., Bertran, F., Guillery-Girard, B., Desgranges, B., Kerrouche, N., Denise, P., Foret, J., and Eustache, F. (2004). Consolidation of strictly episodic memories mainly requires rapid eye movement sleep. Sleep: Journal of Sleep & Sleep Disorders Research, 27(3), p. 395-401. Reid, R.L. & Taylor, J. (1965). Programmed learning with and without responding. Occupational Psychology, 39(2), p. 145-149. Reigeluth, C.M. & Moore, J. (1999). Cognitive education and the cognitive domain. In C.M. Reigeluth (Ed.), Instructional-Design Theories and Models Volume II: A New Paradigm of Instructional Theory (pp. 51-68). Mahwah, NJ: Lawrence Erlbaum Associates, Publishers. Rowntree, D. (1966). Basically Branching: A handbook for programmers. London: MacDonald & Co. Ltd. 82

PAGE 92

Saettler, P. (1990). The Evolution of American Educational Technology. Englewood, CO: Libraries Unlimited, Inc. Senter, R.J., Nieberg, A., Abma, J.S., & Morgan, R.L. (1963). An evaluation of branching and motivational phrases in a scrambled book. USAF Aerospace Medical Research Laboratories Technical Documentary Report 1963, No. 63-122. Senter, R.J., Abma, J.S., Johnson, K.A., & Morgan, R.L. (1966). An experimental comparison of an intrinsically programmed text and a narrative text. UASF Aerospace Medical Research Laboratories Technical Report 1966, No. 65-227. Shah, G.B. (1971). Effectiveness of four response modes in programmed learning: An exploration. Indian Educational Review, 6(1), p. 101-117. Sime, M. and Boyce, G. (1969). Overt responses, knowledge of results and learning. Programmed Learning, 6(1), p. 12-19. Skinner, B.F. (1954). The Science of Learning and the Art of Teaching. Harvard Educational Review, 24(2), p. 86-97. Skinner, B.F. (1958). Teaching Machines. Science, 128, October 24, 1958. Skinner, B.F. (1968). The Technology of Teaching. New York: Appleton-Century-Crofts. Skinner, B.F. (1961). Teaching Machines. Scientific American, 205(5), p90-107. Smith, W. & Moore, J.W. (1962). Size-of-step and achievement in programmed spelling. Psychological Reports 10(1), p.287-294. Snelbecker, G.E., Sherman, L.J., Rothstein, E., & Cramer, H. (1963). Programmed instruction and neuropsychiatric diabetic patients: A preliminary study. Journal of Programmed Instruction, 2(4), p. 47-51. Stephens, A.L. (1953). Certain Special Factors Involved in the Law of Effect. Reprinted in Lumsdaine & Glaser (Eds.), Teaching Machines and Programmed Instruction: A source book. 1960. National Education Association of the United States. Stolurow, L.M (1961). Teaching by Machine. Cooperative Research Monograph No.6, OE-34010, U.S. Department of Health, Education, and Welfare. Subedi, B.S. (2004). Emerging trends of research on transfer of learning. International Education Journal, 5(4), pp 591-599. 83

PAGE 93

Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), pp. 257-285. Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction, 4(4), pp. 295-312. Sweller, J., van Merrienboer, J., and Paas, F. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10, pp. 251-296. Tabbers, H.K., Martens, R.L., & van Merrienboer, J.G. (2004). Multimedia instructions and cognitive load theory: Effects of modality and cueing. British Journal of Educational Psychology, 74, 71-81. Tenebaum, G., Naidu, S., Jegede, O., and Austin, J. (2001). Constructivist pedagogy in conventional on-campus and distance practice: an exploratory investigation. Learning and Instruction, 11 (1), 87-111. Tennyson, R.D. and Elmore, R.L. (1997). Learning Theory Foundations for Instructional Design. In Tennyson, Schott, Seel, and Dijkstra (Eds.) Instructional Design: International Perspectives, Volume 1: Theory, Research, and Models. Mahwah, NJ: Lawrence Erlbaum Associates, Publishers. Tennyson, R.D. and Schott, F. (1997). Instructional Design Theory, Research, and Models. In Tennyson, Schott, Seel, and Dijkstra (Eds.) Instructional Design: International Perspectives, Volume 1: Theory, Research, and Models. Mahwah, NJ: Lawrence Erlbaum Associates, Publishers. Ter Keurst, A.J. (1965). The intrinsic programmed approach to the discovery of concepts in general psychology. Journal of Experimental Education, 33(4), p. 351-355. Thorndike, E.L. (1911). Animal Intelligence: Experimental Studies. Lewiston, NY: Macmillan Press. Tobias, S. (1969). Effect of creativity response mode, and subject matter familiarity on achievement from programmed instruction. Journal of Educational Psychology, 60(6), p. 453-460. Tobias, S. & Abramson, T. (1971). Interaction among anxiety, stress, response mode, and familiarity of subject matter on achievement from programmed instruction. Journal of Educational Psychology, 62(4), p. 357-364. Tudor, R.M. & Bostow, D.E. (1991). Computer-programmed instruction: The relation of required interaction to practical application. Journal of Applied Behavior Analysis, 24(2), p. 361-368. 84

PAGE 94

85 University of Idaho Engineering Outreach. (1995). Print in Distance Education: Distance Education at a Glance, Guide 7, October 1995. University of Idaho, College of Engineering, Engineering Outreach. http://www.uidaho.edu/eo Valverde, H.H. (1968). Effect of confirmation pe eking and response mode on programmed instruction. US AFHRD Technical Report Air Force Systems/Material Command, US 1968, No. 67-225. Van Gerven, P.W.M., Paas, F.G.W.C., van Merrienboer, J.J.J, & Schmidt, H.G. (2002). Cognitive load theory and aging: Effects of worked examples on training efficiency. Learning & Instruction, 12(1), p. 87-105. Van Merrienboer, J.J.G., Schuurman, J.G., de Croock, M.B.M., & Paas, F.G.W.C. (2002). Redirecting lear ners attention during training: Effects on cognitive load, transfer test perfor mance and training efficiency. Learning & Instruction, 12(1), p. 11-37. Verduin, J.R. & Clark, T.A. (1991). Distance education: The f oundations of effective practice. San Francisco: Jossey-Bass. Verhaeghen, P., Cerella, J., & Basak, C. ( 2004). A working memory workout: How to expand the focus of serial attention from one to four items in 10 hours or less. Journal of Experimental Psychology: Learning, Memory, and Cognition, 30(6), 1322-1337. Wagner, E.D. (2003). Where learni ng thrive across the enterprise. Learning and Training Inovations, 4(5), p.20. Watson, J.B. (1913). Psychology as the behaviourist views it. Psychological Review, 20(2) p. 158-177. Williams, J.P. (1966). Combining response modes in programmed instruction. Journal of Educational Psychology, 57(4) p. 215-219. Yarmey, A.D. (1964). Overt and cove rt responding in programmed learning. Ontario Journal of Educational Research, 7(1) p. 27-33.

PAGE 95

Appendices 86

PAGE 96

Appendix A Terminal Behaviors Consciousness 1. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term consciousness (discrimination). 2. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term awareness (discrimination). 3. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term attention (discrimination). 4. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term altered state of consciousness (discrimination). 5. In the presence of appropriate stimuli, the learner will be able to identify instances of the concept altered state of consciousness (generalization). 6. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term daydreaming (discrimination). Biological Rhythms 7. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Biological Rhythm (discrimination). 8. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Circadian rhythm (discrimination). 9. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Infradian rhythm (discrimination). 10. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Ultradian rhythm (discrimination). 11. In the presence of appropriate stimuli, the learner will be able to identify instances of the concept biological rhythm (generalization). 87

PAGE 97

Appendix A (Continued) 12. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Chronobiology (discrimination). 13. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Free running (discrimination). 14. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Zeitgeber (discrimination). 15. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Suprachiasmatic nucleus [SCN] (discrimination). 16. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Jet lag (discrimination). 17. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Shift work (discrimination). The Study of Sleep 18. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Electroencephalogram [EEG] (discrimination). 19. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Electro-oculogram [EOG] (discrimination). 20. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Electromyogram [EMG] (discrimination). 21. In the presence of appropriate stimuli, the learner will be able to identify the electrode location diagram described in the International 10-20 Electrode Placement System (discrimination). Sleep Staging 22. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Sine wave (discrimination). 23. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Amplitude (discrimination). 88

PAGE 98

Appendix A (Continued) 24. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Frequency (discrimination). 25. The learner will demonstrate the ability to identify discriminate EEG waveforms by frequency and amplitude (discriminate the difference between fast and slow frequency EEG activity; high and low amplitude EEG activity) (generalization). 26. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Alpha Frequency EEG (discrimination). 27. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Beta Frequency EEG (discrimination). 28. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Theta Frequency EEG (discrimination). 29. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Delta Frequency EEG (discrimination). 30. The learner will be able to identify the EEG frequency bands associated with the different stages of sleep (concept). 31. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Sleep Staging (discrimination). 32. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Stage Wake (discrimination). 33. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Stage 1 Sleep (discrimination). 34. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Stage 2 Sleep (discrimination). 35. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Stage 3 Sleep (discrimination). 36. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Stage 4 Sleep (discrimination). 89

PAGE 99

Appendix A (Continued) 37. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term REM Sleep (discrimination). 38. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Rapid Eye Movements (discrimination). 39. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Vertex Wave (discrimination). 40. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term K-complex (discrimination). 41. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Sleep Spindle (discrimination). 42. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Delta Waves (discrimination). 43. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Slow-wave sleep (discrimination). 44. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Tonic REM (discrimination). 45. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Phasic REM (discrimination). Sleep Cycles 46. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Sleep Cycles (discrimination). 47. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Sleep Architecture (discrimination). Sleep Debt 48. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Sleep Debt (discrimination). 90

PAGE 100

91 Appendix A (Continued) 49. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Short Sleeper (discrimination). 50. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Long Sleeper (discrimination). 51. In the presence of appropriate stimuli, the learner will be able to identify the definition of the term Homeostatic drive for sleep (discrimination).

PAGE 101

Appendix B Chi Square Analyses Pilot Study Initial Data 27 USF undergraduate psychology students completed a test that combined the items from the initially developed short-term and long-term tests. These students were drawn from three different courses. The raw data collected is shown below. question # Class A B C D E Totals question # Class A B C D E Totals 1 1 0 0 4 0 5 1 1 1 1 2 0 5 2 2 0 1 14 0 17 1 6 5 Sum 6 4 12 5 0 27 Sum 4 # 6 7 0 27 2 2 1 13 1 0 17 3 0 1 1 3 0 5 3 1 0 2 2 0 5 Sum 3 1 2 21 0 27 Sum 4 2 16 5 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 1 2 1 0 5 1 0 3 1 1 0 5 2 3 5 8 1 0 17 2 2 8 3 4 0 17 2 3 1 1 2 1 0 5 7 3 2 1 1 1 0 5 Sum 5 7 12 3 0 27 Sum 4 # 5 6 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 3 0 1 1 0 5 1 1 2 0 2 0 5 2 8 0 3 6 0 17 2 7 3 3 4 0 17 3 3 2 0 1 2 0 5 8 3 1 3 1 0 0 5 Sum 13 0 5 9 0 27 Sum 9 8 4 6 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 2 2 0 0 5 1 3 0 1 1 0 5 2 4 6 5 2 0 17 2 4 3 6 4 0 17 4 3 1 3 1 0 0 5 9 3 1 0 3 1 0 5 Sum 6 11 8 2 0 27 Sum 8 3 10 6 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 0 2 2 0 5 1 0 2 0 3 0 5 2 5 4 7 1 0 17 2 4 6 5 2 0 17 5 3 0 0 3 2 0 5 10 3 0 2 1 2 0 92

PAGE 102

Appendix B (continued) question # Class A B C D E Totals question # Class A B C D E Totals 1 2 1 1 0 1 5 18 1 0 2 2 1 0 5 2 4 2 5 3 3 17 2 2 7 7 1 0 17 11 3 2 1 2 0 0 5 3 2 2 1 0 0 5 Sum 8 4 8 3 4 27 Sum 4 # 10 2 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 0 1 2 2 0 5 19 1 1 2 2 0 0 5 2 5 4 7 1 0 17 2 4 7 4 2 0 17 12 3 2 0 1 2 0 5 3 1 2 2 0 0 5 Sum 7 5 10 5 0 27 Sum 6 # 8 2 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 13 1 2 0 1 1 1 5 20 1 2 0 0 3 0 5 2 10 0 3 1 3 17 2 8 1 0 8 0 17 3 2 0 2 0 1 5 3 2 0 0 3 0 5 Sum 14 0 6 2 5 27 Sum # 1 0 # 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 14 1 0 0 1 1 3 5 21 1 0 2 1 1 1 5 2 2 4 1 1 9 17 2 4 4 1 4 4 17 3 0 1 1 1 3 6 3 0 3 1 1 0 5 Sum 2 5 3 3 # 28 Sum 4 9 3 6 5 27 question # Class A B C D E Totals question # Class A B C D E Totals 15 1 0 3 0 2 0 5 22 1 0 1 3 1 0 5 2 0 6 1 10 0 17 2 1 8 4 4 0 17 3 0 1 0 4 0 5 3 1 1 2 1 0 5 Sum 0 10 1 16 0 27 Sum 2 # 9 6 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 16 1 3 0 1 1 0 5 23 1 1 1 1 1 1 5 2 9 1 2 5 0 17 2 2 4 5 3 3 17 3 2 1 1 1 0 5 3 1 0 2 2 0 5 Sum 14 2 4 7 0 27 Sum 4 5 8 6 4 27 question # Class A B C D E Totals question # Class A B C D E Totals 17 1 2 0 1 2 0 5 24 1 1 3 0 1 0 5 2 7 0 4 6 0 17 2 3 8 4 2 0 17 3 1 1 1 2 0 5 3 1 2 1 1 0 5 93

PAGE 103

Appendix B (continued) question # Class A B C D E Totals question # Class A B C D E Totals 25 1 0 0 1 2 2 5 1 2 1 1 1 0 5 2 2 1 4 6 4 17 2 7 6 0 4 0 17 3 1 1 1 1 1 5 31 3 2 2 0 1 0 5 Sum 3 2 6 9 7 27 Sum 11 9 1 6 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 26 1 0 2 2 1 0 5 1 1 1 2 1 0 5 2 2 4 7 4 0 17 2 7 4 3 3 0 17 3 1 2 1 1 0 5 32 3 2 1 2 0 0 5 Sum 3 8 10 6 0 27 Sum 10 6 7 4 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 0 1 2 2 0 5 1 1 1 0 3 0 5 2 2 4 6 5 0 17 2 6 4 3 4 0 17 27 3 0 1 1 3 0 5 33 3 3 0 1 1 0 5 5 Sum 8 3 4 10 2 27 Sum 6 6 8 7 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 2 1 2 0 0 5 1 0 2 1 1 1 5 2 5 4 4 3 1 17 2 3 7 2 1 4 17 30 3 1 2 2 0 0 5 36 3 1 2 1 0 1 5 Sum 8 7 8 3 1 27 Sum 4 11 4 2 6 27 Sum 2 6 9 10 0 27 Sum 10 5 4 8 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 0 1 2 2 0 5 1 3 1 0 1 0 5 2 0 4 5 8 0 17 2 10 4 2 1 0 17 28 3 0 2 1 2 0 5 34 3 2 1 1 1 0 5 Sum 0 7 8 12 0 27 Sum 15 6 3 3 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 0 1 1 2 5 1 1 1 2 1 0 5 2 5 3 2 7 0 17 2 4 4 5 4 0 17 29 3 2 0 1 2 0 5 35 3 1 1 1 2 0 94

PAGE 104

Appendix B (continued) question # Class A B C D E Totals question # Class A B C D E Totals 1 3 1 1 0 0 5 1 0 1 3 1 0 5 2 7 4 3 3 0 17 2 5 5 3 4 0 17 37 3 2 2 0 1 0 5 43 3 0 3 2 0 0 5 Sum 12 7 4 4 0 27 Sum 5 9 8 5 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 1 1 2 0 5 1 1 2 0 2 0 5 2 2 5 5 5 0 17 2 1 14 1 1 0 17 38 44 5 Sum 9 12 6 0 0 27 Sum 16 3 1 7 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 2 2 1 0 0 5 1 1 3 0 1 0 5 2 6 5 5 1 0 17 2 4 4 6 3 0 17 42 3 1 2 2 0 0 5 48 3 0 1 2 2 0 5 Sum 9 9 8 1 0 27 Sum 5 8 8 6 0 27 3 1 3 1 0 0 5 3 1 4 0 0 0 5 Sum 4 9 7 7 0 27 Sum 3 20 1 3 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 2 2 0 1 0 5 1 0 1 3 1 0 5 2 3 3 6 5 0 17 2 2 3 10 2 0 17 39 3 0 1 2 2 0 5 45 3 1 2 1 1 0 5 Sum 5 6 8 8 0 27 Sum 3 6 14 4 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 0 3 2 0 6 1 0 2 1 2 0 5 2 3 4 5 5 0 17 2 1 8 5 2 0 16 40 3 1 1 1 2 0 5 46 3 2 2 1 0 0 5 Sum 5 5 9 9 0 28 Sum 3 12 7 4 0 26 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 3 1 0 0 5 1 3 1 0 1 0 5 2 6 7 4 0 0 17 2 10 2 1 4 0 17 41 3 2 2 1 0 0 5 47 3 3 0 0 2 0 95

PAGE 105

Appendix B (continued) question # Class A B C D E Totals question # Class A B C D E Totals 1 1 0 2 2 0 5 1 1 1 1 2 0 5 2 4 3 6 4 0 17 2 2 4 4 7 0 17 49 3 0 1 3 1 0 5 55 3 0 3 0 2 0 5 Sum 5 4 11 7 0 27 Sum 3 8 5 11 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 1 2 1 0 5 1 0 2 3 0 0 5 2 5 4 6 2 0 17 2 4 5 7 1 0 17 50 3 2 1 2 0 0 5 56 3 0 0 5 0 0 5 Sum 8 6 10 3 0 27 Sum 4 7 15 1 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 1 2 1 0 5 1 2 1 1 1 0 5 2 2 6 6 3 0 17 2 3 11 1 2 0 17 51 3 1 1 3 0 0 5 57 3 1 3 1 0 0 5 Sum 4 8 11 4 0 27 Sum 6 15 3 3 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 2 2 1 0 0 5 1 1 3 1 0 0 5 2 5 2 7 2 0 16 2 1 11 5 0 0 17 52 3 3 2 0 0 0 5 58 3 0 5 0 0 0 5 Sum 10 6 8 2 0 26 Sum 2 19 6 0 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 2 2 0 0 5 1 2 2 0 1 0 5 2 5 8 2 1 0 16 2 9 3 3 2 0 17 53 3 2 2 1 0 0 5 59 3 1 3 1 0 0 5 Sum 8 12 5 1 0 26 Sum 12 8 4 3 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 0 2 1 2 0 5 1 2 0 2 1 0 5 2 1 5 5 6 0 17 2 6 3 6 2 0 17 54 3 0 3 0 2 0 5 60 3 1 2 2 0 0 5 Sum 1 10 6 10 0 27 Sum 9 5 10 3 0 27 96

PAGE 106

Appendix B (continued) question # Class A B C D E Totals question # Class A B C D E Totals 1 1 3 1 0 0 5 1 0 3 1 1 0 5 2 3 5 4 5 0 17 2 1 8 3 5 0 17 61 3 3 2 0 0 0 5 67 3 1 1 1 2 0 5 Sum 7 10 5 5 0 27 Sum 2 12 5 8 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 3 1 1 0 0 5 1 1 1 1 1 1 5 2 8 6 2 1 0 17 2 2 3 4 4 4 17 62 3 0 2 1 2 0 5 68 3 0 1 1 2 1 5 Sum 11 9 4 3 0 27 Sum 3 5 6 7 6 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 2 0 0 3 0 5 1 0 0 1 2 2 5 2 8 1 1 8 0 18 2 1 3 5 4 4 17 63 3 1 0 0 4 0 5 69 3 1 2 1 1 0 5 Sum 11 1 1 15 0 28 Sum 2 5 7 7 6 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 2 1 1 0 5 1 1 1 3 0 0 5 2 2 1 2 12 0 17 2 2 3 9 3 0 17 64 3 0 1 0 4 0 5 70 3 1 2 1 1 0 5 Sum 3 4 3 17 0 27 4 Sum 9 11 5 2 0 27 Sum 5 11 5 5 0 26 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 1 1 2 0 5 1 0 2 1 2 0 5 2 2 3 2 10 0 17 2 4 6 3 4 0 17 66 3 1 1 1 2 0 5 72 3 2 1 1 1 0 5 Sum 4 5 4 14 0 27 Sum 6 9 5 7 0 27 Sum 4 6 13 4 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 3 1 0 0 5 1 1 2 1 1 0 5 2 6 5 4 2 0 17 2 3 6 4 4 0 17 65 3 2 3 0 0 0 5 71 3 1 3 0 0 0 97

PAGE 107

Appendix B (continued) question # Class A B C D E Totals question # Class A B C D E Totals 1 2 0 2 1 0 5 1 0 0 0 5 0 5 2 5 3 5 4 0 17 2 0 1 1 14 1 17 73 3 2 0 1 2 0 5 79 3 0 0 0 5 0 5 Sum 9 3 8 7 0 27 Sum 0 1 1 24 1 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 1 2 1 0 5 1 1 2 1 1 0 5 2 7 3 4 3 0 17 2 3 6 5 2 1 17 74 3 2 0 2 1 0 5 80 3 2 2 1 0 0 5 Sum 10 4 8 5 0 27 Sum 6 10 7 3 1 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 1 2 1 0 5 1 2 2 0 1 0 5 2 6 4 3 4 0 17 2 7 5 1 4 0 17 75 3 1 3 1 81 5 Sum 1 18 3 5 0 27 Sum 8 11 5 3 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 0 1 2 1 1 5 1 0 3 2 0 0 5 2 2 3 3 3 5 16 2 5 7 5 0 0 17 78 3 1 2 0 0 2 5 84 3 1 2 2 0 0 5 Sum 3 6 5 4 8 26 Sum 6 12 9 0 0 27 0 0 5 3 2 2 0 1 0 5 Sum 8 8 6 5 0 27 Sum 11 9 1 6 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 0 1 2 2 0 5 1 1 2 1 1 0 5 2 2 2 5 8 0 17 2 3 5 1 8 0 17 76 3 0 2 2 1 0 5 82 3 0 3 0 2 0 5 Sum 2 5 9 11 0 27 Sum 4 10 2 11 0 27 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 3 0 1 0 5 1 1 2 1 1 0 5 2 0 10 3 4 0 17 2 6 6 3 2 0 17 77 3 0 5 0 0 0 5 83 3 1 3 1 0 0 98

PAGE 108

Appendix B (continued) question # Class A B C D E Totals question # Class A B C D E Totals 1 2 1 2 0 0 5 1 1 2 1 1 0 5 2 1 7 6 3 0 17 2 4 7 2 4 0 17 85 3 1 2 2 0 0 5 86 3 1 4 0 0 0 5 Sum 4 10 10 3 0 27 Sum 6 13 3 5 0 27 Results of Chi Square Analyses The following table shows the results of the Chi Square analyses for the initial data. Highlighted cells indicate significant cell values. These questions were modified and retested. 99

PAGE 109

Appendix B (continued) Initial Data Question # 2 df Question # 2 df Question # 2 df 1 42.694 6 30 5.518 8 59 13.235 6 2 7.494 6 31 9.565 6 60 7.400 6 3 7.897 6 32 5.329 6 61 9.847 6 4 8.059 6 33 8.718 6 62 13.706 6 5 12.012 6 34 15.871 6 63 25.588 6 6 26.976 6 35 1.377 6 64 28.200 6 7 9.282 6 36 10.235 8 65 11.259 6 8 8.529 6 37 8.529 6 66 11.729 6 9 8.718 6 38 5.988 6 67 10.694 6 10 9.659 6 39 5.988 6 68 2.941 8 11 7.529 8 40 5.447 6 69 8.706 8 12 8.812 6 41 6.015 6 70 11.635 6 13 18.600 8 42 7.871 6 71 6.718 6 14 24.294 8 43 9.847 6 72 3.918 6 15 22.485 6 44 40.624 73 5.047 6 16 13.518 6 45 14.929 6 74 5.329 6 17 9.565 6 46 11.518 6 75 5.518 6 18 11.635 6 47 20.671 6 76 10.224 6 19 7.400 6 48 7.118 6 77 31.212 6 20 12.418 6 49 7.118 6 78 7.471 8 21 10.118 8 50 4.859 6 79 76.129 8 22 10.224 6 51 7.400 6 80 11.059 8 23 5.529 8 52 11.894 6 81 8.812 6 24 9.282 6 53 11.518 6 82 12.294 4 25 8.471 8 54 11.071 6 83 7.400 4 26 5.800 6 55 9.000 6 84 3.670 6 27 8.059 6 56 24.812 6 85 9.753 6 28 5.356 6 57 19.165 6 86 12.200 6 29 14.588 8 58 29.638 6 100

PAGE 110

Appendix C Chi Square Output Modified Pilot Study Modified Test Data In the second phase 15 USF undergraduate students from two introductory psychology courses took a test made up of the combined questions from the short-term and long-term tests. These tests contained the questions modified based on the results of the analyses of the first phase. The following table shows the results of this phase. question # Class A B C D E Totals question # Class A B C D E Totals 1 3 1 4 1 0 9 1 1 5 2 1 0 9 1 3 1 3 2 0 0 6 7 6 Sum 2 1 6 6 0 15 Sum 3 3 4 5 0 15 3 2 1 1 2 0 6 Sum 4 4 6 1 0 15 Sum 3 6 3 3 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 2 1 5 1 0 9 1 2 3 0 4 0 9 2 3 1 2 2 1 0 6 8 3 2 3 1 0 0 6 Sum 3 3 7 2 0 15 Sum 4 6 1 4 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 5 1 1 2 0 9 1 3 2 3 1 0 9 3 3 3 0 1 2 0 6 9 3 2 0 3 1 0 6 Sum 8 1 2 4 0 15 Sum 5 2 6 2 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 2 3 3 1 0 9 1 0 3 2 4 0 9 4 3 2 3 1 0 0 6 10 3 0 3 1 2 0 6 Sum 4 6 4 1 0 15 Sum 0 6 3 6 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 2 3 3 0 9 1 4 1 2 1 1 9 5 3 0 1 4 1 0 6 11 3 2 2 1 1 0 6 Sum 1 3 7 4 0 15 Sum 6 3 3 2 1 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 1 3 4 0 9 1 1 2 3 3 0 9 6 3 1 0 3 2 0 6 12 3 2 1 1 2 0 101

PAGE 111

Appendix C (continued) question # Class A B C D E Totals question # Class A B C D E Totals 1 3 0 3 2 1 9 1 3 0 0 4 2 9 13 3 3 0 2 0 1 6 20 3 2 0 0 3 1 6 Sum 6 0 5 2 2 15 Sum 5 0 0 7 3 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 0 2 2 1 3 8 1 1 3 2 1 2 9 14 3 0 1 1 1 3 6 21 3 0 3 1 2 0 6 Sum 0 3 3 2 6 14 Sum 1 6 3 3 2 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 0 5 1 3 0 9 1 1 2 5 1 0 9 15 3 0 1 2 3 0 6 22 3 1 1 3 1 0 6 Sum 0 6 3 6 0 15 Sum 2 3 8 2 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 5 1 2 1 0 9 1 2 1 2 2 2 9 16 3 3 1 1 1 0 6 23 3 1 0 2 3 0 6 Sum 8 2 3 2 0 15 Sum 3 1 4 5 2 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 4 0 2 3 0 9 1 3 5 0 1 0 9 17 3 2 1 0 3 0 6 24 3 2 2 1 1 0 6 Sum 6 1 2 6 0 15 Sum 5 7 1 2 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 3 3 2 0 9 1 0 0 2 4 3 9 18 3 2 2 2 0 0 6 25 3 1 1 1 1 2 6 Sum 3 5 5 2 0 15 Sum 1 1 3 5 5 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 2 3 3 1 0 9 1 1 3 3 2 0 9 19 3 1 2 3 0 0 6 26 3 1 3 1 1 0 6 Sum 3 5 6 1 0 15 Sum 2 6 4 3 0 15 102

PAGE 112

Appendix C (continued) question # Class A B C D E Totals question # Class A B C D E Totals 1 0 2 4 3 0 9 1 4 2 1 2 0 9 27 3 0 2 1 3 0 6 34 3 2 1 2 1 0 6 Sum 0 4 5 6 0 15 Sum 6 3 3 3 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 0 1 4 4 0 9 1 3 2 3 1 0 9 28 3 0 2 1 3 0 6 35 3 1 1 1 3 0 6 Sum 0 3 5 7 0 15 Sum 4 3 4 4 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 2 1 1 2 3 9 1 2 2 1 1 3 9 29 3 2 0 2 2 0 6 36 3 1 3 0 0 2 6 Sum 4 1 3 4 3 15 Sum 3 5 1 1 5 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 3 2 3 1 0 9 1 4 3 1 1 0 9 30 3 1 2 2 1 0 6 37 3 2 3 0 1 0 6 Sum 4 4 5 2 0 15 Sum 6 6 1 2 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 3 2 1 3 0 9 1 1 3 2 3 0 9 31 3 2 3 1 0 0 6 38 3 1 3 2 0 0 6 6 Sum 3 3 7 2 0 15 Sum 3 6 2 4 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 3 2 1 3 0 9 1 2 0 5 2 0 9 33 3 4 0 1 1 0 6 40 3 2 1 1 2 0 6 Sum 7 2 2 4 0 15 Sum 4 1 6 4 0 15 Sum 5 5 2 3 0 15 Sum 2 6 4 3 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 1 5 2 0 9 1 3 4 0 2 0 9 32 3 2 2 2 0 0 6 39 3 0 2 2 2 0 103

PAGE 113

Appendix C (continued) question # Class A B C D E Totals question # Class A B C D E Totals 1 3 3 3 0 0 9 1 1 4 1 3 0 9 41 3 2 3 1 0 0 6 48 3 1 1 2 2 0 6 Sum 5 6 4 0 0 15 Sum 2 5 3 5 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 3 4 2 0 0 9 1 1 0 4 4 0 9 42 3 1 3 2 0 0 6 49 3 1 1 3 1 0 6 Sum 4 7 4 0 0 15 Sum 2 1 7 5 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 3 3 2 0 9 1 1 3 4 1 0 9 43 3 0 3 3 0 0 6 50 3 2 1 2 1 0 6 Sum 1 6 6 2 0 15 Sum 3 4 6 2 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 4 1 3 0 9 1 1 2 5 1 0 9 44 51 6 Sum 5 6 1 3 0 15 Sum 4 5 5 1 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 4 1 2 2 0 9 1 0 4 2 3 0 9 47 3 3 0 1 2 0 6 54 3 1 3 0 2 0 6 Sum 7 1 3 4 0 15 Sum 1 7 2 5 0 15 3 1 3 1 1 0 6 3 1 1 3 1 0 6 Sum 2 7 2 4 0 15 Sum 2 3 8 2 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 3 1 3 2 0 9 1 3 1 4 1 0 9 45 3 1 2 1 2 0 6 52 3 3 2 0 1 0 6 Sum 4 3 4 4 0 15 Sum 6 3 4 2 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 2 4 1 2 0 9 1 2 3 4 0 0 9 46 3 3 2 0 1 0 6 53 3 2 2 1 1 0 104

PAGE 114

Appendix C (continued) question # Class A B C D E Totals question # Class A B C D E Totals 1 2 1 2 4 0 9 1 4 2 1 2 0 9 55 3 1 3 0 2 0 6 62 3 1 2 1 2 0 6 Sum 3 4 2 6 0 15 Sum 5 4 2 4 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 2 4 2 0 9 1 3 1 2 3 0 9 56 3 1 1 3 1 0 6 63 3 1 0 1 4 0 6 Sum 2 3 7 3 0 15 Sum 4 1 3 7 0 15 question # Class A B C D E Totals question # Class 6 Sum 3 2 7 3 0 15 Sum 2 7 3 3 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 4 3 1 0 9 1 1 1 1 3 3 9 61 3 2 2 1 1 0 6 68 3 0 2 1 2 1 6 Sum 3 6 4 2 0 15 Sum 1 3 2 5 4 15 A B C D E Totals 1 2 3 3 1 0 9 1 2 3 2 2 0 9 57 3 1 3 1 1 0 6 64 3 0 1 1 4 0 6 Sum 3 6 4 2 0 15 Sum 2 4 3 6 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 6 2 0 0 9 1 2 4 2 1 0 9 58 3 2 2 1 1 0 6 65 3 2 3 1 0 0 6 Sum 3 8 3 1 0 15 Sum 4 7 3 1 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 3 3 0 3 0 9 1 1 1 2 5 0 9 59 3 1 3 1 1 0 6 66 3 1 2 1 2 0 6 Sum 4 6 1 4 0 15 Sum 2 3 3 7 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 2 0 5 2 0 9 1 1 5 2 1 0 9 60 3 1 2 2 1 0 6 67 3 1 2 1 2 0 105

PAGE 115

Appendix C (continued) question # Class A B C D E Totals question # Class A B C D E Totals 1 0 0 2 3 4 9 1 1 1 3 4 0 9 69 3 1 2 1 1 1 6 76 3 1 2 3 0 0 6 Sum 1 2 3 4 5 15 Sum 2 3 6 4 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 2 1 5 1 0 9 1 2 5 1 1 0 9 70 3 1 2 2 1 0 6 77 3 1 3 1 1 0 6 Sum 3 3 7 2 0 15 Sum 3 8 2 2 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 2 3 2 2 0 9 1 0 1 4 3 1 9 71 3 1 4 1 0 0 6 78 3 1 2 1 0 2 6 Sum 3 7 3 2 0 15 Sum 1 3 5 3 3 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 5 1 2 0 9 1 1 2 1 4 0 8 72 3 2 1 2 1 0 6 79 3 0 1 1 4 0 6 Sum 3 6 3 3 0 15 Sum 1 3 2 8 0 14 question # Class A B C D E Totals question # Class A B C D E Totals 1 4 0 3 2 0 9 1 2 5 1 1 0 9 73 3 2 1 1 2 0 6 80 3 2 2 1 1 0 6 Sum 6 1 4 4 0 15 Sum 4 7 2 2 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 1 1 5 2 0 9 1 4 3 0 2 0 9 74 3 2 1 2 1 0 6 81 3 2 2 1 1 0 6 Sum 3 2 7 3 0 15 Sum 6 5 1 3 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 3 1 4 1 0 9 1 1 4 1 3 0 9 75 3 1 3 1 1 0 6 82 3 0 3 1 2 0 6 Sum 4 4 5 2 0 15 Sum 1 7 2 5 0 15 106

PAGE 116

Appendix C (continued) question # Class A B C D E Totals question # Class A B C D E Totals 1 3 3 3 0 0 9 1 4 2 3 0 0 9 83 3 2 2 2 0 0 6 85 3 1 2 3 0 0 6 Sum 5 5 5 0 0 15 Sum 5 4 6 0 0 15 question # Class A B C D E Totals question # Class A B C D E Totals 1 3 4 2 0 0 9 1 1 3 3 2 0 9 84 3 1 3 2 0 0 6 86 3 1 3 1 1 0 6 Sum 4 7 4 0 0 15 Sum 2 6 4 3 0 15 Results of Chi Square Analyses. The revised shortand long-term questions were tested and the results of the Chi Square analyses are listed in the table below. There were no significant cell differences found. 107

PAGE 117

Appendix C (continued) Revised Data Question # 2 df Question # 2 df Question # 2 df 1 6.333 3 30 1.889 4 59 5.000 3 2 5.444 3 31 4.556 3 60 6.333 3 3 5.917 3 32 6.778 3 61 3.667 3 4 4.556 3 33 7.222 3 62 2.778 3 5 7.222 3 34 2.778 3 63 7.222 3 6 6.333 3 35 3.222 3 64 6.333 3 7 5.444 3 36 7.222 4 65 5.444 3 8 7.222 3 37 6.333 3 66 5.444 3 9 4.556 3 38 4.556 3 67 5.444 3 10 7.222 3 39 5.889 3 68 5.000 4 11 6.111 4 40 6.333 3 69 7.778 4 12 1.889 3 41 2.583 3 70 5.444 3 13 6.444 4 42 7.222 3 71 6.333 3 14 7.000 4 43 7.222 3 72 5.444 3 15 6.139 3 44 5.000 3 73 4.556 3 16 6.778 3 45 1.889 3 74 5.444 3 17 7.222 3 46 5.444 3 75 5.000 3 18 3.222 3 47 5.444 3 76 6.333 3 19 4.556 3 48 3.667 3 77 6.778 3 20 3.278 3 49 7.667 3 78 8.333 4 21 7.222 4 50 3.667 3 79 6.583 4 22 6.778 3 51 6.778 3 80 5.444 4 23 6.111 4 52 6.333 3 81 4.556 3 24 7.222 3 53 4.556 3 82 6.333 2 25 7.778 4 54 7.222 3 83 5.000 2 26 3.222 3 55 5.444 3 84 1.667 3 27 7.222 3 56 4.111 3 85 1.667 3 28 5.250 3 57 3.222 3 86 4.022 3 29 5.556 4 58 7.472 3 108

PAGE 118

Appendix D Retention/Transfer Question Matrix The following table contains a matrix that will allow quick identification of the retention and transfer questions for both the Short-Term and Long-Term tests. Short-Term Test Long-Term Test Retention Questions Transfer Questions Retention Questions Transfer Questions 1 11 1 12 2 13 2 21 3 21 3 23 4 23 4 25 5 26 5 28 6 27 6 30 7 29 7 34 8 30 8 35 9 35 9 37 10 36 10 39 12 39 11 40 14 41 13 41 15 42 14 42 16 43 15 43 17 16 18 17 19 18 20 19 22 20 24 22 26 24 28 26 31 27 32 29 33 31 34 32 37 33 38 36 40 38 109

PAGE 119

Appendix E Short-Term Assessment 1. Which of the following are NOT elements of Tabers Medical Dictionary definition of consciousness: a. A state of awareness b. Orientation to time, place, and person. c. consciousness is a composite of memory and the comprehension of external reality. d. All of the above (a, b, & c) are included in Tabers definition. e. None of the above (a, b, & c) are included in Tabers definition. Question 1 is a RETENTION question and relates to items 1 6 of the Terminal Behaviors list regarding Consciousness. 2. Consciousness can be thought of as flowing. William James compared consciousness to a ________. a. pool b. lake c. stream d. highway Question 2 is a RETENTION question and relates to items 1 6 of the Terminal Behaviors list regarding Consciousness. 3. Sleep and consciousness share several qualities. Depth is one of these qualities. The more deeply asleep one is, the more: a. disassociated from reality one is. b. close to waking one is. c. focused ones attention is. d. vivid ones dreams are likely to be. Question 3 is a RETENTION question and relates to items 1 6 of the Terminal Behaviors list regarding Consciousness. 4. Sleep can be thought of as: a. a state of total unconsciousness. b. a reversible behavioral state of perceptual disengagement from and unresponsiveness to the environment. c. a behavioral state of disengagement from and unresponsiveness to others perceptual states. d. an non-reversible state characterized by perceptual enhancement and responsiveness to the unconscious mind. Question 4 is a RETENTION question and relates to items 1 6 of the Terminal Behaviors list regarding Consciousness. 110

PAGE 120

Appendix E (Continued) 5. A biological rhythm with a period (from peak to peak or trough to trough) of about 24 hours. This is the definition of ________ __________. a. ultradian rhythms b. infradian rhythms c. circadian rhythm or circadian rhythms d. time Question 5 is a RETENTION question and relates to items 7 12 of the Terminal Behaviors list regarding Biological Rhythms. 6. This chemical, which is produced in the pineal gland, is thought to play an important role in regulating waking-sleeping cycles. a. triptophan b. testosterone c. melatonin d. norepinephrine Question 6 is a RETENTION question and relates to items 7, 8, & 11 of the Terminal Behaviors list regarding Biological Rhythms. 7. The migratory patterns of birds and the hibernation of bears are examples of ______ rhythms. a. infradian b. circadian c. ultradian d. imperian Question 7 is a RETENTION question and relates to items 7, 8, & 11 of the Terminal Behaviors list regarding Biological Rhythms. 8. Biological functions which follow a cycle shorter than 24 hours are following what is known as a(n) ____________ cycle. a. infradian b. circadian c. ultradian d. imperian Question 8 is a RETENTION question and relates to items 7 12 of the Terminal Behaviors list regarding Biological Rhythms. 9. The branch of science that studies biological rhythms is called _________. a. chronobiology b. Temporal science c. There is no such science d. rhythmology Question 9 is a RETENTION question and relates to items 7 12 of the Terminal Behaviors list regarding Biological Rhythms. 111

PAGE 121

Appendix E (Continued) 10. A human in a free-running pattern will shift his internal clock 24 hours in ______ days of temporal isolation. a. 20 b. 18 c. 25 d. 24 Question 10 is a RETENTION question and relates to items 7, 11, 12, & 13 of the Terminal Behaviors list regarding Biological Rhythms. 11. Which of the following is not an example of a zeitgeber? a. sunrise b. car radio while driving to work c. telephone d. watch e. sunshine Question 11 is a TRANSFER question and relates to items 7, 11, 12, 13 & 14 of the Terminal Behaviors list regarding Biological Rhythms. 12. The hormone melatonin reaches peak levels in the body during the ________. a. night b. morning c. middle of the night d. mid-afternoon Question 12 is a RETENTION question and relates to items 7, 8, 11, 12, 14, & 15 of the Terminal Behaviors list regarding Biological Rhythms. 13. Which of the following would experience the most severe jet lag? a. Tom: He traveled by plane from the United States to Europe b. Marcia: She works the late shift at the local convenience store c. Eddie: He traveled by car from New York to Los Angeles. d. Allyson: She studied all night for an exam e. James: He traveled by plane from Miami to New England Question 13 is a TRANSFER question and relates to items 7, 8, 12, 13, & 16 of the Terminal Behaviors list regarding Biological Rhythms. 14. Shift work disrupts people's circadian rhythms and consequently causes which problem? a. hallucinations b. inability to restore the circadian rhythm c. disorientation as to time and place d. inability to tell night from day e. disrupted sleep-wake cycle Question 14 is a RETENTION question and relates to items 7, 11, 12, & 17 of the Terminal Behaviors list regarding Biological Rhythms. 112

PAGE 122

Appendix E (Continued) 15. The EOG is used to record and measure what behavior? a. Heart rate b. Brain wave activity c. Blood pressure d. Eye movements Question 15 is a RETENTION question and relates to item 19 of the Terminal Behaviors list regarding The Study of Sleep. 16. The EMG is a measurement of the electrical activity associated with _____________. a. muscle tension (movement) b. eye movements c. cardiac activity d. dreaming Question 16 is a RETENTION question and relates to item 20 of the Terminal Behaviors list regarding The Study of Sleep. 17. A machine that measures and records the various electrophysiological paramaters of sleep is called a(n): a. electroencephalograph b. polygraph c. electrocardiograph d. polysomnograph Question 17 is a RETENTION question and relates to items 18 21 of the Terminal Behaviors list regarding The Study of Sleep. 18. The International 10-20 system is a systematic process for a. locating electrodes on a subjects head for the recording of EEG. b. scoring and staging human sleep based on gender and age. c. determining the rate of oxygen consumption in sleeping humans of various ages. d. treating people with sleep disorders. Question 18 is a RETENTION question and relates to items 18 21 of the Terminal Behaviors list regarding The Study of Sleep. 19. Immediately before sleep the EEG shifts to patterns of larger and slower waves called ______ waves. a. alpha b. beta c. delta d. spindle Question 19 is a RETENTION question and relates to items 22 30 of the Terminal Behaviors list regarding Sleep Staging. 113

PAGE 123

Appendix E (Continued) 20. The four basic EEG patterns are a. alpha, beta, gamma, delta b. beta, theta, omega, gamma c. theta, delta, beta, omega d. alpha, beta, theta, delta Question 21 is a RETENTION question and relates to items 22 30 of the Terminal Behaviors list regarding Sleep Staging. Left EOG EOG Right EOG Chin EMG Central EEG Occipital EEG Alpha Rhythm 21. What stage does the figure above represent? a. Stage Wake b. Stage 1 c. Stage 2 d. Stage 3-4 e. Stage REM Question 21 is a TRANSFER question and relates to items 22 30 & 32 of the Terminal Behaviors list regarding Sleep Staging. 22. What stage of sleep does the average persons spend the LEAST time in? a. Stage 3-4 b. Stage 1 c. Stage 2 d. REM Question 22 is a RETENTION question and relates to items 22 30 & 33 of the Terminal Behaviors list regarding Sleep Staging. 114

PAGE 124

Appendix E (Continued) Left EOG Right EOG Chin EMG Central EEG C3 Occipital EEG O2 Rolling eye movements Vertex waves 23. What stage does the figure above represent? a. Stage Wake b. Stage 1 c. Stage 2 d. Stage 3-4 e. Stage REM Question 23 is a TRANSFER question and relates to items 22 30 & 33 of the Terminal Behaviors list regarding Sleep Staging. 24. What are the brainwave patterns that signify stage 2 sleep? a. Delta waves and alpha rhythm b. Beta waves and sleep spindles c. Sleep spindles and K-complexes d. K-complexes and delta waves Question 24 is a RETENTION question and relates to items 22 30 & 34 of the Terminal Behaviors list regarding Sleep Staging. Left EOG Right EOG Chin EMG Central EEG C3 Occipital EEG O2 K Complexes 115

PAGE 125

Appendix E (Continued) 25. What stage does the figure above represent? a. Stage Wake b. Stage 1 c. Stage 2 d. Stage 3-4 e. Stage REM Question 25 is a TRANSFER question and relates to items 22 30 & 34 of the Terminal Behaviors list regarding Sleep Staging. 26. The difference between stage 3 and 4 sleep is ________ a. the presence of alpha waves in stage 3, but not in stage 4. b. the proportion of delta waves present c. the presence of rapid eye movement occurs in stage 4, but not in stage 3. d. stage 4 is referred to as deep sleep, but not stage 3. Question 26 is a RETENTION question and relates to items 22 30, 35 & 36 of the Terminal Behaviors list regarding Sleep Staging. 27. A loud noise awakens you; you emerge from sleep confused and do not remember the noise. You were in which stage of sleep? a. stage 1 b. stage 2 c. stage 3 4 d. REM Question 27 is a TRANSFER question and relates to items 22 30, 35 & 36 of the Terminal Behaviors list regarding Sleep Staging. 28. Which sleep stage represents the deepest sleep? a. Stage 1 b. Stage 2 c. Stage 3-4 d. Stage REM Question 28 is a RETENTION question and relates to items 22 30, 35 & 36 of the Terminal Behaviors list regarding Sleep Staging. 116

PAGE 126

Appendix E (Continued) Left EOG Right EOG Chin EMG Central EEG C3 Occipital EEG O2 Delta Waves 29. What stage does the figure above represent? a. Stage Wake b. Stage 1 c. Stage 2 d. Stage 3-4 e. Stage REM Question 29 is a TRANSFER question and relates to items 22 30, 35, 36, 42 & 43 of the Terminal Behaviors list regarding Sleep Staging. 30. Lorraine has passed into REM sleep. If Lorraine were connected to an EEG, the recording would show the presence of : a. sleep spindles and K-complexes. b. gamma waves. c. low amplitude, irregular brain wave patterns. d. delta waves. Question 30 is a TRANSFER question and relates to items 22 30, 37, 38, 44 & 45 of the Terminal Behaviors list regarding Sleep Staging. 31. Dreaming occurs _________________. a. only in REM. b. in both REM and NREM. c. only in deep sleep. Question 31 is a RETENTION question and relates to items 22 30, 37, 38, 44 & 45 of the Terminal Behaviors list regarding Sleep Staging. 32. Which of the following is NOT typical of REM sleep? a. Full arousal of brain activity. b. Virtual paralysis of body musculature. c. Periods of REM sleep get shorter as the night continues. d. Rapid eye movements. Question 32 is a RETENTION question and relates to items 22 30, 37, 38, 44 & 45 of the Terminal Behaviors list regarding Sleep Staging. 117

PAGE 127

Appendix E (Continued) 33. REM sleep is distinguished from NREM in that a. the eyes move rapidly during REM sleep b. dreams are not present in NREM sleep c. the brain waves are less active in REM sleep d. sleep spindles appear in REM sleep but not in NREM sleep Question 33 is a RETENTION question and relates to items 22 30, 37, 38, 44 & 45 of the Terminal Behaviors list regarding Sleep Staging. 34. Most vivid dreaming occurs during _________ sleep. a. REM b. Stage 3-4 c. Stage 2 d. Stage 1 Question 34 is a RETENTION question and relates to items 22 30, 37, 38, 44 & 45 of the Terminal Behaviors list regarding Sleep Staging. 35. A night school student who falls asleep in class sleeps for about 25 minutes before his professor notices and tries to awaken him. At this point, he is in such a deep sleep that it is almost impossible to awaken him. An EEG would show his brain producing delta waves. His is probably in stage _______ sleep. a. REM b. 2 c. 4 d. 1 Question 35 is a TRANSFER question and relates to items 22 30, 35, 36, 42 & 43 of the Terminal Behaviors list regarding Sleep Staging. 36. Your friend, who considers herself to be a night person, is planning to study all night for an important exam that will be given the next morning at 9 a.m. What would be the main reason you would advise her to go to bed at about her normal bedtime? a. She might fall asleep while studying and oversleep the next morning. b. Sleep deprivation might hurt her performance on the exam, especially at 9:00 a.m. c. As a night person, she wil be unable to study effectively late at night. d. If she doesnt get enough sleep, she might fall asleep during the exam. e. Sleep deprivation will impair her ability to study. Question 36 is a TRANSFER question and relates to items 22 47 of the Terminal Behaviors list regarding Sleep Staging and Sleep Cycles. 118

PAGE 128

Appendix E (Continued) 37. Marie is drowsy and her eyes are closed. She has not yet begun stage 1 sleep, but she is extremely relaxed. If Marie were connected to an EEG, the recording would most likely show the presence of _______ _________. a. alpha waves. b. beta waves c. delta waves d. sleep spindles Question 37 is a RETENTION question and relates to items 22 33, 46 & 47 of the Terminal Behaviors list regarding Sleep Staging and Sleep Cycles. 38. Short bursts of brain activity in stage 2 sleep are called a. delta waves b. recurrent beta waves c. sleep spindles d. cortical waves Question 38 is a RETENTION question and relates to items 22 34, 40, 41, 46 & 47 of the Terminal Behaviors list regarding Sleep Staging and Sleep Cycles. 119

PAGE 129

Appendix E (Continued) 39. scribed as: The figure above is a polysomnogram of a patient diagnosed with a condition called confusional arousals. The unique combination of EEG waveforms that marks this condition could best be de a. an Alpha-Delta pattern. b. a Beta-Delta pattern. c. a Theta-Delta pattern. d. a REM sleep disorder. Occi p ital Central Chin EMG Right EOG Left EOG Delta W a v e s 10 Hz activity Question 39 is a TRANSFER question and relates to items 22 34, 40, 41, 46 & 47 of the Terminal Behaviors list regarding Sleep Staging and Sleep Cycles. 120

PAGE 130

Appendix E (Continued) 50 microvolts 1 second Alpha 813 Hz Beta > 13 Hz Theta 47 Hz Delta < 4 Hz Alpha 813 Hz 40. The illustration above shows the four frequency ranges of EEG. Which letter corresponds to a frequency range of 4 7 Hz? a. Alpha b. Beta c. Theta d. Delta Question 38 is a RETENTION question and relates to items 22 47 of the Terminal Behaviors list regarding Sleep Staging and Sleep Cycles. 121

PAGE 131

Appendix E (Continued) Occipital EEG O2 Central EEG C3 Chin EMG Right EOG Left EOG Occipital EEG O2 Central EEG C3 Right EOG Chin EMG Left EOG Occipital EEG O2 Central EEG C3 Chin EMG Right EOG Left EOG c. b. a. 41. REM Behavior Disorder (RBD) is a condition in which the neural mechanism that inhibits voluntary muscle movement during REM sleep fails. In this condition the patient can act out dreams. In RBD, we frequently see arm movements, kicking, and movement of the head and neck. Which of the following polysomnograms would lead you to suspect RBD? Question 41 is a TRANSFER question and relates to items 22 47 of the Terminal Behaviors list regarding Sleep Staging and Sleep Cycles. 122

PAGE 132

123 Appendix E (Continued) 42. A government study has revealed that t housands die each year in accidents resulting from drivers falling asleep. It has been suggested that a device be designed that will alert a driver when he/she is too sleepy to continue driving. Which of the following three de vices will prevent the most accidents by catching a sleepy driver earliest? a. A device that detects alpha rhythms. b. A device that detects K-complexes or sleep spindles. c. A device that detects muscle atonia (paralysis). Question 41 is a TRANSFER question and rela tes to items 22 47 of the Terminal Behaviors list regarding Sleep Staging and Sleep Cycles. 43. The observation of microsleep during sleep deprivation suggests that sleep: a. is important for emotional stability. b. can intrude into our waking state. c. is important for cognition. d. is important for intellectual performance. Question 41 is a TRANSFER question and rela tes to items 46 51 of the Terminal Behaviors list regarding Sleep Staging and Sleep Cycles.

PAGE 133

Appendix F Long-Term Assessment 1. The ability to access past experiences while processing external stimuli is part of a. consciousness b. long-term memory c. short-term memory d. sensory integration Question 1 is a RETENTION question and relates to items 1 6 of the Terminal Behaviors list regarding Consciousness. 2. William James equated the flow of consciousness to a. an electric current. b. a stream. c. a river. d. a highway system. Question 2 is a RETENTION question and relates to items 1 6 of the Terminal Behaviors list regarding Consciousness. 3. Like consciousness, sleep also has the quality of depth. The more deeply alseep one is, a. the more close to waking one is. b. the more disassociated from reality one is. c. the more vivid ones dreams are likely to be. d. longer one has been asleep. Question 3 is a RETENTION question and relates to items 1 6 of the Terminal Behaviors list regarding Consciousness. 4. Biological functions which follow a cycle of about 24 hours are following what is known as a. a solar cycle. b. a lunar cycle. c. an endocrine cycle. d. a circadian cycle. e. biorhythms. Question 4 is a RETENTION question and relates to items 7 12 of the Terminal Behaviors list regarding Biological Rhythms. 5. Sleep is distinguished from coma in that sleep is a. associated with dreaming while coma is not. b. easily reversible. c. not associated with head trauma and coma always is. d. only seen after a period of prolonged wakefulness, while coma occurs at any time. Question 5 is a RETENTION question and relates to items 1 6 of the Terminal Behaviors list regarding Consciousness. 124

PAGE 134

Appendix F (Continued) 6. This chemical, which is produced in the pineal gland, is thought to play an important role in regulating waking-sleeping cycles. a. triptophan b. testosterone c. melatonin d. norepinephrine Question 6 is a RETENTION question and relates to items 7, 8, 11, & 15 of the Terminal Behaviors list regarding Biological Rhythms. 7. Biological functions which follow a cycle longer than 24 hours are following what is known as a(n) ____________ cycle. a. infradian b. circadian c. ultradian d. imperian Question 7 is a RETENTION question and relates to items 7, 8, 9, & 11 of the Terminal Behaviors list regarding Biological Rhythms. 8. Many human ultradian rhythms run on an approximately _______ cycle. a. 90 minutes b. 24 hour c. 25 hour d. 15 minute Question 8 is a RETENTION question and relates to items 7, 8, 10 & 11 of the Terminal Behaviors list regarding Biological Rhythms. 9. Chronobiology is the branch of science that studies a. biological processes associated with aging. b. sleep cycle changes associated with the disease processes of the elderly. c. biological rhythms. d. sleep cycles in mammals. Question 9 is a RETENTION question and relates to items 7 17 of the Terminal Behaviors list regarding Biological Rhythms. 10. Temporal isolation studies, such as cave studies described in the text, resulted in the knowledge that the human circadian system is not perfectly synchronized to the 24 hour day. When isolated from temporal (time) cues, the human system tends to run on a _______ hour cycle. a. 20 b. 18 c. 25 d. 30 Question 10 is a RETENTION question and relates to items 7 17 of the Terminal Behaviors list regarding Biological Rhythms. 125

PAGE 135

Appendix F (Continued) 11. An environmental cue that signals the brain as to the correct time is called a(n) _________. a. temporal spike b. zeitgeber c. chronomarker d. temporal stimulus Question 11 is a RETENTION question and relates to items 7 17 of the Terminal Behaviors list regarding Biological Rhythms. 12. Jet lag is a. more pronounced when traveling from east to west. b. purely a function of distance traveled, the further you fly the more severe your jet lag. c. more pronounced when flying overnight. d. more pronounced when traveling from west to east. Question 12 is a TRANSFER question and relates to items 7, 8, 12, 13, & 16 of the Terminal Behaviors list regarding Biological Rhythms. 13. When a person is shut off from the world such that he or she cannot tell what time it is by light or dark, their sleep-waking cycle a. stays at about 24 hours. b. shortens to an average of about 20 hours. c. lengthens to an average of about 25 hours. d. becomes completely disrupted. Question 13 is a RETENTION question and relates to items 7 17 of the Terminal Behaviors list regarding Biological Rhythms. 14. Shift work will have the greatest negative effect on what shift? a. 1 st shift. b. 2 nd shift. c. 3 rd shift. d. Both 2 nd and 3 rd shifts will show equal negative effects. e. All three shifts have equal necative effects due to rapid shift changing. Question 143 is a RETENTION question and relates to items 7 17 of the Terminal Behaviors list regarding Biological Rhythms. 15. The EOG measures and records a. cardiac activity. b. brain wave activity associated with sleep. c. the electrical changes in muscle activity. d. eye movements. Question 15 is a RETENTION question and relates to item 19 of the Terminal Behaviors list regarding The Study of Sleep. 126

PAGE 136

Appendix F (Continued) 16. What do we call the recording of the electrical activity associated with muscle tonus (tension and movement)? a. An EMG b. An EOG c. An EEG d. An EMF Question 16 is a RETENTION question and relates to item 20 of the Terminal Behaviors list regarding The Study of Sleep. 17. The device used to record the electrophysiological activity of sleep is called a(n): a. electroencephalograph b. polygraph c. polysomnograph d. electrocardiograph Question 17 is a RETENTION question and relates to items 18 21 of the Terminal Behaviors list regarding The Study of Sleep. 18. Because everyones head is slightly different in shape and size a system for consistently locating electrode locations for recording EEG was needed. This system is called the: a. International 20-20 system. b. International 10-20 system c. American standard system of electrode placement (ASSEP). d. National Standard for Electrode Placement (NSEP). Question 18 is a RETENTION question and relates to items 18 21 of the Terminal Behaviors list regarding The Study of Sleep. 19. What frequency band of brain waves is produced during relaxed wakefullness or the twilight stage between waking and sleep? a. gamma b. beta c. delta d. alpha e. theta Question 19 is a RETENTION question and relates to items 22 30 & 32 of the Terminal Behaviors list regarding Sleep Staging. 20. From slowest to fastest, the four basic EEG frequency bands are: a. alpha, theta, delta, beta. b. delta, theta, alpha, beta c. alpha, beta, delta, theta. d. alpha, beta, delta, gamma. e. alpha, delta, gamma, theta. Question 20 is a RETENTION question and relates to items 22 30 of the Terminal Behaviors list regarding Sleep Staging. 127

PAGE 137

Appendix F (Continued) Left EOG Right EOG Chin EMG Central EEG C3 Occipital EEG O2 Alpha Rhythm 21. The red box in this stage wake recording highlights a. beta activity. b. sleep spindling. c. alpha activity. d. delta activity. Question 21 is a TRANSFER question and relates to items 22 30 & 32 of the Terminal Behaviors list regarding Sleep Staging. 22. Stage 1 sleep lasts for __________. a. only a few minutes. b. 45 minutes c. 90 minutes d. about 3 hours Question 22 is a RETENTION question and relates to items 22 30 & 33 of the Terminal Behaviors list regarding Sleep Staging. 128

PAGE 138

Appendix F (Continued) Left EOG Right EOG Chin EMG Central EEG C3 Occipital EEG O2 Rolling eye movements Vertex waves 23. We know that the figure above is a recording of Stage 1 sleep because a. it contains vertex waves. b. the presence of slow rolling eye movements c. the Central EEG has a background of low voltage, mixed frequency activity. d. Answers a and c. e. Answers a, b, and c. Question 23 is a TRANSFER question and relates to items 22 30 & 33 of the Terminal Behaviors list regarding Sleep Staging. 24. Sleep spindles are difinitive markers of which stage of sleep? a. Stage 1 b. Stage 2 c. Stage 3-4 d. REM Question 24 is a RETENTION question and relates to items 22 30 & 34 of the Terminal Behaviors list regarding Sleep Staging. 129

PAGE 139

Appendix F (Continued) Left EOG Right EOG Chin EMG Central EEG C3 Occipital EEG O2 K Complexes 25. The presence of K-complexes tells us this recording is an example of what stage of sleep? a. Stage Wake b. Stage 1 c. Stage 2 d. Stage 3-4 e. Stage REM Question 25 is a TRANSFER question and relates to items 22 30 & 34 of the Terminal Behaviors list regarding Sleep Staging. 26. The earliest stage of sleep to have long, slow, high-amplitude brain waves and in which it is difficult to wake the sleeper is ____________ sleep. a. stage 1 b. stage 2 c. stage 3 d. stage 4 e. REM Question 26 is a RETENTION question and relates to items 22 30, 35 & 36 of the Terminal Behaviors list regarding Sleep Staging. 27. Stages 3 and 4 sleep are associated with ______ frequency waves in EEG recordings a. beta b. theta c. delta d. alpha Question 27 is a RETENTION question and relates to items 22 30, 35 & 36 of the Terminal Behaviors list regarding Sleep Staging. 130

PAGE 140

Appendix F (Continued) Left EOG Right EOG Chin EMG Central EEG C3 Occipital EEG O2 Delta Waves 28. The marker waveforms in the Stage 3 recording are a. alpha waves b. Delta waves c. Sleep Spindles d. Vertex waves Question 28 is a TRANSFER question and relates to items 22 30, 35, 36, 42 & 43 of the Terminal Behaviors list regarding Sleep Staging. 29. The difference between stage 3 and 4 sleep is ________ a. the presence of alpha waves in stage 3, but not in stage 4. b. the proportion of delta waves present c. the presence of rapid eye movement occurs in stage 4, but not in stage 3. d. stage 4 is referred to as deep sleep, but not stage 3. Question 29 is a RETENTION question and relates to items 22 30, 35 & 36 of the Terminal Behaviors list regarding Sleep Staging. 131

PAGE 141

Appendix F (Continued) Left EOG Right EOG Chin EMG Central EEG C3 Occipital EEG O2 Marker waveforms Marker Waveforms 30. The figure above demonstrates Stage 4 sleep as opposed to Stage 3 sleep because a. the marker waveforms have higher amplitudes in Stage 4 than Stage 3. b. the waves identified as marker waves are really K-complexes. c. more than 50% of the segment is made up of marker waves. d. Stage 3 sleep does not contain these marker waves. Question 30 is a TRANSFER question and relates to items 22 30, 35, 36, 42 & 43 of the Terminal Behaviors list regarding Sleep Staging. 31. For many years it was thought that dreaming only occurs in REM sleep. We now know that people experience dreams in a. all stages of sleep, but the most vivid dreams occur only in REM and deep sleep. b. all stages of sleep with the exception of stage 1. c. all stages of sleep, but the most vivid dreams occur only in REM sleep. d. REM and deep sleep. Question 31 is a RETENTION question and relates to items 22 30, 37, 38, 44 & 45 of the Terminal Behaviors list regarding Sleep Staging. 32. Which of the following is NOT typical of REM sleep? a. Full arousal of brain activity. b. Virtual paralysis of body musculature. c. Periods of REM sleep get shorter as the night continues. d. Rapid eye movements. Question 32 is a RETENTION question and relates to items 22 30, 37, 38, 44 & 45 of the Terminal Behaviors list regarding Sleep Staging. 132

PAGE 142

Appendix F (Continued) 33. A dramatic feature of this stage of sleep is total paralysis of the voluntary muscles (atonia). a. Stage 1 b. Stage 2 c. Stage 3-4 d. REM Question 33 is a RETENTION question and relates to items 22 30, 37, 38, 44 & 45 of the Terminal Behaviors list regarding Sleep Staging. 34. What stage of sleep can be accurately identified without the use of monitoring technology? a. Stage 1 sleep b. Stage 2 sleep c. Stage 3 sleep d. Stage 4 sleep e. Stage REM sleep Question 34 is a TRANSFER question and relates to items 22 30, 37, 38, 44 & 45 of the Terminal Behaviors list regarding Sleep Staging. Left EOG Rapid eye movements Occipital EEG O2 Central EEG C3 Chin EMG Right EOG 35. The recording above shows someone in REM sleep. This recording differs from an awake person rapidly moving their eyes because a. the chin EMG would not be so low in stage wake. b. it is impossible to voluntarily move the eyes that rapidly. c. the central EEG channel shows sleep spindles. d. Both a and b. e. Both b and c. Question 35 is a TRANSFER question and relates to items 22 30, 37, 38, 44 & 45 of the Terminal Behaviors list regarding Sleep Staging. 133

PAGE 143

Appendix F (Continued) 36. Studies indicate that partial sleep deprivation begins to cause impairment in a. attention. b. reaction time. c. decision making. d. a and b. e. a, b, and c. Question 36 is a RETENTION question and relates to items 46 51 of the Terminal Behaviors list regarding Sleep Staging and Sleep Cycles. 37. Martin has been selectively deprived of slow-wave sleep for five nights. You would expect that on the sixth, undisturbed night, he will show a. increased stage 4 sleep b. increased REM sleep c. decreased stage 4 sleep d. decreased REM sleep Question 37 is a RETENTION question and relates to items 46 52 of the Terminal Behaviors list regarding Sleep Cycles and Sleep Debt. 38. A woman celebrating her 50 th birthday complains that she is a much lighter sleeper than she used to be. This is probably because: a. REM sleep lessens with age. b. stage 4 sleep lessens with age. c. stage 2 sleep lessens with age. d. people become more sensitive to distractions that wake them up at night as they age. Question 38 is a TRANSFER question and relates to items 46 52 of the Terminal Behaviors list regarding Sleep Cycles and Sleep Debt. 134

PAGE 144

Appendix F (Continued) 39. 10 Hz activity Delta W a ves The figure above is a polysomnogram of a patient diagnosed with sleep walking demonstrates an Alpha-Delta EEG pattern. A patient with this unique combination of waking and deep sleep waveforms could also be diagnosed as having a. an arousal disorder. b. insomnia. c. jet lag. d. a REM sleep disorder. Left EOG Right EOG Chin EMG Central Occi p ital 10 Hz activity Delta Waves Question 39 is a TRANSFER question and relates to items 22 34, 40, 41, 46 & 47 of the Terminal Behaviors list regarding Sleep Staging and Sleep Cycles. 135

PAGE 145

Appendix F (Continued) Occipital EEG O2 Central EEG C3 Chin EMG Right EOG Left EOG Occipital EEG O2 Central EEG C3 Right EOG Chin EMG Left EOG Occipital EEG O2 Central EEG C3 Chin EMG Right EOG Left EOG c. b. a. 40. REM Behavior Disorder (RBD) is a condition in which the neural mechanism that inhibits voluntary muscle movement during REM sleep fails. In this condition the patient can act out dreams. In RBD, we frequently see arm movements, kicking, and movement of the head and neck. Which of the following polysomnograms would lead you to suspect RBD? Question 40 is a TRANSFER question and relates to items 22 47 of the Terminal Behaviors list regarding Sleep Staging and Sleep Cycles. 136

PAGE 146

137 Appendix F (Continued) 41. A government study has revealed that t housands die each year in accidents resulting from drivers falling asleep. It has been suggested that a device be designed that will alert a driver when he/she is too sleepy to continue driving. Which of the following three de vices will prevent the most accidents by catching a sleepy driver earliest? a. A device that detects alpha rhythms. b. A device that detects K-complexes or sleep spindles. c. A device that detects muscle atonia (paralysis). Question 41 is a TRANSFER question and rela tes to items 22 47 of the Terminal Behaviors list regarding Sleep Staging and Sleep Cycles. 42. A night school student who falls asleep in class sleeps for about 25 minutes before his professor notices and tries to awaken him. At this point, he is in such a deep sleep that it is almost impossible to awaken him. An EEG would show his brain producing delta waves. His is probably in stage _______ sleep. a. REM b. 2 c. 4 d. 1 Question 42 is a TRANSFER question and relate s to items 22 30 & 34 of the Terminal Behaviors list rega rding Sleep Staging. 43. Millie falls asleep after a long day at work and a night of college classes. About an hour after she falls asleep, her muscles become very relaxed and her breathing and heart rate become irregular. She is MOST likely entering a. Deep sleep b. REM sleep c. Stage 2 sleep d. None of these Question 43 is a TRANSFER question and rela tes to items 22 47 of the Terminal Behaviors list regarding Sleep Staging and Sleep Cycles.

PAGE 147

Appendix G Five Page TXT Extract 138

PAGE 148

Appendix G (Continued) 139

PAGE 149

Appendix G (Continued) 140

PAGE 150

Appendix H Five Frame LPI Extract 141

PAGE 151

Appendix H (Continued) 142

PAGE 152

Appendix H (Continued) 143

PAGE 153

Appendix I Five Frame IPI Extract 143

PAGE 154

Appendix I (Continued) 144

PAGE 155

Appendix I (Continued) 145

PAGE 156

Appendix J Six Frame CLT-PI Extract This segment contains auditory narration. Frame 15 is from a remedial sequence. The learner has responded incorrectly and is redirected to a multimedia sequence explaining the concept again. The learner is then tested and once more enters the normal sequence of instruction. 147

PAGE 157

Appendix J (Continued) 148

PAGE 158

Appendix J (Continued) 149

PAGE 159

Appendix K Instructional Programs in Windows Executable Format The enclosed compact disk (CD) contains five Window executable (.exe) files. These files are the four programs used in this study plus a Summary program that presents the first 10 frames of each program sequentially. This CD also contains a Read me file with the following information. This CD contains five Windows executable files. These files contain the programs used in my dissertation project. These are Windows executable (.exe) files and should run from your CD drive. You may wish to view the Samples program (#5 below) first. You may need to install this Macromedia file to run these programs. The five programs are: 1. TXT this is the program used in the TXT condition. 2. LPI 2 this is the program used in the LPI condition. 3. IPI this is the program used in the IPI condition. 4. CLT-PIA this is the program used in the CLT-PI condition. This program contains auditory narration. 5. Samples this program contains the first 10 frames of each program presented sequentially. 150

PAGE 160

About the Author Michael Brazas returned to school in his late 30s majoring in Psychology. He graduated Magna Cum Laude from the Universi ty of South Florida in 1994. He entered the Cognitive and Neural Sciences program in the Psychology Department at the University of South Florida in 1994 where he taught a number of courses concentrating on the psychology of learning and experimental methodology.


xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam Ka
controlfield tag 001 001681103
003 fts
005 20060215071238.0
006 m||||e|||d||||||||
007 cr mnu|||uuuuu
008 060104s2005 flu sbm s000 0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0001011
035
(OCoLC)62774284
SFE0001011
040
FHM
c FHM
049
FHMM
090
BF121 (Online)
1 100
Brazas, Michael L.
0 245
Cognitive load theory and programmed instruction
h [electronic resource] /
by Michael L. Brazas.
260
[Tampa, Fla.] :
b University of South Florida,
2005.
502
Thesis (Ph.D.)--University of South Florida, 2005.
504
Includes bibliographical references.
516
Text (Electronic thesis) in PDF format.
538
System requirements: World Wide Web browser and PDF reader.
Mode of access: World Wide Web.
500
Title from PDF of title page.
Document formatted into pages; contains 160 pages.
Includes vita.
520
ABSTRACT: Programmed Instruction was a dominant force in educational research during the 1960's and early 1970's. During this period a new cognitively oriented psychology of education arose that eventually replaced the behaviorally driven research into programmed instruction. For some reason little work was done on integrating these two approaches. This study was designed to test a programmed instructional system based on the intrinsic programmed instructional technology of Crowder (1961) but incorporating design elements derived from cognitive load theory (Sweller, 1988). Specifically, this study tested a multimedia programmed instructional system (CLT-PI) based on work by Mayer (2001) against a text based system (TXT), a traditional linear programmed instructional system based on Skinner (1954, 1958)(LPI), and an intrinsic programmed instructional system based on the work of Crowder (1960, 1961)(IPI).Three hypotheses were tested; 1) Programmed instruction would produce higher long-term retention and transfer scores than TXT, 2) IPI would produced higher long-term retention and transfer scores than LPI, and 3) CLT-PI would produce higher long-term retention and transfer scores than LPI or IPI. 115 undergraduate university students were randomly assigned to one of four conditions where they studied a chapter on human sleep. Each condition was presented on a computer with a test following study. A long-term test was given 4 weeks later. Both short- and long-term tests contained retention and transfer questions. Analyses were conducted using repeated measures MANOVA. A series of ANOVA tests were conducted to determine specific effects and interactions. The first hypothesis was partly supported in that CLT-PI and IPI produced higher long-term retention scores than TXT. LPI, however, did not.
590
Adviser: Toru Shimizu.
Co-adviser: Maria dePerczel Goodwin
653
Working memory.
Learning theory.
Linear programmed instruction.
Intrinsic programmed instruction.
690
Dissertations, Academic
z USF
x Psychology
Doctoral.
773
t USF Electronic Theses and Dissertations.
4 856
u http://digital.lib.usf.edu/?e14.1011