USF Libraries
USF Digital Collections

The effects of multimedia annotations on L2 vocabulary immediate recall and reading comprehension

MISSING IMAGE

Material Information

Title:
The effects of multimedia annotations on L2 vocabulary immediate recall and reading comprehension a comparative study of text-picture and audio-picture annotations under incidental and intentional learning conditions
Physical Description:
Book
Language:
English
Creator:
Chen, Zhaohui
Publisher:
University of South Florida
Place of Publication:
Tampa, Fla
Publication Date:

Subjects

Subjects / Keywords:
Cognitive
Modality
Attention
Look-up
Noticing
Task type
ESL
Dual-coding
Dissertations, Academic -- Secondary Education -- Doctoral -- USF   ( lcsh )
Genre:
bibliography   ( marcgt )
theses   ( marcgt )
non-fiction   ( marcgt )

Notes

Abstract:
ABSTRACT: This dissertation investigated the effects of multimedia annotation on L2 vocabulary learning and reading comprehension. The overarching objective of this study was to compare the effects of text-picture annotation and audio-picture annotation on L2 vocabulary immediate recall and reading comprehension. This study also sought to examine the different effects under incidental and intentional learning conditions. The participants were 78 intermediate adult ESL learners from three universities in northwest U.S. The participants read an Internet-based English text. Twenty target words, annotated in either text-picture or audio-picture, were embedded in the reading text. The participants accessed the annotations by clicking on the highlighted target words. Two instruments were used for measuring vocabulary immediate recall: Vocabulary Knowledge Scale and Word Recognition Test.^ Two measurements were used to assess reading comprehension: Multiple-choice Reading Comprehension Questions and L1 Written Recall. In term of annotation types, the results indicated that the audio-picture annotation group did significantly better than the text-picture group in L2 vocabulary immediate recall. However, there was no significantly different effect between the two annotations on L2 reading comprehension. In terms of learning conditions, the intentional learning condition resulted in significantly better performance in L2 vocabulary immediate recall than the incidental learning condition. However, the incidental learning condition resulted in significantly better L2 reading comprehension than the intentional learning condition only in the Written Recall measure, but not in the multiple-choice Reading Comprehension Test.^ ^In terms of interaction between annotation type and learning condition, there was not interaction between annotation type and learning condition on L2 vocabulary immediate recall. The interaction between annotation type and learning condition on L2 reading comprehension was not significant in multiple-choice Reading Comprehension Text. However, the interaction was found to be significant in Written Recall: in the incidental learning condition, the difference between text-picture annotation and audio-picture annotation was not significant; in the intentional learning condition, participants in text-picture did significantly better than those in audio-picture on Written Recall.
Thesis:
Dissertation (Ph.D.)--University of South Florida, 2006.
Bibliography:
Includes bibliographical references.
System Details:
System requirements: World Wide Web browser and PDF reader.
System Details:
Mode of access: World Wide Web.
Statement of Responsibility:
by Zhaohu Chen.
General Note:
Title from PDF of title page.
General Note:
Document formatted into pages; contains 178 pages.
General Note:
Includes vita.

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
aleph - 001931111
oclc - 213499646
usfldc doi - E14-SFE0001727
usfldc handle - e14.1727
System ID:
SFS0026045:00001


This item is only available as the following downloads:


Full Text
xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam Ka
controlfield tag 001 001931111
003 fts
005 20080321135221.0
006 m||||e|||d||||||||
007 cr mnu|||uuuuu
008 080321s2006 flu sbm 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0001727
035
(OCoLC)213499646
040
FHM
c FHM
049
FHMM
090
LB1601 (ONLINE)
1 100
Chen, Zhaohui.
4 245
The effects of multimedia annotations on L2 vocabulary immediate recall and reading comprehension :
b a comparative study of text-picture and audio-picture annotations under incidental and intentional learning conditions
h [electronic resource] /
by Zhaohu Chen.
260
[Tampa, Fla] :
University of South Florida,
2006.
3 520
ABSTRACT: This dissertation investigated the effects of multimedia annotation on L2 vocabulary learning and reading comprehension. The overarching objective of this study was to compare the effects of text-picture annotation and audio-picture annotation on L2 vocabulary immediate recall and reading comprehension. This study also sought to examine the different effects under incidental and intentional learning conditions. The participants were 78 intermediate adult ESL learners from three universities in northwest U.S. The participants read an Internet-based English text. Twenty target words, annotated in either text-picture or audio-picture, were embedded in the reading text. The participants accessed the annotations by clicking on the highlighted target words. Two instruments were used for measuring vocabulary immediate recall: Vocabulary Knowledge Scale and Word Recognition Test.^ Two measurements were used to assess reading comprehension: Multiple-choice Reading Comprehension Questions and L1 Written Recall. In term of annotation types, the results indicated that the audio-picture annotation group did significantly better than the text-picture group in L2 vocabulary immediate recall. However, there was no significantly different effect between the two annotations on L2 reading comprehension. In terms of learning conditions, the intentional learning condition resulted in significantly better performance in L2 vocabulary immediate recall than the incidental learning condition. However, the incidental learning condition resulted in significantly better L2 reading comprehension than the intentional learning condition only in the Written Recall measure, but not in the multiple-choice Reading Comprehension Test.^ ^In terms of interaction between annotation type and learning condition, there was not interaction between annotation type and learning condition on L2 vocabulary immediate recall. The interaction between annotation type and learning condition on L2 reading comprehension was not significant in multiple-choice Reading Comprehension Text. However, the interaction was found to be significant in Written Recall: in the incidental learning condition, the difference between text-picture annotation and audio-picture annotation was not significant; in the intentional learning condition, participants in text-picture did significantly better than those in audio-picture on Written Recall.
502
Dissertation (Ph.D.)--University of South Florida, 2006.
504
Includes bibliographical references.
516
Text (Electronic dissertation) in PDF format.
538
System requirements: World Wide Web browser and PDF reader.
Mode of access: World Wide Web.
500
Title from PDF of title page.
Document formatted into pages; contains 178 pages.
Includes vita.
590
Adviser: Tony Erben, Ph.D.
653
Cognitive.
Modality.
Attention.
Look-up.
Noticing.
Task type.
ESL.
Dual-coding.
0 690
Dissertations, Academic
z USF
x Secondary Education
Doctoral.
773
t USF Electronic Theses and Dissertations.
856
u http://digital.lib.usf.edu/?e14.1727



PAGE 1

The Effects of Multimedia Annotations on L2 Vocabulary Immediate Recall and Reading Comprehension: A Comparative Study of Te xt-Picture and Audio-Picture Annotations under Incidental and Inte ntional Learning Conditions By Zhaohui Chen A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy Department of Secondary Education College of Education and Department of World Language Education College of Arts and Sciences University of South Florida Major Professor: Tony Erben, Ph.D. Jeffra Flaitz, Ph.D. William Kealy, Ph.D. John Ferron, Ph.D. Date of Approval: September 22, 2006 Keywords: cognitive, modality, attention, l ook-up, noticing, task type, ESL, dual-coding Copyright 2006, Zhaohui Chen

PAGE 2

Dedication In loving memory of my father

PAGE 3

Acknowledgements I will always be grateful to my major pr ofessor, Dr. Tony Erben, for inspiring me and encouraging me to take up this project. Thank you for the many hours you devoted to discussing my progress, reading and res ponding to my numerous drafts, and giving me your insightful suggestions. Your support a nd guidance have made my academic journey at the University of South Florida an enjoyable and valuable one. I was fortunate to have a committee of scholars who helped me to become a better researcher. Thank you to my ot her committee members: Dr. Jeffr a Flaitz for her belief in me, for her understanding and encouragemen t; Dr. John Ferron for sharing his wisdom and providing me with invaluable feedback on statistical analysis; Dr. Bill Kealy for his sharp wit and expertise in multimedia learning. I am thankful to the other faculty me mbers in Second Language Acquisition and Instructional Technology program, particularly Dr. Carine Feyten, Dr. Joyce Nutta, Dr. Wei Zhu, Dr. Linda Evans and Dr. Jim White. I have appreciated and benefited from their instruction and guidance both aca demically and professionally. I am also grateful to a number of coll eagues and friends who shared their insights and gave me encouragements, especially wh en I was finishing my dissertation from a distance. Special thanks to Dr. Phil Sm ith for being generous, considerate and understanding; thanks to Aline Harrison, Ru i Cheng, Annmarie Zoran and Iona Sarieva for their helpfulness, support and warm hugs; Thanks to Darunee Dujsik and Nader

PAGE 4

Morkus for helping me with the date processi ng; thanks also to Ma rtha Castaneda, Ruth Ban, Robert Summers, Michelle Macy, J eannie Ducher, Farah Khorsandian-Sanchez and Li Jin for being supportive; and thanks to all the other SLAIT st udents and colleagues who have helped me one way or another. My deepest appreciation al so goes to the directors a nd instructors at the three English Language Institutes who have ope ned their classrooms to me and helped generously with the project. Among them are Christa Hansen, Cindy Kieffer, Jodi Webber, Ronda Sheffield, Lee Pettigrew, Norman Johnson. Sandy Greenstreet. I would also like to thank the students from the th ree English Language Inst itutes for participating in the project. Without them, this project would not have been possible. Finally I would like to thank my family members in China for their support. My heartfelt appreciation is to my wife, Yifang Zhang, for her unconditional support and love.

PAGE 5

i Table of Contents List of Tables List of Figures List of Acronyms Abstract Chapter I: Introduction Statement of the Problem Theoretical Framework Purpose of the Study Research Questions Hypotheses Significance of the Study Definitions of Terms Limitations and Delimitations Organization of the study Chapter II: Review of the Literature L2 Vocabulary Learning Incidental and In tentional Learning Incidental Vocabulary Learning Annotation Traditional Annotation Multimedia Annotation Text Annotation Text Annotation in Printed Reading Texts Text Annotation and L2 Vocabulary Learning Text Annotation a nd L2 Reading Comprehension Text Annotation in Multimedia Texts Text Annotation and L2 Vocabulary Learning Text Annotation a nd L2 Reading Comprehension Summary Picture Annotation Annotation and L2 Vocabulary Learning Annotation and L2 Reading Comprehension Summary Video Annotation v vii ix x 1 4 4 6 6 7 8 9 11 13 14 15 16 18 20 21 24 25 25 25 29 31 31 33 34 35 36 42 43 44

PAGE 6

ii Annotation and L2 Vocabulary Learning Annotation and L2 Reading Comprehension Summary Audio Annotation Annotation Studies Summary Cognitive Theory of Multimedia Learning Generative Theory of Learning Dual-coding Theory Cognitive Theory of Multimedia Learning Extension of Cognitive Theory of Multimedia Learning Application of Cognitive Theory of Multimedia Learning to Multimedia Annotation Chapter Summary Chapter III: Methods and Procedures Design of the Study Variables Dependent Variables Independent Variables Types of Annotation Types of Learning Other Independent Variables Controlled Participants and Setting Instruments Questionnaire Reading Material and Target Words Program Posttests Vocabulary Knowledge Scale Word Recognition Test Multiple-Choice Reading Comprehension Tests Written Recall Validity Issue of the Posttests Scoring Vocabulary Knowledge Scale Word Recognition Test Multiple-Choice Reading Comprehension Test Written Recall Procedures Data Analysis Chapter IV: Results Pilot Study Participants Procedures Results 44 50 51 52 56 57 58 59 60 62 64 67 70 71 74 74 74 74 75 75 76 79 79 80 81 83 83 84 84 85 85 86 86 87 87 87 87 89 91 91 92 92 92

PAGE 7

iii Discussion Annotation Look-up Behavior Results Posttest Results Vocabulary Knowledge Scale Scores Word Recognition Test Scores Multiple-Choice Reading Comprehension Test Scores Written Recall Scores Results by Research Questions Questi on 1: Effects of Annotation Type on Vocabulary Immediate Recall Questi on 2: Effects of Annota tion Type on Reading Comprehension Question 3: Effects of Learning Condition on Vocabulary Immediate Recall Question 4: Effects of Learning Condition on Reading Comprehension Question 5: Interacti on between Annotation and Learning on Vocabulary Immediate Recall Question 6: Interac tion between Annotatio n and Learning on Reading Comprehension Summary of Findings Chapter V: Discussion Interpretations of Results Effect s of Annotation Type on L2 Vocabulary Immediate Recall Effect s of Annotation Type on L2 Reading Comprehension Effect s of Learning Condition on L2 Vocabulary Immediate Recall Effects of Learning Condition on L2 Reading Comprehension Inter action between Annotation T ype and Learning Condition Theoretical Implications Pedagogical Implications Directions for Future Research Conclusions References Appendices Appendix A: Questionnaire Appendix B: Reading Text Appendix C: Target Words Appendix D: Pictures for Target Words Appendix E: Screens hot of Reading Passage with Text-picture Annotation 95 97 99 100 103 107 111 115 115 118 119 121 123 124 125 129 129 129 132 133 135 137 139 140 141 142 144 159 160 161 162 163 167

PAGE 8

iv Appendix F: Screens hot of Reading Passage with Audio-picture Annotation Appendix G: Vocabulary Knowledge Scale Appendix H: Word Recognition Test Appendix I: Mul tiple-choice Reading Comprehension Test Appendix J: Written Recall About the Author 168 169 173 176 178 End page

PAGE 9

v List of Tables Table 3-1 2 x 2 Factorial Design Table 3-2 Test Scores a nd Matching Levels at AEI Table 3-3 Countries and Langua ges Represented by the Participants Table 3-4 Adapted VKS scoring categories and meaning of scores. Table 4-1 Descriptive Statisti cs for Tests of the Pilot Study Table 4-2 Means and Standard Deviat ion for the Tests of the Pilot Study by Annotation Type Table 4-3 Means and Standa rd Deviation for the Tests of the Pilot Study by Learning Condition Table 4-4 ANOVA results of the Pilot Study Table 4-5 Descriptive Sta tistics for Look-up Behavior Table 4-6 Means and Standard Deviations for Look-up Behavior Table 4-7 ANOVA fo r Look-up Behavior Table 4-8 Correlation between Lookup Behavior and Posttest Measures Table 4-9 Descriptive Statistics for Vocabulary Knowledge Scale by Annotation and Learning Table 4-10 Means and Standard Deviat ions for Vocabulary Knowledge Scale Table 4-11 ANOVA for Voca bulary Knowledge Scale Table 4-12 Descriptive Statistics for Word Recognition Test by Annotation and 71 77 78 84 93 94 94 94 97 98 98 99 101 101 102 104

PAGE 10

vi Learning Table 4-13 Means and Standard Devi ations for Word Recognition Test Table 4-14 ANOVA for Wo rd Recognition Test Table 4-15 Descriptive Statistics for Reading Comprehension Test by Annotation and Learning Table 4-16 Means and Standard Deviati ons for Reading Comprehension Test Table 4-17 ANOVA for Read ing Comprehension Test Table 4-18 Descriptive Statistics fo r Written Recall by Annotation and Learning Table 4-19 Means and Standard Deviations for Written Recall Table 4-20 ANOVA for Written Recall Table 4-21 Means and Standard Devi ations for Vocabulary Posttests by Annotation Table 4-22 Means and Standard Deviat ions for Reading Comprehension by Annotation Table 4-23 Means and Standard Devi ations for Vocabulary Posttests by Learning Table 4-24 Means and Standard Deviat ions for Reading Comprehension by Learning Table 4-25 A Summary of Interaction between Annotation and Learning on Vocabulary Recall Table 4-26 A Summary of Interaction between Annotation and Learning on Reading Comprehension Table 427 A Summary of the Findings in the Study 105 107 108 108 110 112 112 114 116 117 120 122 124 125 127

PAGE 11

vii List of Figures Figure 2-1 Cognitive Theory of Multimedia Learning Figure 2-2 Processing of Text Annotation Figure 2-3 Processing of Audio Annotati on Figure 2-4 Processing of Pictur e Annotation Figure 3-1 Detailed Research Design Figure 4-1. Results of Pilot Study Figure 4-2. Means for the Vocabulary Knowledge Scale by Annotation Types Figure 4-3. Means for the Vocabulary K nowledge Scale by Learning Conditions Figure 4-4. VKS Scores by Annot ation and Learning Condition Figure 4-5. Means for the Word R ecognition Test by Annotation Types Figure 4-6. Means for the Word Recognition Test by Learning Conditions Figure 4-7. WRT scores by Annotat ion Type and Learning Condition Figure 4-8. Means for the Reading Co mprehension Test by Annotation Types Figure 4-9. Means for the Reading Comp rehension Test by Learning Conditions Figure 4-10. RC Scores by Annotation Type and Learning Condition Figure 4-11. Means for Written Recall by Annotation Type Figure 4-12. Means for Written Recall by Learning Condition Figure 4-13. Written Recall Scores by Annotation Type and Learning Condition Figure 4-14. Means for Vocabular y Posttests by Annotation Type 63 65 66 66 72 96 102 103 103 105 106 106 109 109 110 113 113 115 117

PAGE 12

viii Figure 4-15. Means for Reading Co mprehension by Annotation Type Figure 4-16. Means for Vocabular y Posttest by Learning Condition Figure 4-17. Means for Reading Comprehension by Learning Condition 118 121 123

PAGE 13

ix List of Acronyms L1: First Language L2: Second Language L3: Third Language CALL: Computer-Assis ted Language Learning EFL: English as a Foreign Language ESL: English as a Second Language NS: Native Speaker NNS: Non-native Speaker SLA: Second Language Acquisition TL: Target Language

PAGE 14

x The Effects of Multimedia Annotations on L2 Vocabulary Immediate Recall and Reading Comprehension: A Comparative Study of Te xt-picture and Audio-picture Annotations under Incidental and Inte ntional Learning Conditions Zhaohui Chen ABSTRACT This dissertation investigated the e ffects of multimedia annotation on L2 vocabulary learning and reading comprehensi on. The overarching objec tive of this study was to compare the effects of text-picture annotation and audio-pi cture annotation on L2 vocabulary immediate recall and reading comp rehension. This study also sought to examine the different effects under incident al and intentional learning conditions. The participants were 78 intermediate adult ESL learners from three universities in northwest U.S. The participants read an Internet-based English text. Twenty target words, annotated in either te xt-picture or audio-picture, were embedded in the reading text. The participants accesse d the annotations by clicki ng on the highlighted target words. Two instruments were used fo r measuring vocabulary immediate recall: Vocabulary Knowledge Scale and Word Recogni tion Test. Two measurements were used to assess reading comprehension: Multiple-choice Reading Comprehension Questions and L1 Written Recall. In term of annotation types, the results indicated that the a udio-picture annotation group did significantly better than the text-p icture group in L2 vocabulary immediate recall. However, there was no significantly di fferent effect between the two annotations

PAGE 15

xi on L2 reading comprehension. In terms of l earning conditions, the intentional learning condition resulted in signifi cantly better performance in L2 vocabulary immediate recall than the incidental learning condition. Howeve r, the incidental learning condition resulted in significantly better L2 read ing comprehension than the in tentional learning condition only in the Written Recall measure, but not in the multiple-choice Reading Comprehension Test. In terms of inter action between annotation type and learning condition, there was not interaction between annotation type and learning condition on L2 vocabulary immediate recall. The interact ion between annotation type and learning condition on L2 reading comprehension was not significant in multiple-choice Reading Comprehension Text. However, the interact ion was found to be significant in Written Recall: in the incidental learning cond ition, the difference between text-picture annotation and audio-picture a nnotation was not significant; in the intentional learning condition, participants in text-picture did sign ificantly better than those in audio-picture on Written Recall.

PAGE 16

1 Chapter I Introduction Second language (L2) lear ners at all levels are fa ced with the difficulty of learning vocabulary. For example, Nation (1993, 2000) suggests that a native speaker of English knows about 20,000 word families. This poses a challenging task for English as a Second Language (ESL) learners. However, voc abulary learning has long been neglected within the field of Second Language Acqui sition (SLA) (Coady, 1993; Coady & Huckin, 1997; Davis, 1989; Gass, 1987; R ead, 2004; Zimmerman, 1997). Recent years have seen increased intere st in L2 vocabulary research. One influential debate over the years is between incidental and intenti onal vocabulary learning (Gass, 1999; Hulstijn, 1992, 2001). The distinct ion between the two learning conditions has been attributed to learning task, learne r attention and the peda gogical context of the learning (Read, 2004). Both approaches ha ve been argued to contribute to the incremental learning of L2 vocabulary (Hul stijn, 2001; Paribakht & Wesche, 1997; Parry, 1997). L2 research has argued for incident al vocabulary learning through reading (Coady, 1997; Krashen, 1989; Nation, 1990, 2001). This conforms to L2 learners’ reports that vocabulary learning happens, in most cas es, accidentally during reading or listening. However, L2 incidental voca bulary learning tends to be in cremental and slow (Hulstijn, 1992; Jacobs et al. 1994). Annotation has been used as a standard feature in L2 reading materials to facilitate comprehension in which L2 voca bulary learning comes a bout as a by-product

PAGE 17

2 (Davis, 1989; Holley & King, 1971; Hulstijn, 1992; Jacobs, Dufon & Fong, 1994; Joyce, 1997; Watanabe, 1997). As an instructiona l intervention, an annotation draws learner attention briefly away from reading, and fo cuses it temporarily on the form and meaning of the annotated word, thus enhancing vocabulary learning and overall reading comprehension. This echoes the interactio nist view of SLA (Long, 1991, 1996) and the depth of processing hypot hesis (Hulstijn, 1992). The effects of text annotation on L2 vocabulary learning and reading comprehension have been examined by studies which produced mixed findings (Davis, 1989; Hulstijn, 1992; Hulstijn et al 1996; Jacobs, 1994; Jacobs et al. 1994; Joyce, 1997; Ko, 1995; Rott, Williams, & Cameron, 2002; Watanabe, 1997). The advance of multimedia L2 learning and teaching has take n the investigation of annotation a step further (Lomicka, 1998). Different from the traditional marginal annotation, multimedia annotations can present vocabulary informati on in multiple modalities, such as audio (sound) and visual (text, pict ure and video) (Al-Seghayer, 2001; Chun & Plass, 1996a). Studies have examined the effects of di fferent types of multimedia annotations on incidental L2 vocabulary learning, in particul ar, the use of pictur e annotation and video annotation coupled with text annotation (Al-Seghayer, 2001; Chun & Plass, 1996a; Jones, 2003; Yoshii, 2000). These studies support dual-coding theory (Paivio, 1971, 1986, 1990) and confirm the cognitive theory of mu ltimedia learning (Mayer, 1997, 2001) that maintains how meaningful learning engages lear ners in both verbal and visual cognitive processing systems. Dual annotation of text and picture or text and video are unanimously argued to be better than single annotations in facilitating incidental L2

PAGE 18

3 vocabulary learning (Al-Seghayer, 2001; Chun & Plass, 1996a; Kang, 1995; Kost et el 1999; Jones, 2003; Yeh & Wang, 2003; Yoshii, 2000). Studies in audio annotation have mainly engaged the use of pronunciation of the target words, and their findings are inc onsistent (Chun & Plass, 1996a; Svenconis and Kerst ,1995; Yeh & Wang, 2003). Audio is a di fferent sensory modality from visual modality, such as printed text and pictures, because audio is pro cessed by the auditory working memory while printed texts and pi cture are processed by the visual working memory; thus it should be treated separately as to its effect on lear ning. Studies of audio annotation should include not only the pronunciation, but also the definition or meaning of the target words. Furthermore, research suggests that th e addition of an audio element to dual annotations does not seem to have a definite effect on L2 vocabulary learning (Chun & Plass, 1996a; Svenconis and Kerst, 1995; Yeh & Wang, 2003). One possible explanation is that the information delivered simulta neously through different modalities (audio, verbal and visual) might overl oad the cognitive processing. Only two studies have addressed the re lationship between vocabulary annotation and reading comprehension in a multimed ia setting (Chun & Plass, 1996b; Lomicka, 1998). Their studies confirmed the effects of annotations on reading comprehension, but the scarcity of research in this area needs further investigation. As an instructional intervention, multimedia annotation does brie fly interrupt the reading process by drawing learner attention away from the intended r eading goal; it is, theref ore, necessary, when examining L2 vocabulary learning, to further inve stigate the effects of annotations on text comprehension.

PAGE 19

4 Statement of the Problem An overview of the studies on L2 voca bulary annotation, particularly multimedia annotation, suggests that there is little information about how different dual annotations, particularly text-picture and audio-picture annotations, aff ect L2 vocabulary learning and reading comprehension. Specifically, in ESL classrooms, there is little information on how these two types of dual annotation faci litate ESL students’ vocabulary learning through reading. This information is need ed to understand the extent to which multimedia learning can be used in L2 readi ng instruction and the role of multimedia in L2 vocabulary learning. The scant informa tion on audio annotation in multimedia L2 learning in comparison to other multimedia annot ations warrants further investigation. In addition, incidental and intentional vocabular y learning in a multimedia setting has never been studied. The effects of multimedia dual annotations employing different modalities on L2 vocabulary learning and reading comp rehension in incidental and intentional learning conditions remain unclear. Theoretical Framework As an interdisciplinary area of inquir y, SLA benefits from a broad range of theories and principles. Annota tion research fits into the in teractionist paradigm of SLA which argues that language input th at learners receive need to be salient and be noticed to engage learners in negotiation of meani ng (Long, 1996). The availa bility of annotation offers comprehensible input for unfamiliar wo rds during the reading process and helps L2 learners to comprehend the text. Meanwhile, incidental learning of those unfamiliar words happens due to multiple inputs. For example, the learners notice the unfamiliar

PAGE 20

5 words for the first time when seeing them in reading, then encounter them when checking the annotations, and finally seeing them when returning to reading (Watanabe, 1997). In addition, the use of multimedia annotation, which is only available upon mouse clicking, does not directly interfere with the reading proc ess. In terms of cognitive explanations of SLA, annotation shifts the learner attenti on temporarily away fr om the main cognitive task of reading and focuses it briefly on the form and meaning of the unfamiliar words. This process is claimed to be facilitati ve of second language acquisition because deliberate attention to the targ et form helps the learners to notice the form and meaning. The noticing can give salience to an unfamiliar word so that it becomes more noticeable in future input, which, therefore, contributes to the learner's psyc hological readiness to acquire that feature (Long, 1996; Schmitt, 2000) Because of the nature of this study and its focus on the use of multimedia annotations in ESL vocabulary learning and reading comprehension, the cognitive theoretical framework of mu ltimedia learning (Mayer, 1997, 2001) was chosen as the theoretical framework. Mayer argues that, in multimedia le arning, materials are normally presented in verbal (includi ng written text and spoken text) and visual (including pictures, video, animation) forms; these pres ented materials enter the learners’ sensory memory through the visual channel (eyes) a nd the auditory channe l (ears); and in the process, the learner is regarded as an activ e knowledge constructor w ho selects, organizes and integrating both visual/picto rial and auditory/verbal inform ation such as printed texts, pictures, or spoken words (Mayer, 1997, 2001). Mayer’s theory describes the process of learning in a multimedia environment from different presentation mode and different

PAGE 21

6 modalities, and extends itself into the realm of second language learning, and in particular multimedia annotations (Chun & Plass, 1996a). Purpose of the Study Motivated by previous studies on multimedia annotation and existing gaps in this literature, the overarching question addressed in this study was how different dual annotations affect L2 vocabul ary learning and reading compre hension in both incidental and intentional settings. This study was de signed to broaden our understanding of the application of multimedia le aning in a second language acquisition environment through the framework of cognitive th eory of multimedia learning to second language vocabulary learning and reading comprehension. It examin ed the ways in which two different types of dual annotation, namely, text-picture a nd audio-picture, aff ected L2 vocabulary learning and reading comprehension. In addition, it addressed the effects of multimedia annotation on L2 students’ vocabulary learning in both incidental a nd intentional learning conditions. Research Questions This study was guided by the fo llowing research questions: 1. What are the different effects of text-p icture and audio-picture annotations in facilitating L2 vocabular y immediate recall? 2. What are the different effects of text-p icture and audio-picture annotations in facilitating L2 reading comprehension?

PAGE 22

7 3. Which learning condition (incidental or intentional) results in better L2 vocabulary immediate recall? 4. Which learning condition (incidental or inte ntional) results in better L2 reading comprehension? 5. Is there an interaction between anno tation type and l earning condition on immediate L2 vocabulary immediate recall? 6. Is there an interaction between annota tion type and learni ng condition on L2 reading comprehension? Hypotheses The following research hypothese s were investigated in order to answer the above six research questions: 1. Hypothesis: Students in the audio-picture annota tion group will perform significantly better than st udents in the text-pictu re annotation group on the immediate vocabulary posttests. 2. Hypothesis: Students in the audio-picture annota tion group will perform significantly better than st udents in the text-pictu re annotation group on the reading comprehension posttests. 3. Hypothesis: Students in the intentio nal learning condition will perform significantly better than st udents in the incidental learning condition on the immediate vocabulary posttests.

PAGE 23

8 4. Hypothesis: Students in the incide ntal learning condition will perform significantly better than st udents in the intentional learning condition on reading comprehension posttests. 5. Hypothesis: There will be an interaction between a nnotation type and learning condition on L2 vocabulary immediate recall. 6. Hypothesis: There will be an interaction between a nnotation type and learning condition on L2 reading comprehension. Significance of the Study This dissertation was conducted for both theoretical and practical reasons. At the theoretical level, the study aimed at cont ributing to the growing body of multimedia L2 vocabulary studies. First, it pr ovides the much-needed inform ation on the nature of audio annotation on L2 vocabulary lear ning. Second, it sheds light on the use of different dual annotations for multimedia L2 learning. Third, it contributes to the extension of the cognitive theory of multimedia lear ning to second language learning. This study provides practical inform ation for CALL vocabulary material designers in choosing the right combination of modalities in facilitating L2 learning. It also informed language teachers and administ rators who need to make solid decisions about multimedia programs to enhance L2 vocabulary learning and reading comprehension.

PAGE 24

9 Definition of Terms 1. Annotation/Gloss – a brief definition or s ynonym or a short explanation of meaning given to an unknown L2 word during read ing. Traditionally, an annotation is provided in the margin of the text or at th e end of the text. In a multimedia setting, an annotation is linked to a hype rtext. In this study, the wo rds “annotation” and “gloss” are used interchangeably. 2. Audio annotation – a multimedia annotation that uses spoken text, preferably the voice of a native speaker of the L2. It can be the pronunciatio n of an unknown word, a short definition, or a senten ce that contains the word. 3. English as a Foreign Language (EFL) – E nglish being studied by non-native speakers as a foreign language in an environment where English is not spoken as the first language, as in China. 4. English as a Second Language (ESL) – Eng lish being studied by non-native speakers as a second language in an English-speak ing country, as in the United States. 5. First language – The native language of a l earner that has been acquired as a child. 6. Incidental learning – before engaging in th e learning task, the lear ner is not told of the learning goal, but he/she is tested on that learning goal afterwards unexpectedly. For example, before reading a multimedia L2 text, an ESL student is told to read the text for comprehension, but is tested on specific pre-selected annotated words afterwards. In this case, the vocabul ary learning is inci dental learning. 7. Incidental vocabulary learni ng – Vocabulary learning that occurs incidentally during the process of reading or lis tening in a second language. In other words, the learners learn the word unintentionally when they are engaged in a reading or listening task.

PAGE 25

10 8. Intentional learning – before engaging in th e learning task, the le arner is told of a subsequent posttest. For example, before reading a multimedia L2 text, an ESL student is told of a vocabulary test on those annotated words they will come across during reading. In this case, the vocabul ary learning is inte ntional learning. 9. L2 (Second Language)this term generally refers to a second language, including second language and foreign language. A second language refers to the language being studied as a second language in a country where it is used as the native language; a foreign language refers to a la nguage being studied as a second language in a country where the language is no t the native language. Second language and foreign language will be used interchangeably in the study. 10. Multimedia annotation – annotation pres ented through a multimedia program, in other words, when L2 learners encounte r an unknown word when reading via the computer, they can click on the unknow n word and access a provided annotation. This type of annotation employs multiple m odalities, such as auditory (sound, spoken text) and visual (printed text picture), and engage different presentation modes, such as verbal (printed text, spoken te xt) and nonverbal/vis ual (picture). 11. Multimedia – the presentation of learning mate rial using both words and pictures via a computer (Mayer, 2001). 12. Multimedia learning – learning of new information from words and pictures via a computer. 13. Multimedia text – an L2 reading text presented via a computer.

PAGE 26

11 14. Reading comprehension – after reading the text, students will be able to recall the main idea units from the text through answering open-ended comprehension questions. 15. Text annotation – an annotation in a written text. In a printed reading text, a text annotation is usually located in the margin, at the bottom, or at th e end of the reading text; in a multimedia text, a text annotation is accessible upon clicking the unknown word. It usually appears in a pop-up window and does not block the text section where the word is embedded. 16. Picture annotation – an annot ation that uses a picture to clarify the meaning or description of an unknown word. It is us ed in both printed reading text and multimedia reading text. 17. Video annotation – a multimedia annotation that uses digitized video or animation. It is used to clarify the meaning of an unknow n word or describe the content of the word in a context. 18. Vocabulary immediate recall – the ability to recognize and recall the word meaning after one exposure of the wo rd in reading context. Limitations and Delimitations This study focused on ESL vocabulary learning and reading comprehension; therefore, other aspects of ESL learning such as grammar, listening and speaking were excluded. The participants were post-second ary adult ESL learners; therefore, ESL learners at the elementary or secondary le vels were excluded. The reading text was

PAGE 27

12 written specifically in its content and length for the research study, catering to intermediate ESL students’ reading level. Twenty target words in the text were hi ghlighted and annotated. Instead of the traditional multiple factors involved in lear ning vocabulary, vocabulary learning in this study was operationalized as vocabulary immediate recall th at indicates cognition of a word and recall of its meaning when presen ted after a one-time exposure during the webbased reading activity. Vocabulary imme diate recall was only measured through immediate posttests; thus, long-term reten tion of word meaning was not measured. The audio annotation in this study was comp rised of a spoken text of the meaning for the annotated word by a native speaker. Ther efore, the auditory m odality in this case was limited to spoken text, excluding a ny other sounds such background music or sounds that are associated with words. Both external and internal validity lim ited the findings of this study. One possible threat to internal validity was selection of participants, participants in different experimental groups might not be functionally equivalent in respect to their knowledge of the target words. With regard to external validity, possi ble threats included the following: (1) Population validity: intermediate ESL stude nts were randomly chosen from English language institutes at universities from northwest U. S. Therefore, the findings cannot be generalized to ESL students at the begi nning and advanced levels. (2) Ecological validity: participants are limited to a certain number of ESL learners from a specific geographic area in the U. S.

PAGE 28

13 To alleviate internal validity threats, pa rticipants were rando mly assigned to four experimental groups. In terms of external valid ity threats, the partic ipants in this study were a unique group of students, but they were representative of th e larger population of intermediate ESL students. Organization of the study The present study is organized in five chapters. Chapter I introduces the study and outlines the significance of the study. The main reasons for conducting study are stated and research questions are pres ented along with hypotheses. Operational definitions of the terms used in this study are also provided. Chapter II elaborates on the L2 vo cabulary learning and L2 reading comprehension literature. This is followed by a detailed review of literature on annotation and multimedia learning. Chapter III discusses the de sign of this study. It provide s the readers the overview of the procedures, including participants, inst ruments, data collecti on and the nature of data analysis. Chapter IV presents the results of data an alysis and answers each the six research questions accordingly. Chapter V discusses the findings based on the research questions. It also presents theoretical implications and pedagogical recommendations in the field of second language acquisition. Directions for fu ture research are also provided.

PAGE 29

14 Chapter II Review of the Literature Traditionally, the teaching and learni ng of vocabulary has been neglected in second language acquisition (SLA) resear ch (Coady, 1993; Davis, 1989; Gass, 1988; Zimmerman, 1997). The argument in SLA to fo cus on form in communicative classroom activities tends to put the acquisition of grammar in the spotlight (Doughty & Williams, 1998; Long & Robinson, 1998). Therefore, in fa vor of syntax, vocabulary has generally been given a secondary place in the langua ge curriculum (Nation, 1990, 2001). However, many L2 learners share the difficulties of l earning vocabulary, and are often frustrated and discouraged by the unfamiliar words contained in reading texts. It seems that when L2 learners are engaged in read ing texts or listening in the target language, the first and foremost challenge they are faced with is vocabulary, as argued by many second language professionals (Folse, 2005; Grabe & Stoller, 1997; Huls tijn, 2001; Laufer & Hulstijn, 2001; Nation, 1990, 2001; Read, 2004). This chapter contains three parts. The first part presents an overview of L2 vocabulary learning in the c ontext of reading, focusing on the distinction between intentional and incidental learning. This is followed by an introduction of annotation as an instructional interv ention to facilitate incidental vo cabulary learning. The second part reviews studies on annotation in both printed texts and multimedia texts, including text annotation, picture annotation, video annotation and audio annotation. The third part discusses the theoretical underpinnings of multimedia annotation, namely, the cognitive theory of multimedia learning and its extens ion in second language vocabulary learning.

PAGE 30

15 The application of the cognitive theory of multimedia learning in multimedia L2 vocabulary annotations is also explained. L2 Vocabulary Learning Researchers have argued for the import ance of vocabulary in SLA because limited vocabulary knowledge prevents L2 lear ners from communicating effectively in the target language (TL) (D avis, 1989; Gass, 1999; Stei n, 1993; Wesche & Paribakht, 1999). As a result, vocabulary research has received increased recognition in SLA since the 1990s, especially after the appearance of Gass’s (1987) thematic collection on L2 lexical issues published in SSLA. Paribakht and Wesche (1999) stat ed that “since that time, a large body of L2 research and theory has developed around these and other topics, and it has become possible to deal comprehensiv ely with single core i ssues in L2 lexical acquisition from multiple perspectives” (p. 175). This trend corresponds to the longoverdue emphasis on vocabulary acquisition. The reciprocal relationships betw een vocabulary knowledge and reading comprehension have been demonstrated for both first language (L1) learning (Anderson & Freebody, 1981; Stanovich, 1986, 2000; Ster nberg, 1987) and second language (L2) learning (Bossers, 1992, cited in Pulido, 2004; Haynes & Baker, 1993; Laufer, 1992). It is reasonable to assume that better readers te nd to have a larger num ber of sight words, and those with a larger vocabulary tend to be better readers. An educated native speaker of English, suggested by some researchers (Laufer, 1992; Nation, 1993, 2001), knows about 20,000 word families. This poses a challenging task for ESL learners. The readability of an L2 text, to a large exte nt, depends arguably more on vocabulary than on

PAGE 31

16 other factors such as sentence structure, syntax and rhetoric style (Bergman, 1978). Therefore, it is necessary for L2 learners to learn and enlarge the size of their vocabulary to be able to comprehend L2 texts. L2 studies have attempted to esta blish a connection between vocabulary knowledge and reading comprehension (Lau fer & Sims, 1985; Coady, 1993; Grabe & Stoller, 1997). Laufer and Sim (1985) entert ained the possibility that L2 learners apply more vocabulary knowledge than th at of the subject matter or syntax in understanding the reading text. In other words, a certain number of known words are necessary for L2 learners to approach an authentic L2 text comfortably. Laufer (1992) claimed a baseline of 3000 words as the threshold vocabulary si ze required for academic English reading. Furthermore, Nation (1990, 2001) argued that in order to comprehend an L2 text with relative ease, readers should be familiar with 95-98 % of the words in the text at any level. Incidental and Intentional Learning The distinction between incidental and intentional vocabular y learning takes its origin from experimental psychology (Hulst ijn, 2001). In the psyc hological l iterature, incidental learning and intent ional learning are distinguished with different pre-learning instructions. Incidental lear ning refers to the si tuation in which the learners are not informed of their responsibility for cert ain information but are evaluated on that information later. On the other hand, in inte ntional learning, the learners are informed of their responsibility for certain in formation before doing the task. In L2 vocabulary learning, there has been debate between incidental and intentional learning over the years (Hulstijn, 1992, 2001; Hulstijn, Hollander and

PAGE 32

17 Greidanus, 1996). Despite the discrepanc y among researchers on operationalizing the two terms, it is generally agreed that in tentional vocabulary lear ning, in which student attention is directly engaged and focuse d on vocabulary, offers a greater chance for vocabulary learning (Hulstijn, 1992. 2001; K o, 1995; Nation, 1991, 2001). In incidental vocabulary learning, students are engaged in other language learning activities such as reading or listening, and vocabul ary learning occurs incidenta lly and does not involve a conscious effort to learn words (Schimitt, 2000). It is, therefore, defined as “a byproduct, not the target, of the main cogn itive activity, reading” (Huckin & Coady, 1999, p. 182). Read (2004) attributed th e distinction between incidental and intentional vocabulary learning to learner attention and pedagogical contex ts of the learning. Factors such as the context in which the words are us ed, the task demands and others, as pointed out by Huckin & Coady (1999), are importa nt to understand incidental vocabulary learning. In addition, tasks be yond reading itself, such as initiating look-up behavior, have been a recent twist added to the inci dental vocabulary learning domain (Laufer & Hill, 2000). It seems that more demanding tasks result in more incidental vocabulary learning through reading. In this respect, Laufer a nd Hulstijn (2001) proposed an important concept of "task-induced involvement" that is three-fold: the need to learn, the ability to search for information or form of the word, and the evaluati on of the available information for the particular word under inquiry (Read, 2004, p. 148). However, the incidental-intentional dic hotomy has been challenged in recent years by a number of researchers (Par ibakht & Wesche, 1997; Parry, 1991, 1993, 1997;

PAGE 33

18 Zimmermand, 1994). Hulstijn (2001) suggested th at it “is the quality and frequency of the information processing activ ities (i.e., elaborations on as pects of a word’s form and meaning, plus rehearsal) that determine rete ntion of new information” (p. 275). It was also emphasized by Hulstijn (1992, 2001) and Hulstijn et al. (1996) that both approaches should coexist in L2 teaching and learning. It is a consensus among L2 researchers that both intentional instruction and incidental learning are necessary for second language learners, and they are two complementary ac tivities (Hulstijn, 2001; Laufer & Hulstijn, 2001; Nation, 2001; Schmitt, 2000; Read, 2004). Incidental Vocabulary Learning It is a commonly held belief in both L1 a nd L2 research that vocabulary is mostly learned incidentally through reading (Coady, 1997; Krashen, 1989; Nagy, Herman & Anderson, 1985; Nation, 1990, 2001; Shu, Anderson & Zhang, 1995). Most L2 learners as well seem to agree that vocabulary learni ng happens in most cases accidentally during reading or listening. After reviewing 144 studies on incidental vocabulary learni ng, Krashen (1989) concluded that incidental vo cabulary learning affords bette r results than intentional vocabulary learning. In a review of incide ntal vocabulary learning, Huckin and Coady (1999) maintained that: Most scholars seem to agree that except for the first few thousand most common words, vocabulary learning predominantly occurs through extensive read ing, with the learne r guessing at the meaning of unknown words. This se condary type of learning is

PAGE 34

19 called “incidental” learning because it is a by-product, not the target, of the main cognitive activity, reading. They suggested three advantages of in cidental vocabulary learning over explicit vocabulary instruction. (1) In comparison to the isolated word lists sometimes used in explicit instruction, incidental learning pr esents the known words in context (Oxford & Crookall, 1990). (2) Pedagogically speaking, incidental learning, serving as a doubleedged sword, promotes simultaneously both vocabulary acquisition and reading comprehension. (3) By triggering selective attention, incidental learning enables more individualized learning of vocabulary (Schmitt, 2000). However, the number of new words lear ned incidentally is relatively small compared to the number of words that can be learned intentionally (Hulstijn, 1992). Even with the use of a dictionary and the inferring strategy, in cidental vocabulary learning tends to be incremental and slow (Hulst ijn, 1992; Jacobs et al., 1994). In their study, Hulstijn, Hillander and Greidanus (1996) summ arized several reasons why L2 learners failed to learn the unknown words incident ally during reading. (1) Sometimes L2 learners failed to notice the new words, or they assumed the familiarity with the new words. (2) Sometimes L2 learners noticed the new words, but ignored them. (3) The learners did not focus their attention on the unknown words. (4) The inferred meanings from context by the L2 learners were inco rrect. (5) The low frequency of most unknown words prevents effective acquisition. One way to facilitate incidental voc abulary learning is through the use of annotation or gloss. When consulting the dic tionary for word meanings, students need to search for and choose the most appropriate meaning out of several possible definitions.

PAGE 35

20 Unlike dictionary look-up, annotation offers th e meanings of words based on the context and presents the meanings in the margin wh ich is easily accessible to students. Annotation As defined by the Merriam-Webster Online Dictionary, gloss refers to “a brief explanation (as in the margin or between the li nes of a text) of a difficult or obscure word or expression.” Annotation is define d as “a note added by way of comment or explanation.” Gloss and annotation, as a vocabulary learning aid, have been used interchangeably in L2 research and pedagogy. The origin of gloss dates back to the Middle Ages, when Scripture verses were circled with notes written in smaller fonts (Davis, 1989). According to lexicographer Werner Hllen (1989, cited in Roby, 1999), early glosses were usually learner-generated. Medieval students produced th ese interlinear or marginal scribbling as they struggled with Latin text. Glosses generated by teacher s or material develope rs came much later. Widdowson (1978) divided thes e professionally generated glosses into two types: priming glossaries that serve as an advance organizer and are usually provided preceding the reading text, and prompting glossaries to which readers turn for consultation during the reading process. Roby (1999) offers a comprehensive taxonom y of glosses that divides glosses using six criteria: (1) gloss authorship that divides glosses into those generated by the learners or by professionals such as inst ructors or material developers, (2) gloss presentation that distinguishes priming gloss and prompting gloss, (3) gloss functions that include procedural functions (for example, metacognitive, highlighting, and clarifying)

PAGE 36

21 and declarative ones (such as encyclopedi c and linguistic), (4) gloss focus that emphasizes either textual or extratextual info rmation, (5) gloss language in L1, L2 or L3, and (6) verbal, visual or audio gloss forms. He defined glosses as any “attempts to supply what is perceived to be de ficient in a reader's proce dural or declarative knowledge” (Roby, 1999, p. 96). In this study, annotation is defined as a brief definition or a short explanation of meaning given to an unknown L2 word duri ng reading. The annotati ons will be provided in a multimedia setting where the students can access the annotations by clicking the highlighted words in the web-based reading text. Two types of multimedia annotation are available: text-picture and audio-picture. Traditional Annotation A vocabulary annotation is defined as a brief definition or synonym (Nation, 1990, 2001) or an explanation of the meaning of an unfamiliar word (Pak, 1986, cited in Jacobs, 1994). Annotations ar e supplied to help L2 lear ners’ vocabulary learning and enhance reading comprehension. Arrows, br ackets, underlining, bol ding, or highlighting in the text are common ways to indicate these annotated wo rds (Stewart & Cross, 1991). Vocabulary annotations are typica lly located in the margins e ither on the side or at the bottom of the page, or at the end of the reading text. Foreign language/second language readi ng materials have long been using annotation as a standard feature (Davis, 1989; Holley & King, 1971; Jacobs, 1994). Holley & King (1971) posited two advantages of annotation in facilitating vocabulary learning: (1) Annotations help L2 learners avoid incorrect gu essing from the context, and

PAGE 37

22 (2) the annotation-checking behavior engages rehearsal of the targeted vocabulary. For example, seeing the unknown word again in the annotation will reinforce the first exposure of the word in the text, making it pos sible for the word to integrate into the learner’s reading vocabulary. Watanabe (1997) explained the second advantage in terms of input frequency; that is, L2 learners expe rience multiple inputs of the targeted words. Take a particular unknown word, for example, L2 learners read the text and meet the unknown word (first input), they check the annotation for the meaning of the word (second input), and they return to the reading text trying to fit the meaning into the context (third input). Nation (1990, 2001) argued for vocabulary annotation being f acilitative to both vocabulary learning and reading comprehe nsion: (1) the knowledge of unknown words provided by annotations facilita tes reading comprehension; th us, authentic L2 texts can be used in an unsimplified manner; (2) annotations prevent in correct guessing from context and provide context-specific m eanings for unknown words during reading; therefore, L2 learners do not need to depe nd on teachers or dictionaries for definitions; (3) L2 learners can choose to access the annotations when needed in order to minimize interruption to the reading pr ocess; (4) annotation promotes individualized learning in that L2 learners are offered the freedom to choose needed annotations for word meanings; (5) annotations draw learners’ attention to the target words and might encourage and reinforce learning. Annotations seem to serve several purposes First, they assist vocabulary learning (Holley & King, 1971; Hulstijn, 1992; Jacobs, Dufon & Fong, 1994; Ko, 1995; Watanabe, 1997). Second, they enhance r eading comprehension (Davis, 1989; Ko, 1995;

PAGE 38

23 Jacobs, 1994; Jacobs et al., 1994; Joyce, 1997; Rott, Williams, & Cameron, 2002). Third, they draw learner attention to the annotated words in both forms and meaning, according to Schmidt’s (1995) noticing hypothesi s. Furthermore, L2 learners prefer vocabulary annotations (Jacobs, Dufon & Fong, 1994) to dictionary use. Research has, therefore, focused on the effects of differe nt types of annotation, and the effects of annotation on L2 vocabulary learni ng and reading comprehension. The use of annotation as an instructio nal intervention draw s learner attention briefly away from the main c ognitive task of reading and towa rd the form and meaning of unknown words. This fits into the focus-onform approach proposed by Long (1991). The approach is claimed to be facilitative to L2 acquisition. Focus-on-form “overtly draws students’ attention to linguistic elements as they arise in cidentally in lessons whose overriding focus is on meaning or comm unication” (p. 45-46). Though generally associated with grammatical acquisition, fo cus-on-form is suitable to explain the facilitative use of annotation in L2 vocabulary learning. According to Schmitt’s (2000) noticing hypot hesis, the provision of annotations, by giving salience to the target forms, make s the annotated words more noticeable and contributes to the learners’ ps ychological readiness to acquire the target forms. Noticing involves the intake both of meaning and form of the annotated words, which helps the learners to progress from initial recognition to internalization of the target forms. When checking the annotations provided in an L2 reading text, lear ners pay deliberate attention to particular lexical forms and they notice the gap between their own performance in the target language and the performance of proficient users of the language.

PAGE 39

24 Multimedia Annotation The investigation of annotation has been ta ken a step further w ith the advance of multimedia application in second language teaching and learning (Lomicka, 1998). Davis (1989) predicted the promise of multim edia annotations because “hypertext is invisible and thus unobtrusive in the reading process, yet the user is able to obtain as much or as little detail about a given con cept as is desired” (p. 42). Jacobs (1994) confirmed that computerized annotation wa s an effective means in L2 vocabulary learning and was preferred by L2 learners. Different from traditional annotation, multim edia annotations are not restricted to textual information. Instead, multiple moda lities can be used to present vocabulary information, such as auditory mode (sound) a nd visual mode (text, picture and video) (Al-Seghayer, 2001; Chun & Plass, 1996a). Word s with annotations in the reading texts are usually indicated by hyperlinks. U pon clicking the hyperlink on the word, the learners can access various forms of annotat ion in a pop-up window or in a designated area of the screen such as the left side, right side or the bottom of th e screen. Usually, if a pop-up window is used, it does not cover the text section where the annotated word appears, in order not to interf ere with the reading process. Overall, there are four ty pes of annotations: text a nnotation, picture annotation, audio annotation, and video annotation. A text annotation offers a textual definition or meaning of the unknown word. A picture annota tion usually employs a static picture to describe the meaning of the unknown word. An audio annotation often uses the voice of a native speaker to read the word, or to read a sample sentence containing the target word,

PAGE 40

25 or to read the meaning of the target wor d. A video annotation employs a video clip to depict the meaning of the target word. In the following section, annotation studies will be reviewed in four subsections: text annotation, picture annotation, video annot ation, and audio annotation. Each of the subsections will cover both printed text a nd computerized text. In fact, annotations involving pictures, video and a udio are mainly located in multimedia environments. Text Annotation In printed reading materials, text annota tions are often located in the margin, at the bottom, or at the end of the reading text In multimedia texts, when students click on an annotated word, they can view the mean ing or definition of the word in a pop-up window or in a certain location of the computer screen. In this section, discussion will be first focused on text annotation in printed r eading texts, followed by a review of text annotation in multimedia texts. Text Annotation in Printed Reading Texts Studies addressing text annotations will be reviewed in two parts: text annotation and L2 vocabulary learning, and text annotation and L2 reading comprehension. Text annotation and L2 vocabulary learning. Hulstijn, Hollander, and Greidanus (1996) examined incidental vocabulary le arning for second language learners. Their study identified the use of marginal text annot ation as an effective method. Other studies confirmed that text annotation in printed reading text could enhance second language

PAGE 41

26 learners’ retention of vocabulary (Hul stijn, 1992; Jacobs, Dufon & Fong, 1994; Ko, 1995; Watanabe, 1997). In Hulstijn, Hollander, and Greidanus’ (1996) study, three di fferent types of instructional scenarios on a French short st ory by Guy de Maupassant for advanced L2 learners were compared: (1) marginal annota tion (annotation in L1 Dutch), (2) dictionary access (bilingual dictionary), and (3) text on ly. The issue investigated was whether marginal annotation or dictionary use woul d lead to better vocabulary retention. The posttests of students’ receptiv e and productive knowledge of the 16 target words showed significantly better performance by the marg inal-annotation group than the dictionary group and text group. Different types of text annotation have been described such as single and multiple-choice annotations, as well as annotatio ns in L1 and L2 (Hulstijn, 1992; Jacobs, Dufon & Fong, 1994; Kang, 1995; Watanabe, 1 997). In Hulstijn’s (1992) study, experiments I, III and V were conducted with adult Dutch as a second language learners. Two sets of conditions were compared: glo ss to no gloss and multiple-choice glosses to single-glosses. The multiple-choice gloss for each target word was comprised of a correct synonym and three distractors, while the si ngle-gloss simply offered a synonym. In experiment I, L1 gloss led to better vocabular y retention than when the meaning had to be inferred from the context. Experiment III proved that the multiple-choice gloss group achieved substantially higher vocabulary scores than th e single-gloss group on the posttest. However, Experiment V did not fi nd any significant differe nce between single gloss and multiple-choice gloss. Therefore, Hulstijn (1992) pointed out that because of the high rate of incorrect inference in the open multiple-choice approach, it did not

PAGE 42

27 necessarily guarantee bette r learning of L2 vocabulary in co mparison to the closed singleglossing format. In line with comparing single-gloss and multiple-gloss, Watanabe (1997) conducted his research with 231 Japanese university students l earning English as a second language. A between-subjects design was used to examine the effects of four conditions in learning 16 unknown words in an English (L2) article: (1) appositive (an explanation in L2 of each difficult word inserted immediately after the word), (2) single marginal glosses (in L2), (3) multip le-choice marginal glosses (in which two choices are presented in L2 for each difficu lt word, and one of them is the correct meaning), and (4) text only with no cue. Both single and multiple-choice gloss groups were found to perform significantly better on the vocabulary posttest than the appositive and text-only groups on both the immediate a nd delayed posttest. Though the finding of the single-gloss group scoring higher in the posttests than the multiple-choice gloss group somewhat contradicted the results of Hulstijn (1992), it is confirmed that adding marginal gloss to a reading text could be an eff ective method to enhance vocabulary learning during reading Dufon and Hong’s (1994) study on L2 Span ish reading used three formats: (1) no gloss, (2) L1 gloss and (3) L2 gloss. Their results showed that students who had access to glosses outperformed students without glosse s on the immediate vocabulary translation posttest. However, the effectiveness of glo ss was not found in the delayed posttest four weeks later. In terms of this, Jacobs et al. warned that although gl oss is preferred over no gloss, gloss only has a potentially positiv e effect on vocabulary acquisition with sufficient L2 competence. In addition, certain proficiency level was necessary to make

PAGE 43

28 effective use of L2 gloss. Therefore, the positive relation between gloss and vocabulary learning was established, at least for immediat e retention if not for long-term retention. With regard to the comparison between L1 gloss and L2 gloss, no difference was found between the languages (L1 and L2) used for glossing. A questionnaire was used to examine students’ preference, and the result reve aled that students preferred L2 glosses to L1 glosses if the L2 glosses were comprehensible. In order to further investigate the possibl e difference between L1 and L2 glosses, Ko (1995) used a similar design as Jacobs, Dufon and Hong (1994) with 189 Korean college students learning English as a foreign language (EFL). Students took a vocabulary pretest and were aske d to read an 854-word Englis h text. In contrast to the finding of Dufon and Hong (1994), the multip le-choice vocabulary posttest immediately administered after the reading showed signi ficant difference between L1 gloss and L2 gloss. In other words, students with access to L1 gloss outperformed significantly better than those with access to L2 gloss. The effect was found significant in the delayed posttest one week later. The effectiveness of L2 gloss over L1 gloss in vocabulary retention was also challenged by Laufer and Shmueli’s (1997) study. Hebrew-speaking high school EFL students (N=128) were asked to read an E nglish text in which 10 target words were glossed in Hebrew and 10 other target wo rds were glossed in English. Multiple-choice assessment was used in the immediate posttest a nd delayed posttest five weeks later. Both tests showed that L1 gloss resulted in more vocabulary retention than L2 gloss. This conflicts with the finding by Jacobs et al. (1994), but the student s’ level of the second

PAGE 44

29 language in Laufer & Shmueli (1997) might be used as an explanation. Certain proficiency of the second language was n ecessary to fully utilize glosses in L2. Text annotation and L2 reading comprehension. Several text annotation studies in printed reading materials have examined the effect of annotation on L2 reading comprehension, but with mixed findings (Dav is, 1989; Ko, 1995; Jacobs, 1994; Jacobs et al., 1994; Joyce, 1997; Rott, Williams, & Cameron, 2002). Davis (1989) tried to investigate the facili tating effect of marg inal glosses on foreign language reading. The study used a between-subjects design among th ree groups: (1) reading text with no aids, (2) reading text with pre-r eading questions and vocabulary definitions, and (3) reading text with glossed vocabulary. The participants in the study were 71 intermediate-level U. S. college students learning French as a s econd language. Out of the 936-word reading text, 28 words and expressions were glossed. After the experiment, students in all the three groups were asked to recall the readi ng text in their L1 (English). Significant difference was found between the no-aid group and the other two groups with either prereading aids or glossing. The re sults supported the idea that use of glosses facilitated text recall or reading comprehension for inte rmediate foreign language learners. Jacobs (1994) confirmed the positive e ffect of marginal glosses on reading comprehension. He attempted to study the poten tial interactive effects of text variables, learners variables and situational variables for 116 U.S. college students in their thirdsemester Spanish reading class. The presence or absence of vocabulary glosses in the reading text was designated as the text variable. Learner L2 proficiency was chosen as one of the learner variables, and time on task was analyzed as one situational variable.

PAGE 45

30 Two reading conditions were pr esented: (1) unglossed L2 (Spa nish) text and (2) L2 text with marginal gloss in L1 (English). Th e glossed group was found to recall 30% more idea units than the unglossed group, while no interaction was found significant between any of the learners and situat ional variables and glossing. Ho wever, one interaction that did approach significant level was the one be tween glossing and time on task for one of the unit idea recall measures. In Rott, Williams and Cameron’s (2002) experiment, 67 English-speaking college students taking 4th-semester German were divided into four reading c onditions: (1) text with multiple-choice glosses; (2) text recons truction in which students were asked to reconstruct the text after reading; (3) text reconstruction and multiple-choice glosses; and (4) text-only control group. L1 recall administ ered after the experiment proved that gloss groups yielded significantly better reading comprehension than the control group. The combined text reconstruction and gloss gr oup had significantly better comprehension than the control group, which further confirme d the positive effect of gloss on reading comprehension. However, arguments have been made ag ainst the effect of glossing on reading comprehension (Ko, 1995; Jacobs et al. 1994; Joyce, 1997). Ko’s (1995) study looked at the effects of glossing on both L2 voca bulary learning and reading comprehension. Although the effect of glossing on vocabular y learning was supported significantly, its effect on reading comprehension was not positive. L2 gloss was found to be the same as no-gloss in facilitati ng reading comprehension, and L2 gloss was found to even worsen reading comprehension.

PAGE 46

31 In her study of comparing glossed and unglossed L2 text on reading comprehension, Joyce (1997) used three levels first-semester, sec ond-semester and third semester, of students learni ng French as a second language at one U. S. university, Students from each proficiency level were divided into glossed and unglossed groups. The results questioned the effect of gloss on L2 reading comprehension because gloss was found to provide no significant help for t hose who studied text with glosses in their comprehension as measured by L1 after-reading recall. Text Annotation in Multimedia Texts As technology has been used more wi dely in second language teaching and learning, one technique used in glossing is known as hypertext (Bunnell, cited in Davis, 1989). Davis (1989) noted that glossing th rough hypertext offers two features: (1) multimedia glosses are usually invisible and, th erefore, do not interf ere with the reading process; and (2) by clicking on any hyperli nked word, readers have the freedom to choose the amount of information needed on a pa rticular word in the text (cited in AlSeghayer, 2001). In text annotation, there have been a few studies that dealt with computerized materials (Lyman-Hayer et al. 1993; Koren, 1998; Nagata, 1999). Text annotation and L2 vocabulary learning. Lyman-Hayer et al (1993) examined the impact of multimedia progr ams on vocabulary acquisition, and analyzed the difference between text annotations in conve ntional printed text a nd text annotation in multimedia text. The delayed vocabulary test ad ministered one week later confirmed that the students who worked in the multimedia text annotation group pe rformed better than

PAGE 47

32 those who worked in the traditional text annot ation group. Therefore, they concluded that multimedia text annotation enhances students’ ability in vocabulary retention. Replicating Watanabe’s (1997) study, Nagata (1999) i nvestigated the effectiveness of computer-assisted interactive glosses. American college students (N=26) taking second-semester Japanese were asked to read a text on the computer in two versions, text with single gloss and text w ith multiple-choice gloss. The reading text and glossing appeared on the computer screen simultaneously. She found that multiplechoice gloss was significantly more effectiv e than single gloss to help students in recalling the target vocabulary and grammati cal items during the immediate posttest. Nagata explained the conflicting finding fr om Watanabe (1997) by arguing that in multiple-choice glossing, students made more ef forts to interpret and paid more attention to the annotated words. However, the delayed posttest which wa s conducted one month after the treatment proved no difference between the two glossing methods on the vocabulary test, while the difference on grammar test was still statisti cally significant, favoring the multiple-choice group. Compared to the immediat e posttest, a dramatic decrease of retention was found in both vocabulary and grammar items, which confirmed that the one-time experiment had immediate effects on vocabulary learning, but might not be effective for long-term retention. Different from other studies using be tween-subjects design, Koren (1998) used a within-subjects design [within-subjects was also used in Hulstijn’s (1993) experiment III] to examine the possible advantage of hypertex t clue over conventional glossing. Firstyear university EFL students (N=129), most of them speaking Hebrew as their first

PAGE 48

33 language, read an English text. In the text, seven words we re highlighted and on clicking contextual or structural clues were provi ded, and another nine words were given a traditional marginal gloss in the L2. On the first posttest administered three days later, students remembered the inferred words four times better than they remembered the glossed words; on the delayed posttest three months later, in comparison to the first posttest, retention of the glossed words was n early identical (8%), but retention of the inferred words went down by nearly half from 33% to 17%. The results confirmed that text glosses were effective on incidental vocabulary learning, wh ich confirmed findings from other studies, but contrary to other studies, the effect of glosses was retainable in this study. Text annotation and L2 reading comprehension. A few studies in text annotation using multimedia text looked into the possi ble link between text annotation and L2 reading comprehension. With the major purpose of comparing annotation in printed text and multimedia text, Lyman-Hayer et al (1993) administered a written recall protocol immediately after the experiment to exam ine whether the annotation influenced L2 reading comprehension. They found positive e ffects of text annotation on students’ reading comprehension. In Hulstijn’s (1993) study, 44 Dutchspeaking high-school students were randomly divided into two task groups. Th e summary group was asked to summarize in Dutch after reading and the question group was asked to answer 10 comprehension questions. Both groups read the same 772-wo rd English text, with 109 words glossed, where the glosses could be retrieved on the comp uter screen with a cl ick of the enter key.

PAGE 49

34 Although different individual look-up behaviors were f ound, no difference in reading comprehension was found, nor was an effect of the reading task variable on gloss use. Summary In summary of text annotation studies, di fferent L1 and L2 are used, ranging from English, Japanese, French, and Hebrew to Du tch, and different reading materials were chosen by the studies as well. One aspect sh ared by most the text annotation studies is that one proficiency level (either elementary or intermediate or advanced) was used in comparing different annotations. All studies used between-subjects design except Koren (1998) and Hulstijn’s (1992) experiment III that used within-subjects design. Different lengths of time were used for the delayed vocabulary posttest, ranging from one week to three months. In both printed text and multimedia te xt, the effect of text annotation on vocabulary learning and reading comprehens ion were confirmed, though challenged by some mixed conclusions. Jac obs (1994) pointed out that “no single variable alone can guarantee a particular effect (context and inte raction between variables). In the case of glossing, the text difficulty, r eaders’ proficiency, number and choice of items glossed, and many other factors determine whether glossing will improve comprehension in a particular situation” (p. 1 29). As echoed by Nagata (1999), “We should keep in mind that the results may vary, depending on the type and difficulty of reading materials, the learners’ proficiency level, th e kind of test procedure and m easurement, and so forth” (p. 476).

PAGE 50

35 Picture Annotation Visual aids have long been assumed to be beneficial to s econd language learning. Tuttle (1975) argued that “foreign language students can benefit from many types of visual material… the still or flat picture can prove to be a rich resource in the foreign language classroom” (p. 9, cited in Omaggio, 197 9). The use of imagery representation of foreign words by actual objects or imagery was also claimed by Kellogg and Howe (1971) to be facilitative to children’s vocabulary acquisition in a foreign language. Subsequently, a number of researchers have ex plored the effect of visual stimuli on L2 vocabulary learning and reading comprehension. Kellop and Howe’s (1971) study compared wr itten words and pictures as cues for oral acquisition of Spanish vocabulary by childre n. The pictures yielded faster learning of new words than the written stimuli. And the effect was retained in the long term as indicated by greater recall of words shown in pictures. Terrel (1986, cited in Kost et al 1999) proposed that combining the form and visual representation of unknown L2 vocabulary helped learners to acquire concre te ideas and references. In reviewing the techniques used in learning L2 vocabul ary, Oxford and Crookall (1990) acknowledged the effectiveness of visual imagery and mainta ined that, “[M]ost lear ners are capable of associating new information to concepts in memory by means of meaningful visual images, and that visual images make learning more efficient” (p. 17) and “the pictorialverbal combination involves many parts of th e brain, thus providing greater cognitive power” (p. 17). Omaggio’s (1979) empirical study provide d pictorial contexts to French-as-asecond-language students as a dvance organizers. It was hypot hesized that the provision

PAGE 51

36 of additional visual context would facilitate reading comprehension. The results showed that students with access to a pictorial cont ext did significantly better on the recognition test and recall than those with access only to text. This supported the positive effect of pictures on reading comprehension. In annotation studies, pictur e annotation has been used to clarify the meaning of those unknown words second language learners en counter in reading. According to dual coding theory, the way learners comprehend pictures differs greatly from that of comprehending textual information (Paivio, 1971). In other words, text is processed by the verbal cognitive subsystem, while a pictur e is processed by the non-verbal cognitive subsystem. Research has compared L2 voca bulary learning from text annotation, picture annotation, and a combination of text and pictur e annotation Annotation and L2 Vocabulary Learning Kost, Foss & Lenzini (1999) and Yoshii ( 2000) both compared three annotations: text only, picture only, and text and picture, respectively in printed text and multimedia text. Their findings suggest the superiority of combining text and picture over single annotation types such as text or picture, establishing the effectiveness of dual presentation types of vocabular y annotations (text + pictures). In terms of the difference between text annotation and pict ure annotation, the results were interpreted in relation to the task type in the posttests Other studies confirmed th e findings (Kang, 1995; Jones, 2003; Yeh & Wang, 2003) Kost, Foss and Lenzini (1999) conducted their study with 56 U. S. university students learning second-semester German. A 272-word reading passage was adopted

PAGE 52

37 with consideration of learners’ proficiency level. Sixteen word s were chosen as the target words based on a vocabulary pretest, and were annotated in three different ways: text (English translation), picture, and a combinati on of text (English translation) and picture. The students were randomly assigned to the three annotation groups, and had access to one of the three annotations when reading. Three assessment tasks, namely production task, picture recognition task, a nd word recognition task, were used to evaluate students’ short-term vocabulary learning and retention two weeks later. It was hypothesized that students with access to both text and pictur e annotations would outperform those with either text-only or pictureonly annotations in both immedi ate and delayed posttests, and the performance of single text or single pict ure annotations would de pend on task types. Due to factors such as the difficulty level and frequency of target word exposure, the production task in which students were asked to provide L1 translation of target words did not show any difference among the thr ee annotation groups and was therefore excluded from the analysis. The analysis of the word recognition and picture recognition tasks showed that the text-only group outpe rformed the picture-only group in the word recognition test, while the pict ure-only group achieved higher scores than the text-only group in the picture recognition test. The resear chers argued that the relative superiority of text annotation or picture annotation depended on the task type. In the picture recognition task, the combination annotati on differed significantly on the picture recognition task from the text annotation, but not from the picture annotation. In the word recognition task, the combination annot ation differed significantly from both text only and picture only annotations in the im mediate posttest, but not from the text annotation in the delayed posttest. In summary, the combined text-picture annotation

PAGE 53

38 produced the best results in both immediate pos ttests and delayed posttests. The authors explained that the combined annotation provide d two different types of information that increased retrieval options from both verbal and visual cognitive systems, which reflected Paivio’s (1971, 1986, 1990) dual coding theory. Yoshii (2000, also Yoshii & Flaitz, 2 002) replicated the above-mentioned study in a multimedia setting. ESL students (N= 151) at the beginning and intermediate proficiency levels participated in the study. A pretest was used to verify the participants’ unfamiliarity with the 14 target words that we re annotated in three conditions: text only, picture only, and a combination of text and pict ure. The text annotation was given in the L2 since participants represented 18 different first languages. The pa rticipants read an English short story embedding the 14 target words on the Internet for the purpose of reading comprehension. They were randomly assigned to one of three annotation conditions. Three unexpected vocabulary posttests were admi nistered both immediately and two weeks after the treatment: a definiti on supply test, a pictur e recognition test, and a word recognition test. A 3x2 ANOVA was performed to examine the scores in immediate posttests and delayed posttests. In terms of single annotation effect, the claims made by Kost et al (1999), i.e. that it was dependent on the posttest task type, was not fully supported by Yoshii (2000). The pi cture-only group did cons istently be tter than the text-only group in all the posttests, and th e difference was statistically significant in the delayed picture recognition test. This mi ght explain the positive effect of visual annotation as suggested by Yoshii (2002), “the results of the study en courage the use of pictures as alternative or as accompaniments to textual cues” (p. 46). In comparison to the single annotations, the combination group demonstrated better performance on both

PAGE 54

39 immediate and delayed posttests. For th e immediate posttests, the combination group performed significantly better than the text annotation group in th e picture recognition task. The combination group also scored sign ificantly higher than the picture annotation group in terms of definition supply task. For the delayed posttests, significant difference was indicated in the picture recognition betw een the combination group and the text-only group, but no significant diffe rence was found in the word recognition test or the definition supply test. Overall, the combination of text and pi cture proved to be the most effective annotation condition for both vocabulary learning and retention. The positive effect of picture annotati on on L2 vocabulary learning, particularly in combination with text annotation, wa s also reflected in Kang’s (1995) study. Involving 98 elementary EFL Korean student s, Kang (1995) compared three computerbased instructional approaches: L1 translation, L1 translation and picture, L1 translation and picture plus an example sentence. Th e target words in the first and second approaches were given in pair-wise definitio n-based form without any context, while the third approach (context-based) embedded th e target words in contextual sentences accompanied by explanatory pictures. Vo cabulary definition recall and knowledge transfer were used as immediate posttests and delayed retention tests. On the recall measure, a statistically significant differe nce was found among th e groups, specifically, the context-embedded words with picture gr oup scored the highest, followed by the text and picture group, with the text only group being the lowest. The third group also showed significantly better performance than the othe r groups on the retenti on tests, suggesting that a contextualized approach using both text and picture was more effective in promoting long-term recall of vocabulary definitions. On the knowledge transfer

PAGE 55

40 measure, students were asked to use the ta rget words communicatively in computergenerated simulations resembling real world situations. The groups with access to text and picture performed much better than the group with access only to text, confirming the use of multiple annotation combining text a nd picture. Interestingly, the study used a listening comprehension test to measure stud ents’ ability of rec ognizing newly learned vocabulary in a spoken context. The result s showed that the groups with access to pictures outperformed those without. Yeh & Wang’s (2003) study investigated th e effectiveness of different vocabulary annotations on vocabulary learning for EFL college students in Taiwan. Freshmen (N=82) from science programs participated in the study. A computer courseware was designed to introduce Thanksgiving to 82 fres hmen enrolled in the freshmen English class. Three annotation types were presen ted for unknown vocabulary: text annotation (Chinese translation and English explanatio n), text and picture annotation, text and picture and sound. For the sound component, a na tive speaker read th e word, spelled the word and read the sentence in which the target word was planted. Three tasks were used in the posttest: word association questions, multiple-choice questions on word meanings, and a cloze test. Analyses of the participants’ performance on the posttests indicated that the text and picture annotation was the most effective for vocabulary learning, and the different effectiveness of the three annot ation types was further confirmed by an ANOVA analysis to be statistically significant, s uggesting generally consis tent findings with previous studies (Kang, 1995; Kost et al 1999; Yoshii, 2000). However, the addition of sound to the combination of text and picture pr oved not to be effec tive, and the result was even worse than the text-only annotation as indicated by the group mean scores: highest

PAGE 56

41 mean score of 23.41071 from group 2 (text a nd picture), followed by mean score of 22.44444 from group 1 (text-only), and lowe st mean score of 20.77777 from group 3 (text and picture plus sound). Furthermore, Fisher’s LSD pair comparison indicated that the text-picture annotation group significan tly outperformed the text-picture-audio annotation group. Instead of using written reading materi al in the above-mentioned studies, Jones (2003) used an aural text to investigate the effects of multimedia a nnotations in a pretestposttest design. The participants were 171 English-speaking students in a second semester French class. Given 14 minutes to lis ten to a passage, the participants had access to one of four annotation conditions: (1) no an notation; (2) text annotation; (3) picture annotation; and (4) text and picture annotation. After the treatment, students took an immediate multiple-choice vocabulary posttest and a delayed vocabulary post-test four weeks later. On both the immediate and delaye d posttest, when eith er text or picture (verbal or visual) annotation was available, students’ perf ormance on the vocabulary test was better than when no annotation was availa ble, which confirmed the findings of the effectiveness of using annotat ions to facilitate second la nguage vocabulary learning from other studies (Davis, 1989; Hulstijn, Ho llander & Greidanus, 1996; Jacob, 1994). No significant difference was found between th e text-only annotati on and picture-only annotation. This echoed mi xed findings from Kost et al (1999) and Yoshii (2000). Furthermore, the results show ed the highest performance in the immediate posttest by students from the text and picture annotati on group (mean score was 19.75), statistically different from all the other groups, alt hough this difference was not found to be significant between the combin ation group and the pictureonly group in the delayed

PAGE 57

42 posttest. Qualitative techniques were also employed in this study to profile students’ perception to the annotation types. Themes emerging from the interview data supported the effectiveness of multiple annotations in learning second language vocabulary. In summary, the use of dual or multiple a nnotations is supported to be helpful for second language vocabulary acquisition, especial ly when both verbal and visual (text and picture) are presented in the annotation. The supportive nature of multimodal annotation is also supported by student preference (Kang, 1995; Jones, 2003; Yeh & Wang 2003). Echoing Kellogg & Howe’s (1971) statement th at multiple annotati on types utilizing visual imageries, rather than single mode annotations, can facilitate the learning and retrieving process of second language vocabulary, Plass et al (1998) remarked that “the organization and interpretation of two different forms of mental re presentations enhance retrieval performance by providing multiple retrieval cues” (p. 34). Annotation and L2 Reading Comprehension The pilot study of Lomicka (1998) ex amined the effects of multimedia annotations on L2 reading comprehension. In her study, 12 college stude nts enrolled in a second-semester French course, and read a text under three multimedia annotation conditions: full glossing, limited glossing a nd no glossing. The limited glossing gave both definitions in French and translations in English of the glossed word; in addition to that, images, references, questions, and pr onunciation concerning the glossed word were added to the full glossing. Online think-al oud was employed as a reading comprehension measurement, and the results of the st udy indicated that full multimedia glossing promoted a deeper level of text comprehens ion. However, their use of the glosses was

PAGE 58

43 oriented toward the goal of translation a nd paraphrasing in order to achieve a minimal level of comprehension Summary Based on the assumption that visual aids such as pictures can facilitate second language vocabulary learning, studies have been carried out to compare picture annotation to text annotation (Kost et al., 1999; Jones, 2003; Yoshii, 2000). The findings are mixed, with text annotation being bette r in word recognition tasks, and picture annotation better in picture recognition tasks in Kost et al (1999), but Yoshii (2000) found consistently better performance by the picture-only group than the text-only group in both word-recognition and picture-r ecognition tasks, and Jones (2003) found no difference between text annotat ion and picture annotation. However, when combined, the dual a nnotation of text and picture was unanimously argued to be better than single annotations, especially better than text-only annotation (Kang, 1995; Kost et al. 1999; Jones, 2003; Yoshii, 2000; Yeh & Wang, 2003). In this respect, the dual-coding theory (Paivio, 1971, 1986, 1990) is supported empirically in the realm of second langua ge vocabulary learning. The simultaneous activation of both verbal and visual cognitive systems offers learners dual channels to process the given information, and, therefore, have dual channels to retrieve information. Furthermore, the addition of audio (the pronunciation of the target word) does not seem to enhance second language vocabulary; quite the contrary, it influences the learning result in a negative manner (Yeh & Wang, 2003). One possible explanation is that the information delivered through di fferent modalities (audio, and visual) might

PAGE 59

44 overload the cognitive processing; on the othe r hand, the learner prefer ence of particular types of information presentation contribu ted to the finding. In Yeh & Wang’s (2003) study, the particular group of pa rticipants (Chinese EFL learne rs) contributed to the effect because Chinese EFL learners are usually mo re visual learners (Chen, 1998, cited in Yeh & Wang 2003). Further research to separate audio and visual in combination with text needs to be addressed. In terms of reading comprehension, the e ffect of dual annotation of both text and picture is not clearly establis hed due to the scarcity of st udies. Further research is necessary in this respect. Video Annotation As streaming video becomes easier and more widely used in second language teaching and learning, annotation research ha s expanded its multimedia application of using static pictures to the use of animated videos. Studi es have examined the effect of video annotation coupled with text in compar ison to text-only, and to text coupled with pictures, specifically, the effect of diff erent media types on second language vocabulary acquisition. Annotation and L2 Vocabulary Learning Chun and Plass (1996a) reported the result s of three studies with a total of 160 second-year German students from three unive rsities in California. The reading text was a 762-word German text compiled using th e multimedia program CyberBuch, covering 11 screen pages. A total of 82 words were a nnotated in the program in three conditions:

PAGE 60

45 (1) text (L1 definition) only, (2) text and picture, and (3) text and short video. Every annotated word was indicated by a unobstrusive symbol (o). When students clicked on the word, icons marking the available annotati ons appeared, and the chosen annotation appeared on the left of the computer screen. A within-subjects design was used for all three studies to eliminate the variable of individual differences. Study 1 and 2 used definition-supply (English equivalents) as th e vocabulary posttest, and study 2 employed word recognition in the posttest. The first focus of their studies was whether the availability of multiple annotations would result in better incidental vocabular y learning. In study 1, students (N=36) were unexpectedly asked to provide En glish equivalents to 15 target words (five each from the three annotation conditions) immediately after reading and two weeks later. The mean scores and percentages of correct answer s were higher than what was expected for typical incidental vocabulary learning. The mean score of 3.89 (25.9%) on the immediate test and mean score of 3.97 ( 26.5%) on the delayed test were higher than the scores of their pretest (2.06/13.7%, a nd 1.94/12.9% respectively). The learning result was also better than the incidental vocabul ary learning percentage of 5-15% reported by Coady (1993) and 5-21% by Knight (1994). Stu dy 2 (N=103) used 36 target words in the posttest and found that 24.1% of them were retained in the immediate posttest. This number is better than both Coady’s ( 1993) 5-15% and Knight ’s (1994) 5-21%. The results confirmed that using multiple annotat ions facilitated second language vocabulary learning. The word-recognition test used in stud y 3 offered much better retention rates of 77% (immediate posttest) and 77.1% (delayed posttest). The higher score on recognition was expected because recognition task tend to be easier than the definition-supply task.

PAGE 61

46 The effects of different media types us ed for annotation was the second issue to be examined in Chun and Plass (1996a). In study 2, a significant difference was found in immediate retention scores for words between text-picture annotations and text-only annotations or text-video a nnotations, as shown by the or dering of mean scores and correct percentages from the hi ghest to the lowest: text-pic ture (3.75/31.2%), text-video (2.76/23%) and text-only (2.15/17.9%). The re sult supported previous studies that showed pictures were effec tive in annotating words for s econd language learners (Kost et al 1999; Yoshii, 2000; Kang, 1995). In particular, dual annotation that combined text and visual information (such as pict ure or video) is shown to be more effective than text-only annotation, which supported the dual-coding explanation that words annotated with both verbal and visual information will be more efficiently memorized. Moreover, in dual annotations, text-picture seems to be more effective than textvideo. The difference is explained by the auth ors in terms of viewing time of the two annotation types, text-picture annotation allowed longer vi ewing time and better mental representation of the presente d information, and text-video annotation was comparatively shorter and offered less time for viewing and mental processing. To further demonstrate the effectiveness of text-picture annotation, st udents were asked to ar ticulate their use of retrieval cues from different annotations. Their answers consistently favored picture annotation, though the difference was only margin ally significant in comparison to text annotation, and was not signifi cant in comparison to video annotation. The trend was claimed by the authors to correspond to the te ndency for better word recall from textpicture annotations. No delayed posttest, however, was used to check the long-term retention of vocabulary learning. It is, therefore, uncertain wh ether the better performance

PAGE 62

47 by the text-picture group in th e immediate posttest would be retained by the time of the delayed posttest. However, the same pattern of text-pic ture superiority was not found in study 3. Due to the small number of participants (N= 21), no statistically signi ficant difference was found among the three groups. Instead, very clos e mean scores and percentages of correct answers for target words were found am ong the three annotation conditions. The immediate posttest exhibited 75.1% correction rate for text annotation, 76.2% for textpicture, and 81.4% for text-video; the delaye d posttest indicated a similar range of 75%, 81% and 77.2%. Though the result further supported the use of dual annotations combining text and visual, it seems inclusiv e to say one combination is favored over the other because at the immediate posttest, the text and video group outperformed the text and picture group, while the result switched in the delayed posttest. Upon a closer look, the better performance by the text-picture group in study 2 resembled that of the delayed posttest in study 3 favoring text and picture annotation, but since no delayed posttest was conducted in study 2, the interpre tation is debatable. In ad dition, the difference reported in study 1 between the two dual annotation type s was not statistically significant. The comparison between picture and vi deo as two annotation types was further explored in Al-Seghayer (2001) Similar within-subj ects design was used here with 30 intermediate-level ESL students. Three annota tion types were availa ble: text annotation, text-picture annotation, and text -video annotation. The 30 target words, 10 in each of the annotation conditions, were chosen based on cr iteria of frequency, grammatical category, morphological category, and visu al complexity. After reading the text via the multimedia program with access to different annotations, students took two vocabulary

PAGE 63

48 posttests: recognition and production. The r ecognition test, in multiple-choice format, asked students to select from four alternat ives the right English word corresponding to the provided information matching the modality in which the word was annotated. The production test required students to offer definitions in English for the 6 selected target words, 2 words from each of the three annotation conditions. The results of the two vocabulary posttest s presented the best performance by the text-video group. The mean and percentage of correct answers from highest to lowest are as follows: text-video (4.3/87%), text-pictu re (3.3/67%), and text -only (2.7/53%). The advantage of the text-video group was also shown by the distribution of frequency score of each annotation mode, more closely clustere d for the text-video mode than the textpicture mode, with the text-only mode having th e widest range. The most frequent mean score was 6 for text-video mode, 4 for text -picture mode, and 3-4 for text-only mode. Group-wise differences were further examined by the Friedman test, and the results indicated significantly better performance by the text-video group than both the text-only and text-picture groups, but no significant difference was found between the text-only group and text-picture group. The effects of different annotation modes were also examined through qualitative data from questionnaires and interviews. Th e questionnaire data showed that students voted video the most helpful annotation m ode (86.6%), followed by picture (70%) and text (10%). The same preference was reflec ted in the interview in which 90% of the participants agreed that vide o annotations presented the word meanings more clearly than picture or text annotations. These qualitative results co rresponded to the results of vocabulary posttests, favori ng text-video annotation.

PAGE 64

49 On one hand, the result of text-video annotation leading to better vocabulary retention than text-only annot ation is consistent with th e dual-coding theory proposed by Paivio (1971, 1986, 1991). The information presen tation in both verbal and visual forms helps learners to construct referential conn ections between the two mental representation processes and achieve better le arning. However, the dual-coding effect was not strongly supported by the text-picture annotation in th is particular study. On the other hand, the finding of text-video being more effective th an text-picture was contradictory to Chun and Plass (1996a). The results obtained by Chun and Plass (1996a) demonstrated higher vocabulary recall scores by text-pic ture group than text-video group. Al-Seghayer (2001) explained the different findings of his study from Chun & Plass (1996a) to issues such as the first langua ge of the participants, language used in the annotations, types of video c lips, target words, and voca bulary tests used in the two studies. In explaining the bett er effect by text-video, Al-S egheyer (2001) posited three theoretical explanations First, he pointed out that th e stimuli offered by video (dynamic visual mode) provided “a gest alt” (p. 224) that helped learners to build mental representations in learning the new vocabular y. This explanation ag reed with Hanley, Herron & Cole (1995) who argued for video c lips as more effective organizers than pictures in foreign language learning. Sec ond, the video annotation stimulated learners’ curiosity and interest level that consequently led to more focused attention and better retrieval. The third reason was related to the redundancy hypothesis (Sherwood, Kinzer, Hasserlbring, & Bransford, 1987) that proposed text-video redunda ncy to facilitate information processing.

PAGE 65

50 Annotation and L2 Reading Comprehension Chun & Plass (1996b) is the only study that examined the effects of multimedia annotations, in particular thos e involving text-picture and text-video annotations, on L2 reading comprehension. Using the same data set from Chun & Plass (1996a), this study hypothesized that dual-coded annot ation (verbal and visual) w ould lead to better reading comprehension than text-only annotation. Following the after-reading vocabulary test, participants were asked to wr ite recall protocols in Englis h (L1), summarizing the story they had just read in the multimedia program The recall was consequently scored based on 12 main idea units or propositions of the reading text. One point was given to each mentioned proposition, and scores for each st udent and each propositi on were tallied for analysis. In the analysis, a correlation was sou ght between annotated words used by students in the recall protocol and the frequency with whic h these words were mentioned in the recall. The twelve pr opositions, therefore, were cat egorized into three groups: propositions that contained words annotated both verbally (text) and visually (picture or video), propositions that contained words annot ated verbally (text), and propositions that contained words that were not annotated. The mean scores for the three categories were 2.32 (verbal and visual annotat ion), 1.53 (verbal annotation) and 1.95 (no annotation). A T-test showed statistically significant differences among the three categories, suggesting the greater recall in reading comprehension of words with both verbal and visual annotations during reading. In other words, words dually annotated are easier and better remembered than words annotated only verbally or no annotated at a ll. This supported the

PAGE 66

51 dual-coding theory (Paivio, 1971, 1986, 1991) and Mayer’s (1997, 2001, 2003) cognitive theory of multimedia learning. Another aspect worth mentioning in Chun & Plass (1996) was the investigation of a possible relationship between vocabular y acquisition and reading comprehension. Specifically, it was hypothesized that better reading comprehension would be shown by students who looked up more annotated word s as indicated by higher scores on the written recall protocol. Unfortunately, no co rrelation was revealed by the data. However, a moderate correlation was found between participants’ vocabulary knowledge and reading comprehension, as shown by the bett er reading comprehension of students who scored higher in the vocabulary posttest. Summary To summarize, the studies reviewed in this section utilized bo th picture annotation and video annotation coupled with text a nnotation. The results support the dual-coding theory (Paivio, 1971, 1986, 1990) that annotatio n provided in both verbal (text) and visual (picture or video) fo rms facilitated L2 learners’ vocabulary acquisition. It also reflects the cognitive theory of multi media learning proposed by Mayer (1997, 2001, 2003) that claims that meaningful learning e ngages learners in bot h verbal and visual cognitive processing systems. However, wh ich dual-annotation is more effective on vocabulary learning was reported differently by the two studies, with Chun & Plass (1996a) favoring text-picture, though the differe nce was not always si gnificant, and AlSeghayer (2001) supporting text-video. Different theoretical explanations have been provided for the evidence of the more effective dual-annotation mode.

PAGE 67

52 In terms of reading comprehension, only Chun and Plass (1996a) addressed this problem. Their finding appears to support the f acilitative role of dua l-annotation in postreading recall and reading comprehension. Howe ver, text-picture and text-video are put into one category of visual-v erbal annotation, and no distinct ion was articulated between text-picture and text-video. As pointed out by Chun & Plass (1996a), “[T]he difference between pictures and videos as annotations require separate analyses and preclude combining them into a composite category of visual annotations” (p. 190). Moreover, the effect of dual annotation of text -picture is not yet cl early established due to the scarcity of studies (Lomicka, 1998); more research separating the two du al-coding modes is necessary to address any possible differentia l effects on L2 learners’ overall reading comprehension. Therefore, more empirical ev idence is needed to clarify whether there are different effects between text-picture and text-video annotations on L2 reading comprehension. Audio Annotation It is worth noting that lit tle research has been done in audio annotation. Audio annotation gives pronunciation, sample sentence definition or meaning of a target word in spoken form. It has never been studied se parately from other annotation modes, but mostly as an additive component. The only format in which audio annotation has been studied is the pronunciation of target words. Findings on audio annotation are rather mixed and uncertain. On one hand, Svenconis an d Kerst (1995) suggest ed that the use of audio could significantly impr ove vocabulary learning, especially when coupled with a second technique such as semantic mapping. On the other hand, Chun and Plass (1996a)

PAGE 68

53 challenged the effect of audio annotation. In addition, it seems that the addition of an audio component to other annotations is not effective (Yeh & Wang, 2003); instead it distracts learners’ attention. Table 24 summaries audio annotation studies. Svenconis and Kerst (1995) investigated the effectiv eness of semantic mapping techniques in L2 vocabulary learning in a hypertext environment. The participants (N=48) of the experiment were Englishspeaking high school st udents in grades 9 through 12 learning Spanish as a second language. The 72 target words were presented in two methods, word listing and semantic mapping. In the word listing method, the words were presented in alphabetic al order on the computer scr een, without any indication of the relationship among the words. In the semantic mapping method, the words were presented in semantic maps that visually ex plained the relationships among the words. In order to investigate whether the addition of sound would improve vocabulary learning, pronunciation of the target words was used as a factor. The pa rticipants were, therefore, divided into four experimental groups: (1) word listing without sound; (2) word listing with sound; (3) semantic mapping without s ound; and (4) semantic mapping with sound. In the multiple-choice vocabulary posttest, no significant effect was found for the word presentation method, which suggested that se mantic mapping does not necessarily lead to better vocabulary retention than the tradi tional word listing method. But the group of semantic mapping with sound produced the highest overall mean score of 20.38, higher than the other three groups (word listing without sound had a mean score of 19.82, word listing with sound 18.81, and semantic mapping without sound 17.98). An interaction was identified between vocabulary presentation method and sound factor: the combination of semantic mapping and sound seemed to be posi tively facilitating L2 vocabulary learning,

PAGE 69

54 while the addition of sound to word listing negatively influenced vocabulary learning. The results suggested that audi o annotation could be a signific ant factor in L2 vocabulary learning if coupled with a s econd variable, such as semantic mapping. In addition, results of the two delayed posttests demonstrated the increasing effect of sound over time. Chun and Plass (1996a) challenged the positive effect of audio annotation. In their studies, an audio component wa s added to three different an notations types (text, textpicture, and text-video); that is, a German native speaker pronounced each target word. Of the three successive studies, participants from study 1 and 2 were asked to report their use of retrieval cues for vocabulary learning. Among the reported cues of text, picture, video and sound, sound was used the least as a retrieval cue, as shown by the percentage of correct answers on vocabular y test, 2.2% and 4.3% for sound cue in the immediate and delayed posttests for study 1, and 0.6% in the immediate posttest for study 2. The authors suggested that the audio component wa s not useful in lear ning vocabulary since it showed very limited importance as a retrieval cue. In comparison to the use of word pr onunciation in the above two studies, Yeh and Wang (2003) elaborated the a udio annotation in which a na tive speaker read the word, spelled the word and read the sentence th at embedded the word. Three types of vocabulary annotation were examined: text an notation, text-picture annotation, and textpicture-audio annotation. Participants (N= 82) were Chinese EFL learners. Analyses of the participants’ performance on the posttests indicated that the text-picture annotation was the most effective for vocabulary learni ng, and the text-picture -audio annotation was the least effective. This show ed that the addition of audio annotation to the text-picture annotation was not effective, and it was even worse than the text-only annotation as

PAGE 70

55 indicated by the group mean scores: highest mean score of 23.41071 from text-picture annotation, followed by mean score of 22.44444 from text-only annotation, and lowest mean score of 20.77777 from text-picture-audio annotation. The difference between textpicture and text-picture-audio was tested by Fisher’s LSD pair comparison to be significant. Yeh and Wang (2003) offered three po ssible reasons for the relative ineffectiveness of text-pic ture-audio annotation. The fi rst reason was L1 processing mechanism transfer as claimed by Chen ( 1998, cited in Yeh & Wang, 2003) that Chinese EFL learners used more visual strategies th an English native speaker s and were therefore less skillful in using the provided audio information. This was confirmed by high preference of visual learning style over low a uditory learning style by the participants in the questionnaire data. Thus, Chinese students did not effectively process the information provided by the audio annotati on. The second reason resided in the fast speech rate of the audio annotation. Coupled with the visual learning style of Chinese students, the fast speech rate distracted the participants and exceeded their listening proficiency. And finally, the combination of text, picture and audio failed to give participants enough time to proces s the available information. In summary, studies in audio annotation have mainly engaged the use of pronunciation of the target words, and their findings are inconsistent. Audio is a different sensory modality from visual modality such as printed text and pictures. It should be treated separately as to its effect on learning. In the studies reviewed above, the audio annotation was never studied by itself. Studi es of audio annotation should include not only the pronunciation, but also the definiti on or meaning of the target words. As

PAGE 71

56 suggested by Chun and Plass (1996b), the au ditory component needs to be further addressed in terms of its e ffect on vocabulary learning. Annotation Studies Sumary Different annotations have been investig ated in L2 vocabulary learning in both printed texts and multimedia texts. The e ffects of text annotation on L2 vocabulary learning and reading comprehension are ra ther mixed (Davis, 1989; Hulstijn, 1992; Hulstijn et al 1996; Jacobs, 1994; Jacobs et al. 1994; Joyce, 1997; Ko, 1995; Rott, Williams, & Cameron, 2002; Watanabe, 1997). Based on the assumption that visual aids such as pictures can facilitate second language vocabulary learning, studies have been carried out to compare picture annotation to text annotation (Kost et al ., 1999; Jones, 2003; Yoshii, 2000). The findings are, however, inconsistent, despite the fact that dual annotation of text and picture was unanimously argued to be better than single annotations, especially better than text-only annotation (Kang, 1995; Kost et al. 1999; Jones, 2003; Yoshii, 2000; Yeh & Wang, 2003). The effect of dual annotat ion of text and picture on L2 reading comprehension is not clearly established due to the scarcity of studies. Studies utilizing both pictur e annotation and video annot ation coupled with text annotation further support the dual-c oding theory (Paivio, 1971, 1986, 1991). Their results of L2 vocabulary learning also c onfirm the cognitive theory of multimedia learning (Mayer, 1997, 2001) that maintains that meaningful learning engages learners in both verbal and visual cognitive processing sy stems. However, different studies found advantage of different dual annotations, a nd offered different interpretations (Al-

PAGE 72

57 Seghayer, 2001; Chun & Plass, 1996a). In ad dition, in both text-pic ture and text-video annotations, verbal (text) and nonverbal (picture or vide) m odes are used to present the information, but only visual sensory-motor is en gaged (text, picture, video) in registering the information (Mayer, 1997, 2001) On the other hand, audio is a different sensory modality from visual modality such as printed text and pict ures, but studies in audio annot ation have mainly engaged the use of pronunciation of the target words, and th eir findings are inconsis tent. It should be treated separately as to its effect on learning. In additi on, studies of audio annotation should include not only the pronunciation, but also the definition or meaning of the target words. Furthermore, the addition of audio (pr onunciation of the targ et word) does not seem to have a definite effect on L2 vocabulary learning (Chun & Plass, 1996a; Svenconis and Kerst ,1995; Yeh & Wang, 2003). One possible explan ation is that the information delivered through different modalities (audio, verbal and visual) might overload cognitive processing; therefore, further research to separate audio and visual in combination with text needs to be addresse d (Mayer, 2001). In comparison to text-picture annotation, text-audio annotation uses verbal mode to present information, but engages both visual and auditory sensory modality channels to register the information. Cognitive Theory of Multimedia Learning The development of information technologi es in recent years has engendered an increasing application of instructional technology in education. Inspired by Wittrock’s (1974, 1990) generative theory and Paivio ’s (1971, 1986, 1990) dual-coding theory,

PAGE 73

58 Mayer (1997, 2001, 2003) proposed the cognitive theory of multimedia learning that extends the two theories and maintains a theoretical foundation for interpreting how people learn in multimedia environments. It has been applied and extended by Plass, Chun, Mayer and Leutner (1998) to multim edia second language learning. Generative Theory of Learning In 1974, Wittrock introduced the generative model of learning. His model posited that human learning was “a function of the abstra ct and distinctive, c oncrete associations which the learner generates betw een his prior experience, as it is stored in long-term memory, and the stimuli” (p. 89). In othe r words, this model emphasizes the active integration of new ideas with the learner's existing schemata, particularly by using four types of learning strategies: 1) recall, 2) integration, 3) organization, and 4) elaboration (Wittrock, 1974, 1990). In recall, the learner uses techniques such as repetit ion, rehearsal, review or mnemonics to pull information out of long-te rm memory. Methods such as paraphrasing, summarizing, or questions-gener alizing are applied to inte grate new information with prior knowledge; thereafter, the learner utilizes tec hniques such as outlining, categorization, clustering or concept mapping to connect new information to prior knowledge in an organized way. Finally, give n an established c onnection between the new information and the learner’s prior knowle dge, the new information is synthesized and elaborated through mental images, writi ng, visual displays and similar methods. Individual strategies are used alone or in collaboration with others to achieve a learning goal.

PAGE 74

59 Dual-Coding Theory The dual coding theory establishe d by Paivio (1971, 1986, 1990) offers an account of both verbal and non-verbal cogni tive processes. Paivio (1986) stated, "Human cognition is unique in th at it has become specialized for dealing simultaneously with language and with nonverbal objects and events. Moreover, the la nguage system is peculiar in that it deals directly wi th linguistic input and output (in the form of speech or writing) while at the same time serving a symbolic function with respect to nonverbal objects, events, and behaviors. Any representational theory must accommodate this dual functionality" (p 53). Dual-coding theory is deeply infl uenced by the imagery-based mnemonic technique that was rooted in the ancient method of loci (Sadoski & Paivio, 2001). The mnemonic technique applies verbal-imaginal dual coding in which the targeted words (verbal) are transformed into non-verbal imag es, and images are consequently translated back into verbal in recall ac tivities. This technique is wide ly used in learning a second or foreign language. One major assumption of dual-coding theory is that human mental process derives perceptions from various sensory modalities, and these modality-specific representations can be divided into two categor ies: verbal representations as those derived from speech or writing, and non-verbal representa tions as those derived from images (Sadoski & Paivio, 2001). In other words, the theory entertai ns two cognitive subsystems, one represents

PAGE 75

60 and processes language input, and the other speci alizes in interpreting nonverbal objects/events such as imagery. Dual-coding theory identifies three types of processing: (1) re presentational, the direct activation of verbal or non-verbal repr esentations, (2) referent ial, the activation of the verbal system by the nonverbal system or vice-versa, and (3) a ssociative processing, the activation of representations within the same verbal or nonverbal system. The two subsystems described by dual-coding theo ry function independently, parallelly and interconnectedly (Paivio, 1986, 1990; Sadoski & Paivio, 2001). Cognitive Theory of Multimedia Learning Drawing on Wittrock’s (1974, 1990) generative theory and Paivio’s (1971, 1986, 1990, Clark & Pavio, 1991) dual-coding theo ry, Mayer (1997, 2001) proposed and developed the cognitive theory of multimedia learning that consists of three important assumptions: dual channels, limited capaci ty, and active processing (Mayer, 2001). The dual channels assumption argues that humans possess separate channels for processing visual/pictorial and auditory/verba l information (Mayer, 2001). In this respect, Mayer combines the presentation-mode approach and sensory-modality approach. On one hand, consistent with the distinction be tween verbal and nonverbal systems that respectively process verbal and nonverbal information (Paivio, 1986), Mayer (2001) conceptualizes the presentation-mode by dist inguishing verbally presented information such as printed text or spoken words, and nonverbally presented information such as pictures or animations. On the other hand, Mayer (2001) adopts the sensory-modality approach (Baddeley, 1986, 1992, 1999) by differen tiating the visual channel (eyes) and

PAGE 76

61 the auditory channel (ears). The visual cha nnel processes information such as pictures, video or printed texts, while the auditory channel processes information such as spoken words or background sounds. In the second assumption, each channel is assumed to have limited cognitive capacity. In other words, when facing new information presented via multimedia, learners are limited in the amount of information that can be processed in either the visual or auditory channel at one time. For example, if the learner is presented a series of pictures on the computer screen, although all th e pictures may be perceived by the visual modality, only a few images can be brought into and held in the working memory at any one time. The same applies to the auditory modality. For example, in language learning, when a learner listens to a dialog, only a por tion of the dialogue can be selected to be processed in the learner’s wo rking memory, with possible su bsequent integration into long-term memory. In active processing, Mayer borrowed fr om generative learning (Wittrock, 1974, 1990) the concept that meaningf ul learning engages three esse ntial cognitive processes, namely, selecting relevant information from newly presented materials, organizing the selected information into coherent mental representations, and integrating organized information with prior knowledge (Mayer, 1997, 2001). In typical multimedia learning, at the first stage, when presented with a mu ltimedia message, the lear ner pays attention to the presented materials, pr ocesses the information thr ough the auditory and visual channels, and brings selected words and pictures from sensory memory into working memory. At the second stage, the selected words and pictures are processed and organized in the working memory into cohe rent verbal and nonve rbal representations;

PAGE 77

62 sometimes the verbal and nonverbal representati ons interact with each other referentially (Paivio, 1971, 1986, 1990). At the third stage, connections are esta blished between the verbal and nonverbal models in relati on to the learner’s existing knowledge. Figure 2-1 presents a model of cognitiv e theory of multimedia learning (Mayer, 1997, 2001, 2003). Three memory stores are represented: sensory memory, working memory and long-term memory. In multimedia learning, materials are normally presented in verbal (includi ng written text and spoken text) and visual (including pictures, video, animation) forms. These pres ented materials enter the learners’ sensory memory through the visual channel (eyes) a nd the auditory channel (ears). In other words, the eyes register visual images (inc luding pictures, printed texts) and bring them into the visual sensory memory; the ears regi ster auditory images (including spoken text or background sounds) and bring them into th e auditory sensory memory. Due to limited cognitive capacity, only selected sounds and images enter the working memory where they are temporarily held, manipulated a nd constructed into verbal and visual representations. Finally, long-te rm memory is activated to integrate the newly selected and manipulated information. In this model, the learner is rega rded as an active knowledge constructor who selects, organizes and integrating both vi sual/pictorial and auditory/verbal information such as printe d texts, pictures, or spoken words (Mayer, 1997, 2001). Extension of Cognitive Theory of Multimedia Learning The use of actual objects or imagery t echniques in foreign language vocabulary learning is well documented in resear ch (Davis, 1989; Kellogg & Howe, 1971),

PAGE 78

63 especially along with the advance of multim edia application in language teaching and learning. Plass et al (1998) argued for an extension of the cognitive theory of multimedia learning to second language learning. In multimedia language learning, both visual and auditory sensory modalities are engaged to process information that is consequently represented in both verbal and visual models. For example, when presenting a new word via a multimedia program, both the word in printed text and a picture describing the words can be presented on the computer scr een; therefore the L2 learners can select relevant information (either text, picture, or both text and picture) to process in the working memory and integrate the newly selected information into the long-term memory, that is to memorize the meaning of the word. Multimedia Sensory Working memory Long-term Presentation Memory Memory Figure 2-1 Cognitive Theory of Multimedia Learning (adapted from Mayer, 1997, 2001, 2003). Their extension addressed two aspects of multimedia second language learning: vocabulary learning and reading comprehens ion. In terms of vocabulary learning, the extension maintained that “second language lear ners possess two separate verbal systems and a common visual system” (Plass et al 1998, p. 26). They further argued that second Verbal Ears ( Auditor y) Eyes ( Visual ) Images Sounds Verbal model Pictorial model Prior Knowled g e Visual Integration

PAGE 79

64 language learners “learn new words when they can establish a direct connection between the words in their native la nguage, the corresponding picture of an object or action, and its foreign equivalent” (Plass et al 1998, p. 26). The second focus of the extension is reading comprehension. Based on the arguments of Wittrock (1974, 1990), the read ing process engages the referential connections between the mental representati ons of information or ideas presented in different modes such as the visual mode and verbal mode. In multimedia language learning, the presentation of information in multiple modes might facilitate L2 learners’ reading comprehension (Chun & Plass, 1996b; Ommagio, 1979). Application of Cognitive Theo ry of Multimedia Learning to Multimedia Annotation Mayer (2001) defines multimedia as the “presentation of material using both words and pictures” (p. 2). Words refer to mate rials that are presented in verbal forms, such as printed text or spoken text; pictures refer to materials that are presented in pictorial form such as static pictures, video, or animation. Mayer (2003) refined the definition by breaking it down into what, how and why: what refers to the instructional content, how refers to the presentation method that entails both the use of computers and different instructional methods such as using different modali ties, and why refers to the purpose of multimedia learning being to promote learning and foster changes in learner’s knowledge and performance. Multimedia learning is designed in accord ance with the cognitive process of the human mind. Dual-coding theory argues for tw o subsystems of information processing, one for verbal information and one fo r visual materials (Paivio, 1971, 1986, 1990).

PAGE 80

65 Therefore, multimedia is taking advantage of the full capacity of human information processing systems. Mayer (2001) explains th e advantages of multimedia learning both quantitatively and qualitatively: (1) presenting the same material in two channels is better than one channel by offering the information twice (redundancy princi ple); (2) words and picture complement each other in facilitating human mental representations of both visual and verbal information. Multimedia annotation, as an instructiona l intervention in L2 vocabulary learning, conforms to the definition of multimedia a nd applies the cognitive theory of multimedia learning. This section will explain this point by breaking the multimedia annotation into three different modes: text, picture and audio. Multimedia Sensory Working memory Long-term Presentation Memory Memory Figure 2-2 Processing of text annotation In Figure 2-2, the shaded boxes indicat e the cognitive processing of text annotation. When the text annot ation appears on the computer screen, L2 learners, using eyes the visual sensory modality, perceive the printed text, select fr om available visual information, and send it to working memory in the form of sensory images. The sensory Text annotation Ears Eyes Images Sounds Verbal Model Pictorial Model Prior Knowledge Picture annotation

PAGE 81

66 images of the printed word s are converted to correspondi ng sounds that are further organized into verbal models of representation. The shaded boxes in Figure2-3 describe the path for processing audio annotation. Audio annotation is perceived by the ears and is held briefly in the L2 learner’s auditory sensory memory. If the sounds are given ac tive cognitive processing, some of them will be selected to be included in the sound ba se of the working memory. Then the selected sound fragments will be organize d into coherent mental repr esentations, and built into the verbal model in collaborati on with the prior knowledge Multimedia Sensory Working memory Long-term Presentation Memory Memory Figure 2-3. Processing of audio annotation Multimedia Sensory Working memory Long-term Presentation Memory Memory Figure 2-4. Processing of picture annotation Audio annotation Ears Eyes Images Sounds Verbal Model Pictorial Model Prior Knowledge Picture annotation Text annotation Ears Eyes Images Sounds Verbal Model Pictorial Model Prior Knowledge Picture annotation

PAGE 82

67 As shown by Figure24, the shaded boxe s explain how picture annotations are processed. When the annotation is given in pict ure form, L2 learners perceive the pictures via visual sensory modality, select relevant vi sual information and hold it temporarily in the working memory. Additional processing wi ll bring the visual information into pictorial model and organize it into mental representation. In th e next step, active cognitive processing connects the new pictoria l representation to ex isting knowledge that results in integrated learning outcome. Chapter Summary This review of the related literature has discussed the importance of vocabulary for L2 learning. Different annotations have been investigated in L2 vocabulary learning in both printed texts and multimedia texts. Supporting Paivio’s dual-coding theory (1971, 1986, 1991), dual annotation of text-and-picture was unanimously argued to be better than single annotations (Kang, 1995; Kost et al., 1999; Jones, 2003; Yoshii, 2000; Yeh & Wang, 2003). Studies utilizing both picture annotation and vi deo annotation coupled with text annotation further supporte d dual-coding theory, and their results also confirmed the cognitive theory of multimedia learni ng (Al-Seghayer, 2001; Chun & Plass, 1996a; Mayer, 1997, 2001) that claims that meaningful learning enga ges learners in both verbal and visual cognitive processing systems. The e ffect of dual annotation of text and picture on L2 reading comprehension is not clearly establishe d due to the scarci ty of studies. However, different studies found an advantag e of different dual annotations, and offered different interpretations (Al-Segha yer, 2001; Chun & Plass, 1996a).

PAGE 83

68 Studies in audio annotation have mainly engaged the use of pronunciation of the target words and their findings are incons istent (Chun & Plass, 1996a; Svenconis and Kerst ,1995; Yeh & Wang, 2003). As a different sensory modality from visual modality such as printed text, pictures and videoes, audio should be treated separately as to its effect on learning. Studies of audio annotat ion, therefore, should include not only the pronunciation, but also the defini tion or meaning of the target words. Finally, more evidence is necessary to invest igate audio and visual separate ly in combination with text (Mayer, 2001). Studies investigating dual multimedia annotations have only examined the different effects between text-picture and text -video annotations. However, in both textpicture and text-video annotations, verbal (t ext) and nonverbal (picture or vide) modes are used to present the information, but onl y visual sensory-motor is engaged (text, picture, video) in registering the inform ation (Mayer, 1997, 2001). In comparison, audiopicture annotation uses both verbal and nonverbal modes to present information, but engages both visual and auditory sensory moda lity channels to register the information. No study has ever examined the possible differe ntial effect of audio-picture annotation in comparison to text-picture annotation. Most annotation studies i nvolve subjects from a single L1 background, except Yoshii (2000) and Al-Saghayer (2001); therefore, there is a need to further examine a participant population with multiple first la nguages. In this case, ESL students from a range of first language backgrounds would be a good sample. Various studies have found that certain L2 proficiency is necessary to effectively use annotations; therefore,

PAGE 84

69 intermediate ESL students were chosen to participate in this study (Dufon & Hong, 1994; Jacobs, 1994; Laufer & Shmueli, 1997). Only two studies have addressed the re lationship between vocabulary annotation and reading comprehension in a multimed ia setting (Chun & Plass, 1996b; Lomicka, 1998). As an instructional intervention, multim edia annotation does briefly interrupt the reading process by drawing learner attention aw ay from the intended reading goal. It is, therefore, necessary, when examining L2 voca bulary learning, to fu rther investigate the effects of annotations on text comprehension. An overview of the studies on L2 vocabul ary annotation, particularly multimedia annotation, suggests that there is little information about how different dual annotations, in particular text-picture a nd audio-picture annotations, aff ect L2 vocabulary learning and reading comprehension. Specifically, in ESL cl assrooms, there is little information on how these dual annotations facilitate ESL students’ vocabulary le arning through reading. This information is needed to understand the extent to which multimedia learning can be used in L2 reading instruction and the role of multimedia in L2 vocabulary learning. The effects of multimedia dual annotations employing different modalities on L2 vocabulary learning and reading comprehension are still an unresolved problem. In addition, there has been no study to compare in cidental learning and intentional learning in a multimedia setting.

PAGE 85

70 Chapter III Methods The previous chapter reviewed the rela ted literature on L2 vocabulary learning, annotation studies, and cognitive theory of multimedia learning. This study used a between-subjects design to compare the eff ects of dual multimedia annotation utilizing different sensory modalities (t ext-and-picture, audio-and-pi cture) on intermediate L2 students’ vocabulary immediate recall and reading comprehensi on in both incidental and intentional learning conditions. In particular, the following six research questions were addressed: 1. What are the different effects of text-p icture and audio-picture annotations in facilitating L2 vocabular y immediate recall? 2. What are the different effects of text-p icture and audio-picture annotations in facilitating L2 reading comprehension? 3. Which learning condition (incidental or intentional) results in better L2 vocabulary immediate recall? 4. Which learning condition (incidental or inte ntional) results in better L2 reading comprehension? 5. Is there an interaction between anno tation type and l earning condition on immediate L2 vocabulary immediate recall? 6. Is there an interaction between annota tion type and learni ng condition on L2 reading comprehension?

PAGE 86

71 The present chapter descri bes the design of the study, the participants and the setting, instruments, the data collection procedures and the methods of data analysis. Design of the Study This study investigated the effect of di fferent types of multimedia annotations on L2 vocabulary immediate recall and read ing comprehension. St udents’ vocabulary immediate recall and reading comprehension will be compared in two annotation types (text-picture and audio-pict ure) and under two learning conditions (incidental and intentional.) In othe r words, the study was designed to investigate different learning conditions and instructional interventions (p roviding different multimedia annotations to unknown words) on vocabulary immediate reca ll and text reading comprehension. This study adopted a between-subjects de sign immediate posttests. There were two dependent variables: L2 vocabulary im mediate recall and read ing comprehension. There were two independent va riables: learning conditions (intentional and incidental) and annotations (text-picture, and audio-picture). In total, as shown by the 2 x 2 factorial design in Table 3-1, there were four treatme nts: intentional leaning in text-picture, intentional learning in audio-picture, incident al learning in text-pic ture, and incidental learning in audio-picture. A more detailed research design can be found in Figure 3-1. Table 3-1 2 x 2 Factorial Design Learning Intentional Incidental Text-picture Annotation Audio-picture

PAGE 87

72 Figure 3-1 Detailed Research Design For the vocabulary posttest, only imme diate posttests, Vocabulary Knowledge Scale (VKS) and Word Recognition Test (WRT), were used in this study. The delayed posttest was not used because the focus of th e study was to investigate the effects of different annotations on vocabulary immediat e recall after an init ial exposure of the targeted words in a reading text. Hulstijn (2001) argues for this practice with two reasons: (1) after in itial exposure to new information, people tend to forget the Web-based L2 Reading Incidental Learning Condition Intentional Learning Condition Text-picture Annotation Audio-picture Annotation Text-picture Annotation Audio-picture Annotation Immediate Vocabulary posttests Reading Comprehension Tests Participants

PAGE 88

73 information if there are no additional exposur es or rehearsals; therefore, the fall in performance on delayed posttest is expected; (2) de layed posttest will be necessary if the research focus is on what happens with the new information after initial exposure under various conditions of rehearsal or multiple exposures. The scores of the immediate vocabulary posttests were used to measure the participants’ vocabulary learning. L2 vocabul ary learning involves multiple factors, such as phonology, morphology, syntax, and semantic s (Laufer, 1990), and knowing a word includes multiple components, such as spelling, pronunciation, grammatical form, relative frequency, and collocations (Richa rds, 1976; Nation, 1990). The distinction between receptive and producti ve vocabulary generally refe rs to having some knowledge of a word and being able to use it in speech and writing (Melka, 1997; Read, 2000). This study adopted the distinction made be tween receptive knowledge and productive knowledge by Nation (1990) and Read (2000) th at receptive knowledge refers to the ability to recognize and recall the meaning of a target wo rd, while productive refers to the ability to use the target word in the learner’s own speech or writing. Considering the fact that th e participants in the study we re not taught the features of the target words explicitly, and they onl y had limited exposure to the target words through reading, vocabulary imme diate recall in this study fa lls into the category of receptive learning. Thus voca bulary immediate recall in the present study means the ability to recognize and recal l the meaning of the target words after reading. The following section of this chapter descri bes in more details the participants and the setting, instruments, the data collection proce dures and the methods of data analysis.

PAGE 89

74 Variables Dependent Variables The dependent variables in this study were students’ vocabulary immediate recall and reading comprehension. Vocabulary im mediate recall was measured by two vocabulary posttests: VKS (Vocabulary Know ledge Scale) and WRT (Word Recognition Test). Reading comprehension was measured by multiple-choice Reading Comprehension questions and a Written Recall. Independent Variables Types of annotation. The primary independent variable was the type of multimedia annotation that was available to students during the reading process. Two different dual annotations were included to i nvestigate whether they had different effects on L2 vocabulary immediate recall and r eading comprehension. A text-picture annotation provides a textual definition/explanation of th e unknown word and a picture that describes the unknown word. An a udio-picture annotation offers a picture description of the unknown word along with an audio explanation of the word spoken by a native speaker. All the pictures were taken from the Wo rld Wide Web. Each picture was chosen to express the essentia l meaning of the target word as understood in the reading context. A native speaker of English was asked to read and record the audio annotation. The definitions and pictures were also given to experienced ESL instructors and non-participating ESL students for verifications. The definitions were examined and

PAGE 90

75 revised to be compressible to ESL students a nd expressive of the contextual meanings of the target words. The pictures were judged on the criteria of being graphically descriptive of the word meanings. Necessary revisions and modifications were carried out. Types of Learning. With respect to the learning conditions, Hulstijn’s (1992) definition for types of learning was used. The type of learning was determined to be either intentional or incident al based on the learners’ respons ibility for the learning task, which was specified in advance of the task. Therefore, participants who were told in advance that they would have a reading comprehension test after reading the text were treated as incidental vocabular y learning subjects. Those par ticipants who were told in advance that they would have vocabulary te sts and reading compre hension tests after reading were treated as intenti onal vocabulary learning subjects. Two different learning conditions were included in order to test whether a predetermined objective would influence st udents’ performance on comprehending the text and retaining vocabulary. Particularly in terms of vo cabulary immediate recall, the intentional learning condition wa s assumed to draw more stude nt attention to the target words, which, in turn, was believed to ensure depth of processing and facilitate vocabulary learning (Hulstijn, 1992). Other Independent Variables Controlled For the control of participants’ previous knowledge of the target words, the target words were identified by non-participating intermediate ESL students and confirmed by their instructors to be unfamiliar to ty pical intermediate ESL students.

PAGE 91

76 Other relevant independent variables such as age, gender, and computer familiarity were controlled by random assignm ent of the participants into the four conditions. Participants and setting The participants of this study were 85 in termediate-level ESL students from three universities in northwest United States. They were enrolled in the Intensive English Program (IEP) at these universities. The individu al institutes define the intermediate level using similar placement tests. These universities provide ES L classes at a variety of proficiency levels, ranging from beginning to advanced levels. All the three IEPs offer similar curricul a that recognize the need for both fluency and accuracy to prepare non-native speakers of English to meet their academic and professional goals. Instruction in IEPs strives to make use of authentic, meaningful content and to present form as a facilitato r of communicative and social interaction, rather than an end. It is the IEP philosophy that authentic, meaningful content involves an integration of, rather th an a separation of, the linguist ic skill areas of listening, speaking, reading, writing, and grammar. Their IEP curriculum offers classes at si x proficiency levels: beginning (levels 12), intermediate (levels 3-4), and advanced (l evels 5-6). The students were assigned into different proficiency levels based on th eir performance on th e Michigan English Placement Test (EPT) which has 20 listeni ng items, 30 grammar items, 30 vocabulary items and 20 reading items. The TPT was supplemented by an oral interview and a writing test. The scores for level cl assification are shown in Table 3-2.

PAGE 92

77 Table 3-2 Test scores and matching levels at one English Language Institute Level EPT Listening (20 items) EPT Structure (80items) Grammar, Vocabulary, Reading) Oral Interview (eg: Fred Test) Writing test (Holistic) 1 0-4 0-19 0-35 1+, 1 2 5-7 20-30 36-48 2+, 2, 23 8-10 31-41 49-60 3+, 3, 34 11-14 42-52 61-70 4+, 4, 45 15-17 53-65 71-80 5+, 5, 56 18-20 66-80 80+ 6+, 6, 6In terms of test scores, an intermediate level ESL student will have a score of 3152 on the structure section of the EPT test, a score 8-14 on the listening section of the EPT test (a total of 39-66 on the EPT test). The oral interview scor e and writing test score will be respectively 49-70, and (3-)-(4+). Th e cutting scores on EPT at these English Institutes correspond to the cu tting scores set by the University of Michigan (46-74 for intermediate level ESL students).

PAGE 93

78 Table 3-3 Countries and Languages Repres ented by the Participants Country Language Number of Participants Japan Japanese 30 Taiwan Chinese 14 Korean Korean 15 Thailand Thai 2 Saudi Arabia Arabic 6 Italy Italian 1 Vietnam Vietnamese 1 Mexico Spanish 3 France French 1 Germany German 2 Brazil Portuguese 1 Peru Portuguese 1 El Savado Spanish 1 The intermediate language proficiency of the participants in this study were validated by the instructors at these institu tes and the program academic coordinators as well as with reference to the participants’ sc ores on the placement tests given when they entered the IEPs. Originally 85 students participated in this study. After data collection, seven students who did not look up any hyperlinked words during reading were excluded from

PAGE 94

79 the study. As a result, the final sample size was 78. Table 3-3 shows the countries and languages represented by the participants. There were 49 female students and 29 male students. 69% of the partic ipants were in the age group of 18-25, 24% were in the age group of 26-45, and 7% were over 45. In terms of length of stay in the United States, the majority of the participants fell into the cat egory of 0-6 months, 82% of the participants reported 0-6 months, 15% repor ted 6-12 months and 3% repo rted over a ye ar. All the participants reported being comfortable using computers and reading a web-based English text on the computer. Instruments Questionnaire A questionnaire was used to collect demographic information from the participants, general Englis h learning experience, and their experience with using computers. The questionnaire can be found in Appendix A. The questionnaire contains three major parts: demographic information, English language learning experience, and computer experience. The portion dealing with demographic information was designed to gath er information about the participants’ age, gender, national origin, native la nguage, and length of stay in the United States. Another portion of the questionnaire ga thered information on the pa rticipants’ English language learning experience before and after arriving in the United States. The last part of the questionnaire relates to the participants’ experience and familiar ity with technology. Participants were asked to offer informati on on their general computer skills, access to the Internet, and onlin e reading ability.

PAGE 95

80 Reading Material and Target Words The reading text, “European Settlers of Australia,” was written by the researcher based on three criteria: text le ngth, syntactic comple xity, and content. In terms of length, the text has 449 words (including the title). The reading text can be found in Appendix B. ESL students at the intermediate level are comfortable with reading texts of this length and syntactic complexity. It consists of short, uncomplicated sentences and simple past tense is used throughout the text. Th ere is an average of 6.8 sentences in each paragraph, and an average se ntence contains 10.8 words. The percentage of simple sentences in the text is over 80%. With re gard to the content, it seems reasonable to assume that ESL students knew more or le ss the same amount of general information about the European coloni zation of Australia and have comparable background knowledge of the reading text (i.e., since none has been to Australia and its history is foreign to all participants). The conten t of the text does not require any specific culturally related knowledge. The readability of the text is considered to be betw een grade level 5 and 6 based on the Flesch-Kincaid measure. It tells of the story of the European colonists in Australia in the 1800s. The text was given to experien ced ESL instructors who teach intermediate reading/writing classes and was confirmed to be appropriate for intermediate ESL students. A cloze procedure was conducted with non-participati ng intermediate ESL students to estimate the difficulty of the text The student’s cloze sc ore of 67% indicate that the reading text was appropriate for interm ediate ESL students in terms of difficulty level.

PAGE 96

81 The 20 target words were all nouns. They were selected for frequency. Based on the word frequency corpora of Francis a nd Kucera (1982), the 20 target words have a mean of 12.7 per million words. The 20 target words can be found (underlined words) in Appendix C. The 20 target words constitute 4.5% of th e total words of the text, leaving 95.5% coverage of the reading text Nation (2001) points out that 98-99% coverage is desirable for relatively easy reading for ESL students. However, he argues for a smaller coverage of the reading text in annotation studies b ecause the mixed findings of the effects of annotations on reading compre hension might be due to the percentage of unknown words being less than 3% of the running texts in most experimental studies. Therefore, he suggests the use of more unknown words that mi ght initiate the effect of annotation on reading comprehension. The reading text was modified into two di fferent forms: a text with text-picture annotations, and a text with audio-pictur e annotations. The 20 target words were highlighted in both texts. Program The interactive multimedia program used in this study was designed by the researcher to help intermediate ESL st udents with vocabulary learning and reading comprehension. The program provided student s with annotations for unknown words via hypermedia links in two different modes: text -picture and audio-pict ure. The annotations were used to assist the learning of unknown words and understanding of the reading text.

PAGE 97

82 The program was written in HTML and JavaScript. HTML was chosen as the programming language due to its user-fri endly integration of hypermedia and its compatibility for both PC and Mac platforms. The picture annotations were processed with Adobe Photoshop 6.0 (Adobe, 2000), and the audio clips were processed with Vegas 4.0 (Sonic Foundry, 2003). The pictures for the 20 target words can be found in Appendix D. The program was delivered over the Internet and was accessed using either Internet Explorer (version 5.0 or higher) on PCs or Safari on Mac computer. The screen layout took into considerati on instructional design principles. The screen was divided into two frames. The left screen was used for the reading text with the title at the top, a nd the right screen was reserved for the annotation. In the textpicture version, when participants click on a highlighted word, the right screen offers a textual definition of the words together with a picture that describes the word. In the audio-picture annotation, when participants click on a highlighted word, they could see on the right screen a picture th at depicts the meaning of the word and hear an audio clip that explains the meaning of the word. The screenshots of text-pic ture and audio-picture can be found in Appendix E and F. In order to count the numb er of times that a partic ipant accesses an annotated word in the reading text, a JavaScript variab le was created for each highlighted word and encoded into the HTML for the reading text. Each time the participant clicked a certain highlighted word, a variable was incremen ted by one with the “OnClick” JavaScript command. These variables were connected with the participant’s identification code and computer number. Upon finishing reading th e text, when the participant clicked the “Submit” button, the PHP sendmail script passe d the variables to the researcher’s email

PAGE 98

83 account. The transferred file contained the pa rticipant’s identification code, computer number, annotation type and th e clicking behavior recorded during the readi ng process. Posttests Vocabulary Knowledge Scale (VKS). The Vocabulary Knowledge Scale (VKS) was modified from Paribakht & Wesche’s (1993, 1997) Vocabulary Knowledge Scale (VKS). The VKS was designed as a measure of the stages in a learner’s developing knowledge of particular words. In the original instrument, a 5-level scale was used to rate vocabulary knowledge that ranges “from co mplete unfamiliarity, through recognition of the word and some idea of it’s meaning, to th e ability to use the word with grammatical and semantic accuracy in a sentence” (Paribakht & Wesche, 1997, p.29). In this study, an adapted VKS (See Table 3-4) is compiled in consideration of three factors: (1) translation is not encourag ed since the participants represent multiple first languages; (2) since the participants only have one e xposure to the targeted words from the reading text, therefore Level IV (re cognize the word and be able to give the meaning of the word) and Level V (be able to use the word in a sentence) are not required; and (3) the focus of the study is vo cabulary learning that is operationalized as the immediate recall of word meaning. Therefore, gram matical accuracy of the explanation is not graded. The VKS test can be found in Appendix G.

PAGE 99

84 Word Recognition Test (WRT). The participants were asked to complete a Word Recognition Test (WRT). In this test, the 20 ta rget words were presented in their original context taken from the reading text. For each wo rd, the participants were asked to choose one correct meaning out of four given choices Of the four choices, one was the correct meaning, and the other three were distractor s. The WRT test can be found in Appendix H Table 3-4 Adapted VKS scoring categorie s and meaning of scores. Categories Description Possible score Meaning of score I I don’t remember having seeing this word before 0 The word is not familiar at all. II I have seen this word before, but I don’t know what it means 1 The word is recognized, but the meaning is unknown. III I have seen this word before, and I think it means_______ (providing explanation) 2 A word is recognized, and a correct meaning is given. Multiple-choice Reading Comprehension Test (RC). The reading comprehension text was comprised of 10 multiple-choice que stions. For each question, the participant was asked to choose the best answers from the four given choices. The questions and choices were given to experi enced ESL instructors for va lidation. The questions were confirmed to be easy to understand and reflectiv e of main idea of the reading text. The Reading Comprehension Test can be found in Appendix I.

PAGE 100

85 Written Recall. Written recall was used to measure participants’ general text comprehension of the reading text. Due to the English level of the participants, the written recall was conducted in the participan ts’ native language. Th e written recall can be found in Appendix J. Validity Issue of the Posttests Validity refers to the degree to which a study accurately reflects or assesses the specific concept that the researcher is attemp ting to measure. In other words, validity is concerned with the study's success at measuring what the researcher set out to measure. Due to the nature of the achievement test us ed in this study, content validity will be the focus of discussion here. Content Valid ity is based on “the extent to which a measurement reflects the specific intended dom ain of content” (Carmines & Zeller, 1991, p.20). In this study, vocabulary immediate reca ll was operationalized as the ability to recognize and recall the meaning of target words when pr esented. For this particular purpose, the VKS has been proven to be a va lid measure as a self-rating report of word knowledge (Paribakht & Wesche, 1993,1997). Fo r WRT, the choice of the right or wrong answer reflects the part icipants’ knowledge of the ta rget word. Experienced ESL instructors and ESL researchers were consulted in terms of the subject matter of the test items to ensure their valid interpretation. The written recall and reading comprehens ion test aimed to assess participants’ general understanding of the reading text. In other words, participants were supposed to recall the main ideas of the text after readi ng and choose the best answer for each of the

PAGE 101

86 comprehension questions. The multiple-choice questions were given to three experienced ESL instructors for their content validity. A consensus was reached that questions reflect and measure the participants’ genera l knowledge of the reading text. Scoring The researcher rated the scores fo r Word Recognition Test and Reading Comprehension Test. Two raters were as signed to rate each student’s Vocabulary Knowledge Scale and Written Recall. One rate r was the researcher and the other was one experienced ESL instructor from one of the IEPs. Each rater scored the tests individually. Inter-rater reliability, which wa s calculated by the percentage of agreement between the two raters, was used to examine the reliability of the measurements. Where there were discrepancies in the rating betw een the two raters, a reexamination of the responses by the two raters was conducted to reach an agreement. If an agreement was not reached, a third rater was used or an average score of two ra ters was accepted. Vocabulary Knowledge Scale (VKS). The VKS was scored using a 0-2-point scale adapted from Paribakht & Wesche (1997) A score of 0 indicates total unfamiliarity of the word; a score of 1 stands for recognition of the word, but the meaning of the word is not provided; a score of 1.5 shows rec ognition of the word and partial meaning provided; and a score of 2 represents familiarity of the wording shown and correct meaning given. The possible total score was 40 points (2 points x 20 words).

PAGE 102

87 Word Recognition Test (WRT). A correct choice received the score of 1 and an incorrect choice received the score of 0. Th e possible maximum score was 20 points (1 point x 20 words). Multiple-Choice Reading Comprehension Test (RC). A 0-1-point scale was used for the reading comprehension test. In the 0-1 point scale, 0 represents no response or an incorrect choice; 1 represents a correct choice. The comprehension score for each participant was calculated by combining the score for each of the 10 multiple-choice questions together, with a possible maximu m score of 10 (1 point x 10 questions). Written Recall The researcher and another experienced ESL instructor independently identified the idea units or propositions in the reading text. Through discussion, it was agreed that there were a to tal of 52 idea units in the reading text. All the written recall protocols were scored in terms of the 52 idea units. Each unit idea presented in the written recal l was given 1 point. Paraphrasing was allowed and counted correct recall of idea units. Procedures The study was conducted during the particip ants’ regular class times, and required two consecutive 50-minute sessions. The part icipants were randomly assigned to the learning conditions: incident al learning and intentiona l learning. The two learning conditions were conducted sepa rately at different times. In each of the learning conditions, participants were further randomly divided into two annotation groups (text-

PAGE 103

88 picture, audio-picture). Each student created a unique iden tification code that they remembered and used in the study. For the iden tification code, they used the combination of the last letter of their last name and the day of the month in which they were born. For example, Bob Smith, born on the 15th of May, would use th e code of H15. For each learning condition, in the firs t 50-minute session, the researcher first gave a brief introduction of the study and an swered any questions that the participants might have. Then the participants signed the consent form and filled out the questionnaire. The questionnaire asked for st udents’ demographic information such as gender, age, first language, years of English l earning, length of stay in the U.S., and their familiarity with computers. In the second session, students went to the language lab at their IEPs. The computers in the lab were preset to pr esent different annotations for every two neighboring computers. In other words, two ne ighboring students ha d access to different annotations, one was text-picture and the othe r was audio-picture. In the computer lab, the researcher gave a brief introduction of the online reading activity, and an online tutorial was used to get the participants familiar with the online reading activities. Headsets were used for those who were in the audio-pict ure annotation group. During reading, the participants clicked the highlighted unknown words to access available annotations, the text -picture group was able to see textual explanation and pictorial description, and the audio-picture group was able to see pictorial description explanation and hear a spoken explanation. When they finished readi ng, they raised their hands to receive the posttests.

PAGE 104

89 Participants in the incident al learning condition were told before reading that they would have reading comprehension tests, but not informed of the vocabulary posttests. Participants in the intentional learning condition were told that they would have vocabulary posttests and reading comprehens ion tests up front. The same reading vocabulary posttest and reading comprehe nsion tests were conducted with these participants after reading. The posttests we re given in the following order: Vocabulary Knowledge Scale (VKS), Word Recogniti on Test (WRT), Multiple-choice Reading Comprehension Test (RC), and Written Recall. Data Analysis The data were analyzed with the SA S package. A 2 x 2 ANOVA was performed for each of the posttests: vocabulary posttest s (VKS, WRT) and reading comprehension tests (Multiple-choice Reading Comprehension Test and Written Recall). The data were analyzed and examined in term s of the effect of annotation type (Text-picture and Audiopicture), the effect of learni ng condition (Incidental and Inte ntional), and the interaction effect between the two f actors (annotation type a nd learning condition). The internal consistency reliability for the dependent measures was estimated using Cronbach’s alpha. Interrater reliability was used as a measure for Vocabulary Knowledge Scale and Written Recall. An additional ANOVA was conducted for lookup frequency behaviors. An interaction was found between the annotation type and learning condition, further correlation analysis was used to examine the relationship between the lookup frequency behavior and dependent measures.

PAGE 105

90 The alpha level was set at .05 for all the an alyses. Descriptive statistics for the posttest scores were calculated for each anal ysis. Relevant statisti cal data were also presented with tables and graphical figures. This chapter describes the design, par ticipants and setting, instruments, procedures of data collection and data analysis of this study. The results of data analysis will be presented in the next chapter.

PAGE 106

91 Chapter IV Results The purpose of this chapter is to communi cate the data analysis results as well as report the findings related to each research question. The first part of this chapter presents the results of a pilo t study conducted as a precursor to the dissertation study. The second part of this chapte r introduces results obtained from the four instruments --Vo cabulary Knowledge Scale, Word Recognition Test, Multiple-choice Reading Comprehensi on Test and Written Recall. After the presentation of results, each of the research questions is examined and answered in the third part of this chapter. Pilot Study A pilot study was conducted in order to validate the instruments and gain some insight into the research de sign of the disserta tion study. The pilot study examined the different effects of two multimedia annotati ons (text-picture and audio-picture) on L2 vocabulary immediate recall and reading co mprehension. Vocabulary Knowledge Scale (VKS) and Word Recognition Test (WRT) were used to measure vocabulary immediate recall. A 10-question multiple-choice read ing comprehension test (RC) was used to measure reading comprehension.

PAGE 107

92 Participants The participants of the pilot study were 20 intermediate level ESL students from an IEP. Five participants were assigned to each of the four treatments. The data were collected during th e 2005 fall term. Procedures The participants were rando mly assigned to one of the f our treatments: incidental text-picture, incidental audi o-picture, intentional text-picture, and intentional audiopicture. Participants in the intentional le arning condition were told in advance of the vocabulary posttests. The resear cher explained the project an d asked the participants to read the English text of “European Settlers of Australia” on the co mputer. After reading, the participants were given the posttests. The order of administering the posttests was as follows: Vocabulary Knowledge Scale, Word Recognition Test, and Reading Comprehension Test. Results The researcher coded and rated all the thre e tests; therefore, inter-rater reliability was not obtained. The reliability of the te st was measured with Cronbach’s alpha (Cronbach, 1951). For the scores obtained from the three posttests, the reliability was respectively .84 for the Vocabulary Know ledge Scale (VKS), .83 for the Word Recognition Test (WRT), and .51 for the Mul tiple-choice Reading Comprehension Test (RC).

PAGE 108

93 The descriptive statistical results of the tests can be seen in Table 4-1. Table 4-2 and Table 4-3 present the means and standard deviation for the th ree tests by annotation type and learning condition respectively. ANOVA results can be seen in Table 4-4. For the Vocabulary Knowledge Scale (VKS ), the audio-picture-intentional group (M=30.5, SD=3.32) obviously outperformed the other three groups (M=24, SD=6,37, M=23.1, SD=8.58, M=25, SD=5.56). If evaluate d by annotation type, the audio-picture type (M=27.75, SD=5.20) did better than th e text-picture type (M=23.55, SD=7.14); if examined from learning condition, the intentional condition (M=26.8, SD=7.27) performed better than the incidental condition (M=24.5, SD=5.66). However, an ANOVA did not show a statisti cally significant difference between the two annotation types, F (1, 19) =2.26, p =.1522 > .05, or betw een the two learning conditions, F (1, 19) =.68, p = .4224 > .05. Nor was there significant in teraction between annotation type and learning condition, F (1, 19) =1.31, p = .2688 > .05. Table 4-1 Descriptive Statistics for Tests of the Pilot Study VKS WRT RC Group n M SD M SD M SD Text-picture/Incidental 5 24 6.37 14.2 1.92 8.2 1.64 Text-picture/Intentional 5 23.1 8.58 16.6 2.07 7 2.24 Audio-picture/Incidental 5 25 5.56 17.4 1.52 8.2 1.48 Audio-picture/Intentional 5 30.5 3.32 17.2 3.03 6.4 .89

PAGE 109

94 Table 4-2 Means and Standard Deviation for the Tests of the Pilot Study by Annotation Type VKS WRT RC Annotation n M SD M SD M SD Text-picture 10 23.55 7.14 15.4 2.27 7.6 1.96 Audio-picture 10 27.75 5.20 17.4 2.26 7.3 1.49 Table 4-3 Means and Standard Deviation for the Tests of the Pilot Study by Learning Condition VKS WRT RC Learning n M SD M SD M SD Incidental 10 24.5 5.66 15.8 2.35 8.2 1.48 Intentional 10 26.8 7.27 16.9 2.47 6.7 1.64 Table 4-4 ANOVA Results of the Pilot Study VKS WRT RC Source df F P F P F P Annotation 1 2.26 .1522 01 .9200 .17 .6871 Learning 1 .68 .4224 .26 .6168 4.21 .0571 Annotation Learning 1 1.31 .2688 .09 .7634 .17 .6871 Error 16

PAGE 110

95 For the Word Recognition Test (WRT), as a whole, the audio-picture-intentional (M=17.4) outscored the other three groups (M =17.2 for audio-picture-intentional group, M=16.6 for text-picture-intenti onal and M=14.2 for text-picture -incidental). If judged by annotation type, the audio-picture annotati on (M=17.4) outperformed the text-picture annotation (M=15.4); if judged by learning cond ition, the intentional condition (M=16.9) did better than the incide ntal condition (M=15.8). However, an ANOVA showed the differences were not statistical ly significant. There was no interaction between annotation type and learning condition. For the reading comprehension test (RC) as a whole, both annotation types in incidental condition had the highest mean (M=8.2), followe d by text-picture annotation group (M=7), and audio-pictur e-intentional group had the lo west mean (M=6.4). From the perspective of annotation ty pe, the text-picture annotatio n (M=7.6) scored higher than the audio-picture annotation (M =7.3) in reading comprehensi on; from the perspective of learning condition, the incidental condition (M =8.2) led to better results in reading comprehension than the intentional cond ition (M=6.7). However, an ANOVA did not show significant effect for either annota tion type or learni ng condition, although the difference between the two l earning conditions was close to being significant (F=4.21, p=.0571). No interaction was found between annotation type and learning condition (F=.17, p=.6871 > .05). The results of the p ilot study are summarized in Figure 4-1. Discussion The purpose of the pilot study was to valid ate the scores from the instruments and investigate the applicability of the research design to be used for this dissertation study.

PAGE 111

96 The result of the pilot study showed the th ree instruments to be reliable. From the ANOVA analysis, there were no statistically significant eff ects for annotation type or learning condition, nor was ther e an interaction be tween annotation type and learning condition. However, the large mean differences along with sizable standard deviations from the pilot study could explain the lack of statistically sign ificant results. This indicated the necessity for further study with a larger sample size. Results of Pilot Study0 5 10 15 20 25 30 35 VKSWRTRC TestsScores Text-picture/incidental Text-picture/intentional Audio-picture/incidental Audio-picture/intentional Figure 4-1 Results of Pilot Study The following part of this ch apter will present th e results of the actual study. First the information on annotation look-up behavior is presented, followe d by the results of the four posttests will be presented, and fi nally the results by research questions.

PAGE 112

97 Annotation Look-up Behavior Results The participants’ annotati on look-up behaviors were tr acked and recorded when they clicked the hyperlinked words for annot ations. One look-up was counted every time a target word was clicked. Seven particip ants restrained from checking the available annotation and did not look up any of target words and were, therefore, excluded from the data analysis. Among the 7 participants 2 were from the text-picture annotation group and the other 5 were from the audio-picture group. Table 4-5 and Table 4-6 show the descrip tive statistics for look-up behaviors. As shown by Table 4-5, the mean score for text-p icture/incidental was the highest, followed in order by audio-picture/intentional, audio-picture/inci dental, and textpicture/intentional. However, as displa yed by Table 4-6, partic ipants in the two annotation types looked up on average the same amount of words (24.05 vs. 23.62), and participants in the tw o learning conditions looked up similar amount of words as well (24.55 vs. 23.18). A two-way (Annotation x Le arning) analysis of variance (ANOVA) was performed on the look-up behavior data. Table 4-5 Descriptive Statistics for Look-up Behavior Group n M SD Sk Kr Text-picture/Incidental 20 26.65 8.76 .23 -1.43 Text-picture/Intentional 19 21.31 4.99 .47 .01 Audio-picture/Incidental 20 22.45 6.89 1.25 2.10 Audio-picture/Intentional 19 24.84 6.88 1.02 .45 Sk = skewness, Kr = kurtosis

PAGE 113

98 Table 4-6 Means and Standard Deviations for Look-up Behavior Annotation n M SD Learning n M SD Text-picture 39 24.05 7.58 Incidental 40 24.55 8.07 Audio-picture 39 23.62 6.90 Intentional 38 23.18 6.19 As indicated by Table 4-7, the differences between annotation types and between learning conditions were not significant, while the interaction between annotation type and learning condition was signi ficant (F = 5.88, p = .0178 < .05). Table 4-7 ANOVA for Look-up Behavior Source df Type III SS Mean Square F Value P Value Annotation 1 2.211 2.211 .04 .8332 Learning 1 42.17 42.17 .85 .3590 Annotation Learning 1 290.83 290.83 5.88 .0178** Error 74 3662.13 49.49 ** p < .05

PAGE 114

99 Table 4-8 Correlation between Look-up Behavior and Posttest Measures VKS WRT RC Recall r -.22 -.20 -.10 -.01 Look-up Behavior p .0515 .0754 .3759 .9153 Since there was interaction effect betw een annotation type and learning condition, a correlation procedure was carried out to ch eck if look-up behaviors were related to the posttest measures. As shown in Table 48, look-up behavior wa s not statistically significantly correlated with a ny of the posttest measures for the dependent variable. Therefore, further data analysis was conduc ted without controlling for look-up behavior. Posttest Results Four instruments were used in the po sttest: Vocabulary Knowledge Scale (VKS), Word Recognition Test (WRT), Multiple-choice Reading Comprehension Test (RC) and Written Recall. All the four tests were immediate posttests. They were administered right after the participants finished reading the English text on the com puter. The tests were given in the following order: VKS, WRT, RC and Written Recall. The results of the Vocabulary Knowledge Scale are discussed first, followed by those from the Word Recognition Test, then the Reading Comprehe nsion Test, and finally the Written Recall. A two-way (Annotation x Lear ning) analysis of varian ce (ANOVA) was performed on each immediate posttest. The alpha level was set at .05 for all the analyses.

PAGE 115

100 Vocabulary Knowledge Scale Scores The results of the Vocabulary Knowledge Scale were based on the participants’ self-rated knowledge of the 20 target words. For each target word, the participants were asked to choose a description of their knowle dge of the word. Two points were given for a correct written meaning, 1.5 points were gi ven for a recognized word and partially correct meaning, one point was given for a recognized word without written meaning, and zero point was given for an unrecognized word. The total possible score was 40 points. A two-way ANOVA (Annota tion x Learning) was used to examine the scores on the Vocabulary Knowledge Scale. The reliab ility of the instrument, as measured by internal consistency, was .86 (Cronbach coeffici ent alpha). The inter-r ater reliability was .90. Table 4-9 shows descriptive statistics for the Vocabulary Knowledge Scale for various combinations of annotation type a nd learning condition. Table 4-10 presents the means and the standard deviations of th e Vocabulary Knowledge Scale scores by the annotation types and learning conditions re spectively. The scores were normally distributed for each of the treatment groups. Fi gure 4-2 illustrates that the audio-picture annotation participants did better on the Vocabulary Knowledge Scale than the textpicture annotation participants. As shown by Figure 4-3, participants in the intentional learning condition performed better than those in the incidental learning conditions. An ANOVA was conducted to establis h whether or not the diffe rences were significant. As seen from Table 4-11 and Figure 44, the ANOVA results show that there is no significant interaction between annot ation type and learning condition on the

PAGE 116

101 Vocabulary Knowledge Scale. However, ther e were significant differences between the annotation types as well as be tween the learning conditions. Participants in the audio-picture anno tation type performed significantly better than those in the text-picture annotat ion type, F (1, 74) = 8.40, p = .0049 < .05. A significant difference was also found for the learning condition factor, with the intentional learning condition ou tperforming the incidental learning condition, F (1, 74) = 7.15, p = .0092 < .05. Table 4-9 Descriptive Statistics for Vocabulary K nowledge Scale by Annotation and Learning Group n M SD Sk Kr Text-picture/Incidental 20 19.33 7.73 .39 -.32 Text-picture/Intentional 19 24.13 7.27 .45 -.10 Audio-picture/Incide ntal 20 24.48 6.56 .04 -.1.01 Audio-picture/Intentional 19 27.87 5.20 -.30 -.81 Sk = skewness, Kr = kurtosis Table 4-10 Means and Standard Deviations for Vocabulary Knowledge Scale Annotation n M SD Learning n M SD Text-picture 39 21.67 7.80 Incidental 40 21.9 7.54 Audio-picture 39 26.13 6.11 Intentional 38 26 6.52

PAGE 117

102 Table 4-11 ANOVA for Vocabulary Knowledge Scale Source df Type III SS Mean Square F Value P Value Annotation 1 384.75 384.75 8.40 .0049** Learning 1 327.58 327.58 7.15 .0092** Annotation Learning 1 9.73 9.73 .21 .6463 Error 74 3390.22 45.81 ** p < .05 Means for VKS by Annotation Type0 5 10 15 20 25 30 Text-pictureAudio-picture VKS Figure 4-2. Means for the Vocabulary Know ledge Scale by Annotation Types

PAGE 118

103 Means for VKS by Learning Condition0 5 10 15 20 25 30 35 40IncidentalIntentional VKS Figure 4-3. Means for the Vocabulary Knowledge Scale by Learning Conditions VKS by Annotation and Learning0 5 10 15 20 25 30 IncidentalIntentionalScore Text-picture Audio-picture Figure 4-4 VKS Scores by Annotation Type and Learning Condition Word Recognition Test Scores The results of the Word Recognition Te st were based on the participants’ identification of the correct meaning (one out of 4 choices) with the 20 target words. One

PAGE 119

104 point was given for a correct answer, and zer o point was given to an incorrect answer. The total possible score wa s 20 points. A two-way ANO VA (Annotation x Learning) was used to analyze the scores on the Word Recognition Test. The reliability of the instrument, measured by internal consistenc y (Cronbach coefficient alpha), was .76. Table 4-12 presents descriptive statis tics for the Word Recognition Test in different combinations of annotation type and learning condition. The means and the standard deviations of the Word Recogniti on Test scores by the annotation types and learning conditions respectively are provided in Table 4-13. The scores were normally distributed for each of the treatment groups. Figure 4-5 illustrates that participants in the audio-picture annotation did better on the Word Recognition Test than participants in the text-picture annotation. As shown by Figure 4-6, participants in th e intentional learning conditi on performed better than those in the incidental learning conditions. An ANOVA was conducte d to establish whether or not the differences were significant. Table 4-12 Descriptive Statistics for Word Reco gnition Test by Annotation and Learning Group n M SD Sk Kr Text-picture/Incidental 20 12.75 2.88 -.98 1.50 Text-picture/Intentional 19 16.21 2.35 -.66 -.30 Audio-picture/Incide ntal 20 15.75 3.02 -.74 -.50 Audio-picture/Intentional 19 17.63 2.43 -1.22 .45 Sk = skewness, Kr = kurtosis

PAGE 120

105 Table 4-13 Means and Standard Deviations for Word Recognition Test Annotation n M SD Learning n M SD Text-picture 39 14.44 3.14 Incidental 40 14.25 3.29 Audio-picture 39 16.67 2.88 Intentional 38 16.92 2.46 As seen from Table 4-14 and Figure 47, the ANOVA results show that there is no significant interaction between annotat ion type and learning condition. However, participants in the audio-pict ure annotation type performed significantly better than those in the text-picture annotation type, F (1, 74) = 13.12, p = .0005 < .05. In addition, a significant difference was also found between th e two learning conditi ons, with those in the intentional learning condition outperfo rming those in the incidental learning condition, F (1, 74) = 19.16, p < .0001 < .05. Means for WRT by Annotation Type0 2 4 6 8 10 12 14 16 18 20 Text-pictureAudio-picture WRT Figure 4-5. Means for the Word Recogn ition Test by Annotation Types

PAGE 121

106 Means for WRT by Learning Condition0 2 4 6 8 10 12 14 16 18 20 IncidentalIntentional WRT Figure 4-6 Means for the Word Recognition Test by Learning Conditions WRT by Annotation and Learning0 5 10 15 20 IncidentalIntentionalScore Text-picture Audio-picture Figure 4-7 WRT scores by Annotation Type and Learning Condition

PAGE 122

107 Table 4-14 ANOVA for Word Recognition Test Source df Type III SS Mean Square F Value P Value Annotation 1 95.22 95.22 13.12 .0005** Learning 1 139.03 139.03 19.16 < .0001** Annotation Learning 1 12.15 12.15 1.67 .1998 Error 74 537.08 7.26 ** p < .05 Multiple-Choice Reading Comprehension Test Scores The results of the Reading Comprehe nsion Test were obtained from the participants’ scores on the multiple-choice comprehension questions. For each of the 10 comprehension questions, participants were asked to choose a correct answer. One point was given for a correct answer, and zero point was given to an incorrect answer. The total possible score was 10 points. A two-way AN OVA (Annotation x Learning) was used to analyze the scores on the Reading Comprehensi on Test. The reliability of the instrument was .56 as measured by internal consistency (Cronbach coefficient alpha). Descriptive statistics of the Reading Co mprehension Test were presented in Table 4-15. Table 4-16 shows the means and the standard deviations for the Reading Comprehension Test score respectively by a nnotation type and learning condition. The scores were normally distributed for each of the treatment groups.

PAGE 123

108 Table 4-15 Descriptive Statistics for Reading Compre hension Test by Annotation and Learning Group n M SD Sk Kr Text-picture/Incidental 20 6.25 2.12 -.22 -.49 Text-picture/Intentional 19 7.32 2.26 -.60 -.87 Audio-picture/Incide ntal 20 6.65 2.03 -.51 -.18 Audio-picture/Intentional 19 6.32 1.45 -.26 .12 Sk = skewness, Kr = kurtosis Table 4-16 Means and Standard Deviations for Reading Comprehension Test Annotation n M SD Learning n M SD Text-picture 39 6.77 2.23 Incidental 40 6.45 2.06 Audio-picture 39 6.49 1.76 Intentional 38 6.82 1.94 Figure 4-8 and Figure 4-9 indicate that in terms of Reading Comprehension Test, the text-picture annotation participants outscored the audio-picture annotation participants, and the intentional learni ng condition participan ts outperformed the incidental learning conditi on participants. An ANOVA was carried out to verify the significance of the differences.

PAGE 124

109 Means for RC by Annotation Type0 1 2 3 4 5 6 7 8 9 10 Text-pictureAudio-picture RC Figure 4-8. Means for the Reading Compre hension Test by Annotation Types Means for RC by Learning Condition0 1 2 3 4 5 6 7 8 9 10 IncidentalIntentional RC Figure 4-9. Means for the Reading Comprehens ion Test by Learning Conditions As presented in Table 4-17, the Reading Comprehension Test results revealed no main effects for annotation type, F (1 74) = .44, p = .5089 > .05, or for learning condition, F (1, 74) = .66, p = .4209 > .05. Nor was the interaction between annotation type and learning condition significant, F (1, 74) = 2.40, p = .1257 > .05. Figure 4-10 did

PAGE 125

110 show some interaction graphically, but the di fferences between sample means are not far enough apart to conclude the population means are different, therefore, the effect seen in the sample is not pronounced enough to conclu de there is an intera ction effect in the population. Table 4-17 ANOVA for Reading Comprehension Test Source df Type III SS Mean Square F Value P Value Annotation 1 1.75 1.75 .44 .5089 Learning 1 2.61 2.61 .66 .4209 Annotation Learning 1 9.55 9.55 2.40 .1257 Error 74 294.51 3.98 ** p < .05 RC by Annotation and Learning0 1 2 3 4 5 6 7 8 9 10 IncidentalIntentional Text-picture Audio-picture Figure 4-10. RC Scores by Annotation Type and Learning Condition

PAGE 126

111 Written Recall Scores In the Written Recall, each participant was asked to recall as much information as they could remember from the reading passa ge. One point was given for each idea unit recalled. There were a total of 52 idea units in the read ing text. A two-way ANOVA (Annotation x Learning) was used to anal yze the scores on the Written Recall. The reliability of the instrument was .64 as m easured by internal consistency (Cronbach coefficient alpha). The inte r-rater reliability was .92. Descriptive statistics of the Reading Co mprehension Test were presented in Table 4-18. Table 4-19 shows the means and the standard deviations for the Reading Comprehension Test score respectively by a nnotation type and learning condition. The scores were normally distributed for each of the treatment groups. Figure 4-11 and Figure 4-12 indicate that in terms of Written Recall, the textpicture annotation participants slightly outscored the audi o-picture annotation participants, and the incide ntal learning condition part icipants outperformed the intentional learning condition participants. An ANOVA was car ried out to verify the significance of the differences. As presented in Table 4-20 and Figure 413, the Written Recall results revealed an interaction between annotation type and learning condition, F (1, 74) = 8.49, p = .0047 < .05. The main effect of learning condition was also significant, F (1, 74) = 8.97, p = .0037 < .05. However, the main effect of annot ation type was not significant, F (1, 74) = .90, p = .3464 > .05.

PAGE 127

112 Table 4-18 Descriptive Statistics for Writte n Recall by Annotation and Learning Group n M SD Sk Kr Text-picture/Incidental 20 15 6.60 .68 1.11 Text-picture/Intentional 19 14.89 7.38 .37 -1.17 Audio-picture/Incide ntal 20 17.6 4.51 .35 -.55 Audio-picture/Intentional 19 9.79 4.25 .10 -.65 Sk = skewness, Kr = kurtosis Table 4-19 Means and Standard Deviations for Written Recall Annotation n M SD Learning n M SD Text-picture 39 14.95 6.90 Incidental 40 16.3 5.73 Audio-picture 39 13.79 5.86 Intentional 38 12.34 6.48

PAGE 128

113 Means for Written Recall by Annotation Type0 10 20 30 40 50 Text-pictureAudio-picture Recall Figure 4-11 Means for Written Recall by Annotation Type Means of Written Recall by Learnng Condition0 10 20 30 40 50 IncidentalIntentional Recall Figure 4-12 Means for Written Recall by Learning Condition The significant effect of learning condition means that participants from the incidental learning condition di d significantly better than par ticipants from the intentional learning condition in recalling th e reading text. The significa nt interaction effect means that effect of learning condition on written recall depends on annotat ion type. As shown

PAGE 129

114 by Figure 4-13, the incidental le arning condition led to better written recall only in the audio-picture annotation group. In the incide ntal learning condition, participants in the audio-picture annotation did bett er than participants in the text-picture annotation; while in the intentional learning condition, participan ts in the text-picture annotation did better than participants in the a udio-picture annotation. Indepe ndent sample T-tests were conducted to further examine whether the di fferences were statistically significant ( =.05). In incidental learni ng condition, the value of t (t =-1.45) failed to confirm the difference between the two annotations was sign ificant. However, in intentional learning condition, the t value (t=2.61) showed that the difference between the two annotations was significant. This means that although th e main effect of annotation type is not statistically significant, the text -picture annotation pa rticipants did signif icantly better in written recall than the audi o-picture annotation participan ts in intentional learning condition. Table 4-20 ANOVA for Written Recall Source df Type III SS Mean Square F Value P Value Annotation 1 30.58 30.58 .90 .3464 Learning 1 305.27 305.27 8.97 .0037 ** Annotation Learning 1 289.24 289.24 8.49 .0047 ** Error 74 2519.75 34.05 ** p < .05

PAGE 130

115 Written Recall Scores by Annotation Type and Learning Condition0 5 10 15 20 25 30 IncidentalIntentional Text-picture Audio-picture Figure 4-13 Written Recall Scores by Annotation Type and Learning Condition Results by Research Questions This section of the chapter is organized according to the research questions of this dissertation study. Each research question will be stated and answered based on the results presented above. Question 1 – Effects of Annotation Type on Vocabulary Immediate Recall The first research question investigated the different effects of text-picture and audio-picture annotations on L2 vocabulary immediate recall. The results of the two vocabulary posttests (Vocabulary Knowledge Scale and Word Recognition Test) were compiled to answer this question. The result s are displayed in Ta ble 4-21 and Figure 414. In terms of vocabulary immediate recall, Table 4-21 shows that the audio-picture annotation type led to bette r results (VKS M=27.75, WRT M=26.13) than the text-picture

PAGE 131

116 annotation type (VKS M=23.55 WRT M=21 .67). As indicated by Figure 4-14, the audio-picture annotation type outperformed the text-picture annotation type, regardless of the type of test (Vocabulary Knowledge Scale and Word Recognition Test). For the Vocabulary Knowledge Scale, there was a statistically significant difference between the two annotation types, F (1, 74) = 8.40, p = .0049 < .05. In other words, participants in the audio-picture annotation type did si gnificantly better than participants in the text-pic ture annotation type on the Vo cabulary Knowledge Scale. A statistically significant effect of annotation was also f ound in the Word Recognition Test, F (1, 74) = 7.15, p = .0092 < .05. This confirms that the audio-picture annotation type resulted in better performance in Word Recognition Test than the text-picture annotation type. Table 4-21 Means and Standard Deviations for Vo cabulary Posttests by Annotation VKS WRT Annotation n M SD M SD Text-picture 39 23.55 7.14 21.67 7.80 Audio-picture 39 27.75 5.20 26.13 6.11

PAGE 132

117 Table 4-22 Means and Standard Deviations for Readi ng Comprehension by Annotation RC Written Recall Annotation n M SD M SD Text-picture 39 6.77 2.23 14.95 6.90 Audio-picture 39 6.49 1.76 13.79 5.86 Means for Vocabulary Posttest by Annotation0 5 10 15 20 25 30 VKSWRT Text-picture Audio-picture Figure 4-14 Means for Vocabulary Pos ttests by Annotation Type In summary, the results indicate that audio-picture annotation resulted in significantly better performance in vocabular y immediate recall than the text-picture annotation. Thus, hypothesis 1, whic h states that students in the audio-picture annotation group will perform significantly better than st udents in the text-picture annotation group was supported.

PAGE 133

118 Question 2 – Effects of Annotation Type on Reading Comprehension The second research question was focuse d on the relationship between annotation type and reading comprehension. The results of the two comprehension tests (Reading Comprehension Test and Written Recall) in te rms of annotation were presented in Table 4-22 and Figure 4-16. For reading comprehension, Table 4-22 s hows that the results for the Multiplechoice Reading Comprehension Test between the two annotation types were very close (M = 6.77 and M = 6.49 respectively). The diffe rence for the Written Recall between the two annotation types was also very sma ll (M = 14.95 and M = 13.79 respectively). As indicated by Figure 4-15, the te xt-picture annotation particip ants slightly outscored the audio-picture annotation participants in terms of reading comprehension. Means for Reading Comprehension by Annotation Type0 2 4 6 8 10 12 14 16 18 20 RCRecall Text-picture Audio-picture Figure 4-15 Means for Reading Comprehension by Annotation Type

PAGE 134

119 An ANOVA analysis did not show any si gnificant effect for annotation type on reading comprehension in either of the two reading comprehension tests: Multiple-choice Reading Comprehension Test (F = .44, p = .5089 > .05) and Written Recall (F = .90, p = .3464 > .05). Because the interaction between annotation type and learning condition was statistically significant for Written Reca ll, independent sample T-tests shows that participants in the text-pictu re annotation group did significan tly better than participants in the audio-picture annotation group in in tentional learning condition. Therefore, hypothesis 2, which states that students in the audio-picture annotation group will perform statistically better than participants in the text-picture annotation group on the reading comprehension postte sts, was not supported. Question 3 – Effects of Learning C ondition on Vocabulary Immediate Recall The third research question examined if a particular l earning condition might affect vocabulary immediate r ecall when participants were reading an English text for comprehension. The results of the two vo cabulary posttests (Vocabulary Knowledge Scale and Word Recognition Test) in terms of learning condition were complied in Table 4-23 and Figure 4-16. Regarding vocabulary immediate recall, Table 4-23 shows that the intentional learning condition led to better results ( VKS M=26, WRT M=16.92) than the Incidental learning condition (VKS M=21.9 WRT M=14 .25). As indicated by Figure 4-16, the intentional learning condition pa rticipants outscored the in cidental learning condition participants consistently across the two vocabulary posttests (Vocabulary Knowledge Scale and Word Recognition Test).

PAGE 135

120 For the Vocabulary Knowledge Scale, there was a statistically significant difference between the two learning condi tions, F (1, 74) = 13.12, p = .0009 < .05. In other words, participants in the intentional learning conditi on did significan tly better than participants in the incidental learning condition in the Voca bulary Knowledge Scale test. For the Word Recognition Test, a statistically significant effect of learning was found, F (1, 74) = 19.16, p < .0001< .05. This means that participants in the intentional learning condition retained significantly more target wo rds than participants in the incidental learning condition. For immediate vocabulary recall, to summa rize, the intentional learning condition resulted in significantly better performance th an the incidental learning condition. Thus, hypothesis 3, which states that students in the intentional learni ng condition will perform significantly better than stude nts in the incidental lear ning condition on the immediate vocabulary posttests, was supported. Table 4-23 Means and Standard Deviations for Vo cabulary Posttests by Learning VKS WRT Learning n M SD M SD Incidental 40 21.9 7.54 14.25 3.29 Intentional 38 26 6.52 16.92 2.46

PAGE 136

121 Means for Vocabulary Posttest by Learning Condition0 5 10 15 20 25 30 VKSWRT Incidental Intentional Figure 4-16. Means for Vocabulary Pos ttest by Learning Condition Question 4 – Effects of Learning Condition on Reading Comprehension The focus of research question four wa s whether the two learning conditions had different effects on the particip ants’ reading comprehension. To answer this question, the results of the two reading comprehension measures (Read ing Comprehension Test and Written Recall) were presented in Table 4-24 and Figure 4-17. As indicated by Table 4-24 and Figure 417, different results were found for the two reading comprehension measures. In the Multiple-choice Reading Comprehension Test, participants in the two learning conditions had very close scores (incidental M = 6.82, intentional M =6.45). In the Written Recall however, participants in the incidental learning condition did better (M =16.3) than participants in the intentional learning condition (M =12.34).

PAGE 137

122 For the Multiple-choice Reading Comprehens ion Test, the effect of learning condition was not found to be significant (F = .66, p = .4209 > .05). This means that there was no significant difference between the two learning conditions on the multiplechoice reading comprehension test. For the Written Recall, however, there was a significant effect for learning condition (F = 8.97, p = .0037 < .05). This indicates that participants in the incidental learning condi tion did significantly be tter than participants in the intentional learning conditi on in recalling the reading text. Therefore, the incidental learning condition resulted in significantly better reading comprehension than the intentional learning condition in the Written Recall measure, but not in the multiple-choice Reading Compre hension Test. Thus hypothesis 4, which states that students in the incidental lear ning condition will perform significantly better than participants in the intentional lear ning condition on the reading comprehension posttest, was partially supported. Table 4-24 Means and Standard Deviations for Re ading Comprehension by Learning RC Written Recall Learning n M SD M SD Incidental 40 6.45 2.06 16.3 5.73 Intentional 38 6.82 1.94 12.34 6.48

PAGE 138

123 Means for Reading Comprehension by Learning Condition0 2 4 6 8 10 12 14 16 18 20 RCRecall Incidental Intentional Figure 4-17 Means for Reading Comprehension by Learning Condition Question 5 – Interaction between Annotati on and Learning on Vocabulary Immediate Recall The fifth research question examined the degree of interactio n between annotation type and learning condition on tasks involving vocabulary imme diate recall. In order to answer this question, all the interaction effects between annotation type and learning condition for vocabulary measurements were re visited. Table 4-25 li sts the interaction between the two factors for the Vocabulary Knowledge Scale and the Word Recognition Test. As indicated by Table 4-25, there was no in teraction between annotation type and learning condition across the two tests. This helps to explain that the learning condition difference did not have any effect on the annot ation type. In other words, regardless of the learning condition, the results found in re search question one remained the same, namely that the audio-picture annotation group significantly performed better than the

PAGE 139

124 text-picture annotation group on vocabulary immediate recall Thus, Hypothesis 5, which states that there will be an interaction be tween learning condition and annotation type on L2 vocabulary immediate recall, was not supported. Table 4-25 A Summary of Interaction between Annota tion and Learning on Vocabulary Immediate Recall Source (Annotation x Learni ng) df, df F Value P Value VKS 1, 74 .21 .6463 WRT 1, 74 1.67 .1998 Question 6 – Interaction between Annotati on and Learning on Reading Comprehension Research question six investigated whether there is interaction between annotation type and learning condition on partic ipants’ reading comprehension. In order to answer this question, all the interaction effects between annotation type and learning condition for reading comprehension measures were revisited and shown in Table 4-26. As indicated by Table 4-26, there was no in teraction between annotation type and learning condition on reading comprehension scores in the Multiple-choice Reading Comprehension Test (F = 2.40, p = .1257 > .05). However, in the Written Recall, there is significant interaction between learning condi tion and annotation type. This can be interpreted that the difference between the tw o annotation types in written recall depends on the learning conditions (F = 8.49, p = .0047 < .05). In other words, in the incidental learning condition, the difference between te xt-picture annotation and audio-picture

PAGE 140

125 annotation was not significant; in the inten tional learning condition, participants in textpicture did significantly better than those in audio-picture on Written Recall. Therefore, hypothesis 6, which states that there will be an interaction between learning condition and annotation type on L2 reading comprehensi on, was partially supporte d. Whether there is a significant interaction depe nds on the task type. Table 4-26 A Summary of Interaction between Annota tion and Learning on Reading Comprehension Source (Annotation x Learning) df, df F Value P Value RC 1, 74 2.40 .1257 Written Recall 1, 74 8.49 .0047 ** Summary of Findings The overarching question addressed in th is study concerns how different dual annotations affect L2 vocabul ary learning and reading compre hension in both incidental and intentional settings. Text-picture annotat ion and audio-picture annotation were the two dual annotations un der investigation. Two vocabulary posttests were used to measure the participants’ vocabulary immediate recall: Vocabulary Knowledge Scale and Word Recognition Test. Two comprehension posttests were used to measur e the participants’ r eading comprehension: Reading Comprehension Test (Multi ple-choice) and Written Recall. The following presents the findings for each of the research questions:

PAGE 141

126 Question 1: What are the di fferent effects of text-p icture and audio-picture annotations in facilitating L2 vocabulary immediate recall? Findings: The audio-picture annotation was the more effective annotation type in facilitating L2 vocabulary immediate recall. Question 2: What are the different effects of text -picture and text-audio annotations in facilitating L2 reading comprehension? Findings: Text-picture annotation and a udio-picture annotation did not have different effects in facilit ating L2 reading comprehension. However in intentional learning condition, the text-pictu re annotation did lead to significantly better written recall than the audio-picture annotation. Question 3: Which learning condition (inten tional or incidental ) results in better L2 vocabulary immediate recall? Findings: The intentional learning c ondition resulted in better L2 vocabulary immediate recall. Question 4: Is there a significant difference in ESL students’ reading comprehension under the two learning c onditions (intentional or incidental)? Findings: the Incidental lear ning condition resulted in significantly better L2 reading comprehension than the intentiona l learning condition in the Written Recall measure, but not in the multiple-choice Reading Comprehension Test. Question 5: Is there an interaction betw een learning condition and annotation type on immediate L2 vocabulary immediate recall?

PAGE 142

127 Findings: There was no interaction betw een learning condition and annotation type on L2 vocabulary immediate recall. Question 6: Is there an interaction betw een learning condition and annotation type on L2 reading comprehension? Findings: There was no interaction betw een learning condition and annotation type on multiple-choice Reading Comprehension test, but the interaction between learning condition and annotation type wa s significant on Written Recall. Table 427 A Summary of the Findings in the Study Posttests VKS WRT RC Recall Annotation ** Audio > Text ** Audio > Text ns ns Learning ** Intentional > Incidental ** Intentional > Incidental ns ** Incidental > Intentional Interactions (Annotation x Learning) ns ns ns ** ** = significant (p < .05), ns = not signi ficant, Audio = audio-text annotation, Text = text-picture annotation, Intenti onal = intentional le arning condition, Incidental = incidental learning condition.

PAGE 143

128 The findings for the 6 research questions are summarized in Table 4-27. The next chapter will present discussion on the results, offer theoretical and pedagogical implications, make recommendations for future research, and provide final conclusions.

PAGE 144

129 Chapter V Discussion This dissertation examined the different effects of text-picture annotation and audio-picture annotation on L2 vocabul ary immediate reca ll and L2 reading comprehension under incidental and intentiona l learning conditions. After an introduction to the problem in Chapter I, Chapter II revi ewed in details the related literature on annotation studies, Chapter III described the method of data collec tion and analysis, and Chapter IV presented the results of the study. This final chap ter will present the interpretations of the results addressing each research question, discuss both theoretical and pedagogical implications, make recomme ndations for future research, and provide final conclusions. Interpretations of the Results Based on the results presented in Chapter IV, the interpretations will address each of the research questions, make references to the literature, and e xplain possible reasons for the obtained results. Effect of Annotation Type on L2 Vocabulary Immediate Recall The first research question addressed the effectiveness of multimedia annotations. On average, participants retained 59.8% of the 20 target words on the Vocabulary Knowledge Scale and 77.7% of the 20 target words on the Word Recognition Test. It was expected that the Vocabulary Knowledge Scale, as a definition-supply test, would

PAGE 145

130 have lower scores than the Word Recogniti on Test. The average retention rates were comparable to previous multimedia annotat ion studies (e.g. Al-Seghayer, 2001; Chun & Plass, 1996a; Yoshii, 2000). The results c onfirmed the effectiveness of multimedia annotation in facilitating second language vo cabulary learning. A fu rther explanation is the dual-coding effect (Paivio, 1971, 1986, 1990) th at words annotated with both verbal (text or audio) and visual (picture) modes of information lead to effective vocabulary retention. However, the retention rate of 59.8% (54.8% if only counting incidental vocabulary retention) on the Vocabulary K nowledge Scale was much higher than the results reported in definitionsupply tests from previous st udies: 25.9% in Chun and Plass (1996a) and 21.4% in Yoshii (2000). This higher rate could be explai ned by the nature of the task used for this study. The Vocabular y Knowledge Scale combines recognition and production in that points were given to th e participants if the target words were recognized in the self-rating scale. This study was designed to compare the effectiveness of text-picture annotation with audio-picture annotati on on L2 vocabulary immediate re call. As shown by Table 414 and Figure 4-14, the audio-picture annotat ion group consistently outperformed the text-picture annotation group, regardless of different postt est measures. The difference was statistically significant for both the Vocabulary Knowledge Scale (effect size f =.33) and Word Recognition Test (effect size f = .41) The dual channel assumption, especially th e modality principle, of the cognitive theory of multimedia learning can be used to explain this finding (Mayer, 2001). Mayer distinguishes the two separate channels for pr ocessing visual/pictorial and auditory/verbal

PAGE 146

131 information. The modality effect articulat es that working memory has partially independent processors for handling visual and auditory information. The effective capacity of working memory could be incr eased by using both visual and auditory channels (Mayer & Moreno, 1998; Mousav i, Low & Sweller, 1995; Penny, 1989). Text annotation and audio annotation ar e both verbally-presented information, and picture annotation is non-verbally-prese nted (pictorial) info rmation; thus both multimedia annotations contain a combinati on of verbal and non-verbal information. Based on the modality principle (Baddeley, 1986, 1992, 1999; Mayer, 2001), text annotation and picture annotation will be pro cessed by the visual channel, while audio annotation will be processed by the auditory channel. Therefore, in text-picture annotations, the simultaneous register of both text and picture cause d the visual channel to be overloaded. This led to an information processing that was, at least initially, carried out solely in the visual working memory. T hus, the cognitive resources available in the visual working memory had to be divided between textual and pictorial information, whereas the auditory (p honological) working memory was left unused. In comparison, in audio-picture annota tions, the audio was registered by the auditory channel and processed in the phonol ogical working memor y, while the picture was registered by the visual channel and pro cessed in the visual working memory. This combination allowed cognitive resources in both working memories to be used. In other words, more cognitive resources were utilized in audio-picture annot ations than in textpicture annotations. The superiority of audio-picture annota tion on L2 vocabulary immediate recall can also be explained with the split-a ttention principle (M ayer & Moreno, 1998;

PAGE 147

132 Mousavi, Low, & Seller, 1995). Participants with access to text-picture annotations had to split their attention in the visual work ing memory between multiple visual resources (written text and picture). Participants with access to audi o-picture annotations approached the audio as an auditory resource and the picture as a vi sual resource through auditory working memory and visual working memory respectively, which did not require an attention split in either of th e working memories. In this way, effective working memory might be increased by pres enting information in a mixed (visual and auditory) rather than a unitary mode (vis ual only). Hence, au dio-picture annotation resulted in higher vocabulary immediate recall than text-pic ture annotation. Effect of Annotation Type on L2 Reading Comprehension The second research question focused on the effects of different multimedia annotations on L2 reading comprehension. The two multimedia annotations compared were text-picture and audiopicture. The ANOVA results did not show any significant effect for annotation type on L2 readi ng comprehension for either of the two comprehension measures (Reading Comprehe nsion Test, F =.44, p = .5089; and Written Recall, F = 3.38, p = .0700). This fails to s upport the modality effect of the cognitive theory of multimedia learning (Mayer, 2001). Studies have reported the s uperiority of a combination of audio and picture in comparison to a combination of text and pi cture when presenting new knowledge (Mayer & Anderson, 1991; Mayer & Moreno, 1998; Mo reno & Mayer, 1999). According to the modality principle, audio-picture annotati on simultaneously engages both the visual working memory and auditory working memor y, while text-picture annotation involves

PAGE 148

133 only the visual working memory; therefore, audio-picture annotation enables more application of available cogniti ve capacity and should conseque ntly lead to more content recall. The results from this study did not indicate the superiority of audio-picture annotations over text-p icture annotations. If reexamined from the data results, as indicated by Table 4-22 and Figure 4-15, text-picture annotations, in contrast to prev ious findings, resulted in better results (RC M = 6.77, Recall M = 14.95) than audio-picture annotations (RC M = 6.49, Recall M = 13.79), with larger differences in the Wr itten Recall measure. The difference was statistically significant in the intentional learning condition. This could be tentatively explained from the perspective of learning st yle. Most of the part icipants (78%) were from Asian countries (Japan, Taiwan, Korean, etc). Previous studies have reported on the preference of visual learning styles over a uditory learning styles among Asian students (e.g. Ye & Wang, 2003). For this study, the visual information presented via text-picture annotation might act as better retrieval cues, in comparison to the visual and auditory information presented via audio-picture annotat ion, to help the participants when taking the comprehension tests. Effect of Learning Condition on L2 Vocabulary Immediate Recall The third research question dealt with the effects of learning condition on L2 vocabulary immediate recall. In particular, two learning c onditions, incidental and intentional, were compared in this study. As shown by Table 4-23 and Figure 4-16, the intentional learning condition led to better vocabulary immediate recall (VKS M =26,

PAGE 149

134 WRT M= 16.92) than the incidental learning condition (VKS M = 21.9, WRT M = 14.95). The differences in both tests were st atistically significant (effect size f was respectively .30 for VKS and .50 for WRT). Th e finding confirms Konop ak et al (1987) who reported significantly better vocabulary learning in the in tentional learning than in the incidental learning condition. However, this differs from the results obtained by Ko (1995). In Ko’s (1995) study, intentional learning was only found to be slightly better than incidental learning (M=18.70 vs. M=18.09), and the difference was not significant. Ko (1995) explained the non-significant difference between learning cond itions in terms of the attention that students paid to the annotations. Student re action to the annotation as collected via a questionnaire indicated a small gap between the two learning conditi ons in the amount of effort they put into the annotation. In other words, participants in the incidental learning condition paid as much attention to the ta rget words as their counterparts in the intentional learning condition. Schmidt (1994) points out that incidental learning is the condition in which an individual learns “without the intent to learn or the lear ning of one thing (e.g. grammar) when the learner’s primary objective is to do something else (e.g. communicate)” (p. 9). This study followed exactly the division betwee n incidental learning and intentional as defined by Hulstijn (1993) by providing diffe rent objectives for different groups. In incidental learning, learner at tention was not focused on th e target words, while in intentional learning, learner attention was focused on both the target words and the reading text. Therefore, alt hough in incidental learning, lear ners might pick up some new words as a by-product while reading, their at tention was to understand the meaning of the

PAGE 150

135 reading text; hence, those new words could be temporarily held in short-term memory, but could not be processed into long-term me mory due to the lack of attention. In contrast, however, intentional learning, in which the intention was generated by a perdetermined objective, motivated the learner’s a ttention to be focused on the target words, thus leading to better word retention after reading. The learning of a new word in a seco nd language depends on how much mental effort the reader devotes to that particular word, especially for productive vocabulary. An interesting finding worth mentioning is that the gap between the tw o learning conditions was larger in the Vocabulary Knowledge Scal e than in the Word Recognition Test (as seen from Figure 4-11). This c ould be explained by the different nature of the two tasks. A Word Recognition Test is used to meas ure receptive vocabulary, while a Vocabulary Knowledge Scale tends to measure both rece ptive and productive knowledge of the target words. Effects of Learning Condition on L2 Reading Comprehension This study attempted to compare tw o learning conditions, incidental and intentional, in their effects on L2 reading comprehension. Since multimedia annotations were provided for target words in the readi ng text, the two conditions were predetermined on the objective for vocabulary immediate recall. In addi tion, vocabulary knowledge is regarded as an important pr edictor for L2 reading comprehension (Bossers, 1992, cited in Pulido, 2004; Haynes & Baker, 1993; Laufer, 1992) Therefore, the discussion is actually focused on comparing the effects of incide ntal vocabulary learning and intentional vocabulary learning on L2 reading comprehension.

PAGE 151

136 It was expected that the incidental vo cabulary learning condition would lead to better reading comprehension because the participants were not responsible for vocabulary retention and their attention was steadily focused on comprehending the text. While in intentional vocabular y learning, the participants we re responsible for vocabulary learning and their attention was focused on learning the target words. Due to limited cognitive capacity, participants would shift more attention to vocabulary learning and pay less attention to comprehendi ng the text. In other words, the intentional vocabulary learning condition more seriously interrupts the reading process and results in worse reading comprehension. As shown by Table 4-24 and Figure 4-17, the results for multiple-choice Reading Comprehension Test were very close for the two learning conditions (incidental M = 6.45, intentional M = 6.82). However, the resu lts for the Written Recall presented a better performance by the incidental learning c ondition participants (M = 16.3) than by the intentional learning conditi on participants (M = 12.34), and the difference was statistically significant (effect size f = .34). This could be explained from the perspect ive of focused attention (Ko, 1995). In the incidental learning condition, participants were told in ad vance that they would have reading comprehension tests; it was anticipat ed that they would focus their attention solely on the content. Whereas in intentiona l learning condition, partic ipants were told in advance that they would have both vocabular y tests and reading comprehension tests, thus focusing their attention on both the content and vocabulary simultaneously. The focused attention in the incidental lear ning condition resulted in better reading comprehension than divided attention in the intentional le arning condition.

PAGE 152

137 From another point of view, the Multiplechoice Reading Comprehension test and The Written Recall are two different tasks. Different tasks may, by their nature, influence an L2 reader’s performance on reading comprehension (Lee, 1986; Riley & Lee, 1996; Wolf, 1996). A multiple-choice test is a discrete-point task that focuses on isolated bits of information. In contrast, writ ten recall is an integrat ive task that focuses on global understanding of the reading text. Resear chers (Bernhardt, 1983, 1991; Swaffer et al., 1991) have argued for the use of recall protocol, instead of multiple-choice test, as a measure of global or holistic reading comprehension. A multiple-choice test itself suggests possible answers and it sometimes tends to be independent of the reading text (Berhardt, 1983). In addition, the questi ons in the multiple-choice test, especially those distractors, might cause the reader to ch ange his or her representation of the reading text. In comparison, writte n recall seems to more accura tely reflect comprehension, especially in its proposi tional content of the text reconstructi on. It is regarded to be more text-dependent as articulated by Berhardt (1991) that recall is a good measure because “generating recall data does not influence the reader’s understanding of a text” (p. 200). Interaction between Annotation Type and Learning Condition The fifth and sixth research questions focused on the interaction between annotation type and learning condition on L2 vocabulary immediat e recall and reading comprehension. As shown by Table 4-25, ther e was no interaction between annotation type and learning condition on L2 vocabulary immediate recall. This means that the superiority of audio-picture over text-picture in facilitating L2 vocabulary immediate recall does not depend on learning condition.

PAGE 153

138 The interaction between annotation type and learning condition on L2 reading comprehension needs to be addressed by the different comprehension measures. For the Multiple-choice Reading Comprehension Test, there was no interaction between the two factors. For the Written Recall, however, th e interaction between annotation type and learning condition was found to be significant. This means that the performance of the participants on the written recall in different annotatio n types depends on learning condition. Upon closer examination of the re sults for research question 6 (see Figure 413), the scores on written recall by the part icipants in the audi o-picture annotations declined as the learning conditi on changed from incidental to intentional, while the score by the participants in the text-p icture annotations did not chan ge much in the two learning conditions. The decline in audi o-picture reflected the focusattention principle used to explain the better reading co mprehension performance in the incidental learning group, but the performance in text-picture did not reflect this principle. Additionally, whether the effects of learning conditions on L2 reading comprehension are related to annotation type depends on the task type. As indicated by Table 4-26, the multiple-choice reading comprehension Test, as a discrete-point testing method, did not reveal any interaction betw een learning condition and annotation type. However, the written recall protocol, as an overall comprehension testing method, did present a significant interaction between l earning condition and annotation type. This reflects that the choice of different task types might cause different findings.

PAGE 154

139 Theoretical Implications This dissertation adds to the growing body of research in multimedia annotation studies in second la nguage acquisition. Previous multimedia annotation studies have focused on the comparison of text-picture a nnotation to text-only annotation or pictureonly annotation (Jones, 2003; Yoshii, 2000) or on the differences be tween text-picture annotation and text-video annotation (Al-Seghayer, 2001; Chun & Plass, 1996a, 1996b). However, audio annotation, as a different sens ory modality from visual (text, picture), has never been studied before The present study fills this gap in the literature. This study provided the much-needed in formation on the effect of audio annotation on L2 vocabulary l earning. By comparing audiopicture annotation to textpicture annotation, it shed light on the use of different dual annotations for multimedia L2 learning. The dissertation has established that audio-picture annotati on is superior over text-picture annotation in fac ilitating L2 vocabulary immediat e recall. This contributes to the extension of the cognitive theory of mu ltimedia learning to second language learning by verifying both the modality effect and splitattention effect. In addition, the present study has determined that in multimedia envi ronments, the intentional learning condition leads to better vocabulary immediate recall th an the incidental learning condition. This adds to the existing litera ture of promoting intenti onal vocabulary leaning. In terms of L2 reading comprehension, the incidental lear ning condition was found to be more facilitative of L2 reading comprehension, as reflected in the written recall protocol. However, the learning condi tion was not found to have any effect on the multiple-choice reading comprehension test.

PAGE 155

140 Although this dissertation has made important contributions to multimedia annotation research in second language ac quisition, some questions still remain unanswered, such as the long-term effects of different multimedia annotations, the effects of different dual multimedia annotations on L2 reading comprehension, and the effects of incidental and intentional vocabulary learni ng conditions on L2 reading comprehension as measured by different tasks. Pedagogical Implications In addition to the contributions and imp lications for the field of second language acquisition, especially in the area of multimedia annotation research, this study carried pedagogical implications. First of all, the study provides some insights for CALL material designers in choosing the right combination of modalities in facilitating L2 vocabulary learning. This study confirmed that the use of audio-pictur e combinations facilitates L2 vocabulary immediate recall in a more effective manner th an text-picture annotation. In designing multimedia courseware or materials, this fi nding could be taken into consideration when making decisions about presenting information in different modes. This could also inform language teachers and administrators in making decisions about the most effective multimedia programs to enhance L2 vocabulary learning. Participants in the intentional learni ng condition outperformed participants in the incidental learning condition in L2 voca bulary immediate recall. Different learning conditions stimulate different allocation of students’ attention de pending on what is regarded as important as defined by the le arning objective. With respect to vocabulary

PAGE 156

141 learning, intentional learning should be encouraged if th e final learning goal is vocabulary acquisition. This implies that drawing students’ atte ntion to a specific learning objective when they are engaged in a task can have a positive impact on their learning performance. In terms of measuring L2 reading comp rehension, this study shows that written recall might be a more reliable and accurate testing method to gauge L2 learners’ overall or global comprehension of th e reading text. This informs language teachers in choosing the most appropriate tasks based on the purpose of testing, the nature of the reading text and proficiency of the learners In addition, it seems that incidental vocabulary learning results in better reading comprehension becau se the students’ attention is focused on the content of the reading text. Directions for Future Research Due to the nature of this study, only imme diate posttests were used to measure L2 vocabulary recall. Future research should invol ve delayed posttests to attest whether the superior effect of audio-picture annotation ca n be retained over time. In addition, instead of conducting comparative studies, research c ould focus on a particular multimedia dual annotation such as audio-picture to examin e its effects on L2 vocabulary immediate recall as well as long-term retention. This study tracked the frequency records of participants’ look-up behaviors. Although the look-up behavior was not found to be correlated to the posttest measures, it is worthwhile for future research to use a device to keep track of the length of time participants spent on th e annotations and include time-on-ta sk as a possible variable. In

PAGE 157

142 addition, possible Hawthorne Effect should be cautioned when participants getting different treatments are placed in one room. Further study should be car ried out to investigate the effects of different annotations on L2 reading comprehension. Si nce this study did not find any significant difference between text-picture annotation and audio-picture annot ation on L2 reading comprehension, it might be a better approach to exclusively examine their effects on reading comprehension. This study involved participants from mu ltiple first languages. The L1 written recall task might, in some way, have skewed the recall data during the translation process because different languages have different rh etorical styles. Fu ture research could engage more linguistically homogeneous part icipants in order to alleviate this shortcoming. Furthermore, this study had different results for reading comprehension from the different tasks. Other holistic co mprehension measures such as summary or think-aloud protocol could be used as complementary methods to measure comprehension. This dissertation used intermediate ESL students as the participants. More crosssectional studies should be carried out to ex amine the effects of multimedia annotations. As well, instead of using a between-subject s design, a within-subjects design could be employed to decrease the influe nce of individual differences. Conclusions Previous studies have examined the e ffects of multimedia annotations on L2 vocabulary learning and reading comprehens ion. These studies ha ve supported the

PAGE 158

143 effectiveness of multimedia a nnotations in facilitating L2 vocabulary learning. However, no study in second language acquisition has ex amined audio annotation in combination with text as a dual multimedia annotation type This dissertation fo cused on this issue by comparing audio-picture annotation to text-p icture annotation in their effects on L2 vocabulary immediate recall and reading compre hension under incidental and intentional learning conditions. The results of the study demonstrate th at audio-picture annotation is more effective than text-picture annotation in f acilitating L2 vocabulary immediate recall, an intentional learning condition is more effective than an inci dental learning condition in promoting L2 vocabulary immediate recall. Furthermore, an incidental vocabulary learning condition results in bett er reading comprehension in written recall than in an intentional vocabulary le arning condition.

PAGE 159

144 References Al-Seghayer, K. (2001). The effect of mu ltimedia annotation modes on L2 vocabulary acquisition: A comparative study. Language Learning & Technology, 5 (1), 202-232. Anderson, R. C., & Freebody, P. (1981). Vocabulary knowledge. In J. Guthrie (Ed.), Comprehension and teaching: Research reviews (pp. 77-117). Newark, DE: International Reading Association. Baddeley, A. D. (1986). Working Memory Oxford, England: Oxford University Press. Baddeley, A. D. (1992). Working Memory. Science, 255 556-559. Baddeley, A. D (1999): Working Memory (Oxford Ps ychology Series, No 11). New York: Oxford University Press Berhardt, E. B. (1983). Testing foreign langua ge reading comprehension: The immediate recall protocol. Die Unterrichtspraxis, 16 17-33. Berhardt, E. B. (1991). Reading Development in a Second Language Norwood, NJ: Ablex. Bergman, J. R. (1978). Reducing reading fr ustration by an innovative technique for vocabulary growth. Reading Improvement, 14 (3), 168-171.

PAGE 160

145 Carmines, E. G. & Zeller, R.A. (1991). Reliability and validity assessment Newbury Park: Sage Publications. Chun, D. M., & Plass, J. L. (1996a). Effect s of multimedia annotations on vocabulary acquisition. The Modern Language Journal, 80 (2), 183-198. Chun, D. M., & Plass, J. L. (1996b). F acilitating reading comprehension with multimedia. System, 24 (4), 503-519. Clark, J. M., & Paivio, A. (1991). Dual coding theory and education. Educational Psychology Review, 3, 149-210. Coady, J. (1993). Research on ESL/EFL vocabular y acquisition: Putting it in context. In: T. Huckin, M. Haynes, & J. Coady (Eds.), Second Language Reading and Vocabulary Learning. Norwood, NJ; Ablex, 3-23. Coady, J. (1997). L2 vocabulary acquisition: A synthesis of the research. In Coady & T. Huckin (Eds), Second Language Vocabulary Acquisition (pp. 273-290). NY: Cambridge University Press. Coady, J. & Huckin, T. (1997) (Eds). Second Language Vocabulary Acquisition NY: Cambridge University Press. Cronbach, L. J. (1951). Coefficient alpha and the internal st ructure of tests. Psychometrika, 16 297-333.

PAGE 161

146 Doughty, C., & Williams, J. (Eds.). (1998). Focus on form in classroom second language acquisition Cambridge: Cambridge University Press. Davis, J. N. (1989). Facilitati ng effects of marginal glosses on foreign language reading. The Modern Language Journal, 73 41-48. Davis, J. N., & Lyman-Hager, M. (1997) Computers and L2 reading: Student performance, student attitudes. Foreign Language Annals, 30 (1), 59-72. Dreamweaver MX [Computer Spftware]. (2004). Macromedia. Ellis, R. (1994). Factors in the incidental acquisition of second language vocabulary from oral input: A review essay. Applied Language Learning, 5 (1), 1-32. Folse, K. (2005) Vocabulary Myth: Applying Second Language Research to Classroom Teaching University of Michigan Press. Folse, K., & Chien, Y. (2003). Using L2 rese arch on multimedia annotations to evaluate CALL vocabulary materials. Sunshine State TESOL Journal, 2 25-37. Francis, W. N., & Kucera, H. (1982). Frequency Analysis of E nglish Usage: Lexicon and Grammar Houghton Mifflin Company: Boston. Gass, S. (1987). (Ed). The use and acqui sition of the second language lexicon. Studies in Second Language Acquisition 9(2).

PAGE 162

147 Gass, S. (1999). Incidental vocabulary learning. Studies in Second Language Acquisition, 21 (2), 319-333. Grabe, W. & Stoller, F.L. (1997). Readi ng and vocabulary development in a second language: a case study. In: Coady, J. & Huckin, T. (Eds.), Second Language Vocabulary Acquisition. Cambridge: Cambridge University Press, 98-122. Grafton, S. (1987). A is for Alibi Crimeline Herron, C., Hanley, J., & Cole, S. (1995). An i nvestigation of the e ffectiveness of using an advance organizer to introduce vi deo in a foreign language classroom. The Modern Language Journal, 78 (2). 387-395 Haynes, M., & Baker, I. (1993). American and Chinese readers learning from lexical familarization in English text. In T. Huckin, M. Haynes, & J. Coady (Eds.), Second language reading and vocabulary learning (pp.130-52). Norwood, N. J.: Ablex Publishing Corporation. Henriksen, B. (1999). Three dimensi ons of vocabulary development. Studies in Second Language Acquisition, 21 303-317. Holly, F., & King, J. (1971). Vocabulary glosse s in foreign language reading materials. Language Learning, 21(2), 213-219. Huckin, T., & Coady, J. (1999). Incidental vocabulary acquisition in a second language: A review. Studies in Second Language Acquisition, 21 (2), 181-193.

PAGE 163

148 Hulstijn, J.H (1992). Retention of inferred a nd given word meanings: Experiments in incidental vocabulary learning. In: P.J.L Arnaud & H. Bejoint (Eds.), Vocabulary and Applied Linguistics, London: Macmillan, 113-125. Hulstijn, J.H (1993). When do foreign-la nguage readers look up the meaning of unfamiliar words? The influence of task and learner variables. The Modern Language Journal, 77 (2), 139-147. Hulstijn, J. H. (2001). Intentional and incide ntal second language vocabulary learning: A reappraisal of elaboratio n, rehearsal and automaticity. In P. Robinson (Ed.), Cognition and second language instruction (pp. 258-286). Cambridge University Press. Hulstijn, J. H., Hollander, M., & Greidanus, T. (1996). Incidental vo cabulary learning by advanced foreign language stude nts: The influence of marg inal glosses, dictionary use, and reoccurrence of unknown words. The Modern Language Journal, 80 (3), 327-339. Jacobs, G. M. (1994). What lurks in the marg in: Use of vocabulary glosses as a strategy in second language reading. Issues in Applied Linguistics, 5 (1), 115-137. Jacobs, G. M., Dufon, P., & Hong, F. C. ( 1994). L1 and L2 vocabulary glosses in L2 reading passages: Their effectiveness for increasing comprehension and vocabulary knowledge. Journal of Research in Reading, 17 (1), 19-28.

PAGE 164

149 Jones, L. C. (2003). Supporting listening co mprehension and vocabulary acquisition with multimedia annotations: The students’ voice. CALICO Journal, 21 (1), 41-65. Joyce, E. E. (1997). Which words should be gl ossed in L2 reading materials? A study of first, second and third semest er French students’ recall. Pennsylvania Language Forum 58-64. Kang, S. (1995). The effects of a contextembedded approach to second-language vocabulary learning. System, 23 (1), 43-55. Kellogg, K. S., & Howe, A. J. A. (1971). Usi ng words and pictures in foreign language learning. Alberta Journal of Educational research, 17 87-94. Knight, S. (1994). Dictionary: The tool of last resort in foreign la nguage reading? A new perspective. The Modern Language Journal, 78 (3), 285-299. Ko, M. H. (1995). Glossing in incidental and intentional learning foreign language vocabulary and reading comprehension. U npublished MA thesis, University of Hawaii at Manoa. Koren, S. (1999). Vocabulary instruction thr ough hypertext: Are ther e advantages over conventional Methods of teaching? TESL-EJ 4(1). Available online: http://wwwwriting.berkeley.edu/TESL-EJ/ej13/a2.html Kost, C. R., Foss, P., & Lenzini, J. J.,J r. (1999). Textual and pictorial glosses: Effectiveness on incidental vocabulary grow th when reading in a foreign language. Foreign Language Annals, 32 (1), 89-113.

PAGE 165

150 Krashen, S. (1989). We acquire vocabulary a nd spelling by reading: Additional evidence for the input hypothesis. Modern Language Journal, 73 440-464. Laufer, B. (1990). Ease and difficulty in vocabulary learning: Some teaching implications. Foreign language Annals, 23 (2), 147-155. Laufer, B. (1992). How much lexis is necessa ry for reading comprhension? In P. Arnaud & H. Bejoint (Eds.), Vocabulary and applied linguistics (pp. 126-132). London: Macmillan. Laufer, B. and Hill, M.. (2000). What lexical information do L2 learners select in a CALL dictionary and how does it affect word retention? Language Learning and Technology 3/2: 58-76. Available online: http://llt.msu.edu/vol3num2/lauferhill/index.html Laufer, B., & Hulstijn, J. ( 2001). Incidental voca bulary acquisition in a second language: The construct of task-induced involvement. Applied Linguistics, 22 (1), 1-26. Laufer, B. & Sim, D. (1985). An attempt to measure the threshold of competence for reading comprehension. Foreign Language Annals, 18(5), 405-411. Laufer, B. & Shmueli, K. (1997). Memorizi ng new words: does teaching have anything to do with it? RELC Journal 28 (1), 89-108. Lee, J. F. (1986). On the use of recall ta sk to measure L2 reading comprehension. Studies in Second Language Acquisition, 8 201-212.

PAGE 166

151 Lomicka, L. (1998). “To gloss or not to gloss”: An investigation of reading comprehension online. Language Learning and Technology, 1 (2), 41-50. Long, M. (1991). Focus on form: a design f eature in language te aching methodology. In de Bot, K., Ginsberg, R. and Kramsch, C.(Eds), Foreign Language research in Cross-cultural Perspective Amsterdam: John Benjamins, 39-52. Long, M. H. (1996). The role of the lingui stic environment in second language acquisition. In Ritchie, W. C ., & Bahtia, T. K. (eds.), Handbook of second language acquisition (pp. 413-68). New York: Academic Press. Long, M. H., & Robinson, P. (1998). Focus on form: Theory, research and practice. In Doughty, C. J., & Williams, J. (eds.), Focus on form in second language acquisition (pp. 15-41). Cambridge: Cambridge University Press. Lyman-Hayer, M., Davis, J. N., Burnett, J ., & Chennault, R. (1993). “Une Vie de Boy: Interactive reading in French”. Proceedings of the CALICO 1993 Annual Symposium, 93-97. Mayer, R. E. (1997). Multimedia learning: are we asking the right questions? Educational psychologist, 32 10-19. Mayer, R. E. (2001). Multimedia Learning. New York: Cambridge University Press. Mayer, R. E. (2003). Elements of a science of E-learning. Journal of Educational Computing Research, 29(3), 297-313.

PAGE 167

152 Mayer, R. E. & Moreno, R. (1998). A split attention effect in multimedia learning: Evidence for dual processing systems in working memory. Journal of Educational Psychology, 90 312-320. Melka, F. (1997). Receptive vs. productive aspects of vocabulary. In Schimitt and McCarthy (eds), Vocabulary: Description, Acquisition and Pedagogy (pp. 84-102). Cambridge University Press. Merriam-Webster Online Dictionary. Available online: http://www.m-w.com/ Mousavi, S., Low R. & Sweller, J. (1995) Reducing cognitive load by mixing auditory and visual presentation modes. Journal of Educational Psychology, 87 319-334. Mundahl, J. (1993). Tales of Courage, Tales of Dreams: A Multicultural Reader Addison-Wesley Publishing, Inc, New York. Nagata, N. (1999). The effectiveness of computer-assisted interactive glosses. Foreign Language Annals, 32 (4), 469-479. Nagy, W., Herman, P., & Andreson, R. (1985). Learning words from context. Reading Research Quarterly, 20 233-253. Nation, P. (1990). Teaching and learning vocabulary New York: Newbury House. Nation, P. (1993) Vocabulary size, growth and us e. In R. Schreuder and B. Weltens (eds) The Bilingual Lexicon (pp. 115-134). John Benjamins, Amsterdam/Philadelphia.

PAGE 168

153 Nation, P. (2001). Learning vocabulary in another language Cambridge University Press. Omaggio, A. C. (1979). “Pictures and second la nguage comprehension: Do they help?” Foreign Language Annals, 12 (2), 107-116. Oxford, R., & Crookall, D. (1990). Vocabulary learning: A critical analysis of techniques. TESL Canadian Journal, 7 (2), 9-30. Paribakht, T. S., & Wesche, M. (1993). Th e relationship between reading comprehension and second language development in a comprehension-based ESL program. TESL Canadian Journal, 11 (1), 9-29. Paribakht, T. S., & Wesche, M. (1997). Vo cabulary enhancement activities and reading for meaning in a second language vocabular y acquisition. In Coady & T. Huckin (Eds), Second Language Vocabulary Acquisition (pp. 174-200). NY: Cambridge University Press. Paribakht, T. S., & Wesche, M. (1999). Reading and "inciden tal" L2 vocabulary acquisition: An introspective study of lexical inferencing. Studies in Second Language Acquisition, 21 (2), 195-224. Parry, K. (1991). Building a vocabulary through academic reading. TESOL Quarterly 629-653.

PAGE 169

154 Parry, K. (1993). Too many words: Learning the vocabulary of an academic subject. In T. Huckin, M. Haynes & J. Coady (eds.), Second Language reading and vocabulary learning (pp. 109-129_. Norwood, N.J.: Ablex. Parry, K. (1997). Vocabulary and comprehension: Two portrait. In J. Coady & T. Huckin (Eds.), Second language vocabulary acqui sition: A rationale for pedagogy (pp. 5568). Cambridge University Press. Paivio, A. (1971). Imagery and verbal processes New York, NY: Holt, Rinehart & Wilston. Paivio, A. (1986). Mental Representations New York: Oxford University Press. Paivio, A. (1990). Mental representation: a dual-coding approach Oxford, UK: Oxford University Press. Penney, C. G. (1989). Modality effect and th e structure of short-term verbal memory. Memory & Cognition, 17 198-422. Plass, J. L., Chun, D. M., Mayer, R. E., & Leutner, D. (1998). Supporting visual and verbal learning preferences in a second language multimedia learning environment. Journal of Educational Psychology, 90 25-36. Pulido, D. (2004). The relationship between text comprehension and second language incidental vocabulary acquisition: A matter of topic familiarity? Language Learning, 54 (3), 469-523.

PAGE 170

155 Read, J. (2000). Vocabulary Assessment. Cambridge University Press. Read, J. (2004). Research in teaching vocabulary. Annual Review of Applied Linguistics, 24 (1), 146-161. Richard, J. C. (1976). The ro le of vocabulary teaching. TESOL Quarterly, 19 77-89. Riley, G. L. & Lee, J. F. (1996). A comp arison of recall and summary protocols as measures of second langua ge reading comprehension. Language Testing, 13 (2), 173189. Roby, W. B. (1999). "WHAT 'S IN A GLOSS?". Language Learning & Technology, 2(2), 94-101. Rott, S., Williams, J., & Cameron, R. (2002). The effect of multiple-choice L1 glosses and input-output cycles on le xical acquisition and retention. Language Teaching Research, 6 (3), 183-222. Sadoski, M., & Paivio, A. (2001). Imagery and Text: A dual coding theory of reading and writing. Lawrence Erlbaum Associates, Publ ishers: Mahwah, New Jersey; London. Savignon, S. (1983). Communicative competence: theory and classroom practice Reading, MA: Addition-Wesley. Schmidt, R. (1995). Conscious and foreign la nguage learning: A tuto rial on the role of attention and awareness. In R. Schmidt (Ed.), Attention and Awareness in Foreign

PAGE 171

156 Language Teaching and Learning (Technica l Report No.9) (pp. 1-64). Honolulu: University of Hawai’i at Manoa. Schmidt, R. (1994). Deconstructing consciousne ss in search of useful definitions for applied linguistics. AILA Review, 11 11-26. Schmidt, R. (1990). The role of cons ciousness in second language learning. Applied Linguistics, 11 (2), 129-158. Schmitt, N. (2000). Vocabulary in language teaching. Cambridge University Press. Sherwood, R. D., Kinzer, C. K., Hasselbring, T. S., & Bransford, J. D. (1987). Macrocontexts for learning: In itial findings and issues. Applied Cognitive Psychology, 1 (2), 93-108. Shu, H., Anderson, R. C., & Zhang, H. (1995). Incidental learning of word meanings while reading: A Chinese and American cross-cultural study. Reading Research Quarterly, 30 (1), 76-95. Stanovich, K.E. (1986). Matthew Effects in Re ading: Some consequences of individual Differences in the acquisition of literacy. Reading Research Quarterly, 21 360407 Stanovich, K. E. (2000). Progress in understanding reading : Scientific foundations and new frontiers. New York: Guilford Press. Sternberg, R.J. (1987). Most vocabulary is le arned from context. In M.G. McKeown & M.E. Curtis (eds.), The nature of vocabulary acquisition (pp89-106). Hillsdale, N.J.: Erlbaum.

PAGE 172

157 Svenconis, D. J., & Kerst, S. (1995). I nvestigating the teaching of second-language vocabulary through semantic mappi ng in a hypertext environment. CALICO Journal, 12 (2/3), 33-57. Stewart, R. A., & Cross, T. L. (1991). Th e effect of marginal glosses on reading comprehension and retention. Journal of reading, 35 (1), 4-12. Vegas 4.0 [Computer Software]. (2003). Sonic Foundry Inc. Watanabe, Y. (1997). Input, in take, and retention: Effects of increased processing on incidental learning of fo reign language vocabulary. SSLA, 19 287-307. Widdowson, H. G. (1978 ). Teaching language as communication. Oxford, UK: Oxford University Press. Wittrock, M. C. (1974). Learning as a generative activity. Educational Psychologist, 11 87-95. Wittrock, M. C. (1989). Generative processes of comprehension. Educational Psychologist, 24 345-376. Wolf, D. (1993). A comparison of assessment tasks used to measure FL reading comprehension. Modern Language Journal, 77 473-489. Yeh, Y., & Wang, C. (2003). Effects of multim edia vocabulary annotations and learning styles on vocabulary learning. CALICO Journal, 21 (1), 131-144.

PAGE 173

158 Yoshii, M. (2000). Second language incidental vocabulary retention: The effect of text and picture annotation types. Unpublished Doctoral Dissertation, University of South Florida. Yoshii, M. & Flaitz, J. (2002). Second langua ge incidental vocabul ary retention: The effect of text and pi cture annotation types. CALICO Journal, 20 (1), 33-58. Zimmerman, C. B. (1997). Hist orical trends in second la nguage vocabulary instruction. In J. Coady & T. Huckin (Eds.), Second language vocabulary acquisition: A rationale for pedagogy (pp. 5-19). Cambridge University Press.

PAGE 174

159 Appendices

PAGE 175

160 Appendix A: Questionnaire Personal Information: Identification code (ID): __________ Computer Number: ___________ Gender: MALE______ FEMALE______ Age Range: 18-25____ 26-35____ 36-45____ What is your home country? ____________ What is your native language? _____________ What other language(s) do you speak? (List all) ________________ How long have you been st udying in America? _________________ Please check one for each of the following questions: 1. Do you like to use computers for your study? Very much Yes It is ok Not very much Not at all 2. Is it easy for you to use computers? Very much Yes It is ok Not very much Not at all 3. Do you like to read on computers? Very much Yes It is ok Not very much Not at all 4. Are you good at using computers? Very much Yes It is ok Not very much Not at all 5. Do you like to look new word s up in a dictionary? Very much Yes It is ok Not very much Not at all 6. How often do you use computers in a week? Almost every day 3-6 times 1-3 times never 7. How often do you use Internet in a week? Almost every day 3-6 times 1-3 times never 8. How often do you do reading in English online in a week? Almost every day 3-6 times 1-3 times never 9. How often do you look it up in a dictionary when meeting a new word in online reading? All the time Often sometimes never 10. How often do you use emails in a week? Almost every day 3-6 times 1-3 times never

PAGE 176

161 Appendix B: Reading Text The European Settlers of Australia During the early 1800s, European people began to settle in Australia. They lived along the coast. Most of them thought th at the inland of Australia was a prairie In fact, a mountain range separates the coast from the in land. The mountain range is called the Great Dividing Range. To the new settlers, it seemed like a labyrinth of unexplored valleys and mountains. Burke and Wills successf ully crossed the Great Dividing Range in 1830. After that, European settlers started to sett le beyond the mountains. Most of them were shepherds They raised animals for meat. They wo re old clothes, ate simple foods, and lived in small houses. Sometimes the native pe ople attacked them. Ot her times, animals such as emus attacked them with sharp beaks They didn’t have armor to protect themselves. Banjo Patterson was one of the most famous early settlers. He was a songwriter. He wrote Australia’s most famous song: Wa ltzing Matilda. The song is about a young traveler. With nothing to do, the young man sat under a tree, put his satchel on the ground and played his accordion Then he saw a herd of cows and decided to steal one to eat. At that time, life was hard and people hate d stealing. The owner of the cow reported the loss to the police. A local pol iceman caught the young traveler. The policeman took away the young man’s saddle and horse, and put him in prison. He used his cornet to call for a meeting at the center of the town. After the meeting, the young man was made to stand on a barrel The policeman put a noose around the young man’s neck and killed him. Most Australian songs are not that sad, but Wa ltzing Matilda tells the stories of the early colonists in the first part of the 1800s in Austra lia. By the 1850s, however, life for these early settlers became better. Gold was discovere d in the colony of Victoria. As a result, thousands of new settlers came and made Au stralia their home. Many of them became rich through the gold rush. In many stories of that time, a damsel had nothing better to do than to be beautiful. She just drank from a golden goblet looked pretty for handsome young men and lived in a big mansion More people came to Victoria for gold. The early settlers were afraid of the newcomers. They organized soldiers to keep order. At times, the newcomers walked on the streets and waved placards as a sign of protest. On the placards, they drew pictures of phantoms Sometimes they threw twigs at the soldiers. However, the soldiers, each armed with a bayonet easily ended the protests.

PAGE 177

162 Appendix C: Target Words This is a list of the 20 target words. The de finitions are used in the text annotation and audio annotation. 1. prairie: a larger area of treeless grassland. 2. labyrinth: a confusing set of connecting paths. 3. shepherd: a person who takes care of sheep. 4. beak: the hard pointed part of a bird’s mouth. 5. armor: metal clothing for protection in battles. 6. satchel: a small bag with a shoulder strap. 7. accordion: a box-shaped musical instrument. 8. herd: a large number of animals living together. 9. saddle: a leather seat fo r the rider of a horse. 10. cornet: a musical instrument that you play by blowing into it. 11. barrel: a large container w ith flat top and bottom. 12. noose: a rope tied in a circle. 13. colonist: one who settle s in a new country. 14. damsel: a young and unmarried woman. 15. goblet: a drinking glass w ith a base and a stem. 16. mansion: a large and impressive house. 17. placard: a sign people hold in a demonstration. 18. phantom: a ghost. 19. twig: a small branch of a tree. 20. bayonet: a long sharp blade fi xed at the end of a gun.

PAGE 178

163 Appendix D: Pictures for Target Words prairie labyrinth shepherd beak armor satchel

PAGE 179

164 Appendix D: Pictures for Target Words (Continued) accordion Herd Saddle Cornet barrel Colonist

PAGE 180

165 Appendix D: Pictures for Target Words (Continued) noose Damsel goblet Mansion placard Phantom

PAGE 181

166 Appendix D: Pictures for Target Words (Continued) twig Bayonet

PAGE 182

167 Appendix E: Screenshot of Reading Passage with Text-picture Annotation

PAGE 183

168 Appendix F: Screenshot of Reading Passage with Audio-picture Annotation

PAGE 184

169 Appendix G: Vocabulary Knowledge Scale (VKS) Directions: For each word, th ere are three choices of how much you know about the word, please circle the one that fits you most. If you choose III, please written down the meaning of the word. 1. shepherd Categories Description I I don’t remember having seeing this word before II I have seen this word before, but I don’t know what it means III I have seen this word before, and I think it means _____________________________________________________________ 2. placard Categories Description I I don’t remember having seeing this word before II I have seen this word before, but I don’t know what it means III I have seen this word before, and I think it means _______________________________________________________________ 3. prairie Categories Description I I don’t remember having seeing this word before II I have seen this word before, but I don’t know what it means III I have seen this word before, and I think it means _______________________________________________________________ 4. phantom Categories Description I I don’t remember having seeing this word before II I have seen this word before, but I don’t know what it means III I have seen this word before, and I think it means _______________________________________________________________ 5 noose Categories Description I I don’t remember having seeing this word before II I have seen this word before, but I don’t know what it means III I have seen this word before, and I think it means _______________________________________________________________

PAGE 185

170 Appendix G: Vocabulary Knowle dge Scale (VKS) (Continued) 6. herd Categories Description I I don’t remember having seeing this word before II I have seen this word before, but I don’t know what it means III I have seen this word before, and I think it means___________________________________________________________ 7. twig Categories Description I I don’t remember having seeing this word before II I have seen this word before, but I don’t know what it means III I have seen this word before, and I think it means ________________________________________________________________ 8. damsel Categories Description I I don’t remember having seeing this word before II I have seen this word before, but I don’t know what it means III I have seen this word before, and I think it means ________________________________________________________________ 9. colonist Categories Description I I don’t remember having seeing this word before II I have seen this word before, but I don’t know what it means III I have seen this word before, and I think it means ________________________________________________________________ 10. saddle Categories Description I I don’t remember having seeing this word before II I have seen this word before, but I don’t know what it means III I have seen this word before, and I think it means ________________________________________________________________

PAGE 186

171 Appendix G: Vocabulary Knowle dge Scale (VKS) (Continued) 11. labyrinth Categories Description I I don’t remember having seeing this word before II I have seen this word before, but I don’t know what it means III I have seen this word before, and I think it means ________________________________________________________________ 12. cornet Categories Description I I don’t remember having seeing this word before II I have seen this word before, but I don’t know what it means III I have seen this word before, and I think it means ________________________________________________________________ 13. armor Categories Description I I don’t remember having seeing this word before II I have seen this word before, but I don’t know what it means III I have seen this word before, and I think it means ________________________________________________________________ 14. bayonet Categories Description I I don’t remember having seeing this word before II I have seen this word before, but I don’t know what it means III I have seen this word before, and I think it means ________________________________________________________________ 15. beak Categories Description I I don’t remember having seeing this word before II I have seen this word before, but I don’t know what it means III I have seen this word before, and I think it means ________________________________________________________________

PAGE 187

172 Appendix G: Vocabulary Knowle dge Scale (VKS) (Continued) 16. barrel Categories Description I I don’t remember having seeing this word before II I have seen this word before, but I don’t know what it means III I have seen this word before, and I think it means ________________________________________________________________ 17. accordion Categories Description I I don’t remember having seeing this word before II I have seen this word before, but I don’t know what it means III I have seen this word before, and I think it means ________________________________________________________________ 18 satchel Categories Description I I don’t remember having seeing this word before II I have seen this word before, but I don’t know what it means III I have seen this word before, and I think it means _______________________________________________________________ 19. goblet Categories Description I I don’t remember having seeing this word before II I have seen this word before, but I don’t know what it means III I have seen this word before, and I think it means _______________________________________________________________ 20. mansion Categories Description I I don’t remember having seeing this word before II I have seen this word before, but I don’t know what it means III I have seen this word before, and I think it means ______________________________________________________________

PAGE 188

173 Appendix H: Word Recognition Test (WRT) For each underlined word, please circle the correct meaning from the four choices. 1. Most of these set tlers were shepherd s. a. a person who travels in the forest b. a person who takes care of sheep c. a person who digs gold for a living d. a person who buys and sells sheep 2. To the new settlers, it seemed like a labyrinth. a. a group of scattered houses b. a set of connecting roads on a map c. a confusing set of connecting paths d. a group of animals living together 3. The policeman took away the young man’s saddle a. a small bag b. a leather seat c. a handgun d. a musical instrument 4. Most of them thought that the in land of Australia was a prairie a. a large area of wetland b. a large area of grassland c. a large area of desert d. a large area of forest 5. They didn’t have armor to protect themselves. a. wooden weapons b. metal tools c. wooden housing d. metal clothing 6. The young traveler put his satchel on the ground. a. a small bag with a shoulder strap b. a wine glass with a base and a handle c. a leather seat used to ride horses d. a long sharp knife used by soldiers 7. In many stories of that time, a damsel does nothing all the time. a. an unmarried girl b. a housewife c. a young soldier d. a rich man

PAGE 189

174 Appendix H: Word Recogniti on Test (WRT) (Continued) 8. They drew a picture of a phantom on the wall. a. a horse b. a ghost c. a person d. a sheep 9. The policeman put a noose around the traveler’s neck. a. a long necktie b. a long metal chain c. a rope tied in a circle d. a thin piece of cloth 10. The newcomers threw twig s at the soldiers. a. a tree branch b. a wood chip c. a small stone d. a smelly plant 11. She drank from a golden goblet a. a drinking glass b. a water bowl c. a table spoon d. a metal plate 12. The policeman used his cornet to call for a meeting. a. a box-shaped musical instrument that you play with hands b. a whistle used by policemen to get people’s attention c. a musical instrument that you play by blowing into it d. a big bell used to inform people of the time in the past 13. The traveler saw a herd of cows. a. a number of people living together b. a number of animals living together c. a number of travelers on the desert d. a number of policemen at the station 14. The colonist arrived in Australia in 1800s. a. a person who writes songs for travelers b. a person who takes care of sheep c. a person who settles in a new country d. a person who fights for his country

PAGE 190

175 Appendix H: Word Recogniti on Test (WRT) (Continued) 15. The young girl lived in a mansion a. a large house b. an old castle c. a tall tower d. a small cottage 16. Each soldier is armed with a bayonet a. a handgun used by policemen to protect people b. a long and sharp blade fixed at the end of a gun c. a whistle used by policemen to get attention d. a knife used by soldiers to kill animals 17. A gold digger waved a placard at the soldiers. a. a flag people wave in public to get attention b. a sign people hold in pu blic in a demonstration c. a piece of cloth people use to cover their heads d. a cotton scarf people we ar in cold weather 18. The young man was made to stand on a barrel a. a large tree trunk for people to stand on it b. a wooden bench for people to sit on it c. a large container with two handles d. a large container with flat top and bottom 19. Emus attacked the settlers with sharp beak s. a. the long and sharp knife used by hunters b. the hard pointed part of a bird’s mouth c. the sharp blade at the end of a gun d. the pointed horn of a large animal 20. The young traveler played his accordion under the tree. a. a whistle used to get attention b. a big bell used to tell people time c. a box-shaped musical instrument d. a musical instrument like a trumpet

PAGE 191

176 Appendix I: Multiple-Choice Reading Comprehension Questions Based on the reading passage, please circle the best answer for each question: 1. When the European settlers arrived in Australia in the 1800s, they had a misunderstanding about the inland of Aust ralia. What is the misunderstanding? A. They thought the inland of Au stralia was a large forest. B. They thought the inland of Aust ralia was a large prairie. C. They thought the inland of Au stralia was a large mountain. D. They thought the inland of Au stralia was a large lake. 2. Which of the following is mentioned as one of the dangerous things the early settlers had to face? A. bad weather of the inland B. thieves who steal cows C. bayonets of the newcomers D. animals with sharp beaks 3. The early settlers did NOT have which of the following? A. old clothes B. metal armor C. small houses D. animal meat 4. The main character in the s ong “Waltzing Matilda” is a ______. A. shepherd B. soldier C. songwriter D. traveler 5. According to the passage, the charac ter in the song did NOT have ______. A. a small bag B. an accordion C. a horse and saddle D. a sharp blade 6. What did the policeman use to call for a meeting at the center of the town? A. an accordion B. a cornet C. a whistle D. a goblet

PAGE 192

177 Appendix I: Multiple-choice Reading Comprehension Questions (Continued) 7. How did the main character in the song die? A. He was killed by a stone. B. He was killed by a gun. C. He was killed by a noose. D. He was killed by a bayonet. 8. In many stories about the European settle rs after the1850s, a character is often used to describe the life at th at time. Who is that character? A. a young woman B. a settler’s wife C. a young soldier D. a young shepherd 9. What did the newcomers do to show their dislike of the soldiers? A. They moved away from the center of the town. B. They walked on the streets and waved signs. C. They threw small stones at the soldiers. D. They used bayonets to fight the soldiers. 10. What pictures did the newcomers draw on the placards? A. pictures of prairies B. pictures of satchels C. pictures of ghosts D. pictures of animals

PAGE 193

178 Appendix J: Written Recall Directions: Please use your fi rst language to write down as much as you know about the passage you just read.

PAGE 194

About the Author Zhaohui Chen obtained his Ba chelor’s degree in English Education from Suzhou Railway Teachers College in 1994 and Master’s degree in English Language & Literatures from Suzhou University in 1997. He worked as an English lecturer in the School of Foreign Language at Suzhou University from 1997 to 2001. In August of 2001, he started his doctoral prog ram in Second Language Acquisition & Instructional Technology at the University of South Fl orida. While in the PhD program, Mr. Chen taught ESOL classes in both face-to-face and distance learning formats, worked as the treasurer of SLAQ (Student Organization for Second Language Acquisition & Instructional Technol ogy), and presented at conferences.