USF Libraries
USF Digital Collections

Effects of deadline contingencies in a web-based course on html

MISSING IMAGE

Material Information

Title:
Effects of deadline contingencies in a web-based course on html
Physical Description:
Book
Language:
English
Creator:
Majchrzak, Tina Laree
Publisher:
University of South Florida
Place of Publication:
Tampa, Fla.
Publication Date:

Subjects

Subjects / Keywords:
Procrastination   ( lcsh )
procrastination
content on demand
retention test scores
personalized system of instruction
massed versus distributed practice
Dissertations, Academic -- Instructional Technology -- Doctoral -- USF   ( lcsh )
Genre:
government publication (state, provincial, terriorial, dependent)   ( marcgt )
bibliography   ( marcgt )
theses   ( marcgt )
non-fiction   ( marcgt )

Notes

Summary:
ABSTRACT: Current learner-centered trends, such as supplying students with content on demand (CoD), coupled with research findings that indicate distributed practice is superior to massed practice in terms of increased memory function and that the Personalized System of Instruction (PSI) is superior to traditional instruction in terms of academic achievement, content retention, and student satisfaction, prompted an investigation merging these two lines of research. Although PSI is more feasible today based on advances in technology and students prefer its self-paced component, they often procrastinate. In fact, this problem is resurfacing in distance education courses and is reflected in low completion rates as well as in the number of nonstarters. Numerous researchers have used deadline contingencies to reduce procrastination without adversely affecting student achievement and satisfaction, but few have considered the benefit of enhanced memory. It was hypothesized that, by providing students with CoD, a lesser form of self-pacing, and by using contingencies to regulate the pace of assignment submissions, procrastination would be reduced and content retention subsequently increased without detriment to immediate achievement and student satisfaction. To quantify differences in procrastination level, a comprehensive, sensitive, and reliable measure of procrastination, called the rate of relative digression from a target response (RDTR), was proposed. Undergraduate, preservice teachers in an instructional technology course were randomly assigned to one of three treatments. All groups were given the same deadlines. For one treatment, the deadlines were recommended (R) with one absolute deadline at the end of the treatment interval. For another they were conditional (C) with opportunities to earn bonus and penalty points for early and late work. For a third, they were all absolute (A) with no assignment accepted for credit after its due date. Although many problems experienced by students in A made findings for this group inconclusive, analysis of differences between students in R and C indicated that C was superior in reducing procrastination and enhancing memory function without detriment to immediate achievement, pacing preference, and course satisfaction. Although more research is needed to replicate, extend, and clarify findings, these results support using conditional deadlines for assignments when learners are supplied CoD.
Thesis:
Thesis (Ph.D.)--University of South Florida, 2001.
Bibliography:
Includes bibliographical references.
System Details:
System requirements: World Wide Web browser and PDF reader.
System Details:
Mode of access: World Wide Web.
Statement of Responsibility:
by Tina Laree Majchrzak.
General Note:
Title from PDF of title page.
General Note:
Document formatted into pages;

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
aleph - 001413330
notis - AJJ0746
usfldc doi - E14-SFE0000024
usfldc handle - e14.24
System ID:
SFS0024715:00001


This item is only available as the following downloads:


Full Text
xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam Ka
controlfield tag 001 001413330
003 fts
006 m||||e|||d||||||||
007 cr mnu|||uuuuu
008 031010s2001 flu sbm s000|0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0000024
9 035
AJJ0746
b SE
SFE0000024
040
FHM
c FHM
049
FHME
090
LB1028.3.Z9 2001
1 100
Majchrzak, Tina Laree.
0 245
Effects of deadline contingencies in a web-based course on html
h [electronic resource] /
by Tina Laree Majchrzak.
260
[Tampa, Fla.] :
University of South Florida,
2001.
502
Thesis (Ph.D.)--University of South Florida, 2001.
504
Includes bibliographical references.
516
Text (Electronic thesis) in PDF format.
538
System requirements: World Wide Web browser and PDF reader.
Mode of access: World Wide Web.
500
Title from PDF of title page.
Document formatted into pages;
520
ABSTRACT: Current learner-centered trends, such as supplying students with content on demand (CoD), coupled with research findings that indicate distributed practice is superior to massed practice in terms of increased memory function and that the Personalized System of Instruction (PSI) is superior to traditional instruction in terms of academic achievement, content retention, and student satisfaction, prompted an investigation merging these two lines of research. Although PSI is more feasible today based on advances in technology and students prefer its self-paced component, they often procrastinate. In fact, this problem is resurfacing in distance education courses and is reflected in low completion rates as well as in the number of nonstarters. Numerous researchers have used deadline contingencies to reduce procrastination without adversely affecting student achievement and satisfaction, but few have considered the benefit of enhanced memory. It was hypothesized that, by providing students with CoD, a lesser form of self-pacing, and by using contingencies to regulate the pace of assignment submissions, procrastination would be reduced and content retention subsequently increased without detriment to immediate achievement and student satisfaction. To quantify differences in procrastination level, a comprehensive, sensitive, and reliable measure of procrastination, called the rate of relative digression from a target response (RDTR), was proposed. Undergraduate, preservice teachers in an instructional technology course were randomly assigned to one of three treatments. All groups were given the same deadlines. For one treatment, the deadlines were recommended (R) with one absolute deadline at the end of the treatment interval. For another they were conditional (C) with opportunities to earn bonus and penalty points for early and late work. For a third, they were all absolute (A) with no assignment accepted for credit after its due date. Although many problems experienced by students in A made findings for this group inconclusive, analysis of differences between students in R and C indicated that C was superior in reducing procrastination and enhancing memory function without detriment to immediate achievement, pacing preference, and course satisfaction. Although more research is needed to replicate, extend, and clarify findings, these results support using conditional deadlines for assignments when learners are supplied CoD.
590
Adviser: White, James
653
procrastination.
content on demand.
retention test scores.
personalized system of instruction.
massed versus distributed practice.
690
Dissertations, Academic
z USF
x Instructional Technology
Doctoral.
650
Procrastination.
773
t USF Electronic Theses and Dissertations.
4 856
u http://digital.lib.usf.edu/?e14.24



PAGE 1

OfceofGraduateStudiesUniversityofSouthFloridaTampa,Florida CERTIFICATEOFAPPROVAL ThisistocertifythatthedissertationofTINAL.MAJCHRZAKinthegraduatedegreeprogramofCurriculumandInstructionwasapprovedonOctober26,2001fortheDoctorofPhilosophydegree.ExaminingCommittee: MajorProfessor:JamesWhite,Ph.D. Member:FrankBreit,Ph.D. Member:JohnFerron,Ph.D. Member:DeweyRundus,Ph.D.

PAGE 2

EFFECTSOFDEADLINECONTINGENCIESINAWEB-BASEDCOURSEONHTMLbyTINAL.MAJCHRZAKAdissertationsubmittedinpartialfulllmentoftherequirementsforthedegreeofDoctorofPhilosophyDepartmentofSecondaryEducationCollegeofEducationUniversityofSouthFloridaDecember2001MajorProfessor:JamesWhite,Ph.D.

PAGE 3

cCopyrightbyTinaL.Majchrzak2001Allrightsreserved

PAGE 4

DedicationDandelionseedsexploreworldonGentleBreezeandeaglesourshigh.Butterydancesinmagicaleldoflightandnewfoundwonder.ThisworkisdedicatedtoDaniel,whotakesmeplacesandliftsmehigherthanIeverdreamedpossible,andtoSamantha,whomakesmyheartdance.

PAGE 5

AcknowledgmentsTherearemanywhomadethisworkpossibleandhelpedtoimproveitsquality.Iwishtothankthemallsincerely.Myhusband,Daniel,whonevertiredofsharinghisenthusiasm,support,andinsight,andmydaughter,Samantha,mademanysacricesandwereeverpatientandunderstanding.Mymother,GloriaStrother,savedmylifein1986andshowedmehowfaryoucouldgowithadreamtoguideyou,couragetobelieveinyourself,andthedrivetoworkhard.She,alongwithmyotherparents,RoseMarieandAmbroseMajchrzakandDorseyStrother,sup-portedmeunconditionallyandspentmanyqualityhourscaringforSamantha.Mygrandfather,Dr.WilliamC.Tremmel,setthebarwithhisownachievementsandmygrandmother,Opal“Mike”Tremmel,wasoneofmygreatestcheerleaders.Imissherdearlyandwillneverforgethowspecialshemademefeel,likeIcouldac-complishanything.Myfather,JamesTremmel,grandfather,WilberJohnson,anddearfriend,MichaelPokorny,alsohavebeensteadfastintheirsupport.Dr.JamesWhiteprovidedinvaluableguidanceonstudymethodologyandwasinstrumentalinhelpingmebalanceperfectionismwithpracticalconstraints.Hegavemethedirection,encouragement,andfreedomIneededtogrowpersonallyandprofessionally.Dr.JohnFerron,anincredibleassetwithhisdeepandex-pansiveunderstandingofalternativeanalyticmethods,providedprompt,copious,andthoughtfulfeedbackonthedirectionofdataanalysis.Bothofthesementorsalsoshowedconsiderablepatienceandkindness,makingitpossibleforSaman-thatoremainbymysideeverystepoftheway.Dr.Bostow,Dr.Rundus,andDr.Breitdevotedvaluabletimetothecarefulreviewofmyworkandofferedtheirkeeninsight.BarbaraMoore,RuthBurkett,WendyMiller,DavidGuilhaus,KristenLutz,MatthewThomas,andKristianGilberthelpedconducttheexperiment,col-lectthedata,improvethecourseware,andfosteradeeperunderstandingofstudynuances.Finally,Iamindebtedtothestudentswhoparticipatedinthepilotandmainstudies.

PAGE 6

TableofContentsListofTables................................................................ivListofFigures...............................................................viAbstract.....................................................................ixChapter1.Introduction......................................................11.1Purpose...............................................................51.2ResearchHypotheses................................................61.3Delimitations..........................................................71.4Limitations.............................................................81.5AcronymsandDenitions............................................9Chapter2.LiteratureReview................................................112.1SelfPacing............................................................112.1.1ProgrammedLearning.....................................112.1.2Keller'sPersonalizedSystemofInstruction..............122.1.3Competency-BasedVocationalTraining..................142.1.4CriticalFactorsforSuccessfulSelf-PacedTraining......152.1.5DisadvantagesoftheSelf-PacedApproach..............162.1.6CurbingProcrastinationwithDeadlineContingencies...162.1.7MitigatingDisadvantageswithCAIandVideo............182.2Massed,Distributed,andSpacedPractice.........................192.3DistanceEducationViaAsynchronousLearningNetworks.......232.4OtherTeachingParadigms...........................................262.4.1Tutoring......................................................262.4.2GroupLectureandDiscussion............................282.4.3ActiveLearning.............................................292.4.4CollaborativeandCooperativeLearning..................302.5Summary..............................................................30Chapter3.Method..........................................................323.1TimeTable............................................................333.2Sample................................................................343.2.1AssignmenttoTreatments.................................343.2.2MissingData................................................37i

PAGE 7

3.2.3EffectSizeandPower......................................433.3Treatments............................................................443.4Instruments...........................................................443.4.1StudentAchievement.......................................523.4.1.1Administration....................................533.4.1.2Development......................................543.4.1.3Validity.............................................553.4.1.4Reliability..........................................603.4.2StudentPacingPreference................................633.4.2.1Validity.............................................643.4.2.2Reliability..........................................653.4.3LevelofProcrastination....................................653.4.3.1Validity.............................................683.4.3.2Reliability..........................................683.4.4AssignmentScores.........................................69Chapter4.Results..........................................................794.1EstimationofRetentionTestReliabilityandCurvingofGrades...794.2AnalysisofPacingPreferenceData.................................804.3AnalysisofProcrastinationData....................................814.4AnalysisofAchievementData.......................................864.5AnalysisofFreeFormatResponses................................93Chapter5.Discussion.......................................................985.1Findings...............................................................995.1.1ProcrastinationLevel.......................................995.1.2PacingPreference..........................................1005.1.3AchievementData..........................................1015.2FutureResearchandPractices.....................................1045.2.1ReplicatingStudy...........................................1045.2.2ImprovingCourseware.....................................1075.2.3ResearchingRelatedIssues...............................1085.2.4IncorporatingLessonsLearned...........................1105.3Conclusions...........................................................112References..................................................................116Appendices.................................................................122AppendixA.PilotStudyConductedDuringFall1999..................123AppendixB.InstructionsGiventoPilotStudyParticipants.............127AppendixC.CoursewareDescription....................................131AppendixD.CoursewareObjectives.....................................153AppendixE.AnnotatedItemsfromallAchievementMeasures........155AppendixF.HTMLPretest...............................................179ii

PAGE 8

AppendixG.HTMLPosttest..............................................183AppendixH.HTMLRetentionTest.......................................197AppendixI.LettertoHTMLExpert......................................204AppendixJ.ExpertEvaluationFormforAchievementMeasures.....208AppendixK.Self-ReportMeasureofPacingPreference...............219AppendixL.LettertoInstructionalParadigmsExpert..................222AppendixM.ExpertEvaluationFormforPacingPreferenceMeasure225AppendixN.ExpertEvaluationFormforAssignments.................228AppendixO.Self-ReportMeasureofStudyGroupPatterns...........235AppendixP.Self-ReportofAssistanceReceived.......................236AppendixQ.FreeFormResponsesfromStudents.....................238AppendixR.SurveyonPlatformandBrowserUsage..................247AppendixS.CCodetoAnalyzeDataViaRandomization..............248AppendixT.RawData....................................................258AppendixU.ContentsofCD-ROM.......................................285AbouttheAuthor......................................................EndPageiii

PAGE 9

ListofTablesTable1.CommonAcronymsUsed............................................9Table2.CommonTermsandPhrases........................................10Table3.GradesforStudentsMissingPosttestandRetentionTest.........38Table4.SampleSizesasDistributedAcrossTAs............................42Table5.SummaryofAlternativeObjectivesIdentiedbyExperts..........59Table6.CorrelationsBetweenAchievementandPerformanceScores.....61Table7.ReportedPriorExperiencebyTreatment............................62Table8.ObjectivesMeasuredbyRubricsandAchievementItems.........74Table9.DescriptiveStatisticsforPacingPreferenceData...................81Table10.DescriptiveStatisticsforProcrastinationLevelData................82Table11.DescriptiveUnivariateStatisticsforAchievementData.............92Table12.WhatStudentsReportedLikingBest................................96Table13.WhatStudentsReportedLikingLeast...............................97Table14.SummaryofPilotStudyPerformanceandAchievementData.....125Table15.RawDatafromPilotStudy............................................126Table16.DueDatesforTreatmentRofPilotStudy...........................129Table17.DueDatesforTreatmentAofPilotStudy...........................130Table18.GradingRubricforEssayQuestion46...............................174Table19.GradingRubricforEssayQuestion47...............................178Table20.FormUsedtoAnalyzeGradingRubricforEssayQuestion46....214Table21.FormUsedtoAnalyzeGradingRubricforEssayQuestion47....218iv

PAGE 10

Table22.StudentResponsesonAchievementMeasures....................258Table23.AssignmentRawData................................................265Table24.ProleandPreferenceRawData....................................271Table25.ScoresforIndividualsandStudyGroups............................278v

PAGE 11

ListofFiguresFigure1.PredictingPosttestScorefromRetentionTestScore...............40Figure2.PredictingRetentionTestScorefromPosttestScore...............40Figure3.AssignmentDueDatesforTreatmentR.............................45Figure4.DueDateDescriptionforTreatmentR...............................46Figure5.AssignmentDueDatesforTreatmentC.............................47Figure6.DueDateDescriptionforTreatmentC...............................48Figure7.AssignmentDueDatesforTreatmentA.............................49Figure8.DueDateDescriptionforTreatmentA...............................50Figure9.DeadlineInformationSuppliedtoAllStudents......................51Figure10.FormUsedtoLogRequestsforDeadlineExtensions..............69Figure11.FormUsedbyStudentstoAccessSubmissionSoftware..........71Figure12.FormUsedbyStudentstoSubmitAssignments....................72Figure13.ConrmationMessageforSuccessfulSubmission.................73Figure14.FormUsedbyTAstoAccessGradingSoftware....................74Figure15.ListofAssignmentstobeGradedforaGivenTA...................75Figure16.GradingRubricUsedbyTAsforAssignment1.....................76Figure17.Assignment1foraGivenStudent...................................76Figure18.GradeE-MailedtoStudent...........................................77Figure19.SolutionSourceCodeforAssignment1.............................78Figure20.ProcrastinationHistograms...........................................83Figure21.CumulativePercentageofAssignmentsSubmittedEachDay.....86vi

PAGE 12

Figure22.ProcrastinationLevelwithEstimatedMissingData.................87Figure23.PercentageofStudentsSubmittingAssignments...................88Figure24.ProcrastinationLevelwithNoMissingData.........................89Figure25.AchievementHistograms.............................................93Figure26.HistogramsforAchievementDifferenceScores.....................94Figure27.TimeandTreatmentInteractionforAchievementData.............95Figure28.ProcrastinationLevelsDuringPilotStudy...........................125Figure29.HomePageforVersion1.2ofCourseware..........................131Figure30.FirstPageofSectiononListsinHTMLTutorial.....................132Figure31.ActiveLearningOpportunity..........................................133Figure32.ImageDisplayedforaWidthof67...................................134Figure33.ImageDisplayedforaWidthof35...................................135Figure34.GradingRubricinListFormat........................................140Figure35.GradingRubricinGraphicalFormat.................................141Figure36.CoursewareNavigation...............................................142Figure37.OverviewofCoursewareProductGoal..............................143Figure38.InitialIndexPageofProduct..........................................144Figure39.InitialPersonalPageofProduct......................................145Figure40.InitialLinksPageofProduct..........................................147Figure41.IndexPageofProductwithClickableImage........................148Figure42.PersonalPageofProductwithNavigationalImages...............148Figure43.IndexPageofProductwithMouseOverProfessional..............149Figure44.PersonalPageofProductwithMouseOverProfessional..........149Figure45.ProfessionalPageofProduct.........................................150Figure46.InitialFramesforLinksPageofProduct.............................150Figure47.FramesforLinksPageofProductAfterClickingEducation........151vii

PAGE 13

Figure48.FramesforLinksPageofProductAfterClickingErgonomics......151Figure49.PersonalPageofProductwithForm.................................152Figure50.HTMLCodeRenderedbyInternetExplorer(Posttest).............170Figure51.SolutiontoEssayQuestion46.......................................173Figure52.HTMLCodeRenderedbyInternetExplorer(RetentionTest)......175Figure53.SolutiontoEssayQuestion47.......................................177Figure54.Self-ReportPriorExperienceItemonPretest.......................204viii

PAGE 14

EFFECTSOFDEADLINECONTINGENCIESINAWEB-BASEDCOURSEONHTMLbyTINAL.MAJCHRZAKAnAbstractofadissertationsubmittedinpartialfulllmentoftherequirementsforthedegreeofDoctorofPhilosophyDepartmentofSecondaryEducationCollegeofEducationUniversityofSouthFloridaDecember2001MajorProfessor:JamesWhite,Ph.D.ix

PAGE 15

Currentlearner-centeredtrends,suchassupplyingstudentswithcontentondemand(CoD),coupledwithresearchndingsthatindicatedistributedpracticeissuperiortomassedpracticeintermsofincreasedmemoryfunctionandthatthePersonalizedSystemofInstruction(PSI)issuperiortotraditionalinstructionintermsofacademicachievement,contentretention,andstudentsatisfaction,promptedaninvestigationmergingthesetwolinesofresearch.AlthoughPSIismorefeasibletodaybasedonadvancesintechnologyandstudentspreferitsself-pacedcomponent,theyoftenprocrastinate.Infact,thisproblemisresurfacingindistanceeducationcoursesandisreectedinlowcompletionratesaswellasinthenumberofnonstarters.Numerousresearchershaveuseddeadlinecontingenciestoreduceprocrastinationwithoutadverselyaffectingstudentachievementandsat-isfaction,butfewhaveconsideredthebenetofenhancedmemory.Itwashypoth-esizedthat,byprovidingstudentswithCoD,alesserformofself-pacing,andbyusingcontingenciestoregulatethepaceofassignmentsubmissions,procrastina-tionwouldbereducedandcontentretentionsubsequentlyincreasedwithoutdetri-menttoimmediateachievementandstudentsatisfaction.Toquantifydifferencesinprocrastinationlevel,acomprehensive,sensitive,andreliablemeasureofpro-crastination,calledtherateofrelativedigressionfromatargetresponse(RDTR),wasproposed.Undergraduate,preserviceteachersinaninstructionaltechnologycoursewererandomlyassignedtooneofthreetreatments.Allgroupsweregiventhesamedeadlines.Foronetreatment,thedeadlineswererecommended(R)withoneabsolutedeadlineattheendofthetreatmentinterval.Foranothertheywereconditional(C)withopportunitiestoearnbonusandpenaltypointsforearlyandlatework.Forathird,theywereallabsolute(A)withnoassignmentacceptedforcreditafteritsduedate.AlthoughmanyproblemsexperiencedbystudentsinAmadendingsforthisgroupinconclusive,analysisofdifferencesbetweenstu-dentsinRandCindicatedthatCwassuperiorinreducingprocrastinationandenhancingmemoryfunctionwithoutdetrimenttoimmediateachievement,pacingpreference,andcoursesatisfaction.Althoughmoreresearchisneededtorepli-cate,extend,andclarifyndings,theseresultssupportusingconditionaldeadlinesforassignmentswhenlearnersaresuppliedCoD.AbstractApproved: MajorProfessor:JamesWhite,Ph.D.AssociateProfessor,DepartmentofSecondaryEducationDateApproved: x

PAGE 16

Chapter1IntroductionEducationalsystemsappeartobemovingrapidlytowardamorelearner-centeredparadigminwhichteachersactasfacilitators,coaches,andtutorsandstudentsaccesselectroniccontentindividuallyorinsmall,cooperativegroups,havemoreinputregardingtopicsofexploration,interactwithengagingmediaelements,andareassessedbasedontheirprogressonrelevanttasksaswellasthequalityoftheproductstheyproduceratherthansolelyontheirperformanceonachievementmeasures(Wilkes,1996).Thisisevidentintheamountoftraditionalcoursemate-rialbeingconvertedforonlinedelivery,asmorecoursesbecomeavailabledailyforthedistantlearner.Furthermore,accordingtothekeynoteaddressbyW.GravestoattendeesoftheAssociationforLearningTechnologyConferencein1994,“600USuniversitiesandcolleges,plus100corporateassociates,arebusilyplanningaNationalLearningInfrastructureInitiativewhichcouldremoveconstraintsoftimeandplacefrommuchofUShighereducation”(Hawkridge,1995,p.5).Ofcourse,atpresent,manyclassesarestillconductedoncampus.Inthetra-ditionalclassroomsetting,teachersoftenfacethechallengeofdetermininghowbesttomeetsimultaneouslytheneedsofmanystudentswithvariedbackgroundsandskills.Manyfeelthatthelecturemethodoffersthemosteconomicalmeansofaccomplishingthistask.Althoughthisinstructor-pacedmethodofdeliveringcon-tentdoesallowthelecturertotransmitabundant,well-organizedinformationtoalargeaudienceinashortamountoftime,itdoesnotfollownecessarilythattheinformationiseffectivelyreceivedbytheintendedrecipientsfor,atleast,tworea-sons.First,instructorstraditionallyhavetaughttothemiddlelevelstudent,movingtooquicklyforslowerstudentstocomprehendallofthematerialandmovingtooslowlytokeepfasterstudentscontinuouslyengaged.Second,researchindicatesthatstudentsarenotabletolistenefcientlyforextendedperiodsoftime,recordingfewerimportantfactsasthelectureprogresses(Bonwell&Eison,1991).Someteachersofon-sitecoursesareattemptingtoaddresstheneedsofin-dividuallearnersbymeetingstudentsincomputerlaboratorieswheretheycandemonstrateskills,engagestudentsinactivelearning,actasfacilitatorandtutorforalargenumberofstudentsatonce,andalloweachstudenttoprogressthroughthematerialathisorherownpace.Inaddition,manyareaugmentingtheirinstruc-tionwithonlinematerialthatcanbeaccessedondemand,orcontentondemand1

PAGE 17

(CoD).InstructorsalsoareusingonlinecomputermanagementpackagestotrackstudentprogressandtheInternetasavastresourceaswellasameansofcom-municatingsynchronouslyandasynchronouslywiththeirstudents.ProponentsofAsynchronousLearningNetworks(ALN),whichcanbeusedfordistanceaswellasface-to-faceeducation,hopethatbymakingresourcesavail-ableonline,supportingcommunicationandcollaborationbetweenlearnersandteachers,andfocusingonsuchinteractions(ratherthanonthelimitedinteractionspossiblewiththestrictlectureformat)instructionwillbemorecosteffectiveandreadilyaccessibletothemasses.Inaddition,theyhopeself-pacingwillbecome“arealizablegoal”(Mayadas,1994).Makinginformationavailabletoalargenum-berofstudentsonlineandondemand,alongwithsoftwaretoolsformanagingthesubmissionandgradingofassignmentsandfortrackingstudentprogress,docontributetothefeasibilityofself-pacingfordistantandon-sitelearners.Advancesintechnologymakeitpossibletoexplorenewinstructionalparadigmsandtotakeasecondlookatoldones.Nowthatlearner-centeredself-pacingisaviablealternative,on-siteanddistantlearnerscanreapthebenetsofsuchaparadigmshift.Forexample,Keller'sPersonalizedSystemofInstruction(PSI)in-cludesself-pacingandaccesstoCoD,alesserformofself-pacing,astwoofitsvefeatures.PSIwarrantsrenewedinterestbasedonitsabilitytoimproveimmediateachievementandretentiontestscores(Kulik&Kulik,1975),itspopularitywithstu-dents(Kulik,C.Kulik,&Cohen,1979),anditssupportofteachersandassistantsactingastutorsforindividualstudents.Unfortunately,completeself-pacingisassociatedwithsomewell-documentedproblems.First,studentstendtoprocrastinate(Lamwers&Jazwinski,1989;Rae,1993).Thiscontributestohigherdrop-outandfailureratesaswellastoasubstan-tialshiftingofworkloadsforgraderstowardtheendofacourse.Second,itishardtoaccommodateself-pacinginmanycurrenteducationalinstitutions,whichrequirethatcoursesbecompletedinagiventimeframe,thatlessonstnicelyintoaxedtimeperiod,andthatgradesreectwhatastudentisabletoaccomplishundertheseconditions(Keller,1981).Furthermore,itismoredifculttokeepthegradingofassignmentsconsistentwhentheyarecominginatdifferenttimes,andinitially,itcanbetimeconsumingtodevelopinstructionalmaterials.Therearealsotheis-suesofdeterminingwhoownstherightstocoursematerials(intellectualproperty),andthelackofsystematicrewardsforinstructorswhodevelopsuchcourseware(Lloyd&Lloyd,1986).Manyofthesesamedifcultiesareresurfacingtodayindistancelearningenvi-ronments.Inparticular,Hiltz(1997)liststhefollowingissuesfacedbytheNewJer-seyInstituteofTechnology'sALN,theVirtualClassroom:studentprocrastination,higherdropoutrates,moreincompletes,homeequipmentanddialupproblems,difcultyingrading,theneedformoreinstructionalassistantsasthenumberofenrolledstudentsincreases,substantialtimerequiredtomakeinitialinstructional2

PAGE 18

materials,ownershipofinstructionalmaterials,compensationforproductionofma-terials,andtheneedforareducedworkloadduringsuchproduction.Furthermore,retention“or`drop-out'rateshavebeenofconcerntodistanceeducationinstitu-tionsinternationallyfordecades”(Evans,1995,p.70).Gillard(1993)arguesthatdistanceeducationis,inessence,individualizededucation.Oneaspectofindi-vidualization,self-pacing,anditsassociatedhigherlevelsofprocrastination,mayaccountforsomeportionofthedistantlearnerswhodrop-out,especiallygiventhefactthatmanydonotcompleteanycourseassignmentsatall.Baathreferstosuchstudentsasnon-starters.Ofcourse,anotherpossibleexplanationfordrop-outratesistheisolationfeltbysomedistantlearners.Infact,bothprocrastinationandisolationmayoperatetogethertoproducedrop-outratesthataresometimesashighas50%(Holmberg,1989).Inanycase,whilethelearnerisaffordedmorefreedominadistanceeducationcourse,s/heisalsorequiredtotakeagreaterresponsibilityinhisorhereducation.Alearner-centeredenvironmentoffersthematurelearnermanyadvantages,suchasgreaterinputontopicsexploredandself-pacedaccesstorichmediaele-mentsondemand.However,thenegativeeffectsofprocrastinationmayoutweighsuchbenets.Manyresearchershaveexploredthepossibilityofsupportingindi-vidualizedinstruction,whilereducingprocrastination.Researchindicatesthatin-corporatingdeadlinesinpersonalizedinstructioncanimprovepacingwithoutdetri-menttoachievement(Lloyd,1978).Therefore,usingalesserformofself-pacingwherestudentsstillhavedeadlinesforassignments,butareabletoaccessallcon-tentondemand,maymitigatestudentprocrastination,yetstillallowforsomelevelofself-pacingalongwithassociatedbenets.InaglossarycompiledbyKaplan-Leiserson(2001),CoDisspecicallydenedas“deliveryofanoffering,packagedinamediaformat,anywhere,anytimeviaanetwork”(Kaplan-Leiserson,2001).Deadlinesmayhaveanadditionalbenetforthestudent.Whileusingdead-lineshasnotbeenshowntoimproveconsistentlyimmediateachievementscores,itmayhaveapositiveeffectonretentiontestscores,becauseitforcesstudentstospaceouttheirexposuretothematerial,ratherthantomassitallatonetime.Althoughsomeresearchershavefoundthatspacedversusmassedexposure(orcramming)tocontentyieldscomparablescoresonimmediateachievementmea-sures,theyalsohavereportedthatspacedexposurecanenhancememoryfunc-tion,andhence,retentiontestscores(Bloom&Shuell,1981).Otherevidencesuggeststhatdeadlinesmaynothaveasignicantimpactonretentiontestscores(Robin&Graham,1974;Morris,Surber,&Bijou,1978),butmore,well-controlledresearchisneeded.Also,studentsreportastrongpreferenceforself-pacing.En-forcingdeadlinesmayjeopardizeaffectivebenetsanddenystudentstheoppor-tunitytolearnhowtomanagetheirownlearning.ProvidingstudentswithCoDallowsthemtohaveself-pacedaccesstocoursecontent,yetrequiresthemtomeetdeadlinesforcompletingassignmentsbasedonthatcontent.ByprovidingCoD,itishypothesizedthattheaffectivebenetsofself-pacingcanbemaintained,3

PAGE 19

whilemitigatingtheproblemofprocrastination,encouragingadistributedpatternofpractice,andraisingthelevelofcontentretention.Thechallengeistomaintainstudentcontroloverpacing,whileplacingcontin-genciesonthestudentthatencouragethetimelycompletionofassignments.Manyresearchershaveinvestigatedthemeritsofusingdeadlinecontingencieswithper-sonalizedinstruction.Althoughmanyalternativeshavebeenexplored,someoftheeasiesttomanageinvolveawardingbonuspointsforearlyworkanddeductingpenaltypointsforlatework.Findingsindicatethatsuchbonusesandpenaltiesareeffectiveinmotivatingstudentsnottoprocrastinate(Glick&Semb,1978a).Often,thepenaltiesemployedhavebeenharsh.Inonecase,25pointsweredeductedforeachdaylate.Inothercases,nocreditwasawardedforlateassignments.Someinvestigatorsevenforcedstudentswhomisseddeadlinestowithdrawfromthecourse(Glick&Semb,1978a).Inordertoexploretheeffectsofprocrastinationonretentiontestscores,threedeadlinecontingencieswereappliedtothecompletionofassignmentsbystudentsinanintroductoryinstructionaltechnologycourseatalargepublicuniversityinthesoutheast.Bonusesandpenaltieswereused,becausetheyhavebeenshowntobeeffectiveinreducingprocrastinationandbecausethisapproachisrelativelyeasytoimplementascomparedtootherapproacheslikeoneswhichrequirenego-tiatingcontractswithindividualstudents.Thestudentswereallgiventhesamesetofdeadlinesandwererandomlyassignedtooneofthreetreatments,whichvariedbydeadlinecontingencyalongacontinuumwithrespecttoharshness.Foronegroup,deadlineswererecommended(R)only.Foranothergroup,theywerecon-ditional(C),withbonuspointsawardedforworksubmittedearlyandpenaltypointsdeductedforworksubmittedlate.Fortheremaininggroup,theywereabsolute(A),requiringthatassignmentsbesubmittedbythedeadlineinordertoreceiveanypoints.Outcomesalongthreedimensionswereconsidered:achievement,studentpac-ingpreference,andprocrastinationlevel.Itwasexpectedthatscoresonaposttestgivenimmediatelyafterthetreatmentswereadministeredwouldreplicatethend-ingsofotherresearchersandindicatethattherewasnosignicantdifferenceinachievementlevelsfordifferentdeadlinecontingencies.Incontrast,itwasantici-patedthatscoresonameasureofretentiongivenonemonthlaterwouldindicateanadvantageforstudentsingroupsCandA,becausetheywouldbeforcedtospaceouttheirexposuretothecoursecontentratherthanbeabletomassalloftheirlearningneartheendofthetreatmentinterval.SuchmassedlearningwouldoccurforstudentsingroupR,iftheydidtendtoprocrastinatemore,asantici-pated.Carefulrecordswerekeptofwhenstudentssubmittedeachassignment,sothatthissuspectedhigherlevelofprocrastinationcouldbeveried.Inaddition,procrastinationwasgaugedbyhowoftenstudentsrequesteddeadlineextensions.4

PAGE 20

Theteachingassistants(TAs)wereaskedtorecordwhentheyreceivedexcusesfromstudentsforlateworkandrequestsforadditionaltime.Inordertoinvestigatetheeffectsofdeadlinecontingenciesonstudentsat-isfaction,aself-reportmeasureonpacingpreferencewasadministeredwiththeposttest.AlthoughresearchindicatesthatstudentspreferPSIovertraditionalin-struction,andsomehaveshownthatapplyingcertaindeadlinecontingencieshasnoeffectoncoursesatisfaction,itwasnotclearhowtheinclusionofthedead-linecontingenciesproposedbythecurrentstudymightaffectpreferenceforself-pacing.Noeffectwasanticipated.However,ifonedidexist,itwasexpectedthatstudentsinCandAwouldreportalowerpreferenceforself-pacingwithstudentsinAreportingthelowestpreference.Althoughconditionalandabsolutedeadlinesmaybeadvantageousinreducingprocrastination,enforcingsuchcontingenciesmaycomewithanaffectivecostforstudents.Assumingsuchacostandthattheutilizationofsometypeofdeadlinecontingencyiswarrantedbyhigherretentiontestscoresandreducedendofcourseworkload,conditionaldeadlinesmaybeabetterchoicethanabsolutedeadlines.Althoughmoredifculttoadminister1,theymaycreatelessangstforstudents.Again,surveyresultswereanalyzedwiththeexpectationthatdeadlinecontingencieswouldnotsignicantlyimpactstudentsatisfactionwithself-pacing.1.1PurposeThisstudyexaminedwhichwasmoreadvantageouswhenstudentsarepro-videdwithCoD–recommended,conditional,orabsolutedeadlines.Itwashopedthatdeadlineswouldreduceprocrastination,whilestudentsatisfactionwouldbemaintainedbyallowingstudentstoaccesscontentondemand.Itwasexpectedthatreducingprocrastinationwouldmakethecourseeasiertomanageforgradersandmightresultinimprovedlearningforstudents.Theidealcontingencywouldreduceprocrastinationlevels,wouldnotadverselyaffectimmediateachievementscoresorpreferencesforself-pacing,andwouldimproveretentiontestscores.Findingsfromthisstudymayhaveimportantimplicationsforteachingingen-eral,butshouldgeneralizebesttolocalanddistantadultlearnerswhoaccessCoD.Ifresultsreplicatetheworkofothersandindicatethatdeadlineseffectivelyreduceprocrastinationwithoutdetrimenttoinitialachievement,thisstudywillprovidefur-therevidencethatdeadlinesareanimportantcomponentofanycoursebasedonsomelevelofself-pacing,inparticular,thosewhichprovideCoD.However,the 1Byhavingstudentssubmitassignmentsonlineandtimestampingandapplyingbonusesandpenaltiesautomatically,neitherconditionalnorabsolutedeadlinecontingenciesaremoredifculttoadminister.However,theinitialcodingoftheonlinescriptsisslightlymoreinvolvedfortheconditionalcontingencies.5

PAGE 21

argumentfordeadlineswillbesupportedbest,ifretentiontestscoresarehigherforstudentsingroupsCandAthanforstudentsingroupR.Thecasefordead-lineswillbestrongerstill,ifdataonpacingpreferencearenotsignicantlydifferentbetweengroups.Ontheotherhand,ifstudentsingroupCreportasignicantlyhigherpreferenceforself-pacingthanstudentsingroupA,thenthiswouldseemtoindicatethatanyadditionaloverheadassociatedwithcontingencyCmaybeworththegaininaffectivebenettothestudent.1.2ResearchHypothesesLet RTg, PTg, Dg, Pg, Rg,and PPgbetheaverageretentiontestscore,posttestscore,differencebetweenposttestandretentiontestscores,procrastinationlevelintermsofdayslateonassignments,numberofrequestsforspecialconsider-ation,andpacingpreference,respectively,forstudentsingroupg,wheregiseitherR,C,orA.Tostatetheresearchquestionsunderinvestigationmorefor-mally,recommended,conditional,andabsolutedeadlinecontingenciesasappliedtoperformance-basedassignmentscompletedbypreserviceteachersinacollegecourseoninstructionaltechnologywillhavethefollowingeffectsonthedependentmeasures.Achievementscoresonaretentiontestgivenonemonthaftertheadminis-trationoftreatmentswillbehigherforstudentsinCandAthanforstudentsinR.Itshouldbenotedthat,duetoalackofresearchcomparingprocrasti-nationlevelsforstudentswithconditionalversusabsolutedeadlines,itisnotclearwhetherstudentsinCwillprocrastinatemoreorlessthanstudentsinA.However,itisanticipatedthatthestudentsinthegroupwiththelowestprocrastinationlevelwillperformbetterontheretentiontest.If Pcislowest,then RTc RTa RTrIf Paislowest,then RTa RTc RTrAchievementscoresonaposttestgivenimmediatelyafteradministrationoftreatmentswillnotdiffersignicantlyacrosstreatments. PTr PTc PTaThetwohypothesesabovecanreallybecombinedintoonehypothesisabouttheinteractionexpectedbetweentimeandtreatment.Althoughallgroupsshouldhavethesamebasicunderstandingofthematerialatthetimetheposttestisadministered,itisanticipatedthatthetreatmentconditionwillhave6

PAGE 22

aneffectontheamountstudentsforgetduringthetimeintervalbetweenad-ministrationoftheposttestandtheretentiontest,wheretheamountastudentforgetsismeasuredbysubtractingtheretentiontestscorefromtheposttestscore.ItisexpectedthatstudentsinRwillforgetmorethanstudentsinCandA.WithrespecttogroupsCandA,thegroupwiththelowestprocrastinationlevelisexpectedtoforgetless.If Pcislowest,then Dr Da DcIf Paislowest,then Dr Dc DaPreferenceforself-pacingreportedbystudentswillnotdiffersignicantlyacrosstreatments. PPr PPc PPaProcrastinationlevelasmeasuredbytheaveragenumberofdayslateonassignmentsperstudentwillbehigherforstudentsinRthanforstudentsinCandA,whiletheaveragenumberofrequestsfordeadlineextensionsperstudentwillbelowerforstudentsinR. Pr Pc Pa Rr Rc RaAgain,withrespecttostudentsinCandA,itisnotclearwhowillprocrastinatemoreorrequestmoreextensions.StudentsinCmayprocrastinatemorethanstudentsinA,whoreceiveamoreseverepenalty.However,studentsinCmayalsoprocrastinatelessinanattempttoearnbonuspoints.1.3DelimitationsTheauthenticcontextandonlinesubmissionofassignmentsbothincreasedtheecologicalvalidityofthestudy.Findingsshouldgeneralizewelltostudentsinotherclasses,includingdistantlearners.Becausetheparticipantswerepreserviceteachers,studyoutcomesapplytoadultlearnersatbest.However,tosupportgen-eralizationtothislargergroup,futurestudiesshouldconsiderothertypesofadultlearners.Inparticular,onemightexaminebusinessmajors,computersciencemajors,and/oremployeeswithprogrammingresponsibilities.Otheragegroupsshouldbeexaminedaswell.Inthisstudy,theacquisitionofaparticularcomputerskill,InternetprogrammingusingHTML,wasinvestigated.Althoughtheresultsshouldgeneralizetoother7

PAGE 23

computer-relatedskills,includingusingapplicationssuchaswordprocessorsandspreadsheets,andpossiblytoprogrammingwithhigherlevellanguages,futureresearchshouldbeconductedtosupportsuchgeneralizations.Also,itwouldbeinterestingtoseeifsubjectsotherthancomputerprogrammingwouldbenetfromsuchamodelofinstruction.1.4LimitationsTheincreaseinexternalvaliditymentionedintheprevioussection,obtainedbyplacingthestudyinarealisticcontext,impactedtheinternalvalidityofthestudy.Theauthenticeducationalsettingmadeitdifculttocontrolforallextraneousvari-ables.However,everyattemptwasmadetoensurethatthesewerekepttoaminimum.Forexample,oneofthegreatestthreatswasthatstudents,priortothetreatmentinterval,mighthaveformedstudygroupsintheongoingclassthattheyintendedtomaintain.Inthatcase,studentswhoworkedonassignmentsto-getherwouldhavedependentachievementscores,whichwouldadverselyaffectdataanalysis.Inordertominimizetheimpactofsuchdependence,asurveywasadministeredbeforethetreatmentintervalaskingstudentstolistthestudentswithwhomtheyregularlyworkedonassignmentsfortheclass.Allstudentsinagivenself-madestudygroupwererandomlyassignedtothesametreatment,andtheirscoresontheachievementmeasureswereaveragedtoproduceasingledatapointforanalysis.OtherattemptstostrengthenthestudyincludedhavingeachTAadministerallthreetreatmentsandrandomlyassigningstudentstotreatments.Also,becausetheTAscouldseriouslythreateninternalvalidityiftheyallowedstudentsingroupsCandAtosubmitassignmentsforcreditafterthedeadlines,allassignmentsweresubmittedelectronicallyandtimestamped.TheTAsalsomighthaveimpactedin-ternalvalidity,iftheyfeltthatanyofthetreatmentswereunjustandexpressedthisopiniontotheirstudents.InordertolessenTAaswellasstudentconcernaboutanypotentialadvantagesforaparticulargroup,bothstudentsandTAsweresup-pliedwithtwoimportantfacts.First,theywereinformedthatitwasnotclearwhichgroupmighthaveanadvantageintheend2.Second,theyweretoldthatthescoresofstudentsinthedisadvantagedgroup(s)wouldberaisedtoeliminatedifferences.Studentsweregiventhisinformationwiththehopethatitwouldreduceanyfeel-ingsofdemoralizationcausedbyassignmenttoaparticulartreatment.Providingthisinformationtothemwasnotexpectedtocauseanynewproblems,becausethenatureofthetreatmentsandtheelectronicsubmissionprocesswithautomatic 2TAsweretoldthatstudentsingroupsRandCmightearnbetterassignmentscores,butalsomightperformworseontheposttestand/orretentiontest,duetohigherlevelsofprocrastination.8

PAGE 24

timestampdidnotallowthestudentstogainaccesstoanytreatmentotherthantheirown.Inaddition,hadthestudentsnotbeeninformedoftreatmentdifferences,itwaslikelythatmanywouldhavelearnedaboutthenatureofthedifferenttreat-mentsfromfellowclassmates.Becausethisinformationmighthavebeenconveyeddifferently,dependingonthesource,itseemedadvisabletoinformallstudentssys-tematically.1.5AcronymsandDenitionsSeeTable1foralistingofcommonacronymsandTable2foralistingofcom-montermsandphrasesusedinthisreport.Table1.CommonAcronymsUsed AcronymRepresents AabsolutedeadlinesALNAsynchronousLearningNetworkAOLAmericaOnline,Inc.CconditionaldeadlinesCAIcomputeraidedinstructionCBVTCompetency-BasedVocationalTrainingCMIcomputermanagedinstructionCoDcontentondemandEdTechinstructionaltechnologycoursesampledFAQfrequentlyaskedquestionsFTFfacetofaceHTMLHypertextMarkupLanguageMACMacintoshcomputerMP3compressedsoundlePCpersonalcomputerPSIPersonalizedSystemofInstructionRrecommendeddeadlinesRDTRrelativedigressionfromatargetresponseTAteachingassistantTCtraditionalclassroomTItraditionalinstructionVCVirtualClassroomWAVuncompressedsoundleWYSIWYGwhatyouseeiswhatyouget 9

PAGE 25

Table2.CommonTermsandPhrases TermorPhraseRepresents self-pacednotimeconstraintsoncoursecompletioncontentondemand(CoD)accesstocoursecontentanytime,anywheredeadlinecontingencyconsequenceofsubmittingworkearly,ontime,orlatewithrespecttosupplieddeadlinesrecommendeddeadline(R)deadlinegiventoguidethestudentwhowishestocompleteallassignmentsinatimelyfashion;notassociatedwithanybonusorpenaltypointsconditionaldeadline(C)deadlinebywhichassignmentmustbesubmit-tedtoreceivefullcredit;assignmentssubmittedasetamount(Cb)earlierreceivebonuspoints;assignmentssubmittedlateuptoasetpoint(Cp)arepartiallypenalized;assignmentsarenotacceptedforcreditafterCpabsolutedeadline(A)deadlinebywhichassignmentmustbesubmit-tedtoreceiveanycreditretentionachievementscoreearnedbystudentsonninemultiplechoiceitemsandone12-pointessayquestioncoveringHTMLonemonthaftertreatmentadministrationposttestachievementscoreearnedbystudentson36multiplechoiceitemsandone48-pointessayquestioncoveringHTMLimmediatelyfollowingtreatmentadminis-trationpacingpreferenceself-reportofpreferenceforteacher-pacingversusself-pacingonaneightitemratingscalewithvaluesrangingfromonetoveprocrastinationlevelcombinationofaveragenumberofdayslateinsubmittingassignmentsandnumberofrequestsforspecialconsideration(i.e.,deadlineextensions) 10

PAGE 26

Chapter2LiteratureReviewThisstudydrewinspirationanddirectionfrommanysources,includingself-pacedinstruction,massedversusdistributedorspacedpractice,distancelearning,tutoring,grouplectureanddiscussion,activelearning,cooperativelearning,andAsynchronousLearningNetworks.Alloftheresearchareasdiscussedbelowin-formedthedesignofthecurrentstudy,whichinvestigatedtheeffectsofassignmentdeadlinesontheimmediateachievement,retentionofknowledge,satisfaction,andprocrastinationlevelofstudentswhowereabletoaccessallcoursecontentonde-mand.Inaddition,thecoursewareusedinthestudyincorporatedactivelearningmodules,supportedcollaborativelearning,andallowedtheinstructorsandcourseassistantstotutorindividualstudentsduringofceandcomputerlaboratoryhoursaswellasviae-mail.2.1SelfPacingSomewouldarguethatself-pacedtrainingdatesbacktoSocratesandhises-tablishmentoftheSocraticmethod.In1915,SidneyL.Presseydesignedtherstmechanizedteachingmachine.It“couldpresentmaterial,requirearesponse,andprovidereinforcement,aswellasadministerandscoremultiple-choiceex-aminations”(Back&McCombs,1984,p.4).Thisdevelopmentwasfollowedbyprogrammedlearning,personalizedinstruction,andcompetency-basedvocationaltraining.2.1.1ProgrammedLearningSkinner(1954)initiatedtheconceptofprogrammedlearningwhenheassertedtheneedformorefrequentandimmediatereinforcementintheclassroom.Healsoindicatedthatinordertoaccomplishthis,theteachermustemploymechanizedaid.Earlyprogrammedlearningstrategiesmadeuseofworkbooks,butmanynowrelyoncomputerstodelivercontent.Thisinstructionaltechniqueprovidesthestudentwithsmallunitsofinformation,eachofwhichrequiresanimmediateresponsefrom11

PAGE 27

thestudent.Eachresponseisevaluatedandfeedbackissuppliedimmediately(Back&McCombs,1984).Studentprogressisself-paced.2.1.2Keller'sPersonalizedSystemofInstructionAnotherformofself-pacedlearning,individualizedinstruction,suppliesthelearnerwithactivitiesthatarebasedonindividualdifferencesinskilllevel,cognitivestyle,aptitude,andbackground(Back&McCombs,1984).OnepopularinstanceofthisteachingstrategyisKeller'sPersonalizedSystemofInstruction(PSI).Itwasdevelopedinthe1960'sandusedextensivelyintheearly1970s.Inhisownwords,KellerdescribesthekeyfeaturesofPSIasfollows(Keller,1968).Thego-at-your-own-pacefeature,whichpermitsastudenttomovethroughthecourseataspeedcommensuratewithhisabilityandotherdemandsuponhistime.Theunit-perfectionrequirementforadvance,whichletsthestudentgoaheadtonewmaterialonlyafterdemonstratingmasteryofthatwhichpreceded.Theuseoflecturesanddemonstrationsasvehiclesofmotivation,ratherthansourcesofcriticalinformation.Therelatedstressuponthewrittenwordinteacher-studentcommunication;and,nally:Theuseofproctors,whichpermitsrepeatedtesting,immediatescoring,al-mostunavoidabletutoring,andamarkedenhancementofthepersonal-socialaspectoftheeducationalprocess.Sothemainfeaturesincludeself-pacing,requiredmastery,motivationalpresen-tations,immediatefeedbackandtutoring,andafthelementwhichisbetterde-scribedasprovidingstudentswithcontentondemand.Infact,anotherproponentofPSIrestatedthefourthiteminKeller'slistandexpandeditscoveragetoin-cludeprovidingstudentswith“essentialsubjectmatter...inwriting,ontape,onlm,bycomputer,orbyanymeansaccessibletothestudentwhenheisreadyforit”(Green,1974,p.5).Ina1979meta-analysisonPSI,Kuliket.al.foundPSItobesuperiorwhencomparedtotraditionalinstruction(TI)inpromotingacademicachievement,inpro-ducinglessvariationonachievementoutcomes,andinstudentratingsofcollegecourses.TheyalsofoundthatcoursecompletionratesandstudentworkloaddidnotdiffersignicantlybetweencoursesdeliveredusingPSIversusTI(Kuliketal.,1979).Inparticular,theyreportthatnalexamscoresforPSIcoursesareabout812

PAGE 28

percentagepointshigherthanforTIcourses.Retentionscoresareabout14per-centagepointshigher.Inaddition,students“ratePSIclassesasmoreenjoyable,moredemanding,andhigherinoverallqualityandcontributiontostudentlearning.”(Kuliketal.,1979,p.317)Finally,theyindicatethatcompletionratesmightactu-allybehigherforPSIcourses,becausefewstudentscompletesuchcourseswithD'sorF's.ItshouldbenotedthatthisisindirectconictwithLloyd(1978),whoreportsthatseveralauthorshave“amplydocumentedthatmorestudentswithdrewfromKellercoursesthanfromtraditionalcourses”(p.505).Althoughonemightarguethat,ifcompletionratesactuallyarelowerforPSIcourses,measuresofachievementmightbeinated,Kulik,Kulik,andBangert-Drowns(1990)counter-claimthat“superiorexaminationscoresarecharacteristicofPSIclasseswithele-vated,normal,andbelow-averagecompletionrates....HigherstudentachievementinPSIclassesisnotanillusioncreatedbythewithdrawaloftheweakerstudentsbeforenal-examinationtime”(p.286).Theybasetheirclaimontheresultsofameta-analysisonmasterylearning,akeycomponentofPSIcourses.Despitethewell-documentedadvantagesofthesystem,itsusehasdeclined.PSIwasresearchedheavilyinthe1970'sandlookedquitepromisingatthattime.Infact,KulikandKulik(1975)commentonthephenomenalndingsof39well-constructedstudiescomparingPSIwithTI.Theyreportthat38foundPSIsuperiorwithrespecttoachievement(34foundstatisticalsignicance),andthatthediffer-encesweregenerallylargeenoughtobeconsideredofpracticalsignicance.Intheiropinion,PSIhas“themostimpressiverecordachievedbyateachingmethodinhighereducation”(Kulik&Kulik,1975,p.230).Still,researchandusageofPSIdeclinedinthe1980's(Lamal,1984;Lloyd&Lloyd,1986).Problemsasso-ciatedwithPSIincludestudentprocrastinationandcorrespondingdrop-outrates,difcultyttingitintothecurrenteducationalsetting,keepinggradingconsistentwhenassignmentsaresubmittedatdifferenttimes,determiningwhoownstherightstocoursematerials(intellectualproperty),andthelackofsystematicre-wardsforinstructorswhodevelopPSIcourseware(Lloyd&Lloyd,1986).ManyofthesesameissuesareresurfacingtodayindistancelearningenvironmentssuchasALNs(Evans,1995;Hiltz,1997).Despitetheseproblems,themeritsofPSIovertraditionalmethods,alongwithadvancesintechnologythatshouldmakecoursemanagementandcoursewaremaintenancemorefeasible(Crosbie&Kelly,1993),warranttakingacloserlookatthesystem.Infact,withthecurrentshifttowarddistanceeducation,PSI,inpartorinwhole,mayproveaneffectiveparadigmfordistantlearners(Murdock,2000).Ifitistosucceed,theneachoftheissuesraisedabovemustbeaddressedsystematically.13

PAGE 29

2.1.3Competency-BasedVocationalTrainingCompetency-BasedVocationalTraining(CBVT)isamethodofself-pacedtrain-ingusedinsomeindustrialandvocationalsettings.Watson(1990)analyzedthreeCBVTprogramsoperatinginCanada,Minnesota,andAustralia.Hisndingsshedlightonissuesthatneedtobeaddressedwhenusinganymethodofself-pacedinstruction.InhisoverviewofCBVT,hepointsoutthatitiswell-suitedforcontentthatcanbeeasilydividedintosmallskills.Healsofeelsitisanappropriatemethodforindus-tryandcollege,becausestudentbackgroundsvarywidely,itpromotesgoodtraitssuchasindependenceandself-reliance,anditprovidesthematurestudentwiththeoptionsofexibleenrollmentandattendance(Watson,1990).PSIresearchershavealsonotedtheadvantageoftheirownself-pacedapproachinpromotingin-dependentscholarship,withstudentsmakingclaimssuchas“`IfI...don'tdowellinthiscourse,Ihavenoonetoblamebutmyself'”(Cooper&Greiner,1971,p.397).Insitevisitsconductedin1988and1989,Watson(1990)observedthat,whileitisrelativelyeasytolistcompetencies,onedrawbackofthisapproachisthatitistimeconsumingandexpensivetodevelopthetrainingpackages.Furthermore,whilemostofthestaffwerehappywiththesystem,someteachersreportedfeel-inginsecure,unsatised,powerless,and/orfrustrated.Theauthorrecommendspreparinginstructorsbetterandhavingthemparticipateinongoingstaffdevelop-mentinordertoimprovemoraleandmotivation.Acommitmentandbeliefinthecompetency-basedapproachcouldbefosteredbyhavingteachersacquirecom-petenciesindevelopingCBVTmaterials(Watson,1990).Studentsreportedthattheywereverysatisedwiththeinstructionalapproach,especiallywiththeself-pacedaspect.Overthethreeyearperiod,however,thegenerallevelofreportedenjoymentdeclined.ThiscouldbeduetoanynumberoffactorsnotdirectlyaddressedbyWatson(1990).Forexample,perhapstheequipmentusedtodeliversomeofthecoursecontentwasnotwell-maintained.Also,theauthorindicatesthatthequalityofcoursematerialstendedtodeclinewithtime.Thismighthavehadanegativeeffectonstudentsatisfaction.Themostfrequentcomplaintsmadebythestudentswerethatstaffwerenotalwaysavail-ableforguidanceandtesting,andthatgradingwasinconsistent.Theseproblemscouldbealleviatedtosomeextentbytheintroductionofcomputeraidedinstruction(CAI)andmanagement(CMI).Inaddition,coursematerialneedstobebettermain-tained.Writtenmaterialshouldbeunderstandablebymoststudents,andaudio-visualmaterialshouldbecoordinatedwithwrittenmaterial.Watson(1990)alsorecommendsthatlearnersbepreparedfortheself-pacedformat,andthatprinci-plesofself-pacedlearningbeincorporatedinthecoursedesign.Theseincludeusingsmallsteps,matchinglearningactivitieswithobjectives,requiringcontinu-14

PAGE 30

ousstudentresponding,andprovidingimmediateandregularfeedback(Watson,1990).2.1.4CriticalFactorsforSuccessfulSelf-PacedTrainingInanextensiveliteraturereview,BackandMcCombs(1984)foundthattheinclusionofacombinationoffactorsisessentialforsuccessfulself-pacedtraining.Carefulconsiderationofinstructionalfactorsensuresthatagoodmatchismadebetweencontentdeliveryandstudentneeds.Carefulconsiderationofmanagerialneedsfostersahighlevelofinstructordedicationandmotivation.Whentheneedsofstudentsandinstructorsaremet,thenself-pacedtrainingisconsideredcosteffectivewithrespecttomoneyandthequalityofgraduatesproduced,andhence,asuccess(Back&McCombs,1984).Specically,BackandMcCombs(1984)foundthatthefollowinginstructionalfactorswerepresentintheliteraturedescribingsuccessfulself-pacing.deliverymethodmatchedcurrentknowledgeandperformancelevelsaswellaseldrequirementscontinualInstructionalSystemsDevelopmentprocessemployedqualityinstructionalmaterials,withanadequatemixofmediastudent-studentinteractionsviateamandgroupactivitiesstudent-instructorinteractionsThefollowingmanagerialfactorswerealsopresent.strongmanagementsupportexibleimplementationapproachthatiseasilyadaptedtocurrentneedseffectiveschedulingoflimitedequipmentstaffinvolvementandparticipatorymanagementadequatestaffandinstructortrainingwelldenedinstructorroles15

PAGE 31

BackandMcCombs(1984)alsofoundthatadequateself-pacedinstruction“requiresacompletetaskanalysis,specicationofgoalsandobjectives,...andperformance-basedevaluation”(Back&McCombs,1984,p.43).Careshouldalsobetakentopreparestudentsfortheself-pacedformatandtheself-responsibilityitdemandsaswellastomatchinstructiontolearningstyles.Student-studentandstudent-instructorinteractionsestablishaforumwherestudentscangetneededencouragement,feedbackonprogress,andhaveachancetodiscussproblems.Finally,“in-housepersonnel[includinginstructors]shouldbeinvolvedindecisionmakingandcurriculumdevelopment”(Back&McCombs,1984,p.43).Inadditiontoincreasingfeelingsofsatisfaction,suchteamworkshouldhavetheaddedbenetofhelpingtodistributethetaskofdevelopingalloftheinstructionalmaterialforacoursethatmightoverwhelmasingleinstructorworkingalone(Smith,1974).2.1.5DisadvantagesoftheSelf-PacedApproachEventhoughself-pacedtrainingseemstoofferanumberofbenets,itsadop-tionishinderedbyseveralinherentaws.First,studentprocrastinationisamajorproblem(Rae,1993).Second,thisapproachdoesnotteasilyintothecurrented-ucationalstructure.In1981,Kellerstatedthathefearedreformwouldcomeslowly.Inhiswords,the“outlookisnotgoodforanysystemofinstructionthatthreatenstochangethelengthofaclassroomhour;thedurationofacourse;theshapeofagradedistribution;thepoliciesofadmission;thepracticesofregistrationandrecordkeeping;theintegrityoftheClassof'84;oranyoftheeverydayroutinesofteachers,administrativeofcers,supportingstaff,orevenstudents”(Keller,1981,p.38).Also,asmentionedearlier,trackingindividualstudentprogress,keepinggradingconsistent,andmaintainingqualityinstructionalmaterialsischallenging.2.1.6CurbingProcrastinationwithDeadlineContingenciesManyresearchershaveconsideredtheissueofprocrastinationandhowitmightbereducedoreveneliminated.Anumberofstrategieshavebeenemployedandincludegivingstudentsbonuspointsforcompletingworkearly(Powers,Edwards,&Hoehle,1973;Bufford,1976;Lu,1976;Lloyd&Zylla,1981;Reiser,1984;Lamw-ers&Jazwinski,1989),deductingpenaltypointsforsubmittingworklate(Reiser,1984;Ross&McBean,1995),applyingbothbonuspointsandpenaltypointsinconjunction(Morrisetal.,1978;Welsh,Malott,&Kent,1980;Crosbie&Kelly,1993),losingallpointsforlatework(Cooper&Greiner,1971;Robin&Graham,1974),requiringadditionalwork(Murdock,2000),havinginstructor-setdeadlinesforsomeoralltests(i.e.,midterm,reviewtests,unittests)(Lloyd,1978;Glick&Semb,1978b;Hobbs,1981),schedulinganearlynalforstudentswhonishallworkearly(Lloyd,1978),frequenttesting(Mawhinney,Bostow,Laws,Blumenfeld,16

PAGE 32

&Hopkins,1971;Wesp&Ford,1982;Wesp,1986),requiringstudentstowithdrawwhentoomanydeadlinesaremissed(Miller,Weaver,&Semb,1974;Roberts&Semb,1980;Lamwers&Jazwinski,1989),supplyingstudentswithtargetdead-lineswhichmustbemettoearnaparticularcourselettergrade(Reiser,1977),asingleearlyinstructor-imposeddeadline(Sweeney,Butler,&Rosen,1979),ver-balreinforcementfromproctors(Lu,1976),allowingstudentstosettheirowndeadlines(Welshetal.,1980;Lloyd&Zylla,1981;Roberts,Fulton,&Semb,1988;Roberts&Semb,1990),givingstudentsthechoicebetweenstudent-setandinstructor-setdeadlines(Roberts&Semb,1989),andallowingstudentstosetupdeadlinecontractsjointlywiththeinstructor(Lamwers&Jazwinski,1989).Moststudieshaveyieldedresultsindicatingthattheuseofdeadlinecontin-genciesisaneffectivemethodofreducingprocrastinationwithoutjeopardizingachievement(Glick&Semb,1978a,1978b;Robertsetal.,1988;Roberts&Semb,1989,1990;Wesp&Ford,1982;Ross&McBean,1995;Wesp,1986;Reiser,1977).Infact,onlytwoofthestudiesreviewedreportedndingasignicantdiffer-encebetweentreatmentsonachievementmeasuresgivenimmediatelyfollowingtreatmentintervals.Inonecase,thestudent-pacedgroupperformedbetter(Pow-ersetal.,1973).Intheother,theinstructor-pacedgrouphadsuperiorperformance(Hobbs,1981).Inbothcases,therewerepotentiallyconfoundingvariables.InastudyconductedbyPowersetal.(1973),studentsreceivedbonuspointsinonetreatment,butnotintheother.Studentswhodidnotreceivebonuspointslikelyweremoremotivatedtodowellonthenal,andinfact,theydidperformsigni-cantlybetter.Intheotherstudy,studentsinthecompletelyself-pacedtreatmentwereallowedtoretestformasteryuptothreetimes,whilestudentsinthegroupwithinstructor-imposeddeadlineswerenotallowedtoretakeunittests(Hobbs,1981).Inaddition,studentsinthelattergrouptookhalfasmanytestswhicheachcoveredtwiceasmuchmaterial.Itisquitelikelythatgradesinthecompletelyself-pacedgroupwerehigher,becausestudentsweretestedonlessmaterialeachtimeandwereabletoretakeunittests.Hence,theylikelywerelessmotivatedtoperformwellonthenal,whichtheyweretoldcouldonlyimprovetheirgradeandnotlowerit.Thelimitednumberofresearcherswhohavereporteddataoncourseratingsbystudentsunderdifferentdeadlinecontingencieshavegenerallyfoundnosignif-icantdifferenceinlevelofself-reportedsatisfactionwiththecourse(Hobbs,1981;Reiser,1977;Robin&Graham,1974).Inonecase(Hobbs,1981),however,thetreatmentconditionswereconfoundedbytheinclusionofmasteryintheself-pacedtreatmentandnotintheinstructor-pacedtreatment.Inanotherstudy(Reiser,1977),thecontingencywasadropinlettergrade,iftheinstructor-setdeadlineswerenotmet.Inanother(Robin&Graham,1974),deadlineswereabsolutefortheteacher-pacedgroup.Satisfactionhasnotbeenreportedforallofthediffer-entcontingenciesthathavebeeninvestigated,andso,moreevidenceisneededtoensurethatthevariousalternativesofinstructor-pacingdonotadverselyaffect17

PAGE 33

studentsatisfaction.Interestingly,someresearchershavesuggestedthatsatis-factionmaybeaffectedbythewaythecontingencyisdescribed(Murdock,2000;Robin&Graham,1974).Murdock(2000)providedtwogroupswiththesamecon-tingencies,inonecasedescribingthemintermsofremediationandintheotherdescribingthemintermsofenhancingmastery.Hisstudy“showedthatdescribingthecontingencyregardingitspotentiallearningbenetsgeneratedmorepositivestudentratings,andlessreportsofanxiety,thanwhenthecontingencywasde-scribedasapenalty”(Murdock,2000,p.151).Becausedeadlinecontingencieseffectivelyreduceprocrastinationwithoutdetri-menttoachievement,andbasedonavailableevidence,neednotadverselyaffectstudentsatisfaction,theyappeartobeanimportantcomponenttoincludeinanyPSIcourseandpossiblyanycourseincorporatingsomelevelofself-pacing,in-cludingcourseswhichprovideCoD.Manyofthecontingenciesresearchedrequiresubstantialadministrativeoverhead.Becauseallseemtobeeffectiveinreducingprocrastinationwithoutdetrimenttoachievementandpossiblysatisfaction,itwouldseemadvisabletoselectonewhichrequiresminimalefforttoimplement,suchasenforcingoneearlydeadlineorawardingbonusand/orpenaltypointsdependingonwhenassignmentsaresubmittedrelativetoduedates.Again,moreresearchisneededtodeterminetheeffectsoftheselectedcontingenciesonstudentsatis-faction.2.1.7MitigatingDisadvantageswithCAIandVideoSeveraloftheproblemsassociatedwithself-pacingcanbelessenedbyintro-ducingcomputeraidedinstruction(CAI)andmanagement.Forexample,ifde-signedwithexibilityinmind,acomputerprogramcanallownewlearningmodulestobeincorporatedintoitwithoutthecostlyneedtoreformatandreprintpaperpack-ets.Coursewaredeliveredonlineisalsoquiteeasytoupdateandnopaperpacketseverneedtobeprinted.Studentscanviewandprint(ifdesired)themostuptodateinformationatanytime.Also,studentswouldnothavetoscheduletimewithstafffortesting.Thiscouldbeaccomplishedviathecomputer.Likewise,incon-sistentgradingcouldbereducedoreliminatedviaautomation.Automatedgradingwouldalsoallowtheinstructortodealeffectivelywiththeincreasedgradingloadandresultingbottleneckimposedbytheneedtoevaluatecompetenciesforeachstudentonalargernumberofsmallactivities.Furthermore,theautomatedgradingcouldbeusedtoprovideformativeevaluationtothestudentthatwouldhelpfocushisorherquestionsduringsessionswiththeinstructorand/oraids.FlexibleCAIwouldallowindividualteacherstoeasilyincorporatetheirownlearningmodulesintothesystem.Thiswouldsatisfytwogoals.First,eachin-structorwouldfeelthats/heplaysanintegralpartintheprocessandthathisorherinputisvalued.Second,thestudentswouldhaveavarietyofresourcesfromwhich18

PAGE 34

todraw.Forexample,saythatseveralinstructorscontributeshortvideoclipsoflec-turesonthesametopic.Then,ifastudentdoesnotunderstandhowoneinstructordescribesaconcept,s/hecanlistentoanotherinstructor'sdiscourse.Otherwise,thestudentcanprogresstonewmaterial.Anotheradvantageofvideoclipsisthatthestudentcanreviewthelectureoverandoveruntils/heunderstands–ataskdifculttoaccomplishwithalivelecturer.Rae(1993)reportsthat,inhiscourseondiscretemathematics,“theuseofvideoandcomputer-deliveredinstructionenablestheuseoffewerandlesswell-qualiedtutors....Discountingthecostofthevideosand[CAI],...thecourseisac-tuallycheapertorunthanifitweretaughtconventionally”(Rae,1993,p.44).Furthermore,theincorporationofvideoallowsforapersonalelementoftenlackinginCAIalone.WhenRaerstintroducedthevideostohiscourse,heclaimsthattheeffectsonstudentperformanceweredramaticandwerereectedinhighexamina-tionmarks.Also,75%ofthestudentsreportedonasurveythatthevideoswere“helpful”or“veryhelpful”,whilelessthan60%reportedthesamefeelingsaboutthecourseCAIalone(Rae,1993).AnotherteamofresearchersreportsthatusingcomputerstoautomatetestingmadeitpossibletoeffectivelyrunaPSIcoursefor51students“withouttheveproctorsthatwouldnormallyberequired”(Crosbie&Kelly,1993,p.366).2.2Massed,Distributed,andSpacedPracticeTheliteratureonmassed,distributed,andspacedpracticehasfocusedmainlyonthepracticeofnonfunctionaltasksandmotorskills(Mulligan,Guess,Holvoet,&Brown,1980;Grote,1995).Althoughthe“superiorityofdistributedpracticeovermassedpracticehasbeenwelldocumentedinpsychologicalliteratureforabout100years,...thebulkoftheresearchinthisarea...occurredinlaboratorysettings”(Grote,1995,p.97).However,afewresearchershaveexploreditsim-plicationsintheclassroomforlearningandretentionofmorecomplexinformationlikePhysicstopics(Grote,1992,1995),Astronomytopics(Lu,1978),andFrenchvocabulary(Bloom&Shuell,1981).Massed,distributed,andspacedpracticearedistinguishedfromoneanotherintermsoftheintervalsbetweentrainingtrials.Inmassedpractice,thedurationoftheintervalisnegligible.Inspacedpractice,thetimebetweentrialsisusedforrest.Indistributedpractice,itisusedtopracticedifferentmaterial.Distributedpracticeismostlikethetypicalexperiencesofastu-dent.Trialstakeplaceduringclass,studytime,andwhilecompletingassignments.Thetimebetweentrialsisdevotedtoothertasks.Researchindicatesthat“skillstaughtwithaspacedordistributedtrialsequencearelearnedbetterthanskillslearnedusingmassedtrials”(Mulliganetal.,1980,p.328).Ofthestudiesreviewedwhereresearchersconsideredtheimmediate19

PAGE 35

acquisitionand/orretentionofcomplexinformationinaclassroomsetting,oneinvestigatedonlyimmediateacquisitionandfoundasignicantdifferenceinfa-vorofdistributedpractice(Lu,1978),twoinvestigatedonlyretentionandfoundasignicantdifferenceinfavorofdistributedpractice(Grote,1992,1995),andoneinvestigatedboth,ndingnosignicantdifferenceinimmediateacquisitionbutsignicantlyhigherretentionratesforstudentswhoengagedindistributedprac-tice(Bloom&Shuell,1981).Clearly,moreevidenceisneededbeforeanydeni-tivestatementscanbemadeaboutthemeritsofdistributedpracticeovermassedpracticeinanauthenticsetting.However,acloserinspectionofthefourstudiesreviewedindicatesthatthendingsofBloomandShuell(1981)arestrongest.ThestudybyLu(1978)equatesthehierarchicalpresentationofconceptsinwhich“precedingideasareintegratedwithineachnewideaasitispresented”(p.254)withdistributedpracticeandtheuseof“advanceorganizers”and“delayorganizers”withmassedpractice.Inallthreetreatmentsconsidered,studentswerepresentedwithapproximately13minuteaudiolessonscoveringthematerial,followedbya12minutefree-recalltestwherestudentswereaskedtorecordasmanyfactsastheycouldremember.Theyweretoldaboutthefree-recalltestpriortohearingtheaudiolessons.Onegroupwasgivenaninitialoverviewofthematerialrst,anotherwasgivenasummaryofthematerialafterwards,andthethirdwastoldhowconceptsrelatedtooneanotherastheywerediscussed.Thesetreatmentconditionsdonotrepresentinstantiationsofmassedversusdistributedpracticeasclearlyastheotherthreestudiesdo.ThetworeportsbyGrote(1992,1995)describedstudiesthatweresimilarinnature.Ineachcase,studentsweredividedintotwogroups.BothgroupsweregivenclassroominstructionontwoPhysicstopics,callthemtopicsAandB.Thefollowingday,onegrouppracticedallstudyquestionsfortopicA,whiletheothergrouppracticedallstudyquestionsfortopicB.Thenthegroupsswitchedtopics,andoverthenextseveralweeks,eachgrouppracticedonlyafewstudyquestionsadayuntilithadstudiedallofthesamequestionsthattheothergrouphadstudiedenmasse.Afterallpracticewascomplete,theauthorwaitedbetweentwoweeksandtwomonths,dependingontheparticularstudy,beforegivingaretentiontestonbothtopics.Inallcases,thematerialthatwaslearnedviadistributedpracticewasbestremembered.Unfortunately,duetothefactthatthemassedpracticetreatmentalwaysoccurredbeforethedistributedpracticetreatment,theretentionintervalwasalwayslongerforthemateriallearnedenmasse.Duringthethreetofourweekintervalittookstudentstocompletethedistributedpracticephaseofthestudy,theylikelywereforgettingwhattheyhadlearnedduringthemassedpracticephaseofthestudy.Inaddition,althoughtheauthordoesnotindicatewhetherornotthestudentswereinformedabouttheupcomingretentiontests,itislikelythattheywereinformed,becausethestudywasconductedaspartofanactualclass.Ifthatisthecase,thenstudentswouldhavehadtheopportunitytostudyfortheexam,andinessence,tolearnorrelearnthematerial.20

PAGE 36

Asmentionedbefore,thestudyconductedbyBloomandShuell(1981)offersthestrongestevidence,indicatingthatmassedanddistributedpracticeyieldcom-parableinitialresultswithrespecttolearning,butthatmemoryisenhancedunderconditionsofdistributedpractice.Fifty-sixFrenchstudentswerestratiedaccord-ingtopreviousperformanceandrandomlyassignedtooneoftwogroups.Bothgroupslearned20vocabularywordsandweretoldtheywouldbetestedonthematerialafterpracticingit.Onegrouppracticedthewordsduringthree10minutesessionsconductedonthreeconsecutivedays,whiletheothergrouppracticedthemduringone30minutesession.Thelast10minutesofthemassedpracticesessioncoincidedwiththelast10minutesessioncompletedbythedistributedpracticegroup.Allstudentsweretestedimmediatelyafterthenalpracticeses-sion.Nosignicantdifferencewasfoundbetweengroups.However,inanunan-nouncedtestgivenfourdayslater,studentsinthedistributedpracticegrouphadsignicantlybetterrecalloftheFrenchwords.Thenatureofthisstudybetterre-ectswhatmightbeexpectedinanactualclassroom,wheresomestudentsstudythematerialthroughoutthecourse,whileotherscramforexamsattheendofthecourse.Thatfact,alongwiththefactthattheretentiontestwasunannounced,sothatstudentsdidnothaveachancetoreviewthematerial,providesstrongevidencethat,whiledistributedpracticemightshownoadvantageonimmediatemeasuresofachievement,itmightshowanadvantageonlaterretentionofthema-terial.Theauthorsexplicitlypointouttheimportanceofmeasuringretentionwhencalculatingthebenetsofdistributedpractice.ThendingsofBloomandShuell(1981)actuallymayexplainwhylittledif-ferenceinachievementhasbeendocumentedwhenusingdeadlinestoreduceprocrastination.Lowerlevelsofprocrastinationshouldcorrespondwithamoredistributedlearningapproach,inwhichcaseadifferenceinachievementmaybeindicatedonaretentiontest.Suchissuesarecriticalforevaluatingtheneedfordeadlinesinacoursewithsomelevelofself-pacing.Afterall,if“studentscannotrememberwhattheyhavelearned,theymightaswellnothavelearneditintherstplace”(Bloom&Shuell,1981,p.247).AlthoughanumberofresearchershaveinvestigateddisparitiesbetweentheretentionscoresofstudentsincoursestaughtusingPSIversusTI,whichdifferfromoneanotheralongimportantdimensionsinadditiontopacing,fewhaveconsideredtheeffectofvariousdeadlinecontingenciesonretentionforgroupswhichareallexposedtoPSI.Ofthosewhohave,oneteamfoundnodifferenceinretentiontestscores(Robin&Graham,1974),whileanotherfoundrelativelyweakevidenceofaninteractionbetweentimeandtreatment(F(2,45),p.094,f.31)withthetrendovertimeindicatingapossibleadvantagefortheself-pacedgroup(Morrisetal.,1978).However,thendingsofbothofthesestudiesshouldbeinterpretedwithcaution.21

PAGE 37

InthestudyconductedbyRobinandGraham(1974),volunteerswererecruitedfromalecturesection,withunder20assignedtoeachtreatmentafterparticipantsdroppedout.Someevidenceindicatesthatthelargernumberwhodroppedoutoftheteacher-pacedgroupmayhavedonesobecauseofthecontingency,butnodescriptivedataaresuppliedtoenablethecomparisonofdropoutsacrosstreat-ments.Furthermore,studentshadadvanceknowledgeoftheretentiontest,whichwasgivenafteronlythreeweeks,thosewhocompletedunittestsearlyexperi-encedalongerretentioninterval,theposttestandretentiontestbothhadalowmasterycriterionof50%,andformanystudents,theseexamshadnoeffectontheirnalgrades.Finally,althoughsubmissionratesappearedtobeslightlymoreuniformforstudentsintheteacher-pacedgroup,accordingtoavisualinspectionoftheirindividualcumulativesubmissionrecords,analysisoftherateofrsttakesandthequarter-life1forstudentsonaveragedidnotprovidestrongevidencethatthegroupsdifferedsignicantlyinresponserate,orinotherwords,inlevelofpro-crastination.ComplicationsinthestudyconductedbyMorrisetal.(1978)alsomaketheirndingsdifculttointerpret.Studentswereinformedthatnoneoftheachievementmeasureswouldaffecttheirgrade,theposttestandretentiontestwereannounced,lessthan43%ofthestudentsvolunteeredtoreturnninemonthslaterandtaketheretentiontest,nostatisticsweresuppliedtosupporttheclaimthatthevolunteersubsetsforeachtreatmentadequatelyrepresentedtheoriginaltreatmentsetswithrespecttoposttestandnalgradedistributions,andthedegreesoffreedomfortheerrortermappearstohavebeenmisreported2as98ratherthan45,possiblyinatingthecalculatedFvaluewithrespecttothecriticalFvalue.Obviously,moreresearchisneededtodetermineifdeadlinecontingenciescanbeusedinacoursewithsomelevelofself-pacinginordertoreduceprocrastinationbyforcingamoredistributedpatternofpractice,andultimately,toincreasestudentretentionofcon-tent.ThecurrentstudyattemptstobringtogetherthesetwolinesofresearchandtondevidencethatwillinformtheuseofdeadlinecontingenciesincourseswhichsupplyCoD. 1RobinandGraham(1974)calculatedtherateofrsttakesforanindividualstudentbydividingthetotalnumberofunitexamsattemptedatleastoncebythetimeintervaloverwhichtheattemptsweremade.Theycalculatedthequarter-lifebydividingthelengthoftimeittookthestudenttocompleteonequarterofthersttakesbythetotaltimeintervaloverwhichrstattemptsweremade.2Morrisetal.(1978)indicatedthat51studentstooktheretentiontest.Becausetheyconsideredtwotreatmentconditions,apretest,aposttest,andaretentiontestintheiranalysis,thedegreesoffreedomfortheerrortermshouldbe512n3r45.22

PAGE 38

2.3DistanceEducationViaAsynchronousLearningNetworksAsmentionedearlier,ALNssupportcollaborativelearningandself-pacing.An-driole,et.al.usedaninstantiationofthesystemtoteachcollegestudentsatDrexelUniversitysoftwaresystemsdesignandreportedtwoofthemainbenetstobetheALN'saccommodationofself-pacinganditscost-effectiveness.Theyfeltitwascost-effective,becausethesystemcouldbeimplementedusingoff-the-shelfhard-wareandsoftware.Also,asthenumberofstudentsincreased,itwasestimatedthatthecostofinstructorswouldbeless,becausecheaperALNassistantscouldbeemployedtohandletheextraload.Intheveinofcompetency-basedtraining,theyrecommenddevelopingalessonforanALNcoursebyrstdecidingonde-siredknowledgeandskilloutcomes,nextconvertingtheseintotopics,subtopics,andassignments,andnally,matchingcoursereadingswiththetopics(Andriole,Lytle,&Monsanto,1995).Inagreementwiththeliteratureonself-pacing,Andrioleet.al.pointoutthatforanALNtobesuccessful,structureisimportant.Theremustbeaclearandpredictablescheduleand“real-timemonitoringofstudentperformance”(Andrioleetal.,1995,p.101).Theyaddthatallmaterialsmustbeonline,thecourseshouldhaveacommon“lookandfeel”,andthatonlinediscussionsshouldbeintroducedandconcludedwith“openingandclosingdiscussionwindows”(Andrioleetal.,1995,p.101).Furthermore,studentsshouldbeabletolearnALNsoftwarein2hoursorless,tocommunicatewithinstructorsandfellowstudentspubliclyandprivately,toseetheworkofotherstudentsandexamplesofgoodassignments,andtopostquestions(Andrioleetal.,1995).Again,inagreementwithotherresearchonself-pacing,studentresponsestosurveysgivenattheendoftheirALNcoursesrevealhighstudentsatisfactionwiththesystem.Forexample,80%foundconventionalcoursesmoreboringthanALNcoursesandsaytheywilltakeanotherALNcourse.Also,75%didnotmisslecturesand70%felttheylearnedmorethantheywouldhaveexpectedtolearninacon-ventionalcourse.Inaddition,85%felttheyhadmoreaccesstotheinstructor,75%felttherewasmorestudent-studentcommunication,and95%feltitwasusefultoseeotherstudents'workandassignments.Anothercontributingfactortothesuc-cessoftheirALNislikelythefactthattheyemployeddatabasestomanageclassdiscussions,coursematerials,assignmentdescriptions,andinstructor/studentdi-aries(Andrioleetal.,1995).AlthoughdataindicatethattheirALNissuccessful,theauthorsstillfearthatALNsmaynotbeuniversallyaccepted.Theybelievethat“notallfacultyorinstitu-tionscan–orwillwantto–”(p.101)switchtoanALN,whichisbasedsoheavilyonself-pacing.Also,initiallypreparingmaterialforanALNcourseisverytimecon-suming,andunlesstheinstructorisallottedextratimefortheseendeavors,s/hewillbelesslikelytochoosetocommittheoverloadtimeneeded.Theystatethat,23

PAGE 39

whiletheyfeelALNsareanadvantageousinstructionalandmanagementstrategy,the“dangertodayisthatasynchronouslearning–alongwithotherformsof`dis-tanceeducation'–willremaininthelabsandinthehandsoftechno-educators–[those]whoseldomrepresentthemainstreamfacultyinterests”(Andrioleetal.,1995,p.101).TheNewJerseyInstituteofTechnologyisalsolookingintotheuseofALNs.Theirsystem,calledtheVirtualClassroom(VC),isusedtoenrichon-campuscourseswheremostclassinteractionsarefacetoface(FTF)andtosupportdis-tancelearningenvironmentsthatutilizeminimalFTFencounters.Lecturesarede-liveredviaaudioandvideochannelsandconferencingistext-based.Thesystemsupportscollaborativelearning,self-pacing,andindividualizedinstruction.Soft-warestructuresareinplacethatpromoteeffectivecollaborationbetweenstudentsandfacultybyorderingtranscriptsofdiscussionsandforcingactiveparticipation.CollaborativetechniquesemployedintheVCinclude“seminar”typeexchanges,debates,groupprojects,casestudydiscussions,simulationandrole-playing,shar-ingofhomeworksolutions,andcollaborativecomposition.Also,studentsareaskedtoidentifykeyskillsandconcepts,makeupquestionsbasedonthose,andthensharethemwitheachotherandattempttoanswerthem.IntheVC,contentistailoredtotheindividualinthesensethatuniquetopicsareassignedandtheequivalentofcontentbookmarksaremaintainedforeachstudent.Also,anelectronicgradebookisinplace.TheVC'ssupportofself-pacinggivesstudentsmoretimetoreectbeforeengagingindiscussions,makesiteasiertotschoolintotheirbusylives,andmitigatespossiblefrustration,becausetheyarenotforcedtoprogressthroughthematerialataratethatistoofastortooslowfortheirabilities(Hiltz,1997).Twocompletedegrees,theB.A.inInformationSciencesandtheB.S.inCom-puterScience,areofferedusingamixtureofvideoandtheVC.Hiltz,inexaminingoutcomesfortheInformationSciencesprograms,reportsthatmasteryintheVCisequalorsuperiortomasteryinthetraditionalclassroom(TC).Also,VCstudentsre-porthighercoursesatisfaction.Inaddition,VCstudentswhocollaborateare“mostlikelytojudgetheoutcomesofonlinecoursestobesuperiortotheoutcomesoftraditionalcourses”(Hiltz,1997).Findingsarebasedonpre-andpost-coursequestionnaires,directobserva-tionsofonlineactivities,interviewswithselectedstudents,tests,coursegrades,andfacultyreports.Hiltzacknowledgesthat“self-reportdatafromquestionnairessuffersfromsomevalidityissues”(Hiltz,1997).Tomitigatesucheffects,conden-tialitywasguaranteedandmeasuresweretakentoreassurethestudentsofthisfact.Responsesofstudentstakingthesamecoursewiththesameteacherwerecompared.SectionsthatincorporatedtheVCwerecomparedtosectionsthatdidnot(TC).StudentsutilizingtheVCwerealsoaskedtocomparetheircurrentexpe-rienceswithexperiencesinpast,moretraditionalcourses.24

PAGE 40

StudentsintheVCreportedthefollowingproblems.Thirteenpercenthadseri-ousPC-relatedproblemsand40%-50%hadseriousproblemswhendialinginduetoreceivingabusysignal.Inaddition,VCstudentsreportedthattheydevelopedfewernewfriendshipsinclass,andthattheyweremorelikelytostop“attendingclass”.Procrastinationwasalsoaproblem(Hiltz,1997).Ontheotherhand,VCstudentsweremorelikelytofeeltheyhadparticipatedactivelyindiscussions,toratethecoursehighly,andwerelesslikelytoreportthattheclasswasawasteoftime.WhenaskedtocomparetheircurrentVCexperiencewithpreviousTCexperiences,71%reportedthattheyfelttheyhadbetteraccesstotheirVCinstructor.Thiswasdependentonthefactthattheinstructorwasavailableonlineatleastonceperday.Sixty-ninepercentfoundtheVCmoreconvenient,55%weremotivatedtoworkharder,becausefellowstudentswouldseetheirwork,and66%foundseeingothers'workbenecial.Fifty-eightpercentdisagreewiththestatementthattheywouldnottakeanotherVCcourse.While40%felttheylearnedmoreintheVC,only21%disagreedwiththatfeeling.While20%disagreedwiththeassertionthattheVCincreasedthequalityoftheireducationalexperience,58%concurredthatitdid(Hiltz,1997).Overall,ratingsofALNbasedcoursesareequalorsuperiortotraditionalcourses.StudentperformanceinanALNenvironmentisgenerallyequaltoorbetterthanperformanceinaTCsetting.Inthecoursesconsidered,50%oftheVCstudentsearnedan`A'ora`B'ascomparedto31%oftheTCstudents.However,ALNdropoutratesandincompleteoutcomesaregenerallyhigherthaninTC.Also,moreinitialcoursetimeisrequiredtoworkoutcourselogistics.Furthermore,qualityALNismoreexpensive,butcostscanbecutbyadoptingadifferentiatedstafngmodelandhavingTAsperformsomeoftheinstructionalduties(Hiltz,1997).Hiltzalsodiscussessomeimportantissuesraisedbyfaculty.First,thevideosusedbymanyofthecoursesaretimeconsumingtoproduce,takingvastamountsoftimetoprepare,rehearse,tape,andreview.Also,distributioniseitheralogisti-calchallengefortheteacherwhoisshippingthemorthestudentwhoisrecordingthem.Inthefuture,digitizedvideomodulesdistributedonaCD-ROMortheInter-netwilllikelybeexplored.Second,theVCcreatestheneedforanewinstructorroleincoordinatinginteractions.Facultyneedtobetrainedandsupportedinac-quiringnecessarynewskills.Third,gradingisalsoalogisticalproblem,becauseassignmentsarriveatdifferenttimesanddays.It'sharderfortheteachertokeepgradingconsistentwhenassignmentsarenotbatchprocessed.Fourth,workloadisdirectlyproportionaltothenumberofstudents.Fourstudentswilllikelyposttwiceasmanyquestionselectronicallyastwo.FacultyneedtobesupportedasthenumberofVCstudentsincreases.Finally,thereistheissueofintellectualproperty.WhoownstherightstothevideotapesandALNmaterialsproducedbytheinstructor?Also,howistheteachercompensatedforthisextrawork?Attheveryleast,aninstructorshouldenjoyareducedcourseloadduringanysemester25

PAGE 41

inwhichs/heisconvertingacoursetoanALN.Also,thereshouldbeclearpoliciesofrewardingsucheffortswhenconsideringpromotionandtenure(Hiltz,1997).Lookingtothefuture,HiltzexpectsthatALNswillincreasecompetitionamongschools,thattherewilllikelybelessschoolsandmoreadjuncts.Shepredictsthatuniversitieswillbetransformedfromplacesyoungpeoplegotonishgrowingupinto“centersforavarietyofdegreeprogramsdesignedtosupportstudentsofallages”(Hiltz,1997).DespitethemanyadvantagesoftheALNs,therearestillsomesubstantialdisadvantagesthatmayhindertheirchancesofbeingwidelyac-cepted.Mostnotably,theydonotteasilyintothecurrenteducationsystem,initialpreparationofmaterialssuchasvideoistimeconsuming,itishardtomaintainconsistentgrading,studentsseemlesscloselyconnectedinthattheymakefewernewsocialbonds,studentstendtoprocrastinate,andhigherratesofincompleteoutcomesanddropoutsarerecorded.Manyoftheseproblemsareconsistentwithwell-documenteddisadvantagesreportedintheliteratureonself-pacedtraining.2.4OtherTeachingParadigmsAsmentioned,thedesignofthecoursewarewasinspiredbyworkinseveralareas.Wheneverpossible,thebestfeaturesofseveralinstructionalparadigmswerebuiltinand/orsupported.Forexample,becauseallcontentisdeliveredviathecoursewareondemand,instructor-studentinteractionsaregenerallyone-on-one.Contentisorganizedandpresentedconciselyintheformofnarratedvisuals.Thesearekeptshort,becauseresearchindicatestheattentionofthepassivelis-tenerwanesquickly.Lessonsarealsoactiveandseparatedfromoneanotherbyassignmentsinwhichstudentsapplyimmediatelytheknowledgetheyhavejustgained.Finally,whilestudentcollaborationisnotdirectlysupportedbythecourse-ware,itispermittedandencouraged.2.4.1TutoringOneofthestrengthsofself-pacedtrainingisthatitcanfosterthegrowthofa“relationshipbetweeneachstudentandanindividualtutor”(Rae,1993,p.48).Thetutormightbetheteacheroranotherstudentintheclass.Raefoundthatsucharelationshipdevelopedwithabouttwo-thirdsofthestudents.Theserelationshipswereallone-on-oneand“initiatedbythestudents'ownwork”(Rae,1993,p.48).Lepper,Woolverton,andMumme(1993)assertthattheuseoftutorsisaneffectiveteachingstrategyandthatthequalityofCAIitselfcanbeenhancedbybasingdesignonthepracticesofexperthumantutors.Theauthorsstudiedsuchtutorsinordertodetermineeffectivetechniquesofone-on-oneinstruction.Their26

PAGE 42

ultimategoalwastooutlinetheelementsthatshouldbeconsideredinthedesignofcomputer-basedtutors.Experttutorswereselectedfromfourgroupsoftutors.Onegrouptaughtsecondgradestudentshowtocarrywhenadding.Onegrouptaught3rdand4thgraderscomplexwordproblems.Twomoregroupsworkedwith4th,5th,and6thgradersononeoftwocomputerdrills–“TheFactory”or“Darts”.Theexperttutorswereselectedbasedon“objectivelearningmeasuresandindependentratingsoftutors'overalleffectiveness”(Lepperetal.,1993,p.77).Theauthorsfeelthatcurrentcomputertutorsapproachthetaskoftrainingwiththeassumptionthatthelearnerisalreadymotivatedandattentive.Cognitiveissuesareoftenaddressed,whileaffectiveandmotivationalissuesarenotconsidered.Forexample,computertutorsgenerallydetermine,fromacognitiveperspectivealone,whatinformationthestudentneedsinordertoclarifyamisconception.Thisdeterminationisoftenmadebyapplyinganalgorithmtoincorrectanswersgivenoractionstakenbythestudent.Thestudentistheninformedofhis/herlackofunderstandingdirectlyandtoldwhatansweroractionwasexpected.Sometimes,dependingonthestudent'saffectivestate,s/herequiresdifferent,perhapslessdirect,feedback(Lepperetal.,1993).Basedontheirobservationsofexperthumantutorsandoninterviewswiththetutorsfollowingtheirtutoringsessions,theauthorsbelievethataconsiderationofcognitive,affective,andmotivationalissuesisessentialforsuccessfultutoring.Specically,inadditiontocognitiveconsiderations,thehumantutorsfocustheirattentiononbolsteringself-condence,maintainingchallenge,evokingcuriosity,andpromotingfeelingsofcontrol(Lepperetal.,1993).Theseareallstrongmo-tivationalfactors,asishighlightingtherelevanceoftopicsofstudytorealworldsituations(Alessi&Trollip,1991).Tutorsenhancethesenseofchallengebymodulatingobjectivetaskdifcultyaswellassubjectivetaskdifculty.Tutorsmodifyobjectivetaskdifcultybygivingeasiertoharderproblemsbasedonstudentunderstanding,byprovidingscaffold-ing(interveningtocorrectsteps),decreasingthesizeofstepstosuccess,andbyincreasingordecreasingthespecicityofhints.Theymodifysubjectivetaskdif-cultybyemphasizingthedifcultyoftheproblem,challengingthestudentdirectly,andengaginginplayfulcompetition(Lepperetal.,1993).Effectivetutorsbolsterself-condencebymaximizingsuccessandminimizingfailure.Theymaximizesuccessdirectlywithpraiseandexpressionsofcondenceandindirectlybyemphasizingproblemdifculty,studentagency,andengaginginplayfulcompetition.Theyminimizefailuredirectlywithreassuranceandcommis-erationandbyredeningsuccessaspartialsuccess.Theyminimizeitindirectlybyemphasizingproblemdifculty,makingexcusesforthestudent,andaskingques-tionsandprovidinghintsratherthanlabelingananswerincorrect(Lepperetal.,1993).27

PAGE 43

Althoughexperthumantutorsareabletoeffectivelydeterminethecombinedaffective,cognitive,andmotivationalstateoftheirstudents,thistaskmayprovequitedifcultforacomputer.Ifitisevenpossible,thecomputerwillhavetobaseitsappraisalonsubtlecuessimilartothosedetectedbythehumantutors–facialexpressions,slowingofresponsetimes,etc.Also,basedonthestudybyLepperetal.(1993),itisnotclearwhichofthemanytutorpracticesidentiedarethemostimportant,orevenclearlybenecial.Infact,somemayhavenegativesideeffectsthatarecounterbalancedbythepositiveeffectsofothers.Furthermore,whendependingontheuseofhumantutors,itisunlikelythattheywillallbeexperts.Somemayactuallyfosteranunhealthydependenceonthepartofthestudent.Fortunately,humantutorscanbetrainedtobemoreeffectivebydiscussingtutorialscenarioslikethosepresentedbyColdewayandSchiller(1974)toPSIproctorsintheirseminar.2.4.2GroupLectureandDiscussionBonwellandEisonstatethat,giventheassumptionthatthelecturerisknowl-edgeableandenthusiastic,thelecturemethodhassomeadvantages.Agoodlecturercanbeascholarlyrolemodel,canpresentnewmaterialthatisnotyetpublished,cancommunicatetheintrinsicvalueofthesubjectmatterinafashiondifferentfromotherformsofmedia,canorganizethepresentationtomeetstudents'needs,andcancommunicatealargeamountofinformationefciently.Further-more,lecturesaregenerallyconsideredcost-effective,becausetheyaredeliveredtomanystudentsatonce.Theyalsoprovideminimalthreattothestudent,be-causes/heisnotrequiredtoactivelyparticipate.Studentswhoenjoylearningbylisteningmaybestbeservedbythisteachingstrategy(Bonwell&Eison,1991).Thereare,however,substantialdisadvantagestousingthelecturemethod.Itisdifcultformoststudentstolisteneffectivelytoalecturerforsustainedperiodsoftime.Inonestudy,thepercentageofcontentrecordedbystudentswasanalyzed.Duringtherst15minutes,students'notesreected41%ofthecontentdelivered.Duringa30minuteperiod,only25%ofthecontentwasrecorded.During45minutes,only20%wasrecorded.Otherstudiesshowthatverylittleofthecontentdeliveredviathelecturemethodcanberecalledbystudents,unlesstheyhaveaboveaverageintelligenceandeducation(Bonwell&Eison,1991).Inananalysisof58studiesreportedbetweentheyearsof1928and1967,lecturesanddiscussionswerecompared.Althoughnosignicantdifferencewasfoundbetweentheabilitiesofthesetwomethodstoimpartthefactsandprinciples,discussionsweresuperiorinhelpingstudentsbuildtheirproblemsolvingskills.Thediscussionformatwasalsopreferredbythestudents.Thus,itwouldappearthatthelengthylectureis,infact,aninefcientdeliverymethodthatisnotpreferredbymoststudents(Bonwell&Eison,1991).28

PAGE 44

Still,itisapopularmethodemployedbyinstructors.Studiesindicatethatlec-turingwasthemainteachingstrategyusedintherecentpast.Inonesurvey(1980)offacultyon24campuses,between73%and83%oftherespondentsstatedthatlecturingwastheirprimarymethodofteaching.Inanother(1987),between61%and89%ofU.S.universityprofessorsreportedthattheyusedthelecturemethod(61%inhumanities,81%insocialsciences,and89%inphysicalsciencesandmathematics)(Bonwell&Eison,1991).AmorerecentsurveybyMcEwen(1996)showsanapparenttrendtowardusingalternativeteachingmethodstoteachsoftwareskillstobusinessstudents.Thisndingisbasedontheresponsesof167businesseducatorsinveMidwesternstatesteachingatalleducationallevels(two-thirdsatthehighschoollevel),79%ofwhomhadmorethan20yearsofteachingexperience.Respondentsreportedthat,whenteachingcomputerapplications,themosteffectiveinstructorsmakeuseofdemonstrations,simulations,andself-pacedlearning.2.4.3ActiveLearningBonwellandEisondeneactivelearningasanythingthat“involvesstudentsindoingthingsandthinkingaboutthethingstheyaredoing”(Bonwell&Eison,1991,p.2).Theydescribeclassroomactivitiesasexistingsomewherealongacontinuumbetweenpassiveandactivelearning.Commoncharacteristicsofactivelearningstrategiesfoundintheliteratureincludethefollowing.Studentsdomorethanjustlisten(i.e.,read,write,discuss).Thereislessemphasisonsimplytransmittinginformation.Thereismoreemphasisondevelopingstudents'skillsandonexplorationofpersonalattitudesandvalues.Higherorderthinkingisstimulated(i.e.,analysis,synthesis,evaluation).Althoughresearchshowsthatactivelearningiscomparabletolecturinginpro-motingmasteryofcontent,itissuperiortolecturinginpromotingthedevelopmentofstudentskillssuchasthinkingandwriting(Bonwell&Eison,1991).Suchthink-ingandproblem-solvingskillsareessentialinmanysciencecourses.Furthermore,inseveralstudies,studentshavereportedthattheypreferactivelearningstrate-gies,andcognitivestudiesrevealthatthelearningstylesofasignicantnumberofstudentsarebestservedbylearningmethodsotherthanlecturing(Bonwell&Eison,1991).29

PAGE 45

2.4.4CollaborativeandCooperativeLearningCollaborativeenvironmentsofferseveraladvantagesoverthetraditionallectureformat.Asmentionedabove,discussionswereshowntobesuperiorinpromotingproblemsolvingskillsandwerepreferredbystudents(Bonwell&Eison,1991).Inaddition,inacollaborativeenvironment,contentisviewedfrommanyperspectives,learnersaremotivatedbythepresenceofotherswhoarestrugglingtomasterthesamematerial,andthetrepidationoftenassociatedwithlearningskillsthatrequirehigherorderthinkingismitigated(Preeceetal.,1994).Inadiscussion,bothreceiv-ingandtransmittingstudentsbenetfromselfexplanationinthatonereceivesaneededexplanation,whiletheotherdeepenshisorherunderstandingbyverbaliz-ingandsynthesizingideastogether.Studentsalsobenetfromappropriationwhentheylearnbywatchingamoreskilledclassmatework.Apprenticeshipisoneformofappropriation.Finally,studentsareabletointernalizeinformationwhen“ver-balizing[it]inaconversation”(Hiltz,1997).Althoughcollaborativeenvironmentshaveseveraladvantages,onedisadvantageisthatonelearnerinagroupmaycompletelydominateitandbetheonlyonetobenet(Preeceetal.,1994).Also,itisdifculttoknowwhichstudentsactuallycontributedtocollaborativeprojectsinsubstantiveways,makingitdifculttoassignindividualsgrades.RecallthatALNsfostercollaboration.Theydothisbyproviding“asynchronousaccesstoremotelearningresources”(Mayadas,1994),includingpeersandex-pertssuchastutorsandfacultyaswellaslibraries,softwaregeneratedsimula-tions,laboratoriesatadistance,andworkproductscreatedbyremotecollabora-tors.AccordingtoMayadas,theProgramOfceroftheSloanFoundation,amajorgoalofALNsistode-emphasizelectures,whileemphasizinginteraction.The“keycomponentsofALNtechnology...existmainlytolinkpeopletootherpeople,andtoprovideaframeworkfortheirinteraction”(Mayadas,1994).2.5SummaryThedesignofboththestudyandthecoursewarewereinformedbytheliteraturediscussedabove.Indesigningthecourseware,anattemptwasmadetoincludeallfactorsidentiedintheliteratureascriticalforsuccessfulself-pacingthatcouldfeasiblybeincludedatthistime.Forexample,TAswereshownhowtonavigatethroughthecoursewareandhowtousethegradingrubricsfortheassignments.Anintroductorysectionofthecoursewaretoldstudentshowtonavigatethroughthecoursewareandadvisedthemofalternativemethodsofprogressingthroughthecontent.ThecoursewarewassuppliedtostudentsonCD-ROM,whichshouldhavebeenreasonableforthepopulationinvestigated,asstudentswereenrolledinacoursetoacquirecomputerskillsandtolearnaboutapplyingeducationaltech-30

PAGE 46

nology.Student-studentandstudent-teachercommunicationswereencouragedexplicitly,andaformfore-mailingquestionsdirectlytotheTAsaswellaslinkstotheirWebsiteswereprovidedinthecourseware.TAssuppliedfeedbacktostu-dentsonaweeklybasisastheygradedassignmentsthatweredueeachweek.Inaddition,TAsdidnotneedtodevotetimetodeliveringcoursecontent,andinstead,wereabletousetheirtimetutoringstudentsandaddressingtheircognitive,affec-tive,andmotivationalneeds.Bothaudioandvisualelementswereutilized,andusersweresuppliedwithaneasymethodofprovidingfeedback.Thecoursewarewasprogrammedwithmodularityinmind,makingitrelativelyeasytoaddnewlessonsandtochangeassignmentstomeetfuturecourseneeds.Finally,lessonswerekeptshortandactive,theywerebasedonataskanalysisofthetargetprod-uct,andassignmentswereconsideredrelevant,guidingeachstudentthroughtheprocessofdevelopingapersonalWebsite.Thestudyitselfwasinformedbyreportsonself-pacing,distancelearning,theuseofdeadlinecontingencies,andtheshorttermandlongtermeffectsofmassedversusdistributedpractice.NewdistancelearningparadigmssuchasALNsarepushingeducationtobecomemorelearner-centeredandtoincorporatesuchele-mentsasself-pacing.Although,self-pacinghasbeenassociatedwithclearadvan-tageswithrespecttoacademicachievementandlearnersatisfaction,italsohasthewell-documenteddisadvantagethatstudentstendtoprocrastinate.Researchshowsthatprocrastinationcanbecurbedbyusingdeadlinecontingencieswith-outdetrimenttoachievementorsatisfaction.Anaddedbonusmaybeenhancedcontentretentionforlearnerswhodonotprocrastinate.Itishypothesizedthatthecurrentinvestigationwillreplicatethendingsofotherresearcherswithrespecttoimmediateachievement,satisfaction,andprocrastinationlevel.Itisfurtherhypoth-esizedthatreducingprocrastinationbyusingdeadlinecontingencieswillincreaseretentionscores.Suchanoutcomewouldsupportthetheorythatstudentswhoprocrastinatelessandspacelearningoutoveralongerperiodoftimehavebetterrecalloftheinformationlearned.31

PAGE 47

Chapter3MethodStudentsfromanundergraduateclassininstructionaltechnologyatalarge,southeasternuniversitywererandomlyassignedtooneofthreegroups.Hence-forth,theclasswillbereferredtoasEdTech.Althoughallstudentsweregiventhesamesetofeightassignmentdeadlines,theconsequencesfornotmeetingthesedeadlinesdifferedbygroup.Studentswithrecommendeddeadlines(R)wereen-couraged,butnotrequired,tomeetthem.However,theywererequiredtosubmitallworkbythenalassignmentdeadline,asnoworkwasacceptedforcreditfromanygroupafterthatdeadline.Studentswithconditionaldeadlines(C)receivedbonuspointsforsubmittingassignmentsearlyandlostpointsforsubmittingthemlate.Studentswithabsolutedeadlines(A)receivednocreditforassignmentssub-mittedaftertheposteddeadlines.CoursewareonHTMLwassuppliedtoeachstudentonaCD-ROMatthestartofthestudy.Eachstudentreceivedoneofthreestudentversionsofthecourse-ware.Eachversioncontainedlessons,assignments,andreferencematerialonHTML.Theonlydifferencebetweenversionswasthespecieddeadlinecontin-gency.Studentswerereassuredthatiftherewasasignicantdifferencebetweengroups,thescoresofstudentsinthedisadvantagedgroup(s)wouldbeadjusted.Pretest,posttest,andretentiontestdataweretobeanalyzedusingrepeatedmeasures,butarandomizationmodelwasadoptedlaterduetotheabnormalna-tureofthedata.Itwasanticipatedthatnosignicantdifferencewouldbefoundbetweenanyofthegroupsonthepretestandtheposttest.However,studentsingroupsCandAwereexpectedtoperformsignicantlybetterontheretentiontestthanstudentsingroupR.Thus,asignicantinteractionbetweentime(posttesttoretentiontest)andtreatmentwasexpected.PacingpreferencedatacollectedwiththeposttestwereanalyzedusingANOVA.Nosignicantdifferencewasexpectedbetweengroups.Dataonstudentprocrastinationwerecollectedintermsofwhenassignmentsweresubmittedrelativetoduedatesandthenumberofrequestsfordeadlineextensions.AlthoughthesedataweretobeanalyzedusingMANOVA,withtheexpectationthatstudentsingroupsCandAwouldrequestsignicantlymoredeadlineextensions,whilestudentsingroupRwouldsubmitassignmentssignicantlylater,thenumberofrequestswastoosparsetojustifyMANOVA.Inaddition,theplatykurticnatureofthesubmissiondatasuggestedthatarandom-32

PAGE 48

izationmodelwouldbemorepowerfulthanANOVA,andthus,theassignmentsubmissiondateswereanalyzedaloneusingrandomizationtests.AllinstrumentsweredesignedspecicallyforthisstudyandincludedthreemeasuresofHTMLachievementaswellasmeasuresofpacingpreference,collaborationlevel,andstudygroupmembership.3.1TimeTableThestudywasconductedduringSpring2000accordingtotheschedulebelow.Jan14.DiscussedstudywithanddemonstratedcoursewaretoTAsFeb3.AdministeredstudygroupssurveytostudentsFeb11.TrainedTAsonhowtousegradingrubricsFeb22.AdministeredpretestanddistributedcoursewareFeb25.Assignments1and2weredueMar3.Assignments3and4weredueMar10.Assignments5and6weredueMar13.WeekofSpringBreakMar24.Assignments7and8weredueMar28.Administeredposttest,andpacingpreference/collaborationsurveyApr27.AdministeredretentiontestTheTAswereinformedofthegoalsofthestudyandtookaninitiallookatthecoursewareinatwohourmeetingonJanuary14.Changestheyrecommendedwereincorporatedintothecourseware,includingprovidingthestudentswithalter-nativeinstructionsintheeventthatthecoursewaredidnotstartautomaticallywheninsertedintoaCD-ROMdriveandprovidingeachTAwithalistofhisorherstu-dentsandtheirrespectivetreatmentgroups.ThedifferencesbetweentreatmentswasexplainedandtheTAsweretoldthatitwasnotclearwhichgroup(s)wouldhavetheadvantageinthelongrun.Theywerereassuredthatifsignicantdiffer-enceswerefound,thegradesforstudentsinthedisadvantagedgroup(s)wouldbecurvedupward.InanothertwohoursessiononFebruary11theTAsweretrainedonhowtousethegradingsoftwareandrubrics.Itwashopedthat,priortothismeeting,theteacherversionsofthecoursewarewouldbeavailableforthemtotakehomeandinvestigate.Unfortunately,theteacherversionswerenotreadyfordistributionduringthattime.TheTAsreceivedtheircopiesofthecoursewareonthesamedaythatthestudentsreceivedtheircopies.ItwasassumedthattheTAs,beingsomeofthetopstudentsfrompreviousEdTechcourses,wouldbeabletocompletetheassignmentsonestepaheadoftheirstudents.Theyindicatedthattheyfeltthis33

PAGE 49

wasthecaseanddidnotrequireanothertrainingsession.Laterinthestudy,whenanadditionaltrainingsessionwasofferedagain,theTAsagainindicatedthattheydidnotrequiresuchinstruction.Presumablytheyfeltthattheaccesstheyhadtothegradingrubricsandassignmentsolutionswassufcient.Theresultsofasurveyonstudygroups,giveninclassonFebruary3,guidedtherandomassignmentofstudentstotreatments.Thisrandomizationisdescribedindetailinsection3.2.1.ThepretestwasgiveninclassonFebruary22.FreeCD-ROMsalsoweredistributedtothestudentsatthattime.AssignmentsweredueonFridays:February25,March3,March10,andMarch24.Theposttestandsurveyonpacingpreference/collaborationlevelweregiveninclassonMarch28.TheretentiontestwasincorporatedintoExam2forEdTech,whichwasgivenonApril27.3.2SampleAllstudentsinterestedinenteringtheCollegeofEducationattheuniversityfromwhichthesamplewasdrawnmusttakeEdTechbeforebeingadmittedtothecollege.Thecourseintroducesstudentstoinstructionaltoolsmadepossiblebyadvancesintechnology.Amongotherthings,thestudentslearnHTML.ThisstudyfocusedontheHTMLportionoftheclassofferedinSpring2000andinvestigatedthesuccessoftheparticipantsinlearningitfromthecoursewareprovided.Allstudentsmetatthesametimetwiceperweekinamasslecturewithoneoftwoleadinstructorstakingturnsinstructingthem.FiveTAsassistedtheleadinstructorsbyinteractingwiththesamesetofroughly40studentseachweek.TheTAs'dutiesincludedgradingcourseworkandhelpingstudentsunderstandcoursematerials.ForeachsetofstudentsassignedtoaparticularTA,one-thirdreceivedtreatmentR,anotherthirdreceivedtreatmentC,andtheremainingthirdreceivedtreatmentA.3.2.1AssignmenttoTreatmentsAsinpastsemesters,studentswereassignedalphabeticallytoTAsinroughlyequalnumbers.Duetothefactthatsomestudentsdroppedthecourseandsomeaddeditafterthisinitialassignmenttookplace,theTAsendedupwithslightlydifferentnumbersofstudentsattheendofthedrop/addperiod.OneTAhad46students,onehad41,onehad35,andtwomorehad37each.Suchdifferencesingroupsizeswerenotexpectedtoaffectthestatisticalmethodsemployed,becauseanalysiswasbasedonassignmenttotreatmentconditions,nottoTAs.34

PAGE 50

Ofthe195preserviceteachersinitiallyenrolledinEdTech,6ofciallydropped1thecourseafterthedrop/addperiodbutpriortothebeginningofthetreatmentinterval.Thesestudentswereexcludedfromalldataanalyses.Theremaining189studentswereassignedrandomlytotreatmentconditionsinthefollowingmanner.First,responsestothesurveyinAppendixO,giveninclasspriortothetreatmentinterval,revealedthatsomestudentsintheclasshadestablishedstudygroups.Itwasdeemednecessarytoaveragethescoresofthemembersofeachself-madestudygroupandtotreateachgroupasasingledatapoint,becauseANOVA,MANOVA,andrepeatedmeasures2allrequirethatobservationsbeindependentandarenotrobusttoviolationsofthisassumption.Furthermore,themembersofeachself-madegroupwereassignedtothesametreatment,sothataverageswerenottakenacrosstreatments.Afterallself-madegroupswereassignedrandomlytooneofthreetreatments,theremainingstudentswereassignedrandomly3,sothateachTAhadroughlyequalnumbersofstudentsineachtreatment.Ofthe19studygroupsidentiedpriortothetreatmentinterval,7wereassignedtoR,6toC,and6toA.Thegroupswerecomprisedof47totalstudentsandeachhadbetween2and6members,withamodegroupsizeof2.Randomassignmentbasedongroup,resultedin18individualsbeingassignedtogroupR,14toC,and15toA.Unfortunately,responsestothesurveyinAppendixPindicatedthatstu-dentsdidnotmaintaintheirstudygroupswhenworkingontheHTMLassignments.Infact,somenewstudygroupsformed.Only7ofthenewgroupsappearedsta-bleinthesensethatallmembersreportedworkingwitheachothermember.All7groupswerecomprisedof2memberseach,4werecontainedwithinasingletreatment4and3spannedtreatments.Also,manystudents'reportsdidnotagreewithoneanother.Onestudentwouldlistaparticularpartner,whilethatpartnerwouldlistnooneorperhapsevenadifferentthirdperson.Thus,itappearedthatmembershipinself-madestudygroupswasmoreuidthananticipated.Ofthe19initialgroups,only8maintainedtheirintegrity5.However,becauserandomassignmentwasbasedontheinitial19groupsandthe“justicationfor[therandomization]approachisclearlystrongestinanexperimentalsituation”(Manly, 1Thesesixstudentsinitiallyhadstudyidenticationnumbers114,135,189,241,273,and277.FourwereassignedtoA,onetoC,andonetoR.2Thedecisiontoanalyzethedatausingarandomizationmodelwasnotmadeuntillater,afteralldatawerecollected.3Threestudents(#292,#293,#294),whohadthesameTAandaddedtheclasslatebutbeforethetreatmentinterval,wereassignedrandomly,onetoeachtreatment,aftertheinitialgroupofindividualswasassigned.4MembersoftwogroupswereinR,membersofonewereinC,andmembersoftheotherwereinA.5Theeightstablegroupswerecomprisedof17individualswithamodeof2memberspergroup.35

PAGE 51

1997,p.22),thesegroupingswerekeptforanalysisoftheachievementandpro-crastinationdata,with19averagescoresreplacingthe47individualscores.Fol-lowingasimilarargument,averageswerenottakenforthemembersofgroupswhichemergedduringthetreatmentinterval,becausethesegroupsdidnotinformtheinitialrandomassignmentofparticipantstotreatments.However,becausethepreferencedatawereanalyzedusingANOVA,whichisalsostrongestinanexperi-mentalsituationbutfurtherrequiresthatobservationsbeindependent,itwasmoreappropriatetoconsideraveragesforthestable,emergentgroupsaswellasfortheinitial19groups.First,basedonresponsestothesurveyinAppendixP,student#174wasaddedtogroup14,sothat48ratherthan47individualscoreswerere-placedwith19averages.Second,4averageswereusedforthe8studentsinthestable,emergentgroupswherebothmembershadthesametreatment.Becauseonly6studentswereinstable,emergentstudygroupswhichspannedtreatments,itwasexpectedthattheirdependencewouldnothaveasignicantimpactonanal-ysis.Attheendofthecourse,datarevealedthatthreestudentshadagradeof0%inEdTech(#136,#246,#249),presumablyhavingnevershowedupfortheclass.Oneofthesethree,#249,actuallydiddroptheclassofciallyduringthetreatmentinterval.Onemorestudent,#158,droppedduringthetreatmentinterval,butcom-pletednoHTMLassignmentsandnostudyrelatedmeasures.Infact,aswasthecasefortheotherthreestudents,thisstudentlikelyneverreceivedthecourse-ware,becausenopretestwastaken.Itwaspresumedthatallofthesestudentsactuallyhadstoppedattendingclasspriortothestartofthestudy.Fourmorestu-dentsexperiencedtechnicaldifcultybeyondtheircontrolduringthestudy.Onestudent'snameincludedpunctuation(#122),whichcausedthescriptforsubmit-tingassignmentselectronicallytomalfunction.Thestudentwasnotabletosubmitseveralassignmentsduringinitialattemptsduetothislimitationinthesubmissionprocedure.AnothertechnicalproblemwhichoccurredduringproductionoftheCD-ROMscausedthreeunreadableCD-ROMsinarowtobeproduced.Theaffectedstudents(#221,#222,#224)wereinalphabeticalorderandallingroupA.EventhoughallfourstudentswhoexperiencedtechnicaldifcultywereingroupA,itwaspresumedthatthesedifcultieswerenotsystematic.ItwasequallylikelythatastudentinRorCmighthavehadanamecontainingpunctuationandthattheun-readableCD-ROMsmighthavebeencreatedduringtheproductionofCD-ROMsforstudentsinRorC.Therefore,datafortheseeightstudentswereremovedfromanalysiswiththeexpectationthatrandomizationwasnotadverselyaffected.Insummary,afterremoving6studentswhodroppedEdTechbeforethestudybegan,3studentswhopresumablyneverattendedEdTechalthoughtheywereofciallyenrolled,1studentwhoappearedtohavedroppedunofciallypriortothestartofthestudyandofciallyafteritstarted,and4studentswhoexperiencedtechnicaldifcultiesduringthestudy,theinitial195participantswasreducedto181with64inR,60inC,and57inA.However,recallthatforthepreference36

PAGE 52

data,23averagesweregeneratedfrom56individualsinself-madestudygroups,yielding148totaldatapoints,51ofwhichwereassignedtoR,50toC,and47toA.Fortheachievementandprocrastinationdata,19averagesweregeneratedfrom47individualsinself-madestudygroups,yielding153totaldatapoints,53ofwhichwereassignedtoR,52toC,and48toA.ThreesetsofCD-ROMsweremade,oneforeachtypeofdeadline.EachCD-ROMinthesetthatcontainedabsolutedeadlineswaslabeled1.2aontheactualCD-ROMandontheinitialsplashpageofthecoursewareaswellasonthebottomofthemainnavigationbarontheleft(seeFigure29).Likewise,thesetscontainingconditionalandrecommendeddeadlineswerelabeled1.2cand1.2r,respectively.TheselabelswereputinplacesothatTAscouldquicklyidentifystudents'treat-mentconditionswhenansweringquestionsinthecomputerlaboratory.Ifastudentaskedaquestionaboutdeadlines,itwasexpectedthatthisinformationwouldbehelpfulinguidingtheTA'sresponse.Italsohelpedtoensurethateachstudentre-ceivedthecorrectversionoftheCD-ROM.Inaddition,aslipofpaperwasincludedwitheachCD-ROMindicatingpreciselywhichstudentshouldreceiveitaswellasthefour-digitpasswordthestudentwouldneedtosubmitassignmentsonline.3.2.2MissingDataSeveralstudentsweremissingdata,with3missingthepretest,34missingtheposttestandpacingpreferencesurvey,and23missingtheretentiontest.Infact,20weremissingboththeposttestandtheretentiontest.Ofthese20,10ofciallydroppedthecourseafterthestudybeganbutonorbeforeMarch10,whichwasthelastdaytodroporwithdrawfromcoursesattheUniversitywithoutacademicpenalty.Thisoccurredattheendofthethirdweekoftheveweektreatmentinterval.All10droppedEdTechwithinaneightdayperiod.Theother10studentswhoweremissingboththeposttestandretentiontest,didnotofciallydropthecourse.However,theirscoresonEdTech'sExam1,takenonthethirddayofthestudy,andtheirnalclassstandings,gavesupportingevidencethatatleastsomemayhavedroppedunofcially(seeTable3).WhiletheirExam1scoresappearedroughlyequivalent,theirnalclassstandingsweremuchmorediverse.Itwaslikelythatsomechosetodrop,butjustmissedtheUniversity'snaldropdate.Inanycase,itwasnotclearthattheirassignmentinformationwascomplete.Thus,therewasnoreasonablewaytoestimatemissingscoresforthese20individuals,andtheirdatawereremovedfromanalysis.Inordertoguagetheimpactofremovingthisdata,rstnoticethatequalnum-bersofstudentswereremovedfromeachtreatment.Next,considerthecollectiveExam1scoresandnalgradesrecordedinEdTechforthesestudents,whichweresignicantlycorrelatedwithposttestandretentiontestscores(seeTable6).Rela-tivelyspeaking,theiraveragescoresontheposttestandretentiontestlikelywould37

PAGE 53

Table3.GradesforStudentsMissingPosttestandRetentionTest Exam1FinalGrade TreatmentnMMinMaxMMinMax StudentswhodroppedR348.673063122.0081166C343.673550117.6798136A455.504762165.25139180 StudentswhodidnotdropR347.334053123.3398142C450.253961209.0090375A350.004754104.0087135 All20studentsR648.003063122.6781166C747.433561169.8690375A753.144762139.0087180 havebeensimilartotheirscoresonExam1andtheirnalgrades.AlthoughtheExam1scoreswerenotcorrelatedashighlywiththestudyachievementscoresasthenalgrades,theyalsowerenotaffectedbythedatethestudent(un)ofciallydroppedthecourse.Therefore,theyprovidedthebestevidenceofanypossibledifferentialeffects.BecausestudentsinA,onaverage,earnedhalfagradehigherthanstudentsinbothRandC,themostlikelyeffectwasthatthemeanposttestandretentiontestscoresforstudentsinAwerealittlelowerrelativetostudentsinRandCthantheywouldhavebeenhadthese20datapointsbeenretained.Still,thestudentsdidnotappeartoberelatedinanysystematicway,andbecausetheamountofmissingdatawasdistributedequallyacrossgroups,itwasexpectedthattheremovalofthescoresofthesestudentswouldnotdrasticallyaltertheresultsofanalyzingtheachievementdata.Furthermore,withlowcorrelationsbetweenthepretestandtheposttest(r.06)andthepretestandretentiontest(r.10)andwithincompleteassignmentdata,therewasnoreasonablewaytoestimatethemissingvalues.Besidesthe20studentsmissingboththeposttestandtheretentiontest,therewere14morewhoweremissingtheposttestonlyand3morewhoweremissingonlytheretentiontest.Inaddition,onestudentinAreportedamissingposttestscoreonthedaytheretentiontestwasgiven.Thatstudentwasallowedtoretaketheposttestthefollowingweek.Becausethisstudenttooktheposttestaftertheretentiontest,theactualrecordedposttestscorewasnotusedduringdataanaly-sis,bringingthetotalnumberofstudentsmissingtheposttestonlyto15.Ofthese38

PAGE 54

15,8wereinR,2wereinC,and5wereinA.Becausethesemissingdatawerenotdistributedequallyacrossgroupsandbecausetheposttestandretentiontestweresignicantlycorrelated(r.83,p.0001),missingposttestscoreswereestimatedusingretentiontestscoresandviceversa.Simplelinearregressionwasperformedontheposttestandretentiontestscoresofthe143studentswhohadcompleterecordsforbothofthesemeasures,gener-atingthepredictionequationp093563r11169201222269ewherepistheestimatedposttestscore,ristheretentiontestscore,andeisarandomvariablefromanormaldistributionwithameanof0andastandarddeviationof1.Thelattertermwasincludedsothatthedistributionofnoiseintheestimatedmissingvalueswouldbesimilartothatfoundduringregressionanalysisontheavailabledata.Thus,itwasnecessarytomultiplyebythestandarderroroftheestimateassociatedwiththeregressionequation,orby12.22269.Aswasthecasefortheposttestscores,themissingretentiontestscoreswerenotequallydistributedacrosstreatments.TwostudentswereinAandonewasinR.Therefore,regressionanalysiswasusedtogeneratetheequationr077697p1514571113819ewhereristheestimatedretentiontestscore,pistheposttestscore,andeisarandomvariableinN(0,1).Thesenaltworegressionequationsaregraphedovertheirrespectivescat-terplotsinFigures1and2.Notethatbothequationsaccountedfor73%ofthevariabilityinthedependentvariables.Whilethetypicalpredictionerrorsfortheequations(12.22and11.14,respectively)indicatethatpredictions,onaverage,wereinaccuratebyalettergrade,theywerestillmoreaccuratethantheywouldhavebeenifthemeanwereused.Thissameargumentisassumedtoholdeventhoughthereliabilitiesoftheposttest(r.89)andtheretentiontest(r.85)implythattheregressioncoefcientswereunderestimatedsomewhat(Pedhazur,1997).Toreviewallvaluesimputedviaregression,seetheboxedposttestandretentiontestscoresinTable25.Analobservationregardingtheposttestisworthmentioning.Notethatonestudent,#187,respondedwiththeanswerCforeveryquestion(seeTable22).Nor-mally,thisstudent'sresponseswouldberemoved.However,becausethecoursematerialdidappeartobemorechallengingforthestudentsthananticipated,withasubsetofstudentsfromeachgroupearningscoresontheachievementmeasures39

PAGE 55

0 20 40 60 80 100 0 20 40 60 80 100 p = 0.93563r + 11.16920Retention Test Score (percentage)Posttest Score (percentage) Figure1.Predictingposttestscorefromretentiontestscore. 0 20 40 60 80 100 0 20 40 60 80 100 r = .77697p + 1.51457Posttest Score (percentage)Retention Test Score (percentage) Figure2.Predictingretentiontestscorefromposttestscore.40

PAGE 56

thatonewouldexpectbychance,thisstudent'sresponseswerekept.Presumably,somenumberofstudentsineachgroupguessedforallquestions.Student#187wasjusttheonlyonethatdidsoinsuchasystematicmanner.Onlythreestudentsweremissingthepretest,butaccordingtoTable6,themultiplechoiceportionofthetesthadlowreliabilitywith.01andwasnothighlycorrelatedwithanyothermeasures.Therefore,thebestestimateforthemissingscoreswastheaveragepretestscoreof27.15andtheaverageself-reportresponsesonitems10and11of1.5and1.34,respectively.Althoughitwasnotincludedinformaldataanalysisduetoitsextremelylowreliability,thepretestdidprovidesomeevidencethatinitialrandomizationproducedthreesimilargroupswithrespecttopriorknowledgeofHTML,withmeanscoresof27%,29%and25%forR,C,andA,respectively.Infact,randomizationtests6basedonthepretestmultiplechoiceitems(p.2797)andself-reportitems10(p.2781)and11(p.1268)showednoinitialdifferencebetweengroups.Itshouldbenotedthatsomestudentsweremissingindividualitemresponsesonthepretest.Theirscant-ronsandtestformswereinspected,andinthreecases,theresponseswereeasilycorrected(seefootnotesforTable22).Inanothervecases,missingresponsesformultiplechoiceitemsweretreatedasincorrectresponses.Finally,eightmissingself-reportitemswerereplacedwiththeaverageresponsesacrossalldatapoints.Specically,1.5wasrecordedforonestudentwhodidnotansweritem10aboutpriorexperiencewithtypesettinglanguages,and1.34wasrecordedforsevenstu-dentswhodidnotansweritem11aboutpriorexperiencewithprogramming.Nowconsidermissingpreferencedata.Onestudentneglectedtoanswerques-tion8,soavalueof3forNoPreferencewasentered.Thesame35studentswhoweremissingtheposttestalsohadnopacingpreferencescores.The20whoalsoweremissingtheretentiontestandwhosedatawerenotincludedintheanalysisoftheachievementdatawereremovedfromconsiderationhereaswell.Fortheremaining15,theaveragepacingpreferencescoreof2.79(N146)wasused,becauseitwasthebestavailableestimateforthemissingvalues.Inthecaseofamissingassignmentscore,itwasassumedthatthestudentdidnotsubmittheassignmentandazeroscorewasentered.Recallthat,tode-terminethelevelofprocrastination,thesubmissiondateforeachassignmentwas 6Thecomparisonteststatisticselectedforthemultiplechoiceitemswasthetwo-sidedomnibustest PREr PREc2 PREr PREa2 PREc PREa2,where PREgistheaveragepretestscoreforgroupgandgrca.Twosimilartwo-sidedomnibustestswereusedtoanalyzethetwoself-reportitemsindividually.Inallcases,estimateswerebasedonpermutationdistributionsofsize1000000.RandomizationtestswerepreferabletoANOVAs,becausethemultiplechoiceitemsforgroupAwereplatykurtic(seeTable11),andtheself-reportitemsgenerallywerepositivelyskewedandleptokurtic(seeTable7).Seesection4.3fordetailsaboutemployingrandomizationtests.41

PAGE 57

Table4.SampleSizesasDistributedAcrossTAs TA Treatmentna12345 AllindividualsR58151113109C5312912911A5011111189 AchievementandProcrastinationdatagroupsbR4814111289C4712911811A411181189 PreferencedatagroupsbcR4614111289C4512811811A401181089 aNumberofindividualsanddatapointsconsidered.Be-causestudygroupmembersmighthavehaddifferentTAs,studygroupsweresometimesincludedinmorethanoneoftheTAtotals.Therefore,rowtotalsdonotnecessarilysumtothisvalue.bCountsincludeindividualsandstudygroups,suchthateachstudygroupwascountedonlyonceforagivenTA,regardlessofhowmanymembershadthatTA.cAstudygroupwasincludedinthecountforagivenTA,ifanyofitsmembershadthatTA.Hence,emergentgroupsthathadmemberswithdifferentTAswereincludedinthecountsofmorethanoneTA.recorded.Forthosestudentswhodidnotsubmitanassignment,thedayafterthestudyendedwasenteredasthesubmissiondate.Insummary,afterhandlingmissingdata,161individualsremainedwith58inR,53inC,and50inA.Afteraveragingthescoresforindividualsinself-madestudygroupsfortheachievementandprocrastinationdata,atotalof136datapointswereavailableforanalysis,with48inR,47inC,and41inA.Afterincludingemergentgroupaveragesforthepreferencedata,atotalof131datapointswereavailable,with46inR,45inC,and40inA.Table4listshowmanyofthe161individualsandthe136or131datapointswereassignedtoeachtreatment/TAcombination.Noticethat,foragivenTA,thenumberofobservationsineachtreatmentwasroughlythesame,beforeandaftertakingstudygroupaverages.42

PAGE 58

3.2.3EffectSizeandPowerGivenreportsbyresearcherswhohavecompareddeadlinecontingenciesun-derconditionsofPSI,nosignicantdifferencewasanticipatedinimmediateachieve-mentscoresontheposttest.Basedoneffectsizesreportedforretentiontestscoresofstudentswhoengagedinmassedpracticeversusdistributedpractice,alargeeffectsizewasanticipatedforretentiontestscores.Usingformulaf k1FN,wherekisthenumberofgroups,FisthereportedF-value,andNisthetotalsamplesize(Stevens,1990),inonestudytheeffectsizecameouttobef.37(Bloom&Shuell,1981),andinanotheritcameouttobef.65(Grote,1995)7.Inbothcases,distributedpracticewassuperiortomassedprac-tice.Therefore,alargeeffectsizewasexpectedfortheretentiontestscoresearnedbystudentsingroupsCandAascomparedtothoseearnedbystudentsingroupR,whowereexpectedtoprocrastinatemoreandtomasstheirlearningnearthenalduedate.Recallthattheachievementdataweretobeanalyzedusingrepeatedmea-sures,butthatarandomizationapproachwasadoptedinsteadduetotheabnor-malnatureofthedata.AccordingtoManly(1997),“itisreasonabletoexpectthatthepowerofrandomizationandclassicaltestsshouldbeaboutthesamewhentheassumptions[fortheclassicaltests]aretrue....Whendataarefromnon-standarddistributions,thereissomeevidencetosuggestthatrandomizationtestshavemorepowerthanclassicaltests”(p.80).Randomizationwasusedtocom-parethedifferencescoresgeneratedbysubtractingtheretentiontestscoresfromtheposttestscores.Theaveragedifferencescoresforthethreegroupsofinterestwerecomparedbysubtractingthemfromoneanother.Therefore,thetestlikelyhadatleastthesamepowerasANOVAwithtwodegreesoffreedom(3-1),andpossiblymorepower.Givenanaveragegroupsizeof45and.05,theFtestwouldhavehada73%chanceofdetectingjustamedium-sizedeffect(f.25)forthedifferencescores.Foralargeeffectsize(f.40),itwouldhavehada99%chance(Stevens,1990).However,itonlywouldhavehadabouta16%chanceofdetectingasmalleffect.Liketheachievementdata,theabnormalnatureoftheprocrastinationdatapromptedtheuseoftherandomizationapproach,andpowerissuesaresimilartothosejustdiscussed.Consideringthepacingpreferencedatawithanaveragegroupsizeof44and.05,thisstudyagainhadsomepowertodetectamedium(73%)orlarge(99%)effect,giventhataone-wayANOVAwasemployed.Insummary,therepresumablywereenoughdatapointstoprovidefairlyreasonablepowerofdetectingeitheramediumoralargeeffectofthetreatmentconditionontheachievement,pacingpreference,andprocrastinationdata,butnotasmalleffect. 7ForBloomandShuell(1981),thecalculationwas 7 12!52.ForGrote(1995),itwas 15 06!36.43

PAGE 59

3.3TreatmentsAsmentionedpreviously,thestudentsweresuppliedwithoneofthreeversions,1.2r,1.2c,or1.2a,ofthecoursewareonCD-ROM.Version1.1wascreatedandusedduringapilotstudyconductedinFall1999.ModicationsinspiredbythepilotstudywereincorporatedintoVersion1.2.SeeAppendixAformoreinformationonthepilotstudy.ForadetaileddescriptionofVersion1.2,includingchangesmadebasedonthepilotstudy,seeAppendixC.Foradiscussionofthecoursewarever-sionssuppliedontheCD-ROMaccompanyingthisdissertation,seeAppendixU.Theonlydifferencebetweenversions1.2r,1.2c,and1.2awasthatthedead-linessuppliedwerespeciedasrecommended(R),conditional(C),orabsolute(A).Allgroupsreceivedalistofthesamedeadlines.GroupRwastoldthatthedeadlinesweremerelyrecommendedandthatallassignmentsweredueabso-lutelynolaterthanMarch24.Fortheexactwording,seeFigures3and4.GroupCreceivedfurtherinformationondatesbywhichassignmentscouldbesubmittedforbonuspointsorwithpenaltypointsdeducted(seeFigures5and6).GroupAwastoldthatabsolutelynoassignmentcouldbesubmittedafteritslistedduedate(seeFigures7and8).Allstudentswereinformedthattheywerepartofastudyandthatthedead-linecontingencieswereslightlydifferentforthedifferentgroups.Becausetheywerelikelytodiscoverthisinformationontheirownandpossiblybeconcernedaboutit,andbecauseknowledgeofthisinformationwasnotexpectedtoaffecttheiractions8,itseemedadvisabletosupplythemwiththisinformationsystemati-cally.Theywerereassuredthatitwasnotclearwhichgroup,ifany,wouldhavetheadvantageinthelongrun.Theywereinformedthat,intheeventthatonegroupdidhaveaclearadvantage,scoresforthisportionofEdTechwouldbecurvedupwardforthedisadvantagedgroup(s),sothatthemeansforeachgroupwerecompara-ble.TheexactinformationsuppliedtoeachofthegroupsappearsinFigure9.3.4InstrumentsTheoutcomesfellintothreemaincategories.Onesetmeasuredachievementintermsofstudentposttestandretentiontestscores.Anothersetfocusedonstudentsatisfactionasmeasuredbypreferenceforself-pacingversusinstructor-pacing.Athirdsetmeasuredprocrastinationintermsofwhenassignmentsweresubmittedrelativetoduedatesandhowoftendeadlineextensionswererequested.Eachsetofmeasuresisdiscussedindetailbelow. 8Theycouldnotswitchgroupsorgainaccesstothevarioustreatments.44

PAGE 60

Figure3.AssignmentduedatesfortreatmentR.45

PAGE 61

DearStudent,Thiscoursewareisbeingprovidedtoyoufreeofchargeforthepur-poseofteachingyouHTMLaswellasforthepurposeofcollectingdataforastudyinvestigatingtheeffectsofdifferenttypesofdead-lines.Basedonrandomassignment,youwillbesuppliedwithREC-OMMENDEDduedatesforeachintermediateassignment.THESEDUEDATESAREMEANTONLYASAGUIDE.YOUNEEDNOTMEETANYBUTTHEFINALONEOFMARCH24.Ifyouwouldliketolearnmoreaboutthevariousdeadlinecontingenciesbeinginvestigated,pleasevisitthestudyWebsite.ALLINTERMEDIATEASSIGNMENTSAREDUEBYMARCH24.ABSO-LUTELYNOASSIGNMENTSWILLBEACCEPTEDAFTERMARCH24.WHILEYOUARENOTREQUIREDTOMEETTHEOTHERDUEDATES,ITISRECOMMENDEDTHATYOUATTEMPTTOMEETTHEMALLINORDERTOCOMPLETEALLINTERMEDIATEASSIGNMENTSINATIMELYFASH-ION.Youneednotworrywhetherornotstudentswithothertypesofdead-lineshaveanadvantageforlab4.Duetoanumberoffactors,itisnotclearwhichgroup(s),ifany,willhaveanadvantage.Ifasignicantdifferenceisfoundbetweengroups,thenthegradesonthislabforthegroup(s)withtheloweraverage(s)willbecurvedupward. Figure4.DuedatedescriptionfortreatmentR.Textinallcapitallettersindicatesthoseportionswhichdifferfromthetextgiventostu-dentsintheothertwotreatments.Noneofthetextsuppliedtothestudentswasactuallyinallcapitalletters.46

PAGE 62

Figure5.AssignmentduedatesfortreatmentC.47

PAGE 63

DearStudent,Thiscoursewareisbeingprovidedtoyoufreeofchargeforthepur-poseofteachingyouHTMLaswellasforthepurposeofcollectingdataforastudyinvestigatingtheeffectsofdifferenttypesofdead-lines.Basedonrandomassignment,youwillbesuppliedwithCON-DITIONALduedatesforeachintermediateassignment.Ifyouwouldliketolearnmoreaboutthevariousdeadlinecontingenciesbeinginvestigated,pleasevisitthestudyWebsite.TORECEIVEFULLCREDITFORANINTERMEDIATEASSIGNMENT,YOUMUSTSUBMITITBYTHELISTEDDUEDATE.IFYOUSUBMITITBE-FORETHEBONUSDUEDATE,YOUWILLRECEIVEANADDITIONALPOINT.FOREXAMPLE,IFTHEASSIGNMENTISWORTH4POINTS,THENYOUCANEARNAMAXIMUMOF5POINTSTOTALFORIT.IFYOUSUBMITTHEASSIGNMENTLATE(AFTERTHEDUEDATE),BUTBEFORETHELISTEDPENALTYDUEDATE,THEN1POINTWILLBEDEDUCTEDFROMYOURGRADEFORTHISASSIGNMENT.INTHATCASE,FORANASSIGNMENTWORTH4POINTS,YOUCOULDEARNAMAXIMUMOF3POINTS.ABSOLUTELYNOINTERMEDIATEASSIGNMENTWILLBEACCEPTEDAFTERITSRESPECTIVEPENALTYDUEDATE.(NOTETHATASSIGNMENTS7AND8WILLNOTBEACCEPTEDAFTER3/24.)THE8POSSIBLEBONUSPOINTSYOUMAYPOTENTIALLYEARNCANBETHOUGHTOFASAPPLYINGTOWARDYOURHTMLPOSTTEST,WHICHISWORTH24POINTS,AND/ORTOWARDPOINTSMISSEDONTHEHTMLASSIGNMENTS.THEBONUSPOINTSWILLNOTBECAR-RIEDOVERTOTHERESTOFTHECLASS.FORTHISLABONHTML,YOUMAYEARNAMAXIMUMOF60POINTS.Youneednotworrywhetherornotstudentswithothertypesofdead-lineshaveanadvantageforlab4.Duetoanumberoffactors,itisnotclearwhichgroup(s),ifany,willhaveanadvantage.Ifasignicantdifferenceisfoundbetweengroups,thenthegradesonthislabforthegroup(s)withtheloweraverage(s)willbecurvedupward. Figure6.DuedatedescriptionfortreatmentC.Textinallcapitallettersindicatesthoseportionswhichdifferfromthetextgiventostu-dentsintheothertwotreatments.Noneofthetextsuppliedtothestudentswasactuallyinallcapitalletters.48

PAGE 64

Figure7.AssignmentduedatesfortreatmentA.49

PAGE 65

DearStudent,Thiscoursewareisbeingprovidedtoyoufreeofchargeforthepur-poseofteachingyouHTMLaswellasforthepurposeofcollectingdataforastudyinvestigatingtheeffectsofdifferenttypesofdead-lines.Basedonrandomassignment,youwillbesuppliedwithAB-SOLUTEduedatesforeachintermediateassignment.ABSOLUTELYNOINTERMEDIATEASSIGNMENTWILLBEACCEPTEDAFTERITSRE-SPECTIVEDUEDATE.Ifyouwouldliketolearnmoreaboutthevari-ousdeadlinecontingenciesbeinginvestigated,pleasevisitthestudyWebsite.Youneednotworrywhetherornotstudentswithothertypesofdead-lineshaveanadvantageforlab4.Duetoanumberoffactors,itisnotclearwhichgroup(s),ifany,willhaveanadvantage.Ifasignicantdifferenceisfoundbetweengroups,thenthegradesonthislabforthegroup(s)withtheloweraverage(s)willbecurvedupward. Figure8.DuedatedescriptionfortreatmentA.Textinallcapitallettersindicatesthoseportionswhichdifferfromthetextgiventostu-dentsintheothertwotreatments.Noneofthetextsuppliedtothestudentswasactuallyinallcapitalletters.50

PAGE 66

DeadlineContingenciesUnderInvestigationYouareparticipatinginastudyinvestigatingtheeffectsandimpli-cationsofvariousdeadlinecontingencies.Youhavebeenassignedrandomlytooneofthreegroups.Studentsineachgroupwillfacedifferentconsequencesbasedonwhentheysubmitassignmentsrel-ativetothesuppliedduedatesforlab4.Keepinmindthatifthereisasignicantdifferenceingradesbetweenthegroups,thenthegradesofthestudentsinthegroup(s)withtheloweraverage(s)willbecurvedupward.Forexample,saythatthestudentsinonegroupearn55outof60points,onaverage.Saythattheothertwogroupsearn40and45points,onaverage.Then15and10pointswillbeaddedtoallstudent'sscoresinthesetwogroups,respectively.Thethreedeadlinecontingenciesunderinvestigationincluderecom-mended,conditional,andabsolutedeadlines.Notethatallgroupshavefourdeadlines,oneoneachFridayduringthestudy.Onlytheconsequencesofnotmeetingthesedeadlinesvariesbetweengroups.RecommendedIntermediateduedatesaremerelyrecommended.Allassignmentsareactuallydueby3/24.Noassignmentswillbeacceptedafterthisdate.ConditionalAssignmentssubmittedbytheMondayprecedingtheduedateearnanadditionalbonuspointthatcanbeappliedtowardtheposttest.As-signmentssubmittedaftertheduedate,butbythefollowingMondayareassessedapenaltypoint.Noassignmentisacceptedafteritspenaltyduedate.Nobonuspointisavailableforthersttwoassign-ments,andthelasttwoassignmentswillnotbeacceptedforpartialcreditafter3/24.AbsoluteNoassignmentwillbeacceptedafteritsrespectiveduedate. Figure9.Deadlineinformationsuppliedtoallstudents.51

PAGE 67

3.4.1StudentAchievementAsearchofthemostrecentMentalMeasurementsYearbookyieldednoin-strumentsmeasuringachievementinHTML.Therefore,instrumentswerecreatedspecicallyforthisstudy.Seethepretest,posttest,andretentiontestinAp-pendixesF,G,andH,respectively.AppendixEcontainsanannotatedlistofthequestionsonallachievementmeasures.Foreachitem,theinstructionalobjective(seeAppendixD)itmeasuresisnotedalongwiththeexamsonwhichitappearedandwhetherornotitwasusedinthepilotstudy.Theposttestessayquestion(#46inAppendixE)wasverysimilar,butnotidenti-cal,totheonethatappearedonthepilotposttest.ThemaindifferencewasthattheposttestquestionfortheactualstudyrequiredthestudentstoreadnewreferencematerialontheWIDTHattributeoftheHRtag.Theyhadtousethisinformationtocompletethetask,demonstratinganunderstandingofthegeneralformatforHTMLtagsandattributesaswellastheabilitytoutilizereferencematerialonanattributenotformallydiscussedinthecourseware.Inaddition,theywereaskedtocenterthephraseCheckoutmyWebsite!.Theessayquestionincorporatedtwelveitems,whilemeasuringthestudents'abilitytoapplywhattheyhadlearned.Acomparableessayquestionappearedontheretentiontest.AsindicatedbytheannotationsinAppendixE,thepretestandretentiontestcontainedallofthesameitems,withtheexceptionoftheessayquestion,whichonlyappearedontheretentiontest.Asimilaressayquestion,posedtothestu-dentsontherstdayofthepilotstudy,revealednodataofvalue.Noneofthestudentswereabletomakeanyreasonableattemptatansweringthequestion.Therefore,noessayquestionwasincludedonthepretestintheactualstudy.Havingidenticalquestionsonthepretestandretentiontestwasnotanticipatedtobeaproblem,becausestudentswerenotinformedoftheirperformanceonthepretest,andtheretentiontestwasgiventwomonthsafterthepretest.Itwasunlikelythatstudentswereabletorecallanyofthequestionsonthepretestwhentakingtheretentiontest.Furthermore,moststudentswerenotfamiliarwiththeHTMLlanguagewhentheytookthepretest,andhence,werenotlikelytorecallanyofthequestions.Becauseitwasmorelikelythatstudentswouldrecallquestionsfromtheposttestwhentakingtheretentiontest,theposttestquestionswerecompletelydistinctfromthoseusedontheretentiontest(andpretest).Whilethismayweakensomewhatstatementsaboutchangesintestscoresacrosstime,itstrengthensthoseconcern-ingrecallofHTMLontheretentiontest,becausestudentswerelesslikelysimplytohavememorizedtheanswerstoparticularquestions.Also,asdiscussedinsec-tion3.4.1.2,carewastakentoensurethatthequestionsontheposttestandthe52

PAGE 68

retentiontestwerecomparableincontent,difculty,andtheweightofthetestedcoursewareobjectives.3.4.1.1Administration.StudentstookthepretestinclassonTuesday,February22.Theywerecompensatedfortheirtimewithveextracreditpointsappliedtowardtheclass.TheywereinformedthattheirresponseswouldprovideimportantdataindeterminingthestrengthofconclusionsdrawnabouttheHTMLportionoftheclass.Inaddition,theywerereassuredthattheirindividualscoreswouldnotbeassociatedwiththeircoursegradeinanyway,norwouldtheirscoresbesharedwithanycourseofcials.Furthermore,theyweretoldthattheywerenotexpectedtoknowanyoftheanswersatthispoint.SeeAppendixFfortheexactwording.Thepretestwasshort,containingonly11questions,sothatitrequiredminimalclassandstudenttime(5-10minutes).Thestudentswerenotinformedoftheirperformanceonthismeasure.Thepurposeofthepretestwasstrictlytoverifythatrandomassignmenttotreatmentsyieldedroughlyequivalentgroups.FourdaysafteralltreatmentsterminatedonMarch24,theposttestwasadmin-isteredinclass(March28).Inkeepingwiththeclasspracticeofprovidingstudentswithreviewspriortoexams,anineminutereviewofthemaincoursewaretopicswasprovidedwithinthecoursewareundertheTestmainmenuoption.Studentswereinformedoftheirscoresontheposttestonline,inthenormalfashionsetupfortheclass.Themultiplechoiceportionwasscoredbymachine,whiletheessayportionwasscoredbyhand(seesection3.4.1.4).StudentexamswerekeptonleintheTAs'ofce,sothatthestudentscouldndoutwhichquestionstheymissedandexaminefeedbackontheirperformanceontheessayquestion.TheTAsweresuppliedwithasinglecopyofthesolutioncodeandgradingrubric,whichwaskeptinasecurelocation.InaninterviewconductedwithfouroftheveTAsandbothofthecoursefacilitatorsafterEdTechended,allpartiespresentreportedthatnostudentsrequestedareviewoftheirposttestresponses.TheretentiontestquestionsweregivenaspartofEdTech'sExam2onApril27.TheyconsistedofthesamenineHTMLachievementquestions9thatappearedonthepretestalongwithanessayquestionverysimilartotheoneontheposttest.Everyattemptwasmadetocreateaninstrumentdistinctfromtheposttest,butofcomparabledifculty(seesection3.4.1.2).Theretentiontestwasshort10inordertoincorporateitreasonablyintoExam2andtominimizetheamountofclassandstudenttimerequired.Studentswereinformedoftheirperformanceintermsoftheirscoreontheentireexam. 9ThepretestcontainedninemultiplechoiceitemsmeasuringHTMLachievementandtwoself-reportitemsmeasuringpriorexperiencewithtypesettingandprogramming.10Seesection4.1foradiscussionofthepreliminarydataanalysisusedtopredicttheimpactthismighthaveonreliability.53

PAGE 69

3.4.1.2Development.Theachievementmeasuresforthepilotstudywerenotcreatedinasystematicmanner.Therefore,newmeasuresbasedonthecoursewareobjectiveslistedinAppendixDwerecreatedfortheactualstudy.Theweightofeachobjectiveisindicatedintermsofthenumberofquestionsthatmea-sureditontheachievementtestsfortheactualstudy.Afterrelativeweightsweredeterminedforeachobjective,proportionatenumbersofitemsmeasuringtheseobjectiveswereborrowedfromthepilotposttest.Newitemswerecreatedwhereneeded.Ofthe37multiplechoiceitemsonthepilotposttest,18wereretainedaswritten11,and8morewereretainedinmodiedform.Inall,26multiplechoiceitemsoutof37wereretained,and19newoneswerecreated.Thenumberofitemscoveringeachobjectiveonthepilotposttestwasnotwellbalanced.Ofthe11multiplechoiceitemsnotretained,6weredeleted,becausetheobjectivestheymeasuredwereoverrepresented.In5outof6cases,thedeci-sionregardingwhichofseveralitemsmeasuringthesameobjectivetodeletewasbasedonwhichitemwouldincreaseCronbach'salphamostwithitsremoval.Inthe6thcase12,thedecisionwasbasedonwhichitembestcoveredtheimplicitcourse-wareobjectives.BecausetheydidnotcoveranyoftheobjectivesinAppendixD,4moreofthe11multiplechoiceitemswereremoved.Finally,the11thitemwasnotretained,becauseitdisagreedsomewhatwiththemessagedeliveredbythecourseware.Nowconsidertheeightitemsfromthepilotposttestthatweremodiedfortheactualstudyinstruments.Fivewerealtered,sothattheywouldhavefourchoices,ratherthanonlytwoorthree.Fortheotherthree,thechoiceswerealteredtobettermatchthecurrentstateofthecoursewareaswellasotherquestionsontheexam.Analysisofpilotstudydataonthe26retaineditemssuggestedthatitwasreasonabletoretainthemall,because.72andremovingasingleitematatimeyielded#"[.68,.74].Everyattemptwasmadetocreateitemsforeachobjectivewithsimilardif-cultylevels.Then,foreachobjective,adiewasrolledtodeterminewhichoftheitemsmeasuringitwouldappearontheposttestandwhichwouldappearonthepretest/retentiontest.AppendixElistsallitemsinthesameorderastheobjectives.Eachitemisannotatedwithitstargetobjective,alternativeobjectivesidentiedbyexperts(seesection3.4.1.3),thecorrectresponse,andtheinstrument(s)uponwhichitappeared. 11Insomecases,themultiplechoiceresponsesfortheseitemsappearinadifferentorder.12Inthiscase,itemanalysisusingCronbach'salphaindicatedthatanalternativecandidatewouldbethebestonetoremove.However,uponconsultationwiththecoursefacilitatorwhoparticipatedinthepilotstudy,akeyelementonanassociatedgradingrubricwasnotcheckedwhenstudentassignmentsweregraded.Ifithadbeenchecked,itwasanticipatedthatstudentperformanceonthisitemwouldhavebeensubstantiallydifferent.Inaddition,onlythreeofthestudentscompletedtheassignmentassociatedwiththisitem.54

PAGE 70

Theorderofthemultiplechoiceitemswasrearrangedforreviewbytwoexpertsandforthecreationofthetheachievementinstruments.Allmultiplechoiceitemsconsistedofastemfollowedbyfourresponses,onecorrectandthreedistractors.Theposttestcontainedsevenitemswiththechoicemorethanoneofthefollowing,whichwasthecorrectresponseinthreecases.Thepretest/retentiontestcon-tainedthreesuchitems,withthischoicecorrectinonecase.Notefurtherthattheposttestcontainedtwoitemswiththechoiceallofthefollowing,whichwascorrectinonecase.Finally,correctresponseswereevenlydistributedacrosschoices.Ontheposttest,Awasthecorrectchoiceninetimes,aswasB,C,andD.Onthepretest/retentiontest,Awasthecorrectchoicethreetimes,whileB,C,andDwereeachcorrecttwice.3.4.1.3Validity.Recreatingtheachievementmeasuresinasystematicfash-ionbasedontheimplicitcoursewareobjectiveswasaninitialattemptatgeneratinginstrumentswithcontentvalidity.ThisvaliditywasfurthercheckedinasystematicreviewbytwoindependentexpertsonHTML.AppendixIcontainstheletterandquestionsposedtotheseexperts.Theletterbeganbyaskingtheexpertstorateaself-reportquestiononthepretestconcerningpriorexperiencewithtypesettinglanguagessuchasLATEX.Basedonrecommendationsfromtherstexpert,thequestionwasexpandedtoincludeprogramminglanguagessuchasAda,BASIC,C,Cobol,Fortran,Java,JavaScript,LISP,Pascal,VisualBasic,andVisualC.Thesecondexpertwasgiventheques-tionintheforminwhichitappearsinAppendixI.Basedoncommentsfromthisexpert13,itwasfurtherexpandedtoincludeauthoringlanguagessuchasAuthor-ware,IconAuthor,andQuest.Itwasalsosimpliedbysplittingitintotwoquestionsandonlyrequiringthestudenttorespondtotwoitemsratherthantofteen.SeethelasttwoquestionsonthepretestinAppendixFtoseethenalformofthesequestions.Inadditiontotheletter,theexpertsweregivenacopyoftheobjectivesinAp-pendixDandaskedtocommentontheirappropriatenessaswellasontherelativeweightingsassignedtoeach.Bothexpertsagreedthattheobjectivesandweight-ingswerereasonable.Neitherindicatedthatanyimportantobjectiveshadbeenleftout.However,examinationoftheretentiontestdatadidindicatethatanadjustmentoftheweightingschemewaswarranted.Theessayportionoftheexamproducedmuchmorereliablescoresthanthemultiplechoiceportion.Therefore,ratherthanrescaleeachessayitembyafactorof0.25asinitiallyplanned,theitemswereleft 13Expert2alsomentionedthat,whilenotsupportedbyempiricaldata,abackgroundinsymbolicnotationsystemssuchasmusic,Mathematics,electroniccircuitry,andschematicsmightgivestudentsanadvantage.WhilesuchexperiencesmaybeadvantageousinlearningHTML,thefacevalidityofthepretestwouldlikelybeloweredbytheinclusionofitemsmeasuringthem.IntruthmanypriorexperiencesmayplayaroleinhoweasilystudentslearnHTML,includingxinganautomobile,whichrequiresskillatproblemsolving.55

PAGE 71

unscaled.ThisincreasedCronbach'salphafortheretentiontestscoresfrom.70to.85.Italsohadtheeffectofincreasingtheweightsofobjectives3and4to29%anddecreasingtheweightsofobjectives6-10to5%(seeTable8).Thisseemedreasonable,becauseobjectives3and4arearguablymoregeneral.Infact,expert2notedthegeneralnatureofobjective4inanunsolicitedwrittencommentduringtheinitialreview.Furthermore,expert1foundthealternativeweightingschemereasonableduringafollowupreview.Toensurethattheposttestandretentionteststillwerecomparablemeasures,theweightingschemewasadjustedfortheposttestitemsaswell.Fordataanalysisonly14,eachcorrectessayitemontheposttestwasrecordedasa4ratherthana1,sothattheweightofeachessayitemonbothmeasureswasincreasedbyafactoroffour.SeeAppendixDforthenalweightingschemeandTable8tocomparethisweightingschemewiththeoriginalone.Asanalnoteonthescopeoftheobjectives,Expert2,havinghadexperiencewithEdTechandthestudentsinvolved,didindicatesomeconcernovertheamountofmaterialcovered,andinfact,thestudentsdidreport(seeitem6inAppendixP)spendingsevenhoursperweek,onaverage,completingassignmentsforthestudy.Thiswasmorethananticipatedbasedonpilotstudydata(seeAppendixC).Theexpertsalsowereaskedtoconsidereachiteminturn.Theyweregivenaform(seeAppendixJ)withaboxtotherightofeachitem.Ineachbox,theywereaskedtoindicatewhichobjective(s)theythoughtthecorrespondingitemcovered.Theywerealsoaskedtoindicateiftheythoughttheitemwasclear,wastoodifcult,neededtobereworded,and/orneededtoberemoved.Spacewasprovidedforthemtomakeanyothercomments,andtheywereencouragedtomarkchangesontheitemswhereneeded.Expert1indicatedthatanswersaandbforquestion29werenotdistinctenoughandmightbeconfused.Belowistheoriginalquestionasreviewedbyexpert1.29.Itispossibleto asubmitbutton.(a)llthecontentsof(b) changethemessagedisplayedon (c)changethesourcereferencedon(d)lltheimageeldof 14Students'HTMLgradesinEdTechwerebasedontheoriginalweightingscheme.56

PAGE 72

Thestemandrstandlastresponseswerechangedasfollowsandsuppliedtoexpert2.29.Itispossibletochangethe asubmitbutton.(a)rangeof(b) messagedisplayedon (c)sourcereferencedon(d)typeofExpert2recommendedthatitems30,35,and43berewordedtoimprovetheirclarity.Unfortunately,theserecommendationswereinadvertentlyleftoutofthenalversionsoftheinstruments.Therecommendedchangetoquestion30wasfairlyminorasindicatedbytheinclusionoftheitalicizedtextbelow.30.WhatattributeoftheFRAMEtagmustbesetinordertouseitasatargetforA(oranchor)tags?(a)TARGET(b) NAME (c)SRC(d)HREFTherefore,itisexpectedthatleavingitunchangeddidnotimpactitseffectivenessgreatly.Thechangesrecommendedforquestions35and43weremoresubstan-tial.Again,changesappearinitalics.35.Ifyouwanttoimproveyourwebpagesothataparticularimagedownloadsanddisplaysfaster,whatshouldyoudo?(a)morethanoneofthefollowing(b)reducethewidthorheightspecicationintheIMGtag(c)donotspecifyawidthorheightfortheimage(d) opentheimageinanimageeditorandmakeitsmaller 57

PAGE 73

43.Ifyouwantanewpagedisplayedafterclickingalinkinaframe,sothatalltheframesarewipedoutandthepageisdisplayedintheentirebrowserwindow,towhatshouldyousettheTARGETattributeoftheA(oranchor)tag?(a) blank(b) window(c) top (d) newAlthoughtheserecommendationsmayhaveyieldeditemsthatwerearguablymoreclearthantheoriginalitemsaswordedinAppendixJ,itisbelievedthattheirvalid-ityremainsintact.Asevidence,bothexpertsselectedthesameobjectivefromalistof34,andinessence,agreedthattheitemsmeasuredthetargetedobjective.Also,expert1foundbothitemsclearlywritten,suggestingnochangesinwording.Onelastconcernraisedbyexpert2wastheamountofrecallrequiredforsettingtheattributesofthebodytagsinessayquestions46and47.Therefore,thestu-dentsweresuppliedwiththeadditionalreferencematerialonpage171forboththeposttestandtheretentiontest.Asmentionedpreviously,itemsinAppendixEareannotatedwiththealternativeobjectivesidentiedbytheexperts.Theywereaskedtolistallpossibleobjectiveswhentheywereuncertainwhichwasthetargetandwereaskedtolistthemostappropriateonerst.Forthoseitemsforwhichbothexpertslistedonlythetargetobjective,noalternativeobjectivesarenoted.ThisinformationissummarizedinTable5.Eachexpertwasaskedtoidentifythetargetobjectivefor69items,sotogethertheyclassied138.In114cases,bothexpertsselectedtheobjectivetargetedbytheitem.Intheother24cases,theyeitherlistedadditionaloralternativeobjectives.In14ofthe24,thetargetobjectivewasinthelistidentiedandwasactuallylistedrst8times.In5oftheremaining10cases,theexpertsagreedontheobjectivecategory,butnotonthespecicfacetoftheobjective.Forexample,item14waswrittentomeasureobjective5b,buttheexpertsbothfeltitmeasured5a.Asimilarconditionheldforitems18,28,and29.Forthenal5items,theexpertsidentiedalternativeobjectivecategories,andinallcases,objective4awasinvolved.Be-causeofthegeneralnatureofthisobjective,whichwasto“usetagsandattributescorrectly,”itisnotsurprisingthatitmightbeconfusedwiththeotherobjectives.Infact,expert2wrote,“objective4arelatestoalmostallofthesequestions.”Inconclusion,noneofthesuggestionsmadebytheexpertsdifferedsubstan-tiallyfromthenalformoftheinstruments.Likewise,identiedobjectivesandtar-58

PAGE 74

Table5.SummaryofAlterna-tiveObjectivesIdenti-edbyExperts Objective ItemaTbEx1cEx2d 21b4a84a3c,4a94a4a,5a3c,4a104a5a,4a5a,3c145b5a5a186a6b288a8a,8b8d298a8b8b,8a369b4a4654a4d,4a4664d4d,4a46103a3a,4a46123a4a3a,4a4754a4d,4a4764d4d,4a47103a3a,4a47123a4a3a,4a Note.Blankentriesindicatethoseitemsforwhichanexpert'sjudgmentwithre-specttotheobjectivemeasuredagreedwiththetargetobjective.aBasedonitemorderinginAppendixE.Subscriptsindicateitemnumberwithrespecttoorderofappearanceonrubric.bTargetobjective.cObjectiveselectedbyrstexpert.dObjectivese-lectedbysecondexpert.59

PAGE 75

getobjectiveswereincloseagreement.Therefore,thecontentoftheachievementmeasuresappearedvalid.Concurrentvaliditywasconsiderednext.Itseemedrea-sonabletoexpectthatposttestandretentiontestscoreswouldcorrelatepositivelywitheachotherandwithassignmentscores.Inparticular,itwasanticipatedthatthecorrelationswithrespecttoassignmentscoreswouldbehighestfortheessayportionsoftheachievementmeasures,becausealloftheseitemsmeasuredper-formanceandtheapplicationofknowledge.Furthermore,correlationsbetweenpilotstudyassignmentscoresandpilotposttestessayquestionscoresweresta-tisticallysignicant15.Asanticipated,scoresontheposttestandtheretentiontestcorrelatedsigni-cantlywithscoresontheassignments.Thisheldforthemultiplechoiceandessayportionsseparatelyaswellasforthemeasuresoverall,andinallcases,thecor-relationbetweentheassignmentscoresandtheessayportionsoftheexamswasgreaterthanorequaltothecorrelationwiththemultiplechoiceportions.Inaddi-tion,allindividualportionsoftheposttestcorrelatedsignicantlywithallindividualportionsoftheretentiontestasdidthetestsintheirentirety.All(sub)measures,ex-cludingthepretest,alsocorrelatedsignicantlywithscoresonEdTech'sExam1aswellaswiththestudents'nalclassgrades.SeeTable6forasummaryofthecor-relationsbetweenachievementandperformancescores.Noticethatthepretestdidnotcorrelatesignicantlywithanyitemexceptthemultiplechoiceportionoftheretentiontest(r.16,p.04).However,evenforthiscase,thecorrelationwassmallandnolinearrelationshipwasreadilyapparentfromanexaminationofthescatterplot.Becauseboththepretestandretentiontestcontainedtheexactsamemultiplechoiceitems,thiswaslikelyjustanartifactofstudentsguessinginaconsistentmanneronitemstheydidnotknow.3.4.1.4Reliability.Basedonthe26multiplechoiceitemsand9essayitemsincludefromthepilotposttestinthemeasuresfortheactualstudy,itwasantici-patedthatthemeasuresfortheactualstudywouldbereasonablyreliable.Forthepilotdata,Cronbach'salphawas.72andintrarater16reliabilityontheessayquestionswasr.96(p.0001).Infact,withtheexceptionofthepretest,allachievementmeasuresusedintheactualstudyhadmoderatelyhighreliabilitywith.89fortheposttest,.85fortheretentiontest,and.87fortheassignmentscoreswhenbonusesandpenaltieswereapplied.ThelowreliabilityofthepretestwaslikelyduetothefactthatmoststudentsknewlittleaboutHTMLwhentheytookit.Onaverage,theyreportedthattheirprior 15Recallthatthepilotessayquestionswereratedtwice.Fortherstrating,thecorrelationbetweenassignmentandessayscoreswasr.57withp.04.Forthesecondrating,r.64withp.02.16Thiswascalculatedbycomparingthescoresobtainedbygradingallessayitemsononedayandthenagainonthenextday.60

PAGE 76

Table6.CorrelationsBetweenAchievementandAssignmentScores 2345 Measure1abcabcabab 1.Pretest.01.09-.01.02.16.06.10-.01-.05-.02-.042.Posttesta.MultipleChoice.80.78.88.66.70.76.58.61.44.55b.Essay.88.98.63.83.84.63.61.37.55c.Overall.89.67.83.85.64.64.41.573.RetentionTesta.MultipleChoice.54.62.84.42.44.41.46b.Essay.85.95.55.55.29.53c.Overall.85.55.57.37.554.AssignmentGradea.Bonus/Penalty.87.89.26.53b.NoBonus/Penalty.86.23.545.EdTechGradesa.Exam1 .42b.FinalGrade Note.ValuesalongthediagonalrepresentCronbach'salphaforthegivenmeasure.Dashesindi-catemeasuresforwhichdataneededtoperformthecalculationswerenotavailable.Allcorrela-tionsexceptthosewiththepretestandwithExam1werestatisticallysignicantwithp$%.0001.ForExam1,theywerestatisticallysignicantwithp$%.014.Thepretestwassignicantlycor-relatedwiththemultiplechoiceportionoftheretentiontestonly(p.0421).61

PAGE 77

Table7.ReportedPriorExperiencebyTreatment TreatmentnMMinMaxSkewnessKurtosis HTMLandLATEXR481.52141.371.08C471.62151.822.53A411.35131.742.21 ProgrammingR481.39141.933.65C471.45152.305.04A411.16133.0510.15 Note.Valueswerecalculatedafterthescoresofindividualsinstudygroupsidentiedpriortothetreatmentintervalwerereplacedwithaveragesandaftermissingvalueswererectied.experienceconsistedoflessthanoneyearofprogrammingandusingformattinglanguagessuchasHTMLandLATEX.SeeAppendixFfortheexactwordingofthequestionsandthepossibleresponses.Althoughafewstudentsineachgroupre-portedmuchgreaterlevelsofpriorexperienceintheseareas,theywerefairlywelldistributedacrosstreatments.Table7liststheaveragelevelsofpriorexperiencereportedonthepretestandprovidesadditionalevidencethatpriorexperiencewasroughlyequivalentacrossgroups,withstudentsinRandCreportingslightlymorepriorexperiencethanstudentsinA.Recallthat,becausepretestdataweresounreliableandbecauserandomassignmentappearedtohaveproducedroughlyequivalentgroupsaccordingtotheresultsfromseveralrandomizationtests,thepretestwasexcludedfromformaldataanalysis.Inadditiontomeasuringtheinternalconsistencyoftheinstruments,intraraterreliabilitywascalculatedfortheessayportionsoftheposttestandretentiontest.TheessaysweregradedbyhandaccordingtotherubricslistedinTables46and47.Bothweregradedbyasinglegrader,reducingTAworkloadandeliminatinginter-raterdifferencesaswellastheneedtotrainseveralTAsonusingtherubrics.Rubriccheckboxesonpages196and203facilitatedgradingbymakingiteasiertorememberandcheckoffeachoftherubrics.Beforegrading,15studentswereselectedrandomly,andtheiressayswereremovedfromtheungradedstackandphotocopied.Theneachoriginalandphotocopywasreturnedtoarandomlocationinthestack.Theywereplacedthroughouttheentirestack,soastoaccountforanypotentialeffectsofgradingorder.Essaysforthesame15studentsweretreatedinlikemannerforboththeposttestandretentiontest,withtheexceptionofstudent#283whowasmissingretentiontestdata.Intraraterreliabilityfortheposttestandretentiontest,respectively,wasr0.99andr1.00withp.0001.62

PAGE 78

3.4.2StudentPacingPreferenceSearchingthelatestMentalMeasurementsYearbookviaSPIRS(SilverPlat-ter,2001)withthesearchstring[(preferORpreference)AND(paceORpacing)]yieldedfourrecords.Onlyone,theProductivityEnvironmentalPreferenceSur-vey,measuredsomethingclosetopacingpreference.Thesubscore,LearningAlone-Peer-OrientedLearner,is1of20subscoresderivedfromthis100itemin-strument.Presumablyaround5itemsaredevotedtothissubscore,whichascer-tainsalearner'spreferenceforworkingindividuallyversusinagroup.ReviewersofthePEPSinstrumentindicatethatconstructandpredictivevalidityforthemea-surearenotadequatelyaddressed.Inaddition,reportedreliabilitiesforseveralofthesubscoresarelow,with5under.50,1in[.50,.60),7in[.60,.80],and7in(.80,.90).BecauseoftheunknownvalidityandpossiblylowreliabilityofthePEPSsubscoresandthefactthatself-pacingencompassesmorethanthesinglecomponentofworkingaloneversusinagroup17,anewinstrumentwasdevelopedforthecurrentstudy(seeAppendixK).Itincorporatedfeaturesliketheoneslistedbelow,whichdistinguishself-pacingfromteacher-pacing.LivelectureversusprerecordedlectureLearninginalargegroupversusasmallgroup(possiblyofsize1)Learningduringstructuredclasstimeeachweekversusexible,self-determinedtimesImmediate,rst-handaccesstoallstudents'questionsandanswersversusdeferred,second-handaccesstoselect,frequentlyaskedquestionsandan-swersOne1-hourlectureversussix10-minutelecturesTeachersetdeadlinesversusstudentsetdeadlinesMorefrequentself-evaluationversuslessfrequentteacher-evaluationStudentsindicatedtheirpreferenceonaratingscalefrom1to5onsuchissuesbyrespondingtotheeightquestionsontheinstrument.Forfouroftheitems,aresponseof5indicatedahighpreferenceforself-pacing.Fortheotherfour,itindicatedahighpreferenceforteacher-pacing.Forallquestions,oneoftheoptionswasalwaysNoPreference.ThisinstrumentwasgiventothestudentsinclasswhentheytooktheHTMLposttest.Aswasthecaseforthepretest,students 17Infact,studentsunderself-pacingconditionsstillhavetheoptionofformingstudygroups,ifdesired.63

PAGE 79

werecompensatedfortheirtimeintakingthesurveywithveextracreditpointsappliedtowardEdTech.Theyalsowereinformedoftheimportanceandfunctionalanonymityoftheirresponses.Forprecisewording,pleaseseeAppendixK.Itshouldbenotedthatthefeasibilityofgivingasimilarinstrumentpriortotreat-mentwasconsidered.Itwouldhaveprovidedameansofdetermininganychangesinself-reportedpacingpreferencethatmighthavebeencausedbythetreatments.However,itmightalsohavesensitizedstudentstotheissueofself-pacingversusteacher-pacing,andmayevenhavepromptedsomeofthesepreserviceteach-erstolearnmoreabouttheseinstructionalparadigms.Thissensitization,inturn,mighthavecausedstudentstoresponddifferentlytotheinstrumentonpacingpref-erencegivenafterexposuretotreatments(Ary,Jacobs,&Razavieh,1996).Therewasalsothedangerthatapretreatmentsurveymightinteractwiththetreatment,decreasingtheexternalvalidityofthendings.Becauseofthesedangersandthefactthat,withrespecttopacingpreference,themainfocusofthecurrentstudywastodeterminewhetherornottherewasadifferencebetweengroupsfollowingexposuretotreatmentconditions,ratherthanwhetherornottherewasachangeinpreferenceduetoexposure,apretreatmentsurveywasnotadministered.Inaddition,itwasassumedthatrandomassignmentwithmorethan30datapointspergroupadequatelydistributedstudentswithvariousinitialpreferencesacrossgroups(Aryetal.,1996).3.4.2.1Validity.Theconstructvalidityofthisinstrumentwasevaluatedbytwoindependentexpertswhohadexperienceusingbothteacher-pacingandstudent-pacinginstructionalparadigms.TheywereeachgiventheletterinAp-pendixLaswellascopiesoftentativepretreatmentandposttreatmentsurveys.Detailsregardingtheirevaluationsofthepretreatmentinstrumentareomitted,be-causeitwasnotused.However,onecommentbyexpert2hadbearingonthepost-treatmentmeasure.Theexpertquestionedwhetherornotitmadesensetoforcearesponsetoanitemmeasuringpacingpreferenceingeneral.Becausethepur-poseoftheposttreatmentinstrumentwastodeterminewhetherornottherewasadifferencebetweengroupswithrespecttopacingpreferencefollowingexposuretotreatments,ratherthantodeterminethenatureofthatpreference,responseswerenotforcedforanyoftheeightitems.InevaluatinghowcomprehensivelythelistoffeaturesidentiedinAppendixLdistinguishedbetweenteacher-pacingandself-pacing,expert1founditcompleteandrecommendednochanges.However,expert2feltitwasunnecessaryforstu-dentstobeabletoaccesseachothers'questionsandanswersonlineandrecom-mendedthealternativewordingofthefourthfeaturelistedinsection3.4.2.Expert2alsorecommendedtheinclusionoftheseventhfeature.Thispromptedthecre-ationofitem7onthesurveyinstrument(seeAppendixK).Otherrecommendationsbyexpert2promptedadditionalchangestotheinstrument.Initially,itcontainedtwoitemsmeasuringtherstfeaturelistedinsection3.4.2.Thesewerecombined64

PAGE 80

toproduceitem1onthesurvey.Inaddition,fouritemsmeasuringthesecondfea-turewerereducedtoitem2onthesurvey.Finally,thewordonlinewasremovedfromitem4.Foreachitemontheinstrument,bothexpertsagreedonwhichendoftheratingscalerepresentedapreferenceforself-pacingwiththeoppositeendrepresentingapreferenceforteacher-pacing.Again,expert1foundthelistoffeaturesmea-suredbytheitemscomprehensive,andanitemwasaddedtocovertheadditionalfeaturerecommendedbyexpert2.Therefore,thenalinstrumentwasassumedtopossessadequateconstructvalidity.3.4.2.2Reliability.Thisinstrumenthadsufcientreliability(.55)giventhatscoreswereusedforresearchpurposesonly(Aryetal.,1996).3.4.3LevelofProcrastinationToprocrastinateissimplytodelaybeginningatask.Therearemanyreasonsastudentmightprocrastinate,includinghavingalackofdiscipline,assigninglowerprioritytothetaskthantoothercompetingtasks,orwishingtoincreasethechal-lengeofthetaskbyshorteningthetimeframeinwhichitmustbecompleted.Recallthatpossiblenegativeconsequencesofstudentprocrastinationincludein-creasedworkloadforgradersattheendoftheterm,increasedadministrativehas-sleswhenstudentsrequestspecialconsiderationforlatework,andlowerlongtermrecallofacquiredknowledge.Inordertoinvestigateanydifferentialeffectsofprocrastinationforstudentsindifferenttreatments,itwasnecessarytoensurethatprocrastinationitselfdifferedbytreatment.Althoughtheliteraturesupportstheex-pectationthatstudentsinRwouldprocrastinatemore,quantitativedataalongtwodimensionswerecollectedinanattempttoverifythefulllmentofthisprediction.First,becauseassignmentsweresubmittedelectronically,itwasaneasymattertologtheexactsubmissiondateofeachone.Thetimestampforagivensubmis-sionwascomparedwiththeassignment'sduedate,d,andarelativevalueindi-catingthedirectionandlevelofdigressionfromdwasrecorded.Ifanassignmentwassubmittedpriortoitsd,anegativeintegerwasrecorded.Forexample,ifitwassubmittedtwodaysearly,then-2wasrecorded.Ifitwassubmittedonedaylate,+1wasrecorded.Ifitwassubmittedond,0wasrecorded.Asmentionedinsection3.2.2,iftheassignmentwasneversubmitted,thenthedayafterthestudyended,s,wastakenasthesubmissiondateandavalueofsdwasrecorded.Thehighertherecordedvalue,thehigherthelevelofprocrastination,asmeasuredbytherelativedigressionfromthetargetresponserate.CallthismeasuretherateofRDTR.65

PAGE 81

UsingtheRDTRmeasuretoquantifydifferencesinprocrastinationlevelisob-viouslypreferabletofailingtoreportanyobservabledifferences(Hobbs,1981).Ofthenumerousmeasuresthathavebeenusedtocharacterizeprocrastination,noneappeartodistinguishbetweenresponsepatternsaswellastherateofRDTR.Incontrasttoothermeasures,therateofRDTRprovidesanindicationofastudent'svaryingresponserateforeachtaskovertheentiretreatmentinterval.Somere-searchershaveonlyconsideredasubsetofcoursetaskssuchasdelayincom-pletingtherstunittest(Lamwers&Jazwinski,1989)andearlycompletionofthelastreviewexam(Roberts&Semb,1990,1989;Robertsetal.,1988).Othershaveonlyconsideredaportionofthetreatmentintervallikethenumberoftaskscom-pletedintherst6of10weeks(Lamwers&Jazwinski,1989)andquarter-life18(Robin&Graham,1974).Ofthoseresearcherswhodidconsideralltaskscom-pletedduringtheentiretreatmentinterval,mostdiscretizedtheintervalintounits.Someusedsingleunitmeasuresthatspannedtheentireintervalsuchastheav-eragenumberofsubmissionsperweek(Bufford,1976;Robin&Graham,1974).Othersusedmultipleunitmeasurestocomparechangingworkdistributionsovertime.Forexample,onegroupcomparedtheaveragenumberofdailysubmissionsduringthreeportionsofthecourse(Milleretal.,1974),anotherexaminedthedis-tributionofaverageweeklysubmissions(Powersetal.,1973),athirdexaminedthedistributionofsubmissionsduringtherstfourofve15-dayperiods(Morrisetal.,1978),andafourthcomparedtheaverageinter-quizinterval19forthersthalfofthecoursewiththesecondhalf(Sweeneyetal.,1979).Someresearcherschosetodiscretizeresponsetypesintoalimitednumberofcategories.Forexample,stu-dentswerepartitionedintogroupsconsistingofthosewhocompleted12lessonsin9weeksversus12weeks(Wesp,1986),thosewhorespondedatorabovetheuniformresponselineversusthosewhodidnot(Wesp&Ford,1982),andthosewhoserateswereconsideredaccelerated20,uniform21,scalloped22,and/orde-layed23(Murdock,2000;Glick&Semb,1978b;Reiser,1977).TherateofRDTRdoesnotattenuatedifferencesbydiscretizingeithertheintervalortheresponse.It 18Quarter-lifeiscalculatedbydividingthelengthoftimeittooktocompleteonequarteroftherstattemptsatunittestsbythetotaltimeintervaloverwhichrstattemptsweremade.19Thenumberofdaysbetweenthemasteryofagivenunittestandtherstattemptatthefollowingone.20GlickandSemb(1978b)denethisasthecompletionoftwo-thirdsormoreoftheworkabovetheuniformresponseline,whileMurdock(2000)denesitascompletingone-halformoreoftheworkabovetheline.21Submissionsapproximatetheuniformresponseline.22GlickandSemb(1978b)denethisashaving,“onatleastoneoccasion,completedthreecoursetasksonthreeconsecutiveclassdays”(p.133),whileMurdock(2000)andReiser(1977)deneitasapauseinresponding.23Halformoreofthetasksarecompletedbelowtheuniformresponseline.66

PAGE 82

yieldsacontinuous,quantitativevaluethatincorporatesbothnegativeandpositiveresponsesinonemeasure.Otherresearchersthathaveconsideredcontinuousvalueshaveusedmeasuresthatfocusonundesirableresponsesordesirablere-sponsesseparately.Forexample,somecountedthenumberofdaysindailytest-ing(Roberts&Semb,1990,1989)andthenumberofmisseddeadlines(Roberts&Semb,1990,1989),whileotherscountedthenumberoftaskscompletedabovetheuniformresponseline(Roberts&Semb,1990,1989).TherateofRDTRisalsoeasiertointerpretandcompareacrosstreatmentsthanarevisualinspectionsofgraphsoftimespentstudying(Mawhinneyetal.,1971),submissions(Ross&McBean,1995;Lloyd&Zylla,1981;Welshetal.,1980),andattendance(Lu,1976).Anotheradvantageoftheproposedmeasureisthatitreadilydistinguishesthosestudentswhosubmitfewerassignmentsfromthosewhosubmitmore.Forexample,theaveragenumberofsubmissionsperweek24,ascalculatedbyRobinandGraham(1974),isthesameforastudentwhosubmittedoneassignmentperweekfortwoweeksasforastudentwhosubmittedoneperweekforfourweeks.AccordingtotheRDTRmeasure,thelatterstudentwouldhaveapreferablelowervalue.ItshouldbenotedthattherateofRDTRisreallyafunctionofthedatetheassignmentwasstarted(perhapsabettermeasureofprocrastination)andofhowdifcultthestudentfoundit.Usingthismeasure,itispossiblethatanalternativescenariocouldexplainwhyanassignmentwaslateeventhoughthestudentdidnotprocrastinate.Thiswouldbethecaseforastudentwhostartedanassignmentearly,intendingtocompleteitbeforetheduedate,butfoundthatitwasmoredifcultthananticipatedandendedupturningitinlate.Amoredirectmeasurewouldhavebeentoaskstudentstoreportwhentheystartedeachassignment.However,therewasconcernthatthiswouldyieldincompleteandlessaccuratedata.Also,itseemedreasonabletoassumethat,foranygivenassignment,thetreatmentconditionwouldnotcorrelatesignicantlywiththedifcultyleveloftheassignment,butthatthetreatmentconditionwouldcorrelatesignicantlywiththedatethestudentstartedit.Ifso,thenanydifferentiallevelsinprocrastinationshouldreectadifferenceinstartdatesduetotreatmentconditions.Additionalevidenceofhigherprocrastinationlevelswascollectedbytrackingthenumberofstudentrequestsfordeadlineextensions.Suchrequestswerenothonoredduringthetreatmentinterval,soastoensuretheintegrityofthestudy.TheTAswereaskedtorecordwhichstudentsrequesteddeadlineextensionsandforwhichassignmentstheyrequestedthem.Afterthetreatmentinterval,rea-sonsgivenforrequestswerereviewedalongwithscoresforthestudentswhoreportedexperiencingtechnicaldifculties.StudentswhoreportedhavingcorruptCD-ROMswererequiredtobringthembackinexchangefornewones.Allofthe 24RobinandGraham(1974)actuallycalledthis“rateofrsttakes”inreferencetotherequiredmasteryofunitquizzes.67

PAGE 83

eightexchangedCD-ROMsweretested.Onlythree,allfromstudentsinA,werefoundtobecorrupt.Theotherve,withthreefromstudentsinCandtwofromstudentsinR,werenotfoundtobedefective.Thus,penaltypointsforlatesubmis-sionswereliftedforthestudentsinAonly,andthesestudentswereremovedfromformaldataanalysis.Penaltypointswerealsoliftedforthestudentwhosenamecontainedpunctuationwhichwasnothandledproperlybytheassignmentsubmis-sionscripts.Finally,penaltypointsalsowereliftedforastudentwhosubmittedtwoassignmentsdirectlyviae-mailbytheirrespectiveduedates,ratherthanviatheonlinesubmissionforms.Noneoftheotherrequestsforspecialconsiderationwerehonored.3.4.3.1Validity.Thevalidityofthesemeasureswasscrutinizedbythesametwoexpertswhoreviewedtheachievementandpreferencemeasures.Bothagreedthataveragedayslateonassignmentsperstudentandaveragenumberofre-questsforextensionsperstudentweresufcientindeterminingprocrastinationlevel.Neithersuggestedtheinclusionofanyotherindicators.Therefore,themea-suresappeartopossessfacevalidity,atleast,withrespecttostudentprocrastina-tionlevel.3.4.3.2Reliability.Ofthetwoindicators,itwasanticipatedthattheonebasedonsubmissiondateswouldindicatethelevelofprocrastinationbest.Val-ueswerehighlyreliable,becausetheywerecalculatedautomaticallywhenas-signmentsweresubmittedelectronically.Inordertoensurethatnoerrorswereintroducedduetoproblemsinthesoftwareusedtologsubmissionsandtoregisterbonusesandpenalties,thesoftwarewastestedbeforeandduringthetreatmentinterval.Eachassignmentwassubmittedseveraltimesunderctitiousnamesonvariousdates.Forexample,assignment5wasdueonMarch10,sotestdataweresubmittedonMarch6,10,13,14,and25forthisassignmentforallthreetreatmentstomakesurethatbonusesandpenaltieswereregisteredandappliedappropriatelyinallcases.Similartests,conductedforallassignments,veriedthatthatthesubmissionandgradingsoftwareworkedcorrectly.Onlyonestudentex-periencedsoftware-relatedproblemsinsubmittingassignmentsbecauseofnamepunctuation.Also,theWebserverusedforthestudywasmonitoredcontinuouslyduringthetreatmentintervalandfoundtobeavailabletohandlesubmissionswithnolapseinservice.Inordertofacilitatethereportingofrequestsfordeadlineextensions,thecourse-waresuppliedtotheTAsincludedconvenient,password-protectedaccesstotheforminFigure10.AlthoughthesedataweresubjecttoerrorinthattheTAsmighthaveforgottentoenterrequestsinsomecases,itwasassumedthatthedistri-butionofsucherrorswasroughlyequivalentacrosstreatments.ThiswouldhavebeenthecaseevenifoneTAhadforgottentoenterallrequests,becauseeachTAhadaboutthesamenumberofstudentsineachcondition.Therefore,although68

PAGE 84

Figure10.Formusedtologrequestsfordeadlineextensions.thesedatawerelessreliablethanthedatabasedonsubmissiondate,theyweremaintainedforanalysis.3.4.4AssignmentScoresInthecourseware,studentsweretoldexplicitlythattheycouldworkonassign-mentstogetherifdesired.ThisinformationwasgivenwiththecaveatthattheywouldberequiredtowriteHTMLcodeontheposttest.Theywerewarnedthatiftheyworkedtoocloselywithothersanddidnotmakesuretounderstandthema-terialthemselves,itwasverylikelythattheywouldearnamuchlowerscoreontheposttest.Becausethestudentshadthefreedomtocollaborateontheassign-ments,assignmentgradeswereextremelylikelytobedependent.Although,onaverage,studentsreported(seeAppendixP)collaboratingwithfellowclassmatesontheassignmentsonly22%ofthetime,andthattheydid72%oftheassignmentscompletelyontheirown,41students(or25%)indicatedthattheycollaboratedon69

PAGE 85

assignmentswithclassmatesatleast40%ofthetime.Consideringonlythose109individualsnotinself-madestudygroups,studentsreported,onaverage,workingwithclassmates13%ofthetimeandthattheycompleted77%oftheassignmentsalone.However,17(or16%)didreportcollaboratingatleast40%ofthetime.Bothsetsofstudentsreported,onaverage,thattheyrarelyreceivedhelpfromeitherthecourseofcialsoranyotheroutsidesources.Still,thelikelihoodofdependentassignmentgradesandthefactthatANOVAandMANOVAareverysensitivetovi-olationsofthisassumption,meantthatsuchmethodsofanalysiswerenottenable.Notethatthisdependencealsomightbereectedintheachievement,preference,andprocrastinationscores,albeittoalesserextent.Implicationsfordataanalysisarediscussedinchapter4.Itshouldbenotedthat4oftheindividualsinstudygroupsreportedhavingcompletedmoreassignmentsontheirownthantheyactuallysubmitted,asdid14oftheindividualsnotinstudygroups.Distributedinroughlyequalnumbersacrossalltreatments,7studentswereinR,6wereinC,and5inA.Reportedvaluesmightbeinatedforanynumberofreasons.Studentsmayhavecompletedcertainassignmentsaftertheyweredue,andso,didnotbothersubmittingthem.Becausetheinstrumentusedtogatherthisinformationwasnotanonymous,theymighthavefelttheneedtooverreportalittle.Ofmoreconcern,theonlinesubmissiontoolsmightnothaveloggedallsubmissionscorrectly.However,thislatterexplanationislesslikely,becausethelikelihoodthatall18studentsfailedtonoticeareductionintheirpostedgradesandreportitwasverysmall.Whateverthereason,thisoverreportingdoesraisedoubtabouttheaccuracyofthedatagarneredwiththisself-reportinstrument.Inanycase,theassignmentscoresthemselvesstillwerevaluable.Forexam-ple,statisticallysignicantcorrelationsbetweenthemandtheachievementscoresprovidedevidenceoftheconcurrentvalidityofthelatter.Also,theywereaperformance-basedmeasurethataccountedfor60%ofthestudents'gradesintheHTMLpor-tionofEdTech.Abriefdiscussionofthereliabilityandfacevalidity,atleast,oftheassignmentscoresseemsappropriate.Therewereeightassignments.Thegoalofthersttwowastoacquaintstu-dentswiththedevelopmentprocessasspeciedinobjectives1-2(seeAppendixD),sotheyweregiventheHTMLsourcecode.Thegoalsofassignments3-8werealignedcloselywithobjectives5-10,respectively.Objectives3-4weremoregen-eralandimplicitlyincludedinallassignments.Foramoredetaileddescriptionofeachassignment,seeAppendixC.Forassignmentdetailsprovidedontheaccom-panyingCD-ROM,seeAppendixU.StudentssubmittedassignmentsbyrstclickingtheappropriateSubmitbuttonintheassignmentlistprovidedwiththecourseware(seeFigure3).ThentheylledinaformliketheoneinFigure11togainaccesstothesubmissionsoftware.Next,theypastedtheirHTMLsourcecodeintoaformliketheoneinFigure12.Finally,70

PAGE 86

Figure11.Formusedbystudentstoaccesssubmissionsoftware.theyreceivedaconrmationmessageonlineliketheoneinFigure13aswellasviae-mail.E-mailedconrmationsincludedtheirname,theexactdateoftheirsubmission,andacompletecopyofthesourcecodereceived.Allassignmentinformationwasmaintainedinadatabasewithrestrictedaccess.TAswerenotabletojeopardizethestudybyallowingstudentstosubmitworklateforcredit.TAsgradedassignmentsusingonline,password-protectedtools,whichtheywereabletoaccessconvenientlyfromlinksincludedwiththeirversionofthecourseware.AfterindicatingtheiridentityusingtheforminFigure14,theyweresuppliedwithaformsimilartotheoneinFigure15,whichcontainedalistofstu-dentswhoseassignmentshadyettobegraded.Afterselectingoneoftheassign-ments,saytherstone,theyweregivenaformcontainingtheassignmentrubricsliketheoneinFigure16.Also,anotherbrowserwindowwasopened,containingthestudent'swork(seeFigure17).Theyhadtheoptionintherubricswindowofeitherclickingonthecheckboxestotheleftofthoserubricsthestudentfullled,orclickingdirectlyonthegraphicalrepresentationoftherubrics.Clickingonthecheckmarksinthegraphicontheleft,causedcheckmarkstobeenteredintotherubricboxesontheright.Clickingonthex'sinthegraphiccausedthecheckmarkstoberemoved.TAsalsohadtheoptionofllinginapersonalmessageinthe71

PAGE 87

Figure12.Formusedbystudentstosubmitassignments.72

PAGE 88

Figure13.Conrmationmessageforsuccessfulsubmission.suppliedtextboxbeforeclickingthebuttononthebottomright,whichsavedthegradeontheserverande-mailedacopytothestudent.Foranexamplee-mailmessagesenttoastudent,seeFigure18.Finally,theTAscouldclickthelinkpro-videdwiththerubric(seeFigure16),iftheywishedtoviewthesolutioncodefortheassignmentinathirdwindowliketheoneinFigure19.Inordertodeterminethevalidityoftheassignments,anHTMLexpertwasgiventheletterinAppendixNalongwithaccesstoalloftheonlinerubricsusedbytheTAs.Foreachrubric,theexpertindicatedwhichobjectivecategoryitmeasured,sometimeslistingmorethanone.Ifarubricdidnotappeartomeasureanyoftheobjectives,theexpertwasaskedtoenteranN.Also,afterconsideringalloftherubricsforagivenassignment,theexpertwasaskedtolistimplicitobjectivesmea-suredbytheassignmentbutnotaddresseddirectlybyanyoftheindividualrubrics.Basedontheexpert'sresponses,eachrubricmeasuredatleastoneoftheobjec-tives,andallrubricsweredistributedfairlywellacrossallobjectives(seeTable8).Itisinterestingtonotethattheweightingidentiedbytheexpertactuallymatchedthealternativeweightingproposedfortheposttestandretentiontestitemsbetterthanitdidtheoriginalweighting.Inanycase,theassignmentsappeartopossesscontentvalidity,andcorrelationsbetweenassignmentscoresandachievementtestscoresprovideevidenceofconcurrentvalidity.Inaddition,theassignmentscoreswerereliablewith.87afterapplyingbonusesandpenalties.Dataoninterraterreliabilitywerenotcollected.73

PAGE 89

Figure14.FormusedbyTAstoaccessgradingsoftware.Table8.ObjectivesMeasuredbyRubricsandAchievementItems Objective Measure12345678910 AssignmentRubrics61024231329455PosttestItems2429261166666RetentionTestItems5529291055555OriginalWeighting461315101010101010 Note.Allvaluesaregiveninpercentagesandroundedtothenearestwholevalue.74

PAGE 90

Figure15.ListofassignmentstobegradedforagivenTA.75

PAGE 91

Figure16.GradingrubricusedbyTAsforassignment1. Figure17.Assignment1foragivenstudent.76

PAGE 92

Subject:RE:YourGradeonAssignment1Randall,Yousuccessfullyaccomplished5of9itemsonthegradingrubricforthisassignment.Therefore,youearned2outof4points.However,becauseyouturnedyourworkinafter03/24/2000,thisassignmentisworthnopoints,anda0hasbeenrecordedasyourgrade.STUDENT:RandallAaronASSIGNMENT:1GRADERECORDED:0Youmissedthefollowingitems:-Yournamedoesnotappearinthebrowserwindow.-Thetextforyournameisnotbigger.-Thetextforyournameisnotdarkblue.-Thebackgroundcolorisnotpaleyellow.AdditionalComments:Nicejobonthelinks! Figure18.Gradee-mailedtostudent.77

PAGE 93

Figure19.Solutionsourcecodeforassignment1.78

PAGE 94

Chapter4ResultsDatawereanalyzedinfourphases.First,thereliabilityoftheretentiontestwasestimatedbasedonposttestdataandcurveswereappliedtostudents'HTMLgradesinordertocounteranypossibletreatmenteffectsonstudents'gradesinEdTech.Next,thepacingpreferencedatawereanalyzedusingANOVA.Then,randomizationtestswereusedtoanalyzetheprocrastinationdata.Inthenalphase,achievementdataalsowereanalyzedusingrandomizationtests.BecauseANOVAisnotrobusttoviolationsoftheindependenceassumption,stepsweretakentoreducetheeffectsofdependencecausedbystudentsworkingtogetherinself-madestudygroups.Asdiscussedpreviously,studentswereaskedtoreportthemembersofsuchgroupspriortothetreatmentinterval,andmembersofthesamegroupwereallassignedtothesametreatment.Followingthetreatmentinterval,studentswereaskedagaintoreportthecompositionoftheirstudygroups,andfouradditionalgroupswereidentied.Thescoresofstudentsinthese22studygroupswereaveragedpriortoanalysiswithANOVA.Becauserandomizationtestsaremostjustiedwhenobservationsarerandomlyassignedtotreatmentspriortothetreatmentinterval,onlythe18studygroups1whichinformedassignmenttotreatmentconditionswerereplacedwithaveragescorespriortoanalyzingtheachievementandprocrastinationdata.4.1EstimationofRetentionTestReliabilityandCurvingofGradesTheposttestdatawereanalyzedbeforetheretentiontestwasadministered.Multiplechoiceresponseswerescoredautomatically,andtheessayswereratedbyhand.Asreportedinsection3.4.1.4,intraraterreliabilityfortheessaywas.99(p.0001)andCronbach'salphawas.89forall48items.Consideringonlythose9multiplechoiceitemsthatmeasuredobjectivescomparabletotheonesmea-suredbytheitemsontheretentiontest,alongwiththe12essayitems,Cronbach's 1Therewere19studygroupsinitially,butbothmembersofgroup2weredroppedfromthestudyduetomissingdata,reducingthenumberofinitialstudygroupsto18.79

PAGE 95

alphawas.69fortheoriginalweightingscheme2proposedfortheretentiontestand.86fortheadoptedalternativeweightingschemeinwhichallitemshadequalweight.Therefore,itwasanticipatedthatthe21itemsontheretentiontestwouldpossesssufcientreliabilityandnochangeswereconsidered.Infact,was.85fortheactualretentiontestandintraraterreliabilityfortheessayportionwas1.00(p.0001).Duringthisphaseofdataanalysis,grades3fortheHTMLportionofthecoursewerecalculatedforeachstudent.Consideringonlythe58studentsineachtreat-mentwhohadsubmittedatleastoneassignmentorhadtakentheposttest,stu-dentsinR,onaverage,earned28.65pointsoutof60,studentsinCearned26.96points,andstudentsinAearned20.28points.Becausedifferencesintheseaver-agescoresmighthavebeenduetoassignmenttoaparticulartreatment,2pointswereaddedtostudenttotalsinCand8pointswereaddedtostudenttotalsinA.Next,becausethetotalswerelowerthananticipatedingeneral,an11pointcurvewasaddedtoallstudenttotals.However,nototalwasallowedtoexceed60points.Extracreditwasearnedonlybycompletingtheoptionalpretest(5points)andpac-ingpreferencesurvey(5points).SeeTable25foralistingofstudentgradesbeforeandafterapplyingthecurveandextracreditpoints.Finally,consideronlythepor-tionofEdTech'sExam2whichcontainedtheretentiontestquestions.Onaverage,outof12points,studentsinRearned4.54,studentsinCearned5.17,andstu-dentsinAearned4.25.Again,toreducepossibletreatmenteffectsonstudents'grades,1pointwasaddedtotheExam2scoresforeachstudentingroupsRandA.4.2AnalysisofPacingPreferenceDataThesecondphaseofdataanalysisfocusedonthepacingpreferencedatacol-lectedonMarch28.Thegoalofeachquestionwastodeterminethestudent'spreferenceforteacher-pacingversusstudent-pacing.Forhalfoftheitemsare-sponseof1representedthehighestpreferenceforself-pacing.Fortheotherhalf,aresponseof5indicatedsuchapreference.Thedirectionofthescaleforeachitemwasdeterminedrandomly.Thepurposeofthisinversionwastoreducethebiasoftheinstrumentandtomakeiteasytodetectanyresponsepatternsthatmightindicateeachquestionwasnotconsideredthoughtfully.Becausenostu-dentsreportedallonesorallves,allresponseswereretainedforanalysis. 2Recallthatessayitemswereenteredaseither0(incorrect)or.25(correct)underthisscheme.3Forgradingpurposes,posttestandretentiontestaverageswerecalculatedbasedontheoriginalweightingschemesproposed.80

PAGE 96

Table9.DescriptiveStatisticsforPacingPreferenceData TreatmentnMSDSkewnessKurtosisMinMax R462.810.640.770.681.624.50C452.820.590.14-0.541.754.12A402.800.580.430.681.624.50 Theratingscaleemployedyieldedroughlyintervaldatawith.55.There-fore,itwasreasonabletocalculateanaveragescoreforalleightresponses,afterinvertingitems3,5,6,and7.Noticethatthelowertheaveragescore,thehigherthestudent'spreferenceforteacher-pacing.SeeTable24foralistingofresponsespriortoinversionandTable25foralistingofaveragescores.Stem-and-leafplotsforthescoresineachgroupappearedunimodal,anddescriptivestatistics(seeTa-ble9)gavefurtherevidenceofroughlynormaldistributionswithskewnessvaluesin[.14,.77]andkurtosisvaluesin[-0.54,.68].Groupsappearedtohaveequalvari-ance.However,theindependenceassumptionmayhavebeenviolated,becausestudentswereallowedtoworkonassignmentstogether.Workingwithfellowstu-dentsunderconditionsofself-pacingmayhavecausedstudentsinthesamestudygrouptoformsimilaropinionsaboutthisteachingparadigm,soasmentionedpre-viously,scoresforstudentsinstudygroupswereaveragedinanattempttomitigatesucheffects.Therewerenozscoresgreaterthanorequaltothree,andhence,novaluesconsideredtobepotentialoutliers.Asanticipated,pacingpreferencedidnotdiffersignicantlyacrosstreatments(F(2,128)0.02).This,coupledwithaneffectsizeof0.02,indicatedthatanytreatmenteffectwassmallatbestandthatdifferencesinpreferencewereofneitherstatisticalnorpracticalsignicance.4.3AnalysisofProcrastinationDataTheprocrastinationdataweretobeanalyzedusingMANOVA.However,thevariablemeasuringthenumberofrequestsfordeadlineextensions,callitx1,wasextremelyleptokurticandpositivelyskewedwithkurtosisvaluesin[12.57,40.45]andskewnessvaluesin[3.48,6.26]forthetreatmentsamples.Outof131datapoints,only11hadnonzeroentriesforx1.Therefore,itwasnotincludedinanyparametricanalyses.Nonparametricproceduressuchasthechi-squaretestforindependencealsowerenotappropriatewithsofewnonzeroentries.Althoughnodenitiveconclusionscanbedrawn,itisinterestingtonotethattwostudentsinRrequestedextensionsasdidveinCandfourinA.Now,considertheRDTR,whichmeasuredtheaverageamountofprocrastinationofeachdatapointforalleightassignments,andcallitx2.Becausetheunivariatenormalityofeachofthe81

PAGE 97

Table10.DescriptiveStatisticsforProcrastinationLevelData TreatmentnMSDSkewnessKurtosisMinMax AllstudentsR487.926.30-0.02-1.08-5.0016.75C473.805.391.060.75-5.7516.75A417.126.010.49-1.03-1.2516.75 StudentswhosubmittedatleastoneassignmentR406.155.360.07-0.65-5.0016.12C432.603.800.41-0.34-5.7511.25A345.134.470.720.21-1.2516.12 contributingvariablesisanecessaryconditionofmultivariatenormality(Stevens,1986)andbecauseoftheradicaldeparturefromnormalityofx1,theconditionofmultivariatenormalitywithrespecttox1andx2wasnotmet.Although,MANOVAisrobustformoderatedepartures,thelackofnormalitywasextremeinthiscase.Therefore,MANOVAwasnottenable,andANOVAwasconsideredasthenextreasonableoptionforanalyzingx2.Nopotentialoutlierswithzscoresgreaterthanorequaltothreewereidenti-edforthesampledistributionsofx2.Unfortunately,theplatykurticnatureofthedistributionsfortreatmentsRandA(seetoppanelofTable10)meantareductioninpower(Stevens,1990).Acloserinspectionofthehistograms(seeFigure20)revealedthatthesevaluesweredue,inpart,tothefactthatanumberofstudentsineachtreatmentsubmittednoassignmentsduringthetreatmentintervalandhadaverageprocrastinationlevelsof16.75.Consideringonlythosestudentswhosub-mittedatleastoneassignment,yieldedmorenormalskewnessandkurtosisvalues(seebottompanelofTable10),althoughgroupRwasstillslightlyplatykurtic.Ofthe22individualswhosubmittednoassignments,11wereinR,4wereinC,and7wereinA.Aftertakingstudygroupsintoaccount,the11inRwerereducedto8.These22individualsweredistributedacrossallTAsinroughlyequalnum-berswithTAs1-5having4,4,3,7,and4ofthem.Althoughonemightarguethatthesestudentsdidnotreallyparticipateinthestudyandthattheirscoresshouldberemovedfromanalysis,italsowaspossiblethattheydidactuallyparticipatebutprocrastinatedtoolongtoseethevalueofcompletinganyoftheassignments.Thefactthatthenumberofsuchstudentsineachtreatmentdifferedgavesomeevidenceinsupportofthislattertheory.Ofcourse,bothexplanationscouldbetrueinpart.Inanycase,knowingthatonetreatmentmightcausemorestudentstodisengagewasdeemedvaluableinformation,anditwasreasonablethatthemeanbehigherforgroupswithmoresuchstudents.Inaddition,these22studentswere82

PAGE 98

0 2 4 6 8 -10 0 10 20 R 0 2 4 6 8 -10 0 10 20 R -10 0 10 20 C -10 0 10 20 C -10 0 10 20 AAverage Procrastination Level Per Assignment (days)Number of Students & -10 0 10 20 AAverage Procrastination Level Per Assignment (days)Number of Students & -10 0 10 20 AAverage Procrastination Level Per Assignment (days)Number of Students & Figure20.Procrastinationhistograms.Procrastinationlevelisplot-tedforallstudentsrelativetoassignmentduedates.Avalueof-5indicatesthat,onaverage,thestudentsubmittedassignments5daysearly.Totalsareaccumulatedbasedonthenextlowerwholeday.Forexample,valuesintherange[1,2)areaccumulatedinthetotalfor1.Thesmoothnormalcurvesprovidevisualreferencesfordeterminingdeparturesfromnormality.nothomogeneouswithrespecttoExam1scoresinEdTech.Onaverage,stu-dentsinRearned47.64(SD18.34),studentsinCearned29.75(SD34.41),andstudentsinAearned51.43(SD11.22).Simplyremovingthisdatalikelywouldhaveimpactedthetreatmentsdifferentially.Therefore,itwasretained,andconsequently,thenormalityassumptionwaslikelyviolated.Becauseofthereduc-tioninpowerassociatedwithusingANOVAonplatykurticdata(Stevens,1990),randomizationtestswereemployed.Therandomizationmodelisjustiedinthiscase,becauseobservationswererandomlyassignedtotreatments.RatherthanrelyonatableofFvalueswhichassumesthatdataarenormallydistributed,randomizationtestsprovideameansofcreatingadistributionbasedonthedataathand.Italsoallowstheresearcherthefreedom“tochooseateststatisticthatisappropriatefortheparticularsituationbeingconsidered.”(Manly,1997,p.23)First,areasonableteststatistic,saythedifferencebetweenthemeansoftwogroups,isselectedandcalculatedfortheoriginaldata.Callthiscalculations0.Thentheoriginaldataarepermuted,sothatanygivenobservation,whichoncebelongedtoonegroup,maynowbelongtoadifferentgroup.Theteststatisticisthenrecalculatedforthisnewpermutation.Theprocessofpermutingthedataandrecalculatingtheteststatisticcontinues,sothatadistributionofteststatisticsisbuilt,callitS.Afterareasonablenumberofrepe-titions,s0iscomparedtotheteststatisticsinS.Ifthenullhypothesisistrue,thenallpermutationsofthedatashouldbeequallylikelyands0willbeatypicalvalueinS.Ifthatisnotthecaseands0appearstobeextreme,thenthereisevidencethatthenullhypothesisshouldberejectedinfavorofthealternative.Determiningthesignicancelevelofs0(oritsp-value)issimplyamatterofcalculatingwhatpercentageofthevaluesinSaregreaterthanorequaltos0.Intheidealsituation,83

PAGE 99

allpossiblepermutationsofthedatawouldbeconsidered.However,evenforrel-ativelysmallN,thetotalnumberofpermutationsisgenerallytoolargeformoderncomputerstohandle,andso,areasonablesubsetofpermutationsisconsideredinordertoapproximatethesignicancelevelofs0.Manly(1997)suggeststhat,fornominal.05,teststatisticsforatleast999permutationsshouldbecalculated,inadditiontos0.Fornominal.01,atleast4999additionalpermutationsshouldbeconsidered.BecausetheSASversionavailableatthetimethisreportwaswrittendidnotcontaingeneralpurposefunctionsforrunningrandomizationteststhatwereeasilymodiedtoincorporatetheteststatisticsofinterest,Ccodewaswrittentoanalyzetheprocrastinationdataviatherandomizationmodeldescribedabove.Eachfunc-tionwasthoroughlytestedasitwaswritten.Inaddition,therandomizationportionsofthecodeweretestedusingtheVitaminEdatasetdescribedbyGood(2000,p.4-8).FollowingGood,theteststatisticemployedwasthesumofthevaluesoftheculturestreatedwithVitaminE.Inagreementwithhiss0349andhispreciselycalculatedp-valueof.05,thecodereporteds0349andcloselyapproximatedthep-valueat.0502,basedon999999additionalrandomizations.Theprocras-tinationdatawereanalyzedusingalternativeteststatistics,whicharediscussedbelow.Toreviewtheactualcode,seeAppendixS.Also,refertoAppendixU,whichdescribesthematerialprovidedontheaccompanyingCD-ROM.First,anomnibusteststatisticwasselectedtoanalyzethe136datapointsbasedonthedifferencesbetweenthemeansforgroupsR,C,andA.Itwasex-pectedthattheaverageprocrastinationlevelwouldbehigherforgroupR( Pr)thanforgroupsC( Pc)andA( Pa),anditwasunclearhowaveragelevelswouldberelatedforgroupsCandA.Therefore,theomnibustest Pr Pc' Pr Pa')(( Pc Pa((whichincorporatedtwoone-sidedtestsandonetwo-sidedtest,seemedappropri-ate.Ityieldedstatisticallysignicantresultswiths08.23,p.0012,infavorofthealternativehypothesisthatthetreatmentconditiondidhaveasignicantef-fectonprocrastinationlevel.Therefore,additionaltestswereconductedinordertodeterminewhichgroupsdifferedsignicantlyfromoneanother.Totestthehypoth-esisthattheaverageprocrastinationlevelforgroupRwashigherthantheaveragelevelsforgroupsCandA,theone-sidedteststatistics, Pr Pcand Pr Pa,wereused.However,becausethedirectionofanydifferenceinlevelsbetweengroupsCandAwasuncertain,thetwo-sidedteststatistic,(( Pc Pa((,wasused.Afterapply-ingaBonferroniadjustmentofafactorofthreeforthethreetestsconsidered,thenominal.05wasadjustedto.017.Thedataforthegroupsbeingcompared84

PAGE 100

werepermutedrandomly999999additionaltimes4accordingtothemethodpro-posedbyManly(1997,p.90)andillustratedinthepermutefunctioninAppendixS.Although,theteststatistic Pr Pawasnotsignicantwiths00.80,p.2710, Pr Pcwassignicantwiths04.11,p.0005aswas(( Pc Pa((withs03.31,p.0078.Theeffectsizeswered0.13,d0.70,andd0.58,respectively,rangingfromquitesmalltomoderatelylarge.Thus,therewassomeevidencethatstudentsinAprocrastinatedmorethanstudentsinCandevenstrongerevidencethatstudentsinRprocrastinatedmorethanstudentsinC.Figure21illustratesthefactthatstudentsinCmaintainedamoreconsistentsubmissionratethroughoutthetreatmentinterval,whilestudentsinRandAsub-mittedlessassignmentsduringtheintervalwithsteeperincreasesinsubmissionratesattheend.ThisscallopedeffectwasmostevidentforstudentsinR,whoseincreaseinassignmentsubmissionratesbetweendays27and32gaveevidenceoftheirmassedlearningneartheendofthetreatmentinterval.Itwasalsointerest-ingtonotethatallgroupsexperiencedaplateaubetweendays19and27,wherelowersubmissionratescoincidedwithSpringBreak.DifferencesbetweengroupsRandCwerearguablypracticalaswell,withstudentsinRturninginassignments,onaverage,aboutfourdays(4.12)laterthanstudentsinC.Similarly,studentsinAsubmittedassignmentsmorethanthreedays(3.32)laterthanstudentsinC.Fig-ure22illustratesthefactthatthispatternheldforallbutthelasttwoassignments.Recallthatthedeadlineforthesetworeallywasmoreofanabsolutedeadlineforallgroups.Nostudentswereallowedtosubmitanyassignmentsforcreditafterthisnalduedate,althoughstudentsinCcouldstillearnbonuspointsforsubmittingtheirworkearly.ItwasexpectedthatprocrastinationlevelswouldbesimilarforstudentsinCandA,andthatstudentsinRwouldprocrastinatesignicantlymorethanstudentsinbothCandA.However,groupmeansandFigures21and22indicatethatpro-crastinationlevelsforAweremoresimilartoRthantoC.AlthoughstudentsinAappearedtostartoutwithsubmissionratesmorelikethoseofstudentsinC,CandAquicklydivergedandratesforAalignedmorewithratesforR.Figure23,whichillustratesthatfewerstudentsinAcompletedeachassignment,providesevidenceofapossibleexplanation.Formissingassignmentscores,recallthatthedayafterthecloseofthestudywasenteredastheofcialsubmissiondate.Thishadtheef-fectofinatingtheprocrastinationlevelsforstudentswhodidnotsubmittheirwork,andhence,levelswereinatedmostforstudentsinA.Infact,Figure24showsthat,whenonlythoseassignmentsactuallysubmittedwereconsidered,procrastinationlevelsweremoresimilarforAandCthanforAandR.Still,itwasnecessaryto 4Atotalof1,000,000permutationswereconsideredforallrandomizationsperformedinthisstudyinordertoobtainp-valueswithprecisiontothethousandthsplace.Comparingp-valuesobtainedfor1,000,000permutationstothoseobtainedfor2,000,000permutationsyieldeductua-tionsintheten-thousandthsplaceonly.85

PAGE 101

0 20 40 60 80 100 0 5 10 15 20 25 30 Day of Treatment IntervalCumulative Assignments Submitted (%) Ideal R C A Figure21.Cumulativepercentageofassignmentssubmittedeachday.R,C,andAshowthesubmissionpatternsforthecorrespond-ingtreatments.Idealshowsthesubmissionpatternexpected,ifallstudentswouldhavesubmittedallassignmentssteadilyoverthetreatmentinterval.Thefourdark,verticallinesrepresentcommonduedatesacrosstreatments.Thetwolight,verticallinesdelimitthestartandendofSpringBreak.maintaininformationaboutthenumberofassignmentssubmittedbyagivenstu-dentaswellashowlatetheywere.Otherwisetheprocrastinationlevelofastudentwhosubmittedonlyoneassignmentbutturneditinonitsduedatewouldbethesameasthatofastudentwhosubmittedallassignmentsontheirrespectiveduedates.Clearly,thesetwostudentswoulddifferintheamountanddistributionovertimeoftheworktheydidaswellasinhowmuchtheylearned.Therefore,noalterationsweremadeinthemethodofhandlingmissingprocrastinationdata.4.4AnalysisofAchievementDataAsdiscussedinsection3.4.1.4,becausethepretestscoreswereunreliable,onlytheposttestandretentiontestscoreswereanalyzed.Theachievementscoresweretobeanalyzedusingrepeatedmeasureswithtreatmentasabetweenfactorandtimeasawithinfactor.However,thisanalyticapproachassumesmultivari-atenormality,whichinturn,requiresthateachofthemeasuresforeachofthe86

PAGE 102

0 5 10 15 20 25 30 35 1 2 3 4 5 6 7 8 AssignmentDay SubmittedR C A p d b Figure22.Procrastinationlevelwithestimatedmissingdata.Miss-ingassignmentswererecordedashavingbeensubmittedthedayafterthecloseofthestudy,whichlasted32days.R,C,andAarethetreatments,disthecommonduedateacrosstreatments,andpandbarethepenaltyandbonusintervalsforstudentsinC.samplesbeunivariatenormal.Unfortunately,thedescriptivestatisticsinTable11indicatethatthedistributionsofposttestandretentiontestscorestendedtobeplatykurtic,withthreeofsixkurtosisvalueslessthan-0.5.ThehistogramsinFig-ure25illustratethisgraphically.Becausetheunivariatenormalityassumptionwaslikelyviolatedandplatykurtosishasbeenassociatedwithareductioninpower,theachievementscoreswereanalyzedusingrandomizationtests.Inordertodeterminewhetherornottherewasaninteractionbetweentimeandtreatment,ateststatisticwasselectedbasedondifferencescores,whichwerecal-culatedbysubtractingtheposttestscoresfromtheretentiontestscoresforeachdatapoint.ThisyieldedscoresthatwereslightlyplatykurticforgroupCandposi-tivelyskewedandleptokurticforgroupA,asevidencedbythedescriptivestatisticsinTable11andthehistogramsinFigure26.Oneextremedifferenceof47.62forstudent#288ingroupAaccountedforsomeofthedeparturefromnormality.Re-movingthisvalueyieldedameanof7.47,standarddeviationof10.01,skewnessof0.43andkurtosisof0.65.However,Good(2000)recommendsretainingalloftheoriginaldatainordertoavoidreducingthepoweroftherandomizationtest.Therefore,thedataassociatedwithstudent#288werenotdiscarded.87

PAGE 103

30 40 50 60 70 80 90 1 2 3 4 5 6 7 8 AssignmentStudents Who Submitted Assignment (%)R C A Figure23.Percentageofstudentssubmittingassignments.UsingCcodesimilartothatusedfortheprocrastinationdata,arandomizationtestwasrstusedtoconductanomnibustestonthedifferencescores.Letting Dr, Dc,and DabethemeandifferencescoresforstudentsingroupsR,C,andA,respectively,theomnibusteststatistics2 Dr2 Dcwasselectedbasedonthefollowingrationale.BecausestudentsingroupRprocrastinatedthemost,andhence,weremostlikelytoengageinmassedlearning,itwasexpectedthattheywouldforgetthemostandthat Drwouldbelargest.Followingasimilarargumentandbasedontheresultsfromanalyzingtheprocrastinationdata,itwasexpectedthatthenextlargestmeandifferencewouldbe Da,sothat Dr Da Dc.Asthedistancebetween Dr, Da,and Dcincreases,sodoesthevalueofs Dr Da' Dr Dc' Da Dc+Aftercancelingafewterms,thisyieldss2 Dr2 Dc,ors2 Dr Dc.Interest-ingly,thisomnibustestreducestoanexaminationofthemagnitudeofthedistancebetweenthetwopointsinthedifferencedimensionthatwereexpectedtobefar-thestfromoneanother.Thisisreasonableconsideringthefactthatthepurposeoftheomnibustestistodeterminewhetherornotalargeenoughoveralldiffer-enceexistsbetweengroupstowarrantpairwisecomparisons.Arandomizationtestbasedonsindicatedthatthevalueofsfortheoriginaldata,s09.59wasex-tremeenough(p.0199)tosuggestthatthenullhypothesisberejectedinfavorof88

PAGE 104

0 5 10 15 20 25 30 35 1 2 3 4 5 6 7 8 AssignmentDay SubmittedR C A p d b Figure24.Procrastinationlevelwithnomissingdata.Missingas-signmentswerenotincludedwhencalculatingaveragesubmissiondays.Hence,theaveragesforA,andtoalesserextentforR,donotreectthefactthatfewerstudentsactuallysubmittedeachassign-ment.R,C,andAarethetreatments,disthecommonduedateacrosstreatments,andpandbarethepenaltyandbonusintervalsforstudentsinC.89

PAGE 105

thealternativethatthetreatmentconditionhadsomeeffectonhowmuchstudentsforgotbetweentakingtheposttestandtheretentiontest.Althoughtheomnibustestabovefocusedonwhetherornottherewasanoveralldifferencebetweenthedifferencescoresinthedirectionindicatedbytheprocrastinationresults,recallthatprocrastinationlevelsdidnotdiffersignicantlybetweengroupsRandA.Usinganalternativeomnibustest,wheres-, Dr Da, Dr Dc' Da Dcs0stillwas9.59andextremeenough(p.0411)towarrantfurtherinvestigationofthedifferencescores.Inordertodeterminewhichdifferencescoresdifferedsignicantly,threesep-araterandomizationtestswereperformed,oneforeachofthetests Dr Da, Dr Dc,and Da Dc.ABonferroniadjustmentbyafactorofthree,convertedthenominalalphafrom.05to.0167.Althoughnoneoftheindividualtests,strictlyspeaking,wasstatisticallysignicantforthisadjustedalpha,thetest Dr Dcwasthemostsignicantwiths04.79,p.0190.Fortest Dr Da,s03.18,p.0891,andfortest Da Dc,s01.61,p.2613.Theeffectsizesfortheselattermeandifferenceswerebothfairlysmall,withd0.29forgroupsRandAandd0.14forgroupsAandC.Witharoughlymediumeffectsizeofd0.43andadifferencebetweenaveragedifferencescoresofapproximatelyone-halfofalettergrade,differencesbetweengroupsRandClikelyaccountedforthesigni-canceoftheomnibustest.Infact,ifthetest Dr Dawerediscardedbasedonthefactthattheprocrastinationlevelsforthesetwogroupsdidnotdiffersignicantly,thentheBonferroniadjustmentwouldchangethenominalalphato.025insteadof.0167,inwhichcasethetest Dr Dcwouldbestatisticallysignicant.Figure27givesfurthervisualevidenceofapossibleinteractionbetweentimeandtreatmentforgroupsRandC.Nowlet PTgand RTgbetheaverageposttestandretentiontestscoresforstudentsingroupg,respectively,whereg"/.rca0.Consideringtheposttestscoresalone,thetwo-sided,omnibusrandomizationtest PTr PTc2 PTr PTa2 PTc PTa2indicatedthat,asexpected,therewasnosignicantdifferencebetweenachieve-mentscoreswiths0114.15,p.1995.Also,theeffectsizeforgroupsCandRwasd0.05.However,studentsinAearnedsubstantiallylowerposttestscores,onaverage.Infact,themeansforgroupsCandAdifferedby8.09pointsornearlyonelettergrade,andtheeffectsizewasd0.36.Also,theeffectsized0.3090

PAGE 106

forgroupsRandAwasrelativelyhighaswasthe6.86pointdifferencebetweenmeans.Next,theretentiontestscoreswereconsideredaloneusingtheone-sidedom-nibustest2 RTc2 RTr.Thetestwasderivedfrom RTc RTa' RTc RTr' RTa RTrwhichwasformulatedbasedonthepredictionthat RTc RTa RTr,because Pr Pa Pc.Statisticallyspeaking,thetreatmentconditionhadnosignicanteffectontheretentiontestscores,s012.06,p.0845.Effectsizesforthere-tentionscoresweresmallerthanexpected,andinonecase,theeffectwasactuallyintheoppositedirection.Specically,forgroupsCandR,wherealargeeffectwasexpected,asmalleffectwasfoundwithd0.28.ForgroupsCandA,wherelessofaneffectwasexpected,amediumeffectwasfoundwithd0.48Evenmoresurprising,forgroupsAandR,wherealargeeffectwaspredictedinitially,asmalleffectwasfoundintheoppositedirectionwithd-0.17.Recall,however,thatprocrastinationlevelsalsodidnotdiffersignicantlybetweenstudentsinRandAasinitiallypredicted,soasmallinsignicanteffectineitherdirectionmaynotbethatstrange.TherewassomeconcernpriortoconductingthestudythatstudentsinC,whoearnedalargenumberofbonuspointsandhighmarksonallassignments,mightbelessmotivatedtoperformwellontheposttest.The8bonuspointsstudentsinCwereabletoearncouldonlybeappliedtowardtheHTMLportionofEdTech,whichincludedaposttestworth24points.Thus,studentsinCcouldhavealreadyearnedone-thirdofthepointsneededtoget100%ontheposttestbeforeeventakingit.Theonlystudent,#256,whoearnedenoughbonuspoints(4)aswellasahighenoughassignmentscore(111%)toraiseconcernactuallyearned98%ontheposttest.Therefore,bonuspointsdidnotseemtoaffectperformanceontheposttestandnoactionwastaken.Consideringbothbonusandpenaltypoints,itisinterestingtonotethat,onaverage,studentsinRacquired-.56points,studentsinCacquired-4.98points,andstudentsinAacquired-5.71points.Although,bonuspointsearnedthroughoutEdTechmayhavehadasimilareffectontheretentiontest,whichwasasubsetofEdTech'sExam2,thepotentialforearningthesebonuspointswasevenlydistributedacrosstreatments.Therefore,theyposedlittleornothreattointernalvalidity.91

PAGE 107

Table11.DescriptiveUnivariateStatisticsforAchievementData TreatmentnMSDSkewnessKurtosisMinMax PretestR4827.1715.030.36-0.230.0055.56C4729.1511.96-0.19-0.310.0055.56A4124.8210.69-0.28-0.730.0044.44 PosttestR4845.9923.940.45-0.9710.1292.86C4747.2222.650.43-0.4011.1698.81A4139.1322.160.58-0.439.5292.86 RetentiontestR4834.3622.420.85-0.094.7695.24C4740.3919.830.03-1.024.7676.19A4130.6821.020.60-0.864.7671.43 DifferencescoresaR4811.6310.410.580.16-5.9540.48C476.8311.820.03-0.53-15.4832.14A418.4511.711.062.37-14.2947.62 aPosttestscoreminusretentiontestscore.92

PAGE 108

0 2 4 6 8 0 50 100 Retention Test Scores (%)Posttest Scores (%)Number of Students & R 0 2 4 6 8 0 50 100 Retention Test Scores (%)Posttest Scores (%)Number of Students & R 0 2 4 6 8 0 50 100 Retention Test Scores (%)Posttest Scores (%)Number of Students & R 0 50 100 C 0 50 100 C 0 50 100 C 0 50 100 A 0 50 100 A 0 50 100 A 0 2 4 6 8 10 12 R 0 2 4 6 8 10 12 R 0 2 4 6 8 10 12 R C C C A A A A Figure25.Achievementhistograms.Totalsareaccumulatedinin-crementsofvepoints.Forexample,scoresintherange[70,75)areaccumulatedin70.Thesmoothnormalcurvesprovidevisualreferencesfordeterminingdeparturesfromnormality.Thecirclesindicatethescoreastudentwouldearnbychance.4.5AnalysisofFreeFormatResponsesInanefforttoacquireinformationonimprovingthecoursewareaswellasadeeperunderstandingofstudyissuesthatmighthaveaffectedoutcomes,students,courseassistants,andinstructorswereaskedtoprovidefreeformatfeedback.Stu-dentswereaskedtoanswerthethreefree-responsequestionsinAppendixPonthedaytheposttestwasadministered.WhenaskedwhattheylikedmostabouttheHTMLlessonsandassignments,responsesfellintothemaincategorieslistedinTable12.Noticethatstudentsweremostlikelytostatethattheybestlikedei-thernothing,therelevant,interestingmaterialandassignments,ortheconvenientcontentondemand.Althoughresponsesinagivencategorygenerallyweredis-tributedfairlywellacrosstreatments,itisinterestingtonotethattwiceasmanystu-dentsinRreportedthattheybestlikedtheCoD.WhenaskedwhattheylikedleastabouttheHTMLlessonsandassignments,themostcommoncomplaintswerethattheinstructionwasnotadequate,thatthematerialandassignmentsweretoohard,andthatanalternativeteachingparadigmwaspreferred.Otherparadigmsmentionedincludedhavinglivelectures,demonstrations,classdiscussions,groupcollaboration,andstructuredclasstimeinacomputerlaboratory.SeeTable1393

PAGE 109

0 2 4 6 8 -20 0 20 40 Difference Scores (%)Number of Students & R 0 2 4 6 8 -20 0 20 40 Difference Scores (%)Number of Students & R -20 0 20 40 C -20 0 20 40 C -20 0 20 40 A -20 0 20 40 A -20 0 20 40 A Figure26.Histogramsforachievementdifferencescores.Totalsareaccumulatedinincrementsoftwopoints.Forexample,scoresintherange[20,22)areaccumulatedin20.Thesmoothnormalcurvesprovidevisualreferencesfordeterminingdeparturesfromnormality.foracompletelistingofresponsecategories.Noticethatresponsesineachcate-goryagainweredistributedfairlywellacrosstreatments.Infact,accordingtotwochi-squaregoodnessofttestsbasedonthenumberofpositiveresponsesandthenumberofnegativeresponsesmadebystudentsineachtreatmentcondition,freeformatresponsesdidnotdiffersignicantlyacrosstreatments.Specically,123.47,p.1765forthepositiveresponses,and126.68,p.0355forthenegativeresponses.AftermakingaBonferroniadjustmentbyafactoroftwo,thenominalalphachangedfrom.05to.025,andso,neitherresponsetypewasstatisticallysignicant.Althoughstudentsgavenearlyfourtimesasmanynegativeresponsesastheydidpositiveones,thegainstheymadeontheirposttestscoresovertheirpretestscoresdidindicatethattheystillwereabletolearnsomeofthematerialsuccessfully.Ofcourse,averageposttestscoresintherange(39,46)indi-catedthattheyalsodidnotlearnasubstantialportionofthematerial,andhence,itisnotsurprisingthatmanyreportedhavingmorenegativefeelingsabouttheexperience.Severalfactorsmayhavecontributedtothereducedeffectivenessofthein-struction,asevidencedbystudentcomments.Forexample,somestudentsre-portedthattheassignmentsweredifcult,timeconsuming,andnotrelevant.Oth-ersreportedthatthecoursewareandsupportfromcoursepersonnelwerenotadequate.Stillothersreportedfeelingthattheywerenottreatedfairly.Providingstudentswithmoretimetocompleteassignmentsandhavingcourseofcialsre-viewallportionsofthecoursewareandmakesuggestionsabouthowtoimproveitpriortogivingittothestudentswouldlikelyreducesomeoftheseproblemssub-stantially.Whenthestudentswereaskeddirectlywhatimprovementstheywouldmaketothecoursewareandassignments,responsesfellintothesevenmaincat-egoriesbelow.94

PAGE 110

20 25 30 35 40 45 50 Pretest Posttest Retention Test Achievement MeasureScore (%) R C A Figure27.Timeandtreatmentinteractionforachievementdata.1.Eitherreducetheamountofworkorgivemoretimetocompleteit.2.Providemoreassistance.3.Augmentcoursematerial.4.Altertheinstructionalparadigmsomewhat.5.Makesoundaccessibleonmoremachinesandeasytomute.6.Provideoptionforprintedmaterial.7.MakeHTMLunitextracreditratherthanrequired.Foralistingoftheexactresponsescountedineachcategoryofsuggestedim-provementsaswellasineachcategoryofpositiveandnegativeresponses,seeAppendixQ.Forcategoryresponsesmadebyeachindividual,seeTable24.Teachingassistantsandfacilitatorswereinterviewedtogetherinacasualset-tingonApril18.Allteachingassistantsexceptoneandbothcoursefacilitatorswerepresentfortheinterview.Theyechoedtherstsuggestionaboveandstatedthatmanystudentswereangryandfrustrated,becausetheyneededmoretimetocompletetheassignments.Thiswasespeciallytrueforlessonve,whichthecourseofcialsindicatedmighthavebeentooadvancedforthestudents.However,theofcialsdidreportthatsomestudentslovedtheexperience,thatfewstudentscametothemforhelp,andthatnostudentsrequestedtoreviewfeedbackontheir95

PAGE 111

Table12.WhatStudentsReportedLikingBest Treatment ResponseCategoriesRCATotal Nothing11172048Prideinownaccomplishments34310Relevant,interestingmaterialandassignments17201047Contentondemand179834Convenientsubmissionprocess2013Exposuretodifferentinstructionalparadigm1034Tutorialrelationshipwithassistant2002Examples,layoutoflessons,andsmallsteps2439Narration1214 Totalpositivearesponses453929113Percentagepositiveresponses403526101 Note.Foragivenstudentandagivencategory,thecategorywascountedonlyonce,evenifthestudentmademorethanonecommentinthatcategory.aTheresponseof“Nothing”isnotincludedinthesetotalsandpercentages,whicharebasedon113totalcategoryresponsesmadebystudents.posttestanswers.Likethestudents,theofcialssuggestedthatfutureversionsofthecoursewareshouldmakeiteasytomutethesound.Also,theyfeltthattheposttestwastoohard,thattextualmaterialshouldaccompanytheaudiomessage,thatthecoursewareshouldhavebeendiscussedmoreinclassatthestartofthestudy,andthatthegradingrubricsdidnotprovideanadequatemeansofdistin-guishingbetweenstudentswhodidexcellentworkfromthosewhojustmettheminimumrequirements.Oneofcialsaidthattheonlinerubricswouldbeeasiertouse,ifa“selectall”optionwasadded.Anotherstatedthatitwouldbenicetobeabletolookatasummaryofassignmentgradestoseehowagivenstudentwasprogressing.Whenaskedwhetherornottheynoticedanydifferencebetweenstu-dentsindifferenttreatments,theofcialsindicatedthatstudentsinAcomplainedalot,studentsinRdidnotcomeforhelpuntilverylate,andthatstudentsinAwerenotabletogetthesamelevelofhelp,becauseassignmentsweredueonFridays.NoneofthecourseofcialshadofcehoursonFridays,andtheyfeltthatmanyofthestudentsdidnotstarttheassignmentsuntiltheprecedingThursdays.UnlikestudentsinA,studentsinRandCcouldgethelpfromtheofcialsonthefollowingMondaysandstillsubmittheirworkforcredit.96

PAGE 112

Table13.WhatStudentsReportedLikingLeast Treatment ResponseCategoriesRCATotal Timeconsuming56516Coursewarelayout25310Instructionnotadequate28292986Assignmentrequirementsnotclear0426Materialandassignmentstoohard11181847Preferalternativeteachingparadigm15221653Narration38718Technicaldifculties711523Submissionprocedure0213Nosound4307Materialnotstimulatingandrelevant45817PreferWYSIWYGa2259Interactionwithteachingassistantsandinstructors711624Deadlines910827Unfairgrading67518Forcedparticipationinstudy1427Feelingoffailureanddefeat83516 Totalnegativebresponses123167145435Percentageofnegativeresponses28383399 Note.Foragivenstudentandagivencategory,thecategorywascountedonlyonce,evenifthestudentmademorethanonecommentinthatcategory.aStandsfor“whatyouseeiswhatyouget”andreferstotheinterfaceprovidedbysoftwareproductssuchasFrontPage.bTheresponseof“Nothing”fromTable12isincludedinthesetotalsandpercentages,whicharebasedon435totalcategoryresponsesmadebystudents.97

PAGE 113

Chapter5DiscussionThecurrenteducationalsystemappearstobeshiftingtowardamorelearner-centeredapproachasevidencedbyincreasedofferingsofdistantcoursesandprograms,bymorecontentbeingdeliveredonlineandondemand,byteachersactingastutors,facilitatorsandcoachesratherthansimplyasdisseminatorsofinformation,bythegrowthofonlinelearningcommunitiessupportedbyALNs,andbyincreasedinterestinperformance-basedoutcomessuchastheattainmentofcompetenciesandthecreationofportfolios.TechnologicaladvancessuchasCAI,CMI,andtheInternethavemadethisparadigmshiftpossiblebygivingpractitionerstheabilitytosupplylearnerswithrichmediaelementsondemandandtotrackeasilytheprogressofindividualstudents.Somespecictoolsincludeoff-the-shelfpackageslikeWebCTandBlackboardaswellasspecial-purposesystemsliketheonedevelopedforthecurrentstudy.Neweducationalapproachesarenowpossibleandsomeolderonesaremorefeasible.Forexample,thelearner-centeredtenetsofPSI,withtheirfocusonself-pacedprogress,masteryofallmaterialbyallstudents,motivationalinstructor-studentinteractions,contentondemand(CoD),andtutorialrelationshipswithse-niorlevelstudents,makeitagoodcandidateforreconsiderationinlightofcurrenteducationaltrends.Itssuccessinraisingtheimmediateacquisitionandlongtermretentionofcontentbystudentsandtheirpreferenceforitovertraditionalinstruc-tionhavebeenwelldocumented.Italsopromotespersonalgrowthbyshiftingtheresponsibilityofknowledgeacquisitiontothestudent.Unfortunately,giventhefreedomtoself-pace,manystudentsprocrastinate,whichpotentiallyleadstohigherdropoutrates,higherworkloadsforgradersattheendoftheterm,andaccordingtothecurrentstudy,reducedlongtermreten-tionofcontent.Theself-pacedcomponentofPSIalsomakesithardertogradestudents'workconsistently,totracktheirperformance,toprovidethemwithsolu-tionsinatimelyfashion,tocatchcheating,andtotthecourseintoaxedtimeframe.Furthermore,onemightarguethatself-pacingfostersanunrealisticworldview.Afterall,isitreallyreasonabletoexpectthatafutureemployerorclientwillwaitindenitelyforareport?Theemployeeorconsultantwhoconsistentlymissesdeadlinesislikelytondlessopportunitiesandchancesforadvancementinthefu-ture.Automationcanhelpalleviatesomeoftheproblemsassociatedwithprocras-98

PAGE 114

tination.Programscanhelpinstructorstrackstudentsandkeepgradingconsistentbyloggingallinformationinadatabaseandbymakingrubricseasytoaccessandupdate.Theycanalsoaidincatchingcheatingbykeepingarecordofallstudentsubmissionsandcomparingnewsubmissionsforclosematcheswithpriorones.Otherproblemsassociatedwithprocrastinationcanbeminimizedbyusingdead-linesandcontingenciestoencouragestudentstomaintainsteadyprogress.Undertheseconditions,completeself-pacingisreducedtohavingtheabilitytoaccessCoD,whichmightbeconsideredalesserformofself-pacing.Thepresentstudyinvestigatedtheeffectsthismighthaveonstudentsatisfaction,procrastination,andachievement.5.1FindingsAlthoughthestudytookplaceinanauthenticsetting,everyattemptwasmadetocontrolforallfactorsotherthanthedeadlinecontingenciesappliedtothethreetreatmentconditions:recommended(R),conditional(C),andabsolute(A)dead-lines.UnlikeearlierstudiesconductedondeadlinecontingenciesunderconditionsofPSI,thisstudyfocusedonthesubmissionofassignmentsratherthanthetakingofquizzes.Themasteryrequirementwasabsentfromalltreatments,sothatitwasnotconfoundedwithself-pacingasinsomepaststudies.Studyndingswerestrengthenedbyrandomlyassigningallstudentsinalargeclasstothetreatmentconditions,havingeachTAadministereachtreatmenttoroughlyequalnumbersofstudents,andbysystematicallyinformingallcourseofcialsandstudentsaboutthenatureofthestudy,theuncertaintyofthendings,andthestepsthatwouldbetakentoequalizeanytreatmenteffectsthatarose.Althoughtheauthenticenvi-ronmentdidmakeitmoredifculttoidentifyandcontrolforallpossibleextraneousvariables,italsoraisedtheecologicalvalidityofthestudy.Resultsshouldgeneral-izeatleasttoundergraduate,preserviceteachersinacollegesetting,andpossiblytoadultlearnersingeneral.Ofcoursemoreresearchisneededwithothersubjectsandgroupsofadultlearnerstojustifysuchanextensionofndings.5.1.1ProcrastinationLevelBeforeconsideringtheprocrastinationresults,areexaminationofthemeasureusedtoobtainthemisappropriate.Recallthattherateofrelativedigressionfromthetargetresponse(RDTR)isactuallyameasureofwhenassignmentsarestartedandoftheirdifculty.Onemightarguethatabettermeasurewouldbeonethatindicateswhenastudentrstbeginsanassignment.However,measuringthisisproblematic.Onewouldeitherhavetodependonlessreliableselfreportdataorontimestampsofwhenstudentsrstaccessassignmentdescriptions.Inthelatter99

PAGE 115

case,studentswouldhavetoberestrictedfromaccessingalldescriptionsatonce.Evenifaccesstoagivenassignmentdescriptionwasnotalloweduntilthepreviousonehadbeensubmitted,thereisstillnoguaranteethatthestudentaccessingitwillstartworkingonitrightaway.Basedonthestudiesreviewed,therateofRDTRappearstobethemostcomprehensive,sensitive,andreliablemeasureoftotalworkdistributionpatternsavailablewhencomparedtoothermeasureswhichonlyconsiderasubsetoftheassignmentsortreatmentinterval,discretizethetreatmentintervalorresponsepattern,consideronlypositiveornegativeresponsesalone,orrelyonthesubjectiveinspectionofgraphs.AccordingtotheformulafortherateofRDTR,studentswhosubmitfeweras-signmentswillhaveahigherRDTRvalue.BecausestudentsingroupAsubmittedfewerassignments,theirrateslikelywereinated.Unfortunately,studentsinAac-tuallymayhavecompletedapproximatelythesamenumberofassignments,butsimplydidnotsubmitthem.Recallthattheircontingencywastoreceivenocreditforlatework,sotheymaynothaveseenthevalueinsubmittinglateworkforfeedbackonly.Therefore,differencesinRDTRrateswithrespecttoAneedtobeinterpretedwithcare.AccordingtorandomizationtestsonRDTRrates,therewasstrongevidencethatstudentsinRprocrastinatedmorethanstudentsinC.Infact,differenceswerearguablypracticalwithstudentsinRsubmittingassignmentsap-proximatelyfourdayslaterthanstudentsinC.TherewasalsosomeevidencethatstudentsinAprocrastinatedmorethanstudentsinC.Again,eventhoughstudentsinAsubmittedassignmentsmorethanthreedayslaterthanstudentsinC,theseresultsmustbeinterpretedwithcare.Likewise,theminimaldifferencebetweenRDTRratesbetweenstudentsingroupsRandA,withRsubmittingassignmentsnearlyonefulldayearlierthanstudentsinA,aresomewhatquestionable.Althoughnoconclusionsaboutsignicancecanbedrawn,itisalsointerestingtonotethatstudentsinCandArequestedslightlymoredeadlineextensionsthanstudentsinR,andthatTAsreportednoticingstudentsinRdidnotcomeforhelpuntillateinthetreatmentinterval.5.1.2PacingPreferenceAsexpected,studentsinthethreedeadlinecontingenciesdidnotappeartodiffersignicantlyintheirattitudestowardself-pacing,asevidencedbyANOVAre-sults.Accordingtochi-squaretestsonfreeformatresponses,theyalsodidnotappeartodiffersignicantlyintheirfeelingsaboutthecourseware.Althoughstu-dentsinAweretheonesmostlikelytosaythattheylikednothingbestaboutthecourseware,andstudentsinRweretheonesmostlikelytosaythattheylikedtheavailabilityofCoD,twoseparatechi-squaretestsindicatednosignicantdifferencebetweengroupsinthetotalnumberofpositiveresponsesandthetotalnumberofnegativeresponses.Thesendingsareinharmonywiththoseofotherresearchers100

PAGE 116

whohavecomparedcourseevaluationsfromstudentsexposedtodifferentdead-linecontingencies.5.1.3AchievementDataBeforeconsideringanyresultsfromtheanalysisoftheachievementdata,itisadvisabletoreviewsomeofthedisproportionateproblemsencounteredbystu-dentsinA.First,althoughnostatisticallysignicantdifferencesbetweengroupswereidentiedatthestartofthestudy,thedirectionofmanydifferencesindicatedadisadvantageforstudentsinA.Theyreportedlesspriorexperiencewithdocu-menttypesettingandprogrammingonthepretestandhadlowerpretestscores.Infact,althoughtheomnibustestofpretestdifferenceswasnotstatisticallysigni-cant,theaveragefourpercentagepointadvantageforstudentsinCoverstudentsinAhadnearstatisticalsignicance(p0.0551)andtheeffectsizeofd0.38wouldbeconsideredpracticalbymanyresearchers(Gall,Borg,&Gall,1996).Inaddition,ofthe20studentswhoweremissingboththeposttestandtheretentiontestandwhoweredroppedfromanalysis,thoseinAearnedhalfalettergradehigheronExam1inEdTechthanthoseinRandC.IfslightlybetterstudentsweredroppedfromA,thenitisreasonabletoassumethatAmight,onaverage,haveslightlyloweroutcomesonachievementmeasures.Likethepretest,differencesinposttestscoreswerenotstatisticallysignicant,butwerearguablypracticalwithanaverageeightpointadvantageforstudentsinC(d0.36)andasevenpointadvantageforstudentsinR(d0.30).Inaddition,fourstudentsinAexperi-encedveriabletechnicalproblemsbeyondtheircontrol,withoneunabletosubmitsomeoftheassignmentsduetoalimitationintheonlinescriptsforhandlingnamepunctuation,andwiththreeothersreceivingcorruptcoursewareCD-ROMs.ItispossiblethatmorestudentsinAreceivedcorruptCD-ROMsandeitherdidnotrealizeitordidnotreportit.Possiblyofgreaterconsequenceweretwoproblemsassociatedwiththena-tureofthedeadlinecontingencyexperiencedbystudentsinA.First,becausetheywererequiredtomeeteachdeadlineorloseallpointsforthegivenassignment,theylikelyweretherststudentstocompleteeachassignment.TheTAs,whowerecompletingtheassignmentsatthesametimeasthestudents,likelywerelessversedinthesolutionsastheseleadstudentsapproachedthemwithques-tions.Inaddition,allassignmentsweredueonFridaysatmidnight,andnoneoftheTAshadofcehoursonFridays.TheTAsreportedthat,asaconsequence,studentsingroupA,whooftenappearedtodelaystartingtheassignmentsuntilThursdays,werelessabletogethelpandcomplainedmore.StudentsinRandCcouldstillgethelponMondaysandturnintheirworkforfull,oratleastpartial,credit.Followingasimilarargument,studentsinAweretheonesmostaffectedbythebriefonedayintervalbetweentheadministrationofExam1andtherststudy101

PAGE 117

deadline.StudentsinCalsowereaffected,ofcourse,buttoalesserextentthanstudentsinA.Theotherproblemassociatedwiththenatureofthedeadlinecon-tingenciesisaconsequenceofapplyingalltreatmentssimultaneouslytostudentsinthesameclass.Withthehopethatnoonewouldfeeldemoralized,allpartieswereinformedsystematicallyofthedifferencesbetweentreatmentsandofhowpossibledifferencesinoutcomeswouldbehandledequitably.However,itisstillpossiblethatstudentsinAdidfeeltheywereatadisadvantageandwerehelplesstodoanythingaboutit.Perhapsmanygaveupastheycontinuedtosubmitworkandreceivenopoints.ThelikelihoodthatthedatagarneredforstudentsinAwereaffectedadverselybytheproblemsoutlinedabovewarrantsusingextremecautionininterpretinganyresultsregardingstudentsinA.Resultsfromanalysisoftheachievementdataindicatethatapplyingcondi-tionaldeadlinesproducesthebestresults.First,therewerefewerstudentsinCwhoappearedtobenon-starters.Presumably,non-starters,whosubmittednoas-signments,eitherchosenottoparticipateorstartedsolatethattheyfelttheycouldnotsucceedanddecidednottoproceed.Therewere11suchindividualsinR,7inA,andonly4inC.Aftertakingintoaccountthepredeterminedstudygroupsfortheseindividuals,the11inRwerereducedto8datapoints.Thus,twiceasmanystudentsinRandAappearedtodisengage.EvenmoreproblematicisthefactthatsomeofthestudentsinRandAwerebetterstudentsbasedonhigherExam1scores,whichwere,onaverage,approximatelytwolettergradeshigherthantheExam1scoresofnon-startersinC.Becausethereweremorenon-startersinRandA,itislikelythatmorestudentsguessedattheanswersontheposttestandretentiontestinthesegroups.This,coupledwiththefactthatsomebetterstudentsinthesetwogroupswereincludedinthesubsetofguessers,likelyhadtheeffectofloweringposttestandretentiontestscoresforstudentsinRandA.AlthoughonemightarguethatretainingthesedatapointsinsubsequentanalyseslikelyinatedthedifferencesinachievementscoresbetweenstudentsingroupCandthoseintheothertwogroups,itwouldseemimportanttoacknowledgethatallstudentsinC,exceptforafewapparentlylowachievers,stayedengaged.Infact,itmightbearguedthatdroppingthenon-startersfromanalysiswouldarticiallyinatethescoresofstudentsinRandA.Thus,dataforthenon-startersweremaintained.Itisinterestingtonotethatthegreaternumberofnon-startersinRlikelyaccountedforthehigherpercentageofassignmentsubmissionsbystudentsinC.However,thepercentageofsubmissionsbystudentsinAwaslowerthancouldbeaccountedforbythepresenceofadditionalnon-startersaloneandislikelyrelatedtosomeoftheproblemsdiscussedabove.Inshort,contingencyCappearssuperiorinkeepingmorestudentsengaged,includingsomehigherachievers.Accordingtotherandomizationtestsperformedontheachievementdata,thetrendovertimeappearstofavorstudentsinCrememberingmoreofwhatthey102

PAGE 118

learned.Thestatisticalsignicanceoftheomnibustestonamountsforgotten1,coupledwiththenearstatisticalsignicanceofthedifferencebetweenstudentsinRandC,providesomeevidencethatstudentsinRtendedtoforgetmoreofwhattheylearnedovertimethandidstudentsinC.Thedifferenceinthediffer-encescoresbetweengroupsRandCwashighest,andhence,wasthemostlikelycandidatetoexplainthesignicanceoftheomnibustest.Furthermore,ifthedif-ferencebetweenRandAisremovedfromconsiderationduetothefactthattheaverageRDTRratesdidnotshowasignicantdifferenceinprocrastinationlev-elsforstudentsinthesegroups,thenthedifferencebetweenRandCisstatisti-callysignicant.TheevidenceinfavorofadifferencebetweenRandCwouldbestrengthenedevenmoreifAwerenotincludedintheanalysis,whichonemightargueisreasonablebasedonthedifferentialproblemsstudentsinAexperienced.Inaddition,thedifferencebetweentheamountsforgottenbystudentsinRandCisarguablypractical(d0.43)withstudentsinRforgettingonehalfofalettergrademorematerialinamonththandidstudentsinC.Also,ifthetrendsdepictedintheinteractiongraphinFigure27continuetochangeatroughlythesameratesgivenmoretime,itisreasonabletoexpectthattheevidenceforCsupportingbetterlongtermretentionofcontentwillgrowstrongerstill.Consideringthesmalltomediumsizedeffects(d"[-0.17,0.48])ofthetreat-mentsonretentiontestscoresalone,theyclearlywerelowerthantheanticipatedlargeeffects.Also,thenegativeeffectbetweenRandAwascontrarytoexpec-tations,butnotsurprisinginlightoftheproblemsAexperiencedandthefactthatevidenceindicatedstudentsinthesetwogroupsdidnotdiffersignicantlyintheirprocrastinationlevels.Asmentionedabove,alongerretentionintervalwouldin-creasetheeffectsdetected,iftrendscontinuedatthesamerate.AnotherfactorwhichmayhavereducedeffectsizesinvolvedinformingstudentsthatHTMLques-tionswouldappearonExam2twodaysbeforetheytookit.Consequently,theyhadampleopportunitytoreviewthematerialandrelearnit,oreventolearncon-ceptstheyhadnotlearnedduringthetreatmentinterval.AlthoughKuliketal.(1979)statedthatitwasbetternottoannounceretentionexams,theretentionquestionsforthisstudywereincorporatedintoExam2andplayedapartindeter-miningthestudents'nalgradesinEdTech.Morally,itseemedmoreappropriatetoinformthestudentsthantowithholdthisinformation.Ofthestudiesreviewedwhichconsideredtheeffectsofpacingonretentiontestscores,theonethatap-pearedtohavethestrongestdesignincorporatedanunannouncedretentiontestandreportedalargeeffect.Intheotherfourstudies,researcherseitherstatedexplicitlythattheyhadinformedthestudentsoftheretentiontestinadvance,oritappearedimplicitlythattheyhad.Unfortunately,thesestudiesalsosufferedfromamyriadofpotentiallyseriousdesignawsandproblems,promptingtheneedtointerprettheirndingswithcaution.Problemsincludedtheuseofvolunteers,small 1Recallthattheamountforgottenbyagivenstudentwasdeterminedbycalculatingadifferencescore,wheretheretentiontestscorewassubtractedfromtheposttestscore.103

PAGE 119

groupsizes,differentialdroppingfromtreatmentconditions,weakevidenceofdif-ferencesinpacingpatterns,exposingsomestudentstolongerretentionintervals,tellingstudentstheirperformancewouldnotaffecttheirgradeinanyway,alowmasterycriterionof50%fortheretentiontest,andhavingretentionscoresforlessthanhalfofthestudents.Insummary,effectsizesmightberaisedinfuturestud-iesbylengtheningtheretentionintervalandbynotannouncingtheretentiontestbeforeitisgiven.5.2FutureResearchandPracticesBeforeanydenitiveconclusionscanbedrawn,moreresearchisneededtoreplicatethendingsofthisstudywithrespecttorecommendedandconditionaldeadlines,toclarifytheeffectsofabsolutedeadlinecontingencies,andtoenablegeneralizationstoothercontentareasandgroupsoflearners.Forthoseinterestedinreplicatingthisstudyusingthesamecoursewareandcomparablestudents,sug-gestionsaremadeforimprovingthemethodologyandthecoursewareaswellasforprovidingthestudentsandcourseofcialswithbettersupport.Otherrelatedareasofresearcharealsosuggested,suchasthedevelopmentofusefultoolsandmethodsofhelpinginstructorsandlearnersmakeasmoothtransitiontotheirnewroles.5.2.1ReplicatingStudyAfewmethodologicalenhancementsshouldstrengthenfuturereplicationsofthiswork.First,becausemanyofthestudygroupingsformedbythestudentsap-pearedtobesomewhatuidinnature,reductionoftheinitialpowerofthestudycouldbeavoidedbynotassigningstudentstotreatmentsbasedonthesegroup-ingsandthensubsequentlyaveragingmembers'scoresintosingledatapoints.StudentscouldbeaskedtoreportthenamesoffellowstudentswithwhomtheycollaboratedontheHTMLassignments.Ifmanystablegroupingsemerged,thenthedatacouldbeanalyzedusingrandomizationtestssimilartothoseemployedinthecurrentstudy.Alternatively,onemightplantoanalyzethedataviatheran-domizationmodelfromthestart,eliminatingtheneedtocollectdataoncollabo-ration.Itshouldbenoted,however,thatrandomizationtestsdonotyieldexactlythesameinformationasdotestsbasedonANOVA.Thelattertechniqueteststhenullhypothesisthatthetreatmenthasnoeffectonmeanscores,whiletheformerteststhenullhypothesisthatthetreatmenthasnoeffectonscoredistributions.Althoughsimilar,thereisasubtledifferenceinfocuswhichshouldbeconsideredwhenchoosingbetweenthesetechniques.104

PAGE 120

Infuturereplications,italsowouldbeadvisabletoanalyzethepretestdatabe-foredistributingtheCD-ROMs,sothatstudentscouldbereassignedrandomlytotreatments,iftheinitialrandomizationproducedunequalgroupswithrespecttothepretest.Ifsimilarstudentsparticipateinthereplication,thenthetreatmentintervalshouldbeincreasedfromonemonthtotwomonths.Theretentionintervalalsoshouldbeincreasedfromonemonthtotwomonthsinordertoincreaseeffectsizeandpower.Preferably,treatmentsR,C,andAwouldbeincludedinordertorepli-catethendingsreportedhereinfavorofCandtoclarifythendingswithrespecttoA.Infact,futurestudiesmayrevealthatAactuallyisbetter,ifonewishestoencouragestudentstodistributetheirworkuniformlyovertheentiretreatmentin-terval.Givenamorereasonabletimeframe,somestudentsinCmayamasstheirworknearthebeginningoftheinterval,hopingtoaccrueasmanybonuspointsaspossible.Thislikelywouldbejustasundesirableasamassinglearningneartheendoftheinterval.Onemightwishtoadjustthedeadlinesslightly,sothatthelastdaystudentscanreceiveanycreditforanassignmentisthesame,atleast,forstudentsinCandA.Then,studentsinAwouldnotbepushedtocompleteassign-mentsfasterthanstudentsinC.Careshouldbetakenindescribingcontingencieswithpositiveratherthannegativeterms.Forexample,ratherthandiscussCintermsofbonusesandpenalties,onemightexplainthatworksubmittedonagivenassignmentpriortoacertaindateisworth5points,priortoanotherisworth4points,priortoanother3points,andpriortotheendofthecourse0points.Finally,studentsmightfeellesscoerced,iftheyhadtheoptionofcompletinganalternativeassignment,eveniftheystillchosetoparticipateinthestudy.Ifpossible,theretentiontestshouldbeunannounced.Morally,thisisreason-able,ifscoresdonotcontributedirectlytothestudents'grades.Atthesametime,however,studentsneedtobeencouragedtodotheirbest.Onescenariomightbetodistributeitwiththelastclassexam.Thisshouldensuregoodattendance.Thenstudentscouldbemotivatedtocompletetheretentiontestbybeingtoldthatitisworthextracredittowardthelastexam,andthebettertheydo,themoreextracredittheyearn.Ofcourse,thereisstillthedangerthatmanymightchoosenottotakeit,causingresultstobeskewedandbasedeffectivelyonvolunteers.However,consideringthenumberofstudentsinthecurrentstudywhocompletedtheextracreditopportunities,itislikelythatmoststudentswouldtaketheretentiontest.Inaddition,onemightconsidergivingtheposttestonthesamedayasanotherclassexam,perhapsthemidterm,inordertoincreaseattendance.Inthiscase,theposttestlikelywillneedtobeshortened.Onemightopttousetheretentiontestfromthisstudywithorwithoutalteration.Ifitisaltered,thenitishighlyrecommendedthatthenewversionalsoincludeanessayquestionwithnovelmaterial.Notonlydidtheessayquestionprovidemorereliabledatathanthemultiplechoicequestions,butitalsotestedamorevaluableskill.Becausethegoalofthecoursewarewastohelpstudentsdevelopanewcomputerskillsimilarinnaturetoprogramming,beingabletoapplythisknowledge105

PAGE 121

wasarguablymorevaluablethanbeingabletorecognizesyntax.Onechangewithrespecttotheessayquestionisrecommended.Basedonacommentmadebyonestudentregardingthedisproportionatelylownumberofpointsassignedtotheessayquestion,therelativevalueofthisquestionshouldberaisedornotreportedatall.Oneoptionmightbetoinformthestudentthattheentireretentiontestisworthtenpointsandhasthepowertoimprovetheirscoreonthelastexambyonelettergrade.Thisshouldprovideenoughinformationtomotivatethestudentwithouttheneedtodisclosetheexactpointdistribution.Itisalsorecommendedthatstudentsupportbeenhanced.First,theCD-ROMsshouldbediscussedandthenavigationofthecoursewaredemonstratedinclassonthedaytheyaredistributed.Studentsshouldbeadvisedtoexaminetheircopiesrightaway,sothatanytechnicalproblemscanbeaddressedatthenextclassmeeting,includingtheexchangeofanycorruptCD-ROMsandclaricationofhowtonavigatethroughthecontent.Inaddition,studentsshouldbeencouragedtotakenotesastheyprogressthroughthematerialandtocontactcourseofcialsviae-mailand/orofcehoursasquestionsarise.Anoutlineofthecoursecontentmighthelpsupportsuchnotetaking.TheyalsoshouldbeprovidedwithalistofFAQsthattheycansearchforhelp.Ideally,solutioncodeshouldbemadeavailableforeachassignmentafteritsdeadlinehaspassed.Unfortunately,thereisnoclearwaytodothiswithouteithergivingthestudentsinRanunfairadvantageorwaitingtoposttheentiresolutionatonetimeafterthetreatmentinterval.Finally,alistservisrecommendedinordertoprocessstudentquestionsasefcientlyaspossible.Ideally,thelistservwouldallowstudentstosendaquestiontoallcourseofcialsatoncethatisnotcopiedtofellowstudents.Oneofcialcouldbedesignatedperdayastheoneresponsibleforrespondingquicklytoallincomingqueries.Theotherscouldmonitorexchangesandaddcommentsorclarications,ifdesired.Answersshouldbesentbacktothestudentandthelistserv,whichwouldbereadableonlybythecourseofcialsandstudyinvestigator.Tomakethewholeprocessmoreef-cient,itisrecommendedthattheinvestigatorcontinuouslymonitortheexchangesandpostFAQsonline.Insupportofthecourseofcials,correctionsintheirunderstandingofthemate-rial,promptedbytheirresponsestostudentsonthelistserv,shouldbediscussedwiththemprivately,sothattheycanimprovetheirgrasponthematerialandpassthisinformationontothestudentthemselves.Duringtheirtraining,theimportanceoftimelyfeedbackonsubmittedworkande-mailedquestionsshouldbeempha-sized.Alternativepolicies,suchashavingonepersondesignatedasthelistservmonitorperdayandgradingallpendingassignmentsonatleastthreepredeter-mineddaysperweekshouldbediscussedwith,agreeduponby,anddistributedinwritingtoallparties.OnealsoshoulddiscusswiththeTAstheirtutorialroleandthequalitiesofagoodtutor.Forexample,theycanaddressthestudents'affectiveneedsbybolsteringself-condence,promotingfeelingsofcontrol,andreducingfeelingsofisolation.Theycanaddressmotivationalneedsbyhelpingmaintain106

PAGE 122

challenge,evokingcuriosity,andemphasizingtherelevanceofthecontent.Also,theycanaddresscognitiveneedsbyunderstandingthematerialwellandreviewingassignmentsinadvance.Inordertofosteradeepunderstandingofthematerialandtoteachthecourseofcialshowtonavigatethecourseware,theyshouldbeaskedtoreviewallmaterialandtocompleteallassignments.Thiscouldbedoneinthecontextofmakingthemapartofthedesignteamandaskingthemtonoterecommendationsforimprovingthecoursewareastheyreviewit.Theirreviewcouldbeconductedduringtherstweekofthesemesterandameetingscheduledattheendoftheweektodiscussrecommendationsaswellastheappropriatenessandrelevanceofthematerialfortheirstudents.Aftersuchathoroughinvestigationofthecourseware,theyshouldbeabletocommentonthedifcultyoftheachievementtestsandoffersuggestionsforimprovingthem.However,thisshouldbedonewithcaution,ifitall,asstudentsarelikelytoaskofcialswhattoexpect.Iftheofcialshavenotseentheexams,thenthereislessofachancethatstudentswillreceivedifferentialinformationaboutthem.Infact,itlikelyisbestnottoinformtheTAsabouttheexistenceoftheretentiontest.Finally,asinthecurrentstudy,theTAsshouldbetrainedhowtousethegradingrubrics.Ifdesired,an“overallquality”categorymightbeaddedtoeachrubric,sothatTAscandistinguishreadilybetweenstudentswhojustmeettheminimumrequirementsandthosewhodomore.5.2.2ImprovingCoursewareSeveralrecommendationsweremadebystudentsandcourseofcialsforim-provingthecoursewareandassignments.Somestudentsindicatedadesireforlessrigidassignmentswiththefreedomtobemorecreative.Otherssuggestedthattherequirementofpresentingresearchonergonomicsbeeliminated.Bothoftheserequestsmightbeaddressedbyallowingstudentstochoosethetopictheyresearch,ratherthanforcingthemtoresearchergonomics.Ifthecoursefacilita-torwantstodirecttheirfocus,aswasthecaseinthecurrentstudy,thenalistofappropriatealternativetopicscouldbesupplied.Inaddition,lessonsshouldbeen-hancedbydiscussingtherelevanceoflearningHTMLandhowitrelatestootherimportanttoolssuchasFlash,Authorware,JavaScript,Java,etc.Difcultlessons,inparticularthefthone,shouldbeaugmentedwithadditionalexamples.Also,newmodulesontroubleshootingandhelpfulpracticeslikeusingHTMLvalidators,addingstartandendtagstogether,addingonenewlineatatime,andcommentingoutsectionsofcodeshouldbeincorporated.Otherimprovementsincludeprovidingaglossary,helpsection,andstaticFAQsection,withsupplementalFAQsfromthecurrentcourseofferingpostedonline.Severalparticipantsalsorecommendedthatthesoundbeaccessibleonmorema-chines,thatitbemutable,andthatwrittentranscriptsbeprovidedonlineorin107

PAGE 123

print.Theabilitytobookmarkpagesandtohighlightmaterialonlinewouldgreatlyenhancethestudents'experience,butsuchalterationswouldrequireextensiverevisionstothecoursewareprogramming.Alessformidableimprovementwouldbetomakeiteasyforstudentstoviewassignmentdescriptionsandcoursecon-tentsimultaneously.Thiscanalreadybeaccomplishedbystartingthecoursewaretwiceintwoseparatebrowsers.However,suchanapproachisnotintuitiveforin-troductorystudentsliketheonesinthecurrentstudy.Studentsshouldeitherbeinformedexplicitlyofthisprocess,orsimilarfunctionalityshouldbebuiltintothecourseware.Asimplerapproachmightbetoprovidestudentswithprintedmaterialdescribingtheassignments.Studentsrequestedthatothermaterialbeprovidedinprintform,includingthereferencematerialbytheWebDesignGroup,thesyl-labus,andabookletcontainingallcoursewarematerial.Also,itmightbeadvisabletoprovidestudentswithalistofsupplementalbooks,andpossiblywithalistofpagestoreadforeachcoursewarelesson,forthosewhoprefertolearnfromabook.Ofcourse,thismightintroduceextraneousvariables,butsuchproblemsareinherentwhenconductingstudiesinauthenticenvironments.Finally,twoim-provementsthatwouldhelpTAsincludeaddinga“selectall”buttontothegradingrubricsandprovidingthemwithcontinuoustrackinginformationontheprogressoftheirstudents.Inmakingmodicationstothecourseware,careshouldbetakentomaintainitsgoodqualities.Forexample,lessonsshouldbekeptshort,preferablyunder10minuteseach.Theyshouldcontainvisualandauditorystimulusandactivelyengagestudents.Assignmentsshouldbesuggestedthroughoutthecourseware,whichpromoteimmediate,distributedpractice.Lessonsshouldberelevant,in-teresting,andbasedontaskanalysesoftargetbehaviors.Allmaterialshouldbesuppliedondemand.Finally,thecoursewareshouldpermitandencouragecollab-orationwithfellowstudentsandsupporttutorialrelationshipswithcourseofcialsbyprovidingdirectlinkstoTAs'Websitesande-mailformsforeasilypostingques-tionstothelistservand/orspeciccourseofcials.5.2.3ResearchingRelatedIssuesRatherthanreplicatethendingsofthisstudy,onemightchoosetodeveloptoolsthatwouldsupportthecurrentparadigmshifttowardalearner-centeredap-proach,whichprovidesstudentswithaccesstomoreandmoreCoDdaily.Forexample,onemightfocusondevelopingaprogramthatwouldallowstudentstohighlightwrittenmaterialtheyreadonline,facilitatenotetakingwithembeddedlinkstocontent,andhelpthemorganizethematerialintosummariesofkeyel-ementsforfuturereview.Methodsmightbedesignedwhichwouldallowuserstoselectlearningparameterssuchaspreferredmodeoflearning.Then,basedontheirselection,theycouldbepresentedwithappropriatecontentversions.Or,108

PAGE 124

onemightdesignanautonomousagentthatcouldobservealearner'schoicesaswellasevaluatethelearner'slevelofunderstandinggiventhosechoices,sothatitmightsuggestcontentinparticularformatsinthefuture.Toolsmightbedevelopedtohandlesystematicallysubmissionsofassignmentsbygroupsofstudentswork-ingtogetherwithfeedbacksentbacktoallmembersaswellastoolsthatfacilitategradinggroupprojectsandportfolios.Alsobenecial,wouldbethedevelopmentofsoftwaretoolsthatcouldanalyzeshortanswersandessaysforkeyelements,evenwhenconceptshavebeenmisspelledorreplacedbysynonyms.Perhapsonecouldwriteaprogramthatwouldcrosscheckentriesinathesaurusforlikelymatchesbetweenanswersandtargetconcepts.OnemightuseHTMLvalidatorsandprogramcompilersinasimilarfashiontoaidincheckingcode.Anotherusefultoolmighthelpcatchacademicdishonestybyautomaticallycheckingsubmissionsforclosematcheswithpastonesandalertingcourseofcialswhentheyarefound.Toolsthatcouldverifytheauthenticityofresponsesmadebydistantlearnerstakingexamswouldbeinvaluable.Finally,withthegrowinguseofCAI,thereisaneedtodevelopmethodsthatwillallowsuchprogramstoaddresstheaffectivesideofthelearner,perhapsbyusingacameratoanalyzefacialexpressionsforemotionalstate.Otherissuesonemightinvestigateincludeeffectivemeansofmanagingthegrowingnumberofcourseassistants,trainingassistantstobeeffectivetutors,andhelpingstudentslearntotakeresponsibilityforknowledgeacquisitionandtimemanagement.Inaddition,onemightfocusonmethodsofmanagingand/orsupportingcollaborationbetweenlearnersandofreducingfeelingsofisolationfordistantlearners.OnemightattempttodiscernthemosteffectivecombinationsofCoD,lectures,demonstrations,discussions,groupwork,andstructuredlabtimeforvariouscontentdomains.And,ofcourse,efcientproceduresareneededforini-tialcoursedevelopment,conversionofclassroompresentationstoCoD,theshar-ingandmaintenanceofcontentmodules,providingcompensationforintellectualproperty,andhandlingtechnicalproblemsthatmightariseaswellasdisparitiesinstudentaccesstotechnology.Toextendthendingsofthecurrentstudyandtoverifythatregulatingpacingsothatitismoreuniformreallydoescontributetohighercontentretention,moreresearchisneeded.Studiesshouldbeconductedwithonsiteanddistantlearnersaswellaswithdifferentagegroupsandcontentdomains.Investigatorsmightalsoconsidermassedversusspacedpracticeintraditionalcourseswhichdonotpro-videCoDinanattempttoreplicatethendingsofotherresearchers(Grote,1992,1995;Bloom&Shuell,1981).AlthoughPSIhasbeenshowntohavedistinctad-vantagesoverTI,itdoesnotnecessarilyfollowthatCoDisakeycomponentcon-tributingtoitssuccess.Researchersmightfocusoncomparingoutcomeswhencontentisdeliveredondemandversusvialecture.Becausemastery,anotherel-ementofPSI,hasbeenshowntobeaveryeffectiveteachingmethod,onemightpursuemethodsofsupportingmasterywhenpacingisregulated.Oftenduringa109

PAGE 125

course,unforeseeneventsoroversightsmakeitnecessarytoadjustdeadlines.Itwouldbeinterestingtostudytheimpactadjustmentsmighthaveontheeffective-nessofvariouscontingenciesinregulatingpacing.Also,thereistheissueofhowtosupplystudentswithanassignmentsolutioninatimelymannerwhennotallstudentshaveturnedintheassignment.Ofcourse,onemightchoosetosharethesolutionandthenhavethesefasterstudentshelptheirclassmates.Forthistobeeffective,systematicmeansofrewardingandtrainingthesestudentsintheartoftutoringwouldneedtobeinplace.Finally,othertypesofcontingenciesmightbeconsidered.Forexample,onemightmodifyconditionaldeadlines,sothattheyfollowaslidingscale,awardingincreasingbonuspointstheearlieranassignmentissubmittedandincreasingthepenaltythelateranassignmentissubmitted.Alternatively,onemightinvestigatetheuseofstudentcontracteddeadlinesthatarelesslikelytopushastudenttoofastandshouldpromotepersonalgrowthintheformofself-disciplineandabetterunderstandingofone'sowncapabilities.Althoughpastresearchersfoundstudentcontracteddeadlinestobemoretimeconsuming,advancesintechnologymaymakeitfeasibletoautomatetheprocess.Ratherthanallowingstudentstosettheirowndeadlines,onemightallowthemtoselectthecontingenciesformeetingthemfromarangeofpossibilitieslikedroppingalowgrade,skippinganassignment,orearningbonuspoints(Murdock,2000).5.2.4IncorporatingLessonsLearnedIndesigningfuturestudies,courses,andinstructionalmaterial,oneshouldat-tempttoincorporateimportantprinciplesidentiedbypastresearchers.Instruc-tionalmaterialshouldbeofhighqualitywithacommonlookandshouldincorporatediversemediawithcarefulcoordinationbetweendifferentelements.Itshouldbecontinuallyandsystematicallymaintainedandupdated,easilyadaptabletospe-ciclearnersandenvironments,basedonobjectivesandtaskanalyses,dividedintoshortpresentations,understandableandeasilynavigatedbymoststudents,andavailableondemand.Althoughitcanbetimeconsumingandexpensivetodevelop,usingoff-the-shelfsoftwarewhenfeasibleandcollaborationbetweende-veloperscanreduceassociatedcosts.Ofcourse,theequipmentusedtodelivertheinstructionalmaterialshouldbemaintainedatahighlevel,andeveryeffortshouldbemadetoensurethatallstudentshaveequalaccesstoit.OneshouldconsidersupplyingcontenttostudentsinCD-ROMformatwhenitrequiresalargebandwidthtodeliveritonline.Goodinstructionalpracticesincludekeepingstudentsactivelyengaged,match-ingactivitieswithobjectives,constantlymonitoringstudentperformance,providingstudentswithimmediateandregularfeedback,schedulingextratimetoworkoutlogisticswhentechnologyisinvolved,evaluatingstudentperformanceaswellas110

PAGE 126

achievement,andprovidingstructurewithaclear,predictableschedule.Ithasalsobeensuggestedthatonlinediscussionsbeprecededbyaformalintroductionandfollowedbyaclosingsummary.Finally,deadlinesshouldbeusedwhencon-tentisavailableondemandtoreduceprocrastination,andpossiblyraiselongtermretentionofcontent.Insupportingstudents,oneshouldutilizesuchlearner-centeredapproachesassupplyingCoD,collaboration,individualizedinstruction,andactiveparticipationindiscussions.Thedeliverymethodshouldmatchtheknowledgelevelofthestu-dents,andideally,theformatshouldsupportdifferentlearningstyles.Theuseofautomationshouldbebalancedwithpersonalattention,especiallywhendistantlearnersareinvolved,inordertoreducefeelingsofisolationaswellasthenum-berofstudentswhodropoutorrequestincompletegrades.Towardthisendstaffshouldbeavailabletoassiststudentsinatimelyfashion,anditisrecommendedthattheinstructorattempttorespondtoe-mailedquestionsatleastonceperday.Studentsalsoreportthatbeingabletoreviewtheworkoffellowstudentsisbene-cialandthattheyaremotivatedtoworkharderwhentheyknowtheirpeersmightseetheirownwork.Itisalsoadvisabletosupplystudentswithgoodexamplesofcompletedassignments.Fosteringpublicandprivatecommunicationswithinstruc-torsandpeershasalsobeenrecommended.Althoughonestudentmaydominatethegroupanditcanbedifculttoassigncredittoindividuals,collaborationallowscontenttobeviewedfrommanyperspectivesandcanmitigatetrepidationandbemotivationalwhenstudentsseethatothersalsoarestrugglingwiththematerial.Furthermore,studentsreportapreferencefordiscussions,whicharesuperiorinpromotingproblemsolving,overlectures.Whileonestudentreceivesaneededexplanation,theothergainsadeeperunderstandingofthematerialthroughver-balizationandbysynthesizingideas.Finally,studentsshouldbeofferedguidanceindonningnewroleswheretheyaremoreresponsibleforthedirectionandpaceoftheirownprogress.Likestudents,teachersalsoneedtrainingfornewrolesasinstructionaldesign-ers,tutors,facilitators,andmanagers/mentorsofagrowingnumberofassistants.Astutors,theyshouldaddresstheaffective,motivational,andcognitiveneedsoftheirstudents,whiletakingcarenottofosteranunhealthydependenceontheirfeedback.Theyshouldtraintheirassistantstobeeffectivetutorsaswell.Lec-tures,whengiven,shouldbeshort.Althoughtheydoallowteacherstopresentalargeamountofmaterialefciently,someofwhichmaybeunpublished,whichisorganizedtobestmeetsstudents'needs,studentattentionwanesquickly.Otheradvantagesofshortlecturesincludeallowingtheinstructortoactasascholarlyrolemodelandtoimpartenthusiasticallytheintrinsicvalueofthematerialinawaythatputslittlepressureonthestudentsandiswellsuitedforauditorylearn-ers.Ofcourse,someoftheseadvantageswouldlikelytranslatetoCoD,whichincorporatesvideo,orsimplynarration.111

PAGE 127

Whendevelopingnewmaterial,instructorsshouldenjoyreducedteachingloads,becauseitistimeconsumingtoproduce,especiallywhenvideoisinvolved.Theyalsoshouldbecompensatedby,atleast,havingtheireffortsnotedsystematicallyinapplicationsforpromotionandtenure.Coordinatingdesignandproductioneffortswithfellowinstructorsaswellascontinuedstaffdevelopmentshouldhelpreducefeelingsofdissatisfaction,frustration,insecurity,andpowerlessnessexperiencedbysometeachersastheyndthemselvesthrustintonewroles.Teamworkshouldincreasefeelingsofsatisfactionanddistributetheworkloadaswellasprovidestu-dentswithaccesstodifferentperspectivesandpresentationstyles.Iftwoinstruc-torsproducemodulesonthesametopic,thenstudentswouldhavetheoptionofreviewingboth,iftheyfeltitnecessary.Finally,asthestudent-teacherratiogrows,theteachermustbesupportedwithadditionalassistantsandstrongadministrativesupport.AdvancesintechnologymakeitfeasibletodeliverqualityCoD.Thisallowsstudentstoskipmaterialtheyalreadyknowandtoreviewmaterialmorethanonce,ifnecessary.Asmentioned,theyalsocanexaminematerialcreatedbydifferentcontentprovidersandselectpresentationsthatbestsupporttheirownlearningpreferencesandneeds.DeliveringcontentviatheInternetmakesiteasierforcontentproviderstoaddnewmodules,updateoldones,andtocollaboratewithotherproviders,possiblyaddinglinkstotheirwork.Toolswhichsupportcomputermanagedinstructionprovideconvenientwaysforstudentstosubmittheirworkandforgraderstoaccessit.Theyalsomakeiteasiertotrackstudents.Inaddition,gradingtoolscanreduceworkloadbyautomaticallygradingobjectivetestsandsendingfeedback.Ofcourse,thereisnowaytoguaranteethatstudentstakingquizzesonlinearenotconsultingothersources.Atthispoint,onlinequizzesshouldbeuseddiagnosticallytoindicatethecurrentlevelofunderstandingandpossibleneedforremediation,withlittleornoeffectonstudents'grades.Itisrecommendedthatdistantandlocalstudentsberequiredtocompleteatleastoneon-sitetestorinterviewtoverifytheiridentityanddepthofunderstanding.5.3ConclusionsCurrentlearner-centeredtrendssuchastheincreasingavailabilityofcontentondemand(CoD)andsupportofdistantlearners,havesomeexpertspredictingthattherewillbefewercolleges,morecompetitionbetweenschools,moreadjuncts,andbettersupportforstudentsofallagesinthefuture.Aslearnerstakegreaterresponsibilityforthedirectionandpaceoftheireducation,theyneedguidanceinsettingpersonaldeadlinesandselectingmaterialandpresentationmodes.Theyalsoneedtobegivenpromptandcontinuousfeedbackonperformanceaswellasachievementlevel.Publicandprivatecommunicationswithfellowstudentsandteachersshouldbefacilitated.Qualityexamplesofexpectedoutcomesaswellas112

PAGE 128

theworkofotherstudentsshouldbeprovided.Instructorsshouldbalanceshortpresentationswithopportunitiesforstudentstoparticipateactively.Likestudents,instructorsneedtraining,support,andrecognitionintheirnewrolesasinstruc-tionaldesigners,tutors,facilitators,andmanagers/mentorsofthegrowingranksofassistants.AlthoughtheInternetcanprovideaconvenientmechanismforsup-portingcommunication,thesubmissionofassignments,andthedeliveryofqualityCoD,careshouldbetakentoensurethatalllearnershaveequalaccess.MaterialthatrequiresalargebandwidthsuchasvideopresentationsandnarrationsmightbestbedeliveredviaCD-ROMatthispoint.Advancesintechnologymakesometypesoflearner-centeredinstructionmorefeasible.Inparticular,Keller'sPersonalizedSystemofInstructionwarrantsre-newedconsiderationwithitsabilitytoimproveimmediatelearning,increaselongtermcontentretention,andgarnermorefavorableevaluationsfromstudents.Al-thoughitsself-pacedcomponentallowsstudentstobalanceschoolworkwithotherdemands,causeslessfrustrationbynotforcingthemtoproceedtooquicklyortooslowly,andencouragesthemtobemoreindependentandself-reliant,italsoleadstohigherlevelsofprocrastination.AnothercomponentofPSI,CoD,reallymightbeconsideredalesserformofself-pacingandmayprovidemanyofthesameadvantages,whileallowingthepaceatwhichstudentssubmittheirworktobereg-ulatedviadeadlinecontingencies.Ingeneral,contingencieshavebeenshowntoreduceprocrastinationwithoutdetrimenttoimmediateachievementandsatisfac-tion.Basedoninvestigationsofmassedversusdistributedpractice,contingenciesalsomayenhancememoryfunction.Thecurrentstudybroughttogetherthesetwolinesofresearch,theuseofdead-linecontingenciesandthebenetsofdistributedpractice,inanattempttoextendthendingsinfavorofcontingenciesandtoofferanexplanationfortheirpossiblelongtermadvantageincontentretention.Becauseofnumerousproblemsexpe-riencedbystudentsrandomlyassignedtotheabsolutedeadline(A)contingency,ndingswerequestionableforthisgroupandthefollowingdiscussionfocusesondifferencesbetweenstudentswithrecommended(R)versusconditionaldeadlines(C).Accordingtoacomprehensive,sensitive,andreliablemeasureofprocrastina-tiondevelopedforthecurrentstudy,relativedigressionfromthetargetresponse(RDTR),studentsinRsubmittedassignmentssignicantlylaterthanstudentsinCwithapracticaldifferenceoffourdays.Therewasnosignicantdifferencebetweengroupsinpacingpreferenceorcoursesatisfaction.Therewerefewernon-startersinCthaninR,andthoseinCallappearedtobethelowestachievers.Also,asignicantomnibustest,withthelargestdifferencebetweengroupsRandC,providedevidencethatstudentsinRwerelikelytoforgetmoreonemonthafterlearningaboutHTML.Infact,studentsinRearnedhalfalettergradelowerontheretentiontest,andtheanalysisofdifferencescoresrevealedaroughlymediumeffect(d0.43)infavorofstudentsinC.Givenmoretimeandanunannouncedretentiontest,differenceslikelywouldhavebeenevengreater.Although,asmen-113

PAGE 129

tioned,ndingswithrespecttoAareinconclusive,thecurrentstudysuggeststhesuperiorityofconditionaldeadlines,whicharesimpletoimplementautomaticallywithonlinescripts.Inadditiontoenhancingmemoryfunctionwithoutdetrimenttoimmediateachievementandstudentsatisfaction,reasonabledeadlinesalongwithconditionalcontingenciesalsofacilitateconsistentgrading,betterdistributetheworkloadforcourseofcials,andprovidestudentswithamorerealisticexperi-ence.Still,moreresearchisneededtoverifyndingswithrespecttoRandC,toclarifyndingswithrespecttoA,andtoextendndingstoothersubjectareasandgroupsofstudents,includingdistantandlocallearnersaswellasdifferentagegroups.Atbest,currentndingsgeneralizetoundergraduate,preserviceteach-ersinacollegesetting,andpossiblysupportpredictionsaboutadultlearnersingeneral.Furthermore,althoughtheauthenticsettingcontributedtotheecologicalvalidityofthestudy,italsomadeitdifculttoensurethatallpossibleextraneousvariableswereidentiedandcontrolled.Thosewhowishtoreplicatethecurrentstudyshouldbeawarethattheymayneedtoanalyzethedatausingrandomizationtestsand/ortoreassignstudentstotreatmentsbasedonpretestoutcomes.Theyalsoshouldconsiderextendingthetrainingandretentionintervalstotwomonthseach,includingcontingenciesR,C,andA,settingdeadlinessothatthedaysonwhichCandAawardnocreditforlateworkcoincide,describingcontingenciesaspositivelyaspossible,andallowingstudentstooptoutofthestudy.Withrespecttotheretentiontest,theyshouldconsiderdistributingitwiththelastexam,includinganessayquestionwithnovelmaterial,notannouncingit,makingitextracredit,andtellingstudentsthatitcanraisetheirnalexamscorebyonelettergrade,de-pendingonhowwelltheydo.Also,theymightconsiderreducingthelengthoftheposttestandincorporatingintotheclassmidterm.Thoseplanningtoreplicatethisstudyalsoshouldplantoenhancestudentandteachersupportandmakesomeimprovementstothecourseware.OfcialsshoulddemonstratenavigationofthecoursewarethedayCD-ROMsaredistributed,en-couragestudentstoimmediatelyinspecttheircopiesandbringtheirquestionsaswellasanycorruptCD-ROMstothenextclassforexchange,encouragenotetak-ingbyprovidingasparseoutline,andmonitorpersonale-mailandalistserv.Theinvestigatorshoulddiscusswiththeinstructorsandassistantsthepracticesofex-perttutorswhoaddresstheaffective,motivational,andcognitiveneedsoftheirstudents,theimportanceoftimelyresponsestoe-mailandfeedbackonassign-mentsubmissions,howtousethegradingrubrics,andpoliciesregardingwhowillmonitorthelistserveachdayandwhenassignmentswillbegraded.Theinvesti-gatoralsoshouldhaveallcourseofcialsreviewalllessons,completeallassign-ments,andenlisttheiraidinimprovingthecoursewarebyhavingthemtakenotesonsuggestedimprovements.Inaddition,theinvestigatorshouldmonitorcommu-nicationonthelistserv,privatelyclarifycourseofcials'understandingofparticulartopicsifnecessary,andpostFAQsonlineinsuchawaythatensurestheintegrity114

PAGE 130

ofthestudy.Finally,investigatorswillwanttoenhancethecoursewarebyimprov-ingthefthlesson,providingargumentsfortherelevanceoflearningHTMLinlightoftheexistenceofprogramslikeFrontPageandFlash,makingthenarrationsmoreaccessibleandmutable,makingiteasiertoviewlessonsandassignmentdescriptionssimultaneously,allowingstudentsmorefreedominchoosingatopictoresearch,andbyincludingaglossary,transcripts,printedmaterial,additionalmaterialontroubleshooting,andpossiblyalistofrecommendedbooksandread-ings.Also,assistantswouldappreciatetheabilitytoselectallrubricsatonceandtomonitorstudentperformance.Researcherswhowishtoinvestigaterelatedissuesmightconsiderdesigningtoolsthatsupportamorelearner-centeredapproach,waysofhelpinginstructorsadapttotheirnewroles,havingfasterstudentshelpslowerones,andproceduresforsupportingcollaborativeinstructionaldesigneffortsaswellasformaintainingqualitymodules.OnealsomightcomparethemeritsofdisseminatinginformationvialectureversussupplyingCoD,howtoincorporatemasterywhenpacingisreg-ulated,andtheeffectsofdeadlineadjustmentsmadenecessarybyunforeseencircumstances.Inaddition,researchersmightinvestigateconditionalcontingen-cieswherepointsareawardedonaslidingscale,dependingonhowearlyorlateassignmentsaresubmittedaswellasstudentcontracteddeadlines,includinghowcontractingmightbeautomated.Ofcourse,onewillwanttoconsidertheamountofworkinvolvedinadministeringanycontingencyconsidered.Finally,onemightallowstudentstoselectapersonalcontingencyplanfromalistofalternatives.115

PAGE 131

ReferencesAlessi,S.M.,&Trollip,S.R.(1991).Computer-basedinstruction(2ed.).Engle-woodCliffs,NJ:PrenticeHall.Andriole,S.J.,Lytle,R.H.,&Monsanto,C.A.(1995).Asynchronouslearningnetworks:Drexel'sexperience.TechnologicalHorizonsinEducation,23(3),97-101.Ary,D.,Jacobs,L.C.,&Razavieh,A.(1996).Introductiontoresearchineducation(5ed.).Orlando,FL:HarcourtBraceCollegePublishers.Back,S.M.,&McCombs,B.L.(1984).Factorscriticaltotheimplementationofself-pacedinstruction:abackgroundreview(AFHRL-TP-84-24).BrooksAirForceBase,Tex.:AirForceHumanResourcesLaboratory.Bloom,K.C.,&Shuell,T.J.(1981).Effectsofmassedanddistributedpracticeonthelearningandretentionofsecond-languagevocabulary.JournalofEducationalResearch,74(4),245-248.Bonwell,C.C.,&Eison,J.A.(1991).Activelearning:Creatingexcitementintheclassroom(ASHE-ERICHigherEducationReportNo.1.).Washington,DC:TheGeorgeWashingtonUniversity,SchoolofEducationandHumanDevelopment.Bufford,R.K.(1976).Evaluationofareinforcementprocedureforacceleratingworkrateinaself-pacedcourse.JournalofAppliedBehaviorAnalysis,9(2),208.Caldwell,E.C.(1985).Dangersofpsi.TeachingofPsychology,12(1),9-12.Coldeway,D.O.,&Schiller,W.J.(1974).Trainingproctorsforthepersonalizedsystemofinstruction.InR.S.Ruskin&S.F.Bono(Eds.),Personalinstructioninhighereducation:Proceedingsoftherstnationalconference(p.111-117).Washington,DC:CenterforPersonalizedInstruction.Cooper,J.L.,&Greiner,J.M.(1971).Contingencymanagementinanintroductorypsychologycourseproducesbetterretention.ThePsychologicalRecord,21,391-400.116

PAGE 132

Crosbie,J.,&Kelly,G.(1993).Acomputer-basedpersonalizedsystemofin-structioncourseinappliedbehavioranalysis.BehaviorResearchMethods,Instruments,andComputers,60(2),366-370.Evans,T.(1995).Thepotentialofresearchwithstudentstoinformdevelopment.InF.Lockwood(Ed.),Openanddistancelearningtoday(p.67-75).London:Routledge.Gall,M.D.,Borg,W.R.,&Gall,J.P.(1996).Educationalresearch:Anintroduction(6ed.).WhitePlains,NY:LongmanPublishersUSA.Gallup,H.F.(1997,April).Fredkellerandpsi[on-line].Available:http://www.lafayette.edu/allanr/keller97.htmlGillard,G.(1993).Deconstructingcontiguity.InT.Evans&D.Nation(Eds.),Reformingopenanddistanceeducation:Criticalreectionsfrompractice(p.182-195).NewYork:St.Martin'sPress.Glick,D.M.,&Semb,G.(1978a).Effectsofpacingcontingenciesinpersonalizedinstruction:Areviewoftheevidence.JournalofPersonalizedInstruction,3(1),36-42.Glick,D.M.,&Semb,G.(1978b).Instructor-setpacingcontingenciesversustheabsenceofsuchcontingenciesinapersonalizeduniversitycourse.JournalofPersonalizedInstruction,3(3),131-138.Good,P.(2000).Permutationtests:Apracticalguidetoresamplingmethodsfortestinghypotheses(2ed.).NewYork:Springer-Verlag.Green,B.A.(1974).Thepersonalizedsystemofinstruction(psi).InR.S.Ruskin&S.F.Bono(Eds.),Personalinstructioninhighereducation:Proceedingsoftherstnationalconference(p.5-7).Washington,DC:CenterforPersonal-izedInstruction.Grote,M.G.(1992).Masteryofphysicsthroughdistributedpractice.ThePhysicsTeacher,30,443-445.Grote,M.G.(1995).Distributedversusmassedpracticeinhighschoolphysics.SchoolandScienceMathematics,95(2),97-101.Hawkridge,D.(1995).Thebigbangtheoryindistanceeducation.InF.Lockwood(Ed.),Openanddistancelearningtoday(p.3-12).London:Routledge.Hiltz,S.R.(1997).Impactsofcollege-levelcoursesviaasynchronouslearningnetworks:Somepreliminaryresults.JournalofAsynchronousLearningNet-works[On-lineserial],1(2).(Available:http://www.aln.org/alnweb/journal/is-sue2/hiltz.htm)117

PAGE 133

Hobbs,S.H.(1981).Acomparisonofstudent-andinstructor-pacedformatsintheintroductorypsychologycourse.TeachingofPsychology,8(4),209-211.Holmberg,B.(1989).Theoryandpracticeofdistanceeducation.London:Rout-ledge.Kaplan-Leiserson,E.(Ed.).(2001).E-learningglossary.AmericanSo-cietyforTraining&Development.Available:http://www.learningcir-cuits.org/glossary.htmlKeller,F.S.(1968).Goodbye,teacher...JournalofAppliedBehaviorAnalysis,1(1),79-89.Keller,F.S.(1981).Psiandeducationalreform.JournalofCollegeScienceTeaching,11(1),37-38.Kulik,C.-L.C.,Kulik,J.A.,&Bangert-Drowns,R.L.(1990).Effectivenessofmas-terylearningprograms:Ameta-analysis.ReviewofEducationalResearch,60(3),265-299.Kulik,J.A.,C.Kulik,C.lin,&Cohen,P.A.(1979).Ameta-analysisofoutcomestudiesofkeller'spersonalizedsystemofinstruction.AmericanPsychologist,34(4),307-318.Kulik,J.A.,&Kulik,C.-L.C.(1975).Effectivenessofthepersonalizedsystemofinstruction.EngineeringEducation,66(3),228-31.Lamal,P.A.(1984).Interestinpsiacrosssixteenyears.TeachingofPsychology,11(4),237-238.Lamwers,L.L.,&Jazwinski,C.H.(1989).Acomparisonofthreestrategiestoreducestudentprocrastinationinpsi.TeachingofPsychology,8-12.Lepper,M.R.,Woolverton,M.,&Mumme,D.L.(1993).Motivationaltechniquesofexperthumantutors:Lessonsforthedesignofcomputer-basedtutors.InS.P.Lajoie&S.J.Derry(Eds.),Computersascognitivetools(p.75-105).Hillsdale,NJ:LaurenceEribaumAssociates,Inc.Lloyd,K.E.(1978).Behavioranalysisandtechnologyinhighereducation.InA.C.Catania&T.A.Brigham(Eds.),Handbookofappliedbehavioranalysis(p.482-521).NewYork:IrvingtonPublishers,Inc.Lloyd,M.E.,&Lloyd,K.E.(1986).Haslightningstrucktwice?useofpsiincollegeclassrooms.TeachingofPsychology,13(3),149-151.Lloyd,M.E.,&Zylla,T.M.(1981).Self-pacing:Helpingstudentsestablishandfulllindividualizedplansforpacingunittests.TeachingofPsychology,8(2),100-103.118

PAGE 134

Lu,P.H.(1976).Modicationofprocrastinatingbehaviorinpersonalizedsystemsofinstruction.InNationalconferenceonpersonalizedsystemofinstructioninhighereducation(third).Washington,DC.Lu,P.K.(1978).Threeintegrativemodelsofkineticstructureinteachingastron-omy.JournalofResearchinScienceTeaching,15(4),249-255.Manly,B.F.(1997).Randomization,bootstrapandmontecarlomethodsinbiology(2ed.).London:ChapmanandHall.Mawhinney,V.,Bostow,D.,Laws,D.,Blumenfeld,G.,&Hopkins,B.(1971).Acomparisonofstudentsstudying-behaviorproducedbydaily,weekly,andthree-weektestingschedules.JournalofAppliedBehaviorAnalysis,4(4),257-264.Mayadas,A.F.(1994,September).Asynchronouslearningnetworks:Alfredp.sloanfoundation'sprograminlearningoutsidetheclassroom[on-line].Avail-able:http://www.sloan.org/education/ALN.new.htmlMcEwen,B.C.(1996).Teachingmicrocomputersoftwareskills.BusinessEduca-tionForum,50(4),15-20.Miller,L.K.,Weaver,F.H.,&Semb,G.(1974).Aprocedureformaintainingstudentprogressinapersonalizeduniversitycourse.JournalofAppliedBehaviorAnalysis,7(1),87-91.Morris,E.K.,Surber,C.F.,&Bijou,S.W.(1978).Self-pacingversusinstructor-pacing:Achievement,evaluations,andretention.JournalofEducationalPsy-chology,70(2),224-230.Mulligan,M.,Guess,D.,Holvoet,J.,&Brown,F.(1980).Theindividualizedcur-riculumsequencingmodel(i):Implicationsfromresearchonmassed,dis-tributed,orspacedtrialtraining.JournaloftheAssociationfortheSeverelyHandicapped,325-336.Murdock,K.(2000).Managementofprocrastinationindistanceeducationcoursesusingfeaturesofkeller'spersonalizedsystemofinstruction.Unpublisheddoctoraldissertation,UniversityofSouthFlorida,Tampa,FL.Pedhazur,E.(1997).Multipleregressioninbehavioralresearch:Explanationandprediction(3ed.).Orlando,FL:Holt,Rinehart,andWinston,Inc.Powers,R.B.,Edwards,K.A.,&Hoehle,W.F.(1973).Bonuspointsinaself-pacedcoursefacilitatesexam-taking.ThePsychologicalRecord,23,533-538.Preece,J.,Rogers,Y.,Sharp,H.,Benyon,D.,Holland,S.,&Carey,T.(1994).Human-computerinteraction.Wokingham,England:AddisonWesley.119

PAGE 135

Quinn,L.(1998).Webdesigngroup'shtml4.0reference[on-line].Available:http://www.htmlhelp.com/reference/html40/Rae,A.(1993).Self-pacedlearningwithvideoforundergraduates:amultimediakellerplan.BritishJournalofEducationalTechnology,24(1),43-51.Reiser,R.A.(1977).Effectsofself-pacingandinstructor-pacinginapsicourse.TheJournalofEducationalResearch,71,8-12.Reiser,R.A.(1984).Reducingstudentprocrastinationinapersonalizedsystemofinstructioncourse.EducationalCommunicationandTechnology,32(1),41-9.Roberts,M.S.,Fulton,M.,&Semb,G.(1988).Self-pacinginapersonalizedpsy-chologycourse:Lettingstudentssetthedeadlines.TeachingofPsychology,15(2),89-92.Roberts,M.S.,&Semb,G.B.(1980).Dailytesting:aconsequencefornotmeetingdeadlinesinapsicourse.JournalofPersonalizedInstruction,4(2),67-69.Roberts,M.S.,&Semb,G.B.(1989).Studentselectionofdeadlineconditionsinapersonalizedpsychologycourse.TeachingofPsychology,16(3),128-130.Roberts,M.S.,&Semb,G.B.(1990).Analysisofthenumberofstudent-setdeadlinesinapersonalizedpsychologycourse.TeachingofPsychology,17(3),170-173.Robin,A.L.,&Graham,M.Q.(1974).Academicresponsesandattitudesengen-deredbyteacherpacingversusstudentpacinginapersonalizedinstructioncourse.InR.S.Ruskin&S.F.Bono(Eds.),Personalinstructioninhighered-ucation:Proceedingsoftherstnationalconference(p.81-89).Washington,DC:CenterforPersonalizedInstruction.Ross,L.L.,&McBean,D.(1995).Acomparisonofpacingcontingenciesinclassesusingapersonalizedsystemofinstruction.JournalofAppliedBehaviorAnal-ysis,28(1),87-88.Silverplatterinformationretrievalsystem[on-line].(2001).Norwood,MA.Available:http://www.silverplatter.com/Skinner,B.(1954).Thescienceoflearningandtheartofteaching.HarvardEducationalReview,24(2),86-97.Smith,V.B.(1974).Individualizedself-pacedinstruction:Potentialandproblems.InR.S.Ruskin&S.F.Bono(Eds.),Personalinstructioninhighereducation:Proceedingsoftherstnationalconference(p.182-186).Washington,DC:CenterforPersonalizedInstruction.120

PAGE 136

Stevens,J.(1986).Appliedmultivariatestatisticsforthesocialsciences.Hillsdale,NJ:LawrenceErlbaumAssociates,Inc.Stevens,J.(1990).Intermediatestatistics:Amodernapproach.Hillsdale,NJ:LawrenceErlbaumAssociates,Inc.Sweeney,P.D.,Butler,R.A.,&Rosen,L.(1979).Themodicationandpredictionofprocrastinationinapersonalizedinstructioncourse.InAnnualconventionoftheamericanpsychologicalassociation(87th).NewYork.Watson,A.(1990).Competency-basedvocationaleducationandself-pacedlearn-ing(Tech.Rep.No.143).Sydney,Australia:TechnologyUniversity.Welsh,T.M.,Malott,R.W.,&Kent,H.M.(1980).Theuseofbehavioralcon-tractingtoeliminateprocrastinationinapsicourse.JournalofPersonalizedInstruction,4(2),103-104.Wesp,R.(1986).Reducingprocrastinationthroughrequiredcourseinvolvement.TeachingofPsychology,13(3),128-130.Wesp,R.,&Ford,J.E.(1982).Flexibleinstructorpacingassistsstudentprogressinapersonalizedsystemofinstruction.TeachingofPsychology,9(3),160-162.Wilkes,G.(1996).Movingtolearner-centeredenvironmentsinanageofelectronicinformation[on-line].Available:http://www.utc.arizona.edu/tact3-3.html121

PAGE 137

Appendices122

PAGE 138

AppendixAPilotStudyConductedDuringFall1999ApilotstudywasconductedduringFall1999with18volunteersfromEdTech.Initially,30studentsexpressedinterestinjoiningthestudythatwasscheduledtotakeplaceduring6one-hoursessionsconductedonconsecutiveFridaysinateachinglaboratoryequippedwith30computers.Studentssignedupforone-hourtimeslotsbetween9a.m.and5p.m.Thentheywererandomlyassignedtooneoftwogroups,recommendeddeadlines(R)versusabsolutedeadlines(A).The2one-hourtimeslotsthataccommodatedthemoststudentswerebetween11a.m.and12p.m.andbetween12p.m.and1p.m.Thisreducedthenumberofeligibleparticipantsto22.Studentswererequiredtoattendall6one-hoursessionsofthepilotstudyinordertoreceivethe20extracreditpointsofferedascompensation.Ontherstday,12studentswerescheduledtoattendtherstsessionand10werescheduledtoattendthesecond.Onestudent(#13)1scheduledtoattendtherstsessionrequestedtochangetothelattersessionduetoatimeconict.Alloftheother11studentssignedupfortherstsessionshoweduptherstday.Ofthese11,9attendedall6one-hoursessions.Only5ofthe10studentssignedupforthesecondsessionshoweduptherstday.However,oneextrastudent(#23)showeduprequestingtoenterthestudyeventhoughshehadnotsignedupforit.Students#13and#23broughtthetotalnumberinthesecondsessionto7ontherstday.Ofthese7,6attendedallone-hoursessions.Itshouldbenotedthat,onoccasion,studentswhowerenotabletoattendoneofthesessionswereallowedtomakeupthemissedtimebyattendingbothone-hoursessionsthefollowingFriday.AllstudentsreceivedacopyoftheinformationinAppendixB.Thosesectionsreceivedonlybystudentsinagiventreatmentarenoted.Thecoursewareandassignmentsusedinthestudyreplacedanassignmentintheregularcourseworth20points.Inordertoenticevolunteerstojointhestudy,theywereoffered20extracreditpointsascompensationfortheirtime.Anyassignmentstheycompletedduringthepilotstudycountedasadditionalpointstowardtheclass.Also,thescoretheyearnedontheposttestgainedthemadditionalpoints.Inessence,studentsonlyhadtoattendthe6one-hoursessionstoearnthe20pointsfortheregularclassassignment.Anyworktheydidduringthepilotstudycountedasbonuspointstowardtheclass.Somestudentstookadvantageofthisopportunityandcompletedmanyoftheassignments.Mostappearedtobeactivelyaccessingthecoursewareduringthesessions.However,atleastonestudent(#8)wasobservedreadinge-mailduringoneofthesessions.Presumablythen,some 1Amasterlistofstudentsindicateswhichnumbercorrespondstoeachstudent.Inordertoensuretheanonymityofthestudents,theyareonlyreferredtobynumberherein.123

PAGE 139

AppendixA(Continued)ofthestudentsjustattendedthesessionstocollecttheir20pointsanddidnotseriouslypursuethecoursematerial.Infact,ofthe15studentswhoattendedallone-hoursessions,5turnedinnoassignments.Ofthose5,observationsmadeduringthesessionsrevealedthatatleast2ofthem(#23and#16)didactivelyengageinthecourseware.Becausethestudentsreceived20extracreditpointssimplyforattendingthesessions,thevalidityofanyconclusionsdrawnabouttheeffectsofdifferentdead-linecontingenciesissomewhatquestionable.However,acarefulrecordwaskeptofwhenstudentssubmittedeachassignment,anditisinterestingtonotethat,inkeepingwiththendingsofotherresearchers,studentsinRtendedtoprocras-tinatemorethanstudentsinA.Figure28graphicallydepictsthisbyshowingtheaveragedayeachassignmentwassubmittedbystudentsineachtreatment.Sub-missionswereacceptedonfourFridaysduringthestudy.AnyreceivedontherstFridaywererecordedashavingbeensubmittedondayone.ThosereceivedonthesecondFridaywererecordedashavingbeensubmittedondaytwo,etc.Thosethatwereneverreceived,wererecordedashavingbeensubmittedondayve.Forallassignments,exceptthelastone,studentsinRsubmittedtheirworklaterthanstudentsinA.Consequently,studentsinRwerenotabletoturninasmanyas-signmentsandearnedlowerassignmentscores.Infact,Table14listsanaverageassignmentscoreof4forstudentsinRascomparedtoanaveragescoreof8forstudentsinA.Alsoinagreementwiththendingsofotherresearchers,Table14showsthatstudentsinbothtreatmentsperformedsimilarlyonseveralmeasuresofachievement.Eventhoughtheeffectsofrecommendedversusabsolutedeadlinesweredif-culttodetermineduetothesmallsamplesizeandthethelargenumberofextracreditpointsawardedforattendance,valuableinformationwasobtainedregardingtheadequacyofthecourseware,theappropriatenessoftheassignments,there-liabilityoftheachievementmeasures,andtheneedtotrackstudentrequestsfordeadlineextensions.Theseissues,alongwithchangesmadetothecoursewarearediscussedinAppendixC.Toreviewtherawdatafromthepilotstudy,pleaseseeTable15.124

PAGE 140

AppendixA(Continued) 2 3 4 5 1 2 3 4 5 6 7 8 AssignmentDay SubmittedR A Figure28.Procrastinationlevelsduringpilotstudy.Table14.SummaryofPilotStudyPerformanceandAchievementData AssignmentPosttestquestions TreatmentGrade37a26b18cE1dE2e RecommendedM41813945SD533222n666666AbsoluteM81812954SD745332nf988877 Note.Pilotandactualstudyessayquestionswereidenticalwiththeexceptionofobjec-tives4band5c.aAllpilotmultiplechoicequestions.bPilotmultiplechoicequestionsthatappearonactualposttestinsomeform.cPilotmultiplechoicequestionsthatappearonactualposttestinexactform.dEssayquestionsratedon1-11-2000.eEssayquestionsratedon1-12-2000.fMultiplechoiceresponsesweremissingforonestudent,andessaysweremissingfortwo.125

PAGE 141

AppendixA(Continued)Table15.RawDatafromPilotStudy AssignmentcPosttest IDaIb12345678G37d26e18fE1gE2h Recommendeddeadlines4x33------3231711566111244-31318139667x3334----7201511661342------314107228--------0141163323x--------018131044 Absolutedeadlines11x11-2-3--7181398712111-3---72520133314111242341519141088151114444-1312753420x2222234-132317125518x11124343151637--16--------0---3217--------016127--19--------0149722 Maximumpossiblepoints15i37261899Averagepointsearnedbyallstudents61812955 Note.Dashesinassignmentcolumnsindicatethosethatwereneversubmitted.Dashesinposttestcolumnsindicatemissingdata.aIdenticationnumberassignedtostudenttomaintainanonymity.bStudentsinterviewedonthelastday.cFridayassignmentwassubmittedatmasterylevel.dAllpilotmultiplechoicequestions.ePilotmultiplechoicequestionsthatappearonactualposttestinsomeform.fPilotmultiplechoicequestionsthatappearonactualposttestinexactform.gEssayquestionsratedon1-11-2000.hEssayquestionsratedon1-12-2000.iDoesnotincludeJavaScriptassignment,whichnoonecompleted.126

PAGE 142

AppendixBInstructionsGiventoPilotStudyParticipantsExtraCredit Youreceive20extracreditpointstowardyourgradeinEdTechforparticipatinginthisstudy.Inordertoparticipatefullyandreceiveanyofthe20points,youmustattendall6one-hoursessions.SemesterProjectEFG TheassignmentsinthisstudyreplacetheSemesterProjectEFG,whichisworth20points.Bycompletingtheassignmentsinthisstudy,youcanearnupto20points(plus5extrapointsfortakingthepostteston10/29/1999)towardProjectEFG.Thatmeansyoucanearnatotalof45pointsbycompletingthestudy,alloftheassignments,andearning100%onaposttestcoveringthecoursewaremate-rial.Thisisincontrasttothemaximum20pointsyoucouldearnforProjectEFGifyoudidnotparticipateinthestudy.Pleasenotethat,inordertoreceivethe5pointsforcompletingtheJavaScriptassignment,youmusthavesuccessfullycompletedallotherassignmentsinthecourseware.Tocompleteanassignmentsuccessfullymeansthatyouearnedfullcreditforit.That,inturn,meansthatyoucompleteditbythedeadlineandthatyoulostnopoints.Afterlookingthroughthecoursewarepackageforthestudyandconsideringtheassignmentsyouwillbeaskedtocomplete,youshoulddecideifyouwouldliketocontinueinthestudy.Alternatively,youcanterminateyourparticipationandcompleteProjectEFGinstead.Youarefreetoterminateparticipationinthestudyatanytime.JustkeepinmindthatyouwillneedtoleaveyourselfenoughtimetodoProjectEFGinstead.DuringOne-hourSessions Youwillprogressthroughthematerialinaself-pacedmanner,listeningtothenarrationsforeachpageusingheadphones.Ifatanypoint,youhaveaquestion,youcanpausethenarrationandlettheteachingassistantknow.Shewillhaveherownsetofheadphonesthatshecanplugintoyourcomputeraswell.Youcanreplaythepartyouhaveaquestionabout,listentothenarrationtogether,andthenposeyourquestion.127

PAGE 143

AppendixB(Continued)Youmayasktheteachingassistantforhelpwhencompletingassignments.Youmayalsoworktogetherwithfellowstudents.Timepermitting,theteachingassistantcanprecheckyourworkbeforeitisdueandletyouknowifyouwouldearnfullcreditfortheassignment.Ifso,youcanturnintheassignmentearlyandhaveitcheckedoffatthatpoint.SubmittingAssignmentsandBringingaFloppyDisk Youwillsubmityourassignmentsduringclass.Timepermitting,theteacherwillcheckthemwithyou,letyouknowhowyoudidrightaway,andrecordyourgrade.Again,timepermitting,youarewelcometogetfeedbackonyourassignmentsbeforeyousubmitthemforgrading.Thecoursewarerecommendsthatyoubringaoppydisktoclasstosaveyourworkforyourself.Youreallyshouldbring2disks(atleastoneisrequired),sothatyoucangiveonetoyourteacherinthecasethattimedoesnotpermithertocheckyourworkoffduringclass.Itisyourresponsibilitytoprovideherwiththisdiskinthatevent.OnlineVersionofCourseware Ifyouwouldliketolookthroughitoutsideofclass,thecourseware(minusnar-ration)willbeavailableonlineathttp://www.math.usf.edu/tmajchrz/courseware/.RecommendedDueDatesandPoints (giventoparticipantsintreatmentRonly)Ifyouhopetocompleteallassignmentsbytheendofthecourse,itisrecom-mendedthatyoufollowanassignmentcompletionschedule.SeeTable16forasummaryoftherecommendedduedatesandtotalpointsforeachassignment.Thesedeadlinesaremerelyrecommended.Youmaysubmitallassignmentsonthelastdayifyouwish.However,youarewelladvisednottowaituntiltheendtodothemall.Inaddition,waitinguntiltheendwillnotallowyoutimetogetfeedbackfromtheteachingassistantonhowyouareprogressing.Pleasenotethatallassignments,withoutexception,aredueon10/29/1999.Absolutelynoassignmentswillbeacceptedforcreditafterthecloseofclasson10/29/1999.Ifyoudonothaveapreferenceonwhatorderyoucompletethese128

PAGE 144

AppendixB(Continued)assignmentsin,justcompletethemintheorderinwhichtheyarelistedinthecourseware.Table16.DueDatesforTreatmentRofPilotStudy RecommendedAssignmentPointsDueDate Texttemplate.htm110/8/1999index.htm210/8/1999personal.htm210/8/1999Lists2*Images2*Tables2*Frames2*Forms2*JavaScriptLite510/29/1999 2Anytwooftheseitemsshouldbecompletedby10/15/1999.Twomoreshouldbecompletedby10/22/1999.Theremainingoneshouldbecompletedby10/29/1999.AbsoluteDueDatesandPoints (giventoparticipantsintreatmentAonly)Table17containsasummaryoftheduedatesandtotalpointseachassignmentisworth.Thedeadlinesareattheendofclassthatdayandareabsolute.Anassignmentabsolutelywillnotbeacceptedforcreditafteritsdeadline.Youmaystillshowittotheteachingassistantafterthedeadlineandreceivefeedbackonit,butitwillnotearnyouanypoints.Ifyoudonothaveapreferenceonwhatorderyoucompletetheseassignmentsin,justcompletethemintheorderinwhichtheyarelistedinthecourseware.129

PAGE 145

AppendixB(Continued)Table17.DueDatesforTreatmentAofPilotStudy AssignmentPointsDue Texttemplate.htm110/8/1999index.htm210/8/1999personal.htm210/8/1999Lists2*Images2*Tables2*Frames2*Forms2*JavaScriptLite510/29/1999 2Anytwooftheseitemsaredueon10/15/1999.Twomorearedueon10/22/1999.Theremainingoneisdueon10/29/1999.130

PAGE 146

AppendixCCoursewareDescriptionCoursewareonHTML4.0wasdesignedforthisstudy.ItwaswritteninHTMLandJavaScript.Acurrentversionisavailableonlineathttp://tarski.math.usf.edu/3tmajchrz/IPcourse/index.htm.Version1.1wasgiventostudentsinapilotstudy.Basedonobservationsandstudentcommentsversion1.2wasdevelopedfortheactualstudy,whichtookplaceduringSpring2000.ThisversionisontheenclosedCD-ROM(seeAppendixU).SeeFigure29forascreenshotofthehomepagefor1.2andtoseethelayoutoffunctionalunits.Noticethatthemainmenuappearsinanavigationbarontheleft.Soundcontrols,backandforwardnavigationelements,acommentbutton,andapromptareaappearonthebottom.Whentheuserisinagivensectionofthecourseware,themaintopicishighlightedinthenavigationbarontheleft,thetitleforthesectionappearsacrossthetopalongwiththepagenum-ber,andthemiddleportionofthewindowisreservedforcontent.SeeFigure30foranexample. Figure29.Homepageforversion1.2ofcourseware.Ingeneral,thecoursewareincorporateselementsofactivelearningandkeepsindividuallessonsshort.Thenarrationforeachlessonisunder11minutes.Inad-dition,studentsareencouragedduringlessonstotryoutnewknowledgedirectly131

PAGE 147

AppendixC(Continued) Figure30.FirstpageofsectiononlistsinHTMLtutorialinthecourseware.Theyareprovidedwithatextboxinwhichtoexperiment,di-rectionsonwhattotry,andabuttontoclickwhenreadytoviewtheresults.SeeFigure31foranexampleofsuchanactivelearningopportunity,wherethestudentisaskedtoexperimentwiththewidthandheightofanimage.Figure32showsthewindowthatpopsupwhentheLet'sSeeItbuttonisclickedafterawidthof67andaheightof66arespecied.Figure33showshowtheimagelooksforawidthof35andaheightof66.Otheraspectsofthecoursewarearediscussedinmoredetailinthesectionsthatfollow.Changesinspiredbythepilotstudyaredescribedrst.Nextthecourse-wareobjectives,lessons,assignments,andsystemrequirementsarediscussedforversion1.2.132

PAGE 148

AppendixC(Continued) Figure31.Activelearningopportunity.Thestudentisaskedtoexper-imentwiththewidthoftheimagegeneratedbythegivenHTMLcode.WhenthestudentclickstheLet'sSeeItbutton,anotherwindowpopsupdisplayingtheimage.ChangesInspiredbyPilotStudySixstudentsfromthepilotstudy,threefromeachofthetwodeadlinecontingen-cies,wereselectedrandomlyandinterviewedonthelastdayofthestudy.Severalchangestothecoursewarewerepromptedbytheirresponses.Thefree-responsequestionsposedandtheiranswersappearbelow.Inthecasewhenthesamere-sponsewasmadebymorethanonestudent,thetotalnumbergivingthatresponseappearsinparenthesesafterit.133

PAGE 149

AppendixC(Continued) Figure32.Imagedisplayedforawidthof67.1.Howdifcultwasittogureouthowtogetaroundinthecoursewareinitially?VeryhardHard(3)EasyVeryeasy2.Howhardistogetaroundinnow?TagsandattributesharderHard,buteasierEnjoynowEasyAloteasierFairlyeasy134

PAGE 150

AppendixC(Continued) Figure33.Imagedisplayedforawidthof35.3.Howdidyouendupnavigatingthesite(assignmentdescriptionortutorialrst)?Assignmentrst(3)Tutorialrst(3)4.Didyouknowyoucouldaskquestionswhilegoingthroughthecourseware?NoNo,wouldhavehelpedYes(4)135

PAGE 151

AppendixC(Continued)5.Howusefulwerethetimings?Didtheyseemtomatchthelengthoftimeittookyou?Didnotsee(2)Didnotuse(2)Useful,seemedtomatchUseful,notsureaboutmatchbutgaverelativeinformation6.Wouldyourecommendanynavigationalchanges?None(4)LinkfromassignmenttorelevanttutorialEasytoforgetwhereyousawit;addindex7.Didthedeadlinesseemspacedaboutright?Ignored,becausehad20pointsalready(2)Toofast(2)LastcoupletoofastAboutright8.Howmanyhoursperweekdidyouworkontheassignmentsoutsideofclass?(Notethattheresponsesaveragetothree-fourthsofanhour.)Zero(3)One(2)Twoandone-half9.Wouldyoundagraphicalgradingrubricliketheteacher'seasiertousethanthegradingrubriclistprovidedinthecourseware?Yes(3)No(2)Wantboth136

PAGE 152

AppendixC(Continued)10.Arethereanyotherchangesyouwouldrecommendforthecourseware,labexpe-rience,assignments,etc.?(Notethatsimilarresponsesaregroupedtogether.)Augmentcourseware–Makeobjectivesmoreclearinbeginning–Helpgettingstartedonrstassignment–IncludeglossaryIncreaseexibilityofaccess–Fivedaysinarowratherthanonedayperweek–Accesstosoundathome–Oneandone-halfhoursperweekatanytimeorsoundathome–Eightweeks,ratherthansixFacilitatenotetaking–Encouragenotetaking–Provideworkbookorplacetotakenotes–PapercopyofassignmentsLikedLet'sSeeIt!opportunitiestoexperimentInversion1.1ofthecourseware,amenuontheleftcontaininglinkstoas-signmentdescriptionsandtutorialsalloweduserstomovebackandforthbetweentheseitems.In1.2thatmenustillexists,butnavigationwasaugmentedwithdirectlinksfromassignmentdescriptionstopertinenttutorialsandwithdirectlinksfromtutorialstoassignmentdescriptions.Thischangewasmadeinresponsetothecommentsprovidedbystudentsforquestions1,2,3,and6.Furthermore,inre-sponsetothecommentsforquestion6,anindexwasprovidedtofacilitatelocatinginformation.Inadditiontotheindexlinkaddedtothemainnavigationalmenuontheleft,alinktoe-mailtheirrespectiveTAwasprovidedtoencouragestudentstoaskquestionswhilereviewingthecoursewarematerial.TheywerealsoencouragedatthebeginningofthecoursewaretovisittheirTAduringofcehoursforaidifdesired.EachTAwasavailableinadesignatedcomputerlaboratoryforfourhoursperweek.Eachwasalsoavailableintheircommonofceafewadditionalhoursperweek.137

PAGE 153

AppendixC(Continued)Therstsetofstudentresponsestoquestion10shedlightonsomekeyin-formationthatwasinadvertentlyleftoutofversion1.1.Thecoursewaredidnotincludeformalinstructiononthedevelopmentalprocessrequiredtocreateandup-dateaWebpageusingasimpletexteditorandabrowser.Thisprocessbeginswiththecreationoftheinitialdocumentusingatexteditor.Thenthedocumentisviewedinabrowser.Next,thecreatorcyclesbetweensavingchangesintheeditorandselectingReloadorRefreshinthebrowser.Aformallessonoutliningthisprocesswasaddedtoversion1.2,alongwithanassignmenttotypesuppliedHTMLcodeintoaneditorandtoviewitinabrowser.Duetotimeconstraints,theadditionofaglossarywasleftforafutureversionofthecourseware.SomestudentsinthepilotstudyindicatedtheyfeltthatportionsoftheHTMLcoursewareweretoofast(seeresponsestoquestion7).However,theyonlyhadaccesstothecoursewarenarrationforatotalofsixhoursinthelab.Theyhadaccesstoanonline,soundlessversionoutsideofclass.EachstudentintheactualstudywassuppliedwithaCD-ROMcontainingthecompletecourseware.Itwasanticipatedthathavingaccesstothecourseware,includingsound,ondemandwouldgivestudentstheextratimetheyneededtonishtheassignmentsandnotfeelrushed.Itwashopedthatitwouldprovidethemwithdesiredexibility(seesecondsetofresponsestoquestion10).Responsestoquestion8indicatedthat,onaverage,studentseachspentthree-fourthsofanhourperweekworkingontheassignmentsoutsideoflabtime.Itwasanticipatedthat,ifthestudentsdevoted4hoursperweek(oratotalof16hours)totheassignments,theywouldbeabletocompletethemalleasily.Insupportofthis,consideronlythosetenstudentswhoactuallycompletedassignmentsdur-ingthepilotstudyandtable??,whichcontainsasummaryofthedatacollectedregardingassignmentcompletiondatesandachievementtestscores.Thesestu-dentswereabletocomplete,onaverage,5.3outof8assignmentsor66%oftheassignments,withtheirmainmotivationsbeingtoearnbonuspointsandanintrin-sicdesiretolearn.Becausethevestudentsinterviewedfromthisgroupoftenreportedworkingoutsideofclass,onaverage,oneadditionalhourperweek,thatmeanstheywereabletocomplete5.3assignmentsin12hours.Therefore,itisassumedthat16hoursshouldbeampletime1formoststudentstocompletealloftheassignments,especiallygiventhemotivationtoearnrequiredclasspoints.Afewmorechangesweremadetoversion1.2ofthecourseware.Basedonresponsestoquestion9,thegradingrubricswerechangedfromtextualliststo 1Itshouldbenotedthatthisassumptionwasmadebasedondatafromvolunteerswhorepresentedadistinctsegmentoftheaccessiblepopulation.However,itwasthebestevidenceavailableatthetimefromwhichtodrawsuchaconclusion.138

PAGE 154

AppendixC(Continued)graphicalrepresentations.SeeFigure34foranexampleofanoriginalgradingrubriclistandFigure35foranexampleofagraphicalgradingrubric.Bothrubricsofferbasicallythesameinformation,butinaslightlydifferentformat.Bothareaug-mentedwiththesameaudioinformation.Studentresponsesindicatedthatsomewouldndtheswitchtoagraphicalrepresentationhelpful,whilenostudentsvoicedanopinionthatsuchachangewouldbedetrimental.Anotherchangeincludedadiscussionofthesoundtimingsaddedtoanearlycoursewaresectiononhowtonavigatethesite(seeFigure36).Itwashopedthatstudentsintheactualstudy,unlikestudentsinthepilotstudy(seeresponsestoquestion5),wouldallnoticethisinformationandnditusefulinbudgetingtheirtime.Finally,inresponsetothethirdsetofcommentsforquestion10,lessonsandassignmentsweremadeavailableinaformthatwaseasytoprint.Eachstudentwasabletodecidewhichportion(s)ofthecoursewaretoprint,ifany.ObjectivesandLessonsThemainobjectivescoveredbythecoursewareincludedlearningbrowserba-sics,thedevelopmentprocess,designandstyleissues,HTMLdocumentstructure,howtousetagsandattributesingeneral,howtochangetheappearanceoftext,andhowtoincludelists,images,tables,frames,andformsonaWebpage.Foramoredetailedlistingofthecoursewareobjectives,seeAppendixD.Notethattheseobjectiveswerenotstatedexplicitlyinthecourseware,butratherwerestatedimplicitlyinthesectionthatdescribestheoverallproductthestudentscreated.AlistingofthecoursewarelessonswiththeircorrespondingassignmentsandreferencematerialappearsinFigure3.Intherstlessonondevelopment,stu-dentslearnedhowtomakechangesinatexteditorandhowtoviewtheresultsinabrowser.Inthelessonondesign,theylearnedaboutissuessuchasusingtemplateles,maintainingaconsistentlookacrosspages,makingtextreadable,andusingsmallimageles.Inthelessonsassociatedwithassignmenttwo,theylearnedabouttagsandattributesingeneralaswellasabouttheoverallstructureofanHTMLdocument.Next,theylearnedhowtochangetheappearanceoftext.Forassignmentfour,theylearnedhowtocreatebulletedandnumberedlists.Forassignmentve,theylearnedhowtoincludesimpleimagesandclickableimagemapsontheirWebpagesaswellashowtoswapinnewimagesdynamically.Forassignmentssix,seven,andeight,respectively,theylearnedhowtocreatetables,frames,andforms.Studentswhowishedtolearnmoreaboutagiventagand/orattributewereencouragedtoutilizethereferencematerialdevelopedbytheWeb139

PAGE 155

AppendixC(Continued) Figure34.Gradingrubricinlistformat.DesignGroup(Quinn,1998)2viaconvenientlinksnexttoeachassignment.Thematerialwasalsoaccessiblefromthecourseware'smainnavigationbarunderref-erences.Infact,inordertocompletethesecondassignment,studentsweretoldtolookupinformationinthisonlinereference.Thehopewasthat,uponcompletionofthecourse,theywouldbeabletoexploreHTMLfurtherontheirownandhavetheskillsneededtotrackdownanswersforthemselves. 2AcopyofthematerialwasprovideddirectlyontheCD-ROMaswaspermittedbytheoriginalauthor.140

PAGE 156

AppendixC(Continued) Figure35.Gradingrubricingraphicalformat.AssignmentsInordertoobtain100%forthisportionofEdTech,whichwasworth60outofapossible430classpoints,astudentneededtocompletealleightofthecourse-wareassignments(36points)andanswerallquestionsonaposttestcorrectly(24points).Eachassignmentwasworth4points,withtheexceptionofassignment4,whichwasworth8(seeFigure??).Assignment4wasworthmore,becauseitincorporatedanotherassignmentfromEdTechinwhichstudentsreportedinfor-mationtheyfoundonergonomics.TheeightassignmentsculminatedintheproductionofapersonalWebsite.SeeFigure37foranoverviewofthesiteproduced.Aclickableimagemapontheinitialpageincludedlinkstopagescontaininginformationaboutapersonalproject,alistingofprofessionalexperience,andlinkstoInternetsitesoneduca-tionandergonomics.TheentiresiteconsistedoffourWebpages,oneofwhich141

PAGE 157

AppendixC(Continued) Figure36.Coursewarenavigation.wasaframesetcomprisedofsixles.Inall,thestudentscreatedninelesandincorporated18suppliedimages.TherstassignmentfamiliarizedthemwiththedevelopmentprocessbyhavingthemtypeinagivenHTMLdocument.Figure38showshowthisdocument,thehomepagefortheirsite,shouldlookwhenrenderedbyabrowser.Thesecondassignmentrequiredthemtodetermineacolorschemefortheirsiteandtocreateatemplatelebasedonit.Forassignment3,theycreatedanHTMLdocumentusingstructuralandlogicaltagsonly(seeFigure39).Assignment4requiredthattheydosomeresearch.Theyhadtolocateamin-imumoffourInternetsitesonergonomicsandtodisplaythisinformationasan142

PAGE 158

AppendixC(Continued) Figure37.Overviewofcoursewareproductgoal.unbulletedlistofannotatedlinks(seeFigure40).Onesiteneededtodiscussthedangersofelectromagneticradiation.Theotherthreeneededtodiscusstheef-fectsofcomputeruseontheeyes,thearmsandhands,andtheskeletonaswellasmethodsforprotectingthesesystems.Studentsaddedsimpleandclickable3imagestotheirpagesinassignment5.Theseimagesweredecorativeaswellasfunctionalnavigationalelements.Swap-pingbetweenthemprovidedenhancedfeedbacktoanysitevisitorwhomovedthemouseoverthem.SeetheenhanceddocumentsinFigures41and42.ForexamplesoftheeffectsofthemouseOverevents,seeFigures43and44.Inassignment6,theycreatedapagethatcontainedatablelistingtheirpro-fessionalqualicationsintermsofcomputerexperience(seeFigure45).Speci-cally,itcontainedalistingofthecomputerlanguages,environments,andtoolswith 3Allcoordinatesforclickableregionsweresuppliedinthecourseware.143

PAGE 159

AppendixC(Continued) Figure38.Initialindexpageofproduct.whichthestudentwasfamiliar.Inassignment7,theyupdatedtheirlinkspagetouseframesandtodisplayeducationalaswellasergonomiclinks(seeFigures46,47,and48).Finally,forassignment8,theyupdatedtheirpersonalpagebyaddingtheforminFigure49.Itcalledanonlinecgi-scriptprovidedforthecourse.SystemRequirementsVersion1.2ofthecoursewarerequiredtheuseofMicrosoft'sInternetExplorer4.0or5.0.InthecaseofInternetExplorer4.0,itwasfurtherrequiredthatMi-crosoft'sMediaPlayer2beinstalled.TheCD-ROMincludedinstructionsonhowtogainaccesstotheseprograms.Duetolimitedtimeandthefactthatthecourse-warewasstillinanearlystateofdevelopmentforresearchpurposes,extensivetestingondifferentbrowsersandcomputerplatformswasnotconducted.There-fore,theonlybrowserofciallysupportedforthisversionofthecoursewarewasInternetExplorer5.0forWindows,althoughInternetExplorer4.0forWindowswithMediaPlayer2wasareasonablealternative.Thiswasthebrowserofchoicefortworeasons.First,itplayedMP3leswithinthebrowserinaconsistentmanner.144

PAGE 160

AppendixC(Continued) Figure39.Initialpersonalpageofproduct.Second,theresultsofasurveyconductedduringFall1999showedthatthevastmajorityofstudentswouldhaveaccesstoaWindows-basedmachine.ThesoundlesforthecoursewareweresavedastypeMP3,sothattheywouldalltonasingleCD-ROM.TheWAVversionsofthesoundlesweretoolargetotonasingleCD-ROM.InternetExplorer4.0withMediaPlayer2andInter-netExplorer5.0(whichcamewithMediaPlayer2)weretheonlybrowser/mediaplayercombinationsfoundatthetimetoplaytheMP3lesconsistentlybymeansofabrowserplugin.OthermediaplayersconsideredincludedRealPlayerG2,QuickTime4.0,andWinAmp.NoneoftheseoptionshadpluginsthatworkedwellconsistentlyunderNetscape4.6or4.7intestsperformed.Consistent,workablepluginsforInternetExplorer4.0and5.0alsowerenotavailableforthesealter-nativeplayers.DuetotheunmanageablesizeoftheWAVsoundlesandthedesiretohavenarrationsplayandbecontrolledinaconsistentfunctionallocationofthecourseware,asopposedtoaseparatewindowthatmaychangelocationsand/orgetconcealedbyotherwindows,MP3lesandtheInternetExplorer/MediaPlayer2combinationwereused.Futureversionsofthecoursewarewillsupportalternativeoptionsastheybecomeavailableandstable.145

PAGE 161

AppendixC(Continued)Outof233studentsenrolledinEdTechduringFall1999,167(71.7%)re-spondedtoasurvey(seeAppendixR)givenonthelastdayofclass.Itrequestedinformationonthetypesofmachinesandbrowsersthestudentsmostoftenusedtocompleteclassassignments.ItwasassumedthatstudentsenrolledinEdTechduringSpring2000wouldtasimilarprole.Basedonresponses,94.6%reportedthattheymostoftenusedaWindows-basedPC4.Therefore,giventimeconstraints,rigoroustestingofthecoursewarewasconnedtotheWindows-basedPCenvi-ronment.Surveyresultsalsoindicatedthat29.9%ofthestudentsusedsomeversionofNetscapeNavigator,39%usedsomeversionofInternetExplorer,andanunexpected32.3%usedsomeversionofAOL5.Ideally,theNetscapeNaviga-tor,InternetExplorer,andAOLbrowsersshouldallbesupported.However,asmentionedabove,timeconstraintsandthelackofaviablepluginforplayingMP3sinlineinNetscapeNavigatormadeitunfeasibletosupportthisbrowseratthistime.Uponacursoryexamination,itappearedthatAOL5.0,atleast,mightbeaviablebrowseroption,becauseitplayedMP3sinline.Onanalnote,67.7%ofthestudentsreportedworkingathome,19.8%re-portedworkingatanopenusecomputerlaboratoryintheEducationbuilding6,and4.8%reportedworkingatbothlocationsjustasoften.Individualcomputerswerecheckedineachlabandfoundtorunthecourseware,includingnarrations.GiveninstructionssuppliedwiththeCD-ROM,studentswereabletoinstalltherequiredbrowserandmediaplayerontheirhomesystems,ifnecessary.Ifnot,theyhadaccesstothemachinesinthecomputerlabuntiltheywereabletodoso.Stu-dentswereadvisedtobringheadphonestothelabwiththem,becauseonlytenheadphoneswereavailableinthelabsduringthetreatmentinterval. 4OnepersonactuallyreportedusingbothaPCandaMacintoshequallyoften.5Notethatthepercentagestotalmorethan100%,becausesomestudentsreportedusingmorethanonebrowserregularly.6Thereweretwosuchcomputerlaboratories.Onehad33computers,theother23.146

PAGE 162

AppendixC(Continued) Figure40.Initiallinkspageofproduct.147

PAGE 163

AppendixC(Continued) Figure41.Indexpageofproductwithclickableimage. Figure42.Personalpageofproductwithnavigationalimages.148

PAGE 164

AppendixC(Continued) Figure43.Indexpageofproductwithmouseoverprofessional. Figure44.Personalpageofproductwithmouseoverprofessional.149

PAGE 165

AppendixC(Continued) Figure45.Professionalpageofproduct. Figure46.Initialframesforlinkspageofproduct.150

PAGE 166

AppendixC(Continued) Figure47.Framesforlinkspageofproductafterclickingeducation. Figure48.Framesforlinkspageofproductafterclickingergonomics.151

PAGE 167

AppendixC(Continued) Figure49.Personalpageofproductwithform.152

PAGE 168

AppendixDCoursewareObjectivesThetenmainobjectivesofthecoursewarearelistedbelow.Thevaluesinbracketsaretherelativeweightingsgiventoeachobjectiveintermsofthenumberofitemsincludedonthepretest(Pre),theposttest(Post),andtheretentiontest(Ret).Per-centagesarelistedinparentheses.Recallthatthepretestandretentiontestarecomprisedofthesame9multiplechoiceitemsandthattheretentiontesthasanadditionalessayquestiongradedbasedon12items.Recallalsothatthereare48itemsontheposttest,whichconsistsof36multiplechoicequestionsand1essayquestioncomprisedof12items.Eachessayitemontheposttestwasworth4points,whileeachmultiplechoiceitemwasworth1.1.BrowserBasics[1Pre(11%),2Post(2%),1Ret(5%)](a)viewapagecreatedandsavedonthelocalmachine(b)recognizetheforgivingnatureofHTMLinterpreters2.Development,Design,andStyle[1Pre(11%),3Post(4%),1Ret(5%)](a)understandtheimportanceofwritingreadableHTMLcode(b)understandthemeritsofusingatemplatele(c)beawareoftheneedtoRefresh/Reloadadocumenttoseechanges3.DocumentStructure[0Pre(0%),6Post(29%),6Ret(29%)](a)includealinktoanotherpage(b)demonstrateknowledgethatHTMLdocumentsarecomprisedoftwomainsections,theHEADandtheBODY(c)demonstrateknowledgethatHTMLdocumentsaredesignatedwithopeningandclosingHTMLtagsthatsurroundthecontent4.TagsandAttributes[1Pre(11%),7Post(26%),6Ret(29%)](a)usetagsandattributescorrectly(b)demonstrateanunderstandingofhowtagsandattributesworkbybe-ingabletolookupandusetagsandattributesnotdiscussedformallyinthecourseware(c)setthetitledisplayedinthetitlebarofthebrowser(d)setthebackground,text,andlinkcolors153

PAGE 169

AppendixD(Continued)5.TextStyle[1Pre(11%),5Post(11%),2Ret(10%)](a)physicallymarkuptext(bold,changerelativefontsize)(b)logicallymarkuptext(headinglevel)(c)centertext6.Lists[1Pre(11%),5Post(6%),1Ret(5%)](a)includeanordered(numbered)orunordered(bulleted)list(b)includeanumberorbullet(c)setthebullettypeforabulletedlist7.Images[1Pre(11%),5Post(6%),1Ret(5%)](a)includeasimpleimage(b)appropriatelysettheCOORDSattributeoftheAREAtaginaclient-sideimagemap,givencoordinatesforaclickableregiononanimage(c)turnofftheborderforaclickableimage(d)usethemouseOverandmouseOutattributesoftheIMGtag8.Tables[1Pre(11%),5Post(6%),1Ret(5%)](a)usetherowspan,nowrap,andvaligncellattributes(b)settheattributesofatable(c)designatetablerows(d)designateheaderanddatacells9.Frames[1Pre(11%),5Post(6%),1Ret(5%)](a)createaframesversionofaWebsite(b)setthetargetofalinktoaspecicframe,tothetoplevelwindow,ortoanewwindow(c)usetheNOFRAMEStagtodisplayalternativecontent10.Forms[1Pre(11%),5Post(6%),1Ret(5%)](a)includeanINPUTelementofTYPEtext(b)includeanINPUTelementofTYPEradio(c)includeanINPUTelementofTYPEreset(d)includeanINPUTelementofTYPEsubmit(e)callacgi-scripttoprocesstheinformationinspecicformelements154

PAGE 170

AppendixEAnnotatedItemsfromallAchievementMeasuresKeytoMarginalNotes ObjectivecoursewareobjectivetestedbyitemEx1alternativeobjectiveidentiedbyrstexpertEx2alternativeobjectiveidentiedbysecondexpertPilotincludedonpilotposttestPre/RetincludedonpretestandretentiontestPostincludedonposttest response correctanswer 1.YoumayuseaWebbrowsertoviewwhichofthefollowing?(a) allofthefollowing (b)onlinedocumentssavedonaWebserver(c)ofinedocumentssavedonthelocalmachine(d)thesourceofanHTMLdocumentObjective1aPost2.Ingeneral,whatwillabrowserdowithatagitdoesnotrecognize?(a)reportanerror(b) ignoreit (c)replaceitwithaclosematch(d)xitObjective1bEx2:4aPre/RetPilot155

PAGE 171

AppendixE(Continued)3.Whilenottechnicallycorrect(accordingtotheHTML4.0specication),browserswillgenerallyallowyoutodowhichofthefollowingandstillrenderyourpageasre-quested?(a) morethanoneofthefollowing (b)B4Itext/B4/I(shouldbe:B5Itext/I4/B)(c)BODYTEXT=“gray”text/BODYTEXT=“gray”(shouldbe:BODYTEXT=“gray”text/BODY)(d)HRtext/HR(shouldbe:HRtext)Objective1bPost4.FuturechangestoyourHTMLdocumentwillbefacilitatedbydoingwhichofthefollowing?(a)usingappropriatecolorsforthetextandlinks(b)makingthesizeofimagessmall,sotheydownloadfaster(c) usingamplewhitespaceandindenting (d)usingtablestolayoutelementsObjective2aPre/Ret5.WhenwritingHTMLcode,usingamplewhitespace(spaces,tabs,blanklines)andliningupendtagsunderstarttagsallows .(a)morethanoneofthefollowing(b)abrowsertoprocessthecodemorequickly(c)abrowsertodetermineifanytagsaremissingmoreeasily(d) ahumantomakefuturechangestothecodemoreeasily Objective2aPost156

PAGE 172

AppendixE(Continued)6.Itisadvisabletouseatemplateleforthefollowingrea-son(s).(a)allofthefollowing(b)User'swillnotbeabletoviewyourindexpagewith-outone.(c)ItwillreducetheamountoftimetheusermustwaittoviewHTMLdocumentsatyourWebsite.(d) ItwillspeedproductionoffutureHTMLdocumentsforyourWebsite. Objective2bPost7.InordertoviewchangestoanHTMLdocumentthatiscurrentlydisplayedbythebrowser,whatmustyoudo?Notethatthechangesweremadeafterthedocumentwasopenedinthebrowser.(a)Youneeddonothing,sincethebrowserdisplaywillautomaticallybeupdated.(b)YouneedtoclicktheBackbuttononthebrowser.(c) YouneedtoclicktheReloadorRefreshbuttononthebrowser. (d)YouneedtoclicktheForwardbuttononthebrowser.Objective2cPost8.GiventhetagspecicationTD[/TD],wherethebracketsindicatethatthispartisoptional,whichofthefol-lowingwouldbevalidwaystousethistag?(a) morethanoneofthefollowing (b)TDtext(c)TDtext/TD(d)/TDtextTDObjective4aEx2:3c,4aPre/RetPilot157

PAGE 173

AppendixE(Continued)9.GiventhetagspecicationI5/I,whichofthefollowingwouldbevalidwaystousethistag?(a)morethanoneofthefollowing(b)Itext(c) Itext/I (d)/ItextIObjective4aEx1:4a,5aEx2:3c,4aPostPilot10.GiventhestarttagFONTSIZE=“+1”,whatshouldtheendtaglooklike?(a)morethanoneofthefollowing(b)/FONTSIZE=“+1”(c)/FONTSIZE=“-1”(d) /FONT Objective4aEx1:5a,4aEx2:5a,3cPostPilot11.Whichtagisusedtomarktextasbold?(a) B (b)D(c)BOLD(d)DARKObjective5aPostPilot158

PAGE 174

AppendixE(Continued)12.Whichtagmustallbrowsersrenderthesame?(a)STRONG(b)EM(c) I (d)KBDObjective5aPostPilot13.Whichtagallowsyoutospecifyeitheranexactorarelativesizefortext?(a)SMALL(b) FONT (c)BIG(d)RELObjective5aPre/Ret14.Differentbrowsersmayrenderwhichofthefollowingtagsastheyseet?(a)I(b)TT(c) EM (d)UObjective5bEx1:5aEx2:5aPost159

PAGE 175

AppendixE(Continued)15.Whichheadingleveltagwillbedisplayedmostpromi-nently?(a)HR(b)H0(c) H2 (d)H6Objective5bPost16.Whichtagisusedtocreateabulletedlist?(a) UL (b)OL(c)LI(d)BIObjective6aPre/Ret17.Whichtagisusedtocreateanumberedlist?(a)LI(b)LN(c)NL(d) OL Objective6aPost18.WhatistheminimumnumberofopeningULtagsrequiredforalistwith3bullets?(a) 1 (b)2(c)3(d)4Objective6aEx2:6bPost160

PAGE 176

AppendixE(Continued)19.WhatistheminimumnumberofopeningLItagsrequiredforalistwith3bullets?(a)1(b)2(c) 3 (d)4Objective6bPost20.Whichtagcausesthebrowsertodisplayabulletornumber(dependingonthekindoflistinwhichitisused)?(a)OL(b)UL(c) LI (d)TYPEObjective6bPostPilot21.Whichattributeisusedtochangethelookofabullet?(a)VALUE(b) TYPE (c)LOOK(d)NAMEObjective6cPostPilot161

PAGE 177

AppendixE(Continued)22.Whichattributeoftheimagetagisusedtospecifywhatnongraphicalbrowserswillseeandwhatgraphicalbrowsersseewhilewaitingfortheimagetodownload?(a)BORDER(b)BOX(c)SUBJECT(d) ALT Objective7aPost23.Onapagethatincludesanimagewithtextfollowingit,thetextthatfollowsmayormaynotappeartodownloadatadifferentrateofspeedwhenthewidthandheightoftheimagearespecied.Willthatratebefaster,slower,thesame,ordependonthesizeoftheimage?(a) faster (b)slower(c)same(d)dependsonimagesizeObjective7aPost24.IfyouwanttohaveanimageonyourWebpagewithanactualwidthof28pixelsdownloadanddisplayfasterfortheuser,whatshouldyoudo?(a)morethanoneofthefollowing(b)specifyawidthof14pixelswhenincludingitonyourpage(c)donotspecifyawidthorheightfortheimage(d) opentheimageinanimageeditorandmakeitsmaller Objective7aPre/Ret162

PAGE 178

AppendixE(Continued)25.Intheimagecoordinatesystem,whereistheorigin(0,0)fortheimage?(a)center(b) top,left (c)top,right(d)bottom,leftObjective7bPostPilot26.Whichattributeoftheimagetagmustbesetto0todisabletheboxthatappearsaroundaclickableimage?(a) BORDER (b)BOX(c)SUBJECT(d)ALTObjective7cPost27.Whichimageattributedoyouchangetobringinadifferentimage?(a)IMG(b)ALT(c)NEW(d) SRC Objective7dPostPilot163

PAGE 179

AppendixE(Continued)28.WhichofthefollowingattributesisNOTdenedforatablecell?(a)NOWRAP(b)COLSPAN(c) WRAPSPAN (d)ROWSPANObjective8aEx1:8a,8bEx2:8dPostPilot29.Whichcellalignmentsarepossibleinatable?(a) morethanoneofthefollowing (b)vertical(c)horizontal(d)angledObjective8aEx1:8bEx2:8b,8aPostPilot30.WhichtableattributeincreasesthedistanceBETWEENcells?(a)WIDTH(b)BORDER(c)CELLPADDING(d) CELLSPACING Objective8bPostPilot164

PAGE 180

AppendixE(Continued)31.Whichtabletagdesignatesarow?(a)morethanoneofthefollowing(b) TR (c)TD(d)THObjective8cPostPilot32.Whichtabletagdesignatesacellinarow?(a) morethanoneofthefollowing (b)TR(c)TD(d)THObjective8dPostPilot33.Whichtabletagmakestextappearstronglyemphasized(boldinsomebrowsers)?(a)morethanoneofthefollowing(b)TR(c)TD(d) TH Objective8dPre/RetPilot34.HowwouldyouspecifyinaFRAMESETtagthatyouwouldliketworowswiththetoprowcontainingaframeofheight50pixels?(a)ROWS=“*,50%”(b)ROWS=“50%,*”(c) ROWS=“50,*” (d)ROWS=“*,50”Objective9aPost165

PAGE 181

AppendixE(Continued)35.FRAMESETtagscanbe .(a)overlayed(b) nested (c)shifted(d)splitObjective9aPost36.WhatshouldyousettheTARGETattributeoftheAtagto,ifyouwantanewbrowserwindowtobeopenedwhenthecorrespondinglinkisclicked?(a) blank (b) window(c) top(d) newObjective9bEx2:4aPostPilot37.WhatattributeoftheFRAMEtagmustbesetinordertouseitasatargetforAtags?(a)TARGET(b) NAME (c)SRC(d)HREFObjective9bPost166

PAGE 182

AppendixE(Continued)38.WhatshouldyousettheTARGETattributeoftheAtagto,ifyouwantthenewpagedisplayedafterclickingalinkinaframetowipeoutallframesandbedisplayedintheentireoriginalbrowserwindow?(a) blank(b) window(c) top (d) newObjective9bPre/Ret39.Itispossibletosupplyalternativecontentinyourframesetforbrowsersthatareunableto .(a)viewimages(b) loadframes (c)frameimages(d)viewtargetsObjective9cPost40.WhichattributeisusedwithformINPUToftypetexttolimittheamountofdatathattheuserisallowedtotypeintothetexteld?(a)LIMIT(b)TYPE(c)SIZE(d) MAXLENGTH Objective10aPost167

PAGE 183

AppendixE(Continued)41.Whichformelement(s)canbeusedtoallowonlyoneiteminagrouptobechecked?(a) radiobuttons (b)groupbuttons(c)checkboxes(d)groupboxesObjective10bPre/Ret42.Inorderforformelementsoftyperadiotoworktogether,theymustallhavethesamevalueforwhichattribute?(a)VALUE(b) NAME (c)SYNC(d)CHECKEDObjective10bPostPilot43.Whatdoestheresetformelementdo?(a)reloadsthepage(b)setsallformelementstoblankstates(c)reloadstheframecurrentlyselected(d) setsallformelementstooriginalstates Objective10cPost44.Itispossibletochangethe asubmitbutton.(a)rangeof(b) messagedisplayedon (c)sourcereferencedon(d)typeofObjective10dPost168

PAGE 184

AppendixE(Continued)45.WheninformationsuppliedbytheuserviaaformelementinanHTMLdocumentistobesenttoandprocessedbyacgi-script,itisnecessarytosetwhichattributeoftheelementappropriately,sothatthescriptcanaccesstheinformation?(a)SIZE(b) NAME (c)SRC(d)ACCEPTObjective10ePost169

PAGE 185

AppendixE(Continued)46.WriteacompleteHTMLdocumentthatincludesthefollow-ingelements.SeeFigure50forascreencaptureofhowyourcodeshouldberenderedbyaparticularbrowser.(12points)“MyPage”appearsinthetitlebar.Thebackgroundcoloriswhite,thetextcolorisblack,andthevisitedandunvisitedlinkcolorsareblue.Twocenteredhorizontalrulesspan50%ofthescreen.(Ifnecessary,refertothereferencematerialonBODYandHRprovidedonpages171and172.)Thepagecontainsthesentence“CheckoutmyWebsite!”,wherethephrase“myWebsite”isalinkthat,whenclicked,opensthele“index.htm”.Post Figure50.HTMLcoderenderedbyInternetExplorer(posttest).170

PAGE 186

AppendixE(Continued) 171

PAGE 187

AppendixE(Continued) 172

PAGE 188

AppendixE(Continued) HTMLHEADTITLEMyPage/TITLE/HEADBODYBGCOLOR=”white”TEXT=”black”LINK=”blue”VLINK=”blue”CENTERHRWIDTH=”50%”PCheckoutAHREF=”index.htm”myWebsite/A!/PHRWIDTH=”50%”/CENTER/BODY/HTML Figure51.Solutiontoessayquestion46.173

PAGE 189

AppendixE(Continued)Table18.GradingRubricforEssayQuestion46 Objective Item(1pointeach)TaEx1bEx2c HTMLtagsaroundall3cHEADsectionrst3bTITLEtagsaroundMyPage4cBODYsectionnext3bBGCOLORandTEXTattributesinopeningBODYtag4a4d,4aBGCOLOR=“white”TEXT=“black”4d4d,4aHRsandCheckoutmyWebsite!centered5cWIDTHattributeinHR4bWIDTH=“50%”4bAtagsaroundmyWebsite3a3a,4aHREF=“index.htm”3aCheckoutand!bothoutsideofAtags3a4a3a,4a Note.Blankentriesindicatethoseitemsforwhichanexpert'sjudgmentwithrespecttotheobjectivemeasuredagreedwiththetargetobjective.aTargetobjective.bObjectiveselectedbyrstexpert.cObjectiveselectedbysecondexpert.174

PAGE 190

AppendixE(Continued)47.WriteacompleteHTMLdocumentthatincludesthefollow-ingelements.SeeFigure52forascreencaptureofhowyourcodeshouldberenderedbyaparticularbrowser.(3points)“HomeoftheUSFBulls”appearsinthetitlebar.Thebackgroundcolorisgreen,thetextcolorisgold,andthevisitedandunvisitedlinkcolorsarewhite.(Ifnecessary,refertothereferencematerialonBODYprovidedonpage171.)Thephrase“WelcometotheUSFBulls'WebPage”isalevel1headingandisushagainsttherightsideofthebrowserwindow.(Ifnecessary,refertothereferencematerialonH1providedonpage176.)Thepagecontainsthesentence“Checkoutthehigh-lightsfromthelatestgame.”,wheretheword“high-lights”isalinkthat,whenclicked,opensthele“hLights.htm”.Ret Figure52.HTMLcoderenderedbyInternetExplorer(retentiontest).175

PAGE 191

AppendixE(Continued) 176

PAGE 192

AppendixE(Continued) HTMLHEADTITLEHomeoftheUSFBulls/TITLE/HEADBODYBGCOLOR=”green”TEXT=”gold”LINK=”white”VLINK=”white”H1ALIGN=”right”WelcometotheUSFBulls'WebPage/H1CENTERPCheckouttheAHREF=”hLights.htm”highlights/Afromthelatestgame./P/CENTER/BODY/HTML Figure53.Solutiontoessayquestion47.177

PAGE 193

AppendixE(Continued)Table19.GradingRubricforEssayQuestion47 Objective Item(1 12pointeach)TaEx1bEx2c HTMLtagsaroundall3cHEADsectionrst3bTITLEtagsaroundHomeoftheUSFBulls4cBODYsectionnext3bBGCOLORandTEXTattributesinopeningBODYtag4a4d,4aBGCOLOR=“green”TEXT=“gold”4d4d,4aCheckoutthehighlightsfromthelatestgame.centered5cALIGNattributeinopeningH1tag4bALIGN=“right”4bAtagsaroundhighlights3a3a,4aHREF=“hLights.htm”3aCheckouttheandfromthelatestgame.bothoutsideofAtags3a4a3a,4a Note.Blankentriesindicatethoseitemsforwhichanexpert'sjudgmentwithrespecttotheobjectivemeasuredagreedwiththetargetobjective.aTargetobjective.bObjectiveselectedbyrstexpert.cObjectiveselectedbysecondexpert.178

PAGE 194

AppendixFHTMLPretestSocialSecurityNumber: YouwillreceiveextracreditpointstowardEdTechfortakingthispretest.YouranswerswillprovidevaluabledatafordeterminingthestrengthoftheresultsofthestudyassociatedwiththeHTMLportionoftheclass.Yourresponseswillnotaffectyourgradeinanyway.Theywillbeusedonlyforpurposesofthestudy,andwillbeviewedonlybytheoutsideresearchercollectingthedata.Yourindividualscorewillnotbereportedtoanyofcialsassociatedwiththiscourse.Pleasedoyourbesttoanswerthequestions,butdonotbeconcernedifyoudonotknowanyoftheanswers.YouwilllearntheanswersasyoucompletetheHTMLassignments.Thankyouforyourthoughtfulresponsestothefollowingquestions.Pleaserecordyoursocialsecuritynumberandyouranswersdirectlyonthispretestandonthescantronprovided.1.FuturechangestoyourHTMLdocumentwillbefacilitatedbydoingwhichofthefollowing?(a)usingappropriatecolorsforthetextandlinks(b)makingthesizeofimagessmall,sotheydownloadfaster(c)usingamplewhitespaceandindenting(d)usingtablestolayoutelements2.GiventhetagspecicationTD[/TD],wherethebracketsindicatethatthispartisoptional,whichofthefollowingwouldbevalidwaystousethistag?(a)morethanoneofthefollowing(b)TDtext(c)TDtext/TD(d)/TDtextTD179

PAGE 195

AppendixF(Continued)3.Whichformelement(s)canbeusedtoallowonlyoneiteminagrouptobechecked?(a)radiobuttons(b)groupbuttons(c)checkboxes(d)groupboxes4.Whichtagallowsyoutospecifyeitheranexactorarelativesizefortext?(a)SMALL(b)FONT(c)BIG(d)REL5.Ingeneral,whatwillabrowserdowithatagitdoesnotrecognize?(a)reportanerror(b)ignoreit(c)replaceitwithaclosematch(d)xit6.Whichtagisusedtocreateabulletedlist?(a)UL(b)OL(c)LI(d)BI180

PAGE 196

AppendixF(Continued)7.IfyouwanttohaveanimageonyourWebpagewithanactualwidthof28pixelsdownloadanddisplayfasterfortheuser,whatshouldyoudo?(a)morethanoneofthefollowing(b)specifyawidthof14pixelswhenincludingitonyourpage(c)donotspecifyawidthorheightfortheimage(d)opentheimageinanimageeditorandmakeitsmaller8.Whichtabletagmakestextappearstronglyemphasized(boldinsomebrowsers)?(a)morethanoneofthefollowing(b)TR(c)TD(d)TH9.WhatshouldyousettheTARGETattributeoftheAtagto,ifyouwantthenewpagedisplayedafterclickingalinkinaframetowipeoutallframesandbedisplayedintheentireoriginalbrowserwindow?(a) blank(b) window(c) top(d) new10.Pleaseindicatethenumberofyearsofexperienceyouhavewithtypesettingand/ordocumentformattinglanguagessuchasHTMLandLATEX.(a)0(b)lessthan1(c)1-2(d)2-5(e)over5181

PAGE 197

AppendixF(Continued)11.PleaseindicatethenumberofyearsofexperienceyouhavewithauthoringenvironmentssuchasAuthorware,IconAuthor,andQuestand/orprogram-minglanguagessuchasAda,BASIC,C,Cobol,Fortran,Java,JavaScript,LISP,Pascal,VisualBasic,VisualC,etc.(a)0(b)lessthan1(c)1-2(d)2-5(e)over5182

PAGE 198

AppendixGHTMLPosttestName: SS#: Pleasecircleyouranswersontheactualexamandmarkthemonthescant-ronprovided.Also,pleasellinyournameandsocialsecuritynumberonboththeexamandthescantron.1.Whilenottechnicallycorrect(accordingtotheHTML4.0specication),browserswillgenerallyallowyoutodowhichofthefollowingandstillrenderyourpageasrequested?(a)morethanoneofthefollowing(b)B4Itext/B4/I(shouldbe:B5Itext/I4/B)(c)BODYTEXT=“gray”text/BODYTEXT=“gray”(shouldbe:BODYTEXT=“gray”text/BODY)(d)HRtext/HR(shouldbe:HRtext)2.WhatshouldyousettheTARGETattributeoftheAtagto,ifyouwantanewbrowserwindowtobeopenedwhenthecorrespondinglinkisclicked?(a) blank(b) window(c) top(d) new3.Whichtagcausesthebrowsertodisplayabulletornumber(dependingonthekindoflistinwhichitisused)?(a)OL(b)UL(c)LI(d)TYPE183

PAGE 199

AppendixG(Continued)4.WhenwritingHTMLcode,usingamplewhitespace(spaces,tabs,blanklines)andliningupendtagsunderstarttagsallows .(a)morethanoneofthefollowing(b)abrowsertoprocessthecodemorequickly(c)abrowsertodetermineifanytagsaremissingmoreeasily(d)ahumantomakefuturechangestothecodemoreeasily5.WhatistheminimumnumberofopeningLItagsrequiredforalistwith3bullets?(a)1(b)2(c)3(d)46.InordertoviewchangestoanHTMLdocumentthatiscurrentlydisplayedbythebrowser,whatmustyoudo?Notethatthechangesweremadeafterthedocumentwasopenedinthebrowser.(a)Youneeddonothing,sincethebrowserdisplaywillautomaticallybeupdated.(b)YouneedtoclicktheBackbuttononthebrowser.(c)YouneedtoclicktheReloadorRefreshbuttononthebrowser.(d)YouneedtoclicktheForwardbuttononthebrowser.7.Whichtabletagdesignatesacellinarow?(a)morethanoneofthefollowing(b)TR(c)TD(d)TH184

PAGE 200

AppendixG(Continued)8.Whichtagisusedtomarktextasbold?(a)B(b)D(c)BOLD(d)DARK9.Whichtagmustallbrowsersrenderthesame?(a)STRONG(b)EM(c)I(d)KBD10.WhichtableattributeincreasesthedistanceBETWEENcells?(a)WIDTH(b)BORDER(c)CELLPADDING(d)CELLSPACING11.YoumayuseaWebbrowsertoviewwhichofthefollowing?(a)allofthefollowing(b)onlinedocumentssavedonaWebserver(c)ofinedocumentssavedonthelocalmachine(d)thesourceofanHTMLdocument185

PAGE 201

AppendixG(Continued)12.Differentbrowsersmayrenderwhichofthefollowingtagsastheyseet?(a)I(b)TT(c)EM(d)U13.Whichcellalignmentsarepossibleinatable?(a)morethanoneofthefollowing(b)vertical(c)horizontal(d)angled14.Whichattributeoftheimagetagmustbesetto0todisabletheboxthatappearsaroundaclickableimage?(a)BORDER(b)BOX(c)SUBJECT(d)ALT15.Itispossibletosupplyalternativecontentinyourframesetforbrowsersthatareunableto .(a)viewimages(b)loadframes(c)frameimages(d)viewtargets186

PAGE 202

AppendixG(Continued)16.GiventhetagspecicationI4/I,whichofthefollowingwouldbevalidwaystousethistag?(a)morethanoneofthefollowing(b)Itext(c)Itext/I(d)/ItextI17.Whichheadingleveltagwillbedisplayedmostprominently?(a)HR(b)H0(c)H2(d)H618.Whichattributeisusedtochangethelookofabullet?(a)VALUE(b)TYPE(c)LOOK(d)NAME19.Whichattributeoftheimagetagisusedtospecifywhatnongraphicalbrowserswillseeandwhatgraphicalbrowsersseewhilewaitingfortheim-agetodownload?(a)BORDER(b)BOX(c)SUBJECT(d)ALT187

PAGE 203

AppendixG(Continued)20.GiventhestarttagFONTSIZE=“+1”,whatshouldtheendtaglooklike?(a)morethanoneofthefollowing(b)/FONTSIZE=“+1”(c)/FONTSIZE=“-1”(d)/FONT21.Inorderforformelementsoftyperadiotoworktogether,theymustallhavethesamevalueforwhichattribute?(a)VALUE(b)NAME(c)SYNC(d)CHECKED22.Intheimagecoordinatesystem,whereistheorigin(0,0)fortheimage?(a)center(b)top,left(c)top,right(d)bottom,left23.Itispossibletochangethe asubmitbutton.(a)rangeof(b)messagedisplayedon(c)sourcereferencedon(d)typeof188

PAGE 204

AppendixG(Continued)24.WhatattributeoftheFRAMEtagmustbesetinordertouseitasatargetforAtags?(a)TARGET(b)NAME(c)SRC(d)HREF25.Whichimageattributedoyouchangetobringinadifferentimage?(a)IMG(b)ALT(c)NEW(d)SRC26.WhatistheminimumnumberofopeningULtagsrequiredforalistwith3bullets?(a)1(b)2(c)3(d)427.Whatdoestheresetformelementdo?(a)reloadsthepage(b)setsallformelementstoblankstates(c)reloadstheframecurrentlyselected(d)setsallformelementstooriginalstates189

PAGE 205

AppendixG(Continued)28.WhichofthefollowingattributesisNOTdenedforatablecell?(a)NOWRAP(b)COLSPAN(c)WRAPSPAN(d)ROWSPAN29.Whichtabletagdesignatesarow?(a)morethanoneofthefollowing(b)TR(c)TD(d)TH30.Onapagethatincludesanimagewithtextfollowingit,thetextthatfollowsmayormaynotappeartodownloadatadifferentrateofspeedwhenthewidthandheightoftheimagearespecied.Willthatratebefaster,slower,thesame,ordependonthesizeoftheimage?(a)faster(b)slower(c)same(d)dependsonimagesize31.WhichattributeisusedwithformINPUToftypetexttolimittheamountofdatathattheuserisallowedtotypeintothetexteld?(a)LIMIT(b)TYPE(c)SIZE(d)MAXLENGTH190

PAGE 206

AppendixG(Continued)32.HowwouldyouspecifyinaFRAMESETtagthatyouwouldliketworowswiththetoprowcontainingaframeofheight50pixels?(a)ROWS=“*,50%”(b)ROWS=“50%,*”(c)ROWS=“50,*”(d)ROWS=“*,50”33.Itisadvisabletouseatemplateleforthefollowingreason(s).(a)allofthefollowing(b)User'swillnotbeabletoviewyourindexpagewithoutone.(c)ItwillreducetheamountoftimetheusermustwaittoviewHTMLdocumentsatyourWebsite.(d)ItwillspeedtheproductionoffutureHTMLdocumentsforyourWebsite.34.Whichtagisusedtocreateanumberedlist?(a)LI(b)LN(c)NL(d)OL35.WheninformationsuppliedbytheuserviaaformelementinanHTMLdoc-umentistobesenttoandprocessedbyacgi-script,itisnecessarytosetwhichattributeoftheelementappropriately,sothatthescriptcanaccesstheinformation?(a)SIZE(b)NAME(c)SRC(d)ACCEPT191

PAGE 207

AppendixG(Continued)36.FRAMESETtagscanbe .(a)overlayed(b)nested(c)shifted(d)split192

PAGE 208

AppendixG(Continued)37.WriteacompleteHTMLdocumentthatincludesthefollowingelements.Seetheimagebelowforascreencaptureofhowyourcodeshouldberenderedbyaparticularbrowser.Pleasewriteyouransweronthelastpageoftheexam.(12points)“MyPage”appearsinthetitlebar.Thebackgroundcoloriswhite,thetextcolorisblack,andthevisitedandunvisitedlinkcolorsareblue.Twocenteredhorizontalrulesspan50%ofthescreen.(Ifnecessary,refertothereferencematerialonBODYandHRprovidedonpages194and195.)Thepagecontainsthesentence“CheckoutmyWebsite!”,wherethephrase“myWebsite”isalinkthat,whenclicked,opensthele“in-dex.htm”. HTMLcoderenderedbyInternetExplorer.Pleasewriteyouransweronthelastpageoftheexam(page196).193

PAGE 209

AppendixG(Continued) 194

PAGE 210

AppendixG(Continued) 195

PAGE 211

AppendixG(Continued)Name: SS#: Writeyouranswertoquestion37here.Also,pleasellinyournameandsocialsecurityonthissheetaswell. ofce use only H: H: T: B: IN: CC: C: IN: P: A: H: O: 196

PAGE 212

AppendixHHTMLRetentionTestName: SS#: Pleasecircleyouranswersontheactualexamandmarkthemonthescant-ronprovided.Also,pleasellinyournameandsocialsecuritynumberonboththeexamandthescantron.1.FuturechangestoyourHTMLdocumentwillbefacilitatedbydoingwhichofthefollowing?(a)usingappropriatecolorsforthetextandlinks(b)makingthesizeofimagessmall,sotheydownloadfaster(c)usingamplewhitespaceandindenting(d)usingtablestolayoutelements2.Whichformelement(s)canbeusedtoallowonlyoneiteminagrouptobechecked?(a)radiobuttons(b)groupbuttons(c)checkboxes(d)groupboxes3.WhatshouldyousettheTARGETattributeoftheAtagto,ifyouwantthenewpagedisplayedafterclickingalinkinaframetowipeoutallframesandbedisplayedintheentireoriginalbrowserwindow?(a) blank(b) window(c) top(d) new197

PAGE 213

AppendixH(Continued)4.IfyouwanttohaveanimageonyourWebpagewithanactualwidthof28pixelsdownloadanddisplayfasterfortheuser,whatshouldyoudo?(a)morethanoneofthefollowing(b)specifyawidthof14pixelswhenincludingitonyourpage(c)donotspecifyawidthorheightfortheimage(d)opentheimageinanimageeditorandmakeitsmaller5.Whichtagisusedtocreateabulletedlist?(a)UL(b)OL(c)LI(d)BI6.Ingeneral,whatwillabrowserdowithatagitdoesnotrecognize?(a)reportanerror(b)ignoreit(c)replaceitwithaclosematch(d)xit7.Whichtabletagmakestextappearstronglyemphasized(boldinsomebrowsers)?(a)morethanoneofthefollowing(b)TR(c)TD(d)TH198

PAGE 214

AppendixH(Continued)8.GiventhetagspecicationTD[/TD],wherethebracketsindicatethatthispartisoptional,whichofthefollowingwouldbevalidwaystousethistag?(a)morethanoneofthefollowing(b)TDtext(c)TDtext/TD(d)/TDtextTD9.Whichtagallowsyoutospecifyeitheranexactorarelativesizefortext?(a)SMALL(b)FONT(c)BIG(d)REL199

PAGE 215

AppendixH(Continued)10.WriteacompleteHTMLdocumentthatincludesthefollowingelements.Seetheimagebelowforascreencaptureofhowyourcodeshouldberenderedbyaparticularbrowser.“HomeoftheUSFBulls”appearsinthetitlebar.Thebackgroundcolorisgreen,thetextcolorisgold,andthevisitedandunvisitedlinkcolorsarewhite.(Ifnecessary,refertothereferencematerialonBODYprovidedonpage201.)Thephrase“WelcometotheUSFBulls'WebPage”isalevel1headingandisushagainsttherightsideofthebrowserwindow.(Ifnecessary,refertothereferencematerialonH1providedonpage202.)Thepagecontainsthesentence“Checkoutthehighlightsfromthelatestgame.”,wheretheword“highlights”isalinkthat,whenclicked,opensthele“hLights.htm”. HTMLcoderenderedbyInternetExplorer.Pleasewriteyouransweronthelastpageoftheexam(page203).200

PAGE 216

AppendixH(Continued) 201

PAGE 217

AppendixH(Continued) 202

PAGE 218

AppendixH(Continued)Name: SS#: Writeyouranswertoquestion10here.Also,pleasellinyournameandsocialsecuritynumberonthissheetaswell. ofce use only H: H: T: B: IN: CC: C: IN: P: A: H: O: 203

PAGE 219

AppendixILettertoHTMLExpertDearHTMLExpert,Thankyoufortakingthetimetoevaluatethecontentvalidityofthemeasuresforthisstudy.Yourtimeandinsightaregreatlyappreciated.PriorExperienceThestudentswillbeaskedtoanswerthequestioninFigure54beforeanytreatmentsareadministered.Pleaserespondtothequestionsthatfollow. Doyouhavecomputerprogrammingortypesettingexperiencewithanyofthelanguageslistedbeloworanylanguagesnotlisted(pleaselist)?Ifyes,pleaseindicatetheapproximatenumberofyearsexperienceyouhavewitheachone. yrsofAda yrsofHTML yrsofPascal yrsofBASIC yrsofJava yrsofVisualBasic yrsofC yrsofJavaScript yrsofVisualC yrsofCobol yrsofLATEX yrsof yrsofFortran yrsofLISP yrsof Figure54.Self-reportexperienceitemonpretest.Doyouagreethatexperiencewithanyofthelanguageslistedwouldlikelygiveastudentenoughofanadvantagetomeritconcern?(Pleasecrossoutanyyouthinkarenotpertinent.)Arethereanyotherpriorexperiencesyoufeelwouldgiveastudentenoughofanadvantagetomeritconcernthatshouldbeascertainedbeforetreatment?204

PAGE 220

AppendixI(Continued)ObjectivesYouwillndtheimplicitobjectivesofthecoursewarestatedexplicitlyintherstattachment.Approximaterelativeweightingsappearnexttoeachandreectthepercentageofquestionsonthepretest,posttest,andretentionteststhatarein-tendedtomeasureeachitem.BasedonyourknowledgeofwhatisimportantwhencreatingHTMLdocumentsusingaplaintexteditor(asopposedtoaWYSIWYGeditor),pleaserespondtothefollowingquestions.Doyoufeelthatthemostimportantobjectiveshavebeenincluded?Pleaselistanyimportantobjectivesyoufeelhavebeenleftout.Pleaselistanyincludedobjectivesyoufeelreallyarenotimportant.Doyouagreewiththerelativeweightinggivenfortheobjectives?205

PAGE 221

AppendixI(Continued)Ifyoudonotagreewiththerelativeweightingoftheobjectives,pleaseindi-cateanalternativeweighting,usingpercentages.Pleasecrossoutanyob-jectivesyoufeelarenotimportantandaddinanyyoufeelshouldbepresent,butarenot. Objective Current Suggested BrowserBasics 4% Development,Design,andStyle 6% DocumentStructure 13% TagsandAttributes 15% TextStyle 10% Lists 10% Images 10% Tables 10% Frames 10% Forms 10% TestItemsYouwillndtheactualtestitemsinthesecondattachment.Thesearealloftheitemsincludedonalloftheachievementmeasures.Nexttoeachmultiplechoiceitemaboxisprovidedforyourconvenienceinratingtheclarityanddifcultyoftheitemaswellasindicatingwhichobjectiveyoufeelistestedbytheitem.Pleaseindicatewhetherornotyoufeeltheitemshouldbeomittedorreworded.Youmaymakechangesdirectlytotheitemand/orlistanyothercommentsyoumayhavetotherightoftheitem.Pleaseevaluatethetwoessayquestionsinlikemanner,withoneexception.Thesolutioncodeandgradingrubricsforthequestionsareprovided.Thegradingrubricbreaksthesolutionintosubitems.Pleaseindicatetheobjectiveyoufeelismeasuredbyeachsubitemintheappropriatecolumnoftheprovidedtable.Whileeachitemisintendedtomeasureonlyoneobjective,youmayattimes,feelthatmorethanoneapplies.Inthatcase,pleaselistallthatseempertinent,206

PAGE 222

AppendixI(Continued)withthemostpertinentonelistedrst.Also,itmaybehelpfultoknow,especiallyindecidingifaquestionistoodifcult,thatthemeasureswillbeadministeredtocollegelevelfreshmanwhohopetobecometeachers,manyofwhomhavejustobtainedtheirrstcomputer.Irealizethatyourtimeisvaluableandappreciateyourevaluationofthesemea-sures.Thankyousomuchforyourfeedback.Itwillbeputtogooduse.Sincerely,TinaL.Majchrzak207

PAGE 223

AppendixJExpertEvaluationFormforAchievementMeasuresKeytoResponseBoxes ObjectiveFillintheobjectivemeasuredbytheitem(e.g.5b).Eachitemwaswrittentomeasureasingleobjective.However,ifmorethanoneobjectiveseemstoapplyforagivenitem,pleaselistallthatseemappropriate,withthemostappropriateonelistedrst.Determin-ingwhetherornot4bappliescanbedifcultwithoutextensiveknowledgeofthecourseware,sokeepinmindthatwheneveraquestionreferstoanattributeand/ortagnotformallydiscussedinthecourseware,thequestionincludesreferencematerialonthetagand/orattribute.ClearCircleYifthequestionisclearlywritten.Otherwise,circleN,andpleasesuggestalternativewording.TooDifcultCircleYifyoufeelthisitemmaybetoodifcult.Otherwise,circleN.Recallthat,forthemostpart,thestudentsarenovicecomputerusers.RemoveCircleYifyoufeelthisitemshouldnotbeused.Otherwise,circleN.RewordCircleYifyoufeelthisitemshouldbewrittendifferently,andpleasesuggestalternativewording.Otherwise,circleN.CommentsPleasemakeanynotesorfurthersuggestionsregardingtheitemhere. 208

PAGE 224

AppendixJ(Continued)1.Whilenottechnicallycorrect(accordingtotheHTML4.0specication),browserswillgenerallyal-lowyoutodowhichofthefollowingandstillrenderyourpageasrequested?(a) morethanoneofthefollowing (b)B4Itext/B4/I(shouldbe:B5Itext/I4/B)(c)BODYTEXT=“gray”text/BODYTEXT=“gray”(shouldbe:BODYTEXT=“gray”text/BODY)(d)HRtext/HR(shouldbe:HRtext) Objective Clear YN TooDifcult YN Remove YN Reword YN Comments:...29.Itispossibletochangethe asubmitbutton.(a)rangeof(b) messagedisplayedon (c)sourcereferencedon(d)typeof Objective Clear YN TooDifcult YN Remove YN Reword YN Comments:209

PAGE 225

AppendixJ(Continued)30.WhatattributeoftheFRAMEtagmustbesetinor-dertouseitasatargetforAtags?(a)TARGET(b) NAME (c)SRC(d)HREF Objective Clear YN TooDifcult YN Remove YN Reword YN Comments:...35.IfyouwanttohaveanimageonyourWebpagewithanactualwidthof28pixelsdownloadanddisplayfasterfortheuser,whatshouldyoudo?(a)morethanoneofthefollowing(b)specifyawidthof14pixelswhenincludingitonyourpage(c)donotspecifyawidthorheightfortheimage(d) opentheimageinanimageeditorandmakeitsmaller Objective Clear YN TooDifcult YN Remove YN Reword YN Comments:...210

PAGE 226

AppendixJ(Continued)43.WhatshouldyousettheTARGETattributeoftheAtagto,ifyouwantthenewpagedisplayedafterclickingalinkinaframetowipeoutallframesandbedisplayedintheentireoriginalbrowserwindow?(a) blank(b) window(c) top (d) new Objective Clear YN TooDifcult YN Remove YN Reword YN Comments:...211

PAGE 227

AppendixJ(Continued)46.WriteacompleteHTMLdocumentthatin-cludesthefollowingelements.Seetheim-agebelowforascreencaptureofhowyourcodeshouldberenderedbyaparticularbrowser.(12points)“MyPage”appearsinthetitlebar.Thebackgroundcoloriswhite,thetextcolorisblack,andthevisitedandun-visitedlinkcolorsareblue.Twocenteredhorizontalrulesspan50%ofthescreen.(Ifnecessary,refertothereferencematerialonHRpro-videdonpage213.)Thepagecontainsthesentence“CheckoutmyWebsite!”,wherethephrase“myWebsite”isalinkthat,whenclicked,opensthele“index.htm”. Objective llinboxesonpage214 Clear YN TooDifcult YN Remove YN Reword YN Comments: HTMLcoderenderedbyInternetExplorer.212

PAGE 228

AppendixJ(Continued) 213

PAGE 229

AppendixJ(Continued)Solutiontoessayquestion46 HTMLHEADTITLEMyPage/TITLE/HEADBODYBGCOLOR=”white”TEXT=”black”LINK=”blue”VLINK=”blue”CENTERHRWIDTH=”50%”PCheckoutAHREF=”index.htm”myWebsite/A!/PHRWIDTH=”50%”/CENTER/BODY/HTML Table20.FormUsedtoAnalyzeGradingRubricforEssayQuestion46 ItemObjective HTMLtagsaroundall HEADsectionrst TITLEtagsaroundMyPage BODYsectionnext BGCOLORandTEXTattributesinopeningBODYtag BGCOLOR=“white”TEXT=“black” HRsandCheckoutmyWebsite!centered WIDTHattributeinHR WIDTH=“50%” AtagsaroundmyWebsite HREF=“index.htm” Checkoutand!bothoutsideofAtags 214

PAGE 230

AppendixJ(Continued)47.WriteacompleteHTMLdocumentthatin-cludesthefollowingelements.Seetheim-agebelowforascreencaptureofhowyourcodeshouldberenderedbyaparticularbrowser.(3points)“HomeoftheUSFBulls”appearsinthetitlebar.Thebackgroundcolorisgreen,thetextcolorisgold,andthevisitedandunvis-itedlinkcolorsarewhite.Thephrase“WelcometotheUSFBulls'WebPage”isalevel1headingandisushagainsttherightsideofthebrowserwindow.(Ifnecessary,refertothereferencematerialonH1providedonpage216.)Thepagecontainsthesentence“Checkoutthehighlightsfromthelat-estgame.”,wheretheword“highlights”isalinkthat,whenclicked,opensthele“hLights.htm”. Objective llinboxesonpage217 Clear YN TooDifcult YN Remove YN Reword YN Comments: HTMLcoderenderedbyInternetExplorer.215

PAGE 231

AppendixJ(Continued) 216

PAGE 232

AppendixJ(Continued)SolutiontoEssayQuestion47 HTMLHEADTITLEHomeoftheUSFBulls/TITLE/HEADBODYBGCOLOR=”green”TEXT=”gold”LINK=”white”VLINK=”white”H1ALIGN=”right”WelcometotheUSFBulls'WebPage/H1CENTERPCheckouttheAHREF=”hLights.htm”highlights/Afromthelatestgame./P/CENTER/BODY/HTML 217

PAGE 233

AppendixJ(Continued)Table21.FormUsedtoAnalyzeGradingRubricforEssayQuestion47 ItemObjective HTMLtagsaroundall HEADsectionrst TITLEtagsaroundHomeoftheUSFBulls BODYsectionnext BGCOLORandTEXTattributesinopeningBODYtag BGCOLOR=“green”TEXT=“gold” Checkoutthehighlightsfromthelatestgame.centered ALIGNattributeinopeningH1tag ALIGN=“right” Atagsaroundhighlights HREF=“hLights.htm” Checkouttheandfromthelatestgame.bothoutsideofAtags 218

PAGE 234

AppendixKSelf-ReportMeasureofPacingPreferenceName: SS#: YouwillreceiveextracreditpointstowardEdTechfortakingthissurvey.TheinformationyouprovideisofparamountimportanceindeterminingthemeritsoftheinstructionalformatoftheHTMLportionofthiscourse.Itwillaidinmak-ingfutureimprovementstoitandwillshedlightontheinstructionalformatmostpreferredbystudentsingeneral.Yourresponseswillbeconsideredseparatelyfromyourcourseperformancebyanoutsideresearcher,whowillonlyknowyoubyanindependentnumberassignedtoyouforthestudy.Yournameandsocialsecuritynumberwillbeconvertedtothisstudynumber.Specifyingthisinformationhereisnecessarytomakesurethatresponsesareplacedintothecorrectgroupsfordataanalysis.Yournameandsocialsecuritynumberwillbeusedforthispurposeonly,andthenwillbedeletedfromyourresponses.Yourresponseswillbegroupedwiththoseofotherstudents.Reportsofthendingswillbeintermsofgroupsratherthanintermsofindividu-als.Yourindividualresponseswillnotbereportedtoanyofcialsassociatedwiththiscourseortoanyotherindividuals,soyoumayconsideryourresponsestobeanonymous. Thankyouforyourthoughtfulconsiderationofthefollowingquestions.Pleaserecordyourname,socialsecuritynumberandanswersdirectlyonthissurveyandonthescantronprovided.1.Givenachoiceinafutureclassbetweenlivelecturesandlecturesprere-cordedonCD-ROMorvideocassette,whichwouldyouprefer? ReallyPrefer SomewhatPrefer No SomewhatPrefer ReallyPrefer Live Live Preference Prerecorded Prerecorded 1 2 3 4 5 2.Givenachoiceinafutureclasstolearnineitheraverylargegroupwith100ormorestudents,alargegroupwith30-50students,asmall,self-madegroupwith2-5students,oralone,whichwouldyouprefer? Prefer100+ Prefer30-50 NoPreference Prefer2-5 PreferAlone 1 2 3 4 5 219

PAGE 235

AppendixK(Continued)3.Givenachoiceinafutureclasstolearnatexible,self-determinedtimesoratexternallyset,structuredtimeseachweek,whichwouldyouprefer? ReallyPrefer SomewhatPrefer No SomewhatPrefer ReallyPrefer Flexible Flexible Preference Structured Structured 1 2 3 4 5 4.GivenachoiceinafutureclasstohearallofthequestionsandanswersoffellowstudentsinpersonortohavedeferredaccesstoselectquestionsandanswersmaintainedinaFrequentlyAskedQuestions(FAQs)archive,whichwouldyouprefer? ReallyPrefer SomewhatPrefer No SomewhatPrefer ReallyPrefer All,Live All,Live Preference Select,Archive Select,Archive 1 2 3 4 5 5.Givenachoiceinafutureclasstobreakaonehourlectureintosix10-minutelecturesviewedatyourdiscretionortoviewtheentirelectureallatonce,whichwouldyouprefer? ReallyPrefer SomewhatPrefer No SomewhatPrefer ReallyPrefer Six10-minute Six10-minute Preference One1-hour One1-hour 1 2 3 4 5 6.Givenachoiceinafutureclasstocompleteassignmentsatapacesetbyyourselforatapacesetbytheteacher,whichwouldyouprefer? ReallyPrefer SomewhatPrefer No SomewhatPrefer ReallyPrefer SelfSet SelfSet Preference TeacherSet TeacherSet 1 2 3 4 5 7.Givenachoiceinafutureclasstousesuppliedgradingrubricstoevaluateyourownworkfrequentlyatsmall,intermediatestagesortohaveateacherevaluateyourworklessoftenattwoorthreemainjunctures,whichwouldyouprefer? ReallyPrefer SomewhatPrefer No SomewhatPrefer ReallyPrefer Self,Frequent Self,Frequent Preference Teacher,Sparse Teacher,Sparse 1 2 3 4 5 220

PAGE 236

AppendixK(Continued)8.Givenachoicebetweenteacher-pacedinstructionusingthelecturemethodandself-pacedinstructiondeliveredviaCD-ROM,whichinstructionalformatwouldyouprefer? ReallyPrefer SomewhatPrefer No SomewhatPrefer ReallyPrefer Teacher-paced Teacher-paced Preference Self-paced Self-paced 1 2 3 4 5 221

PAGE 237

AppendixLLettertoInstructionalParadigmsExpertDearExpertonTeacher-PacedandSelf-PacedInstruction,Thankyoufortakingthetimetoevaluatethecontentvalidityofthemeasuresforthisstudy.Yourtimeandinsightaregreatlyappreciated.Pleasenotethattheoverallpurposeofthemeasuresistodetermineifdiffer-encesexistbetweentheresponsesofstudentsinthreetreatmentgroups.Thetreatmentsarefocusedondeadlines.Allgroupshavethesamedeadlines.How-ever,foronegroup,thesearemerelyrecommended.Foranother,theyarecon-ditionalwithbonuspointsawardedforearlysubmissionsandpenaltypointsde-ductedforlatesubmissions.Forthethird,theyareabsolutewithnolatesubmis-sionsaccepted.Pleasekeepthiscontextinmindduringyourevaluation.AspectsofSelf-PacingThestudentswillbeaskedtorespondtoeightself-reportitemsafterthetreat-mentshavebeenadministered.Eachisbipolarinnature,requiringthemtoexpressapreferenceforeitherself-pacedinstructionorinstructor-pacedinstructionalongavepointscale.Thequestionscovercommonfeaturesofthelearningenviron-mentwhichdifferentiatethesetwoinstructionalapproaches.Theyarelistedbelowinnoparticularorder.Pleaselookthemoverandthenrespondtothequestionsthatfollow.LivelectureversusprerecordedlectureLearninginalargegroupversusasmallgroup(possiblyofsize1)Learningduringstructuredclasstimeeachweekversusexible,self-determinedtimesAccesstoallstudents'questionsandanswersinpersonversusonlineaccesstoselectquestionsandanswersOne1-hourlectureversussix10-minutelecturesTeachersetdeadlinesversusstudentsetdeadlines222

PAGE 238

AppendixL(Continued)1.Doyoufeelthatthemostimportantfeaturesthatdistinguishself-pacedin-structionfrominstructor-pacedinstructionhavebeenincluded?2.Pleaselistanyimportantfeaturesyoufeelhavebeenleftout.3.Pleaselistanyincludedfeaturesyoufeelreallyarenotimportant.Self-ReportItemsYouwillndtheactualself-reportitemsattached.Nexttoeachitemaboxisprovidedforyourconvenienceinratingtheclarityoftheitemaswellasindicatingwhichvalue(1or5)representsthehighestpreferenceforself-pacing.Pleaseindicatewhetherornotyoufeeltheitemshouldbeomittedorreworded.Youmaymakechangesdirectlytotheitemand/orlistanyothercommentsyoumayhavetotherightoftheitem.ProcrastinationLevelProcrastinationlevelwillbemeasuredintermsoftheitemslistedbelow.Itisanticipatedthatthedifferenttreatmentconditions,whichfocusondeadlines,mayyielddifferentvaluesfortheseitems.Pleaseconsiderthemandanswerthequestionsthatfollow.averagenumberofrequestsfordeadlineextensionsperstudentaveragenumberofdayslateonassignmentsperstudent223

PAGE 239

AppendixL(Continued)1.Inthecontextofrecommendedversusconditionalversusabsolutedeadlines,doyoufeelthatthemostimportantindicatorsofstudentprocrastinationlevelhavebeenincluded?2.Pleaselistanyimportantindicatorsyoufeelhavebeenleftout.3.Pleaselistanyincludedindicatorsyoufeelreallyarenotimportant.Irealizethatyourtimeisvaluableandappreciateyourevaluationofthesemea-sures.Thankyousomuchforyourfeedback.Itwillbeputtogooduse.Sincerely,TinaL.Majchrzak224

PAGE 240

AppendixMExpertEvaluationFormforPacingPreferenceMeasureKeytoResponseBoxes Self-PacedCircle1ifthelowendofthescaleimpliesapreferenceforself-pacing.Circle5ifthehighendofthescaleimpliesapreferenceforself-pacing.ClearCircleYifthequestionisclearlywritten.Otherwise,circleN,andpleasesuggestalternativewording.RemoveCircleYifyoufeelthisitemshouldnotbeused.Otherwise,circleN.RewordCircleYifyoufeelthisitemshouldbewrittendifferently,andpleasesuggestalternativewording.Otherwise,circleno.CommentsPleasemakeanynotesorfurthersuggestionsregardingtheitemhere. 1.GivenachoiceinafutureclassbetweenlecturesrecordedonCD-ROMandlivelectures,whichwouldyouprefer? PreferCD-ROMNoPreferencePreferLive 12345 Self-Paced15 ClearYN RemoveYN RewordYN Comments:2.Givenachoiceinafutureclassbetweenlecturesrecordedonvideocassetteandlivelectures,whichwouldyouprefer? PreferVideoCassetteNoPreferencePreferLive 12345 Self-Paced15 ClearYN RemoveYN RewordYN Comments:3.Givenachoiceinafutureclasstolearneitherinalarge,nondescriptgroupwith30-50studentsortolearninasmall,self-madegroupwith2-5,whichwouldyouprefer? PreferNondescript,30-50NoPreferencePreferSelf-made,2-5 12345 Self-Paced15 ClearYN RemoveYN RewordYN Comments:225

PAGE 241

AppendixM(Continued)4.Givenachoiceinafutureclasstolearneitherinalarge,nondescriptgroupwith30-50studentsortolearnalone,whichwouldyouprefer? PreferNondescript,30-50NoPreferencePreferAlone 12345 Self-Paced15 ClearYN RemoveYN RewordYN Comments:5.Givenachoiceinafutureclasstolearneitherinaverylarge,nondescriptgroupwith100ormorestudentsortolearninasmall,self-madegroupwith2-5,whichwouldyouprefer? PreferNondescript,100+NoPreferencePreferSelf-made,2-5 12345 Self-Paced15 ClearYN RemoveYN RewordYN Comments:6.Givenachoiceinafutureclasstolearneitherinaverylarge,nondescriptgroupwith100ormorestudentsortolearnalone,whichwouldyouprefer? PreferNondescript,100+NoPreferencePreferAlone 12345 Self-Paced15 ClearYN RemoveYN RewordYN Comments:7.Givenachoiceinafutureclasstolearnatexible,self-determinedtimesoratexternallyset,structuredtimeseachweek,whichwouldyouprefer? PreferFlexibleNoPreferencePreferStructured 12345 Self-Paced15 ClearYN RemoveYN RewordYN Comments:8.Givenachoiceinafutureclasstohearalloftheques-tionsandanswersoffellowstudentsinpersonortohaveaccesstoselectquestionsandanswersonlineinaFrequentlyAskedQuestions(FAQs)archive,whichwouldyouprefer? PreferAll,LiveNoPreferencePreferSelect,Archive 12345 Self-Paced15 ClearYN RemoveYN RewordYN Comments:9.Givenachoiceinafutureclasstobreakaonehourlectureintosix10-minutelecturesviewedatyourdis-cretionortoviewtheentirelectureallatonce,whichwouldyouprefer? PreferSix10-minuteNoPreferencePreferOne1-hour 12345 Self-Paced15 ClearYN RemoveYN RewordYN Comments:226

PAGE 242

AppendixM(Continued)10.Givenachoiceinafutureclasstocompleteassign-mentsatapacesetbyyourselforatapacesetbytheinstructor,whichwouldyouprefer? PreferSelfSetNoPreferencePreferTeacherSet 12345 Self-Paced15 ClearYN RemoveYN RewordYN Comments:11.Givenachoicebetweeninstructor-pacedinstructionusingthelecturemethodandself-pacedinstructionde-liveredviaCD-ROM,whichinstructionalformatwouldyouprefer? PreferInstructor-pacedNoPreferencePreferSelf-paced 12345 Self-Paced15 ClearYN RemoveYN RewordYN Comments:227

PAGE 243

AppendixNExpertEvaluationFormforAssignmentsDearHTMLExpert,Thankyoufortakingthetimetoevaluatethecontentvalidityoftheassignmentsforthisstudy.Yourtimeandinsightaregreatlyappreciated.ThestudentswillbeaskedtocreatetheWebsitedescribedinthecourse-warecompletingeightintermediateassignments.Pleaseseeprovidedcourse-warefordetails.Forthersttwoassignments,thestudentsareprovidedwiththesourceHTML,sothattheymayfocusonlearningthedevelopmentcycleofmakingchangesinatexteditorandviewingthembyrefreshingthebrowserwindow.Theyarealsoprovidedwithallnecessaryimages.Alistoftheobjectivescoveredbytheassignmentsappearsbelowfollowedbytherubricsthatwillbeusedtogradetheeightassignments.Toviewthecompleteformoftherubricsprovidedtothegraders,includinggraphicaldirections,pleaserefertotheprovidedcourseware.Foreachrubriclistedbelow,pleaseindicatewhichofthetenobjectivesyoufeelitmeasuresbyplacingavaluebetween1and10intheboxnexttoeachone.Ifyoudonotfeelitmeasuresanyofthem,putanNinthebox.Ifyoufeelitmeasuresmorethanoneorarenotsurewhichitmeasures,pleaselistallpossibilitieswiththemostappropriateonelistedrst.Finally,ifyoufeeltherubricsforagivenassignmentmeasureanyotherobjectivesimplicitlythatyouhavenotalreadylistedforanyoftheindividualrubrics,pleaselisttheseobjectivesintheboxattheendofthelistofrubricsfortheassignment.Irealizethatyourtimeisvaluableandappreciateyourevaluationofthesemea-sures.Thankyousomuchforyourfeedback.Itwillbeputtogooduse.Sincerely,TinaL.Majchrzak228

PAGE 244

AppendixN(Continued)Objectives1.BrowserBasicsViewapagecreatedandsavedonthelocalmachineRecognizetheforgivingnatureofHTMLinterpreters2.Development,Design,andStyleUnderstandtheimportanceofwritingreadableHTMLcodeUnderstandthemeritsofusingatemplateleBeawareoftheneedtoRefresh/Reloadadocumenttoseechanges3.DocumentStructureIncludealinktoanotherpageDemonstrateknowledgethatHTMLdocumentsarecomprisedoftwomainsections,theHEADandtheBODYDemonstrateknowledgethatHTMLdocumentsaredesignatedwithopeningandclosingHTMLtagsthatsurroundthecontent4.TagsandAttributesUsetagsandattributescorrectlyDemonstrateanunderstandingofhowtagsandattributesworkbybe-ingabletolookupandusetagsandattributesnotdiscussedformallyinthecoursewareSetthetitledisplayedinthetitlebarofthebrowserSetthebackground,text,andlinkcolors5.TextStylePhysicallymarkuptext(bold,changerelativefontsize)Logicallymarkuptext(headinglevel)Centertext229

PAGE 245

AppendixN(Continued)6.ListsIncludeanordered(numbered)orunordered(bulleted)listIncludeanumberorbulletSetthebullettypeforabulletedlist7.ImagesIncludeasimpleimageAppropriatelysettheCOORDSattributeoftheAREAtaginaclient-sideimagemap,givencoordinatesforaclickableregiononanimageTurnofftheborderforaclickableimageUsethemouseOverandmouseOutattributesoftheIMGtag8.TablesUsetherowspan,nowrap,andvaligncellattributesSettheattributesofatableDesignatetablerowsDesignateheaderanddatacells9.FramesCreateaframesversionofaWebsiteSetthetargetofalinktoaspecicframe,tothetoplevelwindow,ortoanewwindowUsetheNOFRAMEStagtodisplayalternativecontent10.FormsIncludeanINPUTelementofTYPEtextIncludeanINPUTelementofTYPEradioIncludeanINPUTelementofTYPEresetIncludeanINPUTelementofTYPEsubmitCallacgi-scripttoprocesstheinformationinspecicformelements230

PAGE 246

AppendixN(Continued)AssignmentRubrics(annotatedwithexpert'sresponses)1.CreateIndexPage(seeFigure2fornalformofpage) 4 “Student'sName”HomePageappearsontitlebar 5,4,3 “Student'sName”appearsonpage 5 Nameandhyperlinksarecentered 5,4 Nameappearsinbiggertextsize 4 Nameisdarkblue 5,4,3 Currencyentitysymbolisusedbetweenandaroundtextualhyperlinks 3,4 Linksaremaroon(orblackifclicked) 4,3 Forhyperlinks,onlytheword(andnotblankspace)isunderlined 4,3 Backgroundispaleyellow 2,1 Otherimplicitobjectivesyoufeeltheassignmentcovers2.CreateTemplateFile 3,4 “Student'sName”HomePageappearsontitlebar 4,3 BGCOLORissettosomevalue 3,4 “Student'sName”appearsincopyrightnoticebetweenHTMLcom-menttags 4,3 TEXTcolorsettosomevalue 4,3 LINKcolorsettosomevalue 4,3 VLINKcolorsettosomevalue 4,3 ALINKcolorsettosomevalue 2,1 Otherimplicitobjectivesyoufeeltheassignmentcovers231

PAGE 247

AppendixN(Continued)3.CreatePersonalPage 4 :Personalappearsontitlebar 5 PersonalFactstextcentered 5 Paragraphnotcentered 5,4,3 Navigationalhyperlinkscentered 5 H1tagusedfortitle 2,1 Otherimplicitobjectivesyoufeeltheassignmentcovers4.CreateLinksPage 3,4 Linksallwork 6 Opencirclesforbullets 5,3 Ergonomicsdened 5,3 Dangersofradiationdiscussed 5,3 Effectsoneyesdiscussed 5,3 Methodsofprotectingeyesdiscussed 5,3 Effectsonarmsandhandsdiscussed 5,3 Methodsofprotectingarmsandhandsdiscussed 5,3 Effectsonskeletondiscussed 5,3 Methodsofprotectingskeletondiscussed 6,5 Allparagraphsinbulletindentedsameamount 1,2,9 Otherimplicitobjectivesyoufeeltheassignmentcovers5.AddImages 7 Imageappearsonindexpage232

PAGE 248

AppendixN(Continued) 7 Foralltextinimageonindexpage,mouseOverworks 7 Foralltextinimageonindexpage,mouseOutworks 7,4 Allimagehyperlinkswork 7 Norectangleappearsaroundimageonindexpage 3,4 Alltextualhyperlinksworkbelowimageonindexpage 7,4 Allnavigationalimagesinplaceonpersonalpage 3,4,2 Forpersonalpage,personalhyperlinkdisabled 3,4,2 Forpersonalpage,allimagehyperlinksotherthanpersonalwork 3,4,2 Forpersonalpage,alltextualhyperlinksotherthanpersonalwork 7 Forpersonalpage,mouseOverworksforallnavigationalimages 7 Forpersonalpage,mouseOutworksforallnavigationalimages 7 Norectangleappearsaroundimageonpersonalpage 1 Otherimplicitobjectivesyoufeeltheassignmentcovers6.CreateProfessionalPage 8 Tabletitlespansallthreecolumns 8 Tablebordervisible 8 MSWindows95staysononelineregardlessofbrowserwindowsize 8 Speciclanguages,environments,andtoolsappearondifferentlines 7 Allnavigationalimagesinplaceonprofessionalpage 3,4,2 Professionalhyperlinkdisabled 3,4,2 Allimagehyperlinksotherthanprofessionalwork 8 TableheadercellscreatedwithTHtags 1 Otherimplicitobjectivesyoufeeltheassignmentcovers233

PAGE 249

AppendixN(Continued)7.UpdateLinksPage 9 splash.htmleloadedinitially 9 Hyperlinksinleftframedisplayinrightframewhenclicked 9 Hyperlinksinrightframedisplayinfullbrowserwindow 7 Allnavigationalimagesinplace 3,4,2 Linkshyperlinkdisabled 3,4,2 Allimagehyperlinksotherthanlinkswork 9,3,4,2 Hyperlinksinbottomnavigationframedisplayinfullbrowserwindow 9 NOFRAMEStagpresentinHTMLsourcecode 1 Otherimplicitobjectivesyoufeeltheassignmentcovers8.UpdatePersonalPage 10 Usermayonlytypeuptothreeitemsineachtexteld 10 Clickingaradiobuttonturnsothersoff 10 Clickingclearbuttonclearsformelements 10 Cleartextappearsonclearbutton 10 ProcessRequesttextappearsonbutton 10 FormcorrectlyconvertsbetweenhexadecimalFF033anddecimal255051 4,1,2 Otherimplicitobjectivesyoufeeltheassignmentcovers234

PAGE 250

AppendixOSelf-ReportMeasureofStudyGroupPatternsName: SS#: Werealizethatsomeofyoumayhaveformedstudygroupsforthisclass,inwhichyouworkonassignmentsandstudyforexamstogether.Inordertosupportacon-sistentexperienceforthemembersofthesegroupsandtosimplifydataanalysisforthestudyassociatedwiththeHTMLlab,werequestthatyousupplyuswithsomeinformation.Thankyouforyourcarefulconsiderationofthefollowingquestions.1.Areyouamemberofastudygroupforthisclass?Ifyes,gotoquestion3.Ifno,gotoques-tion2.YesNo2.Doyouhopetojoinastudygroupforthisclass?Ifyes,continue.Ifno,stopandturninthissheet.YesNo3.Ifyouansweredyestoeitherquestion1or2,pleaselisttherstandlastnames(tothebestofyourknowledge)ofthestudentsinyourstudygroup,thegroupyouhopetojoin,orthegroupyouhopetoform. 235

PAGE 251

AppendixPSelf-ReportofAssistanceReceivedName: SS#: WesolicityouraidindetermininghowtoanalyzethedataregardingassignmentandtestscoresfortheHTMLportionoftheclass.Dependingonwhichstudentsworkedtogetherandwhatgroupstheywerein,thedatamustbeanalyzedwithdifferentprocedures.Itdoesnotmatterifyouworkedalone,withfellowstudents,and/orreceivedaidfromacourseofcialand/oranoutsidesource.Thisinfor-mationwillmerelyhelpusdeterminehowtoanalyzethedata.Again,yournameandsocialsecuritynumberwillbereplacedwithyourstudynumber,andyoumayconsideryouranswerstobeanonymous. Pleasellinyournameandsocialsecuritynumberaboveandcirclethemostappropriateresponsebeloworwriteinananswer.Thankyouforyourcarefulconsiderationofthesequestions.1.WhatpercentageofthetimedidyouworkontheHTMLassignmentswithfellowclassmates? 020406080100 2.Pleaselisttheclassmateswithwhomyouworked. 3.HowoftendidyoureceivehelpcompletingtheHTMLassignmentsfromoneofthecourseofcials(acoursefacilitator,acourseassistant)? neverrarelysometimesoftenalways 4.HowoftendidyoureceivehelpcompletingtheHTMLassignmentsfromanoutsidesource? neverrarelysometimesoftenalways 236

PAGE 252

AppendixP(Continued)5.HowmanyoftheHTMLassignmentsoutofeightwouldyousayyoudidcompletelyonyourown? 012345678 6.Howmanyhoursperweekdidyouspendcompletingtheassignments? 0123456789101112131415+ 7.WhatdidyoulikemostabouttheHTMLlessonsandassignments?8.WhatdidyoulikeleastabouttheHTMLlessonsandassignments?9.Whatrecommendationswouldyoumakeforimprovingthecoursewareand/orassignments?237

PAGE 253

AppendixQFreeFormResponsesfromStudentsOnthedaytheposttestwasadministered,thestudentswereasked“WhatdidyoulikemostabouttheHTMLlessonsandassignments?”Theiractualresponsesandtheidentiedresponsecategoriesappearbelow.1.Nothingnothing(blank)2.Prideinownaccomplishmentsprideincompletingassignmentsfeelinglikeaprofessionalenhancedcomputerskillsoutcomeofhardwork3.Relevant,interestingmaterialandassignmentslearningHTMLlanguagelearninghowtomakeWebpageschallenging/stimulatingmaterialrelevantmaterialinterestingmaterialrstbasicstep-by-stepassignments(2-4)nalproductlearningaboutbrowsersgraphicsandcolorsmakingswappableimagesmakingframeslearningterminologyassociatedwithWebpages4.Contentondemandcouldworkathomeself-pacedaccesstocontentfast-pacedCD-ROMformat(versuslecture)referencetoolforthefuture238

PAGE 254

AppendixQ(Continued)abilitytoreviewmaterialoverandovercouldskippartsIalreadyknew5.Convenientsubmissionprocessabilitytosubmitworkviae-mailimmediateconrmationofassignmentsubmissiondueatmidnight6.Exposuretodifferentinstructionalparadigmexposuretoadifferentteachingapproach7.Tutorialrelationshipwithassistantbeingabletogetone-on-onehelpwhenneededhelpfulassistant8.Examples,layoutoflessons,andsmallstepsexamplesclear,easytouseCD-ROMorganizedlessonsbrokendownintosmallstepshavingweeklyassignments9.Narrationnarrationhelpful,whencouldbeheardpreferrednarrationoverwritteninstructionsThestudentsalsowereasked“WhatdidyoulikeleastabouttheHTMLlessonsandassignments?”Theiractualresponsesandtheidentiedresponsecategoriesappearbelow.1.Timeconsumingtootimeconsumingcompetedwithotherassignments239

PAGE 255

AppendixQ(Continued)2.Coursewarelayoutdifculttonavigatenotabletobookmark,hadtostartfromindexeverytimeneedprintoptiononreferencematerialnotwellorganizedippingbetweennotepadandcoursewareippingbetweenassignmentsandlessonsdifcultwithonlyonewindowavailable,havetwoinsteadCD-ROMnotuser-friendly3.Instructionnotadequatevague,hardtounderstandneedsmoredetail,especiallylaterassignmentsinsufcientsupplementalresourcesneedhandoutsneedsmoreexamplesneedexamplesthatmatchtheassignmentsmorecloselythereweremistakesunnecessarymaterialcoveredaddedtoconfusionneededhelpsectionneededFAQsectionneededtrouble-shootingsectionbreakinformationdownmore4.Assignmentrequirementsnotclearnotclearwhattodomissingimportantinformationassignment5onimages5.Materialandassignmentstoohardtoohardforstudentswithlittletonocomputerexperiencenotgearedtowardthecomputernoviceprojectscareyterminologyabarrierfornovicestudents240

PAGE 256

AppendixQ(Continued)lessononHTMLterminologyrstmaterialtoohardassignmentsrequiredpreviousknowledgeofHTMLsomeassignmentsdidnotmatchtutorialexamplesassignmentsharderthantutorialexamples,alwaysrequiredsomethingmore(later)assignmentstoohardassignmentstoolongtoomanyassignmentshardtomatchexamplesshowntoomuchresearchrequiredforergonomicsassignment6.PreferalternativeteachingparadigmCD-ROMshouldbesupplemental,notprimaryresourceinformationnotcoveredinclassshouldtakequestionsanddiscussmaterialinclassteachershouldbesoleproviderofcourseinformationrigidformatwithoutlivehelpjustawayfortheinstructortobelazyitshouldbeagroupprojectshouldbedoneinlabsettingwithsmallnumberofstudentseachatacomputerallowmoregroupworkindividualassignmentduedatesnotannouncedinclass,hadtohavecomputertoreviewthemnoonetotalktonoteacheravailabletohelpnothavingfacetofacediscussionsdemonstrationswouldbehelpful7.NarrationnarratordidnotholdattentionnarrationboringsamevoiceoverandoversaidthesamethingeverytimetheCD-ROMstarted241

PAGE 257

AppendixQ(Continued)narrationforcedmetogofasterthandesirednotgearedtowardvisuallearnersshouldhavetextualinstructionsaswellnowritteninstructionsaccompanyingnarrationprovideprintedbookletaswelluselessvoicepromptsitgetsreallyannoyingwhenyou'refrustratedabitwordy8.TechnicaldifcultiesCD-ROMofpoorqualitynotgivenpasswordCD-ROMgotcorruptedCD-ROMcrashedCD-ROMnotcompatiblewithMacintoshhomecomputerdevelopedavirusnotreadableinsomelabsoncampusshoulduseprofessionaltorecordnarrations,narratornotclearnarratorsvoiceseemsstiffnarrationchoppy9.Submissionproceduredidnotlikesubmissionproceduresubmitwithoppyinsteadofelectronicallytransformationofworkduringtransmissionworknotbeingreceivedwhensentnotclearhowtosubmitassignmentssubmissionprocessunclear10.Nosoundnosoundvolumetoolow242

PAGE 258

AppendixQ(Continued)11.Materialnotstimulatingandrelevantboringunabletopursueamorecreativedirectiononlyprogrammersneedthisinformationfocusshouldbeonteaching,notbuildingaWebpagenotsurehowassignmentshelpteachersfocuswasontechnicaltools,noteducationalapplicationsofthemassignmentsmonotonousmaterialredundant12.PreferWYSIWYG(Whatyousee,iswhatyouget.)pagemakingprogramseasiertouseandmoreunderstandableHTMLisobsoletewithalloftheshortcutsbuiltintootherprogramsdidnotlikehavingtouseNotepadHTMLoutdated,Flash4.0isthewaveofthefutureteachersdonotneedHTMLwhyarewelearningHTMLwhenothereasier,moretimeefcientpro-gramsexist13.Interactionwithteachingassistantsandinstructorsassistantstook6-7daystorespondtoe-maildelayingradingtoolongcorrectivefeedbackinadequatehardtoaskquestionsnonighttimeofcehourshardtogettoassistants'ofcehoursassistants'notalwaysthereduringofcehoursnohelpfromcourseassistantassistants(andprofessor)didnotknowhowtodotheassignmentsassistanthadstillnottriedassignment2daysbeforeitwasdueassistantsnotwillingtohelp243

PAGE 259

AppendixQ(Continued)14.DeadlinesdifferedfromhowotherdeadlineswerehandledintheclassrstassignmentduetoosoonCD-ROMsneededtobedistributedearlierspreadoutdeadlinesmorehaveoneassignmentdueperweekdon'thave2assignmentsdueonthesamedaynothave3and4dueatsametimeshouldhavelatedeadlineforlasttwoalsoforcedtomoveonbeforereadyduetodeadlinespaceseemedharriedassignmentsrushedfelttorn-turnitinontimeandnotunderstandorunderstandandturnitinlatefornopointsnotenoughtimetocompleteassignmentshavenodeadlinesIneededmoretime15.Unfairgradingithurtourgradesgradingunfairshouldnothavebeengradedmakeitforextracreditworthtoomanypoints(relativetootherlabs)assignmentsworthtoomanypointsnotworthenoughpointsfortheamountofworkittookpointdistributionsforindividualassignmentsnotappropriategradingrubrictoostrictIreceivedzeroswhenIdidtheworkIthoughtitworked,butlostpointshavingaposttestwherereferencematerialisnotavailable16.Forcedparticipationinstudyfeltlikeforcedtoparticipateinastudyfeltlikealabrat244

PAGE 260

AppendixQ(Continued)peoplegotdifferentpartsoftheassignmentsthesoftwarewastestedonusdifferentdeadlinecontingenciesunfair17.FeelingoffailureanddefeatIdidverypoorlyIfailedthislabIcouldn'tevencompletesomeoftheassignmentsIjustkeptguessinguntilitlookedrightmademenotwanttolearnmoreaboutcomputersfrustrationanddisappointmentinnevergettingittoworkIhadtohave100%helpallwedidiscopy,notreallylearnfrustratingwhenIcouldn'tgetittoworkmyownprocrastinationalwayshavingtoaskforhelpandstressaboutitIdidnotreceivefullcreditforanythingIalmostgaveuponthewholethingIneededalotofassistanceItwouldhavebeennicetohaveendedupwithanactualwebpageIgaveupFinally,thestudentswereasked“Whatrecommendationswouldyoumakeforim-provingthecoursewareand/orassignments?”Theiractualresponsesandtheidentiedresponsecategoriesappearbelow.1.Eitherreducetheamountofworkorgivemoretimetocompleteit.don'tmake2bigassignmentsdueeachweekbreaklongerassignmentsintosmallerstepslessmaterialkillsectiononergonomicslongertimetodowork245

PAGE 261

AppendixQ(Continued)2.Providemoreassistance.24hourhotlinedoitinalabwithteacheronhandtoanswerquestionshanddisksoutinlabandaddressproblemstogetherrightawayFAQbutton3.Augmentcoursematerial.detailed,stepbystepinstructionsforcompletingassignmentsmoreexamplessolutioncodeaftersubmissionllinmissinginformationforassignment54.Altertheinstructionalparadigmsomewhat.havesomelecturealsoallowformorecreativityallowstudentstosubmitworkingroups5.Makesoundaccessibleonmoremachinesandeasytomute.xsoundtoworkonmoremachinesputonamutebutton6.Provideoptionforprintedmaterial.printsyllabusatleastbooklettoaccompanyCD-ROMprintoptiononreferencematerial7.MakeHTMLunitextracreditratherthanrequired.extracreditinstead246

PAGE 262

AppendixRSurveyonPlatformandBrowserUsage1.PlaceanXnexttothelocationofthecomputeryoumostoftenusetocom-pleteclassassignments. home openuselabinEducation otheropenuselaboncampus(pleasespecifywhichone: ) other(pleasespecify: )2.PlaceanXnexttothetypeofcomputeryoumostoftenusetocompleteclassassignments. PCwithWindows98 PCwithWindows95 PCwithWindows(notsureif95or98) MAC other(pleasespecify: )3.PlaceanXnexttothebrowseryoumostoftenuse. NetscapeNavigator3 NetscapeNavigator4 NetscapeNavigator4.6 NetscapeNavigator4.7 NetscapeNavigator(notsurewhichversion) InternetExplorer4 InternetExplorer4.5 InternetExplorer5 InternetExplorer(notsurewhichversion) AOL Other(pleasespecifywhichone: )247

PAGE 263

AppendixSCCodetoAnalyzeDataViaRandomization687978797979797879797979787979797978797979797879787978797879787879797878797978787979787879797:7979797:7979797:7979797:7979797:79797:7979797:7977<;>=@?:A9BrC@DFE8G)H9I9IrJLKNMrCPORQTSVUXWRQ9Y>Z[E>B9\:Q^]_Ka`cb9bedfC@DFE8Ghgid>jRg^jFBlk>jlm7en>jFB^opC9g9g8CF=@OqClgiD9B:Q^O8G>jFmrGc=TQsO:AT?RQFBlG:AtGh=vurgFjg^j9QlBcZ[E7v?9u:BF?h=:g^j>gxKy?:BR=skrC@m:jFmzG9ERQ^GrGlEfClgt=sBhC@DrCPORQ:b{Zl=|?:A9BhC@DlE8GrOh=|GrClZFj)C9geB:jFG>QRCPORjlm7}QsO:m~Q:b9bzZ[ERQsO:D>j>giGc=eG9ERjz=^BhC@DhCPOcQ:b~Zl=sm:jrQFB:j{ZlbFj9QlBRbsAC@m:j^O8GrC@whC^jFmVU77v?RjFB^o:n:BR=8Z€UZi‚>QRge‚:BhC[G9G>j^OC[O)=sB9m>jFBrGc=}Q^ORQ:bsA9\>j}G9ERji?>BR=8Z|B:QRg@GrCPORQFGrCF=@Oƒm>Q^G>Q7iwR=sBtGlERjTmhClg9g^jlBlG>Q^GrCl=@O„jsO8GrC@GcbFjFm†…+‡:w9w:j>Z@Grgt=^wTˆ>j8QFmRb:CPOcjt;R=@O8GrCPO>D:jsOrZ8C^j>g7zCPO~Q<‰RjsŠR‹lŒ:Q>g^jFm{;R=@u:Bcg^j~=@Oz9MlW:S'…/Š>ATMrC[ORQ}SŽUWRQ8Y>Z[E:B9\>Q^]ŽUMc=iGlERj<ŠRj>g|G7t=swm:jFBcg@GRQsO:mhCPO>DKaGlERjj>gz=@u8Gcb>CPORjFmrŠ:ArŒ9B9A>QsOzŽU‘€UXWRQsOhbsA7zCPO-…+dRQsO:mR=[oLC@\:Q^GrCl=@OVK’Œc=9=|Ghg@G:B:Qs?ƒQsO:mTWh=@O8GRjt;:QFBRb9=iWRj^G9Eh=smcg~CPOzŒhCF=9b9=^D9AL…fK7rg^j>Zl=@O>mzjFmrC[GrCF=@O”“PJ@•9•l–c—xKy‚RjFB:jTwR=9b9b9=s‚>jFmVU77eMlERjiocj^GlEh=^m{=swrD:jsOcjFB:Q^GrC[O:D~B:QsO:mc=[o~O9ulo8ŠRjlBcgi‚c=sBl]>jlm)=|O{QDrSrCPO9u:VU7z™[GjFmzwR=sB{Q:bsG>jFBFORQFGrC[k>jT?hblQ^G8wR=sBFofg€U77eMlERj{Zl=sm:j}‚>Q>grZl=[o8?pCFbFjFmšCPO8Gh=TQ^O{jF>j>Z[u8G>Q^ŠhbFjT‚rC[GlE~GlEcj}wR=8b9b9=|‚rC[O:D„Zl=[o9ocQ^O:mVU7DcZ9Z}‹:=e?RjFB^o>n8BR=8Ze?RjFBFo:n8BR=8Z›UZ797978797979797879797979787979797978797979797879787978797879787879797878797978787979787879797:7979797:7979797:7979797:7979797:79797:7979797:7l6œCPOfZlb@u:m:jz>g|G8mhCF=›UERŸœCPOfZlb@u:m:jz>g|G8mRb:CPŠ U¡ERŸœm:jlwhCPORjt¢J[£8¤œm:jlwhCPORjTdz8¦œm:jlwhCPORjr;t8–œm:jlwhCPORjt`rfJœm:jlwhCPORjƒ™[M9‡8d8`lMr™s§l¢:¨J@I9I9I8I9I9IUI)686iwR=^B}?:B>j>Z8Clg8Cl=@O~Gc=eG9ERjZ^jwRb9=lQ^G)Q^k>jFB:QlD:j“wRb9=FQFG{Q_K/CPO8G„g@G>QFBlGVK/C[O8G~j^O:mh—kc=:C|m?RjFB^o8u:G>j“wRb9=FQFGzmŽK/CPO8GtO9ulop—kc=:C|mg@‚>Qs?“wRb9=lQ^G~mVK/CPO8Gƒb9=8Z>J'K/C[O8G)b8=8Z^HR—wRb9=lQ^Gšg@G>Q^G>lBcZ^Q“wRb9=lQ^G~mVK/CPO8GtO9ulop—xwRb9=lQ^Gšg@G>Q^G>lBcZ“wRb9=FQFGzmŽK/CPO8GtO9ulop—wRb9=lQ^Gšg@G>Q^G>lB:Q“wRb9=FQFGzmŽK/CPO8GtO9ulop—wRb9=lQ^Gšg@G>Q^G>RZ^Q“wRb9=FQFGzmŽK/CPO8GtO9ulop—kc=:C|mmFulo8?R8QFB9B:QFA“wRb9=FQ^G{Q_K/C[O8GTO8ulop—kc=:C|mB:j9QFm:cCPO“Z[ERQFBrwFORQsocjKawcb9=FQ^Gt?KCPO:Gƒg@GRQFBlG_KCPO8G{jsO:mr—kc=:C|mmhClg[?hblQFA“Z[ERQFBƒ7eG>j>g@G_KNwRb9=lQ^Gšg^I›KNwRb9=FQ^Gr?RFk>Q:b@uRjc—248

PAGE 264

AppendixS(Continued)CPO8GTocQRC[O“kh=:C@mh—CPO8GCKZ[O8GVwRb9=FQ^G?¢9wRb9=FQ^Gg^IK’gxK?R^kRQ:b@uRj›696879797879797979787979797978797979797879797979787978797878797978787979787879797696<§|o:OfCPŠ9urgiG>jRg@G_K/Zl=@Org:C@m:jFBhC[O:Dzd_K;KQ^O:mt`ŽU696vd>j8QFmrGlERjTm:Q^G>QTw9BR=[o~GlE:B>j9j)g^js?RQFB>Q^G>jtm:Q^G>QtwhClbFj>gzCPO8Gc=tQzg:CPO:DRbFjzQFB8B:QFAt?UB:j9QFm:cCPO“[…?:BR=:Z@dŽUm:QFG'…fKN?VK#JLKydr—B:j9QFm:cCPO“[…?:BR=:Zs;_Um:QFG'…fKN?VKyd:rJLKNd:8;c—B:j9QFm:cCPO“[…?:BR=:Z|`VUm:QFG'…fKN?VKyd:9;9fJLKNd:9;9l`r—696i;8Q>b8Z[uhbFQFG>jtGlERj}G>j>g@G„g@G>QFGrClg@GrC9Z}wR=sBtGlERj~=sBhC@DhC[ORQ:bTm:Q^GRQUg^I})g@G>Q^GRFBcZ^Q“?VKN¢h—696vn>jlB^o8u8G>jTGlEcjTm>Q^G>QrQsO:mrD:jsORjFB>Q^G>jTGlERjTmhClg@G:BhCPŠ9u8GfCF=@Oq=swtG>j>g|Gƒg@GRQ^GrClg@GfClZ9g€UZ[O8G~tI›wR=sB“+C|rJ'CsR™[M9‡8d8`lMr™s§l¢:¨›Cs9c—696:C|w“9“CF6RJ@I9Ic—|9›“+Cl6RJ@I9IUIR—9—~i?:BrCPO8G8w“[…+‚c=^Bl]rCPO:D„=|OšC[GRjFB:Q^GrCl=@O{9mR@O…pK/C8—?RjFBFo8u8G>j“?VKN¢h—g<~g@G>QFG>FBcZ^Q“?VKy¢r—696};>=@u9O:GtGlEcjj>g@Gšg@GRQ^GrClg@GfClZ9giGlERQ^G)QFB>jzbFjRg9gvG9ERQsOšg^IUC@w“g}zg^Ic—Z[O8G:9696i;8Q>b8Z[uhbFQFG>jT?R‹^k>Q>b@uRjU?R^k>Q:b|uRjrƒJe‹“9“wcb9=FQ^Gr—zZ@O8Gr—F6:™@M9‡9d8`lMf™s§F¢:¨›696vˆrC9g[?hbFQFArGlEcjTB>j>g[uhb|Grg€UmhClg[?hblQFA“[…@=[o8OpCPŠ9urgc…pKg^I›K?R^k>Q:b|uRjR—249

PAGE 265

AppendixS(Continued)69687979787979797978797979797879797979787979797978797879787879797878797978787696vM9‚h=iD9Bc=@u9?~G>j>g@GVKZ9=@Org8C@m>jFBhCPO:Dzd~QsO:mz;_U696vd>j8QFmrGlERjTm:Q^G>QTw9BR=[o~G9‚c=zg^js?cQFB:Q^G>jrm>Q^G>QTwhCFbFj>g~CPO:Gc=}Q{g8CPO>DRbFjrQFB9B:QFAt?UB:j9QFm:cCPO“[…?:BR=:Z@dŽUm:QFG'…fKN?VK#JLKydr—B:j9QFm:cCPO“[…?:BR=:Zs;_Um:QFG'…fKN?VKyd:rJLKNd:8;c—696i;8Q>b8Z[uhbFQFG>jtGlERj}G>j>g@G„g@G>QFGrClg@GrC9Z}wR=sBtGlERj~=sBhC@DhC[ORQ:bTm:Q^GRQUg^I})g@G>Q^GRFBcZ“?VKyd:9;c—x696vn>jlB^o8u8G>jTGlEcjTm>Q^G>QrQsO:mrD:jsORjFB>Q^G>jTGlERjTmhClg@G:BhCPŠ9u8GfCF=@Oq=swtG>j>g|Gƒg@GRQ^GrClg@GfClZ9g€UZ[O8G~tI›wR=sB“+C|rJ'CsR™[M9‡8d8`lMr™s§l¢:¨›Cs9c—696:C|w“9“CF6RJ@I9Ic—|9›“+Cl6RJ@I9IUIR—9—~i?:BrCPO8G8w“[…+‚c=^Bl]rCPO:D„=|OšC[GRjFB:Q^GrCl=@O{9mR@O…pK/C8—?RjFBFo8u8G>j“?VKyd:9;h—g<~g@G>QFG>FBcZ“?VKyd:9;h—696};>=@u9O:GtGlEcjj>g@Gšg@GRQ^GrClg@GfClZ9giGlERQ^G)QFB>jzbFjRg9gvG9ERQsOšg^IUC@w“g}zg^Ic—Z[O8G:9696};8Q:b:Z[uhbFQ^GRjT?R‹^k>Q:b|uRjU?R^k>Q:b|uRjrƒJe‹“9“wcb9=FQ^Gr—zZ@O8Gr—F6:™@M9‡9d8`lMf™s§F¢:¨›696vˆrC9g[?hbFQFArGlEcjTB>j>g[uhb|Grg€UmhClg[?hblQFA“[…+d>‹l;L…fK/g^I›K?R^k>Q:b|uRjR—250

PAGE 266

AppendixS(Continued)69687979787979797978797979797879797979787979797978797879787879797878797978787696vM9‚h=iD9Bc=@u9?~G>j>g@GVKZ9=@Org8C@m>jFBhCPO:Dzd~QsO:mr`VU696vd>j8QFmrGlERjTm:Q^G>QTw9BR=[o~G9‚c=zg^js?cQFB:Q^G>jrm>Q^G>QTwhCFbFj>g~CPO:Gc=}Q{g8CPO>DRbFjrQFB9B:QFAt?UB:j9QFm:cCPO“[…?:BR=:Z@dŽUm:QFG'…fKN?VK#JLKydr—B:j9QFm:cCPO“[…?:BR=:Z|`VUm:QFG'…fKN?VKyd:rJLKNd:9`h—696i;8Q>b8Z[uhbFQFG>jtGlERj}G>j>g@G„g@G>QFGrClg@GrC9Z}wR=sBtGlERj~=sBhC@DhC[ORQ:bTm:Q^GRQUg^I})g@G>Q^GRFB:Q“?VKyd:l`h—x696vn>jlB^o8u8G>jTGlEcjTm>Q^G>QrQsO:mrD:jsORjFB>Q^G>jTGlERjTmhClg@G:BhCPŠ9u8GfCF=@Oq=swtG>j>g|Gƒg@GRQ^GrClg@GfClZ9g€UZ[O8G~tI›wR=sB“+C|rJ'CsR™[M9‡8d8`lMr™s§l¢:¨›Cs9c—696:C|w“9“CF6RJ@I9Ic—|9›“+Cl6RJ@I9IUIR—9—~i?:BrCPO8G8w“[…+‚c=^Bl]rCPO:D„=|OšC[GRjFB:Q^GrCl=@O{9mR@O…pK/C8—?RjFBFo8u8G>j“?VKyd:l`r—g<~g@G>QFG>FB:Q“?VKyd:l`r—696};>=@u9O:GtGlEcjj>g@Gšg@GRQ^GrClg@GfClZ9giGlERQ^G)QFB>jzbFjRg9gvG9ERQsOšg^IUC@w“g}zg^Ic—Z[O8G:9696i;8Q>b8Z[uhbFQFG>jT?R‹^k>Q>b@uRjU?R^k>Q:b|uRjrƒJe‹“9“wcb9=FQ^Gr—zZ@O8Gr—F6:™@M9‡9d8`lMf™s§F¢:¨›696vˆrC9g[?hbFQFArGlEcjTB>j>g[uhb|Grg€UmhClg[?hblQFA“[…+d>‹F`'…fK/g^I›K?R^k>Q:b|uRjR—251

PAGE 267

AppendixS(Continued)69687979787979797978797979797879797979787979797978797879787879797878797978787696vM9‚h=iD9Bc=@u9?~G>j>g@GVKZ9=@Org8C@m>jFBhCPO:D{;rQsO:mr`VU696vd>j8QFmrGlERjTm:Q^G>QTw9BR=[o~G9‚c=zg^js?cQFB:Q^G>jrm>Q^G>QTwhCFbFj>g~CPO:Gc=}Q{g8CPO>DRbFjrQFB9B:QFAt?UB:j9QFm:cCPO“[…?:BR=:Zs;_Um:QFG'…fKN?VK#JLK;c—B:j9QFm:cCPO“[…?:BR=:Z|`VUm:QFG'…fKN?VK;9rJLKa;99`h—696i;8Q>b8Z[uhbFQFG>jtGlERj}G>j>g@G„g@G>QFGrClg@GrC9Z}wR=sBtGlERj~=sBhC@DhC[ORQ:bTm:Q^GRQUg^I})g@G>Q^GR>Z^Q“?VK;9l`h—x696vn>jlB^o8u8G>jTGlEcjTm>Q^G>QrQsO:mrD:jsORjFB>Q^G>jTGlERjTmhClg@G:BhCPŠ9u8GfCF=@Oq=swtG>j>g|Gƒg@GRQ^GrClg@GfClZ9g€UZ[O8G~tI›wR=sB“+C|rJ'CsR™[M9‡8d8`lMr™s§l¢:¨›Cs9c—696:C|w“9“CF6RJ@I9Ic—|9›“+Cl6RJ@I9IUIR—9—~i?:BrCPO8G8w“[…+‚c=^Bl]rCPO:D„=|OšC[GRjFB:Q^GrCl=@O{9mR@O…pK/C8—?RjFBFo8u8G>j“?VK;9l`r—g<~g@G>QFG>>Z^Q“?VK;9l`r—696};>=@u9O:GtGlEcjj>g@Gšg@GRQ^GrClg@GfClZ9giGlERQ^G)QFB>jzbFjRg9gvG9ERQsOšg^IUC@w“g}zg^Ic—Z[O8G:9696i;8Q>b8Z[uhbFQFG>jT?R‹^k>Q>b@uRjU?R^k>Q:b|uRjrƒJe‹“9“wcb9=FQ^Gr—zZ@O8Gr—F6:™@M9‡9d8`lMf™s§F¢:¨›696vˆrC9g[?hbFQFArGlEcjTB>j>g[uhb|Grg€UmhClg[?hblQFA“[…;8‹F`'…fK/g^I›K?R^k>Q:b|uRjR—B:j^Glu:BlO{I›252

PAGE 268

AppendixS(Continued)687978797979797879797979787979797978797979797879787978797879787879797878797978787979787879797:7979797:7979797:7979797:7979797:79797:7979797:7977imhClg[?hblQFAt?:BhCPO8Grgt=|u8GtG9ERjg@GtŠRjRCPO:D„Zl=|O:mFurZ@GRjFmKGlERj{g@G>QRg@GrClZQKgFI›KG9ERji?cjFBcZ^jsO:G>QFD:j)=swtB:QsO:mc=[opC@\:QFGrCF=@OrgTGlERj{g@G>Q^GrC9g@GrClZ{Clg7iD9B:j9Q^GRjFBrGlERQsOVK#J@I9I8‹s?R^k>Q>b@uRj>7>J|I9I›KC@Ghgtg:C@DFOfC@wrClZ^QsOrZFj)bFj^k>j:b€K?c^k>Q:b@ucj>7>J@I9IK7}QsO:mT?R^k>Q:b|uRjU797978797979797879797979787979797978797979797879787978797879787879797878797978787979787879797:7979797:7979797:7979797:7979797:79797:7979797:7l6kc=:C|mtmhC9g[?hbFQFA“Z@ERQFB„7vGRj>g@G_KNwRb9=FQFG„gFI›Kywcb9=FQ^Gt?R^k>Q>b@uRjR—?:BhCPO8G:w“[…|@Org^IrtVUHFwrwR=sBtGlERjTcgrZl=[o8?RQlBhClgl=@O„Q^O:mƒC9geD9B>j9Q^G>jFBrGlEcQsOzŽUHFw…fKg^I›KyG>j>g@G_K#J@I9I8‹s?R^k>Q>b@uRj>7>J|I9IR—?:BhCPO8G:w“[…|=swR@OG9ERj}B>QsO:mR=[oLC@\:Q^GrCl=@OrgxK#g9=rg^I{Clgrg8C@DFOfC|whClZ^QsO:GƒQ^GtG9ERj}ŽUHFw…s—?:BhCPO8G:w“[…+?RjFBcZ^j^O8GšbFj^k>j>br|O…fK?R^k>Q:b|uRj>7>J@I8IR—?:BhCPO8G:w“[…“?RlVU9wr—%U@O…pK?R^k>Q>b@uRjR—B:j^Glu:BlOV687978797979797879797979787979797978797979797879787978797879787879797878797978787979787879797:7979797:7979797:7979797:7979797:79797:79797977iB:j9QFm:cCPO{B:j9QFmcgim:Q^GRQTw8BR=[o~GlERjTwhCFbFjTwFORQ|ohjrQ^O:mT‚:BhC[G>j>g~C[GrGc=eGlERjrQFB9B:QFAt?U7z™[G}ŠRjlDhCPOrgi‚8BhC@GrCPO:D{Q^Gƒb9=8Z^QFGrCF=@Oqg@GRQFBlG>‹hJ}=sw}?{Q^O:mtwrCFb9b8g~CPOrGlERj~bFQ>g@G7ek>Q:b@uRjrQ^Gƒb9=8Z^QFGrCF=@OƒjsO:m>‹hJ}=sw}? UM9ERjz=|OhbsArB:jl^ufC|B:j|ocjsO:GqClgvG9ERQ^GrGlERjtB:j9Q:b7vO9ulo8ŠRjlBcg~CPOtGlEcjTwrCFbFj<ŠRj~gFjs?RQFB:QFG>jFmtŠ:At‚lEfC[G>j)g[?cQ>Z^jU797978797979797879797979787979797978797979797879787978797879787879797878797978787979787879797:7979797:7979797:7979797:7979797:79797:79797l6kc=:C|mtB:j8QFm:RCPO“Z@ERQFBzwFORQ|ohjKwRb9=lQ^Gr?K/CPO8G„g@G>QFB9G_KCPO8G{jsO:mh—CPO8GšCh™@Sl‡„78CPOVC@w“9“CPO)}wR=@?cjsO“wFORQ|ohj›Kv…BL…s—9—<9T¢^>S9Sr—?:BhC[O8G8w“[…;8Q^O9Oh=|G„=@?Rj^OC[O9?9u8GzwhCFbFjtcg›U@O…fKwFOcQ|ocjR—j:b8g^jwR=sB“+CsRg@G>QFB9G>‹hJLCs9jsO:m_C|9c—}wcg9Z^QsO>w“CPOVK…9wL…fKF? C@h—B:j^Glu:BlOV253

PAGE 269

AppendixS(Continued)687978797979797879797979787979797978797979797879787978797879787879797878797978787979787879797:7979797:7979797:7979797:7979797:7977imFulo8?R8QFB9B:QFAt?:BrCPO8Ghgr=@u8GrGlERj{Zl=@O:G>jsO8Ghgz=^wTGlEcj}whC|Bcg@GtO9ulošb9=8Z^Q^GfCF=@Org7t=sw}GlEcjtQFB8B:QFA{QU™[GšClgufg^jFmrwR=sBrm:jsŠ9u:D8DhCPO:Dr?9u>BF?h=8g^jRg€U797978797979797879797979787979797978797979797879787978797879787879797878797978787979787879797:7979797:7979797:7979797:7979797:7l6kc=:C|mtmFu9o8?R9QFB8B:QFA“wRb8=FQ^G)QK/CPO8GrO9u9op—CPO8GšCwR=sB“+C|8ICssO9uloC|9c—Q^G>lBcZ^Q)Z^Q:b8Z@uhbFQ^G>jRgQ^GrC9g@GrClZz=sw)Z[Eh=:C9Z^jU™POzGlEfC9gtZ^QRg^j›K/C[G7rZ^Q:b8Z[urbFQ^G>j>giGlEcj~=@o8OfCPŠ9ufgeGRj>g@G”^“d>‹l;h—i“d>‹F`r—i¡;8‹l` "VU797978797979797879797979787979797978797979797879787978797879787879797878797978787979787879797:7979797:7979797:7979797:7979797:79797:7979797:7l6wRb9=lQ^Gšg@G>Q^G>lBcZ^Q“wRb9=lQ^G~mVK/CPO8GtO9ulop—wRb9=FQ^GB:QFk8DK/Z^Q^k8D_KaQ8Q^k8DKgxKyGFo8?V696ehC|Bcg@G_Km:j^GRjFB^opCPOcjtGlERjiocj9QsOrgrJLKyd:9;c—Q9Q^k8DztQ^kRjFB:QFD:j“mKd>9;9rJLKyd:9;8l`h—696@?:BhC[O8G8w“[…B:QFk8D8lVUHFwKZ^Q^k:D8lVUHlwK’Q9Q^k8D:lVUHFwc@O…fKB>Q^k8DK/Z^Q^k:DKQ8Q^k8Dh—696e¢:jllG_K/Z^Q:b8Z@uhbFQ^G>jtG9ERjtQ^?9?:BR=@?>BhCsQ^G>jrmrC@w9w:jFB>jsOrZ^j~QsO>mtB:jFGlu:BFOqC[GŽU696T=[o:OfCPŠ9urgiG>jRg@G~wR=sB~Q:b9b_UI8IhJ@HGFo8?{~Z^Q^k:Dz‹rQ9Q^k8D_C@w“Glo8?ƒTIR—<^GFo8?š7sz‹hJL~696iG9‚c=F‹Rg8C@m:jFmge“B:Q^k8D~‹{Z^Q^k8Dh—i“B:Q^k8D~‹rQ9Q^k8Dh—<}GFo8?VB:j^Glu:BlO„gx254

PAGE 270

AppendixS(Continued)687978797979797879797979787979797978797979797879787978797879787879797878797978787979787879797:7979797:7979797:7979797:7979797:79797:7979797:7977rg@G>Q^G>lBcZzZ^Q:b8Z[urbFQ^G>j>gQ^GrClg|GrClZr=swƒZ[Eh=:ClZFjU™POrGlEfClgzZ^Q>gFj›KC@G7rZ^Q:b8Z[urbFQ^G>j>giGlEcj~=|ORj9‹>g8C|m:jFmrG>j>g@G“dR‹l;c—%U797978797979797879797979787979797978797979797879787978797879787879797878797978787979787879797:7979797:7979797:7979797:7979797:79797:7979797:7l6wRb9=lQ^Gšg@G>Q^G>lBcZ“wRb9=FQFGzmŽK/CPO8GtO9ulop—wRb9=FQ^GB:QFk8DK/Z^Q^k8D_Kg%696ehC|Bcg@G_Km:j^GRjFB^opCPOcjtGlERjiocj9QsOrgrJLKyd:9;c—696e¢:jllG_K/Z^Q:b8Z@uhbFQ^G>jtG9ERjtQ^?9?:BR=@?>BhCsQ^G>jrmrC@w9w:jFB>jsOrZ^j~QsO>mtB:jFGlu:BFOqC[GŽU696T=@Ocj9‹>g8C@m>jFmšZl=[o8?cQFBhClgl=|O=swtD8BR=@u9?rgid~QsO:mz;K’gFIllVUJ8JLKy?>_UI9I8I9getB:Q^k8D~‹zZ^QFk8DB:j^Glu:BlO„gx687978797979797879797979787979797978797979797879787978797879787879797878797978787979787879797:7979797:7979797:7979797:7979797:79797:7979797:7977rg@G>Q^G>lB:Q)Z^Q:b8Z[urbFQ^G>j>gQ^GrClg|GrClZr=swƒZ[Eh=:ClZFjU™POrGlEfClgzZ^Q>gFj›KC@G7rZ^Q:b8Z[urbFQ^G>j>giGlEcj~=|ORj9‹>g8C|m:jFmrG>j>g@G“dR‹F`h—%U797978797979797879797979787979797978797979797879787978797879787879797878797978787979787879797:7979797:7979797:7979797:7979797:79797:7979797:7l6wRb9=lQ^Gšg@G>Q^G>lB:Q“wRb9=FQFGzmŽK/CPO8GtO9ulop—wRb9=FQ^GB:QFk8DKQ9Q^k8D_Kg%696ehC|Bcg@G_Km:j^GRjFB^opCPOcjtGlERjiocj9QsOrgrJLKyd:l`h—696e¢:jllG_K/Z^Q:b8Z@uhbFQ^G>jtG9ERjtQ^?9?:BR=@?>BhCsQ^G>jrmrC@w9w:jFB>jsOrZ^j~QsO>mtB:jFGlu:BFOqC[GŽU696T=@Ocj9‹>g8C@m>jFmšZl=[o8?cQFBhClgl=|O=swtD8BR=@u9?rgid~QsO:mr`K’gFIl8IU¦:I›Ky?>_UHl–fJ@IgetB:Q^k8D~‹TQ9QFk8DB:j^Glu:BlO„gx255

PAGE 271

AppendixS(Continued)687978797979797879797979787979797978797979797879787978797879787879797878797978787979787879797:7979797:7979797:7979797:7979797:79797:7979797:7977rg@G>Q^G>RZ^Q)Z^Q:b8Z[urbFQ^G>j>gQ^GrClg|GrClZr=swƒZ[Eh=:ClZFjU™POrGlEfClgzZ^Q>gFj›KC@G7rZ^Q:b8Z[urbFQ^G>j>giGlEcj}G8‚c=F‹>g8C|m:jFmrG>j>g@G'¡;:‹F` ^U797978797979797879797979787979797978797979797879787978797879787879797878797978787979787879797:7979797:7979797:7979797:7979797:79797:7979797:7l6wRb9=lQ^Gšg@G>Q^G>RZ^Q“wRb9=FQFGzmŽK/CPO8GtO9ulop—wRb9=FQ^GZ^QFk8DKQ9Q^k8D_Kg%696ehC|Bcg@G_Km:j^GRjFB^opCPOcjtGlERjiocj9QsOrgjtG9ERjtQ^?9?:BR=@?>BhCsQ^G>jrmrC@w9w:jFB>jsOrZ^j~QsO>mtB:jFGlu:BFOqC[GŽU696vG9‚h=F‹>g8C@m>jFmšZl=[o8?cQFBhClgl=|O=swtD8BR=@u9?rg};rQsO:mr`K’gFIl9£_U£fJLKy?>_UI9I9–9¦ge)Z^Q^k8D~‹TQ9QFk8DC@w“g}TIR—<>gr7sr‹rJLB:j^Glu:BlO„gx687978797979797879797979787979797978797979797879787978797879787879797878797978787979787879797:7979797:7979797:7979797:7979797:79797:797977}Q^k>jFB:QlD:jtB:j^Glu:BlOrgiGlERjrQ^k>jFB:QlD:j{=swTG9ERjg~CPO)QFB9B>QFAzQ{g@G>QlBlGrCPO:D{Q^G7tb9=8Z^Q^GfCF=@Og@G>QFB9G>‹hJiQsO:m~jsO:mhC[O:D{Q^G)b8=8Z^Q^GrCl=@OƒjsO:m:‹hJxU797978797979797879797979787979797978797979797879787978797879787879797878797978787979787879797:7979797:7979797:7979797:7979797:79797:797l6wRb9=lQ^G)Q^k>jFB:QlD:j“wRb9=FQFG{Q_K/CPO8G„g@G>QFBlGVK/C[O8G~j^O:mh—CPO8GCwRb9=FQ^G„g[uloc8I›wR=sB“+C|Rg|G>QFBlG>‹rJLCs9j^O:mC|9h—g[ulo)9zQC@_g[uloš6^“j^O:m:‹>g@GRQFBlG:rJl—B:j^Glu:BlO„g[u9oŽ256

PAGE 272

AppendixS(Continued)687978797979797879797979787979797978797979797879787978797879787879797878797978787979787879797:7979797:7979797:7979797:7979797:79797:7979797:7977v?RjFB^o8u:G>jtB:QsO:mR=@orbsAt?RjFB^o:u8G>j>giGlERjTk>Q>b@uRj>gzCPO)QFB9B:QFA{QiŠ:A)g|G>QFBlGrC[O:D{Q^GtG9ERj7}jsO:m)=sw}GlEcjtQFB8B:QFAK/g^j:blj>Z@GrCPO>D)Q}B:Q^O:mR=[o~k>Q:b@ucjTw8BR=[o~GlERQ^Gr?h=8g:C[GrCF=@O7iwR=sBl‚>QlB9mKQsO:mƒg@‚>Qs?8?fCPO:DrGlERj{Zl=@O:G>jsO8Ghgz=^wzQtQ^GtG9ERj>g^jTG9‚c=rb9=8ZFQ^GrCF=@Ofg€UMlERj^O7zC[GtB:j^?Rj9Q^GhgiGlEcjBR=8Z^j>g8giwc=sBTG9ERj~gFj>Zl=@O:mrGc=eGlERj{bFQ>g|G~jsO:G8B9A„CPO{QK7rZl=@O8GrC[O9ufCPO:DšCPO„b:C[]RjiohQsO9ORjFBTu9O8GfCFb{C[GrB>j9Q>Z[ERjRgeG9ERjiŠcjFDhCPO9OpCPO:D„=swzQU797978797979797879797979787979797978797979797879787978797879787879797878797978787979787879797:7979797:7979797:7979797:7979797:79797:7979797:7l6kc=:C|m}?RjlB^o8u8G>j“wcb9=FQ^G)Q_K/C[O8GTO8ulop—CPO8GšCKNY›wR=sB“+C|^O8uloŽCsŸhJ'Cs‹9‹R—Yt“9“9“wRb9=FQ^Gf—@B:QsO:mc=[oV“+—F6sd8`9¢lˆ>^W>`9h—97:C9—|fJ'g@‚>Q^?“QKCs‹rJLKY9‹hJF—696@?>BhCPO8G8w“[…sg@‚>Qs?9?pCPO:D~9m~QsO:mr9mR@Ox…fKCKYc—687978797979797879797979787979797978797979797879787978797879787879797878797978787979787879797:7979797:7979797:7979797:7979797:77rg@‚>Qs?šg@‚>Qs?fgvGlEcj}kRQ:b@uRj>gzCPO)QFB9B:QlA~QtQ^Gƒb9=8Z^Q^GfCF=@Orgzb9=:Z>JvQ^O:m{b8=8Z^HU797978797979797879797979787979797978797979797879787978797879787879797878797978787979787879797:7979797:7979797:7979797:797979786kc=:C|m)g@‚RQs?“wRb9=FQFG{Q_K/CPO8Gƒb9=8Z>JLKCPO:G)b9=:Z^HR—wRb9=FQ^GrGFo8?ŽGFo8?{tQb8=8Z>JPQb9=8ZRJP~tQb8=8Z^HFQb9=8ZFHF~
PAGE 273

AppendixTRawDataTable22.StudentResponsesonAchievementMeasures PosttestRetentionTest IdaTbPretestMCEssayMCEssay key311221443113433113413112332442222414321434422111111111111313412412111111111111100cR41321343211321331231411431244223243132132413110011110011133423213211010 0101101 101cC413414424113433114411112332442232414321434432111111101111311112412101001001110102eA423333333123123331412123113323142333424123000000000000000132141142000000000000103A413423324141433431412114332244233122314431122101101000000211222132000000000000104cC313412434142433214313142331243134132124144432111111 100111332411442101101001111105R424414412341423311413111322442224412323334432101111111111414122432101101111111106ceR413214443121433210310113342442222434124431234101000100000313412412000000000000107dA433221114 334142332100101101101108R133212244342133231241112312342114411114434433100100000001334222132101100000001109R413213234142433413413112332442224414324114422101111111101314112222101101001101110C214234434141423213413122332443211324124111442101101100001334422112101101101000111A423221321142333114422121312313244113111141422111100000001232112012111101000000112C323242134213113213443311314441114411322444220101101100100332412432101100001110113R433214344341413332412111312423434433124121334000000000000 115C343212432414113323313114312441133434324421442101101101011311432212101101001101116dR134214231141133213413111312412124222321331334100001100101434443332100101100101117cR1334142243434132124221113124432344341214444321101011010 11134412312101101101100118A423413121333133333114121343321121334124431232000000000000412241112000000000000119C433211134 120hA434214231 121A423211142144223113412141312242124432323421432111101000111114111432101100001101122dlA424211434243133433113143322342213231124111211000000100001434422432001101001001123A231213223431413213423312312243223134321431444001000100111313212412000100001111124deiA413414114341133212414112332441241412124431432101111100001412143434000000000000125A433411134142413111311112132442121114321434132111101100111414512432101101100111126R331211432142433312310112332442221214321434432110111111111313431412101111100111127C31 211324313313412111311342441234434121321432111101000101111212232111100000111128C243211424241113214413112342441214124324431432111111100101414212432101111100101129dR333214234342132213412133311323423314424441332000000000000234242312000001000000130C233231134122243313423122342334233132324422432100101000000434412132100101000101 (tablecontinues)258

PAGE 274

AppendixT(Continued)Table22.(continued) PosttestRetentionTest IdaTbPretestMCEssayMCEssay key311221443113433113413112332442222414321434422111111111111313412412111111111111131dC223214114123133113444141312443424134314131331110101000101111141132011100000000132cR431421314113433113213112332442223414111434422111111111111313112432100111011111133A414411414142123112112141111321113241221332413000000000000423231212000000000000134C323211414413123213412132342241224132324421332101110010100231442132101101000000137R342214422421413112413112342444213111122441432110101000000334212432101100000000138cR3222241441431332134131131124432344143141214421111001011013344124121011010010 01 139eR233414123142133213312112342412214432144134422101101101001311312314000000000000140A413211234123433133423112312323421234121431232000101000000432111312000000000000141C413211234114413432414111342423224312122411331000000000000324422132100101000000142R433214431223433212412111312143124434324443341111111100001434212232101111000000143R413213414313143314314123331344144134312431424100100000000431122312001101001000144A343213234242233224422211322441124131132433434101000100111321421432100100100000145R433413411223133030312112311423124132124431132000000000000434431332000000000000146cR413214244224133213422112112341214333124421132000000000000444211332000000000000147R143211424243433212412111341342234132121134233100100000000344432232100000000000148A 113133313323112332442431414324434412101101101111311412112101101101110149C413211434343133214413112332442223414121431432111111111111313112332101111001111150R 113134114412141333321213111344131412100100000001334212132001000001100151eA323443232142443210412112343412214432124431322101100100000432112322000000000000152C213214232141133213423123142341121334311241323101101100001414411332101001001000153C434211411341223113313131312212113344124421333010100000101434233142110100100101154C323243223 155A214211234144242212112122232323224321331334333000000000000434111132000000000000156C333224123 157cA4232144311312333134142133114421343314242411341011111 00000233212332101100100000159C433214124223433212412111312443124434324443341111101100001314441342101100000100160A434211422322133234312232323431122323144231323000000000000444214232000000000000161R233213412 162cR412233434142333213424111142233132234424431131000000000000431312000000001001000163A433212144422133214312111322441212434311323122100101100000424321432101100100000 (tablecontinues)259

PAGE 275

AppendixT(Continued)Table22.(continued) PosttestRetentionTest IdaTbPretestMCEssayMCEssay key311221443113433113413112332442222414321434422111111111111313412412111111111111164A233213232241113213413112332441324424314441434111101110011314211332111101001000165R414211134112333212113111312241424434124444322001001100000434242332001100001000166C133212434133433113313112342441222414321334412110111000111331212332110101001111167A 443143212314131341312234144114424433101001001001324331134101101000101168R423211432 443111222000000000000169R113214224 343441234000100000000170A 43244132 171C133415531 172A111213412223133112313111132423133134324224341101100100111214241132101000001111173jC213214244222333212413112332453123314124134442101100100000212422234000001000000174C414213114343133412313111312443224114124121431101100100001432212332111101000000175C433214134144243212413144312422123224414321332100001000000231220132000101001001176C313214234434143213343214312423124234324123431000000000000434122132000101000000177A11323 414433213413112332442221414324434442111111111111331212332111111011111178R223214134221423132112113312441431204114314422110000010000134242132101000000000179R433214324421413112413112342442213111122441432110100000000245232312000000000000180dC312221311142333112313111322441243434311331422110100000100431211132111100000000181A433211222 182eA232213223 441231232000000000000183cC134214132313433314313112312442223114324434422111111111111313412412101111111111184A323212331233133213412113332342244432114431421101101101001111331432100100000000185C133211434342413213114111312142123214324411333111101000001214212432011101001100186C134414434313433313413112132442223114324444422101111111111211132212101101101101187A113311132333333333333333333333333333333333333000000000000232111231000000000000188A424223432423443213121143112243241134124341442101101000000332412112001100001001190A1322114313422341134 3113332341234434114431422101000100001234411332101101100000191dC334214412122421213324111312242124122221441231000000000000114131132000001001000192R433223132123413114412112212442231414311434324001111100111311142132001110001111193A413212313142413213443142332343144134211121432110101000000132412432101101001000194R232211223442443232412312312223224324344341423001100000000334211332000100000001 (tablecontinues)260

PAGE 276

AppendixT(Continued)Table22.(continued) PosttestRetentionTest IdaTbPretestMCEssayMCEssay key311221443113433113413112332442222414321434422111111111111313412412111111111111195C413214133123313212414121112323211222324121344000000000000431122342000100000000196C423423122211433113312311141342424134111434432101110110111313432412101001100111197A433314124441123312411343322323221334124423333101001000000334241342001000001000198cR313212445142432213312322314221234134322421132011101100000133241234001100000000199dR123214233 134242332101000000000200R133211422 333241242000000000000201dR433214234312433311311112332441224412341444232101001101111414232332101101001111202C323123224243413212111112312443214414321431311000000000000322122131000000000000203dA232233214 413112432101101011111204R223414114122123414323111322333234221114411333010100000000434111134000000000000205cdC212211432144133212422114332323241334121414432101000000000334421132001100001000206R413214314343131214411111312443134134122431132011100000001424431132100100000001207dC223411422433433113413112122441444231311434412111101100101334412432101100100100208A233211224302133122422111322442223412324434322111101100101314112412101101001101209C223221214213433213414211112442124134124411132000100000000312112122000101000000210A433214334333333313322212222223421134124431232000000000000434411424000000000000211dR113212422 212hA434212434 213dA223211434122133313213122312321132324224131311000000000000234111432000000000000214C333212424132433314311112332441123134343441412111001000101311431432111001101111215R313224424342113211121112312422144314124121433001000000000434111212000000000000216R413223332242433113413111312442222414311434422111110111111411412112111111001001217R423413242141133212313111312412124222321331334000000000000532141434000000000000218dhC233413232 219eR113213144 234121344000000000000220hA433113432 221fA443214443142113212324112332342223434321431334100111100101214212332101101101000222fA133211432 223R423214431243133212411121312221224313321131412010100000000435112232010100001000224dfA232213244243153413414314322343144133234332332110101110000433222322101101000001 (tablecontinues)261

PAGE 277

AppendixT(Continued)Table22.(continued) PosttestRetentionTest IdaTbPretestMCEssayMCEssay key311221443113433113413112332442222414321434422111111111111313412412111111111111225A132222414141423113313112332443222424341434432100101100111344422412000101001111226C 321 434441332000000000000227C134413324123131111113141311242224132414321432000100000000332122112000101101000228R233214314222233414413111351443124421121131441111101100011334212432101101000001229dC213411422311133132312113332442213111114434234100111111101343212312101111100101230R213211141143413213413112232441141414324431432111101000101314412332001101001000231C231314132121143214323111322243223134311111432101101100100414322134000000000000232hR311411432 233ckmA2332141443434133134131313323132244121442114421011000001 013342321321001000001 10234hC224214421 235hnR413212243 236A133224432 431142332000000000000237R342215532 334431342000000000000238C233211434414413313442111332443124113224222113100000110001212113132101000000000239C331214214123113314413312332322124413121431134101101000101322432132101001001101240R323234214 334323132000000000000242A323234432323123212443112342343124434124344332000000000000324331132101100000000243R434211134441433112412141342223234133314111332001000100101134111232001100001000244R313412234312413213313112332412221434324434422101110111111314412432110101011111245C413214413421431213412311311243123332314431332100000000000232441234000100000000247A313414234341133413312121312131121132131414332000000000000234241112000101000001248eA413214234341423213313212321443124434124134443000000000000334122314000000000000250R113214124133131114413143442241423134314431412110100000000331112332000001000000251R413211114122131112443131412313444334114321313000000000000234111342000000000000252deR314211412342122331442123324232112343421232133000000000000433141132000000000000253dC413222434143133413342111334441122214321433432100101100001434112422101101000000254eA424234113332113213412141132441123414123424422111111001000314122334000000000000255A423214312342333113414112112342124434123441434101001100111311411112101001101000256C134223231313433113413112332442222414321434422111111111111414212432101111100111257C233411144432133213413111312442234112314242432100001100111233112132100101000000 (tablecontinues)262

PAGE 278

AppendixT(Continued)Table22.(continued) PosttestRetentionTest IdaTbPretestMCEssayMCEssay key311221443113433113413112332442222414321434422111111111111313412412111111111111258C313214434142133113413112332442421414324434432111111101111343412432110110100111259R124212434143133114412111332442344314314411242111101101111314122131001101001101260dR333211432 261R333221344413433113413112332442222414321434432111110111111313412412111111101111262A413214414311133212421142312343424134123433432101101100000331421122101101100000263oC113211132413433114413112332442224414123331432111111100111313132312101101101110264A413214434241233234113312412342141224123131243000100000000314431342001100000000265dC413214112 434141122000000000000266C423214443223113411312111342343234231124131232001000100000432112112101000101000267R233221444232223413114113312441212133312441113100100100010213131132100101000000268A134212211334423114413344412323221322124334221000000000000334123314000000000000269R431214441113433113414111132442221434311334422101101111111311112112101101000110270A413234234112133313413211342442224414324431432111100100111111432434101100101111271R324414433414433213413112332442223414321434422101101111111314412412101111011011272A233214224342323214413323113443234133212231141100000000000 274C113414244242133113234112312342124414324434122101100000101313312432101100001000275A412231414243133223414111343313114423124431232100000000000334442132000100000000276hA423214231 278hC233214324 279R123213433313133413313112332442224414321434422101111111111011511312101101011111280C413233131143233232412141212321121134124123243000100000101424141332000100001000281A413233421123131213412111312343124334123441343000000000000432431444000000000000282R413213134142423211413221332441124424124434433101111100111334212422101101001101283cA432111433242124313422113112443113134114431333000101001000 284C233211424342133113413141312312223312124441422101101000101333112114100101100001285C213214444213433111413113332412324432124331431110100000101231412432100000000100286R123213434331123313113243311323124122134131311000000000000234442332000000000000287C433214334243213312414311312443214412121441423100100100101214432432100100001001288A232231442112421111341113342442421434321333123111001101111411432132000000000000289hR 23212232 (tablecontinues)263

PAGE 279

AppendixT(Continued)Table22.(continued) PosttestRetentionTest IdaTbPretestMCEssayMCEssay key311221443113433113413112332442222414321434422111111111111313412412111111111111290C313224432442332413412212132323122133144241323100000000000332431444100000000000291R433414231241132212313112323343434234324131332111111101001234441232001100000000292gC234114434321423212411112332442434223424441432101100000000314132331000000000000293gR432211414 234411132000000000000294gmA413214314 Note.Dashesindicatemissingdata.aStudentidenticationnumber.bTreatment.cEssaygradedtwiceformeasureofintraraterreliability.Underlineditemsindicatethosewhichwererateddifferently.dTookpretesttwodayslaterwithEdTech'sExam1.Becausedelaywasrelativelyshort,responseswerekept.eRetentionessayactuallymissing.Severalessayshadnoname,sozeroswereenteredforstudentswhohadmultiplechoicedata.fReceivedcorruptcoursewareCD-ROMs,sodataremoved.gLateaddsnotoninitialrollwhowererandomlyassignedtotreatmentslaterandreceivedCD-ROMsbyFebruary24.hRemovedfromformalanalysis,becauseofciallydroppedcoursebyweekthreeofveweektreatmentinterval.Thesestudentsweremissingtheposttestandretentiontestandassignmentdatawereincomplete.iTookposttestafterretentiontest,soposttesttreatedasmissingdata.jTookposttestfollowingdayandretentiontestfourdayslaterinStudentDisabilityServiceslocation.Becausedelaywasrelativelyshort,responseswerekept.kAccidentallyskippedposttestitem36andmarked37asresponse2,soa2wasrecordedfor36.lPunctuationinnamecausedproblemswithonlinesubmissionofwork,sodatawereremoved.mNotonApril25classroll.nPretestitems8,9,10,and11shifteddown,socorrectederrorbyshiftingbackup.oFixedincompleteerasureforitem8onpretest.264

PAGE 280

AppendixT(Continued)Table23.AssignmentRawData InitialGradecBonus/PenaltydFinalGradeeProcrastinationf IdaTb12345678123456781234567812345678Rg 100R443832 000000 4438320045-1-1121311101C443843 -4-100-1-1 0338320043-1-13311102A 0000000029292222151511103A442 -4-40 0020000055022151511104C343844440000000034384444000-100-1-1105R44374444000000004437444400-2-1-10-12-11106R333 33 000 -3-3 3330000011022151511107A4418 -4-400 0018000011-6-2151511108R443823 000000 443823000000-1011109R242 33 3000 00 024203303-1-11722111110110C4407 -4-40-7 0000000077441515113-4111A43 00 43000000-1-12222151511112C4447 -4-4-4-7 000000001214781515111-6113R4431 0000 4431000000021151511115C44472013-1-100-10003347101322001100116R444723 30000-20 04447030355021151410117R443 000 4430000000022151511118A444 -4-4-4 00000000111113221515111-4119C141 -1-4-1 0000000067322151511120iA4438 0000 44380000-10-3-2151511121A101500000000000010150000-1-1-100000122hA44384333-4-4-3-8-4-3-3-30000000066221515114123A443 333300-3 00004400333300322-1-100124A4438 330000 004438003300001515-1-2125A44383333-4-40000000038333344-10-1-2-1-1126R043744340000000004374434992444-10127C043833 4000000 00438330400-200010128C33481213001-1-10013357021400-43100-4129R 00000000292922221515114130C1027 2 -1000 0 0027020033-1-115-111131C443823130000-1-1004438121300-2-23300132R333743440000000033374344-15-2-1-13-1-1 (tablecontinues)265

PAGE 281

AppendixT(Continued)Table23.(continued) InitialGradecBonus/PenaltydFinalGradeeProcrastinationf IdaTb12345678123456781234567812345678Rg 133A44 00 44000000002222151511134C343 0-1-1 3320000003322151511137R41382 00000 413820000004141511138R4448 0000 4448000011241717151511139R44384403000000004438440300-14-1-100140A 0000000029292222151511141C44362 -4-40-1-1 00351000440131511142R2424 2 0000 0 24240200-1-1-3-115-111143R442512 000000 442512000002141411144A44472301000000004447230100000000145R444 000 4440000056-122151511146R 0000000029292222151511147R0435 0000 0435000006521151511148A444734040000-3-4004447000400001414005-6149C4447 34000-7 004440003400-221151500150R342 000 34200000002122151511151A4437 000-7 4430000000071515113-4152C3435 -3-40-1 003400006602151511153C2421 1 -2-400 0 00210100570015011154C444 -1-4-4 30000000310422151511155A 0000000029292222151511156C00 00 00000000332222151511157A343711130000000034371113-1000-100-1159C3425 3 -3-400 0 0025030044-3015-1111-2160A44 00 44000000-1-12222151511161R442723 000000 44272300558103311162R 0000000029292222151511163A4426 0000 4426000000-10151511164A344533230000-3-3003445002300-10111100165R44362 00000 4436200017252121141511166C443544130-110000143454414-11-4-1-1-1-3-4167A44361 0000-1 443600000000141511 (tablecontinues)266

PAGE 282

AppendixT(Continued)Table23.(continued) InitialGradecBonus/PenaltydFinalGradeeProcrastinationf IdaTb12345678123456781234567812345678Rg 168R44421303000000004442130311112121141400169R140800000000000014080000262619191212-2-2170A 0000000029292222151511171C4435 -4-4-3-5 0000000012121021151511172A 0000000029292222151511173C4447 -1-10-7 3340000033041515117-8174C443714440000-1-40144370045-1-10044-2-45-6175C44374404-1-100-4-4003337000433001212-2-25-6176C4436 0010 44460000-1-1-4-2151511177A444844440000000044484444000000-1-1178R0004 0000 000400007700151511179R44182 0000-2 441800000066151511180C443844340000-4-40044380034-1-10044-2-1181A444 -4-40 0040000066-122151511182A4448 00-4-8 44000000-1-12020151511183C444744440010000044574444-1-1-4-1-3-30-2184A444743230000000044474323-1-1-3-1-1-1-1-1185C44484444000000004448444400-1000-1-1186C444744440000000-444474440-10-100001187A24482 -2-4-4-8-2 0000000028282121141511188A44 00 44000000002222151511190A0447041 0-40-70-40 00400010770771201191C44372 1-4-4-1-10 0002620011010330151-2192R443834140000-3-4-1-44438000027272020151511193A4447 -4-40-7 004000005606151511194R4448 0000 4448000067020151511195C 44 -4-1 00300000294322151511196C44484444001-11-10044575344-2-1-41-41-10197A44472 -4-400-2 004700007700141511198R4426444400000000442644440000111100199R4436 0 0000 0 44360000131366151311200R44 00 44000000002222151511 (tablecontinues)267

PAGE 283

AppendixT(Continued)Table23.(continued) InitialGradecBonus/PenaltydFinalGradeeProcrastinationf IdaTb12345678123456781234567812345678Rg 201R44483434000000004448343400-3-1111000202C443134 -4-4-3-1-3-4 000000001111422151511203A44484444-4-40000000048444444-10-2-1-1-1204R 0000000029292222151511205C444 12 -4-4-4 00 000012001212522-1-111206R 0000000029292222151511207C442713 -4-4-1-1-1-3 00160000552351411208A41172323-4-10-70-3000010202355-230300209C440712010010100044172201-1-1-5-2-4-2-3-2210A24482 -2-4-4-8-2 0000000028282121141511211R 0000000029292222151511212iA3438 3 0000 0 34380300-10-3-215-211213A341 -3-40 0010000046-122151511214C00004444000000000000444400-2-2-1-1-1-2215R444 000 4440000055422151511216R4448440400000000444844046633141400217R 44823 4 000-20 0044803042911421151410218iC24 -2-4 0000000010102222151511219R 0000000029292222151511220iA44 00 44000000-1-12222151511221hA444814 4-4-40000 00048140477000010222hA4437334 -4-4-3-7000 00003340121255-2-201223R4447 0000 44470000112021151511224hA 0000000029292222151511225A444844340000-4-400444800340000141400226C 0000000029292222151511227C4448 000-8 4440000000-214151511228R100. 000. 1000000025251829151511229C4433330 0000000 4433330000-100001230R1038243 0000000 10382430002121141401231C3438 0-400 30380000-16-10151511232iR443 000 44300000-1-1-222151511 (tablecontinues)268

PAGE 284

AppendixT(Continued)Table23.(continued) InitialGradecBonus/PenaltydFinalGradeeProcrastinationf IdaTb12345678123456781234567812345678Rg 233A343 3-3-40 -30030000055022151511234iC 0000000029292222151511235iR44 00 44000000002222151511236A 0000000029292222151511237R 0000000029292222151511238C44472323000000004447232300000000239C44281334-4-40000000028133466-1-1-2-2-2-2240R444844 30000-40 04448040377021151410242A34364 000-6-4 3430000000021151511243R444704 0000000 0444704004401141410244R4448444400000000444844440000-1-1-1-1245C 0000000029292222151511247A444 000 44400000-1-1022151511248A442 3000 044200003-1-1-122151510250R 0000000029292222151511251R 0000000029292222151511252R 0000000029292222151511253C4447000 -4-400000 0047000044-1-1111101254A44483344000000004448334400-1-10000255A443703 4000-700 0443003040005-101-2256C444844440010101144585455-20-4-1-8-3-14-14257C44373334-1-10000003337333411-20-1-10-1258C3 483434-3 -1-1-1-1-310037230552911331-4259R4447 4 30000 0 0444704037112121151410260R 0000000029292222151511261R444724240000000044472424-1-1-2-2-1-3-15-15262A444744340000000044474434-1-1-1000-1-1263C44484444000000004448444400-1000-30264A44 200 044000002002222151510265C444723 2-4-4-4-7-2-3 0000000021111421151410266C4447 -4-4-1-1 003600006633151511267R44483333000000004448333300000000 (tablecontinues)269

PAGE 285

AppendixT(Continued)Table23.(continued) InitialGradecBonus/PenaltydFinalGradeeProcrastinationf IdaTb12345678123456781234567812345678Rg 268A 0000000029292222151511269R44474444000000004447444433-1006-8-1270A44482203-4-40000000048220333000000271R44484444000000004448444428282121141400272A333 2 -3-3-3 -2 00000000110322141511274C444744430000-1-11144473354000012-4-4275A44403404-4-4-40-3-40000000004662121141400276iA4 -4 000000007292222151511278iC442 -4-40 0020000066-122151511279R44483443000000004448344355-2011-110280C 0000000029292222151511281A4347 3 0000 -3 43470000000015111282R443842 00000-2 4438400016162021141511283A 0000000029292222151511284C443722 1-4-40000 10037220266-1-1-1-11-4285C4448431300000-1004448421300000200286R443813 000000 44381300001718141211287C4426140400-1-1-1-41144150015-1-11244-4-4288A343602 -3-40000 0036020044-1-1-1-1111-2289iR 0000000029292222151511290C 0000000029292222151511291R334722 3000000 03347220300342210292C4438344400-1-10000442734440033-2-2-2-2293R 00000000292922221515113-6294A 0000000029292222151511 Note.Dashesindicatemissingdata.aStudentidenticationnumber.bTreatment.cGradebeforeapplicationofbonus/penalty.dAmountaddedto(deductedfrom)gradeforearly(late)submissionsasdeterminedbytreatment.Thedayafterthetreatmentintervalendedwasusedinthosecaseswhentheassignmentwasneversubmitted.eGradeafterapplicationofbonus/penalty.fRelativelevelofprocrastinationasdeterminedbysubtractingtheduedayfromthesubmissionday.gAssignmentsforwhichstudentrequestedadeadlineextension.hRemovedfromstudyduetotechnicaldifculties.iRemovedfromformalanalysis,becauseofciallydroppedcoursebyweekthreeofveweektreatmentinterval.Thesestudentsweremissingtheposttestandretentiontestandassignmentdatawereincomplete.270

PAGE 286

AppendixT(Continued)Table24.ProleandPreferenceRawData SGdPrioreCollaborationfPreferencei IdaTbTAcBA1212345E1gFinalh12345678BestjLeastk 100R11.002.00022412653871242511231,2,3,14101C11.001.0002163634121221222282,3,14,15,16102A13.003.00803506453211423513113103A11.001.0001134633752442551163,6104C14.003.00401352614121413114138,15105R11.001.00021715724454413232445,13106sR11.001.00022613511695312111543107A11.00 1.34 60340 108R11.001.008052510503601242215223,14109R11211.001.00401345654005451551443110C11.001.0004517573741223424213,5,6,11111A11.001.0001224633992243542211,2,3,5,6,15112C11.001.00021410553692215324213,8,10113R11.001.0001143441521244212483,13115C1151.001.00024612553971211511235,6,15116R152.001.00201355393093234333333117sR11.001.0001182533551442444245,7118A11.001.0004306.5613781221521512,6,7119C11.001.00 54137 120qA11.001.00 57180 121A172.002.0002469473641211411133,5,9122lA11.001.00202444503901241542131,5123sA11.001.0002386633982241412283,12,14124tA11.001.002032610543651213422436,13125A11.001.00034615584161411111111,6126R13.001.00012735531232224214311127C12.003.0002355463221241441115,6128C12.001.002012815573931232442132,3,5129sR12.002.00022135533212515511310130C14.002.00 1556543571451111212,3,5,9,11131C13.001.00033815413301445133333,6,11,13132R11213.003.00201378694274223233448,15 (tablecontinues)271

PAGE 287

AppendixT(Continued)Table24.(continued) SGdPrioreCollaborationfPreferencei IdaTbTAcBA1212345E1gFinalh12345678BestjLeastk 133A11.001.0002326593141411111112,5,6,14134C11.001.0002506633631351542113,5,7,8,10137R1311.001.0040115556391144444114,95138R132.003.0001143674241421542232,3,5,6,8,10,14139R11.001.0002454584002243444233,5140A11.001.00402204593491253553113,11,12141C11.001.00 54360124442214,96,7142sR23.002.00502365492562221423246,11,14143R22.001.00043615483474451423133,11,17144A2103.00 1.34 803441357378124441112,33,11145R21.001.0002133583341212411245,6,10,17146R21.001.001001302593313423313411,6,8147R21.001.00202333503541422121253,6,12148A2 1.50 1.34 2032612524072224422243,13149C21.001.00404334613851422244336150R23 1.50 1.34 203338433381241524173,5151A21.001.0002234503631251334483,6152C21.001.0002142562703222534233153sC2321.001.002012612472821111512113,13,17154C2134.002.00 61375 155A22.001.0002101322961224523113,5,6,13156C2152.001.00 3990 157A28231.00 1.34 8022315464061441211145,6,12,13,16159C21.001.00404214423481231353216160A283.003.0001120553622442442215,17161R21.001.00 49142 162R21.001.0001202462092452341246,8163A2101.00 1.34 201136483711245442113,5,6,15164A28231.001.008033315523721415324233,8165R2192.00 1.34 02255553821211511183,17166C22.001.00021715563912433221141,3167A2922 1.50 1.34 801255393304423222443 (tablecontinues)272

PAGE 288

AppendixT(Continued)Table24.(continued) SGdPrioreCollaborationfPreferencei IdaTbTAcBA1212345E1gFinalh12345678BestjLeastk 168R22.002.00 36209 169R21.001.00 63325 170A21.00 1.34 4790 171C21.001.00 47234 172A22.001.00011005736614414421 173C21.001.000414154936644154131314,16174uC2141.001.0010024012503791441244216,13175C214261.002.0010014011443431441344116,11,15176C21.001.0001346553461323553133,6177A21.001.0001 415594322221444343,5178R21.001.00041410473102221544133,6,15179R3311.001.00601144413554424312413,17180C314261.001.001001415473711411514113,6181A33.001.00 49135 182A33.001.00 37318 183C316251.002.00603143.566436244424144,83,5,14,15184A31.001.0004227503881411211213,14,17185C3181.001.002041815453902131413433,5,13,14186C316251.001.0080244560428144524252,33,7,8,9,14187A3301.001.00402233.503301453551215188A31.001.0002123573971231443131,3,5,6,8,16,17190sA32.001.000118547323123351333,83,13,14191sC32.001.00023812483162224122511,3,6,8,14,17192R34.004.0001278643504224515317,12,14193A371.002.00021310503801421522213,6,15194R341.001.00801424382441323324125195sC31.001.0001274503741422112425,6196C31.001.000118168432422134144,83,8197A39221.001.001001306322831443412193,7198R31.001.006024310313662443443351,5,11,17199R32.003.00 46336 200R31.001.00 49202 (tablecontinues)273

PAGE 289

AppendixT(Continued)Table24.(continued) SGdPrioreCollaborationfPreferencei IdaTbTAcBA1212345E1gFinalh12345678BestjLeastk 201R31.001.0080326654396222553222,36,13202C31.001.00201365643724522113443203A32.001.00 57408 204sR32.00 1.34 402212633671221442116,15,16205C32.005.000453849356242431443,91,8,10,14206R3333.001.0020110 543291251552113,14207C3131.001.002022615683982425442 3 25,7,13208A32.00 1.34 02288563981442342113,5209C31.00 1.34 02188443682234514215210A3301.001.001002204523551421444113,6,8,11,17211R32.002.00 53130 212qA31.001.00 56173 213A31.001.000233640361421212233,63214C3324.003.002052815554101213311111,3,4,7,15215R31.001.0002324573801244442113,6216R341.001.0001284604095155551543217R31.003.001001202.545282141555511 218rC31.001.00 35119 219R331.001.00 62328 220mA42.004.00 62169 221lA41.001.00802119533821443554136,8,9,14222lA44.003.00 60215 223R4192.001.0001146533761211511243,6,13,17224lA44.00 1.34 0442103521251554114,6,8,15,17225A411271.001.008023313734422411412411,3,5,6,7,11,15226C41.001.00 62221 227C41.00 1.34 02342663761212551113,6,16228sR41.001.00031515554001111141233,6,13,14229C417241.001.002021710583813214342343230R431.001.000151.511553861241411213,5,13231sC4134.005.004014586238045255224111,12232qR421.001.00 63166 (tablecontinues)274

PAGE 290

AppendixT(Continued)Table24.(continued) SGdPrioreCollaborationfPreferencei IdaTbTAcBA1212345E1gFinalh12345678BestjLeastk 233A41.001.0001321003643223322431,4,14234oC41.001.00 4698 235qR421.001.00 53119 236A41.001.00 67208 237R431.001.00 54286 238C417242.002.006024310453751251522143,5,8,13239C42.001.00402347.5463981243542283,5,6,12,13240R453.004.00 44322 242sA42.002.0002368482232223323214243R41.001.000217403611211544143,7,15244R41.002.000228366440124224124,715245sC42.001.0004218573222423242334,5247sA481.001.0001241.5523721254554135,7,12,15248A412281.001.00801432613784443242463,7,14250R4331.001.00201101573621212512118,10251R461.002.00 021225423132 252sR461.003.0001374312754244443315253C41.001.00602136573692223322241,6,8,13254A4101.001.0001517624031255542145,11255A41.001.006023215463301211512113,5,13256C41.002.0002284694494522221534,8,14257C5181.001.008045110523721411412135,6,17258C51.001.0004338694211251541232,3,4,7259R51.001.0003161551366244145243,41,3,8,14260R51.001.00 4098 261R53.002.000226351398454224143,413,15262A51.001.0001284554004324242523,14263C52.001.00202174594072321242233264A512281.001.00403322713823224534413,6265C55.004.00 39245 266sC51.001.0001155593594212312443,6,8,13267R53.001.008044410513871251242133,6 (tablecontinues)275

PAGE 291

AppendixT(Continued)Table24.(continued) SGdPrioreCollaborationfPreferencei IdaTbTAcBA1212345E1gFinalh12345678BestjLeastk 268A51.001.0020110149325342353333,411269R51.001.00021715704295515111111,3270A51.001.00024210.5523964333243455,14271R51.001.00011815614285224211548,17272sA53.002.002015710361305415114523,11274C51.001.000118566422222442222,32,3,7,15275A511271.001.00100 304643312234421415,7,8276pA53.001.00 47139 278qC51.001.00 50136 279R52.001.000217563404125211423,63,6280C51.00 1.34 0210403172222412235,6,13281A51.001.00015025438324411434411,12282R51.001.000116356403242242223,43,14283A51.001.0001200511532422212245,8284C51.001.0003156653911243445133,5,13285C54.003.00012855238612242441 286R533.001.0002324623762241414233,6287C514262.001.008011315423521231521516,14288A51.001.00205108654251351523113,6,7,17289nR51.001.00 3081 290C52.001.000430303081421551115,6,7,16291R52.002.00402137613921412412533,17292C11.001.00015210493801411411131,3,6,14 (tablecontinues)276

PAGE 292

AppendixT(Continued)Table24.(continued) SGdPrioreCollaborationfPreferencei IdaTbTAcBA1212345E1gFinalh12345678BestjLeastk 293R12.001.00 43139 294A11.001.00 5487 Note.Dashesindicatemissingdata.Valuesinboxesindicatemissingdataforwhichvalueswereestimatedinwholeorinpart.aStudentidenticationnumber.bTreatment.cTeachingassistant.dStudygroup.Students,priortothetreatmentinterval,anticipatedbeinginthegroupslistedincolumnB.Afterthetreatmentinterval,theyreportedthattheywereactuallymembersofthegroupslistedincolumnA.eSelf-reportofpriorexperiencewithdocumentformattingandprogramming/authoringlanguages(seeAppendixF).fSelf-reportofcollaborationlevelonHTMLassignments(seeAppendixP).gScoresonEdTech'sExam1withvaluesintherange[0,73].hFinalclassstandinginEdTechwithvaluesintherange[0,449].iSelf-reportofpacingpreference(seeAppendixK).jReportofwhatwaslikedbestaboutthelessonsandassignments(seeAppendicesPandQ).kReportofwhatwaslikedleastaboutthelessonsandassignments(seeAppendicesPandQ).lRemovedfromstudyduetotechnicaldifculties.mrRemovedfromformalanalysis,becauseofciallydroppedcoursebyweekthreeofveweektreatmentinterval.Thesestudentsweremissingtheposttestandretentiontestandassignmentdatawereincomplete.EachdroppedoneitherMarch2m,3n,7o,8p,9q,or10r.sReportedcompletingalargernumberofassignmentsalonethanactuallysubmittedforgrading.tTookpacingpreferencesurveyafterretentiontest,sopreferencedatatreatedasmissingdata.uNotidentiedingroup14priortostudy,butidentiedafterwardsandincludedingroup26.Studentwasincludedingroup14foranalysisofpreferencedataonly.277

PAGE 293

AppendixT(Continued)Table25.ScoresforIndividualsandStudyGroups AchievementAssignmentsClassPointsProcrastination IdaTbGcPrePostRetDiffsNdNBPeBPfBeforegEhFinaliLatejReqkPrefl 100R33.3361.9047.6214.296.0066.6766.6738.0010.0059.004.252.50101C22.2290.4861.9028.576.0072.2252.7840.5010.0063.501.622.75102A11.119.524.764.760.000.000.004.0010.0033.0016.752.75103A22.2240.4814.2926.193.0027.785.5613.0010.0042.008.002.25104C33.3366.6761.904.768.0094.4494.4447.0010.0070.00-0.383.25105R11.1179.7666.6713.108.0094.4494.4451.0010.0070.00-3.383.88106mR55.5639.2942.86-3.575.0041.6725.0021.0010.0042.007.004.38107A33.33 59.9 47.6212.284.0047.2225.009.005.0033.003.25 2.79 108R22.2232.1433.33-1.196.0066.6766.6733.0010.0054.000.122.38109R122.2282.1457.1425.006.0047.2247.2236.5010.0057.507.502.75110C33.3352.3852.380.004.0041.670.0013.0010.0036.006.752.002.50111A33.3341.6742.86-1.192.0019.4419.4417.0010.0046.009.252.25112C22.2250.0057.14-7.144.0052.780.0012.0010.0035.009.126.003.12113R22.2213.10 16.13 -3.034.0033.3333.3317.5010.0038.506.623.25115C1533.3359.5261.90-2.388.0069.4461.1135.0010.0058.000.752.75116R511.1146.4338.108.337.0075.0069.4437.0010.0058.007.623.00117mR0.0061.9057.144.763.0030.5630.5625.0010.0046.006.752.12118A0.0015.4814.291.193.0033.330.006.5010.0035.5011.124.002.88121A733.3360.7147.6213.108.0019.4419.4420.5010.0049.50-0.382.75123mA33.3347.6261.90-14.297.0063.8955.5632.5010.0061.502.882.50124nA11.11 19.63 9.5210.116.0069.4469.4425.005.0049.003.38 2.79 125A11.1171.4361.909.528.0086.1163.8939.5010.0068.500.253.38126R55.5684.5276.198.338.0080.5680.5648.0010.0069.003.883.25127C44.4453.5752.381.197.0069.4469.4437.0010.0060.00-0.122.00128C33.3366.6761.904.768.0069.4469.4439.5010.0062.50-0.502.12129mR22.2215.4819.05-3.570.000.000.006.5010.0027.5016.751.001.62130C22.2233.3342.86-9.525.0033.3330.5620.5010.0043.502.503.00131C11.1148.8123.8125.008.0077.7872.2237.5010.0060.500.253.25132R133.3395.2476.1919.058.0086.1186.1153.0010.0070.000.123.38133A33.3311.9014.29-2.382.0022.2222.2213.0010.0042.009.503.38134C44.4450.0033.3316.673.0027.7822.2220.0010.0043.007.501.75137R33.3342.8638.104.765.0050.0050.0030.0010.0051.004.382.62 (tablecontinues)278

PAGE 294

AppendixT(Continued)Table25.(continued) AchievementAssignmentsClassPointsProcrastination IdaTbGcPrePostRetDiffsNdNBPeBPfBeforegEhFinaliLatejReqkPrefl 138R344.4463.1061.901.194.0055.5655.5634.5010.0055.5012.622.38139R11.1157.1423.8133.338.0083.3383.3343.5010.0064.500.122.12140A33.3334.5214.2920.240.000.000.0011.5010.0040.5016.751.62141C33.3319.0533.33-14.295.0052.7825.0017.0010.0040.003.622.50142mR22.2259.5238.1021.435.0038.8938.8927.0010.0048.001.252.50143R33.3327.3833.33-5.956.0050.0050.0026.5010.0047.504.002.50144A1022.2242.8633.339.528.0069.4469.4434.0010.0063.000.002.75145R11.1116.679.527.143.0033.3333.3319.0010.0040.008.003.00146R33.3316.679.527.140.000.000.007.0010.0028.0016.753.62147R33.3329.7623.815.954.0033.3333.3321.5010.0042.508.003.38148A 26.65 75.0071.433.578.0083.3363.8941.005.0065.003.502.003.00149C44.4489.2971.4317.866.0072.2252.7838.5010.0061.506.122.75150R3 26.65 34.5233.331.193.0025.0025.0019.005.0035.009.381.75151A11.1141.6714.2927.384.0050.0030.5622.5010.0051.504.882.002.12152C22.2246.4338.108.334.0041.6719.4417.5010.0040.505.752.38153mC33.3334.5233.331.195.0027.7811.1112.5010.0035.503.622.38155A33.3315.489.525.950.000.000.006.5010.0035.5016.752.50157A822.2244.0538.105.958.0063.8963.8932.5010.0061.50-0.382.88159C11.1155.9538.1017.865.0047.2227.7823.0010.0046.002.622.002.00160A833.3311.909.522.382.0022.2222.2213.0010.0042.009.252.50162R33.3316.6719.05-2.380.000.000.007.0010.0028.0016.752.62163A1022.2240.4828.5711.904.0044.4444.4427.0010.0056.003.882.38164A811.1166.6747.6219.058.0075.0058.3335.5010.0064.502.623.25165R1933.3338.1023.8114.295.0052.7852.7830.5010.0051.5014.382.62166C22.2273.8157.1416.678.0077.7880.5648.0010.0070.00-1.753.25167A9 26.65 38.1033.334.765.0050.0047.2225.505.0049.503.883.88168R33.33 32.69 14.2918.408.0058.3358.3321.005.0037.0011.50 2.79 169R22.22 30.44 19.0511.398.0036.1136.1113.005.0029.0013.75 2.79 172A44.4452.3838.1014.290.000.000.0011.5010.0040.5016.752.12173C33.3344.0519.0525.004.0052.7827.7822.5010.0045.505.252.003.62174oC22.2245.2438.107.148.0086.1175.0038.5010.0061.500.002.002.25175C1411.1126.1923.812.388.0083.3355.5628.0010.0051.003.252.002.00 (tablecontinues)279

PAGE 295

AppendixT(Continued)Table25.(continued) AchievementAssignmentsClassPointsProcrastination IdaTbGcPrePostRetDiffsNdNBPeBPfBeforegEhFinaliLatejReqkPrefl 176C33.3315.4819.05-3.574.0047.2250.0024.5010.0047.503.002.12177A22.2292.8671.4321.438.00100.00100.0057.0010.0070.00-0.252.25178R11.1132.1419.0513.104.0011.1111.1113.0010.0034.005.751.88179R11.1139.2914.2925.005.0052.7847.2229.0010.0050.005.504.00180C1455.5645.2428.5716.678.0094.4472.2239.0010.0062.000.382.50182A22.22 21.54 4.7616.784.0055.5622.228.005.0032.008.75 2.79 183C1611.1190.4895.24-4.768.0097.22100.0056.0010.0070.00-1.883.38184A22.2257.1423.8133.338.0086.1186.1144.5010.0070.00-1.253.38185C1833.3348.8152.38-3.578.00100.00100.0047.5010.0070.00-0.382.62186C1611.1186.9057.1429.768.0097.2286.1151.0010.0070.00-0.123.38187A22.2210.714.765.955.0055.560.004.5010.0033.5016.122.25188A33.3334.5247.62-13.102.0022.2222.2216.5010.0045.509.501.88190mA33.3342.8638.104.767.0055.5613.8917.0010.0046.005.122.62191mC33.3316.6719.05-2.386.0058.3330.5618.0010.0041.005.003.75192R22.2267.8652.3815.488.0086.1152.7835.5010.0056.5015.753.00193A733.3339.2947.62-8.334.0052.7811.1114.5010.0043.506.122.62194R433.3326.1923.812.384.0055.5655.5628.0010.0049.008.122.62195mC33.3313.1014.29-1.192.0022.228.338.5010.0031.5011.253.62196C11.1165.4871.43-5.958.00100.00100.0050.0010.0070.00-1.253.12197A90.0027.3819.058.335.0058.3330.5618.0010.0047.005.622.75198R55.5640.4814.2926.198.0088.8988.8941.5010.0062.502.752.62199R22.22 36.06 19.0517.015.0047.2247.2217.005.0033.008.50 2.79 200R33.33 17.61 14.293.322.0022.2222.228.005.0024.009.50 2.79 201R11.1165.4852.3813.108.0094.4494.4449.5010.0070.002.122.88202C22.2225.009.5215.486.0052.780.0010.5010.0033.5010.004.00203A11.11 87.2 71.4315.778.00100.0077.7828.005.0052.000.25 2.79 204mR0.0021.434.7616.670.000.000.006.0010.0027.0016.752.12205C44.4430.9528.572.385.0041.678.3313.0010.0036.006.383.50206R22.2238.1023.8114.290.000.000.0010.0010.0031.0016.751.50207C1322.2265.4852.3813.106.0058.3319.4422.5010.0045.504.503.25208A22.2264.2966.67-2.388.0063.8922.2223.0010.0052.001.752.38209C33.3327.3833.33-5.958.0052.7858.3331.0010.0054.00-2.502.62 (tablecontinues)280

PAGE 296

AppendixT(Continued)Table25.(continued) AchievementAssignmentsClassPointsProcrastination IdaTbGcPrePostRetDiffsNdNBPeBPfBeforegEhFinaliLatejReqkPrefl 210A11.1116.6714.292.385.0055.560.007.0010.0036.0016.122.12213A33.3317.8614.293.573.0022.222.788.5010.0037.507.883.62214C33.3352.3866.67-14.298.0044.4444.4429.0010.0052.00-1.123.12215R55.5620.2414.295.953.0033.3333.3319.0010.0040.008.502.25216R433.3388.1066.6721.438.0088.8988.8952.5010.0070.005.753.00217R11.1120.244.7615.486.0069.4463.8931.5010.0052.5011.882.38219pR333.33 0 0.000.000.000.000.000.005.0016.0016.75 2.79 223R1922.2230.9528.572.384.0052.7852.7829.0010.0050.009.382.75225A1133.3366.6757.149.528.0097.2275.0044.5010.0070.003.503.38226C11.11 21.85 9.5212.330.000.000.000.005.0018.0016.75 2.79 227C0.0022.6238.10-15.484.0055.5633.3320.0010.0043.005.502.25228mR11.1158.3347.6210.714.002.782.7813.5010.0034.5016.122.75229C1733.3364.2966.67-2.387.0055.5655.5633.5010.0056.500.003.25230R344.4463.1047.6215.487.0058.3358.3337.0010.0058.008.882.50231mC1311.1147.629.5238.104.0050.0038.8925.0010.0048.004.503.88233A22.2245.2433.3311.904.0036.118.3314.5010.0043.508.003.38236A33.33 13.78 9.524.260.000.000.000.005.0024.0016.75 2.79 237R322.22 36.36 14.2922.070.000.000.000.005.0016.0016.75 2.79 238C1733.3338.1023.8114.298.0080.5680.5639.0010.0062.000.001.88239C33.3352.3847.624.768.0080.5658.3334.0010.0057.000.252.12240R522.22 22.38 9.5212.867.0086.1175.0027.005.0043.008.12 2.79 242mA33.3320.2423.81-3.575.0055.5627.7818.5010.0047.506.622.88243R22.2234.5223.8110.717.0063.8963.8931.5010.0052.504.751.88244R22.2279.7676.193.578.00100.00100.0054.5010.0070.00-0.502.50245mC44.4421.439.5211.900.000.000.007.5010.0030.5016.753.25247mA822.2215.4823.81-8.333.0033.3333.3318.5010.0047.506.501.62248A1222.2221.4314.297.144.0036.1136.1122.0010.0051.006.253.38250R22.2235.7123.8111.900.000.000.0010.5010.0031.5016.752.62251R633.3310.719.521.190.000.000.004.5010.0025.5016.753.00252mR655.569.529.520.000.000.000.004.0010.0025.0016.752.75253C44.4451.1938.1013.107.0052.7830.5625.0010.0048.003.623.00254A1022.2257.1414.2942.868.0094.4494.4447.5010.0070.00-0.252.12 (tablecontinues)281

PAGE 297

AppendixT(Continued)Table25.(continued) AchievementAssignmentsClassPointsProcrastination IdaTbGcPrePostRetDiffsNdNBPeBPfBeforegEhFinaliLatejReqkPrefl 255A11.1155.9552.383.577.0069.4450.0031.0010.0060.000.382.50256C22.2298.8166.6732.148.00100.00111.1163.5010.0070.00-5.754.12257C1822.2251.1933.3317.868.0086.1180.5641.5010.0064.50-0.382.88258C44.4486.9071.4315.487.0080.5655.5640.0010.0063.004.881.88259R22.2272.6242.8629.766.0072.2272.2241.5010.0062.5011.252.50261R55.5692.8695.24-2.388.0086.1186.1153.5010.0070.00-5.003.50262A33.3344.0538.105.958.0094.4494.4445.0010.0070.00-0.623.75263C33.3380.9566.6714.298.00100.00100.0055.0010.0070.00-0.502.75264A1233.3319.0528.57-9.523.0027.7827.7816.5010.0045.509.382.88265C22.22 11.16 4.766.407.0072.225.562.005.0020.009.62 2.79 266mC44.4425.0038.10-13.104.0052.7825.0016.5010.0039.506.253.62267R55.5635.7128.577.148.0088.8988.8941.0010.0062.000.002.00268A11.1116.679.527.140.000.000.007.0010.0036.0016.753.00269R44.4482.1457.1425.008.0097.2297.2254.5010.0070.000.254.50270A22.2269.0557.1411.908.0075.0052.7836.0010.0065.000.753.25271R33.3385.7180.954.768.00100.00100.0057.0010.0070.0015.754.25272mA11.1115.48 8.31 7.174.0030.560.005.0010.0034.008.384.50274C22.2251.1952.38-1.198.0094.4494.4448.0010.0070.00-0.623.00275A1144.4421.4323.81-2.388.0063.8911.1111.5010.0040.5010.253.25279R33.3389.2961.9027.388.0094.4494.4455.0010.0070.00-0.122.50280C22.2227.3814.2913.100.000.000.007.0010.0030.0016.752.88281A33.3319.059.529.525.0058.3350.0026.0010.0055.002.252.88282R22.2266.6757.149.526.0069.4463.8937.5010.0058.5013.003.00283A33.3327.38 30.19 -2.810.000.000.007.0010.0036.0016.753.38284C33.3352.3847.624.767.0063.8944.4429.0010.0052.000.621.75285C44.4450.0033.3316.678.0086.1183.3343.5010.0066.500.252.50286R322.229.5214.29-4.766.0063.8963.8927.0010.0048.007.882.25287C1411.1145.2442.862.388.0069.4455.5631.5010.0054.500.122.75288qA44.4466.6719.0547.626.0050.0030.5625.5010.0054.500.752.001.88290C55.5619.0519.050.000.000.000.006.5010.0029.5016.752.25291R0.0059.5219.0540.487.0066.6766.6735.5010.0056.501.503.50292C11.1135.7114.2921.438.0094.4488.8942.5010.0065.50-0.253.00 (tablecontinues)282

PAGE 298

AppendixT(Continued)Table25.(continued) AchievementAssignmentsClassPointsProcrastination IdaTbGcPrePostRetDiffsNdNBPeBPfBeforegEhFinaliLatejReqkPrefl 293R33.33 38.6 14.2924.310.000.000.000.005.0016.0016.754.00 2.79 G1R127.7888.6966.6722.027.0066.6766.6744.7510.0063.753.813.06G3R3 32.22 34.43 28.575.863.3333.8033.8019.587.5038.0812.04 2.41 G4R433.3357.1445.2411.906.0072.2272.2240.2510.0059.506.942.81G5R516.67 34.40 23.8110.597.0080.5672.2232.007.5050.507.88 2.90 G6R644.4410.129.520.600.000.000.004.2510.0025.2516.752.88G7A733.3350.0047.622.386.0036.1115.2817.5010.0046.502.882.69G8A822.2234.5229.764.765.2548.6144.4424.8810.0053.884.502.56G9A9 13.32 32.7426.196.555.0054.1738.8921.757.5048.254.753.31G10A1022.2246.8325.4021.436.6769.4469.4436.1710.0063.001.212.42G11A1138.8944.0540.483.578.0080.5643.0628.0010.0055.256.883.31G12A1227.7820.2421.43-1.193.5031.9431.9419.2510.0048.257.813.12G13C1316.6756.5530.9525.605.0054.1729.1723.7510.0046.754.503.38G14C1425.9338.8931.757.148.0082.4161.1132.8310.0055.831.250.672.42G14rC1425.0040.4833.337.148.0083.3364.5834.2510.0057.250.941.002.38G15C1533.3359.5261.90-2.388.0069.4461.1135.0010.0058.000.752.75G16C1611.1188.6976.1912.508.0097.2293.0653.5010.0070.00-1.003.38G17C1733.3351.1945.245.957.5068.0668.0636.2510.0059.250.002.56G18C1827.7850.0042.867.148.0093.0690.2844.5010.0067.25-0.382.75G19R1927.7834.5226.198.334.5052.7852.7829.7510.0050.7511.882.69G30A3016.6713.699.524.175.0055.560.005.7510.0034.7516.122.19G31R3122.2241.0726.1914.885.0051.3948.6129.5010.0050.504.943.31 (tablecontinues)283

PAGE 299

AppendixT(Continued)Table25.(continued) AchievementAssignmentsClassPointsProcrastination IdaTbGcPrePostRetDiffsNdNBPeBPfBeforegEhFinaliLatejReqkPrefl G32C3233.3343.4550.00-6.556.5036.1127.7820.7510.0043.751.252.75G33R3322.2236.9023.8113.100.000.000.0010.2510.0031.2516.752.06 Note.Valuesinboxesindicatemissingdataforwhichvalueswereestimatedinwholeorinpart.Scoresforindividualsingroupswereanalyzedviagroupaverages.aStudentidenticationnumber.NumbersbeginningwithGindicatestudent-madegroups.GroupsG1-G19wereidentiedpriortotreatment,andG30-G33emergedduringtreatment.bTreatment.cStudent-madestudygroup.Valuesin[1,19]indicategroupsidentiedpriortothetreatmentcondition.Valuesin[30,33]indicategroupsidentiedafterthetreatmentcondition.dNumberofassignmentscompletedoutofeight.eAssignmentgradebeforeapplicationofbonusandpenaltypoints.fAssignmentgradeafterapplicationofbonusandpenaltypoints.gHTMLgradeearnedtowardEdTechbeforeapplyingcurveandextracredit.hExtracreditearnedforcompletingthepretestand/orpacingpreferencesurvey(vepointseach).iHTMLgradeearnedtowardEdTechafterapplyingextracreditpointsandacurve.Toequalizedifferencesinmeangrades,studentsreceiveddifferentcurvesdependingontreatment,with11pointsforstudentsinR,13forstudentsinC,and19forstudentsinA.jAveragelevelofprocrastinationoneachassignmentintermsofdaysearly(negativevalues)orlate(positivevalues).kNumberofrequestsfordeadlineextensions.lSelf-reportofpacingpreferencewithvaluesintherange[1,5]andhighervaluesindicatingahigherpreferenceforstudent-pacing.mReportedcompletingalargernumberofassignmentsalonethanactuallysubmittedforgrading.nTookposttestandpacingpreferencesurveyafterretentiontest,sodatatreatedasmissing.oNotidentiedingroup14priortostudy,butidentiedafterwardsandincludedingroup26.Studentwasincludedingroup14foranalysisofpreferencedataonly.pConverted-6.12to0forimputedposttestscore.qDifferencescoreisanoutlierwithz3.35.rAveragesincludevaluesfromstudent#174andwereusedforanalysisofpreferencedataonly.sPosttestscoreminusretentiontestscore.284

PAGE 300

AppendixUContentsofCD-ROMTheCD-ROMaccompanyingthedissertationincludesacopyofeachofthefourcoursewareversionsusedinthestudy.Threeversions,1.2r,1.2c,and1.2a,weredeliveredtothestudents,dependingonthetreatmentconditiontowhichtheywereassigned.Thefourthversion,1.2,wasutilizedbythecourseofcials.Theonlydifferencebetweenthestudentversionswasthediscussionofdeadlines.Theteacherversionalloweduserstoselectthetreatmentconditionandtoreviewthelanguagepresentedtoallstudents.Thestudentversionshadspeciclinksallow-ingthemtocontactcourseofcialseasily.Theteacherversionprovidedtoolsforloggingdeadlineextensionrequestsandforgradingsubmissionsaswellasalistofallstudentsandthetreatmentstowhichtheywereassigned.Version1.4ofthecoursewareisalsoincludedontheCD-ROM.Studyspecicinformation,includ-ingexactdeadlines,hasbeenremovedoraltered,sothatthecoursewaremaybeutilizedbyamoregeneralaudience.Version1.4incorporatesimprovementswhichallowtheusertoturnoffthesound,toviewthecoursewareonaWindows98PCusingeitherInternetExplorer5orNetscapeNavigator6.1,andtohearthenarrationsusingQuickTimePlayer5.0.2.Italsoincludesanupdatedindexandcopyrightinformationaswellasnarrationcorrectionsanddisplayenhancements.TheCD-ROMalsocontainsinformationontheassignmentscompletedbythestudents,includingworkingproductversions,solutioncode,andgradingrubrics.Inaddition,thecompletedissertationisaccessibleinPDFformat,andthedefensemaybereviewedinwholeorinpart.ItconsistsofonevideosegmentpresentedinMOVformatandnumerousaudiosegmentspresentedinMP3format.TherawdatainAppendixTareprovidedinaformatwhichiseasytoaccessviaananal-ysisprogramsuchasSAS.Finally,sourcecodeforconductingtherandomizationtests(C)andmoretraditionalanalyses(SAS)aswellasforgeneratingthegraphs(gnuplot)inthedissertationareprovided.285

PAGE 301

AbouttheAuthorTinaLareeMajchrzakreceivedthefollowingdegreesfromtheUniversityofSouthFlorida:BachelorofArts,MagnaCumLaude,inMathematicswithReli-giousStudiesminor(1991);MasterofScienceinComputerScience(1995),andDoctorofPhilosophyinCurriculumandInstructionwithemphasisinInstructionalTechnology(2001).Shereceivedfourscholarshipsandfellowshipsandworkedasaresearchassistant,ateachingassistant,andanInternetprogrammer.Dur-ingthepasttenyears,shehastaughtundergraduatesandgiftedmiddleandhighschoolstudentsMathematicsandcomputerprogramming.Mrs.MajchrzakwasawardedtheProvost'sCommendationforOutstandingTeachingByAGraduateStudent.Shepublishedthreeconferencepapers,oneofwhichwaslaterincludedinabook,onejournalarticle,threesymposiumreports,andthreetutorialsonHTML,JavaScript,andCprogramming.ShealsocollaboratedwiththeFloridaCenterforInstructionalTechnologyonacomputernetworkingtutorialandanFCATtoolforTeachers.