xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam 22 Ka 4500
controlfield tag 007 cr-bnu---uuuuu
008 s2010 flu s 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0004570
Systematic development and validation of a course of instruction in prior learning assessment
h [electronic resource] /
by John McNally.
[Tampa, Fla] :
b University of South Florida,
Title from PDF of title page.
Document formatted into pages; contains X pages.
Thesis (EDS)--University of South Florida, 2010.
Includes bibliographical references.
Text (Electronic thesis) in PDF format.
Mode of access: World Wide Web.
System requirements: World Wide Web browser and PDF reader.
ABSTRACT: Many post-secondary schools across the country offer adult working students an opportunity to obtain at least partial credit for work and life experiences in their curriculum through portfolios. The primary goal of this project was to design, develop and evaluate a portfolio course for adult students at a small independent university. Design emphasized adult learning theory and incorporated instructional design best practices throughout. Also significant to the design was the implementation of the Quality Matters Â™ Rubric. The project focus was to intertwine the six assumptions of adult learning theory while implementing best practices and effective instructional strategies, and to conduct formative and summative evaluations. The study incorporated a pre-test post test instrument and satisfaction questionnaire for quantitative data collection. The results of this project are positive based on the evaluation data collected during this project.
Advisor: Jim White, Ph.D.
Adult Learning Theory
Prior Learning Assessment
x Secondary Education
t USF Electronic Theses and Dissertations.
Systematic Development and Validation of a Course of Instructi on in Prior Learning Assessment by John D. McNally A thesis submitted in partial fulfillment of the requirements for the degree of Education Specialist Department of Secondary Education College of Education University of South Florida Major Professor: James White, Ph.D. Ann E. Barron, Ed.D Tina Hohlfeld, Ph.D. Date of Approval : June 10 20 10 Keywords: Andragogy, Adult Learning Theory, Self Directed Learners, Experiential Learning, Pr oject Based Learning, Kolb Prior Learning Asse ssment, Portfolio, Knowles Copyright 2010 John D. McNally
i Table of Contents List of Tables i List of fi .... iv .1 Identified Problem and its Context .. .. .. 2 Des ig 6 Adults as Self Adults Learners have a lot to 8 Adult learners are ready to learn when they realize what they are learnin g does affect some aspect of their lives .9 Adult learners prefer problem ... 10 Adult Learners are motivated intrinsically first, extrinsically second .... ......... .10 ..1 1 Recommended Instructional Strategies and .. 12 Strategies for Learning .13 ... Results and Ana Potential for Future R .38
ii ..40 Appe Appendix A: Ko lb Experiential Learning Theory Model Appendix B: Appendix B: Kolb Learning Styles 6 Appendix D: Verbs for Educational Objectives .4 8 Appendix E: PLA Course Outcomes matched to Blo Educational Objectives 49 Appendix F : Directions to Expert Reviewers (Content/Template) 5 0 Appendix G : Directions to Expert Reviewers (Course Design/Rubric) 1 Appendix H: Directions to Usability Testers 2 Appendix I : Usabilit y Measurement Inventory for Course Website Prior Learning Assessment Portfolio Appendix J : Severity Ratings for Usabilit y Problems Appendix K: Pre test Post test for level 2 evalu Appendix L: Satisfaction Questionnaire (Summative Evaluation) 59
iii List of Tables Table 1 Research for Prior Learning Assessment Portfolio Basic Informati 4 Table 2 Course Outcome 13 Table 3 Course Out Table 4 Course Outcome Numbe 17 18 Table 6 Cours Table 7 Course Outcom 20 Table 8 Course Outcome Numbe 2 1 Tab le 9 presents the development milestones for Prior Learning Assessment .29 Table 10 Results of Satisfaction Questionnaire 34 Table 11 Results of Pre test 35 Table 12 Course Outcomes for PLA101 matched to Table 13 Feedback form for usability testers
iv List of Figures Figure 1 Stevenson Univ. Template for M 28 Figure 2 Kolb Experiential Learning Theory Model
v Systematic Development and Validation of a Course of Instruction in Prior Learning Assessment John D. McNally ABSTRACT Many post secondary schools across the country offer adult working students an opportunity to obtain at least partial credit for work and life experiences in their curriculum through portfolios. The primary goal of this project was to design, develop and evaluate a portfolio course for adult students at a small i ndependent university. Design emphasized adult learning theory and incorporated instructional design be st practices throughout. Also s ignificant to the design was the implementation of the six assumptions of adult learning theory while implementing best practices and effective instructional strategies, and to conduct formative and summative evaluations. The study incorporate d a pre test post test instrument and satisfaction questionnair e for quantitative data collection. The results of this project are positive based o n the evaluation data collected during this project.
1 Analysis Introduction The estimated population of adult students over the age of 24 at postsecondary institutions across the nation currently lingers at about 44 percent. The U.S. Department of Labor states that mi llions more Americans need postsecondary credentials to succeed economically (U.S. Department of Labor, 20 07). Approximately 34 million working adults have absolutely no college experience (Pusser Breneman, Gansneder, Kohl Levin, Milam, and Turner, 2007 ). Research indicates adult students in the postsecondary setting require a different approach in the facil Adult learners must be recognized as a diverse and complex set of individuals with widely divergent aspirations, Pusser, et al., 2007). Because of various important commitme nts, such as work and family responsibilities, many adults choose a nontraditional path to postsecondary educations part time students, possibly with assistance from employers. These students can bring a wealth of information and life experiences to the cl assroom. These life and work experiences, or other prior non credit learning activities could translate into attainment of college credits by demonstrating students come into the postsecondary environment with a strong sense of goal orientation. As a result, many adults seek credit for prior learning, because they do not
2 want to be taught something they already know. This credit also allows them to move more quickly throu gh their program than their traditionally aged fellow students. Identified Problem and its Context The po pulation at Stevenson university mirrors the national description of adult student learners a ge is 24 or older, most have transfer credits, work i n a professional environment with a full or part time job in addition to go ing to school and ma ny are coming back to school to advance their career Currently, there is no local mechanism or course in place to direct or guide adult accelerated students in th e development of a prior learning experience portfolio at Stevenson University. Stevenson University directs its students to other schools, such as Regis University, to complete a portfolio development course. After the development of a portfolio to demo nstrate competencies of the learning objectives of a particular course, the student submits a challenge in the form of a print portfolio to their program coordinator at Stevenson University for review. The portfolio is then forwarded to the appropriate s ubject matter expert teaching in the accelerated programs, and after review it is approved, returned for rewrite, or returned for additional support Portfolios are graded as Pass/Fail. According to Patricia Ellis, Associate Dean of Accelerated Undergr aduate Programs, al l of the Stevenson students who have completed a portfolio course for prior learning assessment thus far have taken the course online at Regis University. Accelerated programs at Stevenson University enable adult undergraduate students taking 5 or 8 week classes in hybrid or online format. The se programs include Business Administration,
3 Business Information Systems, Computer Information Systems, Interdisciplinary Studies, RN t o BS Nursing, Criminal Justice and Paralegal Studies. An analysis of some successful programs provided direction in approach and research. Most institutions reference The Council for Adult and Experiential Learning (CAEL) as a primary source of informat ion and guidance. This international organization has worked with accrediting bodies, employers, and academic institutions since 1974. CAEL has taken the lead in articulating the philosophical basis for prior learning assessment and has developed ten stan dards for assessing prior learning experience (Bamford Rees, 2009). The design of the course was directed by Adult Learning Theory and standards of design. The course structure is similar to the model presented by Charter Oak State College. This decision was based on the simplicity of the Charter Oak Model, which focuses on the development of a portfolio for a single course and concentrates primarily on a specific writ ing styl e and the portfolio preparation. These two models (Charter Oak and Stevenson) are presented below in separate list s and Table 1 provides an indication of how representative schools across the country approach and deliver portfolio courses and wor kshops.
4 Portfolio Assessment at Charter Oak State College is based on evaluation of a document composed of five elements: 1. A description of a college course against which knowledge will be measured. 2. A biographical introduction in which sources of learni background are identified. 3. A summary of the learning outcomes for the course being challenged. 4. A narrative essay in which the student describes what he/she did and learned, and how that knowledge was applied. 5. Evidence from a variety of sources to support the claim and to demonstrate knowledge and skills. From Prior Learning Portfolios: A Representative Collection (Page 40 ). Portfolio Assessment at Stevenson University is based on evaluation of a document composed of five main elements: 1. A description of a college course and its required course outcomes, against which knowledge will be measured. 2. A biographical introduction in which major milestones, critical events and other learning e vents are identified and discussed in detai l 3. A competency chart listing each course outcome for the challenge course, and several competency statements that each work toward achieving the listed outcome. 4. Each course outcome (chart) will be accompanied by a narrative that explains in detail how the competency has been mastered through learning and application, and how it equates to college level learning. 5. Evidence (documents) from valid sources to support the claim and to demonstrate knowledge and skills.
5 Table 1 Presents Research Information for Prior Learning Assessment Portfolio Basic Information School Course Ref. Credit Length Software Delivery Pre Reqs Regis University ED 202 3 8 week Angel Traditional & Online N/A Sprint Arbor LLP Workshop N/A 4 hour UNK Traditional & Online N/A St. Edward's University PLA Seminar 1 1 year Blackboard Hybrid N/A St. Joseph's College GS 110 3 UNK UNK Traditional & Online ENG 103 Valdosta State University PLA 2000 2 UNK WebCT Online N/A Vermont State College APL 3 UNK Blackboard Hybrid N/A Univ. of Alabama ASK (manual) N/A 6 months N/A Self N/A Sinclair Comm College Advisor Monitor N/A UNK N/A Self N/A Ashford University EXP 200 3 5 week Blackboard Online PSY 202 Athabasca University PSY 205 PLAR 3 UNK UNK Online N/A Empire State College PLA Workshop N/A UNK N/A Traditional N/A Charter Oak State College IDS 102 3 8 week UNK Online 6 ENG Cr. Stevenson University PLA 101 3 8 week Blackboard Online ENG 152
6 Design Adult learning theory served as the guiding theoretic al framework of the design of the course in Prior Learning Assessment This rese arch based course enable s students to create an effective, standards based portfolio through a systematic, yet flexible, process that successfully implemen ts the components of adult learning theory This implementati on require s strategies and a learning environment not typically f ound in most higher education settings. Malco l m S. Knowles, a central figure in the development of Adult Learning Theory, determined environment to be critically important. Environmental conditions can often be a barrier to learning and should be a primar y consideration for the planning process for adult educators. Knowles (2005) believed in experiential learning and in a very strong connection between living and learning. Adult Learning Theory identifies six assumptions about Adult Learners and how they approach lear ning These six assumptions state that adult learners: 1 ) are self directed 2) need to know why, how and what they are learning 3) have a lot to offer the class through years of experience and their own mental models 4) must know that w hat they are learning has immediate application and benefit to life and/or work 5) prefer problem centered instruction over subject cent ered instruction 6) are motivated intrinsically first, externally second.
7 Adults as Self Direct ed Learners Self directed learners take responsibility for their own learning and research indicates that adults exhibit this preference. The notion of providing the adult student opportunities to control some aspects of the learning is equally important to, and poss ibly more important than the actual content or the manner in which it is being presented (Hiemstra, 1997). A self directed teaching and learning environment provides for the consideration adequate opportunities for student control (Hiemstra, 1997). This method also provides an environment conducive to developing a high internal Locus of Control by enabling students to take credit for their own successes ( Northwest Regional Educational Lab oratory 2009) Research has suggested that distance education students with a high internal locus of control are successful because they are more likely to be able to work independently (Dillie & Mezack, 1991). The implications for the use of technology regarding the self directed learner are positive as well, particularly for the experienced online learner. Web based instruction provided in a non linear format allow s the adult learner to proceed as desired instead of as directed (Fidishun, 2009). T he a dult learner new to the online environment, however, and possi bly even new to adult learning in the higher education setting may require s ome type of support system, as well as a structure to fall back on
8 Being a self directed learner does not necessarily mean one that is entirely self teaching, or one that is completely autono mous in the learning environment. A class with many students will have learners at various stages of ability regarding skills for s elf teaching, as well as the ability to direct their own learning. There are many variables that would contribute to, or determine the level of ability, including student background and experience, as well as the content and the environment for learning. In some cases, especially in a topic where the student may have little experience, the learner may prefer a totally structured environment because it is the easiest (Knowles, Holton Swanson, 2005). Adults want to know Why am I learning th is? Adult learners must see some benefit in having to learn something. If they are taking responsibility for their own learning they do not want to waste time learning something unnecessarily. Providing for mutual planning of what is to be learned, and involving adult learners in how it is to be learned can have great benefit. Simply providing the opportunity to collaborate in the learning process can be very effective, can improve self concept and from the student perspective validates the need for lea rning (Knowles, et al., 2005). Adult Learners have a lot to offer as a resource for learning Most adult learners have work and life experie nces that can benefit the class in some way. This is a resource that should be accessed throughout the course, but must remain closely monitored. Mental models can inhibit new learning if the new material conflicts with what is already known and requires a change in existing schema There
9 has been a tremendous amount of research regarding the learning process and h ow new content fits in with knowledge the learner already possesses or has stored in long term that this scheme must be changed if the new information does not fit appropriately into this already existing way of knowing continuous process grounded in experience, which means that all learn ing can be seen as relearning. This is particularly true for adults who have such a large reservoir of Adult learners are ready to learn when they realize that wh at they are learning does affect some aspect of their lives Adult learners ar e ready to learn once they realize what they are learning is important to them in some way, but that does not mean that they are fully prepared or capable, and do not need at least some type of guidance and support The effective or successful instructor or facilitator must be able to identify what type of support is required. Pratt (1988) determined that there are two types of assistance for adult learners in this regard direction and support Direction has to do with assistance or guidance regarding t he knowledge and skills necessary to learn or apply the content; s upport is primarily affective in nature, and may involve a strategy such as feedback to fulfill an emotional need and boost confidence. Of course there can be any variation, such as a learn er who needs direction and support, a le arner who needs only direction, or a learner who needs only support, and this can change throughout the course.
10 Adult Learners prefer problem centered instruction Jonassen (1981) felt that immersing a student in a problem to solve is the ideal context for learning, but states that it should be authentic and that everything needed in the way of information and tools should be provided for the students to manipulate various solutions as th ey work to solve the problem. Adult learners prefer some type of project to complete or a problem to solve in an education setting This approach, when taken, naturally creates a student centered learning environment and one that is conducive to developin g, or requiring, creative and critical thinking skills. The instructor during this type of instruction is not necessarily just on the sideline watching, but should model the steps, cognitive processes or problem solving skills and then facilitate as neces sary (Brown, Collins & Newman, 1989). The best case scenario would be for the students to successfully model processes for other students based on their prior experiences Knowles, et al., strongly support David Kolb and his Experi ential Learning Model (A ppendix A ). The Kolb model has four stages: Concrete Experience; Observations and reflection; Formation of abstract concepts and generalizations; and Testing implications of new concepts in new situations. Kolb also outlines four distinct learning styles that align with the four stages of the model: Accomodator; Converger; Diverger; Assimilator (Appendix B ). Adult Learners are motivated intrinsically first, extrinsically second Knowles, et al., cite Expectancy theory as a support for this sixth assumption Valence or the value on an outcome; Instrumentality or the probability that the valued
11 outcomes will be received given that certain outcomes have occurred; and Expe ctancy or the belief a person has that certain effort will lead to outcomes that get rewarded. Knowles, et al., simplify the theory by stating that adult learners will be highly motivated if they believe they can learn the material, believe it will be b eneficial and believe it is important to their life. Project Goal and Course Outcomes The overall goal of this project wa s to create and validate a course of study, Prior Learning Assessment Portfolio Course (PLA 101) woven successfully around a theoretical framework of Adult Learning Theory while also implementing best practices of instructional design and evaluation. The purpose of the product is to enable Adult Accelerated students at Stevenson University to examine prior learning experiences and events and create a standardi zed portfolio that demonstrates higher level learning, and also demonstrates achievement of course outcomes in t The successful achievement of this goal will be met through completion of the following course outcomes for PLA 101: 1. Identify and differentiate types of learning, learning styles and levels of learning. 2. Conduct research to identify resources, critical events, and other learning events for analysis, description and classification. 3. P rovide a critical analysis of portfolio elements created by someone else (peer review). 4. Demonstrate a strong correlation between prior learning experiences and the course learning outcomes of a course selected for Portfolio Assessment by documenting critic al and other learning events in proper format and writing style.
12 5. Develop a written narrative and an autobiography to present evidence and reasonable argument to support the proposed competencies for which the student has claimed mastery. 6. Create/ Assemble a n electronic version of the portfolio to enab le Internet presentation/review and that may also be printed for review. 7. Demonstrate an understanding of The Council for Adult and Experiential Learning Standards for the Assessment of Credit, and the Stevens on University Portfolio Evaluation Process. Recommended Instructional Strategies and Rationale The rationale for the proposed solution strategy incorporates components of adult learning theory throughout the course of instruction as appropriate. The instru ctional strategies motivate and provide for self direction by allowing the adult learners to assist in planning for learning. The content is meaningful because there is personal benefit; upon successful completion students can create portfolios to earn cr edit. The instructional strategies enable students to achieve the course outcomes in a student centered environment that is safe and supportive. The proposed instructional strateg y implements the course over nine modules. The following charts outline the Course Outcome, Module, Module Objective, Learning Activity, Assessment, and the application or implementation of the pertinent component of Adult Learning Theory. The charts are organized by Course Outcome.
13 S trategies for Learning The strategies for the students to achieve all succeeding course outcomes were developed after breaking down the outcome into more manageable pieces, or tasks, that work towa rd completion of the whole task, or an understanding of the concept. These take the form of presented in the charts below and an explanation follows each chart. This Task Centered Instruction systematically incorporates the first principles of instruction (Merrill, Barclay van Schaak, 2007) Table 2 presents Course Outcome number 1. 1. Identify and differentiate types of learning, learning styles and levels of learning. Module Objective Activities Assessment Pre Class Assist in Planning for course content & assignment Participate in Discussion Board Pre Class Demonstrate an understanding of Learning Theory and Application Read Chapter 7 in text Colvin text Complete Reading Guide Questions Pre Class Determine Learning Style Complete Learning Style Inventory (LSI) Respond to LSI on Discussion Board Pre Class Determine Multiple Intelligence Strengths Complete Multiple Intelligence Test (MIT) Respond to MIT on Discussion Board
14 Module One Determine the level Taxonomy for the outcomes of y our challenge course Taxonomy in text & via link(s) in module Complete Reading Guide Questions The first module (pre class) allows for the students to assist in the planning for the course, introduces the students to the learning management system and initiates learning important factor regarding the pre class module is to d etermine the needs of the student. Most of the activities, while certainly important, do not count for grade s This provides time for the student to find out what is required, get a feel for the approach and to ask questions about anything that must be d one now or in the future without feeling pressure They can see what type of course it is, make sure they have the right book, read the syllabus and schedule, explore the course etc. The Reading Guide Questions (RGQs) identify specific areas of content that are critical ly important for what lay ahe ad. This directs the students not to spend more time than necessary, unless they prefer to learn more. So, the RGQs allow the learners to focus and applies directly to what they need to accomplish Two of the reading guide questions immerse the student in a problem that will be encountered later on but will be more complex. They are designed to prepare the students ( Merrill, et al., 2007 ) for the critically important task of applying t hese learning theories early on, possibly providing early indicators of problem areas for individual students.
15 During the planning session, the option to make this an activity modeled on the discussion board will be proposed. Identify and differentiate types of learning, learning styles and levels of learning consider what is taking place cognitively as they learn. They may apply different strategies that have proven successful for them in the past, but typically they do not think beyond that application. In this course they will have to consider different strategies and perspectives. post secondary education. Therefore, it is critical that the students in this course know something about learning in general and also have an understanding of not only their preferred style of learning, but other styles as w ell. The course content presents and requires implementation or an application of the Kolb Experiential Le arning Model or Cycle (Appendix A Intelligences (Appendix C (Appendix D ). The Kol b Model, as it is presented in the text, Earn College Credit for what you know provides prompts at each quadrant to assist the student in the recall of circumstances surrounding a learning event. The prompts simplify the overall process by asking What did you see? determination based on what you saw? (Colvin, 2006). on because of its experiential nature, it is just one of many theories regarding how people learn. Howard
16 Adult Learning Theory. In order for the student to equate something they feel they have learned on their own to something the University says they should know before they have achieved a Good enough would equate to the knowledge, skills and abilities required at the ta xonomy and the required course outcomes of their challenge course. The discussion board responses are intended primarily to identify what the instrument well as student), but also to build community in the course and get an informal measure of how accepting the students are of the proposed theories. This strategy can also se rve to draw out any life experiences that are pertinent to the learning, and possibly initiate a connection between theories and concepts Learning is facilitated when existing knowledge is activated as a foundation for new knowledge ( Merrill, et al., 200 7 ) This asynchronous communication tool provides opportunity for Student Content Interaction, Student Student Interaction and Student Instructor Interaction (facilitation) for these two assignments.
17 Table 3 presents Course Outcome number 2 2. Conduct research to identify resources, critical events, and other learning events for analysis, description and classification. Module Objective Activities Assessment Pre Class Meet and greet your classmates Locate and Post your challenge course description see Online Catalog, Web Express or University Catalog Post an intro/biography on the Discussion Board with your goals included. Also, identify what course you intend to challenge and post the course description as a separate paragrap h Peer Response greet classmates Pre Class Locate the Course Outcomes for the course being challenged Contact Advisor or Course Instructor if necess a ry Necessary for later assignments Module One Prepare organizational documents to assist planning timeline, learning chart, chart Read Chapters 8 & 9 Reading Guide Questions & Chapter Assignments The RGQ assignment in Module One un der this course outcome requires the students to determine the level of the course outcomes of the student s challenge course knowledge is applied by the learner). This task is modeled in the local conten t (Module One) using the course outcomes for this course prior to the assignment (Appendix E ). Learning is facilitated when new knowledge is demonstrated to the learner ( Merrill, et al.,
18 2007 ) The RGQ assignment also requires the student s to submit a resum chart, resource chart and timeline. These stimulate memories surrounding critical events An example o f each is provided in the text. This assist s student s in planning their approach and in identifying events that provide applicable information regarding the learning of required material. Table 4 presents Course Outcome number 3 3. Demonstrate a strong correlation between prior learning experiences and the course learning outcomes of a course selected for Portfolio Assessment. Module Objective Activities Assessment Module One Analyze the Course Outcomes for the challenge course and write a critical reflection View Course Outcomes video & Reflective Writing video Read Chap 8 & 9 Identify & analyze the course outcomes for the course you want to challenge. Module One Provide a critical analysis using the Kolb Model and Taxonomy. Review text (89 to 93) and some of the links provided in the modules to outside materials Select a learning event from your past and provide a critical analysis that relates elements of the Kolb model & The module objectives for this course outcome immerse the student in solving the problem of analyzing one critical event using the Kolb model to collect the necessary information to document and demonstrate learning and determine what level of learning has been obtained based on achieved competencies (module one). This activity will be modeled prior to student engagement. Module two immerses the students in their own
19 required activities to determine achieved competencies and document them in the proper fo rmat (chart). Table 5 presents Course Outcome number 4 4. Develop a written narrative and an autobiography to present evidence and reasonable argument to support the proposed competencies for which the student has claimed mastery. Module Objective Activities Assessment Module Two Write a 4 to 5 paragraph Narra tive to support each course outcome at the appropriat e level Write supporting narrative Module Three Write an autobiographical summary. Read Chapter 11 Read Appendix 2, 3 & 4 (Page 159 165) The activities in Appendix 2, while potentially helpful, are optional. Submit all writing to Smart hinking prior to submitting to instructor Write autobiographical summary Module two and three are for writing the supporting narrativ e and the autobiographical summary, totally immersing the student in the activity of demonstrating and proving learning and synthesizing these elements into a panoptic montage of their prior learning. The student can use various methods of support during t his time, before or in addition to submitting to SmarT hinking, an online tutoring service, which is required.
20 Support systems include various discussion board forums within the course: Narrative Questions; Autobiography Questions; Content Questions, Process Questions, Coffee House; direct contact with the instructor via email or telephone, and also a community organization on Blackboard Table 6 presents Course Outcome number 5 5. Create/Assemble an electronic (digital) version of the portfolio to enable Internet presentation/review, and that may also be printed for review. Module Objective Activities Assessment Module One Demonstrate an understanding of the contents of the portfolio View the Portfolio Contents Powerpoint Review Chapters 8 & 9 in text Completed Portfolio Module Three Create a list of the Port folio appendices, & supporting documentation Read Appendix 5 (Section III) for information purposes only Submit to instructor Match list with actual documents all documents must be digitized Module Five Create a Final Draft list of the Portfolio appendices, including all supporting documentation Review Append ix 7 Save/combine with portfolio Match all supporting documents? Module Seven Create a Master Document of your Portfolio Review all portfolio requirements and standards Finalize Portfolio, convert to pdf and submit to the instructor Module Five Create a Final Draft list of the Portfolio appendices, including all supporting documentation Review Appendix 7 Save/combine with portfolio Match all supporting documents?
21 To achieve this course outcome, primarily during module three and four, there is a continuation of the writing process and beginning development of the master document in module four. This also provides some extra time for writing if necessary. The same support methods are available for the technical aspects of building the por tfoli o in Microsoft Word These support systems including various discussion board foru ms within the course Content Questions P rocess Questions, Coffee House direct contact with the instructor via email or telephone. Table 7 presents Course Outcome number 6 6. Provide a critical analysis of written portfolio elements created by someone Module Objective Activities Assessment Module Three Analyze someone portfolio elements Use the Discussion Board Provide a written review/critique of competency statements and narrative This module objective provides an opportunity to collaborate, to see how peers are approaching the problem, and to offer positive comments and constructive feedback, as well as receive input from peers as they continue to work on the writing, and use technology.
22 Table 8 presents Course Outcome number 7 7. Demonstrate an understanding of The Council for Adult and Expe riential Learning Standards for the Assessment of Credit, and the Stevenson University Portfolio Evaluation Process. Sequence Objective Activities Assessment Module Seven Self assess your portfolio based on the CAEL standards and course criteria. Read Chapter 4 Read Appendix 9 (Page 182) Write your self assessment, identifying weaknesses and strengths, any gaps in learning that you have filled or not filled, and that you may have just now recognized and want to fill before submitting for evaluation for credit. Module Eight Demonstrate an understanding of The Council for Adult and Experiential Learning (CAEL) Standards Review the first three standards Open book Exam This course outcome is achieved by taking a final look at the completed pr oject to make a determination regarding what the student did right or wrong, and how it can be fixed before submission for review by a subject matter expert for credit. There is also an assessment to determine how well the student knows the st andards by which they have
23 been, and will be evaluated on future portfolio submissions. Learning is facilitated when new knowledge is integrated into the learner's world ( Merrill, et al., 2007 ) Most of the theoretical assumptions can only be implemented u In other words, only so much can be done outwardly to create an environment that allows for self direction; for justifying the curriculum to the student; for creating a comfort zone and opportunity for revealing or sharing life experie nces; for creating a level of authenticity in a project, and fo r knowing and understanding motivation needs. The critical element is to ensure that the avenues to provide for implementation are persistent and consistent throughout the course. Success dep ends greatly on instructor with itness in the virtual classroom, and the ability to react and respond appropriately. A critical element that will contribute tremendously to this area is ongoing evaluation. Support There are methods or avenues of assistan ce not specifically addressed within the strategies and curriculum above, and they include two discussion board forums which are part of the course 1) the Content Questions forum allows the students to pose questions regarding a specific piece of content (i.e. How do we categorize competencies?). These questions can be answered by the instructor but the desire is f or peers other students to provide direction. 2) The Process Questions forum allows the students to pose questions regarding how to perform a task directed within the course (i.e. How do I upload this assignment?) These two forums, while somewhat different, align direction as discussed above There is also a discussion board forum called The Coffee House which is in the course, but not really part of the course, and can be used for any type of general information
24 amongst the students (and facilitator) possibly as an avenue for emotional suppo rt, although it should be constantly monitored by the facilitator for inappropriate use. There is also a Community Organization site outside of the PLA course for adult accelerated undergraduate students to use primarily as The Coffee House above, but incl udes all students at Stevenson University in this community (online adult undergrad students) support New students are provided an orientation to the school and to the Blackboard learnin g management syst em both in face to face format and online. The online version is called GPS 100 and is required by all students taking online courses.
25 Development The development phase of the project started on December 7 following I RB approval. This phase based upon review by instructional technology experts and usability reviewers. All of the reviewers are current employees of Stevenson University, including a graduate student The expert reviewers included Stevenson University administrative staff and faculty with curriculum and instruction education and background, instructional design and educational technology ed ucation and experience, online curriculum development and delivery experience, as well as a subject matter expert in the areas of prior learning, adult learning, and accelerated higher educational programs. All expert reviewers have advanced degrees and a minimum of ten years experience in higher education. One of these expert reviewers is the Associate Dean of Distance Education and Ed.D. ; she has over 20 years of experience in higher education. Two of the expert reviewers are Instructional Designers wi th over 10 years of experience in higher education; one with an Ed.D., the other an M.Ed. and also a certified Master Reviewer for the Quality Matters (QM) organization. The one faculty expert reviewer has been a fulltime faculty member at Stevenson Unive rsity for over ten years and a certified (QM) Master Reviewer for three years. The content expert is the Associate Dean of Accelerated Undergraduate Programs and a J.D. with more than 20 years of college level teaching experience, and extensive
26 experience with the Co m m i s s i o n for Accelerated Programs (CAP) and the Council for Adult and Experiential Learning (CAEL). She is currently the Vice Chair for CAP. of Graduate and Professional Studies requested the course be expanded from a five week, one credit course to an eight week, three credit course. The development of the modules, pre class through module eight proceeded in order with iterations based on feedback (formative assessment) along the way as necessary. This rapid development phase implemented a process known as successive approximation The expert reviewers were provided instructions (Appendix F and Appendix G) based on their area of expertise and assignment. All course materials were copied from the development course into the actual course web site in Decembe r 2009 following the completed implementation. There was a recommendation by one instructional designer to int roduce the students to the Smart hinki ng support organization earlier in the curriculum. This would prepare the students for using the system before the time when they will be required to use it. Th is same designer provided critical information regarding the Satisfaction Q uestionnaire, identifying that there was not a rating scale incorporated as a measure The author implemented a rating scale and set the criteria of an overall rating of 3.5 or less in any area required consideration for change This was with the exception of question 9 which would require a rating of 3.0 or above since it is written in the
27 A second instructional designer recommended a move from using Microsoft Office On line to submit and review portfolio materials to the use of the Blackboard Discussion Board Feature ; o All of these changes were considered appropriate and incorporated immediately. There were thre e usability reviewers; two are current staff members who perform this same type of task often in their professional positions. O ne of these is a student at the universit y as well. The third reviewer wa s a graduate student but is now employed in the depar Beta testing was conducted and completed prior to winter break (December 18). Usability reviewers were provided instructions using Appendix H, and provided Appendix I to report their findings. Minimal recommendations were offe red during these phases, so much so that the Severity Rating phase (Appendix J) was deemed not necessary by the author. There was pages. The most reported finding was broken links that most likely resulted from copying materials. The links were repaired and the recommendation reported was implemented immediately. The course is delivered via the Blackboard Learning Management System, Version 8. The software is hoste The interface design of the course incorporates a standard template utilized by the University (Figure 1) that provides navigation to eight main areas Annou ncements, Staff
28 Information, Start Here, Syllabus & Documents Mod ules, Discussion Board, Resources My Grades. Figure 1: Stevenson University Standard Navigation Template The features of the learning management system used include the announcement page w hich is viewed whenever the student logs in; the folder and learning unit features will be used to create the module structure; internal and external linking features available on individual pages are used as appropriate, along with the assignment feature for students to submit assignments; email feature; the asynchronous discussion board and the grade center. The interactive (html) syllabus created using Softchalk and imported to Blackboard was determined to be too time consuming to update each cycle and will not be used in future iterations. The module structure is composed of a f older for the week (i.e. Module One), and within the folder are the Course Outcomes that apply which are listed along with the Module Objectives underneath, then a Learning Unit presents information and activities
29 for learning, assignment directions and submission links. A learning unit enables a table of contents feature that supports nonlinear navigation within the learning unit itself. The course site incorporates web design best practices such as the 7 plus or minus 2 rule and also the three click rule as much as possible. The former is based upon psychology research that describes the amount of information a person can effectively process cognitively at once (chunks of seve n to ten pieces of information). The latter is the result of web usability studies that prescribe a site design and architecture that allow a user of a web site to get to the information they need within three clicks of the mouse (Nielsen, 2000). While bo th of these techniques have been the subject of much debate, the candidate believes that in an educational site these are good, if not best, practices. Within the course, wherever appropriate and possible, there is a link to whatever activity or resource may be required next by the student. For instance, in the module where a discussion board scenario is presented there is a link directly to the discussion board area. Instructional design and interface design are also influenced by the Quality bric, which is a research based tool to guide best practices. This document also influenced the structure of the course implementing the Start Here section. The development process implemented the instructional strategies and provide d sufficient opportuni ty for interaction between student and content, student and instructor and student and student. The development schedule of PLA 101 is outlined in Table 9 as a list of milestones.
30 Table 9 present s the development milestones for Prior Learning Assessment Phase Start Date Duration End Date Analysis 11/17/2008 53 1/9/2009 Design 1/15/2009 110 5/5/2009 Development 9/21/2009 111 1/10/2010 Formative Evaluation 10/12/2009 153 3/14/2010 Implementation 3/4 /2010 28 4/1 /2010 Summative Evaluation 3/4 /2010 28 4/1 /2010 The structure of the course was guided by three documents -the Course Map and Syllabus created by the instructor (content expert or SME) and in part by the research The Course Map outlines Module Objectives, Learning Activ ities, Assessments and the Course Outcome(s) partially or fully met during each module. The Course Map helps identify any gaps that exist and elements that might possibly be unnecessary. The development of the modules proceeded in order, with iterations b ased on feedback (formative assessment) along the way as necessary. This development phase implemented to a limited degree a process known as successive approximation This process is supported by Michael Allen in his Guide to eLearning where he maintai ns that moving ahead in several repeated small steps is better than trying to leap ahead in one giant perfect step (Allen, 2003).
31 Changes were based on the consideration of recommendations by expert reviewers and feedback received during formative evaluati on from usability testers and students. Expert reviewers were tasked with reviewing the site and content in terms of their area of expertise and assisting with usability testing for interface design. The three main areas prescribed for review were: the e ffectiveness of the implementation of the theoretical framework; content, in regards to the sequencing, level and coverage (breadth), the effectiveness of selected media and any external resources as well as how well the course meets the stated outcomes; u sability of the interface. Beta testing was conducted by three fulltime staff personnel employed by the University. Beta testers conducted testing on the course in its fully operational environment. They were provided directions from Appendix H and recor ded their findings on Appendix I. They also conducted their investigations independently and did not communicate with any other evaluator until the testing was completed. Upon completion of usability testing the data was collected and compiled and it was determined that using t he Severity Rating (Appendix J) by Jakob Nie recommendations was not necessary as state above. There were very few recommendations.
32 Implementation A signifi cant adjustment to the planned i mplementation phase was necessary after only one student enrolled in the actual course. To compensate for this lack of approved and subsequently implemented to solicit staff, faculty and volunteer students from the School of G raduate and Professional studies normal population Eventually 14 student volunteers were recruited; ultimately, course in the actual environment and provided evaluation material s as r equested These particip ants were briefed on March 4 on the type of feedback that was necessary to complete the research project by obtaining appropriate summative informat ion. Supplemental information regarding how to find the necessary pieces to make d eterminations regarding the summative questionnaire was presented in an n four weeks to assess the areas addressed in the questionnaire and complete the post test. The instruct or was available via the means available in the course, as well as in person. The pre test was comple ted during the initial briefing. Consent forms (Appendix M) were explained and signatures were obtained during this initial brief. The actual course of i nstruction as evaluated can be viewed online at the following url https://blackboard.stevenson.edu/webapps/portal/frameset.jsp This is the Blackboard log in page for Stevenson University. Access to the guest account to view the course will be granted using the following log in: Username: pla & Password: guest. The course should be available after you log in. If you do not see the course, please email John McNally to request access at firstname.lastname@example.org
33 Evaluation Results and Analysis Overall the ratings presented in Table 10 (below) from the Satisfaction Questionnaire are positive and indicate the course was successfully implemented, and probably could be successfully implemented in future semesters. There is one area rated 3.5 and below (except for question 9 which is a high rating at 1.3 because it is fram ed in the negat ive), which would require attention. Question #6 was rat ed somewhat low at an average of 3.3 response overall While many participants chose not to answer certain questions, half chose not to answer question #3, and two chose not to answer question #6, t he author feels there may have been some confusion about what was being asked. Additionally, th e low score on #3 is probably due to the fact that the evaluators were not actually completing the course. A student in the actual course would certainly thin k that the completion of this c ourse would be relevant to his life, and probably livelihood (#6) This however, would be something to keep an eye on in future iterations of the course. the negative and the author now suggests this is not a recommended procedure and should be changed if presented in the future.
34 Table 10 presents the r esults of the Satisfaction Questionnaire for PLA101 The results of the post test scores provided in Table 11 ( below ) indicate that learning took place for the eight participants, with a range of individual increase in performance from 27 to 53 per cent over the pre test. This presents an overall average increase of 35 per cent. This instrument should be utilized over several iterations of the course to determine reliability. The pre test post test instrument is a good indicator of lear ning. T he author, however, intends to also track the outcome of portfolios submitted after future iterations of the course, documenti ng the results of student portfolio s that are submitted and assessed by the appropriate subject matter expert. Rating of question is on the degree to which it is true (5 is highest) 5 4 3 2 1 Avg 1. Did you feel as if you had some control over what you learned as you participated in this course? 4 2 1 4.3 2. Did you have control over the system or did it have control over you (were you able to get to the pages you wanted to get to and perform the functions you needed)? 5 1 1 4.4 3. Were you able to participate in decisions made regarding how you would demonstrate what you had learned? 2 1 1 4. 0 4. Do you feel as if it was clear to you why you were completing each exercise? 4 3 1 4.3 5. Did you have an opportunity to share information you had regarding a previous work or life experience? 5 1 1 1 3.9 6. Was any part of this curriculum relevant to something currently taking place in your life? 1 2 2 1 3.3 7. Did this course involve working on a meaningful project? 5 1 1 4.6 8. Did you accomplish work that was personally rewarding? 4 2 4.7 9. At any time did you feel there was not enough guidance? 2 5 1.3 10. Were the instructor and support organizations helpful? 4 1 1 4.0
35 Table 11 presents the r esults of the pre test and post test Student Pre test Score Post Test Score Change A 11 7 +4 B 7 3 +4 C 6 2 +4 D 8 2 +6 E 6 2 +4 F 10 4 +6 G 10 4 +6 H 14 6 +8
36 Potential for Future Research Most aspects of this course curriculum are perfect for a design based on Adult Learning Theory. The research on Adult Learning Theory as a whole, however, is somewhat limited. There is more research on individual components, such as self directed learners or problem based learning, and this research is mostly qualitative. The downturn in the economy has stimulated growth at our post s econdary institut i ons, many of these students are adults looking for a new career or seeking to improve themselves to keep the job they have. This is a critical time to continue research in adult learning and to provide information or demonstrate new ways to become more successful at providing an optimum environment for adult learning.
37 Lessons Learned This project provided a unique experience in that it provided me the opportunity to wear two hats and serve as instructional designer and content expert at the same time. As primarily an instructional designer, I had always viewed these two areas as very distinct Throughout the course of this project, particularly during the content development, the distinction between the two perspectives became blurred at times. It is difficult to say how and why, but since the content does involve discussion and review of several learning theories that may have something to do with this occurrence. It was an awkward feeling to not be able to make the d istinction having worked on so many courses with a variety of faculty over the years and never having felt that before I can only assume that because of my instructional design experience, I automatically developed content with the design in mind. Prob unexpected, and have a back up plan for everything. I thought that I was fully prepared and ready to forge ahead with m y project when only one person registered for the course which was off pring term. Since that would not yield complete my study. In that process, since I changed the protocol I had to get a new approval from the IRB, which all took away valuable time.
38 List of References Allen, M. (2003). Guide to eLearning. Hoboken, NJ: John Wiley and Sons, Inc. Bamford Rees, D. (2009) Thirty Five Years of PLA In Hart & Hickerson (Ed.). Prior Learning Portfolios: A Representative Collection (pp. 1 10) Dubuque, IA : Kendall/Hunt Professional. Colvin, J. (2009) Earn College Credit Now For What You Know 4 th Ed Dubuque, IA : Kendall/Hunt Professional. Council for Adult and Experiential Learning and National Center for Higher Education Management Systems. Adult Learning in Focus: National and State by State Data Retrieved February 2009 from World Wide Web: http://www.cael.org/adultlearninginfocus.htm Department of Labor (2007). Adult Learners in Higher Education: Barriers to Success and Strategies to Improve Results. Retrieved February 2009 from World Wide Web: http://www.jff.org/Documents/adultlearners.dol.pdf Dillie, B. and Mezack, M. (1991). Identifying predictors of high risk among commun ity college telecourse students [Electronic version] American Journal of Distance Education 5 (1): 24 35 Fidishun, D. Andragogy and Technology: Integrating Adult Learning Theory As We Teach With Technology. Retrieved April (2009) from the World Wide Web: http://frank.mtsu/~itconf/proceed00/fidishun.htm Hart, Denise M. and Hickerson, Jerry H. Prior Learning Portfolios: A Representative Collection Dubuque, IA : Kendall/Hunt Professional, (2009) Hiemstra, R. Self Directed Learning: Present and Future. Retrieved March 2009 from World Wide Web: http://www distance.syr.edu/montrsdl.html Knowles, M., Holton III, E., Swanson, R. The Adult Learner 6 th Ed Burlington, MA: Elsevier, (2005).
39 Merrill, D. Barclay, M. & van Schaak, A. (2007) Prescriptive Principles for Instructional Design In Spector, M., Merrill, D. van Merrinboer, J. & Driscoll, M. (Ed) Handbook of Res earch on Education Communications and Technology 3 rd Ed. 14 (pp. 173 183 ). New York: Routledge/Taylor & Francis Group Nielsen, J. (2000). Designing Web Usabili ty Indianapolis, IN: New Riders Publishing. Nielsen, J. Severity Ratings for Usability Problems. Retrieved August 2009 from the World Wide Web: http://www.useit.com/papers/heuristic/severityrating.html Northwest Regional Educational Laboratory (NWREL). Developing Self Directed Learners Retrieved March 2009 from World Wide Web: http://www.nwrel.org/planning/reports/self direct/index.php Pratt, D., (1988) Andragogy as a Relational Construct. Adult Education Quarterly 38, 160 161 Pusser, Brian. Breneman, David W., Gansneder, Kohl, Kay J., Levin, John S., Milam, John H., Turner, Sarah E. (2007). Returning to Learning: Adults Succession Indianapolis, IN: Lumina Foundation for Education. Retrieved February 2009 from World Wide Web: http://www.luminafoundation.org/publications/ReturntolearningApril20 07.pdf
40 Bibliography Allan, J Snapshot of a Generation: Bridging the theory practice divide with project based learning. Australian Journal of Adult Learning 47 (1): 79 83. Ashburn, E. Mapping the Misunderstood Population of Adult Students Retrieved February 2009 from World Wide Web: http://chronicle.com/weekly/v53/i32/32a03501.htm Crooks, T. The Validity of Formative Assessments. Retrieved March 2009 from World Wide Web: http://www.leeds.ac.uk/educol/documents/00001862.htm Fahy, P. and Ally, M ( 2005 ) Student Learning Style and Asynchronous Computer Mediated Conferencing (CMC) Interaction. The American Journal of Distance Education 19 (1): 5 22. Gallow, D. What is Problem Based Learning? Retrieved April 2009 from the World Wide Web: http://pbl.uci.edu/whatispbl.html Harris, W. Why Colleges Should Welcome the Return of the Boomers Retrieved February 2009 from World Wide Web: http://chronicle.com/weekly/v53/i32/32a03501.htm Hou, H., Chang, K. and Sung, Y. (2007). An Analysis o f Peer Assessment Online Discussions within a Course that uses Project based Learning. Interactive Learning Environments 12 (3): 237 251. Kearsley, G. Andragogy (M. Knowles). Theory into Practice website. Retrieved February 2009 from the World Wide Web : http://tip.psychology.org/knowles.html Michelson, E. and Mandell, A. (2004). Portfolio Development and Assessment of Prior Learning Sterling, VA: Stylus Publishing, LLC. Nunley, C. Community Colleges May Be Losing Their Edge in Educating Adults. Retrieved February 2009 from World Wide Web: http://chronicle.com/weekly/v54/i09/09b01801.htm
41 Pfaffman, J. (2009). Computer Mediated Communications Technologies. In Spector, M., Merrill, D. van Merrinboer, J. & Driscoll, M. (Ed) Handbook of Research on Education Communications and Technology 3 rd Ed. 14 (pp. 173 183 ). New York: Routledge/Taylor & Francis Group Richardson, J. and King, E. Adult Students in Higher Education burden or boon? [Electronic version] Journal of Higher Education 69 (24): 65 Sander, L. Colleges Woo Adults Who Have Some Credits but No Degree. Retrieved February 2009 from World Wide Web: http://chronicle.com/weekly/v54/i23/2302901.htm University of Maryland University College. EXCEL: Experiential Portfolio Development at UMUC Retrieved February 2009 from World Wide Web: http://www.umuc.edu/prog/ugp/prior/faq.shtml Zanich, M.L. (1991). Learning styles/teaching styles, Unpublished Manuscript, Indiana University of Pennsylvania, Teaching Excellence Center, Indiana, P A. Retrieved March 2009 from World Wide Web: http://www.coe.iup.edu/rjl/instruction/cm150/selfinterpretation/kolb.htm
43 Appendix A: Kolb Experiential Learning Theory Model Figure2 Kolb Experiential Learning Theory Model 1. Concrete Experience What happened? 2. Reflection and observation What did you observe? 3. Abstract conceptualization Were there any rules or concepts th at apply? 4. Active Experimentation How did this experience affect what you did in future similar situations? Active Experimentation Concrete Experience Abstract Conceptualization Reflection & Observation Experiential Learning Theory
44 Appendix B: Kolb Learning Styles Diverging (feeling and watching CE/RO) These people are able to look at things from different perspectives. They are sensitive. They prefer to watch rather than do, tending to gather information and use imagination to solve problems. They are best at viewing concrete situations from several dif ferent viewpoints. Kolb called this style 'Diverging' because these people perform better in situations that require ideas generation, for example, brainstorming. People with a Diverging learning style have broad cultural interests and like to gather infor mation. They are interested in people, tend to be imaginative and emotional, and tend to be strong in the arts. People with the Diverging style prefer to work in groups, to listen with an open mind and to receive personal feedback. Assimilating (watching a nd thinking AC/RO) The Assimilating learning preference is for a concise, logical approach. Ideas and concepts are more important than people. These people require good clear explanation rather than practical opportunity. They excel at understanding wid e ranging information and organizing it into a clear logical format. People with an Assimilating learning style are less focused on people and more interested in ideas and abstract concepts. People with this style are more attracted to logically sound theo ries than approaches based on practical value. This learning style is important for effectiveness in information and science careers. In formal learning situations, people with this style prefer readings, lectures, exploring analytical models, and having t ime to think things through.
45 Appendix B: Kolb Learning Styles (Continued) Converging (doing and thinking AC/AE) People with a Converging learning style can solve problems and will use their learning to find solutions to practical issues. They prefer technical tasks, and are less concerned with people and interpersonal aspects. People with a Converging learning style are best at finding practical uses for ideas and theories. They can solve problems and make decisions by finding solutions to questions a nd problems. People with a Converging learning style are more attracted to technical tasks and problems than social or interpersonal issues. A Converging learning style enables specialist and technology abilities. People with a Converging style like to exp eriment with new ideas, to simulate, and to work with practical applications. Accommodating (doing and feeling CE/AE) The Accommodating learning style is 'hands on ,' and relies on intuition rather than logic. These p eople use other people's analyse s, an d prefer to take a practical, experiential approach. They are attracted to new challenges and experiences, and to carrying out plans. They commonly act on 'gut' instinct rather than logical analysis. People with an Accommodating learning style will tend to rely on others for information rather than carry out their own analysis. This learning style is prevalent and useful in roles requiring action and initiative. People with an Accommodating learning style prefer to work in teams to complete tasks. They set targets and actively work in the field trying different ways to achieve an objective
46 Appendix C: Multiple Intelligences Howard Gardner Visual Spatial people think in terms of physical space as do architects and sailors, and are v ery aware of their environment They like to draw, do jigsaw puzzles, read maps, daydream. They can be taught through drawings, verbal and physical imagery. Tools include models, graphics, charts, photographs, drawings, 3 D modeling, video, videoconferencing, television, mul timedia, texts with pictures/charts/graphs. Bodily kinesthetic people use the body effectivel y, like a dancer or a surgeon, and have a k een sense of body awareness. They like movement, making things, touching. They communicate well through body language an d should be taught through physical acti vity, hands on learning, acting out, role playing. Tools include equipment and real objects. Musical people show sensitivity to rhythm and sound. They love music, but they are also sensitive to sounds in their enviro nments. They may study better with music in the background. They can be taught by turning lessons into lyrics, speaking rhythmically, tapping out time. Tools include musical instruments, music, radio, stereo, CD ROM, multimedia. Interpersonal people unders tand and interact with others. These students learn through interaction. They have many friends, empathy for others, street smarts. They can be taught through group activities, seminars, dialogues. Tools include the telephone, audio conferencing, time and attention from the instructor, video conferencing, writing, computer conferencing, E mail.
47 Appendix C: Multiple Intelligences Howard Gardner (Continued) Intrapersonal people understand their own interests, goals. These learners tend to shy away from others. They're in tune with their inner feelings; they have wisdom, intuition and motivation, as well as a strong will, confidence and opinions. They can be taught through independent study and i ntrospection. Tools include books, creative materials, diaries, privacy and time. They are the most independent of the learners. Linguistic people use words effectively. These learners have highly developed auditory skills and often think in words. They li ke reading, playing word games, making up poetry or stories. They can be taught by encouraging them to say and see words, read books together. Tools include computers, games, multimedia, books, tape recorders, and lecture. Logical Mathematical people reas on, calculate think conceptually and abstractly and are able to see and explore patterns and relationships. They like to experiment, solve puzzles, ask cosmic questions. They can be taught through logic games, investigations, mysteries. They need to learn and form concepts before they can deal with details.
48 Appendix D: Verbs for Knowledge arrange, define, duplicate, label, list, memorize, name, order, recognize, relate, recall, repeat, reproduce state. Comprehension classify, describe, discuss, explain, express, identify, indicate, locate, recognize, report, restate, review, select, translate. Application apply, choose, demonstrate, dramatize, employ, illustrate, interpret, operate, practice, schedul e, sketch, solve, use. Analysis analyze, appraise, calculate, categorize, compare, contrast, criticize, differentiate, discriminate, distinguish, examine, experiment, question, test. Synthesis arrange, assemble, collect, compose, construct, create, des ign, develop, formulate, manage, organize, plan, prepare, propose, set up, write. Evaluation appraise, argue, assess, attach, choose compare, defend estimate, judge, predict, rate, core, select, support, value, evaluate.
49 Appendix E: PLA Objectives Table 12 Course Outcomes of PLA101 match ed of Educational Objectives Course Outcomes PLA Identify and differentiate types of learning, learning styles and levels of learning. Comprehension & Analysis Conduct research to identify resources, critical events, and other learning events for analysis, description and classification. Comprehension Provide a critical analysis of portfolio elements created by someone else (peer review). Analysis & Evaluation Demonstrate a strong correlation between prior learning experiences and the course learning outcomes of a course selected for Portfolio Assessment by documenting critical and other lea rning events in proper format and writing style. Analysis Develop a written narrative and an autobiography to present evidence and reasonable argument to support the proposed competencies for which the student has claimed mastery. Evaluation Create/As semble an electronic (digital) version of the portfolio to enable Internet presentation/review, and that may also be printed for review. Knowledge Demonstrate an understanding of The Council for Adult and Experiential Learning Standards for the Assessment of Credit, and the Stevenson University Portfolio Evaluation Process. Application
50 Appendix F : Directions to Expert Reviewers (Content/Template) After being provided a copy of the course map, syllabus and outline, expert reviewers for content are to determine if the course outcomes are written at the appropriate level for the college level course (100 to 400 ). They are also asked to review the course materials and other resources and comment if these are adequate or inad equate. Finally, they are asked to confirm that all objectives, content and assessments align. Upon completion of the development phase, these expert reviewers are enrolled in the course for a Final Review. This review entails a quality control che ck on all aspects of the course to ensure any recommendations have been implemented, the template is in place and the course is set up in standard format and presentation.
51 Appendix G : Directions to Expert Reviewers (Course Design/Rubric) After being provide d a copy of the course map, syllabus and outline, design reviewers are provide any recommendations regarding the course materials and other resources. Design reviewers are a sked to confirm that all objectives, content and assessments align. These reviewers will ensure the latest materials (textbook, etc.) are being used and also to determine if there are any supplemental materials available through a Blackboard cartridge, c ompanion web site, etc. These reviewers are also asked to comment on the implementation of the theoretical framework throughout the course. They are to provide feedbac k regarding how appropriately the framework is implemented and make recommendations on a lternative strategies. C onsider these components regarding adult learners as the review takes place: adult learners are self directed need to know why, how and what they are learning have a lot to offer the class through years of experience must know wha t they are learning has immediate application and benefit to life and/or work prefer problem centered instruction over subject centered instruction are motivated primarily intrinsically first, externally second. Upon completion of each section of the c ourse during the development phase, these reviewers are asked to make note of any instructional design and interface issues, items possibly missing or not in proper format as recommended by the Quality Matters Rubric and/or instructional design best practi ces. This is an iterative process that continues throughout the development phase.
52 Appendix H: Directions to Usability Testers You are being asked to assist with the interface design and usability of a course web site. You are assigned to test Modules O ne through Eight and also the administrative areas of Staff Info, Start Here, Syllabus, Modules, Discussion Board, SU Links, and My Grades. You are in this course as a student, and your assignment is to proceed as any student would through all of the mod ules. Please start with the pre class module, as this will give you some idea of what the course is about and what you will have to do. You do not have to complete any assignment. You will, however, go through the motions of completing assignments. In other words, if one of the modules asks you to submit an assignment, you will have an assignment document located in a folder on the desktop with your last name. Any document to su bmit will be in this folder. If you do not see this folder notify me immediately. Also, if an assignment in a module asks you to post to the discussion board, please go through all of the motions that enable you to post on the discussion board (i.e. mak e sure you are able to post). If you are unsure of any directions in any assignment, ple ase make note of the assignment to report later on the Survey for each module.
53 Appendix H: Directions to Usability Testers (Continued) As you proceed through each module, please make notes of anything that causes you to hesitate, causes you to rethink or back up to a previous page, or makes you think you are not sure what to do next. If you have trouble getting to a page, back to a certa in page or just off of the page you are on, make note of it. Please note any links that no do not work correctly, or any direction(s) you do not understand. Is the text hard to read? Does a page take too long to load? After each module, please reflect over the entire process and note if it was a positive experience, negative experience or perhaps just neutral. Then ple ase fill out the appropriate twenty question survey. Surveys are labeled at the top, one for each module/area. NOTE: P lease do this after each module; do not complete all modules and then attempt to complete several questionnaires at once. You should complete no more than two modules, or one module and the administrative sections in a day. So, the entire process should take you three days. You will have one week to complete this testing.
54 Appendix I : Usabilit y Measurement Inventory for Course Website Prior Learning Assessment Portfolio Module One Survey/Feedback Please complete this survey immediately after reviewing/testing Mod ule One. You are encouraged to make notes on any specifics you can underneath the survey chart and on the back of this sheet if necessary Please place a check mark or an X in the box that best indicates how you feel under Agree, Undecided, or Disagree beside each question. Your answers will help make this a better course. Table 13 Feedback form for usability testers Please list any specifics of any experience you can regarding an instance where a n improvement could be made to the web site. Please provide the number (1 20 a bove) that you are referencing as well as the location of the problem (i.e. Module 2 discussion board) and the specific problem (i.e. I click on the discussion board link and it takes me to any empty page). Please use the other side of this sheet to list your specific notes. Module One Agree Undecided Disagree 1. This portion of the web site was easy to use. 2. I could always find what I was looking for. 3. I always knew where I should go next. 4. I could always return to the home page. 5. Sometimes I became frustrated using this site. 6. It took to o 7. I found it difficult to submit assignments. 8. There were always directions when needed. 9. The directions were always easy to follow. 10. I found the design of the course attractive. 11. The site structure made it easy to use. 12. The navigation made sense to me. 13. Navigation is consistent throughout the course. 14. I was able to print any documents I wanted. 15. I could view all media within the course. 16. I was able to view external links with no problem. 17. All links worked properly. 18. All pages loaded quickly. 19. Periodically events took place without warning. 20.
55 Appendix J : Severity Ratings for Usability Problems Severity Ratings in Heuristic Evaluation will be used to determine what action, if any, will be taken to fix identified usability problems. The f ollowing 0 to 4 rating scale will be used to rate the severity and act upon usability problems. After the usability is sues are collected and compiled, three expert reviewers will rate the severity of each issue. Expert reviewers will be asked to rate the issues independently and to not discuss the issues until applied by t hree expert reviewers will determine the action to be taken below. 0 = I don't agree that this is a usability problem at all 1 = Cosmetic problem only: need not be fixed unless extra time is available on project 2 = Minor usability problem: fixing this should be given low priority 3 = Major usability problem: important to fix, so should be given high priority 4 = Usability catastrophe: imperative to fix this before product can be released
56 Appendix K : Pre test Post test for level 2 evaluation (advancing student skills) 1. Knowledge that cannot really be explained or put on paper is known as: a. Tacit Knowledge b. Unknown Knowledge c. Sub conscious Knowledge d. Auto synchronic 2. Musical intelligence is one of the multiple intelligences. T F 3. Learning that takes place outside of the classroom is known as: a. Field Experience b. Experiential Learning c. Adult Learning d. Workforce Experience 4. Multiple Intelligence is a theory developed by: a. Robert Mager b. Howard Gardner c. Albert Einstein d. Malcolm Knowles 5. According to Adult Learning Theory, Adults should be taught differently than Children. T F 6. The following are associated with Adult Learning theory. Adult Learners are: (choose all that apply) a. More Experienced b. Self Directed c. Hon est d. Smarter
57 Appendix K: Pre test Post test for level 2 evaluation (Continued) 7. A competency is similar to a: (Choose all that apply) a. Course Outcome b. Module Objective c. d. Learned Task 8. The organization that helps establish standards for prior learning assessment is: a. Department of Labor b. Council for Adult Experiential Learning c. Maryland Higher Education Committee d. Maryland Occupational Standard Department 9. Higher Education for adults should implement _______________. a. Andragogy b. Pedagogy c. High Standards d. The Council for Adult Education Programs 10. CAEL is an organization that has established standards and assists schools nationwide in the area of assessing ________________________. (Choose all that apply) a. Adult Learning b. Andragogy c. Prior Learning d. Experiential Learning
58 11. The Kolb model assists in the application, planning and facilitation of: a. Adult Learning b. Child Learning c. Traditional Learning d. Experiential Learning 12. One aspect of learning typically lacking or less adequ ate during experiential learning is: a. Observation b. Application c. Theory d. Reflection 13. Upon successful completion of this course, students may submit as many portfolios for courses as they want. T F 14. Competency statements are supported by the: a. Module Objective b. Course Outcome c. Narratives d. Autobiography 15. The Kolb model can be used to analyze ___________ ____________, which can then be documented to demonstrate learning. a. Critical events b. Competency Statements c. Performance Evaluations d. Supporting Documents
59 Appendix L : Satisfaction Questionnaire (Summative Evaluation) Adult Learning (How well implemented?) Please fill out this questionnaire as completely and honestly as you can. The answers to these questions will help build a better course for future students. The higher the rating number you choose, the more confident you are the question is true (except for number 9, which is framed in the negative). 1. Did you feel as if you had some control as you participated in this cour se? Rating: 1 2 3 4 5 Please explain. 2. Did you have control over the system or did it have control over you (were you able to get to the pages you wanted to get to and perform the functions you needed)? Rating: 1 2 3 4 5 Please explain. 3. Were you able to participate in decisions made regarding what you would learn, or how you would demonstrate what you had learned? Rating: 1 2 3 4 5 Please explain.
60 Appendix L: Satisfaction Questionnaire (Continued) 4. Do you feel as if it was clear to you why you were completing each exercise? Rating: 1 2 3 4 5 Please explain. 5. Did you have an opportunity to share information you had regarding a previous work or life experience? Rating: 1 2 3 4 5 Please explain. 6. Was any part of this curriculum relevant to something currently tak ing place in your life? Rating: 1 2 3 4 5 Please explain. 7. Did this course involve working on a meaningful project? Rating: 1 2 3 4 5 Please explain.
61 Appendix L: Satisfaction Questionnaire (Continued) 8. Did you accomplish work that was personally rewarding? Rating: 1 2 3 4 5 Please explain. 9. At any time did you feel there was not enough guidance, even after askin g fo r additional help? Rating: 1 2 3 4 5 Please explain. 10. Would you say the instructor was supportive? Were support organizations helpful (I.e. Presidium, Tech Connection) ? Rating: 1 2 3 4 5 Please explain.
62 Appendix M: Consent Form Informed Consent to Participate in Research Information to Consider Before Taking Part in this Research Study IRB Study # ____Pro0000029_ Researchers at the University of South Florida (USF) and Stevenson University study many topics. To do this, we need the help of people who agree to take part in a research study. This form tells you about this research study. We are asking you to take part in a research study that is called: Systematic Development and Validation of a Course of Instruction in Prior Learning Assessment. The person who is in charge of this research study is John D McNally. This person is called the Principal Investigator. He will be explaining the research t o you. The research will be done at Stevenson University Purpose of the study The purpose of this study is to Determine if the procedures used in the teaching of this course are effective. This study is being conducted as part of a thesis project. Study Procedures If you take part in this study, you will be asked to [ Provide: Answers to a test that will be given before the actual start date of the course, and you will be asked to provide answers to the same test at the end of the course. The test has 15 questions that are: multiple choice, multiple answer and true/false. This test will be given online using the Blackboard learning system. This test is not part of the course and will not affect your grade regardless of whether you complete it or n ot. Answers to a 10 question survey at the end of the course. This survey will be given during module 8. You can answer the questions all at once or save the survey and go back to it several times. This questionnaire is not part of the course and will no t affect your grade whether you complete it or not. The test questions should take no long than 30 minutes each time. The 10 question survey should take no longer than 60 minutes. T hese three activities take place within the normally scheduled course sess ion at Stevenson University
63 No videotaping or audio recording will take place. Appendix M: Consent Form (Continued) This research study is not part of the course Alternatives This research study is not part of the course, your decision to participate or not will have no influence on your grade for this course. There is no alternative study. Benefits R isks or Discomfort This research is considered to be minimal risk. That means that the risks associated with this study are the same as what you face every day. There are no known additional risks to those who take part in this study. Voluntary Participation / Withdrawal You should only take part in this study if you want to volunteer. You should not feel that there is any pressure to take part in the study, to please the investigator or the research staff. You are free to participate in this research or withdraw at any time. There will be no penalty or loss of benefits you are entitled to receive if you stop taking part in this study Your decision to participate or not to participate will not affect your student status or grade Questions, concerns, or complaints If you have any questions, concerns or complaints about this study, call John McNally at 443 352 4045 If you have questions about your rights as a participant in this study, general questions, or have complaints, concerns or issues you want to discuss with someone outside the research, call the Division of Research Integrity and Compliance of the University of South Florida at (813) 974 9343.
64 Appendix M: Consent Form (Continued) Consent to Take Part in this Research Study It is up to you to decide whether you want to take part in this study. If you want to take part, please sign the form, if the following statements are true. I freely give my consent to take part in this study. I understand that by signing this form I am agreeing to take part in r esearch. I have received a copy of this form to take with me. _____________________________________________ ____________ Signature of Person Taking Part in Study Date _____________________________________________ Printed Name of Person Taking Part in St udy Statement of Person Obtaining Informed Consent I have carefully explained to the person taking part in the study what he or she can expect. I hereby certify that when this person signs this form, to the best of my knowledge, he or she understands: What the study is about. What procedures/interventions/investigational drugs or devices will be used. What the potential benefits might be. What the known risks might be. Signature of Person Obtaining Informed Consent Date Printed Name of Person Obtaining Informed Consent
About the Author John D. McNally was born in Gaithersburg, Md. After serving for 20 years in the United States Navy as a Journalist, John retired and started teaching Digital Media Ar ts in high school in Jacksonville, Fl. Once he completed the major part of his studies at the University of South Florida in the Instructional Technology program John moved back to his home State of Maryland and also changed careers. He currently serves as Senior Professiona l Studies and is an adjunct He also conducts local and regional workshops, and presents at national conferences. Medical Laboratory Technician from Montgomery College (with national certification from the American Society of Clinical Pathologists in MLT), Bachelor of Science in Workforce Education from Southern Illinois University, and Master of Arts in Teaching from Jacksonville University.