xml version 1.0 encoding UTF8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchemainstance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam Ka
controlfield tag 001 001461876
003 fts
006 med
007 cr mnuuuuuu
008 040406s2004 flua sbm s0000 eng d
datafield ind1 8 ind2 024
subfield code a E14SFE0000233
035
(OCoLC)54957635
9
AJQ2288
b SE
SFE0000233
040
FHM
c FHM
090
LB3051
1 100
Kerstyn, Christine.
0 245
Teachers' mathematics preparation and eighth grade student mathematics achievement
h [electronic resource] :
can an integrated learning system provide support when teachers' professional preparation is limited? /
by Christine Kerstyn.
260
[Tampa, Fla.] :
University of South Florida,
2004.
502
Thesis (Ph.D.)University of South Florida, 2004.
504
Includes bibliographical references.
500
Includes vita.
516
Text (Electronic thesis) in PDF format.
538
System requirements: World Wide Web browser and PDF reader.
Mode of access: World Wide Web.
Title from PDF of title page.
Document formatted into pages; contains 150 pages.
520
ABSTRACT: Teaching vacancies are increasing nationally and the task of placing an experienced, subjectcertified teacher in the classroom is getting more difficult for school districts. About 23% of all secondary teachers do not have a minor in their main teaching field. This is true for more than 30% of mathematics teachers and the proportion of such teachers is much greater in highpoverty schools and lowerlevel classes. In schools with the highest minority enrollments, students have less than a 50% chance of getting a science or mathematics teacher who holds a license and a degree in the field which he or she teaches. While placement of probationary teachers may help to relieve the shortages of licensed teachers, school districts should consider the consequences of placing these teachers in the classroom. One solution school districts have looked to is the use of technology. The Integrated Learning System (ILS) is a virtual classroom which may offer a solution for school districts not able to fill teaching vacancies with a fully prepared teacher. The focus of this study was on the impact of the ILS and teachers' mathematics preparation on 8th grade student achievement. Hierarchical linear modeling was used to analyze existing data. The participants included 1223 students in 76 classes taught by 30 teachers. The results indicated that 8th grade student achievement in ILS classes was significantly higher compared to classes not using the ILS. When teachers' preparation in mathematics was added in to the model, the impact of the ILS was reduced. Results from this study suggested that the ILS may be beneficial for MJ3 students and that the ILS may offer school districts an alternative technique in raising student mathematics achievement, specifically with lowincome or minority students. In addition, the ILS may be a practical solution for school districts when dealing with teacher vacancies in mathematics classrooms. In regard to teacher preparation, the results from this study confirm the importance for school districts to hire mathematics teachers with knowledge of mathematics content and pedagogy; support the belief that alternative certification programs should emphasize pedagogy; challenge state licensing boards' option of certifying teachers without documentation of completed subject area courses; and support NCTM's vision of a wellprepared mathematics teacher.
590
Coadviser: Dedrick, Robert F.
Coadviser: Ferron, John
653
ILS.
methods of teaching.
teacher quality.
certification.
690
Dissertations, Academic
z USF
x Measurement and Evaluation
Doctoral.
773
t USF Electronic Theses and Dissertations.
4 856
u http://digital.lib.usf.edu/?e14.233
PAGE 1
TeachersÂ’ Mathematics Preparation and Eighth Grade Student Mathematics Achievement: Can an Integrated Learning System Provide Support When TeachersÂ’ Professional Preparation is Limited? by Christine Kerstyn A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy Department of Measurement and Evaluation College of Education University of South Florida CoMajor Professor: Robert F. Dedrick, Ph.D. CoMajor Professor: John Ferron, Ph.D. John A. Hilderbrand, Ph.D. Sherman Dorn, Ph.D. Date of Approval: March 3, 2004 Keywords: ILS, Methods of teaching, teacher quality, certification Copyright 2004, Christine Kerstyn
PAGE 2
Passion in teaching, and the courage to pursue it, does not grow out of the heads of studentteachers. It has to be fostered and nurtured in teacher education where individuals confront their teaching subject and through which they first meet with children who are learning. A teaching career demands that an individual grow as he or she assumes a more complex role and deepens his or her understanding and experience. (Hugh Sockett, 1994)
PAGE 3
Acknowledgments I would like to thank each of the members of my dissertation committee for their patience and their input directed toward my completing this degree. Each memberÂ’s contribution was unique but ultimately brought closure to this product. To Sherman Dorn, (who knows more about HLM than he lets others to believe), thank you for the challenges in examining HLM in relationship to educational settings. To John Ferron, thank you for your stability which provided calm and balance during this process. Your input often contributed to making connections among the scattered thoughts. To John Hilderbrand, thank you for the opportunity to realize my goals. I sincerely appreciate your confidence. To Robert Dedrick, who initially offered the challenge, thank you for thinking out of the box. Like Pandora, who found hope when all chaos escaped, I believe that your efforts in this process will continue to be a guide.
PAGE 4
i Table of Contents Table of Contentsi List of Tablesv List of Figuresvii Abstract viii Chapter One Introduction1 Definitions6 Chapter Two Review of the Literature7 The Teaching Profession9 National Board for Professional Teaching Standards10 NCATEÂ’s Quality Assurance System for the Teaching Profession11 Teacher Quality12 Definition.12 Subject Area Knowledge12 Professional Preparation13 Teacher Licensing15 Types of certification15 Fully certified15 Alternative certification16 Outoffield teaching16 Teacher Certification In Florida17 Mathematics certification17 Teacher Shortages19 National Trends19 FloridaÂ’s Teacher Shortage20 A Large School District in Florida21 Technology21 Integrated Learning Systems21 Student achievement22 ILS at the College Level23 Support System for New Teachers23 Modeling the Problem in the Study24 Hierarchical Linear Modeling25 Advantages of HLM.25 Characteristics of HLM26
PAGE 5
ii Choosing the number of levels28 Dependence and correlated data29 Power, effect size and sample size31 Summary32 Chapter Three Method35 Sample36 Schools36 Students36 Design38 Variables38 Dependent Variable38 Student Mathematics Achievement38 Independent Variables39 Instructional Method39 Professional Preparation41 Control Variables43 Instructional Time43 Student Prior Achievement44 Analysis44 The HLM Model45 Twolevel model: Learning EnvironmentStudent Model45 Threelevel model: Teacherclassstudent Model47 Summary50 Chapter Four Results52 Descriptive Information for Classes53 NonILS Classes53 ILS Classes54 Two Level Analysis55 Overview55 Unconditional model56 Conditional Model Including the Level1 Covariate57 Conditional Model Including Random Intercepts and Slopes60 Conditional Model Including the Level2 Covariate, Time61 Conditional Model Including the Level2 Predictor, ILS64 Conditional Model Including the Level2 Predictor, Mathematics Preparation65 Conditional Model Including Main Effects68 Conditional Model Including Main Effects and Interaction70 Three Level Analysis71 Overview72 Unconditional model72 Conditional Model Including the Level1 Covariate74 Conditional Model Including Random Intercepts and Slopes76
PAGE 6
iii Conditional Model Including the Level2 Covariate, Time.77 Conditional Model Including the Level2 Predictor, ILS79 Conditional Model Including the Level2 Predictor, Mathematics Preparation80 Conditional Model Adding All Predictors82 Conditional Model Including All Covariates, Predictors, and Interactions84 Summary85 Chapter Five Discussion87 Study Overview87 FCATIssues in Measurement89 Research Question 1 TeachersÂ’ Mathematics Preparation90 Research Question 2 Instructional Method (ILS vs. nonILS)93 Research Question 3 Interaction of ILS and TeachersÂ’ Mathematics Preparation96 Instructional Time96 Recommendations and Further Research 97 TeachersÂ’ Mathematics Preparation97 ILS.98 Method. 100 Implications for Practice102 Teaching Vacancies in Mathematics103 Alternative Certification Choices103 Use of Technology to Support Instruction104 References106 Bibliography113 Appendices119 Appendix A NCATEÂ’S Quality Assurance System for the Teaching Profession120 Appendix B FloridaÂ’s Mathematics Certification Requirements123 Appendix C NCTM Strategies for Teaching124 Appendix D MJ3 Syllabus (Grade 8 Mathematics)126 Appendix E Teacher Survey133 Appendix F Survey Results136 About the Author End Page
PAGE 7
iv List of Tables Table 1 Certification Areas for Teaching M/J Mathematics 3 in Florida18 Table 2 Percent of FloridaÂ’s Fall Vacancies Filled by Teachers Not Certified in the Field of Instruction in Florida 20 Table 3 Percent of Teachers Not Certified in Field of Instruction in a large Florida School District, 19961999 21 Table 4 Grade 8 MJ3 Student Demographics37 Table 5 District Means for 2001 FCAT Math Scale Score and 2000 FCAT Norm Referenced Test Given As A Normal Curve Equivalent Score 39 Table 6 Number of Teachers in Classes by Professional Preparation and Instructional Method From 11 Participating Middle43 Table 7 Number of ILS and NonILS Classes By Time and Teacher Professional Preparation44 Table 8 Descriptive Information for Participating NonILS Classes53 Table 9 Descriptive Information for Participating ILS Classes54 Table 10 Fixed and Random Effects for the 2Level Unconditional Model57 Table 11 Fixed and Random Effects for the 2Level Conditional Model with Level1 Covariate 59 Table 12 Fixed and Random Effects for Conditional Model with Random Slopes and Intercepts61 Table 13 Fixed and Random Effects for the 2Level Conditional Model with Level1 and Level2 Covariates: Prior Math Achievement and Time63 Table 14 Fixed and Random Effects for the 2Level Conditional Model with Level1 and Level2 Covariates and ILS65
PAGE 8
v Table 15 Fixed and Random Effects for the 2Level Conditional Model with Level1 and Level2 Covariates and Professional Preparation67 Table 16 Fixed and Random Effects for the 2Level Conditional Model Including the Predictors 69 Table 17 Fixed and Random Effects for the Fullyfitted 2Level Conditional Model71 Table 18 Fixed and Random Effects for the 3Level Unconditional Model73 Table 19 Fixed and Random Effects for the 3Level Conditional Model with Level1 Covariate75 Table 20 Fixed and Random Effects for 3Level Conditional Model with Random Slopes and Intercepts76 Table 21 Fixed and Random Effects for the 3Level Conditional Model with Level1 and Level2 Covariates: Prior Math Achievement and Time78 Table 22 Fixed and Random Effects for the 3Level Conditional Model with Level1and Level2 Covariates and ILS79 Table 23 Fixed and Random Effects for the 3Level Conditional Model with Level1 and Level2 Covariates and Professional Preparation81 Table 24 Fixed and Random Effects for the 3Level Conditional Model with Level1 and Level2 Covariates and All Predictors83 Table 25 Fixed and Random Effects for the 3Level Model With All Variables and Interactions84
PAGE 9
vi List of Figures Figure 1 Hypothesized study results.33
PAGE 10
vii TeachersÂ’ Mathematics Preparation and Eighth Grade Student Mathematics Achievement: Can An Integrated Learning System Provide Support When TeachersÂ’ Professional Preparation Is Limited? Christine Kerstyn Abstract Teaching vacancies are increasing nationally and the task of placing an experienced, subjectcertified teacher in the classroom is getting more difficult for school districts. About 23% of all secondary teachers do not have a minor in their main teaching field. This is true for more than 30% of mathematics teachers and the proportion of such teachers is much greater in highpoverty schools and lowerlevel classes. In schools with the highest minority enrollments, students have less than a 50% chance of getting a science or mathematics teacher who holds a license and a degree in the field which he or she teaches. While placement of probationary teachers may help to relieve the shortages of licensed teachers, school districts should consider the consequences of placing these teachers in the classroom. One solution school districts have looked to is the use of technology. The Integrated Learning System (ILS) is a virtual classroom which may offer a solution for school districts not able to fill teaching vacancies with a fully prepared teacher. The focus of this study was on the impact of the ILS and teachersÂ’ mathematics preparation on 8th grade student achievement. Hierarchical linear modeling was used to analyze existing data. The participants included 1223 students in 76 classes taught by 30 teachers. The results indicated that 8th grade student achievement in ILS classes was
PAGE 11
viii significantly higher compared to classes not using the ILS. When teachersÂ’ preparation in mathematics was added in to the model, the impact of the ILS was reduced. Results from this study suggested that the ILS may be beneficial for MJ3 students and that the ILS may offer school districts an alternative technique in raising student mathematics achievement, specifically with lowincome or minority students. In addition, the ILS may be a practical solution for school districts when dealing with teacher vacancies in mathematics classrooms. In regard to teacher preparation, the results from this study confirm the importance for school districts to hire mathematics teachers with knowledge of mathematics content and pedagogy; support the belief that alternative certification programs should emphasize pedagogy; challenge state licensing boardsÂ’ option of certifying teachers without documentation of completed subject area courses; and support NCTMÂ’s vision of a wellprepared mathematics teacher.
PAGE 12
1 Chapter One Introduction Teaching vacancies are increasing nationally and the task of placing a teacher with subject and pedagogical training in the classroom is getting more difficult for school districts (Kanstoroom & Finn, 1999; NCES, 1994; Neuman, 1994; Parker, 1992). In 199091, 15% of all schools reported having teacher vacancies they could not fill with a certified teacher in the course or gradelevel taught (Smith, 1995). As suggested by the report of the National Commission on Teaching and AmericaÂ’s Future (1996), more than 50,000 people who lack the training required for their jobs have entered teaching annually on temporary or emergency licenses. About 23% of all secondary teachers do not have even a minor in their teaching field. A minor typically consists of 18 semester hours of completed subject area courses. More than 30% of mathematics teachers do not have a minor in their teaching field. The proportion of such teachers is much greater in highpoverty schools and in lower track classes. In schools with the highest minority enrollments, students have less than a 50% chance of getting a science or mathematics teacher who holds a license and a degree in the field which he or she teaches. Recently, Goldhaber and Brewer (1998) documented that the current demand for teachers on emergency or probationary licenses is significantly higher than the demand in the early 1990's. Placing individuals in teaching vacancies w ith various levels of training magnifies the need to articulate the fundamental meaning of the teaching profession. The National Council for Accreditation of Teacher Edu cation (NCATE) designed a system that recommends better preparation and more rigorous standards for the professional teacher. NCATE is recognized by the U.S. Department of Education as the professional
PAGE 13
2 accrediting body for teacher preparation in the United States (NCATE, 2000). The resulting continuum, The Quality Assurance System, is composed of three stages of teacher preparation that assures the general public of quality in the practice of teaching: preservice preparation, extended clinical preparation and assessment, and extended professional development (National Council for Accreditation of Teacher Education, 1995). While each phase provides teachers with different experiences for growth as professionals, all three phases are grounded in a foundation represented by subject matter and teaching knowledge. This is to ensure that as society requires a higher level of skill and knowledge of all individuals than ever before, teachers must be prepared to help their students increase conceptual understanding and analytical ability. Yet, every year as the school year begins, school administrators struggle to provide a qualified teacher for every classroom. A copy of the Quality Assurance System continuum can be found in Appendix A. While professional organizations call for the placement of teachers with subject area knowledge, teaching knowledge, and clinical experiences in classrooms, teacher shortages have altered that entry level standard. The role of the professional standard is to provide training and direction toward a st ate teaching license. However, not all newly hired teachers take the traditional route toward certification. A newly hired teacher is a college graduate, but may be recruited for a teaching position from outside the college network. As such, these teachers have not had the opportunity for the appropriate recommended professional training. These teachers may not have the minimum number of subject area or methods courses needed for a teaching license. Also, experienced teachers often choose to change their teaching assignments. While they have completed general teaching methods courses, the subject specific methods courses as well as the subject area courses may be missing. In Florida, certification is awarded to individuals upon completion of requirements determined by state regulatory boards. In the case of mathematics, (the focus of this study), the certification area delineates the number of mathematics courses taken by the teacher (FDOE, 2001d). For example, high school certification requires at least twice the number of courses completed successfully as does the middle school
PAGE 14
3 certification in mathematics. The high school certification also requires the applicant to have completed course work in calculus, which is optional for the middle school certification. In addition to subject area preparation, prospective teachers are expected to have completed course work in methods of teaching mathematics, specifically a subjectrelated methods of teaching course. The importance of certification is to signal that an individual has completed sufficient amounts of training to certify them as an expert. Teachers serve communities that value their skills and are satisfied with the regulations imposed by state authorities who assume the responsibility of assuring that their teachers are competent. Changes to the certification process resulting from pressures to alleviate the shortage of teachers highlight the problem that is now widely recognized; there exists a shortage of competent teachers in key fields. One of the purposes of this study was to investigate the effect of teachersÂ’ varying amounts of mathematics and methods of teaching mathematics training on 8th grade studentsÂ’ mathematics achievement. While certification is expected to validate the completion of subjectarea courses, that may not continue to be the case as states are forced to wrestle with the problem of teacher shortages. The relationship between student achievement and certification may no longer address the issue of subject area competence for teachers. However, in this study, the relationship between teachersÂ’ varying levels of training in mathematics and completion of methods of teaching mathematics course and student achievement was examined. The problem of teacher shortages and the accompanying variation in teachersÂ’ training is widely recognized and school dist ricts are addressing the problem in a number of ways. One solution school districts have turned to when dealing with teacher shortages is technology. With the growing availability of computers in schools, a strong interest exists in knowing the extent to which these technologies are being used and for what purposes (Greene & Zimmerman, 2000). Multimedia and large storage devices allow technology to provide state of the art learning environments complete with elaborate graphics and user interactivity. One such system is called an Integrated Learning System.(ILS). Integrated Learning Systems have been used as pedagogical
PAGE 15
4 tools since 1980. These sophisticated systems offer a large variety of subjects that individualize and monitor studentsÂ’ instruction. Although they can be relatively expensive for some school districts to implement in the classroom, in a synthesis of evaluation reports, Becker (1990) suggested that students generally do better than expected using the ILS and sometimes the results are substantially superior. The promise of technology in supporting pedagogical changes may help school districts in the placement of newly hired teachers with minimal professional preparation in the classroom (CadieroKaplan, 1999; Tuinman, 2000; Zeon, Lundeberg, Costello, Gajdostik, Harmes, & Roschen, 1999). There are studies that looked at the effect s of teachersÂ’ subject area preparation (Hawk, 1985; NCTM, 1989; Monk, 1994; Da rlingHammond, Wise & Klein, 2000). The results from these studies concluded that a teachersÂ’ knowledge in mathematics as measured by course work and certification was the strongest correlate of student mathematics achievement (with or without controlling for poverty and language). Complimenting these studies on teachersÂ’ mathematics preparation are those studies that looked at the effect of pedagogical strategi es. Begle and Geeslin (1979) found that the number of methods courses in mathematics was a stronger correlate of student mathematics achievement than was the number of credits in mathematics teachers possessed. Findings from studies that looke d at the effect of technology were positive (West & Marcotte, 1994; Van Dusen & Worthen, 1995). The focus of this study was the Integrated Learning System (ILS) that is a result of recent developments in technology. While studies that looked at the effect of the ILS reported positive results (Becker, 1993; Taylor, 1999; West & Marcotte, 1994), conflicting results stemmed from the various ways in which the technology was used (e.g., small group instruction, time spent on computerized instruction). When looki ng at the effect of technology on student achievement, some studies reported significant improvement in mathematics achievement by middle school students when they worked in small groups (Taylor, 1999).
PAGE 16
5 To deal with the shortage of wellprepared mathematics teachers for inner city middle schools, districts have turned to integr ated learning systems. Integrated learning systems are capable of delivering the instruction, providing assessments, and reporting studentsÂ’ progress at regular intervals. Cons equently, the integrated learning system may be one solution for districts dealing with teaching vacancies in mathematics. Although there have been studies that separately looked at teachersÂ’ level of preparation in mathematics and use of technology in the classroom, there have been no studies that have addressed the relationship between teachersÂ’ level of mathematics preparation and use of the integrated learning system. Three concerns were the focus of this study: 1) relationship between teachersÂ’ training in mathematics and student achievement, 2) the effect of an ILS on student achievement, and 3) the interaction between teachersÂ’ training and the use of an ILS. To address these concerns, this study examined the relationship of teachersÂ’ varying levels of mathematics training and student achievement. Teachers with the highest levels of mathematics training were expected to positively influence student mathematics achievement. Also, this study explored wh ether an integrated learning system (ILS) positively affected student achievement when compared to students placed in nonILS classrooms. Lastly, this study investigated student achievement for ILS supported teachers with the lowest levels of mathematics training compared to teachers with higher levels of mathematics training. Essentially, the integrated learning system provided students with content instruction and presented the material using subject specific methods. It was hypothesized that the ILS environment would be more effective for teachers with the lowest level of mathematics training when compared to similar teachers using a nonILS environment, as indicated by student achievement. The following questions were addressed in the study. 1.What effect does the amount of teachersÂ’ mathematics preparation, defined by number of mathematics courses and presence of methods of teaching mathematics course, have on 8th grade studentsÂ’ mathematics achievement?
PAGE 17
6 2.What effect does the instructional method (ILS vs. NonILS) have on 8thgrade studentsÂ’ mathematics achievement? 3.To what extent does an integrated learning system (ILS) interact with a teacherÂ’s level of mathematics preparation, defined by number of mathematics courses and presence of methods of teaching mathematics course, on 8th grade studentsÂ’ mathematics achievement? Definitions Professional Teacher : A teacher with at least 3 years of teaching experience, during which time an appropriate certificate was held for the subject taught. Integrated Leaning System (ILS) : A computer software system that delivers instruction, assesses the studentÂ’s abilities and maintains records. Teacher Licensure : The process of obtaining credentials to provide evidence of subject matter knowledge, pedagogy, and teaching skill. Certification: The possession of a certificate that provides evidence of subject matter competence needed to teach a specific subject in schools. Newly Hired Teacher : An individual who has been hired to teach but does not have any previous experience in the profession. NCATE : National Council for Accreditation of Teacher Education. Preservice teacher : An individual who is in training for a teaching position; usually a college student who has entered the College of Education and is in the process of completing a specific teaching certification track leading to a teaching position upon completion. Probationary Teacher: A newlyhired teacher with less than professional (permanent) certification. Professional Training: Courses leading to an area of certification which includes the content specialty and methods of teaching.
PAGE 18
7 Chapter Two Review of the Literature The intent of this study was to investigate the effects of two factors on student achievement: teacher training and instructional delivery. Teacher training in this study was discussed in terms of the efforts by prof essional organizations to define how teachers are prepared and what they must know to be effective. A professionally prepared teacher has documentation that leads to certification which suggests they are more likely to be successful as teachers. This includes completion of subject area courses and courses in methods of teaching. However, the growing trend in teaching vacancies has individuals entering the teaching profession with various levels of subject area and teaching preparation (FDOE; 2001a; Ingersol & Gruber, 1996). As such, school districts are hiring these individuals without the subject area knowledge and teaching skills recommended by professional organizations. Among the ways some districts have dealt with the problem is in the use of technology. Current trends in technology suggest that student achievement is positively affected by the use of Integrated Learning Systems (ILS). These systems provide the student with instruction, practice, and assessments of their academic progress. The role of the classroom teacher is to support and foster studentsÂ’ progress toward increased subject knowledge. The intent of this study was to explore the effect of the ILS on student achievement in classrooms headed by teachers with various amounts of mathematics training. The literature review presents background information explaining the circumstances that support the need for this study. Topics that provide background information on the teaching profession will focus on the National Board for Professional Teaching Standards and the National Council for Accreditation of Teacher Education
PAGE 19
8 (NCATE) Quality Assurance System. While other organizations provide accreditation to schools with teacher education programs (i.e. Teacher Education Accreditation Council), the NCATE is the only one sanctioned by the United States Department of Education. Teacher quality was defined using current literature on the importance of teachersÂ’ subject area knowledge and professional prepar ation. The rationale for teacher licensing, certification, alternative certification, and outo ffield teaching in light of the national trend in teacher shortages is presented. Current trends in technology are described with a focus on the use of Integrated Learning Systems that have been reported as useful in promoting student achievement. Finally, given the multilevel nature of the data used to address the effects of teacher preparation and ILS on student achievement (i.e., students are nested in classes and classes are nested in teachers), Hierarchical Linear Modeling (HLM) and technical concerns such as modeling data, choosing the number of levels, dependence and correlated data will be discussed. The National Council for the Accreditati on of Teacher Education (NCATE, 1997) reported findings from a recent study that showed that student achievement increased when students had teachers who were trained in developing higher order thinking skills, who were skilled at implementing handson experiences in the classroom, and who were trained to work with special populations. Because subject matter knowledge is necessary to teach effectively, the findings also supported the need for content specific pedagogy to be included as part of teachersÂ’ development. The current research strongly suggested that skilled teachers have a positive effect on student achievement; however, teachers with little or no professional preparation are hired by districts in greater numbers each year to fill teaching vacancies (NCES, 1996, 1999). Technology may offer a solution for school districts in bridging the gap between teacher effectiveness and student achievement, particularly when placing teachers with little or no professional preparation in the classroom. With current advances in technology, the ILS has become a sophisticated learning environment that allows for student interactivity and assessment of their learning. Consequently, the teacherÂ’s role changes from deliverer of knowledge to facilitator of the learning activity. Placing a teacher with little or no subject area knowledge or pedagogical skill in this environment
PAGE 20
9 may be a symbiotic condition for both the teacher and student. The opportunity may allow newly hired teachers (i.e., not trained in mathematics education) the time needed to meet the professional requirements set by the state certification rules without compromising student achievement. In the following literature review, evidence is presented that supports the need for the study. One aspect of the study was to fill the void in research concerning the placement of teachers with little or no professional training in particular kinds of instructional environments afforded by technology. The Teaching Profession The preparation of teachers is a controversial issue. For at least the last 20 years, various groups of individuals with divergent philosophies have debated how to improve the quality of teachers. These groups include federal agencies, state agencies, philanthropic organizations, teachersÂ’ unions, sc hool districts, colleges of education, and parents. Each group had a vision of how to improve the occupation of teaching, however, policy makers understood that parents wanted their childrenÂ’s teachers to impart specific skills and knowledge. The skills and knowledge view of student learning can be advanced by treating teachers as expert technicians who must also be held accountable to others outside the profession for educational outcomes (Kanstoroom & Finn, 1999). This fundamental view of the t eacher as an expert technician overlaps a number of philosophies supported by groups that are instrumental in defining what teachers should know. Teacher preparation in this study was based on the efforts of two professional organizations that have been instrumental in defining how teachers are prepared and what they must know to be effective: the National Council for Accreditation of Teacher Education (NCATE) and the National Counc il of Teachers of Mathematics (NCTM). NCATE organized the fragmented results of a number of groups that participated in developing an aspect of what teachers should know, while the NCTM described teachersÂ’ professional characteristics in the context of the mathematics classroom. In 1986, NCTM presented their standards for the teaching profession, focused on the view that good teachers of mathematics know their subject and that they know how to teach it. By 1995,
PAGE 21
10 NCATE presented a view of the teaching prof ession that included subjectlevel expertise for teachers. The view of teacher as expert did not fundamentally change in that span of 10 years and is not likely to change in the philosophies of professional organizations. The hypotheses in this study are based on NCTMÂ’s assumptions that teachers are key figures in changing what and how mathematics is taught and that teachers should have support to initiate those changes. The following review presents a historical and empirical perspective to defining teacher professionalism. National Board for Professional Teaching Standards In response to the need for policy and programs that addressed the educational reform movement of the 1980's, the Carnegie Forum on Education and the Economy established its Task Force on Teaching as a Profession in 1985. One of its significant recommendations was the formation of the National Board for Professional Teaching Standards (Carnegie Forum, 1986). The National Board for Professional Teaching Standards (NBPTS) was formed in 1987 with support from governors, teachersÂ’ unions, educators, corporate leaders, and concerned citizens. The mission of the NBPTS is to establish high and rigorous standards for what teachers should know and be able to do, certify those teachers who meet the standards, and advance related educational reforms, all for the purpose of improving student learning in the U. S (National Board for Professional Teaching Standards, 1989). Standards for what teachers should know and be able to do are being developed so that they Â“emphasize teacher performanceÂ” (Wise, 1996, p. 192). Wise (1996) reports that the National Council for Accreditation of Teacher Education (NCATE), the Council of Chief State School Officers (CCSSO), and the National Board for Professional Standards (NBPTS) are each developing and implementing standards and assessments that teachers will meet along the path of preparation and continuing development. All three organizations are working together to develop complementary standards, so that preparation standards reflect the skills and knowledge needed for state licensing examinations. While National Board certification complements state licensing, it does not replace it (Buday & Kelly, 1996). State licensing systems set entrylevel standards for beginning teachers while the National Board Certification establishes standards for
PAGE 22
11 accomplished teachers who have completed at least three years of certified teaching. NCATEÂ’s Quality Assurance System for the Teaching Profession The National Council for Accreditation of Teacher Education (NCATE) has been an active participant in educational reform issues. Recently, the Teacher Education Accreditation Council (TEAC) was organized for improving academic degree programs for professional educators. Its support of educational reform issues started in 1997 and its activities should complement the efforts of NCATE that has a long history of participation in the reform movement. Very early on, NCATE recognized the need to align policy and practice of teacher preparation and development. They developed a quality assurance system for the teaching profession. At the time of the reform impetus (1980's), some of the components were already existing and others were beginning to evolve. The resulting quality assurance system emerged as a continuum using three phases: preservice preparation, extended clinical preparation and assessment, and continuing professional development. Appendix A presents a diagram of the continuum which is described in the next section. Phase I, the NCATE/State quality assurance stage, deals with the individualÂ’s entry into the teaching career. This phase begins with the individualsÂ’ recruitment into the teaching profession. Recruitment usually takes place while the individual is enrolled in a college program of study. The program of study should include a liberal arts education, subject matter preparation, professional and pedagogical studies, and clinical studies. The second phase, the extended clinical preparation quality assurance, takes the academically prepared candidate through the stateÂ’s rigorous standards of licensing. This may include more clinical practice, mentoring and professional development that leads to a regular license in the state. The last phase deals with the experienced teachers who must continue professional development in order to renew their regular certificate or progress through the National Board certification process. The entry point in the professionalization of teaching begins when an individual is in college. During this time, a preservice teacher completes the appropriate course work and clinical experiences that leads to certification. Entry points into the continuum of teacher preparation may now vary depending on the individualÂ’s background because
PAGE 23
12 teacher shortages nationwide have school di stricts recruiting teacher candidates from other sources than college campuses. The expectations for all individuals is the same, regardless of the amount of training they en ter the profession with; individuals placed in a classroom are expected to be proficient in their subject and effective in communicating it to students (Koziol, Minick, & Sherman, 1996). Teacher Quality Definition. Teacher quality is a complex phenomenon entailing a number of different definitions. However, in a re port presented by the National Center for Education Statistics (NCES, 1999), two broad elements that characterize teacher quality were presented. One element was teacher preparation and qualification which referred to preservice learning. This includes postsecondary education, certification, teaching assignment and professional development. The second refers to the actual quality of teaching that the teacher exhibits in the classroom. This element involves the teacherÂ’s skill in providing instruction to students in such a way that studentsÂ’ knowledge of the subject increases. The two elements described in the report by NCES are the basis for examining teacher effectiveness in this study. Because teacher preparation is aligned with state licensing standards, certification is an appropriate measure of teacher preparation. It is a measure of subject area courses completed by the teacher that is necessary for the teaching assignment. The second element, teaching preparation, is measured by the number of methods course taken by the teacher. Two courses, general methods and subject specific methods will measure the preparation of teachers in this study. The subsequent review of literature provides support for the use of these elements as descriptors of prepared teachers. Subject Area Knowledge. When the National Council of Teachers of Mathematics (NCTM) released its curriculum standards in 1989, the document described what a highquality mathematics education for North American, K12 students should comprise. The NCTM recognized that the curriculum, as well as the environment where teaching and learning are to take place will be very different from much of what was practiced. As such, the NCTM produced a document in 1991 that described a set of professional standards for teaching mathematics. The Professional Standards for
PAGE 24
13 Teaching Mathematics is based on the following two assumptions: (1) Â“Teachers are key figures in changing the ways in which mathematics is taught and learned in schools. (2) Such changes require that teachers have longterm support and adequate resourcesÂ” (NCTM, 1991, p. 2). Essentially, a well prepared teacher of mathematics must know mathematics, model good mathematics teaching, and know how students learn mathematics. Mathematics teachers with subject area preparation have a positive effect on student achievement. Using data from a 50state survey of policies, state case study analyses, the 199394 Schools and Staffing Surveys (SASS), and the National Assessment of Educational Progress (NAE P), DarlingHammond, Wise and Klein (2000) concluded that subject area knowledge and certification are the strongest correlates of student achievement in mathematics, both before and after controlling for student poverty and language status. Monk (1994) reported that teachersÂ’ subject content preparation, as measured by course work in the subject area was positively related to student achievement in mathematics and science but that the relationship was curvilinear, with diminishing returns for student achievement when teachersÂ’ subject matter courses were above a threshold (e.g., 5 courses in mathematics). MonkÂ’s findings elaborate on a study done by Begle and Geeslin (1972) where they reported that the number of course credits in mathematics that teachers earned was not linearly related to teacher performance as indicated by student achievement. However, Hawk (1985) reported the findings that the relationship between teachersÂ’ training in mathematics and student achievement was greater in higher level mathematics courses. Professional Preparation Curriculum reformers of the late 1980's began to describe environments where Â“goodÂ” teaching took place. The classroom was a community of learners who are engaged in activity, discourse, and reflection, where the teacher provides concrete and contextually meaningful experiences. This type of environment allowed students to raise their own questions and construct models, concepts and solving strategies. School mathematics has a sociocultural perspective and as such requires student and teacher acknowledgment of each otherÂ’s contribution (Telese, 1999). Teachers must be prepared to integrate their studentsÂ’ personal meanings
PAGE 25
14 with content defined mathematical meanings. They must come to understand the value of promoting an active mathematical classroom that models or reflects practices of the wider mathematical community. As such, the classroom teacher is expected to demonstrate a deep and flexible knowledge of subject matter and pedagogical strategies (Confrey, 1993; Dimock & Boethel, 1999; Telese, 1999). Studies in different subject matter areas that compared teachers with and without preparation have typically found higher ratings and greater student gains for teachers who have more formal preparation for teaching. A number of studies suggested that the typical problems of beginning teachers are re duced when adequate preparation prior to entry of the profession is completed by the individual (Adams & Martray, 1980; Glassberg, 1980). Studies of teachers admitted with less than full preparation (e.g., with no teacher preparation or through very short alternate routes) have found that such recruits tend to be less satisfied with their training and they tend to have greater difficulties planning curriculum, teaching, managing the classroom, or diagnosing studentsÂ’ learning needs (DarlingHammond, 1987). In a review of findings of the National Longitudinal Study of Mathematics Abilities, Begle (1979) found that the number of credits a teacher had in mathematics methods courses was a stronger and positive correlate of student performance than was the number of credits in mathematics courses or other indicators of preparation. Similarly, MonkÂ’s (1994) study of studentsÂ’ mathematics and science achievement found that teacher education coursework had a positive effect on student learning and was sometimes more influential than additional subject matter preparation. In a study of more than 200 graduates of a single education program, Ferguson and Womack (1993) examined the influences on 13 dimensions of teaching performance that included education and subject matter course work. They found that the amount of education coursework completed by the teachers explained more than four times the variance in teacher performance than did measures of content knowledge (National Teacher Exam scores and GPA in major). Similarl y, Guyton and Farokhi (1987) found a strong, positive relationship between an individualÂ’s performance on teacher education coursework and teacher performance in the classroom.
PAGE 26
15 DarlingHammond et al. (2000) suggested that the positive effects of subject matter knowledge are augmented or offset by knowledge of how to teach the subject to various kinds of students. The degree of pedagogical skill may interact with subject matter knowledge to bolster or reduce teacher performance. Byrne (1983) suggested that although teachersÂ’ knowledge provides the basis for their effectiveness, the most relevant knowledge will be that which concerns the particular topic taught and the relevant pedagogical strategies for teaching it to a particular kind of student. Teacher Licensing Licensing or certification status is a measure of a teacherÂ’s qualifications that combine aspects of knowledge of subject matter and pedagogy. The following section highlights the kinds of licenses that are available for teachers to get. Types of certification Fully certified. Most states provide administrative regulations that specify the professional knowledge and subject matter content prospective teachers must study in approved teacher education programs (Sch alock, Schalock, Cowart, & Mynton,1993). Consequently, a stateÂ’s certification process has as its purpose the identification of teacher candidates who have a Â“high probability of success in accomplishing the kind of learning outcomes in pupils that are desired by the state and the community in which a teacher is hiredÂ” (Schalock, Schalock, Cowart, & Mynton, 1993, p. 108). Certification status or licensing is a measure of a teacherÂ’s qualifications that combine aspects of knowledge of subject matter and pedagogy. In addition to a bachelorÂ’s degree, teacher certification includes clinical experiences and some type of formal testing and the assessments typically examine whether teachers have a basic college education and expertise in a specialty field (Greene & Zimmerman, 2000). Individuals who meet state teacher licensing regulations are awarded a license in an area that the state certifies and are, as such, fully certified teachers. Alternative certification. Other types of certification teachers may hold are waivers, provisional, probationary, tempor ary, or emergency certificates. Educational reforms that included alternative certification models began in the early 1980's in an attempt to address teaching vacancies (FL DOE, Educator Certification, 2001). State
PAGE 27
16 departments of education are Â“pressured to respond to the critical shortage of teachers by supplying a cadre of employees who hold some type of stateissued certificate to teachÂ” (Fl DOE, Educator Certification, 2001, p. 75). The licenses that allowed for individuals with various levels of preparation to be placed in the classroom took the form of emergency certificates. Consequently, alternative certification is an indication of gaps in professional training. Outoffield teaching. In order to a ddress the issue of qualified teachers, it is important to distinguish between teaching certification and teaching assignment. The substance of assignment focuses on whether teachers are matched to their work assignment by their training, hence the phenom enon of outoffield teaching. Outoffield teaching can be defined as a certified teacher teaching one or more classes without at least an undergraduate or graduatelevel major or minor in the particular subject. Outoffield teaching is extensive in U. S. schools and mathematics is the subject with the highest percentage of outoffield teachers (Ingersoll & Gruber, 1996). Ingersoll and Gruber (1996) reported that in 199091, many students were taught core academic subjects by teachers without adequate qualifications in the fields they were assigned to teach. The study (Ingersoll, 1996) pointed out that this outoffield teaching was not due to a lack of basic education or training on the part of the teachers; in fact, almost all public school teachers hold bachelorÂ’s degrees, about half have graduate degrees, and over 90% are certified. The source of outoffield teaching lay in the lack of fit between teachersÂ’ fields of training and their teaching assignments. The data also provided evidence that many teachers were assigned to teach courses in fields that did not match their formal background preparation and as such about onefourth of all public school students enrolled in mathematics classes in grades 712 were taught by teachers without at least a minor in mathematics or mathematics education. In many fields, students in 7th and 8th grade classes were more often taught by outoffield than were senior high students. Teacher Certification In Florida. Florida awards applicants either a professional or temporary certificate. Alternative certificates are not available. In the report, An Overview of FloridaÂ’s Current Certification System (FL DOE, 2001), a number of
PAGE 28
17 alternative routes to certification are suggested. An applicant can Â“hold a temporary certificate while completing professional preparation requirements in addition to satisfaction of the education competency demonstration and the certification examinationsÂ” (p. 88). The following information is available online at the Florida DOE certification website. There are three steps in the process of obtaining a professional certificate [teaching license] in the state of Florida. All applicants receive the Official Statement of Status of Eligibility This status is awarded if the applicant holds an acceptable bachelorÂ’s or higher degree, meets the specialization requirements, and obtains a 2.5 grade point average on a 4.0 scale in the initial certification subject to be shown on the certificate. During this process, any deficiencies in the area of specialization would be determined and required to be satisfied prior to issuance of a Temporary Certificate If the applicant does not have subject area deficiencies, a temporary certificate will be issued. This signifies that the applicant has the proper academic training for the subject area to be shown on the certificate. To complete the process for a Professional Certificate, the applicant must pass the testing requirements for the certification and successfully complete an approved system for demonstration of professional education competence. Mathematics certification. As established in State Board of Education Rules, Chapter 6A4, Florida Administrative Code, FloridaÂ’s Certification structure offers 64 academic coverages, 6 degreed vocational coverages, one nondegreed vocational coverage, nine academic endorsements and two vocational endorsements (Review of Florida Educator Certification, FL DOE, 2001). A corollary to the certification structure is found in the provisi ons of State Board of Education Rule 6A1.09441, Florida Administrative Code, which establishes the Course Code Directory and Instructional Personnel Assignments. The Course Code Directory provides a list of all approved courses at the elementary, middle and high school levels. It also provides the certification coverages appropriate for an i ndividual to teach the specific course. For example, Table 1 provides a list of subject area coverages and grade levels that are needed by teachers to teach M/J Mathematics 3 (MJ3), the mathematics course appropriate for 8th grade. Otherwise, the state certification board believes the teachers
PAGE 29
18 have not completed enough mathematics courses to have the necessary background for teaching M/J3 Mathematics. Table 1 Certification Areas for Teaching M/J Mathematics 3 in FloridaCertification Areas Allowed for Teaching M/J Mathematics 3 CertificationGrades/Levels Mathematics6th 12th Middle Grades Mathematics5th 9th Middle Grades Integrated Curriculum5th 9thThe three coverages presented in Table 1 represent the 19992000 certification standards in Florida for teaching mathematics in the middle school. Each certification area requires the completion of specific mathematics courses that provides evidence of subject matter knowledge. The minimum number of hours needed to teach the 8th grade mathematics course in Florida is 18 semester hours. Individuals who certify with the Integrated Curriculum must meet the standards required by that certification. However, to teach mathematics, they also must have the 18 semester hours of coursework needed for middle school mathematics certification. The number of semester hours and the type of courses needed for the certification are listed in a table which can be found in Appendix B. The difference between the Middle Grades Mathematics or Integrated Curriculum certification and the secondary Mathematics certification (612) is in the number of courses needed and the level of coursework. The Mathematics certification requires 6 hours of Calculus within the 30 hours of mathematics preparation. Clearly, a teacher whose academic preparation does not meet the state certification requirements for middle school mathematics coverage does not have the course work completed. Teacher Shortages When the National Commission on Teaching & AmericaÂ’s Future was formed in 1994, its purpose was to provide an action agenda for meeting AmericaÂ’s educational challenges in linking higher student achievement with the need for teachers who are knowledgeable, skillful, and committed to meet the needs of all students. In their final
PAGE 30
19 report published in 1996, What Matters Most: Teaching for AmericaÂ’s Future, a plan for recruiting, preparing, and supporting teachers in all of AmericaÂ’s schools was proposed. This plan was aimed at ensuring that all communities have teachers with the knowledge and skills they must have in order to teach. Because the Commission believed that students are entitled to teachers who know their subjects and have the skills to reach all students, they proposed a goal that, by the year 2006, every student in America should have access to competent, caring, qualified teachers in schools organized for success; however, teaching vacancies are increasing nationally and the task of placing an experienced, subjectcertified teacher in the classroom is getting more difficult for school districts. National Trends. In 199091, 15% of all schools reported having teacher vacancies they could not fill with a certified teacher in the course or gradelevel taught (Smith, 1995). The following observations were described in a report of the National Commission on Teaching and AmericaÂ’s Future (1996). More than 50,000 people who lack the training required for their jobs have entered teaching annually on temporary, emergency or substandard licenses. About 23% of all secondary teachers do not have a minor in their main teaching field. This is true for more than 30% of mathematics teachers and the proportion of such teachers is much greater in highpoverty schools and lowertrack classes. In schools with the highest minority enrollments, students have less than a 50% chance of getting a science or mathematics teacher who holds a license and a degree in the field which he or she teaches. Recently, Goldhaber and Brewer (2000) documented that the current demand for teachers on emergency or probationary licenses is significantly higher than the demand in the early 1990's. FloridaÂ’s Teacher Shortage. The 13,436 teachers hired between July 1, 2000 and November 1, 2000 in the state of Florida represent the largest number of newlyhired teachers in the last four years (Critical Shortage Areas:20012002, FL DOE, 2000). This number is about 10.1% of the total workforce in the state during the 20002001 school year. In mathematics, about 11.1% of the total number of teachers were newly hired. As evidenced in Table 2, about 15.5% of the total teaching vacancies in the state were filled with teachers who were not certified in the field of instruction which includes the Basic
PAGE 31
20 Fields, Exceptional Education, and Vocational Education. About 12.6% of the Basic Fields fall vacancies were filled by outoffi eld teachers, the highest percentage in 10 years. In 1999, the number of teachers wit hout certification hired by school districts statewide to fill vacancies in mathematics was almost twice the number as the number hired in the previous year. Table 2 Percent of FloridaÂ’s Fall Vacancies Filled by Teachers Not Certified in the Field of Instruction in FloridaPercentage of Fall Vacancies Filled By Teachers Not Certified in the Appropriate Field1199019921994199619981999 Basic Fields4.07.011.08.49.712.6 Exceptional Education18.7 26.5 27.822.2 27.0 27.6 Vocational5.010.2 12.39.010.9 15.1 Mathematics5.87.511.19.414.320.0 Total7.211.9 14.911.313.115.51 Florida Department of Education. (December 12, 2000). Critical Teacher Shortage Areas: 2001 2002. Online. Available:www.firn.edu/doe/tchdataA Large School District in Florida. During the last four years, the number of vacancies also increased in a large Florida school district that will be the focus of this study. The two years with the highest percent of uncertified teachers were 1996 and 1999 at 12.4% and 10.0%, respectively. Table 3 presents the percent of vacancies filled by uncertified teachers in a large Florida school district from 1996 to 1999. Table 3 Percent of Teachers Not Certified in Field of Instruction in a large Florida School District, 19961999Percent of Vacancies in A Large Florida School District Filled with Teachers Not Certified in Field of Instruction11Fall, 1996Fall, 1997Fall, 1998Fall, 1999 Total Vac.OutofField Total Vac. OutofField Total Vac. OutofField Total Vac. OutofField 50912.4%7368.6%7508.7%84310.0%1 Florida Department of E ducation. New Hires: 20012002. Online: Internet. Available: www.firn.edu/doe/tchdata.
PAGE 32
21 Technology As Districts continue to hire teachers with minimal professional training, technology may help fill the gaps in teacher preparation. Multimedia computer systems, known as integrated learning systems, are designed to deliver the instruction, assess the learning of students, and provide assessment reports. The classroom teacherÂ’s role changes from provider of instruction to facilitator of instruction. Integrated Learning Systems. Since the early 1960's, various forms of technology have been incorporated into schools as instructional tools. From television to distance learning and from drill and practice to interactive multimedia intelligent tutors, updating or placing technology in every school has beco me a national agenda. Student exposure to technology is seen as a critical goal for all schools that reflects the needs of society as well as the students. A type of computer software that recently entered the technological area for use in instructional settings is the Integrated Learning System (ILS). This term has evolved during the last ten years, but essentially in all cases, the computer system delivers the instruction through an interactive method, provides the assessment, and generates reports of student achievement. In a attempt to define models of individualized systems of instruction, Fletcher (1992) defined Intelligent Computer Assisted Instruction (ICAI) as a tutorial program that operates on the studentsÂ’ input in order to generate instructional material, including the questions and answers to them, and responds to inquiries initiated by the student. Consequently, the model for an ICAI has three characteristics. First, an ICAI program must be able to represent the relevant knowledge domain. The program must be able to portray a subject matter expert and convey that information to the learner. Second, it must be able to represent the studentÂ’s state of knowledge. Diagnostic support is provided for instruction while delineating a representation of the studentÂ’s misconceptions. Finally, it must provide the means to transition a student from one state of knowledge to another. West and Marcote (1995) defined an Integrated Learning System (ILS) as a sophisticated form of CAI. ILSÂ’s are computers that network to several microcomputers and usually have a computer management system that controls and keeps track of student
PAGE 33
22 progress. Lessons are carefully planned, sequenced, and incorporate Lessons include branching, frequent feedback and may include diagnostic software. Computer use implies constant student interaction and its ability to individualize and pace individuals is more likely to keep the learner engaged in the learning process. Research suggests that increased time on task in an ILS environment results in increased achievement (Becker, 1993). However, students must be provided with adequate time on the system and the engaged time appears to explain the advantage CAI has over traditional instruction. Student achievement. Integrated Learning Systems (ILS) can transform the classroom if the computer environment is accompanied by increased time on task, effective assessment and reporting, individualized instruction and facilitative teachers (Van Dusen & Worthen, 1995). West and Marcotte (1994) reported the findings of a study they did using an ILS that suggested studentsÂ’ mathematics achievement increased proportionally to the amount of time spent on the computer. The time on task appears to explain the advantage CAI has over traditional instruction because Â“the use of the computer requires constant student interaction and the attention of the student userÂ” (West & Marcotte, 1994, p. 285). They also suggest that the ability to individualize and pace individuals is more likely to keep the learner engaged in the learning process. Waxman and Huang (1997) found similar results from their study suggesting that students in classes where the computer was used were on task significantly more often than students from any other group they studied. When comparing studentteacher interactions, Swan and Mitrani (1993) found that these interactions were more student centered and individualized during computerbased instruction than in traditional teaching and learning environments. Stude nt achievement was the focus of study by Taylor (1999), which found significant improvement in mathematics performance for middle school students on an endofyear exam. There was some evidence that the regularity of use and time spent on the computer had a positive effect on student performance in mathematics. As evidenced by the results of these studies, students who used an ILS are likely to have increased mathematics achievement resulting from increased time on task and individualized instruction.
PAGE 34
23 ILS at the College Level. Very little research has been done at the collegelevel in regard to the ILS. The relationship of the ILS to student mathematics achievement was examined by Tilidetzke (1992) with two instructors using the ILS in an experimental design. The results indicated no statistically significant differences in gains from a pretest to posttest for students using the a computer for instruction; however, a very small sample was reported. The lack of interest by colleges for the ILS may be a result of the growing demand for webbased classes. Students attend their classes through the use of the Internet and communicate with the instructor using the email system. Support System for New Teachers. At least a third of teachers nationwide do not have the degree or license to teach mathematics. The number of probationary licenses is growing significantly with the teacher unlik ely to have professional training. The training afforded by preservice preparation for skilled teachers is generally provided while on the job for individuals with probationary licenses. Research suggests that effective teachers, particularly at the middle or high school levels, rely on prior training experiences in choosing teaching strategies to help students learn (McConney, 1998). Based on the premise that teachers need support and adequate resources to teach, the use of technology is hypothesized as a resource to support teachers with various levels of preparation. Classroom responsibilities for all teachers include instructional delivery, student assessment, classroom management, and professional development. To promote student achievement, practicing professionals have the opportunity to reflect on their practice and increase their teaching skills while beginning teachers rely on their preservice experiences. However, the expectations for all teachers, including those with limited preparation, are reflected in their studentsÂ’ achievement. Advances in technology, such as the ILS, are currently ready to help manage the instructional needs of the classroom without compromising student achievement and consequently support the gaps in professional training of new teachers. The intent of this study was to investigate the effect of the ILS classroom on student achievement considering the classroom teachersÂ’ various levels of professional training. Modeling the Problem in the Study
PAGE 35
24 There are several levels of interest in this study. The goal of the study was to examine student achievement, the relationship between teachersÂ’ mathematics training and student achievement, and the effect of the type of instructional delivery used in the classroom. To analyze the problem required a closer look at the data structure found in educational settings. Among some of the factors that place students into a particular mathematics class is an indicator of their prior achievement. Some of the mathematics classes may be in computer classrooms that integrate the student learning with a specific software, while other mathematics classes are traditionally instructed (i.e., more emphasis on teacher delivering instruction). The literature suggests that students have a greater likelihood of higher achievement when placed in an Integrated Learning environment while in a traditionally instructed classroom; student achievement is affected by the skill and training of the teacher. At the classroom level, student achievement is affected by the teachersÂ’ subject area knowledge and pedagogi cal training. As such, the contribution of the student, classroom, and teacher level variables cannot be ignored when analyzing the data collected in this study. To examine relationships among the variables within the three levels represented by students, classrooms and teachers, statistical techniques such as hierarchical linear modeling (HLM) should be considered. Hierarchical Linear Modeling Hierarchical Linear Modeling (HLM) is a method of analyzing data organized in a nested structure. More levels can be determined by grouping further into the hierarchy of educational settings. Teachers represent a level that groups classes. Levels continue to grow by grouping the teachers by school, the schools by district, and so on. The use of HLM to analyze data from educational settings has some advantages compared to classical statistical approaches. Advantages of HLM. There are several advantages for using HLM to analyze data when compared to classical statistical approaches. One advantage of the approach is that it models the data structure created by educational settings. When students are nested in classes and classes are nested with teachers, data can be collected at each level. These data can be analyzed in the context of the level and in relation to the other levels.
PAGE 36
25 Other advantages of using HLM over other approaches address the issues of level of analysis and aggregation bias. Characteristics associated with individuals who are clustered in groups pose special problems for data analyzed using classical statistical approaches. Classical statistical analyses are likely to be biased if the lack of independence is ignored; however, by taking a multilevel approach these data can be examined (Marsh, Kong, & Hau, 2000). A multilevel model will evaluate studentlevel, classroomlevel, and the betweenlevel effects simultaneously. Some of the problems associated with classical statistical approaches are related to the levels of analysis and aggregation bias. Choosing a level of analysis is dependent on a number of factors. One such factor may be that the data can only be collected a particular level. The use of these data limits the inference to the level of analysis used; however, inappropriate inferences are often directed to other associated levels of the data creating a biased situation. Aggregation bias is one of the most debated and routinely committed error in statistics (Hanushek, Rivkin, & Taylor, 1997). By using HLM, the data are analyzed at the lowest level as well as at other levels using the errors found within and between the levels making inferences possible from any level of the model. Another advantage of using HLM over traditional statistical approaches is related to sample size. Although a large sample was expected, a rule of thumb allows the number of students per class to range from 11 to 28 and the number of classes to range from 30 to 62. Groups consisting of a single observation can be included as long as there are other multiple groups. This means that all the data can be used in the analysis in contrast to classical approaches where the sample number must meet specific criteria. In this study, a number of factors supported the use of HLM for analyzing the data. One consideration is the nested data structure; students were grouped by classes, and by teachers, each with specific characteristics. Students were not randomly placed in classes and teachers were not randomly assigned to classes. The number of students in all the participating classes tended to vary in size with one class having a single enrolled student. The research questions addressed concerns that needed to be answered by two levels of the analysis. The concerns in this study suggested that HLM was an appropriate choice compared to classical statistical methods for analyzing data.
PAGE 37
26 Characteristics of HLM. Hierarchical Linear Modeling (HLM) was designed to handle data consisting of multiple levels or units of analysis. In the present study, Level1 represents the smallest unit of analysis, the students. Within the first level, the analysis uses data which are representative of the student (i.e., prior achievement). Level2 information is represented by the grouping variable, such as the classroom led by the teacher with various amounts of expertise. A large amount of variation within and between the levels can exist and HLM allows for all the variance to be included. Bryk and Raudenbush (1992) suggested that 8090% of the within group variation is lost when aggregating student characteristics within classes. The basic concept behind hierarchical linear modeling is similar to that of Ordinary Least Squares (OLS) regression. At the individual level (level 1), an outcome variable is predicted as a function of a linear combination of one or more level 1 variables, yij = 0j + 1jX1 + .... + rijwhere: 0j represents the intercept of the group, 1j represents the slope of variable X1 of group j, and rij represents the residual for individual i within group j. On subsequent levels, the level 1 slope(s) and intercept become dependent variables being predicted from level 2 variables, as shown in the following models: 0j =00 + 01w1 + ....+ u0j1j = 10 + 11w1 + ... + u1j where 00 and 10 are the intercepts and 01 and 01 are the slopes predicting 0j and 1j, respectively from variable w1 Through this process, the effects of level 1 and level 2 variables can be modeled to reflect the organizational structure of the data. By predicting
PAGE 38
27 the slopes as well as the intercepts (means), an attempt is made to explain the differences in the relationship between level one and level two. There are some problems associated with modeling data as organizational structures. The research on hierarchical modeling in educational settings is limited but sufficient to address data issues in the modeling process for this study. The major concerns are choosing the number of levels, dependence and correlated data, and power, effect size and sample size. Choosing the number of levels. Making a decision about the number of levels needed to represent the data is extremely important. Although HLM is based on Ordinary Least Squares (OLS) analysis, traditional OLS methods that do not take into account the multilevel nature of the data may provide inaccurate estimates of the impact of important predictors. Porter and Umbach (2000) reported that if level 2 or higher group effects are not accounted for in the type of modeling procedure employed, inaccurate coefficients and subsequently poor analyses are likely to be the result. Similar results were noted by Webster, Mendro, Orsak, and Weerasingghe (1998) when OLS methods were compared to multilevel models. A one level model is represented as a linear regression. Because these individuals are drawn from the same classroom or school, they tend to share the same characteristics and data on these individuals may not be fully independent. Osborne (2000) sugge sts that OLS regression produces standard errors that are too small and in turn leads to a higher probability of rejecting a null hypothesis than if an appropriate statistical analysis were performed or if in fact the data were truly independent. The modeling process requires a discriminating approach and a particular sensitivity to the understanding of the relationships that interact within the organizational setting. There are concerns when levels are ignored in the model development. Opdenakker and Van Damme (1998) reported that ignoring a top level in a model can distort the regression coefficients and errors at various levels. For example, when ignoring a top level in the model, 1.an overestimation of the variance belonging to the highest level considered occurs while the others are not affected,
PAGE 39
28 2.the standard error of the variance estimate of the highest level considered is overestimated, and 3.it causes unstable regression coefficients of the explanatory variables belonging to the highest level considered. When an intermediate level is ignored, 4.it causes an overestimation of the variance belonging to the level just above and below the ignored level, 5.the standard error of the variance of the level just under an ignored intermediate level is overestimated whereas the standard error of the variance estimate of the level just above ignored is underestimated, and 6.it causes unstable regression coefficients estimates of the variables belonging to the level just above and just below the ignored level. The standard errors of the intercept estimate appears to be underestimated in models with ignored levels. Consequently, combining the grouping levels may lead to erroneous results from the analysis. In determining whether two or three level models should be used, Webster, Mendro, Orsak, and Weerasingghe (1998) reported that two level models are more convenient and efficient than three level models because they can accommodate more level1 and level2 contextual variables and are not nearly as sensitive to multicolinearity and low variance in conditioning variables as are three level models. Their 3level model would not run in either a onestage or twostage form. The number of contextual variables used in the study ranged from nine to eleven in any one of the three levels. Although not reported by the studyÂ’s authors, the large number of variables at each level may have required a larger sample size for convergence using three levels. However, a 3level model ran successfully when a large number of contextual variables were eliminated from the equations. The conclusion reached by the researchers for their study was that an HLM twostage, two level model with a full range of student and school level contextual variables produced the most biasfree estimates of school effect (level2). The Webster et al.Â’s study used nine level1 variables and eleven level2 variables.
PAGE 40
29 Dependence and correlated data. Stevens (1990) suggests that the assumption of independence is the most important assumption and even a small violation of it produces a substantial effect on both the level of significance and the power of the F statistic. Any dependence among the observations causes the actual alpha to be several times greater than the nominal alpha. The situation is not improved with larger samples but rather gets worse. Stevens (1990) points out that type I error is dramatically affected when dependence among observations occurs. The Intraclass Correlation (ICC) calculated for the data in this study is compared to a table Stevens (1990) used to show how large the actual alpha is to the nominal alpha of .05. Of the 26 ICCÂ’s calculated, 15 were negative values and 11 were positive values. The ICCÂ’s for the positive values ranged from .24 to .83. Given the size of the ICC, number of groups, and number of observations, the actual alphas ranged from .45 to .87. One of the ways to deal with correlated data is to test at a more stringent level of significance. However, the actual alphas are more than five times the nominal alpha which would have suggested using the more stringent alpha of .01. The actual alphas suggested from the correlated data range from 9 to 16 times the nominal alpha of .05 which would require a more stringent test using alpha ranging from .003 to .005. Sheehan and Han (1996) reported that using HLM is recommended when the intraclass correlation is high, because Â‘the parameters are assumed to be unique for each context and are modeled accordinglyÂ’ (p. 4). It is assumed that the context of the grouping level exerts some influence on individuallevel variables resulting in positive intraclass correlations. Sheehan and Han (1996) further state that if the context is ignored in the statistical model, then there is the strong possibility of confounding effects operating at the higher levels of the model. A particular type of multilevel model that is often used to make crosslevel inferences is one in which the regression coefficients are not assumed to be constant for all context. In multilevel models, the parameters are allowed to differ over different groups and are treated as a function of the grouping variables and random, but unique group variations that are assumed to be constant in other classical analyses. In addition
PAGE 41
30 to providing a more realistic model of the data, the random coefficients model is technically also an improvement over conventional regression models because it calculates the correct standard errors. Moreover, the random coefficients model improves the estimation of the parameters for the grouping variable. Power, effect size and sample size. Literature from the last decade was sought to help determine sample size at each level that optimally captures the variance needed to make inferences about the effects at each level. Researchers have begun to describe various methods of choosing their sample sizes at each level in order to ensure a desired level of power given a hypothesized effect size and a chosen level of significance, alpha. Snijders and Bosker (1992) suggest that the total sample size can be determined using the formula, N(n), where N = the number of classes and n = the number of students in the class. To keep the standard error below the desired level they recommend the number of students (n) be less than 28 and the number of classes (N) be greater than or equal to 30, for a given total, N(n). Additionally, if the number of students per class ranges from 11 to 28 and if the number of classes ranges from 30 to 62, the standard error of the crosslevel interaction will also be low. Maximum power for the test of the crosslevel interaction could be obtained by choosing 42 classes and 19 students per class, but the maximum power for the secondlevel could be obtained by choosing an N as large as possible. They conclude that N should be taken as large as possible and that at least an n>10 be used if the more appropriate rule of n>25 cannot. Small within group data are not of great concern to one researcher. Kreft (1992) suggests that groups consisting of a single observation do not prohibit the inclusion of these groups. The only condition is that the data include at least two groups with multiple observations. More recently, Raudenbush and Liu (2000) described a model for determining power in hierarchical linear modeling. The model included a standardized effect size measure that is commonly used in social science research (Cohen, 1988), a standardized measure of siteby treatment variance, and a standardized measure of sitelevel moderating effects. Raudenbush and Liu (2000) suggested that the key elements to the planning of multisite designs include deciding the number of individuals sampled at
PAGE 42
31 each site and the number of sites. They also suggested that sampling a large number of persons per site will increase the precision of the treatment effect estimate at each site. One of their observations is that although the number of groups and the number in each group both contribute to power, the number of groups is more important. Educational researchers appear to agree that a large sample size is needed. However, the issue of sample size for use in HLM analyses is inconclusive. Webster, Mendro, Orsak, and Weerasingghe (1998) reported that a basic 3level model used by the group would not run. In order to successfully run a 3level HLM model, many important contextual variables had to be eliminated from the equations resulting in models that produced unacceptably high correlations with noncontrolled contextual variables. Although the number of students was not give n, the Dallas School District was named as the source of data. A 3level model may be highly desirable; failing to get convergence using the three level model helps support the use of a 2level model and given the existing research, neither model appears to be incorrect. The growth of numerical approaches to the iterative process of calculating the covariance component estimation, in addition to the computer programs that fit the models mathematically, have provided researchers the tools to analyze multilevel data in educational settings (Bryk & Raudenbush, 1992). Summary Teachers normally prepare lesson plans, deliver the instruction, assess students, and manage the classroom. For the professionally prepared teacher, these tasks may be accomplished skillfully and effectively. The literature (DarlingHammond, 2000; Ferguson & Womack, 1993; Monk, 1994;) indicated that teachers with subject knowledge and completed methods of teaching courses have a positive influence on student achievement. Computer environments are suggested to positively impact student achievement. However, whether a teacherÂ’s professional training impacts student achievement with the use of technology has not been addressed. A purpose of this study is to provide empirical evidence that deals with this void in literature. The studyÂ’s hypothesis suggests that teachers with variable amounts of subject area and methods of
PAGE 43
32 teaching courses can positively influence student achievement when using an Integrated Learning System. Figure 1 below describes the studyÂ’s predicted outcome. Figure 1 Hypothesized study results. As shown in Figure 1, it was anticipated that students whose teachers have subject area preparation and methods of teaching courses would have higher academic achievement, regardless of the instructional methods used in the classroom when compared to other teachers with less training. Teachers without the mathematics or methods of teaching courses will positively influence student achievement when using the ILS. All teachers were expected to have high student achieve ment when using the ILS; however, teachers with mathematics and methods of teaching mathematics were expected to have the highest student achievement. Although the ILS delivered the instruction, the highly trained teacher of mathematics is expected to demonstrate a deeper understanding of the subject matter and knowledge and related strategies that will influence student learning (Confrey, 1993; Dimock and Boethel, 1999; Telese, 1999). The role of the classroom
PAGE 44
33 teacher is not reduced when the ILS provides much of the instruction. The teachers of mathematics will be more responsive to problems that occur in student learning (i.e., students asking questions, the use of supplementary material). Because instruction is individualized, studentsÂ’ questions may vary gr eatly within the range of topics found in a specific curriculum. Consequently, teachers with mathematics and methods of teaching mathematics are likely more prepared than teachers with less training to support studentsÂ’ learning with a broader knowledge base.
PAGE 45
34 Chapter Three Method The purpose of this study was to investigate the effects of middle school teachersÂ’ type of professional preparation and instru ctional approach (Integrated Learning System vs. nonIntegrated Learning System) on 8th grade studentsÂ’ mathematics achievement. Teachers with evidence of mathematics and methods of teaching mathematics courses (i.e. teacher preparation) were expected to positively impact student mathematics achievement. The Integrated Learning System (ILS) was hypothesized to positively impact student mathematics achievement with the strongest effect observed for teachers with less preparation. The following research questions were addressed: 1.What effect does the amount of teachersÂ’ mathematics preparation, defined by number of mathematics courses and presence of methods of teaching mathematics course, have on 8th grade studentsÂ’ mathematics achievement? 2.What effect does the instructional method (ILS vs. NonILS) have on 8thgrade studentsÂ’ mathematics achievement? 3.To what extent does an integrated learning system (ILS) interact with a teacherÂ’s level of mathematics preparation defined by number of mathematics courses and presence of methods of teaching mathematics course and 8th grade studentsÂ’ mathematics achievement? Descriptions of the sample, study design, data collection instruments, independent and dependent variables are provided in this chapter. Hierarchical linear modeling (HLM) was used as the primary method of analysis, and therefore, methodological points related to HLM are described.
PAGE 46
1The ILS used in this study was I CAN Learn Education Systems by JRL Enterprises, Inc (www.icanlearn.com). It was indicated by the company that the school district was the first in Florida to place the ILS fo r mathematics instruction. 35 Sample Schools. The school district that participated in the study is among the larger school districts in Florida with a student population of over 150,000. In the spring of 2000, the district was the first in Florida to implement an Integrated Learning System1(ILS) for 8th grade mathematics instruction in middle schools with 60% or more of their students receiving free or reduced price lunc h. Eleven out of 36 middle schools in the district had 60% or more of their students receiving free or reduced lunch paid lunch and these 11 schools provided the data for this study. Schools ranged in size from 615 to 1718 students. Each of the 11 participating middle schools had at least one dedicated computer room with the 8th grade mathematics curricula programmed within the ILS. A total of 76 prealgebra classes named MJ3 Mathematics were used in this study (a copy of the course syllabus is provided in Appendix C). These classes were distributed across the 11 schools as follows: one school had four classe s, one school had five, six schools had six, one school had seven, one school had eight, and one school had 16 classes. Students. All participating students in the study were in the 8th grade and enrolled in MJ3 Mathematics during the 20002001 school year. District enrollment in MJ3 Mathematics was 9,214 students; approximately 38% of that total were enrolled in MJ3 Mathematics at the participating middle schools. MJ3 students were included in the analysis if they had a record of scores from two assessments: the 2000 FCAT Math NRT and the 2001 FCAT Math Sunshine State Standards. Table 4 presents demographic information for four groups of students: students enrolled in nonparticipating middle schools and students enrolled in participating middle schools. Data for these students can be found in the first two columns. Demographics about the participating students were disaggregated to include students with complete data and students enrolled in participating schools without complete data. Students with complete data had a record of
PAGE 47
36 both FCAT Math scores. Students with only one FCAT Math score were withdrawn from the analysis. Data for these students can be seen in Table 4 in the last two columns. As seen in Table 4, percentages of males and females in the participating middle schools were similar to the nonparticipating middle schools in the District. The criterion for selecting schools to implement the ILS program was a high poverty rate at the school; 60% or more of the students at a school were eligible for free or reduced price lunch. Table 4 Grade 8 MJ3 Student DemographicsNonparticipating Middle Schools n=25 Participating Middle Schools n=11 Participating Middle Schools n=11 Complete Data*1Incomplete Data*2n%n%n%n% Gender Male291943.998938.651243.447734.5 Female244136.795537.254245.941329.8 Missing129019.462024.212610.749435.7 Ethnicity White258738.956121.930926.225218.2 Black123218.565825.737832.028020.2 Hispanic134820.367426.334329.133123.9 Other193 2.9 51 2.0 24 2.0 24 2.0 Missing129019.462024.212610.749435.7 SES Eligible278041.8135752.969258.666548.0 Not eligible387058.2120747.148841.471952.0 Instructional Method ILS143656.066556.477155.7 NonILS112844.051543.661344.3 Note. District level data provided for 20002001 school year.*1 Students with two achievement scores used in the analysis*2 Students with less than two achieve ment scores dropped from the analysisConsequently, the schools were located in less a ffluent parts of the district and served a large minority population. Also seen in Table 4, a larger proportion of minority students in the participating middle schools were enrolled in MJ3 than in nonparticipating
PAGE 48
37 middle schools. Approximately 10% more students are eligible for free or reduced price lunch at the participating middle schools than at the nonparticipating middle schools. Design This study was a secondary analysis of existing databases from the tenth largest Florida school district located in west central Florida. Data retrieval for this study was done through the districtÂ’s Department of In formation Services and Division of Human Resources using student and personnel records. Variables Dependent Variable Student Mathematics Achievement. The Florida Comprehensive Assessment Test (FCAT) is an assessment program consisting of various tests that measure reading, writing, and mathematics. The test includes a normreferenced test and a criterionreferenced test that are based on the Florida Sunshine State Standards (SSS) (http://www.firn.edu/doe/menu/sss.htm). Prior to 2001, the FCAT reported results from the FCAT Math SSS as a composite scale score combining the results from the multiple choice part and performance part of the FCAT Math SSS test. The 2001 administration of the FCAT reported results for the multiple choice part of the test only. Technical information about the FCAT was provided from the most current available report; the 1998 FCAT Technical Report presents the psychometric properties of the FCAT Math SSS The report describes the purpose of the FCAT as documentation of Â‘student performance in the areas of Reading and Mathematics as defined by the 1996 Sunshine State StandardsÂ’ (Technical Report, 1999, p. 8). Several reliability scores are provided in the technical report. CronbachÂ’s alpha for the content areas ranged from .89 to .93. The Stratified alpha and the FeldtRaju reliability scores ranged between .90 and .93. More information about the technical considerations of the FCAT can be found in Appendix D. The dependent variable for mathematics achievement for participating students in this study was measured using the 2001 FCAT Math SSS scale score. The 2001 FCAT Math SSS was used as the dependent variable and the 2000 FCAT Math NRT NCE was used as a covariate. As seen in Table 5, the district mean for all 8th grade students taking
PAGE 49
38 the test in 2001 was 318.6 with a standard deviation of 51.5. The correlation between the grade 7 FCAT Math NRT NCE and the grade 8 FCAT Math SSS was .517. Table 5 District Means for 2001 FCAT Math Scale Score and 2000 FCAT Norm Referenced Test Given As A Normal Curve Equivalent Score GradeMeanStandard Deviation nMinMax FCAT Math Scale Score8318.651.511027100500 FCAT Math NRT NCE*1755.023.610847199 Note: Scores are for the entire DistrictÂ’s 7th and 8th grade students.*1 FCAT Norm Referenced Test Given As A Normal Curve Equivalent Score Independent Variables Instructional Method. Two instructional models were examined in the study: the Integrated Learning System (ILS) and the nonILS model. The ILS classroom has computer stations where students logon and begin work on lessons in mathematics. The software presents the lesson which includes diagnostic information from pretests, the lesson presentation, practice and an assessment of the lesson. Students sit at the station, watching the lesson on the monitor while listening to an audio presented through headphones. Lessons are presented using textbased information as well as digitallybased, fullscreen video. Students take notes and practice the material while they work in small groups or individually. The classroom teacher is there to help facilitate the lesson if students need assistance. The teacherÂ’s role is to help students keep on task as they progress toward completing the course. Classroom activities of ILS teachers include: (a) directing students to lessons in the computer, (b) answering questions regarding the lessonÂ’s material, (c) maintaining discipline, (d) discussing with students and parents student progress using computer generated reports, and (e) augmenting the computer lessons with supplemental materials. Students are typically on different lessons when approaching the classroom teacher for assistance.
PAGE 50
2Teachers begin instruction when the tardy bell rings by assi gning a task that students must complete within a few minutes. It may be a review or an introduction to the new lesson that opens the instruction. Typically it is used to maintain classroom discipline and focus the studentsÂ’ attention on the lesson. 3The extent to which the ILS im pacts the teachersÂ’ role in th e classroom was not a research question. 39 With the nonILS instructional method, bell work2 is started by students as they ready for the teacher to begin class. Instruction is generally delivered through lecture. The material presented to students is created by the teacher beforehand. Students take notes and practice the material presented in the lecture. Small groups take shape during the practice session or students may work individually at their desks. The teacher circles the room to provide additional assistance. Homework is assigned and is discussed during the next class. Students must keep pace with the instruction to have the background for new material that will likely be presented the following day. A teacher survey was used to collect data that described differences between teachers using an ILS for instructional delivery and teachers using other instructional strategies. The survey items were based on NCTMÂ’s teaching standards recommended for all teachers of mathematics. The purpose of the survey was to provided descriptive information based on teacher recall about the instructional practices of the participating ILS teachers in contrast to the instructional practices of the participating nonILS teachers. The survey results suggested that both groups of teachers, those using the ILS for instruction and those who do not, reported using a variety of instructional strategies (i.e., lecture, small group instruction, peer teaching, etc). The rates at which these instructional strategies were used by both groups of teachers were similar for all except for one item. As expected, the percent of time technology was used by teachers who did not use the ILS reported was much lower than teachers who used the ILS. The intent of the survey was to provide a brief documentation of how the participating classes were conducted as reported by teachers.3 Tables of survey results can be found in Appendix E.
PAGE 51
40 Any student enrolled in an MJ3 class at one of the 11 middle schools implementing the Integrated Learning System (ILS) and had two FCAT scores was included in the study. Student enrollment in MJ3 is determined by the scheduling process used in the district; student schedules were created by a computer program that assigns the appropriate number of classes based on student course selection. Teacher experience or instructional method were not used as factors when placing students in classes. As a result of the scheduling process, MJ3 students were instructed in one of two instructional methods: an ILS or a nonILS instructional environment. Teachers were assigned to the computer class by personal request or by principal request. The teachers assigned to the ILS instructional method taught the MJ3 curriculum in grade 8 for all their classes each day for the entire year. The 76 MJ3 classes included in the study were taught by 30 teachers. The number of MJ3 classes taught by each teacher ranged between one and four. In the ILS classes, the mean number of students was 24, the median number of students in a class was 25 and the number of students per class ranged from 14 to 35. In the nonILS classes, the mean number of students was 20, the median number of students in a class was 19.5 and the number of students ranged between 9 and 31. Professional Preparation. Existing databases were examined to provide the following descriptive information about the participating teachers. There were 30 teachers who participated in the study: 13 teachers were male and 17 were female. As required by Florida state law, all teachers possessed at least a B. A. degree. The number and type of degree were as follows: (a) two teachers in Biology, (b) three teachers in Elementary Education, (c) ten teachers in Secondary Mathematics, (d) one teacher in Psychology, (e) one teacher in Specific Learning Disabilities, (f) one teacher in Mathematics, (g) nine teachers had degrees that could not be matched to the stateÂ’s 256 B.A. coding system, and (h) three teachers had missing records. Approximately a third of the teachers (11) also earned a MasterÂ’s degree. In Florida, teachers with professional certificates or temporary certificates in mathematics have completed the subject area requirements for certification. Among the areas of certification the Florida DOE recommends teachers have for teaching MJ3
PAGE 52
41 Mathematics are the following: (a) 612 Mathematics (secondary), (b) Middle Grades Mathematics, and (c) Integrated Curriculum. In this study, 16 teachers had a professional certificate, nine teachers had a temporary certificate, and five did not have any professional or temporary teaching certificate. Some teachers had one area of certification. The following single areas were recorded for participating teachers with a professional certificate: (a) one with secondary mathematics, (b) three with middle grades mathematics, and (c) one with middle grades social studies. Some participating teachers with professional certificates had certifi cation in two areas: three had secondary mathematics with a middle grades endorsement and one had psychology and middle grades mathematics. Seven teachers with professional certificates had multiple certifications: four had middle grades mathematics and two to five other areas of certification and three had secondary mathematics, a middle school endorsement and one to four other areas of certification. Participating teachers held temporary certificates in the following areas: (a) one had elementary education, (b) two had secondary mathematics, (c) three had middle grades mathematics, and (d) one had middle grades integrated curriculum. Some teachers with temporary certificates had multiple areas of certification: 1 teacher had biology and middle grades general science and one had political science, and sociology recorded on their certificate. Among the requirements for qualification of a Florida Professional Certificate is an approved method for demonstrating mastery of professional preparation and education competence. Subjectspecific and general methods of teaching courses are typically taken while participating in teacher education programs; these types of classes are not reviewed by the state. In this study knowledge of pedagogy was defined as completion of a methods of teaching mathematics course. Teacher transcripts were reviewed for the study in order to categorize teachers who had completed a methods of teaching mathematics and those who did not. In this study, teacher professional preparation was defined by two components: certification in mathematics needed to teach MJ3 Mathematics and completion of a methods of teaching mathematics course. Table 6 presents the number of teachers in the ILS and nonILS classes by preparation.
PAGE 53
42 Table 6 Number of Teachers in Classes by Profe ssional Preparation and Instructional Method From 11 Participating Middle ILS (n=16)NotILS (n=14) n%n% Gender Male743.8642.9 Female956.2857.1 Professional Preparation Mathematics certification and methods of teaching mathematics 968.8642.9 Mathematics certification only518.8214.3 No mathematics certificati on and no methods218.8635.7 Note. Data represented 30 teachers, 77 classes, and 1223 students.Control Variables Instructional Time. The 11 participating middle schools operated on various time schedules that determined the number of minutes allowed for class. Students had mathematics instruction all year long in one of three time periods: (a) 45minute classes, (b) 50minute classes, and (c) 90minute classes. The 39 ILS classes in the study consisted of: (a) 9 ILS classes held during 45minute periods, (b) 19 ILS classes held during 50minute periods, and (c) 11 ILS classes held during 90minute periods. The 37 nonILS classes in the study consisted of: (a) 15 nonILS classes held during 45minute periods, (b) 9 nonILS classes held during 50minute periods, and (c) 13 nonILS classes held during 90minute periods. While time was determined by three periods of instructional time allowed at a specific school: 45 minutes, 50 minutes and 90 minutes, any questions about existing differences that may have occurred between 45 minute classes and 50 minute classes was resolved during an initial review of the data. It was determined that differences in group means on grade 8 mathematics achievement for teachers using similar instructional methods in 45minutes classes vs. 50minute classes were not statistically significant (p > .05). Class means for ILS teachers with training in mathematics and methods of teaching mathematics in 45 minute classes were similar to ILS teachers with training in mathematics and methods of teaching mathematics in 50 minute classes; similar results
PAGE 54
43 were noted for other matched pairs of classes. Consequently, time was categorized by two levels, less than or equal to 50 minutes (coded 0) and 90minutes (coded 1). Table 7 presents the number of classes taught by ILS teachers and nonILS teachers. Table 7 Number of ILS and NonILS Classes By Time and Teacher Professional PreparationTime Teacher Professional Preparation Mathematics certification + methods of teaching mathematics Mathematics certification only No mathematics certification and no methods ILS 50 minutes 1855 90 minutes740 NotILS 50 minutes 8313 90 minutes715 Note. Data represented 30 teachers, 76 classes, and 1223 students.Student prior achievement. In order to control for prior achievement, the 2000 FCAT Math NRT NCE was collected for participating students. The district mean for the 2000 FCAT Math NRT NCE score was 55.0 with a standard deviation of 23.6. Participating students had a mean of 39.54 and a standard deviation of 17.05. Analysis Data drawn from educational settings are often organized in nested or hierarchically structured models. A numbe r of different scenarios are possible when describing data in educational settings. In this study, students were nested within classes that have particular characteristics and classes were nested within teachers. In this three level model, students represent the lowest level in the hierarchical structure, classes the next level and teachers are the third level. Although a three level model would investigate differences among the teachers, the classes they teach and the students within those classes, there are potential problems with this model given the small number of levels of level2 and level3 units. A two level model would increase the sample at the second level and increase the variation within by including more variables, however, a
PAGE 55
44 two level model may not model the data well. To analyze the hierarchically structured data represented by either the twolevel or threelevel models, HLM was utilized. An HLM analysis is capable of capturing the variation within individual levels and between all levels. In the twolevel model, differences were captured among students in classes and among the classes. In the threelevel model, differences were captured among students in classes, among the classes, and among the teachers. The HLM Model. Research suggests that students differ in achievement as a result of past performance and the following characteristics of the classroom: (a) the time allowed for instruction, (b) the kind of instructional delivery that takes place in the classroom, and (c) the teachersÂ’ professional preparation. HLM allows the analysis to capture the differences among students as well as among the classrooms. While the concept of nested data structure can be easily applied here, the question as to what number of levels may be appropriate for these data is complicated. Given the results provided by other researchers using HLM for the analysis of data, exploratory analyses of alternative models were conducted. The intent of these analyses was to look at student mathematics achievement and its relation to a number of classroom effects that include teachersÂ’ professional preparation and the use of technology for instruction while controlling for amount of instructional time and prior student mathematics achievement. To answer the research questions for this study, the following two models were examined: a twolevel model described as learning environmentstudent and a threelevel model described as a teacherclassstudent. Twolevel model: Learning environmentstudent model. A 2level model was considered based on information drawn from existing research. Level 1 relates to any variables pertaining to the student. Level two includes variables that impact student achievement at the class level. Variables for both levels are defined below. Level One (Student) Â•X1: Prior student math achievement (2000 FCAT NRT NCE Score)
PAGE 56
45 Level Two (Class) Â•W1: Teacher mathematics preparation Â•W2: Instructional method Â•W3: Time for instruction Â•W4: Teacher mathematics preparation X Instructional method Using these defined variables, the model representing level1 is given as: yij = 0j + 1j ( Xij .. ) + rij where: yij represents the 2001 FCAT Math achievement 0j represents the expected 2001 FCAT Math achievement for a student whose value on Xij is equal to the grand mean, 1j represents the expected change in 2001 FCAT Math achievement for a unit change in student prior achievement ( FCAT NRT NCE Score) rij represents the residual for individual i within group j (random error of the prediction). On the subsequent level, the level1 slope ( 1j) and intercept ( 0j) become dependent variables being predicted from level2 variables. Each of the predictor variables was captured around its grand mean. As seen in the following models, 0j = 00 + 01 (Teacher mathematics preparation) + 02 (Instructional method) + 03 (Time for instruction) + 04 (Interaction of teacher mathematics preparation and instructional method) + u0j 1j = 10
PAGE 57
46 where: 00 is the expected 2001 FCAT Math scale mean score for a class assuming all class level variables are centered and the classes are averages, 01 is a regression coefficient based on the relationship between the class mean FCAT achievement ( 0j) and teacherÂ’s class variables, 02 is a regression coefficient based on the relationship between the class mean FCAT achievement ( 0j) and the instructional method (W2j) after controlling for other class variables, 03 is a regression coefficient based on the relationship between the class mean FCAT achievement ( 0j) and instruction time (W3j) after controlling for other class variables, 04 is a regression coefficient based on the relationship between the class mean FCAT achievement ( 0j) and the interaction of teacherÂ’s professional training by instructional method (W4j) after controlling for other class variables, 10 is the fixed value of the slope ( j ) representing the relation between 8thgrade mathematics achievement and 7th grade mathematics achievement, and u0j is the unique effect of class j on the average achievement after controlling for W1j, W2j, W3j, and W4j Threelevel model: Teacherclassstudent model. The data in this study can also be represented as a 3level model. The analysis will likely provide less biased coefficients than the two level model. The threelevel model in this study separated the teacher characteristics from the classroom characteristics. The decision to assign variables to a specific level was based on those variables which were fixed to a condition. TeachersÂ’ professional preparation is a characteristic that teachers carry with them to any school/classroom. Instructional time and the ILS are characteristics of the
PAGE 58
47 school/classroom where teachers work. These characteristics change if teachers work at different locations; however, teachersÂ’ mathematics preparation is not changed by moving to another location. As such, teachersÂ’ mathematics preparation is used in the 3rdlevel of the model and the classroom characteristics, instructional time and use of ILS, remain in the 2nd level. Research on sample sizes for threelevel models does not exist. This study was completed on the assumption that sufficient variability existed among the 30 participating teachers that taught 76 classes for a total of 1223 students. There is a possibility that the sample size is a limitation that may prevent the threelevel model from converging. Of the 30 participating MJ3 teachers, one taught one class, 15 taught two classes, ten taught three classes, and four taught four classes. Should the model converge and provide results of the coefficients, the information when compared to a two level model would provide insight into the effects due to the teacher. The following variables are used to represent the data at each level: Level 1 (Student) Â•X1: Prior achievement Level 2 ( Class) Â•W1: Instructional method Â•W2: Instructional time Level 3 (Teacher) Â•S1: Teacher mathematics preparation Â•S2: Teacher mathematics preparation X Instructional method The model representing level1 is given as: yijk = 0jk + 1jk ( Xijk ... ) + rijkwhere: yijk represents the 2001 FCAT Math Achievement, 0jk represents the intercept of class j with teacher k, 1jk represents the slope of variable X1 of class j with teacher k, and rijk represents the residual for individual i within class j and teacher k.
PAGE 59
48 In the next level, the level1 slope and intercept become dependent variables being predicted from level two variables. Each class level variable was the grand mean centered in the predictor variables. The model representing the 2nd level is given as: 0jk = 00k + 11k(Instructional method) + 12k (Instructional time)+ u0jk and 1jk = 10k where: 00k is the expected 2001 FCAT Math achievement for teacher k, if, all class level variables are centered, 11k is the regression coefficient based on the relationship between the expected FCAT Math achievement mean of teacher k and the class mean represented by instructional method, 12k is the regression coefficient based on the relationship between the expected FCAT Math achievement mean of teacher k and the class mean represented by instructional time, and u11k is the unique effect of class j of teacher k on the class mean achievement after controlling for instructional method. In level three, the level two slope and intercept become dependent variables being predicted from level three variables. Each teacher level variable was the grand mean centered predictor variable. The model representing the 3rd level is given as: 00k = 000 + 001 (Teacher mathematics preparation) + 002 (Interaction of mathematics preparation and instructional method) + t00k 11k = 110where:
PAGE 60
49 000 is the expected 2001 FCAT Math Achievement for the group if all teacher level variables given are centered, 001 is the regression coefficient based on the relationship between the expected class mean FCAT Math achievement and professional preparation after controlling for other teacher variables, 002 is the regression coefficient based on the relationship between the expected class mean FCAT Math achievement and the interaction of professional preparation and instructional method after controlling for other teacher variables, and 110 is the fixed value of the slope ( 11k) across all teachers (pooled withinteachers regression coefficient), and t11k is the unique effect of teacher k on the average achievement after controlling for S1j Summary Past research has indicated that students within classrooms differ in achievement as a result of past performance. Student achievement has also been shown to vary as a result of the characteristics of the classroom that include time allowed for instruction and the kind of instructional delivery that takes place. Research also suggests that teachersÂ’ subject area knowledge is critical in explaining student achievement. These contextual variables are recognized as contributory in predicting student achievement. While the concept of nested data structure can be applied easily here, the question still remains as to what number of levels may be appropriate for this data. Given the evidence provided by other researchers who use HLM, exploratory analyses of the modeling process were advised in order to determine the robustness of the results. While the intent of this study was to look at differences in student mathematics achievement as impacted by teachersÂ’ training in mathematics and the use of technology to deliver instruction, any concerns about the method of analysis need to be addressed as well. Although the three level model more accurately reflects the hierarchical structure of the data, there are very few classes (level 2 units) for each level 3 unit (teachers).
PAGE 61
50 Sparseness of data (i.e., number of classes per teacher ranged from one to four) has been shown to affect estimation and convergence. Therefore, both a 2and 3level model were run. These models are defined as follows: two level model represented by students nested in classes and a three level model represented by students nested in classes which are nested in teachers.
PAGE 62
51 Chapter Four Results The purpose of this study was to investigate the effects of middle school teachersÂ’ preparation in mathematics and instructional method (Integrated Learning System vs. nonIntegrated Learning System) on 8th grade studentsÂ’ mathematics achievement. Teachers with completed courses in mathematics and in methods of teaching mathematics (i.e., teacher mathematics preparation) were expected to positively impact student mathematics achievement. The Integrated Learning System (ILS) was hypothesized to positively affect student mathematics achievement with the strongest effect observed for teachers with less preparation. The following research questions were addressed: 1.What effect does the amount of teachersÂ’ mathematics preparation, defined by number of mathematics courses and presence of methods of teaching mathematics course, have on 8th grade studentsÂ’ mathematics achievement? 2.What effect does the instructional method (ILS vs. NonILS) have on 8thgrade studentsÂ’ mathematics achievement? 3.To what extent does an integrated learning system (ILS) interact with a teacherÂ’s level of mathematics preparation, defined by number of mathematics courses and presence of methods of teaching mathematics course, on 8th grade studentsÂ’ mathematics achievement? To address these research questions, three levels of data were examined in this study: student level, classroom level and teacher level. In both the twoand threelevel models the same dependent variable was used (eighth grade mathematics achievement) as
PAGE 63
52 were the independent variables (i.e., student prior achievement, time, instructional method, mathematics preparation). The major difference in the twoand threelevel models was that the twolevel model combined the teacher characteristics with the classroom characteristics and in the threelevel model the teacher characteristics were separated from the classroom characteristics. The results from the twolevel and threelevel analyses are presented in this chapter. A resolution on the choice of using the twolevel model or threelevel model in relation to the research questions is presented in the summary. Descriptive Information for Classes NonILS Classes. Table 8 presents descriptive information for the 37 participating nonILS classes. As can be seen, class means ranged from 235.4 to 335.1 and standard deviations ranged from 23.98 to 61.82 on FCAT Nineteen of the 37 classes had skewness and kurtosis values in the interval of 1 and 1. Class sizes for these classes ranged from 9 to 22 students. For classes that had skewness and kurtosis values outside the range, the largest skewness was 2.22 and largest kurtosis was 6.29. Table 8 Descriptive Information for Participating NonILS Classes Mathematics PreparationInstructional Time FCAT Math Scale Score NMSDskew.kurt. No mathematics or methods 5012287.448.332.005.29 No mathematics or methods 504277.861.821.552.11 No mathematics or methods 5017307.827.921.282.92 No mathematics or methods 508305.393.751.242.13 No mathematics or methods 5014297.628.890.870.81 No mathematics or methods 5010282.039.960.650.69 No mathematics or methods 5019315.744.640.481.75 No mathematics or methods 5016293.245.600.300.42 No mathematics or methods 5010292.838.680.080.12 No mathematics or methods 5011298.746.680.120.55 No mathematics or methods 509290.331.200.300.71 No mathematics or methods 5022298.938.050.530.81 No mathematics or methods 5014307.537.960.650.50 No mathematics or methods9016235.456.641.932.98 No mathematics or methods9021288.832.961.293.63 No mathematics or methods9017276.540.100.840.40 (table continues)
PAGE 64
53 Mathematics Preparation Instructional Time FCAT Math Scale Score NMSDskew.kurt. No mathematics or methods9015283.127.190.501.10 No mathematics or met hods9015271.923.980.930.96 Mathematics only 5013273.935.132.246.19 Mathematics only 5016274.222.210.650.03 Mathematics only 5017295.335.740.071.18 Mathematics only9019272.652.012.226.29 Mathematics and methods of teaching mathematics 5020273.866.991.823.27 Mathematics and methods of teaching mathematics 5016284.834.231.293.76 Mathematics and methods of teaching mathematics 5018283.136.110.810.15 Mathematics and methods of teaching mathematics 5018313.622.670.790.12 Mathematics and methods of teaching mathematics 509294.937.610.710.64 Mathematics and methods of teaching mathematics 5015300.230.480.441.68 Mathematics and methods of teaching mathematics 5013297.036.220.061.55 Mathematics and methods of teaching mathematics 5020279.134.810.650.32 Mathematics and methods of teaching mathematics909258.277.161.211.03 Mathematics and methods of teaching mathematics9016302.230.771.102.40 Mathematics and methods of teaching mathematics9011284.729.010.370.32 Mathematics and methods of teaching mathematics9016284.331.420.010.52 Mathematics and methods of teachi ng mathematics9015335.127.470.550.92 Mathematics and methods of teachi ng mathematics9013296.622.550.961.27 Mathematics and methods of teachi ng mathematics909270.237.690.971.03 Summary Means NMSDskew.kurt. No mathematics or methods13.929042.50.61.3 Mathematics only16.3279 36.31.32.8 Mathematics and methods of teaching mathematics14.529137.00.41.0Note. The FCAT ranges from 100 to 500 with a mean of 300 (SD=65). The District level mean was 319 (SD=51.5) ILS Classes. Table 9 presents descriptive information for the 39 participating ILS classes. As can be seen, class means ranged from 270.0 to 335.9 and standard deviations ranged from 21.97 to 62.60. Twentythree of the 39 classes had skewness and kurtosis values in the interval 1 and 1. Class sizes for these classes ranged from 9 to 24 students. For classes that had a skewness and kurtosis outside the range, the largest skewness was 3.42 and largest kurtosis was 12.73.
PAGE 65
54 Table 9 Descriptive Information for Participating ILS ClassesMathematics Preparation Instructional Time FCAT Math Scale Score NMSDskew.kurt. No mathematics or methods 5016296.736.170.680.46 No mathematics or methods 5021295.029.360.330.23 No mathematics or methods 509298.222.730.890.44 No mathematics or methods 5018303.836.311.091.68 No mathematics or methods 5016277.936.081.502.58 Mathematics only 5011301.530.000.910.82 Mathematics only 5019290.140.240.690.11 Mathematics only 5022282.033.170.420.61 Mathematics only 5017293.128.200.320.21 Mathematics only 509287.439.930.341.24 Mathematics only9019271.455.041.904.77 Mathematics only9015287.162.601.766.08 Mathematics only9017295.930.140.370.76 Mathematics only9016282.321.970.230.58 Mathematics and methods of teaching mathematics 5021313.623.400.440.27 Mathematics and methods of teaching mathematics 5015330.735.370.420.33 Mathematics and methods of teaching mathematics 5020296.234.070.390.58 Mathematics and methods of teaching mathematics 5022299.034.970.380.15 Mathematics and methods of teaching mathematics 5016304.156.993.4212.73 Mathematics and methods of teaching mathematics 5023289.043.021.814.46 Mathematics and methods of teaching mathematics 5016303.045.661.502.70 Mathematics and methods of teaching mathematics 5023290.234.001.213.02 Mathematics and methods of teaching mathematics 5021306.830.931.043.81 Mathematics and methods of teaching mathematics 5020317.429.060.770.61 Mathematics and methods of teaching mathematics 5022305.832.180.140.36 Mathematics and methods of teaching mathematics 5024310.327.380.140.37 Mathematics and methods of teaching mathematics 5022304.830.020.120.62 Mathematics and methods of teaching mathematics 5023309.328.000.311.06 Mathematics and methods of teaching mathematics 5023298.323.010.370.97 Mathematics and methods of teaching mathematics 5015335.928.290.441.14 Mathematics and methods of teaching mathematics 5018305.625.090.490.51 Mathematics and methods of teaching mathematics 501270.0... Mathematics and methods of teachi ng mathematics 9012297.332.310.111.83 Mathematics and methods of teachi ng mathematics 9015270.940.140.700.06 Mathematics and methods of teaching mathematics 9020291.039.230.690.57 Mathematics and methods of teachi ng mathematics 9021297.125.160.310.14 Mathematics and methods of teachi ng mathematics 9021279.934.540.300.52 Mathematics and methods of teaching mathematics 9020275.448.842.469.05 Mathematics and methods of teaching mathematics 9011299.932.740.912.17 (table continues)
PAGE 66
55 Summary Means NMSDskew.kurt. No mathematics or methods16.0294 32.10.11.1 Mathematics only16.1288 37.90.60.9 Mathematics and methods of teaching mathematics18.630033.90.61.5Note. The FCAT ranges from 100 to 500 with a mean of 300 (SD=65). The District level mean was 319 (SD=51.5) Two Level Analysis Overview. The two level analysis began with the unconditional model (no predictor variables at level one or level two). The covariate, prior mathematics achievement (7th grade FCAT NRT ) was then added to the model. The adjusted 8th grade mathematics achievement class mean (0j) derived from the level one model was then used as the outcome to be explained by the level two variables. Examining different models as each level two variable was added (one at a time) allowed for determining the amount of explained variance contributed by the variable(s) and determining the extent of the variableÂ’s effect. Two variance components were presented in the tables: the variability within the class associated with the ith student in the jth class was given by the random error rij and the variability between classes was given by the random error u0j Unconditional model. In the two level analysis, students represented the first level and classes represented the second level. To determine the amount of variation in mathematics achievement between classes versus the amount of variability within classes, analyses began with an unconditional model. The unconditional means model did not include any predictors and provided a ceiling on the amount of variation in classroom means that could be explained by a level2 factor (Singer, 1998). A comparison between the variance components found in the unconditional model and variance components derived from other models was used to determine the percent of explained variance that was accounted for by various factors. At the first level, studentsÂ’ Grade 8 mathematics achievement was expressed as the sum of an intercept for the studentsÂ’ class (0j) and a random error (rij) associated with the ith student in the jth class:
PAGE 67
56 yij = 0j + rij where rij ~ N ( 0, 2 ). (1) At the second level, the class level intercepts were expressed as the sum of an overall mean (00) and a series of random deviations from that mean: 0j = 00 + u0j where u0j ~ N ( 0, 00 ). (2) Substituting the two equations, results in the multilevel model: yij = 00 + u0j + rij (3) The equation is an example of a mixed model that contains two parts: fixed and random. The fixed part of the model contains 00 the single fixed effect that described the average 8th grade mathematics achievement for 76 classes. The random part contains two random effects: u0j and rij. The random effect that described the variability between class means was u0j and the random effect that described the variability within classes was rij. The results from this model are presented in Table 10. Table 10 Fixed and Random Effects for the 2Level Unconditional ModelFixed EffectCoefficientssetratioPr (t) Intercept, 00 293.31.89154.81<.0001 Random EffectVariance ComponentssezratioPr (z) Intercept, 00177.244.14 4.01 <.0001 Residual, rij1423.159.3823.97 <.0001 Note. Data represented 30 teachers, 76 classes, and 1223 students.The intercept for fixed effect, 293.3 was the average class mean for the 76 classes under study. There are two estimates for the random effects portion of this model. The variance component for class intercepts was 177.2 and the variance component among students within classes was 1423.1. Hypothesis tests of both random estimates indicated that the variance components were significantly different from 0; classes varied in their
PAGE 68
57 average mathematics achievement and even more variation existed among students in classes. The intraclass correlation coefficient (ICC) is found using the variance components. The ICC equals .11 [177.2 / (177.2 + 1423.11)] which indicates some clustering of mathematics achievement existed within classes. Conditional Model Including the Level1 Covariate. The effect of the level1 covariate, Grade 7 mathematics achievement, on Grade 8 mathematics achievement was examined. Prior mathematics achievement was measured using the studentsÂ’ seventh grade FCAT Norm Referenced Test (NRT). The normreferenced test was reported as an NCE score between 1 and 99 points. The covariate, prior mathematics achievement, was centered at the grand mean; the grand mean was subtracted from the studentsÂ’ score resulting in a value that was used in the analysis. At the first level, studentsÂ’ Grade 8 mathematics achievement was expressed as the sum of an intercept for the studentsÂ’ class (0j), the expected change in Grade 8 FCAT Math achievement for a unit change in the studentÂ’s grade 7 mathematics achievement ( 1j), the studentÂ’s grade 7 achievement score centered around the grand mean ( Xij .. ), and a random error (rij) associated with the ith student in the jth class: yij = 0j + 1j ( Xij .. ) + rij (4) At the second level, the class level intercepts are expressed as the sum of an overall mean (00), a series of random deviations from that mean (u0j), and the fixed value of the slope ( j ) across all classes ( 10 ):0j = 00 + u0j (5) and 1j (6) Substituting the two equations, results in the multilevel mixed model:yij = 00 + u0j + rij (7) rij ~ N ( 0, 2 ) and u0j ~ N ( 0, 00 ).
PAGE 69
58 As seen in Table 11, the estimate for the fixed effect intercept in the conditional model was 293.5. This represented the adjusted class mean for mathematics achievement across classes while controlling for studentsÂ’ prior mathematics achievement. There is very little difference between it and the unconditional modelÂ’s class mean (293.3). The estimate for the average regression slope reflecting the relationship between 7th grade and 8th grade FCAT scores was 1.15. On average, student prior mathematics achievement was significantly related to 8th grade mathematics achievement within classes (t = 20.03, p < .0001). Table 11 Fixed and Random Effects for the 2Level C onditional Model with Level1 CovariateFixed Effects CoefficientsetstatisticProb(t) Intercept, 00Unconditional293.31.89154.81<.0001 Conditional w/ Level1 Covariate293.51.38213.11<.0001 Prior Mathematics achievement, 1jUnconditionalConditional w/ Level1 Covariate1.20.0620.03<.0001 Random Effects Variance Components sezstatisticProb(z) Intercept, 00Unconditional177.244.144.01<.0001 Conditional w/ Level1 C ovariate71.923.383.01.0013 Residual, 2Unconditional1423.159.3823.97<.0001 Conditional w/ Level1 C ovariate1101.946.0523.93<.0001 Note. Data represented 30 teachers, 76 classes, and 1223 students.By introducing 7th grade mathematics achievement as a level1 covariate, the variance estimates were expected to be reduced. As suggested by current research (Nunez & Dosett, 2003; Erbe, 2000), including the student level covariate in the model
PAGE 70
59 resulted in a noticeable drop in the variance components. As seen in Table 10, the variance component for the intercept, describing the between class variability, dropped from 177.2 in the unconditional model to 71.9 in the conditional model including the student level covariate. The inclusion of 7th grade mathematics achievement, accounted for [(177.271.9)/177.2 = .59], or 59% of the e xplainable variation between classes (Singer, 1998). The within class variability estimate, the residual, dropped from 1423.11 in the unconditional model to 1101.87 in the conditional model that included the level1 covariate. About 23% [(1423.111101.87)/1423.11 = .23] of the within class variation was explained when grade 7 mathematics achievement was introduced into the model. Hypothesis tests for the variance component for the intercept suggested that after including grade 7 mathematics achievement there was variance to explain and increasing the number of variables in the model was warranted. Conditional Model Including Random Intercepts and Slopes. A model that contained random components for both the intercepts and the slopes was examined. The difference between this model and the previous model that included the level1 covariate is that the slopes (7th grade 8th grade achievement relationship) were free to vary, changing the combined model given in (7) to the resulting combined multilevel mixed model:yij = 00 + ( Xij .. ) + u0j + rij (8) where 00 is the expected grade 8 FCAT Math scale score class mean controlling for student prior mathematics achievement, 10 is the random value of the slope allowed to vary across all classes, and the random effect of the class is represented by u0j and the random effect within the class was represented by the residual associated with rij Results from the previous model suggested that variation in the slopes for 7thgrade 8th grade achievement across classes existed and therefore a model that contained slopes as a random component needed to be examined. The extent to which the intercepts and slopes varied across classes was given by the variance components presented in Table 12.
PAGE 71
60 Table 12 Fixed and Random Effects for Conditional Model with Random Slopes and InterceptsFixed EffectCoefficientssetratioPr (t) Intercept, 00 293.61.38212.31 <.0001 Prior Mathematics Achievement, 1j1.20.0618.80<.0001 Random Effect Variance ComponentssezratioPr (z) Intercept, 0072.924.23.02.001 Slopes, 110.10.041.01.157 Covariance (between intercepts and slopes), 011.20.831.40.161 Residual, rij1423.159.3823.97 <.0001 Note. Data represented 30 teachers, 76 classes, and 1223 students.The variability in intercepts, 00, was 72.94 with a standard error of 24.2 (p = .001). A rejection of the null that the variance component was equal to 0 was made; classes differed in Grade 8 mathematics achievement after controlling for student prior mathematics achievement. The variability in slopes, 11 was 0.05 with a standard error of 0.04 (p=.157). The null hypothesis that the variance component was equal to 0 was not rejected; the relationship between student prior mathematics achievement and mathematics achievement was not significantly different among the classes. The estimate for the covariance component, 01 was 1.17 with a standard error of 0.83. Because the test for the null that the variance component was equal to 0 could not be rejected (p = .161), there was little covariation between intercepts and the slopes. There was no evidence that the effects of 7th grade mathematics achievement on 8th grade mathematics achievement differed across classes. Because the variation in slopes did not differ significantly from 0, the random component for the slopes was not included in any further analyses (i.e., slope was represented as a fixed effect). Conditional Model Including the Level2 Covariate, Time. In order to examine the relation between the adjusted 8th grade achievement and instructional time, the level2 predictor, instructional time, was added to the model. Time was represented by three periods of instructional time: 45 minutes, 50 minutes and 90 minutes. Any question about existing differences that may have occurred between 45 minute classes and 50 minute classes was resolved during an initial review of the data. It was determined that
PAGE 72
61 differences in group means on grade 8 mathematics achievement for teachers using similar instructional methods in 45minutes classes vs. 50minute classes were not statistically significant. Simply, class means for ILS teachers with mathematics and methods of teaching mathematics preparation in 45 minute classes were similar to ILS teachers with mathematics and methods of teaching mathematics preparation in 50 minute classes; similar results were noted for other pairs of classes. Consequently, time was categorized into two levels, less than or equal to 50 minutes (coded 0) and 90minutes (coded 1). The level1 model did not change from the earlier models that were presented (4) and remained defined as: yij = 0j + 1j ( Xij .. ) + rij(9) where, yij represented the predicted grade 8 FCAT Math achievement, 0j represented the expected 2001 FCAT Math achievement for a student whose value on Xij equaled the grand mean, 1j represented the expected change in grade 8 FCAT Math achievement for a unit change in student prior mathematics achievement and rij represented the residual for individual i within group j. The addition of the covariate, instructional time, the level2 model was defined as: 0j = 00 + 1j (Instructional time) + u0j(10) j = 10 (11) Table 13 presents the fixed and random effects for the level2 variable, instructional time. The intercept in the conditional model, 295.7, estimated the mathematics achievement for a class with an average prior mathematics achievement using less than or equal to 50 minutes of instructional time. The fixed effect estimate for instructional time was 7.01 (se = 2.89, tstatistic = 2.43, p = .018), indicating that there was a statistically significant relationship between the amount of instructional time and
PAGE 73
62 the mathematics achievement scores of its students; classes that had 90 minutes of instructional time tended to be 7 points lower on mathematics achievement compared to classes of 50 minutes or less. Table 13 Fixed and Random Effects for the 2Level Conditional Model with Level1 and Level2 Covariates: Prior Mathematics Achievement and TimeFixed Effects VariableModelCoefficientsetstatisticProb(t) Intercept, 00Unconditional293.31.89154.81<.0001 Conditional*1293.51.38213.11<.0001 Conditional*2295.71.62182.93<.0001 Prior Mathematics achievement, 1jUnconditionalConditional*11.20.0620.03<.0001 Conditional*21.11.3819.68<.0001 Instructional Time, 10UnconditionalConditional*1Conditional*27.02.892.430.018 Random Effects Variance Components sezstatisticProb(z) Intercept, 00Unconditional177.244.144.01<.0001 Conditional*171.923.883.01.001 Conditional*263.222.492.81.003 Residual, 2Unconditional1423.159.3823.97<.0001 Conditional*11101.946.0523.93<.0001 Conditional*21101.846.0423.93<.0001 Note. Data represented 30 teachers, 76 classes, and 1223 students. *1 Conditional with Level1 Covariate *2 Conditional with Level1 and Level2 CovariatesAs seen in Table 13, the conditional component for the variance within the class, 2 barely changed with the addition of time. The variance component representing between classes variance (00) dropped slightly to 63.2 when compared to the conditional model that included the level1 covariat e. About 12% [(71.9063.17)/71.90 = .12] of the variation between classes was captured by the level2 covariate, time.
PAGE 74
63 Conditional Model Including the Level2 Predictor, ILS. The variable ILS, represented the instructional component of the class (Integrated Learning System). ILS was measured dichotomously using a value of 1 for teachers using the ILS and a value of 0 for teachers not using the ILS. The level1 model did not change from the earlier model that was presented and remained defined as: yij = 0j + 1j ( Xij .. ) + rij(14) In this model, the Level2 model is defined as: 0j = 00 + 01(Instructional time) + 02(ILS) + u0j(15) j = 10 (16) Table 14 presents the fixed and random effects for the level2 variable, ILS. The intercept in the conditional model, 292.4 estimated the mathematics achievement for a class with an average prior mathematics achievement in a class with less than or equal to 50 minutes of instruction and not using the ILS for instruction. The relationship between the level2 variable, ILS and mathematics achievement was given by the fixed estimate for ILS that equaled 5.9; classes that use the ILS for instructional purposes tended to increase their class mean by almost 6 points on mathematics achievement. With a standard error of 2.63 contributing to a tstatistic of 2.24 (pvalue = .028), a significant relationship existed between the use of an ILS for instruction and the mathematics achievement scores of its students.
PAGE 75
64 Table 14 Fixed and Random Effects for the 2Level Conditional Model with Level1 and Level2 Covariates and ILSFixed Effects VariableModelCoefficientsetstatisticProb(t) Intercept, 00Conditional*1295.71.62182.93<.0001 Conditional*2292.42.17134.55<.0001 Prior Mathematics Achievement, 1jConditional*11.10.0619.68<.0001 Conditional*21.40.0619.69<.0001 Time, 00Conditional*17.02.892.43.018 Conditional*26.52.842.29.028 ILS, 10Conditional*25.92.632.24.028 Random Effects Variance Components sezstatisticProb(z) Intercept, 00Conditional*163.222.492.81.003 Conditional*257.521.492.67.004 Residual, 2 Conditional*11101.846.0423.93<.0001 Conditional*21101.145.9823.95<.0001 Note. Data represented 30 teachers, 76 classes, and 1223 students. *1 Conditional with Level1 and Level2 Covariates *2 Covariates and ILSAs seen in Table 14, the variance component for within the class barely changed when the variable ILS was included. The variance component representing between classes variance dropped from 63.17 in the conditional model including the covariates to 57.45 when ILS was included in the model. About 9% [(63.1757.45)/63.17 = .09] of the remaining variation between classes was captured by the variable, ILS. Conditional Model Including the Level2 Predictor, Mathematics Preparation. Three categories of teachers were defined: (a) teachers with documentation of completed mathematics courses and methods of teaching mathematics, (b) teachers with completed mathematics courses only, and (c) teachers with no documentation of sufficient mathematics courses to teach MJ3 and no methods of teaching mathematics. In order to compare the three groups of teachers, dummy coding of the three categories was done using two variable names, MATH and MATH/METHOD. MATH compared teachers who had completed mathematics courses only to the group of teachers with no
PAGE 76
65 documentation of sufficient subject area content to teach MJ3. MATH/METHOD compared teachers with documentation of mathematics and methods of teaching mathematics to teachers without documentation of mathematics and methods of teaching mathematics. The level1 model did not change from the earlier models that were presented (4) and remained defined as yij = 0j + 1j ( Xij .. ) + rij .(19) The level2 model was defined as: 0j = 00 + 01(Instructional time) + 02 (MATH) + 03(MATH/METHOD) + u0j (20) j = 10 .(21) Table 15 presents the fixed and random effects for the level2 variable, MATH/METHOD and MATH. The relationship between the level2 variable (MATH/METHOD) describing teachers with mathematics and methods of teaching mathematics was given by the coefficient that equaled 6.3; classes taught by teachers with mathematics and methods of teaching mathematics preparation tended to have higher class means, by over 6 points on class mathematics achievement compared to teachers without mathematics preparation (subject matter or methods). With a standard error of 3.00 contributing to a tstatistic of 2.08 (pvalue = .041), a significant relationship existed between teachers with methods of teaching mathematics preparation on the mathematics achievement scores of their students.
PAGE 77
66 The relationship between MATH and grade 8 mathematics achievement is given by the coefficient that equaled 1.2. With a standard error of 3.97 contributing to a tstatistic of 0.29 (pvalue = .770), there was no evidence of a significant difference between teachers with subject area content and teachers without the mathematics preparation on grade 8 mathematics achievement scores. Table 15 Fixed and Random Effects for the 2Level Conditional Model with Level1 and Level2 Covariates and Professional PreparationFixed Effects VariableModelCoefficientsetstatisticProb(t) Intercept, 00Conditional*1295.71.62182.93<.0001 Conditional*2 292.62.52115.89<.0001 Prior Mathematics Achievement, 00Conditional*11.10.0619.68<.0001 Conditional*2 1.10.0619.57<.0001 Time, 00 Conditional*17.02.892.43.018 Conditional*2 7.22.822.53.014 MATH, 10Conditional*1Conditional*2 1.23.97.29.770 MATH/METHOD, 10Conditional*1Conditional*2 6.33.012.08.041 Random Effects Variance Components sezstatisticProb(z) Intercept, 00Conditional*163.222.502.81.005 Conditional*2 55.521.372.60.009 Residual, 2Conditional*11101.846.0323.94<.0001 Conditional*2 1101.546.0123.94<.0001 Note. Data represented 30 teachers, 76 classes, and 1223 students. *1 Conditional with Level1 and Level2 Covariates *2 Covariates with MATH and Method (Mathematics Preparation)As seen in Table 15, the conditional component for the variance within the class dropped some from the parameter given in the conditional model fitted with the covariates. The within class variance in the conditional model including MATH and MATH/METHOD (1101.5) differs little from the conditional model including only the
PAGE 78
67 covariate (1101.8). The variance component representing between classes variance dropped considerably from 63.2 in the conditional model that included only the covariates to 55.5 in the conditional model that included MATH and METHOD. About 12% [(63.2 55.5)/63.2 = .123] of the remaining variation between classes was captured in this model that added the teacher mathematics preparation variables, MATH and MATH/METHOD. Conditional Model Including Main Effects. A model that included all the variables was tested. The effects of the variables, prior mathematics achievement, instructional time, ILS, MATH and MATH/METHOD were included in the model. The level1 model did not change from the earlier models that were presented (4) and remained defined as: yij = 0j + 1j ( Xij .. ) + rij (24) The level2 model was defined as: 0j = 00 + 01 (Instructional time) + 02 (ILS) + 03 (MATH) + 04 (MATH/METHOD) + u0j (25) j = 10 .(26) As seen in Table 16, the relationship between the level2 variable MATH/METHOD (teachers with mathematics and methods of teaching mathematics training) while controlling for ILS and prior mathematics achievement was given by the coefficient that equaled 4.00; classes taught by teachers with subject matter and methods of teaching mathematics preparation tended to have higher class mean by almost 4 points on class mathematics achievement. With a standard error of 3.21 contributing to a t
PAGE 79
68 statistic of 1.24 (pvalue = .218), no significant relationship existed between teachers with mathematics and methods of teaching mathematics preparation and the achievement scores of their students. The estimate for th e effect of MATH (teachers with mathematics and no methods of teaching mathematics preparation) while controlling for ILS, was 3.7; classes taught by teachers with mathematics preparation tended to have lower class means by almost 4 points and class mathematics achievement than teachers without the mathematics training. With a standard error of 4.14 contributing to a tstatistic of 0.89 (pvalue = .376), no significant relationship existed between teachers with mathematics and no methods preparation on the mathematics achievement scores of their students. The estimate for the effect of ILS (integrated learning system) while controlling for teachersÂ’ mathematics preparation was 5.3; classes using the ILS tended to have higher class means by over 5 points compared to nonILS classes. With a standard error of 2.81 contributing to a tstatistic of 1.88 (p = .065), no statistically significant relationship existed between ILS classes and the mathematics achievement scores of the students. Table 16 Fixed and Random Effects for the 2Level Conditional Model Including the Predictors Fixed Effects ConditionalWith CovariatesConditional Main effects CoefficientProb(t)Coefficients.e.dftvalueProb(t) Intercept, 00295.7<.0001291.22.6076111.86<.0001 Prior achievement, 11 1.1<.00011.10.06121719.57<.0001 Time7.0.0186.42.81702.29.025 ILS5.32.81691.88.065 MATH3.74.14700.89.376 MATH/METHOD4.03.2171 1.24.218 Random Effects CoefficientProb(z) Variance Componentss.e. zstatisticProb(z) Intercept, 00 ( 00 ) 63.20.00352.320.81 2.51.012 Residual, rij ( 2 )1101.8<.00011100.945.9623.95<.0001 Note. Data represented 30 teachers, 76 classes, and 1223 students.
PAGE 80
69 As seen in Table 16, the conditional component for the variance within the class dropped some from the parameter given in the conditional model fitted with the covariates. The within class variance in the conditional model including ILS, MATH and MATH/METHOD (1100.9) differs little from the conditional model including only the covariate (1101.8). The variance component representing between class variance dropped considerably from 63.2 in the conditional model that included only the covariates to 52.3 in the conditional model that included ILS, MATH and METHOD. About 12% [(63.255.5)/63.2 = .121] of the rema ining variation between classes was captured by ILS, MATH and MATH/METHOD. Conditional Model Including Main Effects and Interaction. A model that included all the variables was tested. The effects of the variables, Prior mathematics achievement, Time, ILS, MATH and MATH/METHOD were included in the model. Two interaction effects were also included in the model: the interaction between MATH and ILS and the interaction of MATH/METHOD and ILS. The level1 model did not change from the earlier models that were presented (4) and remained defined as: yij = 0j + 1j ( Xij .. ) + rij (24) The level2 model is defined as: 0j = 00 + 01 (Instructional time) + 02 (ILS) + 03 (MATH) + 04 (MATH/METHOD) + 05 (MATH/METHOD x ILS) + 06 (MATH x ILS) + u0j (25) j = 10 .(26)
PAGE 81
70 As seen in Table 17, none of the main effects or interactions involving ILS and Mathematics Preparation were statistically significant (p > .05). The main effect of ILS (8.4) was not statistically significant related to mathematics achievement (s.e. = 5.82, p=.153). Class means are predicted to be 8 poi nts higher for students if placed in classes that used an ILS for instruction. The estimate for the effect of MATH (0.8) indicated a drop of almost 1 point in mathematics achievement for the average class mean for classes taught by teachers with mathematics preparation only. The differences in means were found to be not significant (p = .893). The estimate for the effect of MATH/METHOD (4.9) indicated an increase of almost 4 points in mathematics achievement for the average class mean if the teachers had subject matter and methods of teaching mathematics preparation. The difference in means was found to be not significant (p = .236). Table 17 Fixed and Random Effects for the Fullyfitted 2Level Conditional ModelFixed Effects ConditionalWith CovariatesConditional Fullyfitted CoefficientProb(t)Coefficients.e.dftvalueProb(t) Intercept, 00295.7<.0001290.42.927899.43<.0001 Prior achievement, 111.1<.00011.10.06121119.51<.0001 Time7.0.0186.12.89682.12.038 ILS8.45.82691.45.152 Mathematics0.86.23680.14.893 Math/Method4.94.08691.20.236 Mathematics X ILS6.38.97680.70.487 Math/Method X ILS3.46.85690.50.618 Random Effects CoefficientProb(z) Variance Componentss.e.zstatisticProb(z) Intercept, 00 ( 00 )63.2.00555.021.592.55.011 Residual, rij ( 2 )1101.8<.00011100.945.9623.95<.0001 Note: Data represented 30 teachers, 76 classes, and 1223 students.
PAGE 82
71 Three Level Analysis Overview. The threelevel model began with the unconditional model (no predictor variables at any level). In this model, students represented the first level, the classes for each teacher represented the second level, and the teachers represented the third level. The covariate, prior mathematics achievement (7th grade) was added to the level one model. The adjusted 8th grade mathematics achievement class means ( 0jk) derived from the level1 model were then used as the outcome to be explained by the level2 and level3 variables. The predictor variables were added into the model one at a time, matching the procedure done in the 2level analysis. Unconditional model. In the three level model, students represented the first level, classes represented the second level and teachers represented the third level. To determine the amount of variation in mathematics achievement among classes taught by teachers and then among teachers versus the amount of variability within classes, analyses began with an unconditional model. The unconditional means model did not include any of the predictors and provided a ceiling on the amount of variation that would ever be explained by a 3level model. A comparison between the variance components found in the unconditional model and the variance components derived from other models was used to determine the percent of explained variance that was accounted for by various factors. At the first level, studentsÂ’ Grade 8 mathematics achievement was expressed as the sum of the intercept for a studentsÂ’ class (j) with a specific teacher (k) and a random error associated with the ith student in the jth class with the kth teacher: yijk = 0jk + rijk where rijk ~ N ( 0, 2 ).(29) At the second level, the class level intercepts were expressed as the sum of an overall mean (00k) and a series of random deviations from that mean:0jk = 00k + u0jk where u0jk ~ N ( 0, 00 ).(30)
PAGE 83
72 At the third level, the teacher level intercepts were expressed as the sum of an overall mean and a series of random deviations from that mean: 00k = (31) Substituting the two equations, results in the multilevel model:yijk = 000 + u0jk + rijk (32) The equation is an example of a mixed model that contains fixed and random components. The fixed part of the model contains 000 the single fixed effect that describes the average class grade 8 mathematics achievement for 30 teachers each teaching from one to four classes. The ra ndom part contains three random effects: u0j and rijk The random effect that described the variability among teachers means was the random effect that described the variability among class means for each teacher was u0j and the random effect that described the variability within classes wasrijk. The results from this model are presented in Table 18. Table 18 Fixed and Random Effects for the 3Level Unconditional ModelFixed EffectCoefficientssetratioPr (t) Intercept292.252.52116.05<.0001 Random EffectVariance ComponentssezratioPr (z) Intercept, (teacher) 129.951.302.53.006 Intercept, u0j (class)49.429.681.67.048 Residual, rijk 1423.859.4023.97 <.0001 Note: Data represented 30 teacher s, 76 classes, and 1223 studentsThe intercept 292.3 was the average class mean for the 30 participating teachers. There are three estimates for the random effects portion of this model. The variability among teacher mean intercepts was 129.9, the variability among class mean intercepts for
PAGE 84
73 teachers was 49.4, and the variability among students within classes was 1423.8. Hypothesis tests of the random estimates among teachers and within classes indicated that the variance components were significantly different from 0; teachers varied in their average mathematics achievement, classes varied within teachers and variation existed among students within the classes. Conditional Model Including the Level1 Covariate. The effect of the level1 covariate, Grade 7 mathematics achievement on Grade 8 mathematics achievement was examined in the 3level model. Prior mathematics achievement was measured using the studentsÂ’ seventh grade FCAT Norm Referenced Test (NRT). At the first level, studentsÂ’ Grade 8 mathematics achievement was expressed as the sum of an intercept for the studentsÂ’ class (0jk) the expected change in Grade 8 FCAT Math achievement for a unit change in the studentÂ’s grade 7 mathematics achievement ( 1jk), the studentÂ’s grade 7 achievement score centered from the grand mean ( Xijk ... ), and a random error (rijk) associated with the ith student in the jth class: yijk = 0jk + 1jk ( Xijk ... ) + rijk .(33) At the second level, the class level intercepts for each teacher were expressed as the sum of an overall mean (00k), a series of random deviations from that mean (u0jk), and the fixed value of the slope ( jk ) across all classes ( 10k ): 0jk = 00k + u0jk, and (34) 1jk = 10k .(35) At the third level, the teacher level intercepts were expressed as the sum of an overall mean (000), a series of random deviations from that mean (u0jk), and the fixed value of the slope ( 11k) across all teachers ( ):00k = 00k(36) 11k = (37)
PAGE 85
74 Substituting the two equations, results in the multilevel mixed model:yijk = + 00k + u0jk + rijk (38) where: rijk ~ N ( 0, 2 ) u0jk ~ N ( 0, 0jk ) and 00k ~ N ( 0, 00k ).As seen in Table 19, the estimate for the fixed effect intercept in the conditional model was 293.0. This represented the adjusted class mean for mathematics achievement across classes while controlling for student prior mathematics achievement. There is very little difference between it and the unconditional modelÂ’s class mean (292.3). Table 19 Fixed and Random Effects for the 3Level C onditional Model with Level1 CovariateFixed Effects CoefficientsetstatisticProb(t) Intercept, 00Unconditional292.32.52116.05<.0001 Conditional*1293.01.80163.26<.0001 Prior Mathematics achievement, 1jConditional*11.10.0619.86<.0001 Random Effects Variance ComponentssezstatisticProb(z) Intercept, TeacherUnconditional129.951.302.53.011 Conditional*161.026.792.28.023 Intercept, ClassUnconditional49.529.681.67.960 Conditional*114.017.810.78.433 Residual, 2Unconditional1423.859.4023.97<.0001 Conditional*11101.746.0123.94<.0001 Note: Data represented 30 teach ers, 76 classes and 1223 students. *1 Conditional with Level1CovariatesBy introducing seventh grade mathematics achievement as a level1 covariate, the variance estimates were expected to be reduced. Including the student level covariate in the model caused a noticeable drop in the variance components. As seen in Table 18, the variance component for the intercept, describing the variability among teachers, dropped
PAGE 86
75 from 129.9 in the unconditional model to 61.0 in the conditional model that included the student level covariate. Inclusion of grade 7 mathematics achievement accounted for [(129.9 61.0)/129.9 = .53], or 53% of the e xplainable variation among teachers (Singer, 1998). The within class variability estimate, the residual, dropped from 1423.8 in the unconditional model to 1101.7 in the conditional model that included the level1 covariate. About 23% [(1423.8 1101.7)/1423.8 = .23] of the within class variation was explained when grade 7 mathematics achievement was introduced into the model. Hypothesis tests for all variance components suggested that after including grade 7 mathematics achievement there was additional variance to explain and increasing the number of variables in the model was warranted. Conditional Model Including Random Intercepts and Slopes. A model that specified random components for the intercepts and the slopes was examined in the 3level model. The difference between this model and the previous model that included the level1 covariate was that the slopes (7th grade 8th grade achievement relationship) were free to vary. The extent to which the intercepts and slopes varied across classes was given by the variance components presented in Table 20. Table 20 Fixed and Random Effects for 3Level Conditional Model with Random Slopes and InterceptsFixed EffectCoefficientssetratioPr (t) Intercept, 00 293.11.81161.83<.0001 Prior Mathematics Achievement, 1j1.20.0716.55<.0001 Random EffectVariance Components sezratioPr (z) Teacher LevelIntercept61.5027.72.22.013 Slopes0.040.040.98.163 Covariance*1 0.750.820.91.364 Class LevelIntercept16.6018.700.88.189 Slopes0.010.050.23.410 Covariance*1 0.500.670.67.501 Residual1086.046.5623.3<.0001 Note: Data represented 30 teachers, 76 classes, and 1223 students. *1 covariance between intercepts and slopes
PAGE 87
76 The variability in teachersÂ’ intercepts was 61.5 with a standard error of 27.7 (p = .013). A rejection of the null that the variance component was equal to 0 was made; mathematics achievement differed across teachers after controlling for student prior mathematics achievement. The variance component describing the variability in slopes among teachers was 0.04 (se = 0.98; p = .163). The null hypothesis that the variance component was equal to 0 was not rejected; the relationship between the intercepts and slopes was not statistically significant among teachers. The variability in class intercepts for teachers was 16.6 with a standard error of 18.7 (p = .189). A rejection of the null that the variance component was equal to 0 could not be made; classes for a given teacher did not differ significantly in Grade 8 mathematics achievement after controlling for student prior mathematics achievement. The variability in slopes was 0.01 with a standard error of 0.05 (p=.410). The null hypothesis that the variance component was equal to 0 was not rejected; the relationship between the intercepts and slopes was not significantly different among the classes of a given teacher. The estimate for the covariance component was 0.45 with a standard error of 0.67. Because the test for the null that the variance component was equal to 0 could not be rejected (p = .501), there was little correlation between intercepts and the slopes. Based on these results, the variability for the slope (the relationship of 7th grade mathematics to 8th grade mathematics achievement) parameter was set to 0. Conditional Model Including the Level2 Covariate, Time. The level2 predictor, instructional time was added to the model. Table 21 presents the fixed and random effects for the level2 variable, instructional time. The intercept in the conditional model, 295.5, estimated the mathematics achievement for a class with an average prior achievement using less than 50 minutes of instructional time. The relationship between time, the level2 covariate and mathematics achievement was given by the fixed effect estimate for time (6.7). A standard error of 3.59 contributing to a tstatistic of 1.87 (p=.072) indicated that there was no statistically significant relationship between the amount of instructional time and the mathematics achievement scores of its students.
PAGE 88
77 Table 21 Fixed and Random Effects for the 3Level Conditional Model with Level1 and Level2 Covariates: Prior Mathematics Achievement and TimeFixed Effects CoefficientsetstatisticProb(t) Intercept, 00Unconditional292.32.52116.05<.0001 Conditional*1293.01.80163.26<.0001 Conditional*2 295.52.15137.44<.0001 Prior Mathematics achievement, 1j UnconditionalConditional*11.10.0619.86<.0001 Conditional*2 1.10.0619.65<.0001 Instructional Time, 10UnconditionalConditional*1Conditional*2 6.73.591.87.072 Random Effects Variance Components sezstatisticProb(z) Intercept, TeacherUnconditional129.951.302.53.011 Conditional*161.026.792.28.023 Conditional*2 52.725.212.09.037 Intercept, ClassUnconditional49.529.681.67.096 Conditional*114.017.810.78.433 Conditional*2 14.317.930.80.424 Residual, 2Unconditional1423.859.4023.97<.0001 Conditional*11101.746.0123.94<.0001 Conditional*2 1101.746.0123.95<.0001 Note : Data represented 30 teachers, 76 classes, and 1223 students. *1 Conditional with Level1 Covariate *2 Conditional with Level1 and Level2 CovariatesAs can be seen in Table 21, adding the level2 covariate, instructional time, impacted the variance components at the third level. The variability among teacher intercepts was given as 52.7 (se = 25.18; p=.018), slightly lower than the conditional modelÂ’s variance component (61.0). A hypothesis test of the random estimate indicated that the variance component at the teacher level was significantly different than 0 suggesting that the average class achievement for teachers varied substantially. About
PAGE 89
78 14% [(61.0 52.7) / 61.0] of the remaining variance among teachers was further explained by the addition of the level2 variable, instructional time. Conditional Model Including the Level2 Predictor, ILS. The variable ILS, represented the instructional component of the class; the instructional delivery known as an Integrated Learning System (ILS) was used in 41 of the 76 participating classes and by 16 of the 30 teachers. Table 21 presents the fixed and random effects for the level2 variable, ILS, when added to the 3level model. The intercept in the conditional model, 292.4 estimated the mathematics achievement for a class with an average prior mathematics achievement in a class with less than or equal to 50 minutes of instruction and not using the ILS for instruction. The relationship between the level2 variable, ILS and mathematics achievement was given by the fixed estimate for ILS that equaled 5.4; classes that use the ILS for instructional purposes had higher class mean by over 5 points on mathematics achievement than nonILS classes. With a standard error of 3.34 contributing to a tstatistic of 1.62 (pvalue = .119), a statistically significant relationship did not exist between the use of an ILS for instruction and the mathematics achievement scores of its students. Table 22 Fixed and Random Effects for the 3Level Conditional Model with Level1 and Level2 Covariates and ILSFixed Effects Coefficien t setstatisticProb(t) Intercept, 00Conditional*1295.52.15137.94<.0001 Conditional*2292.42.81103.89<.0001 Prior Mathematics Achievement, 00Conditional*11.10.0619.86<.0001 Conditional*21.10.0619.65<.0001 Time, 00Conditional*16.73.591.87.072 Conditional*26.33.491.81.081 ILS, 10Conditional*25.43.341.62.119 (table continues)
PAGE 90
79 Random Effects Variance Components sezstatisticProb(z) Intercept, TeacherConditional*152.725.212.09.037 Conditional*246.624.721.89059 Intercept, ClassConditional*114.317.930.80.424 Conditional*215.818.400.86.391 Residual, 2Conditional*11101.746.0123.95<.0001 Conditional*21101.245.9723.95<.0001 Note: Data represented 30 teacher 76 classes, and 1223 students. *1 Conditional with Level1 and Level2 Covariates *2 Covariates and ILSAs seen in Table 22, the variance component representing the variability among teachers equaled 46.6 when the variable ILS was included. About 11% [(52.7 46.6) / 52.7] of the remaining variance at the teacher level was captured when ILS was added to the model. The variance components at level1 and level2 did not change much. Conditional Model Including the Level2 Predictor, Mathematics Preparation. Three categories of teachers were defined in this study: (a) MATH/METHOD: teachers with completed mathematics courses and a methods of teaching mathematics course, (b) MATH: teachers with completed mathematics courses only, and (c) teachers without sufficient mathematics or methods of teaching mathematics courses to teach MJ3 or methods of teaching mathematics. Table 23 presents the fixed and random estimates for the 3level model that include the level1 and level2 covariates with the teacherlevel variable, mathematics preparation. The relationship between the level3 variable, MATH/METHOD, describing teachers with mathematics and methods of teaching mathematics was given by the coefficient, 6.9; classes taught by t eachers with the highest amounts of training tended to have higher class mean by almost 7 points on mathematics achievement. With a standard error of 3.91 contributing to a tstatistic of 1.76 (pvalue = .091), no statistically significant relationship existed between teachers with the highest mathematics preparation and teachers without mathematics and methods of teaching courses on mathematics achievement scores of its students.
PAGE 91
80 The relationship between MATH and mathematics achievement is given by the coefficient that equaled 0.7; classes taught by teachers with completed mathematics courses had lower class means by almost 1 point on mathematics achievement when compared to teacher without the mathematics preparation. With a standard error of 5.00 contributing to a tstatistic of 0.13 (pvalue = .896), no statistically significant relationship existed between teachers with mathematics training only and teachers without the sufficient preparation on grade 8 mathematics achievement scores of its students. Table 23 Fixed and Random Effects for the 3Level Conditional Model with Level1 and Level2 Covariates and Professional PreparationFixed Effects CoefficientsetstatisticProb(t) Intercept, 00Conditional*1295.52.15137.44<.0001 Conditional*2292.03.2888.97<.0001 Prior Mathematics Achievement, 00Conditional*11.10.0619.86<.0001 Conditional*21.10.0619.58<.0001 Time, 00Conditional*16.73.591.87.072 Conditional*27.03.502.01.055 Mathematics, 10Conditional*20.75.000.13.896 Method, 10Conditional*26.93.911.76.091 Random Effects Variance Components sezstatisticProb(z) Intercept, TeacherConditional*152.725.212.09.037 Conditional*246.523.891.95.052 Intercept, ClassConditional*114.317.930.80.424 Conditional*213.817.680.78.435 Residual, 2Conditional*11101.746.0123.95<.0001 Conditional*21101.545.9923.95<.0001 Note: Data represented 30 teacher 76 classes, and 1223 students. *1 Conditional with Level1 and Level2 Covariates *2 Covariates with Mathematics a nd Method (Professional Preparation)
PAGE 92
81 As seen in Table 23, the variance component describing the variability between classes and the variance describing the variability within classes changed very little from the corresponding parameter in the unconditional model. There is some evidence that including the predictors in a model shaped the variance component describing the variability among teachers. About 12% [(52.7 46.5) / 52.7 = .117] of the remaining variance between teachers was captured by including the teacher mathematics preparation variables, MATH and MATH/METHOD. Conditional Model Adding All Predictors. A model that included all the variables was tested. The effects of the variables, Prior mathematics achievement, Instructional time, ILS, MATH and MATH/METHOD were included in the model. As seen in Table 24, the relationship between the level3 variable MATH (teachers with mathematics training) while controlling for ILS on grade 8 mathematics achievement was given by the coefficient that equaled 2.8; classes taught by teachers with mathematics training tended to have lower class means by almost 3 points on grade 8 class mathematics achievement compared to teachers without the mathematics training. With a standard error of 5.17 contributing to a tstatistics of 0.55 (p=.587), no significant relationship existed between teachers with only mathematics preparation compared to teachers without the mathematics training and grade 8 mathematics achievement scores of their students. The estimate for the effect of MATH/METHOD (teachers with mathematics and methods of teaching mathematics preparation) while controlling for ILS, was 5.0; classes taught by teachers with mathematics and methods of teaching mathematics tended to have higher class means by 5 points on grade 8 mathematics achievement compared to teachers without the mathematics preparation. With a standard error of 4.09 contributing to a tstatistic of 1.21 (p=.238), no statistically significant relationship existed between teachers with mathematics and methods of teaching mathematics preparation and the mathematics achievement scores of their students. The estimate for the effect of ILS while controlling for teachersÂ’ mathematics preparation was 4.7; classes using the ILS tended to have higher class means by almost 5
PAGE 93
82 points on class mathematics achievement than classes not using the ILS on grade 8 mathematics achievement scores of the students. Table 24 Fixed and Random Effects for the 3Level Conditional Model with Level1 and Level2 Covariates and All PredictorsFixed Effects CoefficientsetstatisticProb(t) Intercept Conditional*1295.52.15137.44<.0001 Conditional*2290.73.3786.31<.0001 Prior Mathematics Achievement Conditional*11.10.0619.86<.0001 Conditional*21.10.0618.57<.0001 TimeConditional*17.03.591.87.072 Conditional*26.43.471.85.076 MATHConditional*20.75.17.055.587 MATH/METHODConditional*25.04.091.21.238 ILSConditional*24.73.521.32.198 Random Effects Variance Components sezstatisticProb(z) Intercept, TeacherConditional*152.725.212.09.037 Conditional*243.023.991.79.073 Intercept, ClassConditional*114.317.930.80.424 Conditional*215.118.100.83.406 Residual, 2Conditional*11101.746.0123.95<.0001 Conditional*21101.545.9623.96<.0001 Note: Data represented 30 teacher 76 classes, and 1223 students. *1 Conditional with Level1 and Level2 Covariates *2 Covariates with MATH and Method (Professional Preparation)As can be seen in Table 24, the variance component describing the variability between classes and the variability component describing the variability within classes changed very little from the parameters in the unconditional model. At the teacher level, the variance components indicated that more explained variance was detected. About 18% [(52.7 43.0) /52.7 = .18] of the variance between teachers was captured by including all the predictors. Conditional Model Including All Covariates, Predictors, and Interactions. A model that included all the variables was examined in the three level model. The effects,
PAGE 94
83 grade 7 mathematics achievement, instructional time, ILS, MATH, MATH/METHOD were included in the model. Two interaction effects were also included in the model: the interaction between MATH and ILS and th e interaction of MATH/METHOD and ILS. Table 25 presents the fixed and random estimates for the 3level model that included the level1 and level2 covariates and all the predictors. The fixed effect describing the grade 8 FCAT Math mean dropped from 295.5 in the model with only the covariates to 289.4 in the model with all the variables. The fixed effects for ILS, instructional time, MATH, MATH/METHOD, MATH X ILS, and MATH/METHOD X ILS were 9.3, 6.1, 0.2, 6.8, 7.4, and 5.6, respectively. None of the predictor variable estimates nor the interactions were found to be statistically significant. Table 25 Fixed and Random Effects for the 3Level Model With All Variables and InteractionsFixed Effects ConditionalWith Covariates Conditional Fullyfitted CoefficientProb(t)Coefficients.e.dftvalueProb(t) Intercept295.5<.0001289.43.902274.14<.0001 Prior achievement1.1<.00011.10.06119219.54<.0001 Time7.00.0726.13.62241.68.107 ILS9.37.54221.24.230 Mathematics0.27.8422 0.03.981 Method6.85.34221.26.219 Mathematics X ILS7.411.29240.66.517 Method X ILS5.68.87220.63.532 Random Effects Variance Components Prob(z)Variance Components s.e.zvalueProb(z) Intercept (teacher)52.7.03748.726.221.86.063 Intercept (class)14.3.42414.717.950.82.412 Residual, rij 1101.7<.00011101.145.9623.96<.0001As seen in Table 25, the variance component for the variance within the class and between classes barely differed between the conditional model with the covariates and the fully fitted model. At the teacher level, about 7% [(52.7 48.7)/52.7 = .074] of the variation between teachers was captured by including all the covariates, predictors, and interaction.
PAGE 95
84 Summary Eight different models were examined using the data collected in this study: 1) an unconditional model, 2) a model with the level1 covariate (7th grade mathematics achievement), 3) a model with random slopes (7th grade 8th grade relationship), 4) a model with level1 and level2 covariates (instructional time), 5) a model with level1 and level2 covariates and ILS, 6) a model with level1 and level2 covariates MATH, and MATH/METHOD, 7) a model with all the predictors and 8) a model with all the predictors and their interactions. The relationships among the variables were also examined using a twolevel model and a threelevel model. The advantage of using a twolevel model is that it is easier to explain the impact of the effects within the context of two levels. However, a twolevel model may be restricting the amount of variance that can be captured between and within levels. The advantage of using a threelevel model is to maximize the variance between and within levels captured in the model; however, the results must be reported in the context of the number of levels increasing the difficulty in the interpretation. In this study, exploring the relationships among the variables in a twoor threelevel model was contingent upon interpreting the structure of the nested data. Three levels of data were collected: student, classroom, and teacher. Either a twolevel or three level model can be used to describe the 3 levels of collected data. One way to interpret the data structure was to include the teacher characteristics with the classroom. Consequently, the interpretation of the results in this model focused on the extent the study variables impact studentsÂ’ grade 8 mathematics achievement. In this study, the classroom factors (i.e., instructional time, instructional model, teachersÂ’ preparation) are expected to influence student achievement. If the three level model is used, the teacher characteristics are separated from the classroom. The interpretation of the studyÂ’s results focused on the differences among teachers and the differences among their classes. Consequently, student achievement is influenced by teachersÂ’ level of mathematics preparation and by the characteristics of the classroom they teach (i.e., ILS vs. nonILS, time).
PAGE 96
85 Overall, the results from the twolevel and threelevel analyses suggested that with the addition of a third level to the model, the fixed or random estimates did not change substantially. The contributions of the predictors on student Grade 8 mathematics achievement were not statistically significant in either the twolevel model or the threelevel model. The errors of the estimates were higher in the 3level model when compared to the errors of the estimates in the 2level model. However, the larger errors in the three level model are not usual because of the more complicated variance and covariance structures created with the greater number of levels and the smaller number of units for the given level. The importance of this study was based on its ability to provide guidance to school districts. Because the results from th e analyses were not different, the twolevel model was used to interpret the results for school district personnel. The twolevel model was interpreted as an environment with various factors (i.e., instructional time, ILS vs. nonILS, teacher characteristics) that influence student achievement. Some of these factors are within the school administratorsÂ’ control, and as such, the results from the twolevel analysis should provide useful information that contributes to an environment that positively impacts student achievement. Other findings resulting from this study were intended for the research community. The issues and concerns resulting from the HLM analysis are relevant for either the twoor threelevel model. Subsequently, the results from the twolevel model are used to interpret the results and the issues resulting from the HLM analysis are addressed in Chapter 5.
PAGE 97
4 TeachersÂ’ mathematics training/prepara tion was defined in this study as ha ving completed 18 or more semester hours in mathematics, enough to earn middle school certification in mathematics, and co mpletion of a methods of teaching mathematics course.86 Chapter Five Discussion This chapter summarizes the major findings of the study and discusses their connections with the literature on teacher preparation and the use of integrated learning systems to teach middle school mathematics. Implications and recommendations from the results are provided along with a discussion of the methods used to obtain the results. Limitations of the study are included and recommendations for further research are made at the end of the section. Study Overview The literature on teacher effectiveness has indicated that teachersÂ’ mathematics training4 is a key variable in the teachinglearning process (NCTM, 1998; NCES, 1999). NCTM (1986) suggests that teachers of mathematics with adequate mathematics preparation are expected to know mathematics, model good teaching of mathematics, and know how students learn mathematics. These conditions are viewed as crucial to student learning in mathematics. Current trends in technology also suggest that student achievement is positively impacted by the use of integrated learning systems. The ILS environment facilitates increased time on task and effective assessment. As such, studentsÂ’ learning of mathematics was expected to be positively shaped while using the ILS, especially with teachers without the specified mathematics preparation. Currently, at least a third of public school teachers, nationwide do not have the degree or license to teach mathematics in middle and high schools; in urban areas with
PAGE 98
5 Typically, having a degree suggests that an individual has completed a minimum number of mathematics courses needed to meet the requirements for the degree. Among other requirements needed for the certification process, it is expected that certification in mathematics includes completed courses in mathem atics and at least a course in methods of teaching mathematics. 87 atrisk populations (i.g., low ses), the percen tage of teachers without certification in mathematics increases to 50%.5 The growing trend of teachers without the appropriate preparation and certification entering the classroom as teachers of mathematics provided the impetus for this study. While current research suggests that teachers with increasing levels of mathematics preparation positively influence studentsÂ’ mathematics achievement, the effect of the ILS on mathematics achievement when used by teachers of varying levels of mathematics preparation was unknown (Brown & Smith, 1997). It was anticipated that a positive relationship would exist between the ILS and student mathematics achievement and that this relationship would be stronger for teachers with less mathematics preparation. The present study was designed to explore the effects of an ILS with teachers with varying levels of mathematics preparation on eighth grade student mathematics achievement. Three major objectives were addressed: 1) the relationship of teachersÂ’ mathematics preparation to grade 8 student mathematics achievement; 2) the relationship of an ILS to grade 8 student mathematics achievement; and 3) the interaction of the ILS and teachersÂ’ mathematics preparation on studentsÂ’ mathematics achievement. These objectives were addressed using secondary data from 11 Title I middle schools in a large Florida school district. Participating in the study were 1,223 students in 76 classes taught by 30 teachers. The dependent variable was grade 8 mathematics achievement that was measured by the FCAT Math SSS (the mathematics portion of the FCAT, a criterionreferenced test). To strengthen internal validity, control variables of studentsÂ’ prior achievement captured by the grade 7 normreferenced test scores on the FCAT and the amount of time used for mathematics instruction at the middle school were included in the analysis.
PAGE 99
88 The existing data, derived from a nonexperimental design, were analyzed using hierarchical linear modeling. Data for this study were complex and measured at several levels: student level, classroom level, and teacher level. At the student level, mathematics achievement data were collected for each student for two years. The grade 7 FCAT NRT (normreferenced test) was used to measure studentsÂ’ prior mathematics achievement and the FCAT Math SSS (criterionreferenced test) was used to measure grade 8 mathematics achievement. Classroom data included amount of time for instruction and instructional method (ILS vs. nonILS). Instructional time was classified into two categories: less than or equal to 50 minute classes and 90 minute classes. Teacher level data included three categories of mathematics preparation: 1) teachers with mathematics and methods of teaching mathematics; 2) teachers with mathematics courses only; and 3) teachers without the mathematics courses needed to teach MJ3 (grade 8 prealgebra) and without the methods course. The outcome variable used in this study was a score the 8th grade FCAT Math SSS a criterionreferenced test. Because of the central role of this outcome variable in this study, measurement concerns with the FCAT introduced in Chapter Three (Method), are revisited, followed by a discussion of the studyÂ’s results for each research question. FCATIssues in Measurement The FCAT item specifications were developed by the Florida Department of Education with the contributions and advice of Florida subject area educators. The FCAT Content Advisory Committee determined which benchmarks in the Sunshine State Standards (SSS) would be assessed on the FCAT and which of the item types would be appropriate. A contractor was hired to pr epare a first draft of the item specifications recommended by the Content Advisory Committee. The first draft was reviewed by the Advisory Committee and further revisions were made based on their recommendations. Items and performance tasks were developed by the contractor and reviewed by the statewide committees of Florida educators. Committee comments were used to clarify and refine the initial draft of the specifications. The specifications included overall considerations that included directions on general specifications pertaining to item writing, the context in which the test item was presented, the cognitive level of the
PAGE 100
89 student, use of graphics, item style and format (i.e., multiple choice, gridded response, short/long response). Detailed information about the specifications for grades 6 to 8 can be found in the Mathematics Task Item and Performance Task Specifications guide. Although, ideally for the present study it would have been desirable to have a variety of measures to assess the effect of teachersÂ’ mathematics preparation and the effect of the ILS on 8th grade studentsÂ’ mathematics achievement, the nature of this secondary analysis precluded having these measures. Having said this, it should be pointed out that the FCAT captures the mathematics skills endorsed by the NCTM. The goals include: 1) number sense, concepts, and operations; 2) measurement; 3) geometry and spatial sense; 4) Algebraic thinking; and 5) data analysis and probability. Research Question 1 TeachersÂ’ Mathematics Preparation In this study, teachers were not randomly selected or randomly assigned to classes; they volunteered to use the ILS or were asked by the principal to use the ILS for instruction. The number of classes varied among the teachers, each of whom had a number of classes ranging from 1 to 4. Each teacher used the same instructional method for all the MJ3 classes the teacher taught. The first research question focused on the relationship between teachersÂ’ mathematics preparation and studentsÂ’ mathematics achievement (measured by the FCAT Math SSS a standardized, statewide test). Much of the research examining teacher quality suggests that subject area preparation is positively related to student achievement; however, the variable defining teachersÂ’ mathematics preparation has been measured in a number of different ways. For example, Hawkins (1998) used an undergraduate or graduate major in mathematics to explain the variance of 8th grade NAEP scores. The results of the Hawkins study indicated that students taught by teachers with an undergraduate or graduate major in mathematics scored higher than students taught by teachers with majors in education or some other field. Fetler (1999) defined teacher subject preparation by the highest degree completed by the teacher and possession of a mathematics authorization for teaching; it was not evident whether the awarded degree was completed in mathematics or some other field. However, the results of the study
PAGE 101
90 suggested that teacher preparation was significantly related to student achievement while controlling for student poverty. The teacher preparation variable used in this study was defined by number of completed mathematics courses and completion of a methods of teaching mathematics course which was aligned with the courses and qualifications legislated by Florida to teach MJ3. Teachers must have completed at least 18 semester hours in mathematics; this was also consistent with the requirements for middle school certification in mathematics. All the participating teachers had at least an undergraduate degree, but not necessarily one in mathematics. In this study, the type of degree was irrelevant in defining teachersÂ’ preparation. The criteria for defining teachersÂ’ preparation in this study were the completion of a minimum number of courses in mathematics and the completion of one course in methods of teaching mathematics. A review of teachersÂ’ transcripts was used to determine the number and type of mathematics courses completed by the teacher as well as the completion of a methods of teaching mathematics course. It was anticipated that teachers with mathematics and the methods of teaching mathematics would have higher class means than other groups of teachers with less mathematics preparation. Participating in the study were 30 teachers in 76 classes. Forty classes were taught by 16 teachers with mathematics and methods of teaching mathematics, 13 classes were taught by 6 teachers with mathematics preparation only, and 23 classes were taught by 8 teachers with no mathematics and no methods of teaching mathematics preparation. In this study, teachers with mathematics and methods of teaching mathematics (teachers with the highest mathematics preparation) tended to have higher class means when compared to teachers without the mathematics preparation. This result was not unexpected and is consistent with the existing body of literature on teacher quality; students benefit from teachers with evidence of mathematics and methods of teaching mathematics preparation. When controlling for prior student achievement and instructional time, teachersÂ’ mathematics training was found to have a statistically significant effect on student achievement. However, when also controlling for the use of an ILS, the magnitude of the
PAGE 102
91 effect of teachersÂ’ preparation was redu ced. Differences among the varying groups of teachers were no longer statistically significant at the .05 level. When controlling for prior student achievement in mathematics and instructional time, teachers with mathematics training only (no methods) did not have a class mean statistically different from teachers without mathematics preparation. Similar results were found when controlling for ILS. This information added support to the results from this study that suggested that teachers with mathematics and methods of teaching mathematics training are better prepared for instruction than other teachers with less training. Methods of teaching mathematics appeared to compliment the teachersÂ’ mathematics background. The lack of difference between the group of teachers with mathematics subject matter (but no methods) training and the group of teachers with no mathematics may be attributed to the basic level of material that is characteristic of the MJ3 course. MJ3 is one of two classes in mathematics available for grade 8 in FloridaÂ’s middle schools. Its curriculum is defined by the concepts and processes typically identified in a prealgebra course. Even the minimum number of college level courses in mathematics taken by individuals who graduate from college may provide a middle school teacher with sufficient knowledge of mathematics to teach the MJ3 curriculum. If the class used in the study was Algebra I, which consists of more advanced concepts, perhaps the differences between the two groups of teachers (mathematics training vs. no mathematics training) would have become more pronounced. One limitation of how teachersÂ’ preparation in mathematics was measured was the use of categorical data to measure the training. The placement of teachers in the no mathematics and methods of teaching category was based on the number of courses the state indicated as a minimum number to teach the course. It is possible that teachers in this group had all but one mathematics class required by the state; however, the dichotomously scored variable would have categorized teachers as not having the requirement completed. TeachersÂ’ mathematics preparation could have been captured with interval data using the number of semester hours completed in mathematics. However, such data are not readily available. To collect these data would necessitate
PAGE 103
92 surveying teachers, an option that was not available in this secondary analysis. Therefore, the teacher preparation variable in this study was aligned with the stateÂ’s requirements needed for teaching MJ3 classes. Although the group means among teachers were not found to be statistically significant, teachers with the highest level of mathematics training while controlling for ILS had class means about 4 to 8 points higher than other groups of teachers with varying levels of mathematics training. The implication of the trend of higher achievement for teachers with the highest levels of mathematics training needs to be evaluated by individual schools. SchoolsÂ’ efforts are dir ected toward helping students achieve their potential. A measure, determined by the Florida Department of Education, that schools use to gauge their efforts is a scale score in the interval whose minimum is 310 (Level 3). Student performance at Level 3 indicates that the student has partial success with the challenging content of the SSS but the performance is consistent. A level 3 student answers many of the questions correctly but is generally less successful with questions that are most challenging (FDOE, 2002). Research Question 2 Instructional Method (ILS vs. nonILS) The second research question focused on the relationship between the instructional method (ILS vs. nonILS) and 8th grade studentsÂ’ mathematics achievement. The effectiveness of the ILS can be defined in a number of ways. Time in ILS was considered by West and Marcotte (1994) as a measure of its effectiveness. They found that for 9th graders enrolled in Algebra I, students using an ILS for longer periods of time had statistically higher achievement than students who used the ILS for shorter periods of time. Another study by Taylor (1999) also considered the effect of time on ILS. Results from this study suggested that longer periods of time on the ILS were associated with significantly higher achievement on a yearend mathematics exam when compared to use of ILS for shorter periods of time. Because the ILS is an expensive purchase for schools, Brush (1996) conducted a study that considered the use of cooperative learning activities combined with the ILSdelivered instruction. Results from the Brush study indicated that using cooperative learning strategies combined with the ILS was shown to positively influence student achievement in mathematics.
PAGE 104
6 An ILS is a network of computers linked to a server where presented lessons must include branching, frequent feedback, and diagnostic informati on. The computer has a management system that controls and keeps track of student progress.93 These examples of research highlight the different ways to measure the effect of the ILS. Inferences can be further complicated when different products are used for study. The fundamental features of the ILS used in a study must be described in order to validate the definition of the ILS 6. Branching is the most important feature distinguishing an ILS. Its ability for d ecisionmaking based on input from the student differentiates it from other types of computerassisted instruction. As such, it is essential that the name of the product be included with the study to ascertain its credibility as an ILS. In this study, the ILS was conceptualized as an instructional strategy and students were coded as either participating in the ILS or not participating in the ILS. Because this study used existing data, information on all that went on in the ILS and nonILS conditions was limited. In this study, 39 classes used the ILS and 37 did not. While controlling for instructional time and studentsÂ’ prior achievement, the relationship between using an ILS for instruction and 8th grade mathematics achievement was found to be statistically significant. Classes that used an ILS for instruction tended to have class means about 6 points higher compared to classes not using the ILS. However, when also controlling for teachersÂ’ mathematics preparation, the magnitude of the effect of ILS was reduced. Differences among the instructional method (ILS vs. nonILS) were no longer statistically significant; however, classes that used the ILS tended to have class means about 5 points higher compared to classes that did not. The implication of the trend of higher achievement for the ILS classes needs to be evaluated by individual schools. SchoolsÂ’ effo rts are directed toward helping students achieve their potential. A measure, determined by the Florida Department of Education, that schools use to gauge their efforts is a scale score of 300 or higher on the FCAT Math SSS Regardless of the teachersÂ’ mathematics preparation, more classes using the ILS (39%) met that goal compared to nonILS classes (22%).
PAGE 105
7Hawthorne Effect was defined in Borg and Gall (1989) as a distortion of research results caused by the response of subjects to the special attention they receive from researchers. 94 This was the first time students and teachers participating in the study used the ILS. The novelty of the ILS may have influenced their attentiveness to the instructional format, assuming that the hardware was working. TeachersÂ’ and studentsÂ’ frustration with hardware downtime or software errors would have shaped the results. Without observational or survey data, information about the first year of implementation was not available. The teachers chose to use the ILS for instruction or were encouraged by their principals to use the ILS. Their interest in the technology may have influenced the effect of the ILS on grade 8 student achievement (Borg & Gall, 1989)7. Very often, teachers who have taught the same class to the same level of student need change. The ILS may have provided some teachers with the needed change to refresh their instructional delivery and attitude toward teaching. The introduction of the ILS may have reduced the percentage of teachers who found teaching MJ3 otherwise boring. Longitudinal research on teachers using the ILS over many years would help evaluate the novelty effects of the ILS. The ILS is defined as an individualized learning environment. Students begin lessons wherever they have left off the previous time. Students are also able to complete more lessons based on their proficiency and allotted instructional time. Students using the ILS may have explored more of the curriculum and mastered some of that content prior to the time of taking the FCAT. Further research should focus on the individualized instruction provided to students. A record of the number of completed lessons would indicate the extent of the curriculum covered by students in the ILS and nonILS classes. Higher student achievement on the FCAT may be a result of ILS students completing more lessons prior to test administration. Research Question 3 Interaction of ILS and TeachersÂ’ Mathematics Preparation The third question centered on the interaction between teachersÂ’ preparation in mathematics and the instructional method. It was hypothesized that the effect of the ILS
PAGE 106
95 on mathematics achievement would be larger for teachers with no mathematics and no methods of teaching mathematics compared to teachers with the highest mathematics preparation. The results of this study did not support this hypothesis in that no statistically significant interaction between teachersÂ’ mathematics training and the use of the ILS was found. The need for this study was motivated by teachers without mathematics preparation entering the classroom as teachers of mathematics. Although it was anticipated that teachers without sufficient mathematics preparation would benefit the most from using the ILS, the results indicated that all groups of teachers, using the ILS, regardless of their preparation in mathematics had higher class means; no one group of teachers exhibited stronger effects when using the ILS. Teachers with varying levels of mathematics preparation who used the ILS tended to have class means about 8 points higher than similar teachers in nonILS classes. Instructional Time While instructional time was not a focus of study, the consistency of its effect across the examined models needs to be addressed. Classes using 50 minutes or less for instruction tended to have statistically significant higher class means compared to classes using 90 minutes for instruction. Using the ILS or having teacherswith the highest preparation in mathematics did not diminish the relationship between instructional time and student achievement. The instructional time for classes within any participating school was determined by school administrators. Often the need for longer periods of instructional time arose from a perception that students are lowachieving and require more time on task. Fiftytwo classes in the study used 50 minutes or less for instruction and their means for prior achievement (grade 7) ranged between 27.8 and 51.8 with a mean of 41.3 (SD = 5.1). Twentyfour classes in the study used 90 minutes for instruction and their means for prior achievement (grade 7) ranged between 26.6 and 50.3 with a mean of 35.2 (SD = 5.5). A ttest was conducted to compare prior achievement (grade 7 mathematics) using 50 minutes or less for class with classes using 90 minutes for instruction. Classes using 50 minutes or less had means significantly higher than classes using 90 minutes. The groups
PAGE 107
96 of students categorized by instructional time were found to be different at the beginning of the analysis. While controlling for grade 7 prior achievement in mathematics, the results of this study indicated that the effect of time was statistically significant on grade 8 student achievement in mathematics. However, there may be other variables that influence student achievement not accounted for in this study (e.g., gender, locus of control); controlling doesnÂ’t equate students on all variables and characteristics that facilitate or inhibit instruction. Because this study was nonexperimental and the finding of the instructional time incidental, future research should examine the impact of instructional time under more controlled conditions. Recommendations and Further Research TeachersÂ’ Mathematics Preparation. The intent of this study was to investigate the relationship between an integrated learning system that delivered a prealgebra curriculum to 8th grade students in classes taught by teachers with various levels of mathematics preparation and student mathematics achievement. While controlling for student prior achievement and instructional time, the results of this study found that teachers with mathematics and methods of teaching mathematics tended to have significantly higher class means compared to teachers without the mathematics and methods of teaching mathematics preparation. However, while controlling for instructional method (ILS), the studyÂ’s results indicated that differences among groups of teachers with varying levels of mathematics training were reduced. While the results were not statistically significant, the teachers with the highest preparation in mathematics teaching tended to have the highest class means of any group of teachers, regardless of the instructional method; about half of the teachers with the highest preparation in mathematics teaching had FCAT class means of 300 or more scale score points. Schools looking to purchase an ILS or schools that have an ILS may find the studyÂ’s results useful. The results support placing teachers with varying levels of mathematics preparation in the ILS; there is no evidence to suggest that one group of teachers would benefit more than another from using the instructional method. Future research focusing on pedagogical and subject matter training and its relationship to student achievement should be continued. Research also should examine
PAGE 108
97 the consequences of other variables (i.e., teacher reflection, collegial interaction, staff development) in relation to how teachers learn to teach mathematics. The extent to which other variables contribute to teachersÂ’ preparation and consequently, influence student achievement, is not known. ILS. While controlling for student prior achievement and instructional time, the results of this study found that classes using the ILS tended to have significantly higher class means compared to classes not using the ILS. Based on this result, schools may find the use of an ILS potentially beneficial for prealgebra middle school students. The effect of the ILS was reduced some while controlling for teachersÂ’ professional preparation (p = .065). Further analysis that includes a larger sample size and balanced groups may provide more substantial evidence of the impact of the effect. The research questions in this study were examined using an analysis of existing data. The use of existing data presented challenges that limited the manner in which variables were conceptualized. Data collected from observations and surveys would have added to explaining the effects of teacher preparation in mathematics and the use of the ILS on student achievement. For example, the management system within the ILS records studentsÂ’ time on the ILS and keeps a record of the completed lessons. Exploring the relationships between these data and student achievement would add to explaining the positive relationship between the ILS and student achievement. `The results from this study suggested a positive relationship between the ILS and student achievement in MJ3, an 8th grade prealgebra class, in Title I middle schools only. Further research is needed to study the relation between the ILS and student achievement at other schools with broader ranges of socioeconomic levels and in different mathematics classes (e.g., Algebra I). Current research suggest that teachers in nonurban areas tend to be more qualified compared to teachers in urban areas especially in mathematics classrooms; teachers are expected to have a degree in mathematics and appropriate certification in mathematics (Jerald, 2002; McDermott, Rothenberg, & Gromley, 1998; Ingersol & Gruber, 1996; NCES, 1995). Inclusion of middle schools from more affluent neighborhoods in the dist rict may mitigate the effect of the ILS.
PAGE 109
98 Because this study involved a secondary analysis of existing data, ILS was measured dichotomously by student enrollment in the class. As such, some aspects of the ILS were not considered as variables but may be crucial in explaining more about the effect of the ILS on student achievement (e.g., teachersÂ’ interest, student use of notebooks). Other components of the system that support studentsÂ’ learning need to be examined. Design features of the ILS, such as the assessment component and its relationship to student achievement, should be explored. The higher student achievement in mathematics found for the ILS may have resulted from the immediate reports provided to the student at the end of an assessment. While using the ILS, students were made aware of their performance quickly which may have affected their persistence and continued use of the ILS. The relationship between assessment reports and student learning could be explored through su rvey data collected from students. Technology issues may affect student achievement indirectly; equipment that is under repair often is not available for student use. An exploration of the relationship between downtime and student achievement may be informative. Also, teachersÂ’ and studentsÂ’ input regarding the use of the IL S would provide candid information about the use of the technology. Issues such as downtime, crashes, updates, and Internet access would empower school districts by providing r eal cost estimates of implementing the ILS. The extent to which technology issues influenced student achievement could have been addressed using survey data and data collected from the computer management system. Other issues not addressed in this study that would be important to uncover relate to the kind of student that is most influenced by the ILS environment. Demographic information such as gender, race, age, and proficiency in English need to be explored in future research. This information may be useful for school districts to know before implementing the ILS program. Method. The implementation of randomized field experiments in K12 educational settings is difficult. To conduct a randomized field study there needs to be random assignment of students to classes, random assignment of classes to ILS or nonILS instructional formats, and random assignment of teachers to the classes. Because this
PAGE 110
99 was not possible in this study, an appropriate method for evaluating the studyÂ’s studentlevel and classroom level effects was a multilevel model. Because of the nested structure of the studyÂ’s data, HLM was used for analyzing the data. This technique allows the simultaneous analysis of data from several levels and provides unbiased estimates of effects when the data are nested. However, a number of gaps existed in the literature regarding technical issues that address using HLM in educational settings. Future research regarding the technical issues that were uncovered during this study are presented next. One of the findings of this study pertained to the size of the sample. Sample size is a complex issue that involves the number of levels and the number of variables at each level. Some researchers (Kreft, 1999; Raudenbush, 2000) have suggested a rule for determining an appropriate sample size based on the number of groups at the second level and the number of observations within the group. However, the number of variables per level was not considered in this rule. A liberal method of exploring the impact of the number of variables per level in relation to the sample size across all levels is to use all the intended variables. In a study by Webster, Mendro, Orsak, and Weerasinghe (1998), a decision to change from a 3level analysis to a 2level analysis was based on a lack of convergence resulting from a large number of variables at all levels. When the 3level model did not converge, the number of variables was reduced and analyzed again until convergence took place. A more conservative method would be to carefully choose variables that address the research questions. Typically, in linear regression, variables are chosen based on their relationship to the dependent measure and the number of variables is limited by the size of the sample. While this is a convenient rule for linear regression, no such rule exists for HLM. Future research should address the relationship between the number of variables at each level and the sample size across all levels. In educational settings, students are nested in classes, classes are nested within teachers, teachers are nested within schools, schools are nested within communities, and so on. A lack of research is evident in supporting decisions in regard to the number of levels that are needed to represent the data structure. Future research should focus on the
PAGE 111
100 manner in which the models are created. Questions for further research should examine if the data structure must exactly represent the nesting of the educational setting or if there is some flexibility in the modeling process. Studies pertaining to school effects typically use 2 levels to analyze data: student level data and school level aggregate data. The nesting levels between the two are eliminated from the analyses. Future research should address the accuracy of the results of these divergent multilevel models. Another issue dealing with sample size that was not discussed in the literature is power for nonrandomized studies using HLM. Given the nonrandomized circumstances that control the data structure in educational settings, collecting sufficient numbers of classes that also have large num bers of student may be difficult. Raudenbush and Liu (2000) presented a power analysis for two designs that are typically found in medical research: randomized trials and randomized cluster trials. Based on the findings from Raudenbush and Liu (2000), power in the present study may have been enhanced by increasing the number of students within classes and increasing the number of classes among teachers at a school. Future research should be conducted to determine what factors involved in the multilevel modeling contribute to power for nonexperimental designs. Twolevel and threelevel HLM analyses were examined to study the effects of ILS and teachersÂ’ mathematics preparation. In the unconditional 2level model, the within class variance component was greater than the between class variance component. The within class variance component was also larger than the other two variance components in the unconditional 3level model. However, the between teacher variance component was larger than the between class variance component. The smaller variance between class may be attributed to the small number of classes that each teacher taught. In this study, one teacher taught one class, 15 teachers each had two classes, 11 teachers each had three classes and 3 teachers each had four classes. Because of the structure of the 3level model, the variance between classes was captured within the teacher. As such, the 3level model reduced the variability among the classes in contrast to the 2level model that captured the between class variability among 76 classes.
PAGE 112
101 Implications for Practice Teaching vacancies occurring in middle school mathematics classrooms nationwide have caused educators to look for alternative ways to address the problem. There is some evidence to suggest that teachers trained in subject area knowledge as indicated by certification or subject area knowledge are more likely to positively impact student achievement. This study continued to explore this area by examining the impact of teachers with varying levels of preparation in mathematics on student achievement. Because the certification process has been typi cally identified with subject area expertise, the studyÂ’s results have some implications for educators involved with certification regulations. The ILS is also seen by some administrators as having the potential of supporting the instructional needs of the student. The ILS used in this study is unique because it employs a variety of multimedia to deliver the instruction. The instructional format of this ILS is selfcontained and the role of the classroom teacher changes to facilitator of instruction. Some school administ rators may use this information to justify decisions to fill a mathematics vacancy with individuals with less mathematics training than what the state certification requirements suggest. Consequently, the studyÂ’s results have implications for three areas pertaining to teaching vacancies: 1) filling teaching vacancies with teachers with varying levels of mathematics training, 2) alternative certification, and 3) use of technology to support the instructional needs of students. Teaching Vacancies in Mathematics. The results from this study suggest that while controlling for student prior achievement and instructional time, teachers with at least 18 semester hours of mathematics and a methods of teaching mathematics course tend to have a statistically significant different class means compared to teachers without the preparation. When controlling for ILS (instructional method), the effect of mathematics preparation was reduced. However, teachers with the highest mathematics preparation tended to have higher class means by about 8 points than other groups of teachers. While teachers with the highest mathematics training would be the most suitable choice to fill a teaching vacancy, there is not strong evidence to suggest that students would be at a critical disadvantage if taught by individuals with less training in mathematics.
PAGE 113
102 Alternative Certification Choices. States that regulate certification processes have created alternative ways to classify classroom teachers with less than the required subject area courses. Among the terms used by states to signal teachers lacking the requirements needed for certification are outoffield and temporary. Teachers with alternative certifications have opportunities to continue their education and complete the requirements towards obtaining the professional license. This process is consistent with norms established by professional organizations that support the attributes of professionalism in teaching that are based on existing and continuing knowledge. Currently, the State of Florida permits individuals to take an area certification exam without having completed the required courses for the area (FLDOE, 2002). These individuals can be directed towards a prof essional certificate by passing the subject area exam for certification. Consequently, the professional license in Florida no longer insures that an individual has demonstrated successful completion of any subject area courses for the area of certification. This study suggested that having training in mathematics and pedagogy is an important component to studentsÂ’ success in mathematics. Eliminating those components from the certification process may negatively affect student achievement. It is not expected that teachers with certification in mathematics need the kinds of support that teachers with less training do as indicated by the results from this study. While the intent of this study was to examine the relationship between teacher preparation and student achievement, changes in the certification process have complicated the st udy of the preparation effect and inferences that related to certification as a variable. While some believe that allowing individuals to get certified without documentation of subject area competency is a solution to finding certified teachers to fill teaching vacancies, the problem has become more complicated for schools administrators. Other than certification, criteria such as transcripts may need to be included to substantiate the professional competencies of teachers as they apply for teaching positions. Use of Technology to Support Instruction. The unique aspect of an ILS is the branching that takes place when students respond to assessment questions within the software. It responds in different ways depending on the studentsÂ’ input giving the
PAGE 114
103 impression of judgements similar to those made by teachers during instruction when they respond to studentsÂ’ questions. While the results from this study suggested a moderately positive relationship between the use of the ILS and student achievement, it is premature to suggest that the ILS will be suitable for mathematics classes that cannot be staffed with trained teachers. In this study, when controlling for student prior achievement and instructional time, achievement in classes that used the ILS had statistically different means from nonILS classes; however, when controlling for professional preparation, the effect of the ILS was reduced. While teachers with the highest mathematics training would be the most suitable choice to fill a teaching vacancy, there is no evidence to suggest that students would be disadvant aged by individuals with less training in mathematics. Under certain conditions (e.g., using an ILS), differences among groups of teachers with varying levels of mathematics training were reduced. Because the ILS was used to support instruction, other types of instructional support (i.e., mentoring newly hired teachers, modeling lessons, coteaching models of instruction) could have similar effects on student achievement similar to those achieved using the ILS.
PAGE 115
104 References Adams, R. D. & Martray, C. R. (1980). Correlates of teacher perceived problems Paper presented at the Annual Conference of the MidSouth Educational Research Association, New Orleans, LA. (ERIC Doc 195 567). Becker, H. (1993). A model for improving the performance of integrated learning systems: Mixed individualized/group/whole class lessons, cooperative learning, and organizing time for teacherled remediation of small groups. In G. Bailey (Ed.), Computerbased integrated learning systems. Englewood Cliffs, New Jersey: Educational Technology Publications. Becker, H. (1990). Computerbased integrated learning systems in the elementary and middle grades: A critical review. ( ERIC DOC ED 348 939). Begle, E. G. & Geeslin, W. E. (1979). Critical variables in mathematics education: Findings from a survey of empirical literature. (EGK 56560). Reston, VA: National Council of Teachers of Mathematics. ( ERIC DOC ED 171 515). Borg, W. & Gall, M. (1989). Educational research: An introduction Longman, NY. Brown, C. A. & Smith, M. S. (1997). Supporting the development of mathematical pedagogy. Mathematics Teacher 90(2), 13843. Bryk, A. S. & Raudenbush, S. W. (1992). Hierarchical linear models: Applications and data analysis methods. New York: Sage Publications. Buday, M. C. & Kelly, J. A. National Board certification and the teaching professionÂ’s commitment to quality assurance. Phi Delta Kappan 5(1), 215219. Byrne, R. (1983). Inservice programs: What are the essentials for making them effective? NASSP Bulletin 67(461), 17. CadieroKaplan, K. (1999). Integra ting technology: Issues for Traditional. Journal of Computing in Teacher Education, 15(2), 1418.
PAGE 116
105 Carnegie Forum on Education and the Economy. Task Force on Teaching as a Profession. (1986). A nation prepared: Teachers for the 21st century Washington, D.C.: Author. Confrey, J. (1993). Learning to see childrenÂ’s mathematics: Crucial challenges in constructivist reform. In K. Tobin, (Ed.), The practice of constructivism in science education. Hillsdale, NJ: Lawrence Erlbaum Associates. DarlingHammond, L. (1997). What matters most: Investing in quality teaching National Commission on Teaching and AmericaÂ’s Future, New York, NY: Author. DarlingHammond, L., Wise, A. & Klein, S. P. (2000). A license to teach. Raising the standard for teaching JosseyBass. Inc. San Francisco, CA. Dimock, K. V. & Boethel, M. (1999). Constructing knowledge with technology. (ERIC DOC ed431 398). Erbe, B. M. (2000). Correlates of school achievement in Chicago elementary schools. (ERIC DOC ed441 832). Ferguson, P. & Womack, S. T. (1993). The impact of subject matter and education coursework on teaching performance. Journal of Teacher Education 44(1), 5563. Fetler, M. (Mar, 1999). High school staff characteristics and mathematics test results. Education Policy Analysis Archives 7(9) Fletcher, J. D. (July, 1992). Individualized systems of instruction. Institute for Defense Analysis ; Washington, D.C. Florida Government. (2002). 2002 Florida State Statutes. [Online] Internet. Available: www.flsenate.gov/statutes/index.cfm Florida Department of Education. (2001a). Critical Teacher Shortage Areas: 2001 2002. [Online] Internet. Available: www.firn.edu/doe/tchdata Florida Department of Education. (2001b). An overview FloridaÂ’s current certification system. [Online] Internet. Available: www.firn.edu/doe/tchdata Florida Department of Education. (2001c). Review of Florida Educator Certification [Online] Internet. Available: www. cpd.fsu.edu/~DOECertification Review
PAGE 117
106 Florida Department of Education. (2001d). Educator Certification. [Online] Internet. Available: www.firn.edu/doe/bin00022/home00022.htm Glassberg, S. (1980). Student teachi ng: A developmental approach. Journal of Teacher Education 31(2), 3538. Goldhaber, D. D. & Brewer, D. J. (December, 1996). Evaluating the effect of teacher degree level on educational performance ( ERIC DOC ED 406 400). Greene, M. & Zimmerman, S. (2000). The effects of fifth dimension on preservice teacher beliefs. (ERIC Doc 444 533). Guyton, E. & Farokhi, E. (1987). Relationships among academic performance, basic skills, subject matter knowledge, and teaching skills of teacher education graduates. Journal of Teacher Education, 38(5), 3742. Hawk, P. (1985). Certification: It does matter. Journal of Teacher Education 36(3), pp.1315. Ingersol, R. & Gruber, K. (1996). Outoffield teaching and educational quality: Statistical analysis report. National Center for Educational Statistics: Washington, DC, American Institute for Research in the Behavioral Sciences. (ERIC Doc 402 302). Jerald, C. D. (2002). All talk, no action: Putting an end to outoffield teaching (ERIC Doc ED468 741). Kanstoroom, M. & Finn, C. ( 1999). Better teachers, better schools. ( ERIC DOC ED 434 124). Koziol, S. M. (1996). What student t eaching evaluations instruments tell us about emphasis in teaching education programs. Journal of Personnel Evaluation in Education : 10(1); 5374. Kreft, I. (April, 1992). The analysis of small group data: A reanalysis of Webb 1982 with random coefficient model. Paper presented at the Annual Conference of the American Educational Research Association, San Diego, CA. Marsh, H. W., Kong, C. & Hau, K. (2000). Longitudinal multilevel models of the big fish little fish pond effect on academic selfconcept: Counterbalancing contrast and reflected glory effects in Hong Kong schools. ( ERIC DOC ED 442 790).
PAGE 118
107 McConney, A. A., Schalock, M. D. & Schalock, H. D. (1994). Focusing improvement and quality assurance: Work samples as authentic performance measures of prospective teachersÂ’ effectiveness. Journal of Personnel Evaluation in Education : 11(4); 34363. McDermott, P. Rothenberg, J. & Gormley, K. (1998). The best teachers in low income, urban schools: How do they maintain their enthusiasm and excellence? (ERIC Doc 419 027). Monk, D. H. (1994). Subject area preparation of secondary mathematics and science teachers and student achievement. Economics of Education Review 13(2), 12545. National Board for Professional Teaching Standards. (1989). Toward high and rigorous standards for the teaching profession: Initial policies and perspectives of the National Board For Professional Teaching Standards. Detroit, MI: Author. National Center for Education Statistics. (1994). AmericaÂ’s teachers ten years after Â“A Nation at RiskÂ” (NCES 95766). U. S. Department of Education; Office of Educational Research and Improvement, (www.//nces.edu.gov/). National Center for Education Statistics. (April, 1996). Are high school teachers teaching core subjects without college majors on those subjects? (IB196). U. S. Department of Education; Office of Educational Research and Improvement, www.//nces.edu.gov/ National Center for Education Statistics. (January, 1999). Teacher quality: A report on the preparation and qualifications of teachers (NCES 1999080). U. S. Department of Education; Office of Educational Research and Improvement, www.//nces.edu.gov/ National Center for Education Statistics. (October, 1996). Which types of schools have highest turnover? (IB595). U. S. Department of Education; Office of Educational Research and Improvement, www.//nces.edu.gov/ National Commission on Teaching and AmericaÂ’s Future. (1996). What matters most: Teaching for AmericaÂ’s future. Woodbridge, VA: Author. National Council for Accreditation of Teacher Education. (1995). Assuring uality in the practice of teaching: The continuum of teacher preparation. [Brochure]. Washington, DC: Author; ncate@ncate.org. National Council for Accreditation of Teacher Education. (1997). Summary data on teacher effectiveness, teacher quality, and teacher qualifications. Washington, DC: Author; www.ncate.org./resources/factsheettq.htm.
PAGE 119
108 National Council of Teachers of Mathematics. (1989). Curriculum and evaluation standards for school mathematics. Reston, VA: Author. National Council of Teachers of Mathematics. (1991). Professional standards for teaching mathematics. Reston, VA: Author. Neuman, R. (1994). Reconsidering emergency teaching certificates and alternative certification programs as responses to teacher shortages. Urban Education 29(1), 89108. Nunez, M. A. & Dosett, D. (2003). A valueadded methodology. (ERIC Doc 478 202). Opdenakker, M. & Van Damme, J. (1998, April). The importance of identifying levels in multilevel analysis: An illustration of the effects of ignoring the top or intermediate levels in school effectiveness research. Paper presented at the Annual Meeting of the American Educational Research Association, San Diego, CA. Osborne, J. W. (November, 2000). The advantages of hierarchical linear modeling. (ERIC Doc 447 198). Parker, F. (Jan, 1992). U. S. teacher education trends, 199092. (ERIC doc ED 340711). Porter, S. R. & Umbach, P. (2000). Analyzing faculty workload data using multilevel modeling. Paper presented at the Annual Forum of the Association for Institutional Research. (ERIC Doc ED 446 504). Raudenbush, S. (1997). Statistical analysis and optimal design for cluster randomized trials. Psychological Methods. 2(2);173185. Raudenbush, S. & Liu, X. (2000). Statistical power and optimal design for multisite randomized trials. Psychological Methods: 5(2);199213. Schalock, H. D., Scalock, M. D., Cowart, B. & Mynton, D. (1993). Extending teacher assessment beyond knowledge and skills: An emerging focus on teacher accomplishment. Journal of Personnel Evaluation in Education, 7, 105133. Sheehan, J. K. & Han, T. (1996). Hierarch ical modeling techniques to analyze contextual effects: What happened to the aptitude by treatment design? MidWestern Educational Researcher 9 (4), 47. Singer, J. (1998). Using PROC MIXED to fit multilevel models, hierarchical models, and individual growth models. Journal of Educational and Behavioral Statistics 23(4), 32355.
PAGE 120
109 Smith, T. The pocket condition of education:1996. National Center for Educational Statistics, Washington, DC: Author. (ERIC Doc 401 599). Snijders, T. & Bosker, R. (Fall, 1993). Standard errors and sample sizes for twolevel research. Journal of Educational Statistics 18(3), 23759. Swan, K. & Mitrani, M. (1993). The changing nature of teaching and learning in computerbased classrooms. Journal of Research on Computing in Education Fall 26(1) 4054. Stevens, J. P. (1990). Intermediate statistics: A modern approach. Lawrence Erlbaum Associates, Hillsdale, NJ. Taylor, L. (1999). An integrated learning system and its effect on examination performance in mathematics. Computers and Education 32, 95107. Telese, J. A. (1999). The role of social constructivist philosophy in the teaching of school algebra and in the preparation of mathematics teachers. Paper presented at the annual meeting of the Association of Teacher Educators. (ERIC DOC ED 432 469). Tilidetzke, R. (1992). A comparison of cai and traditional instruction in a college algebra course. Journal of Computers in Mathematics and Science Teaching : 11(1); 5362. Tuinnan, J. (2000). Educational technology: The answer to a fundamental educational agenda. Education Canada Sum, 40(2), 2427. Van Dusen, L. M. & Worthen, B. R. ( 1995). Can integrated instructional technology transform the classroom? Educational Leadership 53(2), 2833. Waxman, H. C. & Huang, S. L. (1997). Differences by level of technology use on studentÂ’s motivation, anxiety, and classroom learning environment in mathematics. Journal of Educational Technology Systems, 25(1), 6777. Waxman, H. C. & Huang, S. L. (1997). Classroom instruction differences by level of technology use in middle school mathematics. Journal of Educational Technology Systems, 25(1), 157169. Webster, W. J. Mendro, R. L., Orsak, T. H. & Weerasinghe, D. (1998). An application of hierarchical linear modeling to the estimation of school and teacher effect. Paper presented at the Annual Meeting of the American Educational Research Association, San Diego, CA. (ERIC Doc 424 300).
PAGE 121
110 West, R. C. & Marcote, D. R. (1994). The effects of an integrated learning system (ILS) using incremental time allotments on ninth grade algebra achievement. Journal of Educational Technology Systems 22(3), 283294. Wise, A. (Nov, 1996). Building a system of quality assurance for the teaching profession: Moving into the 21st century. Phi Delta Kappan #(#), 191192. Zeon, S., Lundberg, M. A., Costello, S. M., Gajdostik, L. J., Harmes, N. R., & Roschen, N. A. (1999). Restructuring at the classroom level: Effects with technology. (ERIC Doc 432 302).
PAGE 122
111 Bibliography Apple, M. (1985). Education and power Boston: Routeledge & Ark Paperbacks. Adams, K., Brower, S., Hill, d., & Marshall, I. (2000). The components of an effective mathematics and science middle school: Standards, teaching practices, and professional development. (ERIC Doc ed449 032). American Psychological Association. (1995). Standards for educational and psychological testing Washington, DC: Author. Ashton, P. & Crocker, L. (1987). Systematic study of planned variations: The essential focus of teacher education. Journal of Teacher Education 38(3), 28. Baird, A. W. (Aug, 1990). Alternative certification of teachers. School Business Affairs : 56(8), 1821. Ball, D. L. (1991). Teaching mathematics for understanding: What do teachers need to know about subject matter? In M. Kennedy (Ed.), Teaching academic subjects to diverse learners New York: Teachers College Press. Begle, E. G. (1979). Critical variables in mathematics eduction: Findings from a survey of empirical literature. (ERIC Doc 171 515) Begle, E. G. & Geeslin, W. E. (1972). NLSMA Reports No. 28: Teacher effectiveness in mathematics instruction. (ERIC Doc 184 130) Borko, H. & Putnam, R. T. (1995). Expanding a teacherÂ’s knowledge base: A cognitive psychological perspective on professional development. (In T. R. Guskey & M Huberman (Eds.), Cognition, education and multimedia: Exploring ideas in high technology Hillsdale, NJ: Lawrence Erlbaum. Bradshaw, L. & Hawk, P. (1996). Teacher certification: Does it really make a difference in student achievement? ( ERIC DOC ED 405 308). Bureau of Curriculum, Instruction, and AssessmentAssessment and Evaluation Section. (1999). Technical Report: 1998 Florida Comprehensive Assessment Test. Tallahassee, Fl: CTB/McGrawHill.
PAGE 123
112 Christiansen, I. (Mar, 1999). Are theories in mathematics education of any use to practice? For the Learning of Mathematics 19 (1), 2023. CochranSmith, M. (2001). Constructing outcomes in teacher education: Policy, practice, and pitfalls. Education Policy Analysis Archives, 9(11). http://olam.ed.asu.edu/epaa/v9n11.html Cohen, D. K. & Hill, H. C. (1998). State policy and classroom performance: Mathematics reform in California (ERIC Doc 418 842). DÂ’Ignazio, F. (October, 1992). Multimedia sandbox: Teaching, learning, and the transfer of knowledge. Computing Teacher 20(2 ), 5455 DarlingHammond, L. (1987). Teacher quality and equality. In Access to Knowledge Keating, P. And Goodlad, J. I. New York: College Board Entrance Examination Board. DarlingHammond, L. (November, 1996). What matters most: A competent teacher for every child. Phi Delta Kappan 79(3 ), 190200. Dial, M. & Stephens, C. (Nov, 1993). Introduction: The context of alternative teacher certification. Education and Urban Society 26(1), 417. Ernest, P. (1998). Social constructivism as a philosophy of mathematics. Albany, NY: State University of New York Press. Evertson, C., Hawley, & Zlotnik. (1985). Making a difference in educational quality through teacher education. Journal of Teacher Education 36(3), 212. Feldt, L. S. & Ankermann, R. D. (1998). Appropriate sample size for comparing alpha reliabilities. Applied Psychological Measurement 22(2), 170178. Florida Department of Education. (2001). State Board of Education Administrative Rules. [Online] Internet. Available: www.firn.edu/doe/rules/rules.htm Fosnot, C. (1996). Constructivism: A psyc hological theory of learning. In. C. Fosnot (Ed.), Constructivism: Theory, perspectives, and practice. New York: Teachers College Press. Gau, S. (March, 1997). The distribution and the effects of opportunity to learn on mathematics achievement. Paper presented at the Annual Meeting of the American Educational Research Association, Chicago, IL. (ERIC DOC 407 231)
PAGE 124
113 Goldhaber, D. D. & Brewer, D. J. (October, 1998). When should we reward degrees for teachers? Phi Delta Kappan 80(2), pp 134138. Hinkle, D., Wiersma, W., and Jurs, S. (1994). Applied statistics for the behavioral sciences. Houghton Mifflin: Boston, MA. Hillsborough County Public Schools (2001). OutofField Teaching. [Online] Internet. Available: www.sdhc.k12.fl.us. Hussar, W. (1999). Predicting the need for newly hired teachers in the United States. U. S. Department of Education; Office of Educational Research and Improvement www.//nces.edu.gov/ Jarrett, D. (1998). Integrating technology into middle school mathematics: ItÂ’s just good teaching. ( ERIC DOC ED 427 961). Jones, J. A. (1994). Integrated learning systems for diverse learners. Media and Methods, SeptOct, 31(1), 1416. Karsenti, T. & Thibert, G. (April, 1998). The interaction between teaching practices and the change in motivation of elementary school children. Paper presented at the Annual Conference of the American Educational Research Association, San Diego, CA. Kasprzyk, D. (October, 1996). National assessments of teacher quality ( Paper No. 9624). U. S. Department of Education; Office of Educational Research and Improvement www.//nces.edu.gov/ Kober, N. (1991). What we know about mathematics teaching and learning. ( ERIC DOC ED 343 793). Kozol, J. (1995). Pedagogical emphasis in practice in NBPTS EA/ELA standards. Paper presented at the Annual Meeting of Nati onal Council of Teachers of English, San Diego, CA. (ERIC Doc 398 572). Kulik, J. A., Kulik, C. C., and Cohen, P. A. (1980). Effectiveness of computerbased college teaching: A metaanalysis of findings. Review of Educational Research 50, 52544. Labaree, D. (1992). Power, knowledge a nd rationalization of teaching. A genealogy of the movement to professionalize teaching. Harvard Educational Review, 62, 12354.
PAGE 125
114 Mandeville, G. and Liu, Q. (1997). The effect of teacher certification and task level on mathematics achievement. Teaching and teacher education, 13(4), pp.397407. McDiarmid, G. W. & Wilson, S. M. (April, 1991). An exploration of the subject matter knowledge of alternative route teachers: Can we assume they know their subject. Journal of Teacher Education, 42(2), pp93103. Means, B. & Olson, K. (1997). Technology and educational reforms. Washington, D. C.: U. S. Department of Education. Miller, J. McKenna, B. & McKenna, M. (1998). A comparison of alternatively and traditionally prepared teachers. Journal of Teacher Education 49(3), 16576. Mosenthal, J. H. & Ball, D. L. (1988) Constructing new forms of teaching: Subject matter knowledge in inservice teacher education. Journal of Teacher Education, 43 (5), 347356. National Center for Education Statistics. (October, 1996). Outoffield teaching and educational quality (NCES 96040). U. S. Department of Education; Office of Educational Research and Improvement (www.//nces.edu.gov/). National Center for Education Statistics. (September,2000). Teachers tools for the 21stcentury: A report on teachersÂ’ use of technology (NCES 2000102). U. S. Department of Education; Office of Educational Research and Improvement, www.//nces.edu.gov/). National School Board Association (author). (2002). Better teachers, better schools. http://www.nsba.org/nepn/newsletter/297.htm Novick, M. R. & Lewis, C. (1968). Coefficient alpha and the reliability of composite measurements. Psychometrika 32, 113. Peck, H. I. (1989). The effect of certification status on the performance of mathematics teachers: A pilot study. Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco, CA. (ERIC DOC ED 307322). Pedhauser and Schmelkin, (1991). Measurement, design, and analysis: An integrated approach Lawrence Erlbaum Associates: Hillsdale, NJ. Pisapia, J. Schlesinger, J. & Parks, A. (1993). Learning technologies in the classroom: Review of the literature. ( ERIC DOC ED 411 365).
PAGE 126
115 Ramp, L. C. & Guffey, J. S. (November, 1998). Technology Effect: The promise of enhanced academic achievement Paper presented at the MidSouth Educational Research Association Conference, New Orleans, LA. (ERIC DOC ED 427 771). Raju, N. (1982). The reliability of a criterionreferenced composite with the parts of the composite having different cutting scores. Educational and Psychological Measurement 42, 113129. Sandholtz, J. H., Ringstaff, C. & Dwyer, D. C. (1997). Teaching with technology: Creating studentcentered classrooms New York: Teachers College, Columbia University. Schalock, M. D. (1998). Accountability, student learning, and the preparation and licensure of teachers: OregonÂ’s teacher work sample methodology. Journal of Personnel Evaluation in Education, 12(3), 269285. Schalock, M. D. Scalock, H. D. & Edwards, J. (1993). Extending teacher assessment and professional development beyond knowledge and skills: Applying student learning gains to a faculty incentive program/reward program. Journal of Personnel Evaluation in Education 7, 159178. Sedere, M. U. & Feldt, L. S. (1976). The sampling distributions of the kristof reliability coefficient, the feldt coefficient, and guttmanÂ’s lambda2. Journal of Educational Measurement, 14(1), 5363. Senese, D. J. (1984). Technological advances Â– new opportunities for educators. Paper presented at a Meeting of the Regional Schools Contract Planning Committee of the Virginia Department of Education, Massanetta Springs, VA. ( ERIC DOC ED 298 882). Shulman, G. M. & Luechauer, D. L. (Nov, 1991). Creating empowered learners: Merging content and process. Paper presented at the Annual Lilly Conference on College Teaching, Indianapolis, IN. ( ERIC DOC ED 361 754). Steffe, L. & Gale, J. (Eds.). (1995). Constructivism in education. Hillsdale, NJ: Lawrence Erlbaum Associates. Van Dusen, L. M. & Worthen, B. R. (1992). Factors that facilitate or impede implementation of integrated learning systems. Educational Technology 32(9), 1621. Von Minden, A. M. Walls, R. T. & Nardi, A. H. (1998). Charting the links between mathematics content and pedagogy concepts: Cartographies of cognition. Journal of Experimental Education, 66(4), 33958.
PAGE 127
116 Walker, C. M. (1999). The effect of different pedagogical approaches on mathematics studentsÂ’ achievement. Paper presented at the Annual Meeting of the American Educational Research Association, Montr eal, Canada. ( ERIC DOC ED 431 011). Wiburg. Karin. (February, 1995). Integrated learning systems: What does the research say? The Computing Teacher, # (). Wiley, D. &Yoon, B. (1995). Teacher reports on opportunity to learn: Analyses of the 1993 California learning assessment system. Educational evaluation and policy analysis, 17(3), 35570.
PAGE 128
117 Appendices
PAGE 129
118 R E C R U I T M E N T E N T R Y PRESERVICE PROGRAM 1. Liberal Arts Education 2. Subject Matter Preparation 3. Professional and Pedagogical Studies 4. Clinical Studies STATE SUBJECT MATTER AND TEACHING KNOWLEDGE ASSESSMENT G R A D U A T I O N INTERN PROGRAM 1. Clinical Practice 2. Mentoring 3. Clinical Studies 4. Professional Development School (PDS) NO COLLEGE ASSESSMENTS COLLEGE ASSESSMENTS INTERNSHIP ASSESSMENTS COLLEGE ASSESSMENTS NO NCATE / State Quality AssuranceASSURING QUALITY IN THE PRACTICE OF TEACHING:The Continuum of Teacher PreparationPhase I Preservice Preparation YES Appendix A: NCATEÂ’S Quality Assurance System for the Teaching Profession
PAGE 130
119ASSURING QUALITY IN THE PRACTICE OF TEACHING:The Continuum of Teacher PreparationPhase II Extended Clinical Preparation and Assessment 1 STATE LICENSE ASSESSMENTS (Application of Knowledge and Skills) REGULAR LICENSE 3 YES NO PHASE I PreService PreparationYESState Licensing Authority Quality Assurance
PAGE 131
120ASSURING QUALITY IN THE PRACTICE OF TEACHING:The Continuum of Teacher PreparationPhase III Extended Professional Development PHASE I PreService Preparation Professional Development NBPTS* Assessment License Renewal Advanced Certification Continuing Professional DevelopmentState /School District Quality Assurance NO NBPTS System*NBPTS National Board for Professional Teaching Standards
PAGE 132
121 Appendix B: FloridaÂ’s Mathematics Certification RequirementsSpecialization Requirements for Certification Area of CertificationCourse Requirements Middle Grades Integrated Curriculum (Grades 69)11. A bachelorÂ’s or higher degree with a degree major in middle grades education which includes a minimum of twelve hours in each of the following areas: English, mathematic s, science, and social science. 2. A bachelorÂ’s or higher degree with a degree major in a subject other than middle grades education and 54 semester hours in English, mathematics, science, and social science. Eighteen semester hours shall be completed in one of the 4 subject areas. Those 18 hours shall be the same as those required for middle school certification in that area. At least 12 semester hours shall be completed in each of the remaining subject areas. Middle Grades Mathematics (Grades 59)21. A bachelorÂ’s or higher degree with an undergraduate or graduate major in mathematics or middle grades mathematics. 2. A bachelorÂ’s or higher degree with 18 semester hours in mathematics to include credit in the ar eas specified below: (a) Calculus, precalculus, or trigonometry, (b) Geometry, and (c) Probability or statistics. Mathematics (Grades 612)31. A bachelorÂ’s or higher degree with an undergraduate or graduate major in mathematics or middle grades mathematics. 2. A bachelorÂ’s or higher degree with 30 semester hours in mathematics to include credit in the ar eas specified below: (a) Six (6) semester hours in Calculus (b) Credit in Geometry, (c) Credit in probability or statistics, and (d) Credit in abstract or linear algebra. 3. A bachelorÂ’s or higher degree with specialization requirements completed for physics and 21 semester hours in mathematics to include credit in the areas specified below: (a) Six (6) semester hours in Calculus (b) Credit in Geometry, (c) Credit in probability or statistics, and (d) Credit in abstract or linear algebra. 1 [Specific Authority 229.053(1), 231.15(1), 231.17(1) FS. Law Implemented 231.02(1), 231.15(1), 231.17(1), FS. History New 42596]2 [Specific Authority 229.053(1), 231.15(1), 231.17(3) FS. Law Implemented 229.053, 231.145, 231.15, 231.17, FS. History New 9192, Amended 7172000]3 Specific Authority 229.053(1), 2 31.15(1), 231.17(3) FS. Law Implem ented 231.02, 231.145, 231.15, 231.17, FS. History New 7190, Amended 7172000]
PAGE 133
122 Appendix C : NCTM Strategies for Teaching NCTM Goals for Professional Teaching Standards The teacher of mathematics should pose tasks that: Â•engage studentsÂ’ intellect, Â•develop studentsÂ’ mathematical understandings and skills, Â•stimulate students to make connections and develop a coherent framework for mathematical ideas, Â•call for problem formulation, problem solving and mathematical reasoning, Â•promote communication about mathematics, Â•represent mathematics as an ongoing human activity Â•display sensitivity to and draw on students diverse background experiences and dispositions Â•promote the development of all studentsÂ’ dispositions to do mathematics Discourse refers to the ways teachers or students think and express themselves in the classroom. The teacherÂ’s role in discourse is to: Â•pose questions and tasks that elicit, engage, and challenge each studentÂ’s thinking Â•listening carefully to students ideas Â•asking students to clarify and justify their ideas orally and in writing Â•deciding what to pursue in depth from id eas that students bring up in discussion Â•deciding when and how to attach mathematical notation and language to studentsÂ’ ideas Â•deciding when to provide information, when to clarify an issue, when to model, when to lead, and when to let a student struggle with a difficulty Â•monitoring studentsÂ’ participation in discussions and deciding when and how to encourage each student to participate The tools teachers use for enhancing discourse are: Â•computers, calculators, and other technology Â•concrete materials used a s models Â•pictures, diagrams, tables, and graphs Â•invented and conventional terms and symbols Â•metaphors, analogies, and stories Â•written hypotheses, explanations, and arguments Â•oral presentations and dramatizations
PAGE 134
123 NCTMÂ’S GOALS for Professional Teaching Standards conÂ’t Environment classroom culture that influences learning Â•provide and structure the time necessary to explore sound mathematics and grapple with significant ideas and problems Â•use the physical space and materials in ways that facilitate studentsÂ’ learning Â•providing a context that encourages the development of mathematical skills and proficiency Â•respecting and valuing studentsÂ’ ideas, ways of thinking, and mathematical dispositions Â•encouraging students to work independently or collaboratively to make sense of mathematics Â•expecting students to take intellectual risks by raising questions and formulating conjectures Â•expecting and encouraging students to display a sense of mathematical competence by validating and supporting ideas with mathematical argument Analysis refers to reflections on teaching/learning Â•a teacher should be observing, listening to, and gathering other information about students to assess what they are learning Â•examining effects of the tasks, discour se, and learning environment on studentsÂ’ mathematical knowledge, skills, and dispositions Â•ensure that every student is learning sound and significant mathematics and developing a positive disposition toward mathematics Â•challenge and extend studentsÂ’ ideas Â•adapt or change activities while teaching Â•make plans, both shortand longrange Â•describe and comment on each studentsÂ’ learning to parents and administrators as well as to the students themselves
PAGE 135
124 Appendix D: MJ3 Syllabus (Grade 8 Mathematics)
PAGE 136
125
PAGE 137
126
PAGE 138
127
PAGE 139
128
PAGE 140
129
PAGE 141
130
PAGE 142
131 Appendix E Teacher SurveyINSTRUCTIONAL PRACTICES SURVEYI Can Learn Classes Traditional ClassesThis folder contains a survey for all MJ3 teachers at schools with an I Can Learn classroom. Teachers, who taught MJ3 during the 20002001 school year, are be ing asked to respond. Survey teach ers include those who used an ICL classroom and teachers who did not. The purpose of this survey is to describe MJ3 instruction. The 20002001 school year was the first year of implementation of the I Can Learn classroom. While some teachers in the I Can Learn classroom were observed, teachers in regular MJ3 cla sses were not. The intent of the surv ey is to provide the reader with a description of how inst ruction took place in a ty pical MJ3 classroom. The survey items are intended to reflect the type of instructional practice taking place in an MJ3 classroom during the 20002001 school year (initial y ear of implementation). Please, try to remember how you taught the MJ3 curriculum during the 20002001 school year if you were using the I Can Learn lessons or if you were teaching in a traditional classroom. Th e responses you make must reflect the instructional activities that took place during the 20002001 school year. The District values your experiences and opinions. By participating and returning the completed survey, you are contributing to decisionmaki ng regarding policy and program developmen t. Future students and teachers will benefit directly from your input. This survey should take about 15 minutes to complete. Be assured that names of schools or teachers will not be revealed for any other purpose other than de scribing the instruction of MJ3 to middle school students. Your particip ation is voluntary. Your responses will be tallied with other teachers and kept anonymous Your participation is voluntary and in no way will affect your teaching position. Your principal is aware that the I Can Learn program is being evalua ted, but does not know which teachers may be involved in survey data collection. Once the survey data are formatted into a data set, the surveys will be destroyed. If there are any questions, please call Christine Kerstyn at (813) 2724341. If you wish to participate in this survey, please return the survey in the enclosed envelope to the Department of Assessment, Accountability, and Evaluati on by October 30, 2002. The survey ite ms will be aggregated to reflect the two groups of MJ3 teachers categorized by the instructional strategy used in the classroom. Demographics Please circle an appropriate answer to each item. 1.At the end of 20002001, how many total years have you taught mathematics at the middle or high school level? a)<= 3 years b) 4 Â– 6 yearsc) 7 Â–10 yearsd) 11 + years 2. Did you use an I Can Learn classroom in 20002001 for teaching mathematics? a) YESb) NO 3.How many minutes per period did you have for classroom activities in 20002001? a) 45 minutes b) 50 minutesc) 90 minutes
PAGE 143
132Survey Items Â– Instructional PracticePlease think about the 20002001 school year when you taught MJ3. Given a two week period, (10 full days of instructional time), what percent of time would the following instructional strategy be used in pr esenting a lesson to your students? Circle the percent that closest meets your answer. 1)lecture to a whole group a) Noneb) 149%c) 50%e) 5175%f) 76 100% 2)lecture to a small group a) Noneb) 149%c) 50%e) 5175%f) 76 100% 3)peer teaching (students teaching students) a) Noneb) 149%c) 50%e) 5175%f) 76 100% 4)use of technology (to support instruction) a) Noneb) 149%c) 50%e) 5175%f) 76 100% 5)discovery learning a) Noneb) 149%c) 50%e) 5175%f) 76 100% 6)use of manipulatives a) Noneb) 149%c) 50%e) 5175%f) 76 100% 7)administer pretest a) Noneb) 149%c) 50%e) 5175%f) 76 100% 8)administer quizzes a) Noneb) 149%c) 50%e) 5175%f) 76 100% 9)administer chapter (unit) tests a) Noneb) 149%c) 50%e) 5175%f) 76 100% 10)administer a cumulative test a) Noneb) 149%c) 50%e) 5175%f) 76 100% 11)give a progress report a) Noneb) 149%c) 50%e) 5175%f) 76 100% Please continue on the other side.
PAGE 144
133Survey Items Â–Instructional Delivery Please think about the 20002001 school year when you taught MJ3. Given the following MJ3 topics, please indicate the instructional strategy you used to teach the topic for the first time to students. Circle all the given choices that apply. If you used a different method to teach the lesson, please write your answer in the provided spa ce. 12)Translate verbal expressions and sentences into al gebraic expressions and equations. Circle all the choices that apply. a) Lecture b) Discovery Techniques c) I Can Learn d) Cooperative Learning/Peer tutoring e) Other ___________________________________________________________________________________________________ ________________________________________________________________________13)Solving and graphing Inequalities. Circle all the choices that apply. a)Lecture b) Discovery Techniques c) I Can Learn d) Cooperative Learning/Peer tutoring e) Other ___________________________________________________________________________________________________ _______________________________________________________________________________________ 14)Use proportions and equations to solve percent problems Circle all the choices that apply. a)Lecture b) Discovery Techniques c) I Can Learn d) Cooperative Learning/Peer tutoring e) Other ___________________________________________________________________________________________________ _______________________________________________________________________________________ 15)Find the perimeter and area of polygons. Circle all the choices that apply. a)Lecture b) Discovery Techniques c) I Can Learn d) Cooperative Learning/Peer tutoring e) Other ___________________________________________________________________________________________________ _______________________________________________________________________________________ 16)Graph two variable relationships. Circle all the choices that apply. a)Lecture b) Discovery Techniques c) I Can Learn d) Cooperative Learning/Peer tutoring e) Other ___________________________________________________________________________________________________ _______________________________________________________________________________________ 17)Evaluate polynomials. Circle all the choices that apply. a)Lecture b) Discovery Techniques c) I Can Learn d) Cooperative Learning/Peer tutoring e) Other ___________________________________________________________________________________________________ _______________________________________________________________________________________ 18)Naming similar or congruent triangles. Circle all the choices that apply. a)Lecture b) Discovery Techniques c) I Can Learn d) Cooperative Learning/Peer tutoring e) Other ___________________________________________________________________________________________________ _______________________________________________________________________________________Thank you! Your responses are valued.Appendix F Survey Results
PAGE 145
134 Frequencies of Responses for Whole GroupFrequency of Teachers Using a Mathematics In struction Strategy in a TwoWeek Period Percent of Time Used in 2 Week Period StrategyNone149%50%5175%76100%Missing lecture to whole group21112672111616 lecture to small group31713721616 peer teaching (students teaching students)161269317211 Use of technology (to support instruction)16739211844 discovery learning317116116317 use of manipulatives5288441631716 administer pretest422950528 administer quizzes844211844 administer chapter (unit) tests16844 1616739 administer cumulative tests317739211633 give a progress report844211211633 Frequency of Responses For Teachers Choosing a Teaching Strategy for Mathematics Objective Strategy*1ObjectiveLectureDiscovery techniques I Can LearnCoop Learning /Peer Tutoring n%n%n%n % Translate verbal expressions and sentences into algebraic expressions and equations 1439261233822 Solving and graphing inequalities*21542381028822 Use proportions and equations to solve percent problems 1439381131822 Find perimeter and area of polygons1233381233925 Graph two variable relationships1541381130822 Evaluate polynomials1338261235721 Naming similar or congruent triangles*214503111139*1 Duplicated Count *2 Not included in ILS lessons
PAGE 146
135 Percent of Time Teachers Use a Strategy in a TwoWeek period for Mathematics Instruction By Type of Number of Minutes in Period Percent of Time Used in 2 Week Period StrategyInstructional Time per period (N) None149%50%5175%76100%Missing lecture to whole group< = 50 min (6)141 90 min (9)1521 Not identified (3)3 lecture to small group< = 50 min (6)231 90 min (9)171 Not identified (3)3 peer teaching (students teaching students) < = 50 min (6)51 90 min (9)531 Not identified (3)12 Use of technology (to support instruction) < = 50 min (6)1113 90 min (9)54 Not identified (3)111 discovery learning< = 50 min (6)15 90 min (9)1413 Not identified (3)12 use of manipulatives< = 50 min (6)231 90 min (9)3411 Not identified (3)111 administer pretest< = 50 min (6)321 90 min (9)153 Not identified (3)21 administer quizzes< = 50 min (6)42 90 min (9)324 Not identified (3)12 administer chapter (unit) tests < = 50 min (6)132 90 min (9)315 Not identified (3)21 administer cumulative tests < = 50 min (6)222 90 min (9)144 Not identified (3)12 give a progress report< = 50 min (6)42 90 min (9)3123 Not identified (3)111
PAGE 147
136 Percent of Time Teachers Use a Strategy in a TwoWeek period for Mathematics Instruction By Type of Instructional Method Percent of Time Used in 2 Week Period StrategyMethod of Instruction (N) None149%50%5175%76100%Missing lecture to whole groupILS (11)281 Not ILS (5)221 Not identified (2)2 lecture to small groupILS (11)1811 Not ILS (5)23 Not identified (2)2 peer teaching (students teaching students) ILS (11)812 Not ILS (5)32 Not identified (2)11 Use of technology (to support instruction) ILS (11)317 Not ILS (5)14 Not identified (2)11 discovery learningILS (11)173 Not ILS (5)131 Not identified11 use of manipulativesILS (11)5321 Not ILS (5)41 Not identified (2)11 administer pretestILS (11)344 Not ILS (5)14 Not identified (2)11 administer quizzesILS (11)515 Not ILS (5)311 Not identified (2)2 administer chapter (unit) tests ILS (11)1415 Not ILS (5)32 Not identified (2)11 administer cumulative tests ILS (11)254 Not ILS (5)122 Not identified (2)2 give a progress reportILS (11) 6113 Not ILS (5)212 Not identified (2)11
PAGE 148
137 Number of Teachers Choosing a Teaching Strategy for Mathematics Objective By Number of Minutes in Period Strategy ObjectiveInstructional Time per period (N) LectureDiscovery techniques I Can LearnCooperative Learning/ Peer Tutoring Translate verbal expressions and sentences into algebraic expressions and equations < = 50 min (6)4051 90 min (9)7254 Not identified (3)323 Solving and graphing inequalities*< = 50 min (6)5131 90 min (9)7254 Not identified (3)323 Use proportions and equations to solve percent problems < = 50 min (6)5042 90 min (9)6353 Not identified (3)323 Find perimeter and area of polygons< = 50 min (6)3051 90 min (9)5355 Not identified (3)323 Graph two variable relationships< = 50 min (6)4151 90 min (9)8244 Not identified (3)323 Evaluate polynomials< = 50 min (6)3051 90 min (9)7253 Not identified (3)323 Naming similar or congruent triangles*< = 50 min (6)5041 90 min (9)6354 Not identified (3)323 *Not included in ILS lessons
PAGE 149
138 Number of Teachers Choosing a Teaching Strategy for Ma thematics Objective By Type of Instructional Method Strategy ObjectiveMethod of Instruction LectureDiscovery techniques I Can LearnCooperative Learning/ Peer Tutoring Translate verbal expressions and sentences into algebraic expressions and equations ILS (11)71113 Not ILS (5)5103 Not identified (2)212 Solving and graphing inequalities*ILS (11)8093 Not ILS (5)5303 Not identified (2)212 Use proportions and equations to solve percent problems ILS (11)70103 Not ILS (5)5303 Not identified (2)212 Find perimeter and area of polygonsILS (11)60113 Not ILS (5)4304 Not identified (2)212 Graph two variable relationshipsILS (11)80103 Not ILS (5)5303 Not identified (2)212 Evaluate polynomialsILS (11)60113 Not ILS (5)5202 Not identified (2)212 Naming similar or congruent triangles*ILS (11)70103 Not ILS (5)5303 Not identified (2)212 *Not included in ILS lessons
PAGE 150
About the Author Christine Kerstyn received a BachelorÂ’s Degree in Mathematics and Computer Science from Northeastern Illinois University in 1973. She started teaching high school mathematics in Chicago, Illinois in1973 and continued her teaching career after a move to Florida in 1985. Shortly after, Ms. Kerstyn received a MasterÂ’s Degree in Instructional Technology from the University of South Florida where she also entered the Ph.D. program in Educational Measurement and Research. While in the Ph.D. at the University of South Florida, Ms. KerstynÂ’s interests were in the area of the design of assessment reports and the use of assessment data as an instructional tool. She coauthored a publication where her design of an assessment report for an ungraded mathematics program was presented. Ms. Kerstyn also made several presentations at educational researchersÂ’ meetings in the area of reporting assessments and the impact of technology on student mathematics achievement.
