USF Libraries
USF Digital Collections

Educational policy analysis archives


Material Information

Educational policy analysis archives
Physical Description:
Arizona State University
University of South Florida
Arizona State University
University of South Florida.
Place of Publication:
Tempe, Ariz
Tampa, Fla
Publication Date:


Subjects / Keywords:
Education -- Research -- Periodicals   ( lcsh )
non-fiction   ( marcgt )
serial   ( sobekcm )

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
usfldc doi - E11-00285
usfldc handle - e11.285
System ID:

This item is only available as the following downloads:

Full Text
xml version 1.0 encoding UTF-8 standalone no
mods:mods xmlns:mods http:www.loc.govmodsv3 xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govmodsv3mods-3-1.xsd
mods:relatedItem type host
mods:identifier issn 1068-2341mods:part
mods:detail volume mods:number 10issue 37series Year mods:caption 20022002Month September9Day 66mods:originInfo mods:dateIssued iso8601 2002-09-06

xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam a22 u 4500
controlfield tag 008 c20029999azu 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E11-00285
0 245
Educational policy analysis archives.
n Vol. 10, no. 37 (September 06, 2002).
Tempe, Ariz. :
b Arizona State University ;
Tampa, Fla. :
University of South Florida.
c September 06, 2002
Effectiveness of "Teach for America" and other under-certified teachers on student academic achievement : a case of harmful public policy / Ildiko Laczko-Kerr [and] David C. Berliner.
x Research
v Periodicals.
2 710
Arizona State University.
University of South Florida.
1 773
t Education Policy Analysis Archives (EPAA)
4 856


1 of 53 Education Policy Analysis Archives Volume 10 Number 37September 6, 2002ISSN 1068-2341 A peer-reviewed scholarly journal Editor: Gene V Glass College of Education Arizona State University Copyright 2002, the EDUCATION POLICY ANALYSIS ARCHIVES .Permission is hereby granted to copy any article if EPAA is credited and copies are not sold. EPAA is a project of the Education Policy Studies Laboratory. Articles appearing in EPAA are abstracted in the Current Index to Journals in Education by the ERIC Clearinghouse on Assessment and Evaluation and are permanently archived in Resources in Education .The Effectiveness of "Teach for America" and Other Under-certified Teachers on Student Academic Achievement: A Case of Harmful Public Policy 1 Ildiko Laczko-Kerr Arizona Department of Education David C. Berliner Arizona State UniversityCitation: Laczko-Kerr, I., & Berliner, D.C.. (2002, September 6). The effectiveness of "Teach for America" and other under-certified teachers on stud ent academic achievement: A case of harmful public policy," Education Policy Analysis Archives 10 (37). Retrieved [date] from academic achievements of students taught by und er-certified primary school teachers were compared to the academ ic achievements of students taught by regularly certified primary scho ol teachers. This sample of under-certified teachers included three t ypes of under-qualified personnel: emergency, temporary and provisionally


2 of 53certified teachers. One subset of these under-cert ified teachers was from the national program "Teach For America (TFA)." Re cent college graduates are placed by TFA where other under-quali fied under-certified teachers are often called upon to work, namely, low -income urban and rural school districts. Certified teachers in this study were from accredited universities and all met state requireme nts for receiving the regular initial certificate to teach. Recently hir ed under-certified and certified teachers (N=293) from five low-income sch ool districts were matched on a number of variables, resulting in 109 pairs of teachers whose students all took the mandated state achievem ent test. Results indicate 1) that students of TFA teachers did not p erform significantly different from students of other under-certified te achers, and 2) that students of certified teachers out-performed studen ts of teachers who were under-certified. This was true on all three s ubtests of the SAT 9—reading, mathematics and language arts. Effect s izes favoring the students of certified teachers were substantial. I n reading, mathematics, and language, the students of certified teachers ou tperformed students of under-certified teachers, including the students of the TFA teachers, by about 2 months on a grade equivalent scale. Studen ts of under-certified teachers make about 20% less academic growth per ye ar than do students of teachers with regular certification. Traditiona l programs of teacher preparation apparently result in positive effects o n the academic achievement of low-income primary school children. Present policies allowing under-certified teachers, including those from the TFA program, to work with our most difficult to teach c hildren appear harmful. Such policies increase differences in ach ievement between the performance of poor children, often immigrant and m inority children, and those children who are more advantaged. There has been growing interest in "teacher quality ," a catch phrase for a host of teacher characteristics, including a teacher's content know ledge, classroom behavior, academic ability, advanced degree work, salary, and teacher education experiences. Among the many characteristics under investigation as an indicator of teacher quality has been teacher certification. This study examines the effects of different kinds of teacher certification on student achievement. Reviews of this issue may be found in Darling-Hammond, 1999 and 2002; Evertson, Hawley & Zlotnik, 1985; and Ashton, Crocker, & Olejnik, 1986. In Arizona, a state with a strong commitment to sta ndards based reform, policies were needed to ensure that quality teachers would be available for students to meet the new and more rigorous mandated standards. Thus the Arizona Educ ator Proficiency Assessment (AEPA) was developed as a tool in the state certification proc ess to ensure the quality of new teachers. One part of the test purports to measure teachers' prof essional knowledge, including pedagogy, teaching methods, and educational theory. The seco nd part of the test covers content knowledge, either elementary content, or for second ary teachers, a subject specific content area. A passing score on the test, clearance by th e police of any criminal record, and an accredited university's recommendation that a perso n is prepared to work as a classroom teacher earns a regular certificate to teach from the State Arizona's efforts are part of a national movement t o improve the quality of teachers through


3 of 53assessments like the AEPA ("Quality Counts," 2000; Higher Education Reauthorization Act, 1998). But not every district in the state or nati on can find regularly certified teachers, giving rise to other policies that appear to work against the goal of increased teacher quality. For example, in Arizona and elsewhere, attempts at impr oving the quality of the teaching force seem contradicted by the continuing practice of iss uing emergency certification (see "Quality Counts," 2000; Olson, 2000). Critics of hiring unc ertified teachers ask whether complex, standards—based reforms can be enacted with teacher s who are, to varying degrees, untrained. Supporters of hiring uncertified teachers claim tha t the advantages of traditional teacher education programs are unproven, and some question, as well, whether such training is even necessary. Stated in its simplest form as a resear ch problem the question is: "Do students taught by teachers with emergency certification lea rn as much or achieve as well as students who are taught by regularly certified teachers?" An answer to this simple question would inform us whether policies designed to improve teac her quality are being undermined by the simultaneous adoption of policies that allow the us e of uncertified teachers. The dilemma associated with using uncertified teach ers is not limited to Arizona where, currently, one out of six teachers are estimated to be uncertified (Go, 2002). For example, the Chicago "Sun Times" (Rossi & Grossman, 2002) report s an audit by the Chicago Board of Education showing that 22 percent of teachers in th e system's 81 probationary schools—those with the greatest academic needs and the lowest tes t scores—were not fully qualified to teach. These were teachers missing what the state calls "i nitial'' or "standard'' certificates. Other teachers were found with certificates, but they wer e teaching subjects they were not certified to teach. In all, 900 teachers, about one of every fi ve in Chicago's worst-performing public schools appeared unqualified to teach during the sc hool year 2001-2002. New York State appears to be no different. Lankford, Loeb and Wyc off (2002) report that in a recent school year, in some New York schools, less than half the teachers held certification for the courses they taught. These schools were invariably urban a nd serving the poor. With the passage of the No Child Left Behind Act of 2001 (HR1), federal law will require schools to have a "highly qualified" teacher in eve ry classroom by 2005-2006. Thus research on the effectiveness of uncertified and certified t eachers takes on added significance as the designation of teachers as uncertified and certifie d becomes entwined with the evaluations of teachers so that the highly qualified can be distin guished from those less qualified.Related ResearchThe review that follows explores the difference bet ween licensure and certification, reasons for teacher certification, the role of certification in the professionalizaton of the teaching workforce, on-the-job training for teachers, and th e assessment of beginning teachers' competency. After those topics are discused briefl y, research that bears on a broad range of certification issues and teacher effectiveness is d iscussed in somewhat more detail. These areas are all highly contested and interpretations of thi s literature are, more than usual, intertwined with the ideology of the researchers (Cochran-Smith & Fries, 2001). Licensure and CertificationTeacher certification, at its core, is based on the need to ensure that every public school teacher has had rigorous screening and training and been ju dged qualified to teach. Certification is designed to protect the public from harm. But there is a difference between certification an d licensure. Lawyers, cosmetologists, and physicians represent a few of the many professions that


4 of 53require a practitioner to hold a license in order p ractice their profession. The goal of licensing tests is to set a minimum level of competency. Pro fessions that require licensure make it illegal for someone without a license to practice that occu pation (Pyburn, 1990). A person without a law or cosmetology license would be committing a cr ime if caught practicing law or working in a hair-dressing salon. No such legal protection is afforded the public when it comes to education. Teachers without certification are simp ly not allowed to use the title of "certified teacher" but there are no legal impediments for tea ching without certification. This difference between certification and licensure allows states t o issue emergency certificates but not emergency licenses. Issuance of these emergency ce rtificates produces considerable moral difficulty. A newspaper report on Chicago public s chools makes this case dramatically (Rossi & Grossman, 2002). The journalists document that a t Howland elementary school, one of Chicago's poorest: 55 percent of teachers were "not fully certified" t o teach all of their students.... That included four of six teachers in high-stakes classr ooms, where kids must repeat a grade if they don't do well on annual tests. All fo ur held substitute certificates, although two were in teacher preparation programs. Assigning uncertified substitutes to low-scoring kids who face high-stake s tests "should be illegal," said Kati Haycock, head of the Education Trust, a Washin gton, D.C., research and advocacy group. "That's educational malpractice." H owever, in Chicago, no policy governs who can teach such students. The "Philadelphia Inquirer" wondered what the city of Philadelphia was going to do about the same problem (Hundreds of teachers, 2002). Preside nt George W. Bush had just asked America to ensure that there was a highly qualified teacher in every classroom. But Philadelphia has 30,000 students in classrooms wher e teachers are uncertified and the city cannot figure out how to solve that problem. Presi dent Bush did not mention that while Governor of Texas, during the 1996-97 school year, he allowed 760,000 of the state's 3.8 million students to be taught by uncertified teache rs. Nor did he note that the students with uncertified teachers were found not to do as well o n the state achievement tests as did students in the classes of regularly certified teachers (Stu dents of certified, 1999). President George W. Bush has now passed on to the nation the problem th at Governor George W. Bush could not solve.Reasons for CertificationThose who defend the process of teacher certificati on claim it is a necessary component in the development and maintenance of the profession of te aching, as well as the means by which the state can ensure the quality of those who enter the profession. Wise (1994a) notes, however, that there are two methods of controlling entry int o the profession, professional control and popular control. Professional control allows the teaching profession to monitor who becomes a teacher. By specifying standards for certification and through various political mechanisms, the profession controls the quality of teachers who enter the prof ession. When professional control is present we often see teacher input in the design of teacher certification tests. On the other hand, popular control allows public demand to control who is placed in classrooms, with much less concern for their qualifications. Emergency certif icates to teach during times of "shortages" are an example of popular control. Wise (1994b) advoca tes professional control as the primary means to allow the promotion of teaching within the economic sector. He believes that without


5 of 53certification teaching becomes a trade rather than a profession. Among other characteristics, a profession is also d efined as possessing a distinct body of knowledge and having control of the education and l icensing of its members (Pratte & Rury, 1991; Burbules & Densmore, 1991). Labaree (1992) d escribes professionalization as the ability to demonstrate formal knowledge and to have autonom y in the work place. He explains that any occupational group: ...must establish that it has mastery of a formal b ody of knowledge that is not accessible to the layperson and that gives it speci al competence in carrying out a particular form of work. In return, the group asks for a monopoly over its area of work on the grounds that only those certifiably cap able should be authorized to do such work and to define appropriate practice in the area (p. 125). Both Wise (1994b) and Roth (1994) fear that demandbased policies allowing for uncertified teachers can be devastating to the profession. The y argue that such policies are likely to reduce the quality of teaching, lower the livable wage of teachers, and change the resources that are spent on and in schools. In effect, downgrading th e importance of certification and training prevents teaching from meeting one of the criteria by which an occupational group calls itself a profession. [A] shift in locus of preparation [from university programs to alternative certification programs] moves teaching in the direc tion of trade. On-the-job training is not characteristic of a profession. Di smissing the requirement of professional preparation and a credential prior to practice is also uncharacteristic of a profession (Roth, 1994, p. 267). This battle over control of training is not new. F or over 150 years who certifies teachers and how that certification is to be done has been a topic of intense debate. At all times, as might be expected, professional educators have fought to con trol the process, using medicine and law as their models (Angus, 2001).On-the-job Training and Teacher CertificationTo counter the argument that teachers can learn all that is required to be effective on the job, Darling-Hammond (2000), Howey and Zimpher (1994), a nd others argue that there is an inadequate amount of supervision and training provi ded to novice teachers by schools. Principals and veteran teachers who could serve as mentors generally do not have the required skills, training, or time to provide novices with q uality supervision for on-the-job training. With few exceptions, school districts do not now ha ve access to the additional resources needed for the training of teachers, and it is unlikely th at such resources can be obtained. Hawley (1992), articulating the views of many others, clai ms also that there is a body of subject-matter content and subject-matter method, as well as skill s and pedagogical knowledge, that needs to be learned prior to teaching. He and other teacher educators argue that it is unlikely that someone without training in subject matter methods could get in front of a class of students and be a successful teacher. This group of scholars re jects the idea that effective teaching can be learned on the job.Ordinarily, certification should assure the public that a minimal level of competency has been


6 of 53achieved, thereby insuring that unqualified people are not practicing the profession. Darling-Hammond (2000) believes the extant data su pports that claim. For example, in an analysis of state level data she found the percent of new teachers in a state who were uncertified correlated negatively with performance on six diffe rent state assessments conducted by the National Assessment of Educational Progress. These correlations ran from –.40 to –.63. She found equally large positive correlations for the p ercent of teachers in a state having regular certification and a major in the field in which the y were teaching, again using six NAEP data sets. These correlations ran between +.61 and +.80 Even for state level data the correlations Darling-Hammond found seem to be quite large, allow ing her to assert with confidence that students achieve better when they have certified te achers as instructors. Certification and Competency Testing for Novice Tea chers There are disputes about what should be covered on teacher certification exams because there is conflict about what is necessary for teachers to kn ow in order for them to be effective. Nevertheless, and despite the problem of defining t hese areas unambiguously, teacher certification exams currently focus on measuring ba sic skills, content and pedagogical knowledge (Kearns, 1984).In 42 states, "Candidates [for certification] are r equired to pass one or more tests of basic skills, general knowledge, subject matter knowledge, or tea ching knowledge" (Mitchell, Robinson, Plake, & Knowles, 2001, p. 44). But critics mainta in that the basic skills certification tests "measure verbal and mathematical achievement at abo ut the 10th grade level. And many states set their passing scores so low that virtually anyo ne can succeed" (Olson, 2000). Even Minnesota, usually one of the highest scoring state s in the nation on standardized tests, has a teacher basic skills test that currently passes ove r 99% of the applicants (Scores needed, 2002). The Arizona basic skills assessment was also of thi s kind. It had only a 1% failure rate (Crehan, Hess, Lawrence, & Sabers, 2000). This bas ic skills test was abandoned, in part, because of its low failure rate and also because of adverse impact on some of Arizona's minority group candidates. The low failure rates n ationwide suggest that teacher competence testing in basic skills areas is much more about sy mbolic politics (the need to appear as if standards have been put in place), and a lot less a bout genuine and systematic attempts to upgrade the quality of the profession. In the area of pedagogical skills and methods, the test developers often determine what good teaching looks like based on some definition of tea cher quality. But teacher quality often is defined as having the necessary knowledge, skills, abilities and behaviors of a good teacher, a circular definition providing little guidance. Mo reover, ideas about quality change from one decade to the next, as well as from one test develo per to another, and the criteria for measuring teacher quality (the knowledge, skills, and abiliti es a teacher possesses) is not readily agreed on from person to person (Mitchell, Robinson, Plake, & Knowles, 2001). The National Research Council (Mitchell, Robinson, Plake, & Knowles, 2001) reviewed past and present definitions of teacher quality and comp etency, finding that past definitions of teacher quality emphasized teachers' virtue. In it s more modern form teachers are still expected to be role models for students, representing high s tandards of personal behavior and expected to transmit worthy cultural and education values. Wit h no diminishment over time, it is still assumed that effective teachers possess certain per sonality traits, including enthusiasm, curiosity, and compassion, as well as democratic va lues. And, as always, effective teachers have been thought talented in teaching the prescrib ed curricula, an increasing concern after


7 of 53states have invested heavily in the development of standards and accountability systems. Today, most definitions of teacher competence from which assessments follow, are the product of three organizations, namely, the National Board for Professional Teacher Standards (NBPTS), the Interstate New Teacher Assessment Supp ort Consortium (INTASC) and the National Council for Accreditation of Teacher Educa tion (NCATE). The National Research Council (Mitchell, Robinson, Plake, & Knowles, 2001 ) notes that all three sets of standards of quality promoted by these organizations examine tea ching in the light of student learning, a relatively new concern. While a focus on learning seems obvious to parents and politicians, this is a much more problematic an issue then it ap pears to be. The three organizations also acknowledge that teachers' actions or performances depend on many different kinds of knowledge as well as the dispositions to use that k nowledge. And they recognize that teachers must also work with others to support the learning and success of all students. The standards of competence described by the three organizations rel ate to a teacher's commitment to students and their students' learning, requiring that teache rs: should act on the belief that all students can lear n; should have deep subject matter knowledge about the substance and structure of their disciplines; need to manage and monitor student learning, identi fy learning goals and choose from teaching styles to meet these goals; need to be reflective about their teaching, evaluat e their decisions and experiences to make adjustments in their teaching; must be part of a larger community consisting of sc hool staff, parents, and the broader non-parent community. As the National Research Council notes, the standar ds currently in use "...illustrate the wide range of knowledge, skills, abilities and dispositi ons that contemporary educators believe competent teachers must possess and demonstrate in the classroom" (Mitchell, Robinson, Plake, & Knowles, 2001, p. 31). Given the wide range and lengthy descriptions of the knowledge, skills, abilities and dispositions that the various assessors of teachers' competence have considered necessary to begin to teach, it seems an omalous that arguments to drop teacher education or to provide emergency certificates woul d have any credibility at all. Test developers find effective classroom teaching to be extremely complex, requiring the coordination of many different kinds of skills and dispositions, many of which cannot be observed directly. It is hard to imagine that an o ccupation with these characteristics can be effectively learned on-the-job.This very same occupational complexity also handica ps the developers of teacher certification testing, leading Sabers to note that, "test develop ers and researchers haven't done a good job of telling the public that they can't measure everythi ng with a test" (in Crehan, et al., 2000). The public believes that a certification exam will elim inate poor teachers from schools and in essence guarantee that teachers who pass these exam s are of high quality. But we do not yet have such tests. At present, it is fair to say tha t many aspects of teaching cannot be assessed by using a multiple choice or essay exam, and if perfo rmance tests of teaching were used such testing would be prohibitively expensive (Crehan, e t al, 2000).


8 of 53Validity problems with certification tests. All teacher certification exams have problems wi th construct, content, consequential and criterion-referenced validity (Laczko-Kerr, 2002). In brief, there is no evidence that the construct measured in teacher certification exams i s understood. In addition, course content varies more widely across teacher training institut es then, say, law schools or medical schools. This invites criticisms about the content validity for teacher certification exams. Additionally, certification tests limit the pool of potential tea chers based on race and ethnic background (Murnane, 1991). As the need for teachers of color increases because of the increase in children of color in our public schools, the numbers of mino rity teachers seems to be decreasing (Gitomer et al., 1999). One reason for this is the increased requirements for insuring teacher quality, including certification testing for teache rs. But these exams have an adverse impact on the teacher supply and this raises concerns about t he consequential validity of the exams. Finally, teacher certification exams do not appear to have criterion-related or predictive validity (Smith & Hambleton, 1990; see also Glass, 2002). C ertification tests simply do not predict success in teaching. Rather, their intent is to sc reen out certain applicants from the teaching pool (Sabers in Crehan, et al., 2000). Jaeger, quo ted in the National Academy of Science/National Research Council report has an add itional concern, namely, that the sorts of experimental or statistical controls necessary to p roduce trustworthy criterion-related evidence [are] virtually impossible to obtain" (Mitchell, Ro binson, Plake, & Knowles, 2001, p. 72). These problems with validity, particularly predicti ve validity, seem to bolster the arguments of those who support emergency certification. If the tests cannot predict teaching competency, they argue, why must they be required for certifica tion? The answer offered to that question by supporters of teacher certification tests is that p assing the tests ensures familiarity with a broad teacher education curriculum, without which beginni ng teachers would not do well. This debate can be restated as a version of the simple q uestion we noted above: Does teacher education and the certification that accompanies su ch programs make a difference in the achievement of students?Research on Certified Teachers and Student Achievem ent Three major areas of research are salient for under standing the importance of certification. First is the research on the effects of certification reg arding teachers' content knowledge, particularly mathematics and science knowledge, as it affects st udent achievement. A second area of research deals with the effects of certification re garding a teachers' pedagogical knowledge, and its effects on student achievement. It is clear th at the federal government is having troubles deciding on the relative importance of these two ar eas, paying lip service to the latter but more often endorsing the former. As "Washington Post" r eporter Jay Mathews notes (2002), first the Bush administration pushes through an education bil l that demands a "highly qualified" teacher in every classroom. Then the administration releas es a report arguing that the nation's education schools spend too much time on classroom methodology. Mathews points out that mixed messages are being sent to the public. But i n fact, they aren't very mixed. Education Secretary Roderick Paige and other Educat ion Department officials claim that schools of education need to spend less time on ped agogical issues and spend more time worrying about whether teachers understand what the y teach. The current mantra of federal educational administrators seems to be "You can't t each what you don't know" (Mathews, 2002). Not mentioned explicitly, but implied, is t hat schools of education should have little role in the training of teachers. Secretary Paige' s comments are all the more puzzling from


9 of 53some one who advocates evidencebased research. H is own Department of Education recently requested a review of "rigorous empirical research" on teacher preparation (Wilson, Floden & Ferrini-Mundy, 2002). The authors of this governme nt commissioned report concluded that subject matter knowledge is not sufficient for effe ctive teaching to take place. [The studies reviewed] suggest that the subject mat ter preparation....prospective teachers currently receive is inadequate for teachi ng toward high subject matter standards, by anyone's definition. [Without traini ng in pedagogy] it appears that prospective teachers may have mastered basic skills but lack the deeper conceptual understanding necessary when responding to student questions and extending lessons beyond the basics (p.192). The third area of research focuses on two sub-areas that both deal with traditional certification and alternatives to it. One of these areas of rese arch is on the effects of regularly certified teachers teaching in or out of their area of expert ise. In this literature in-field vs. out-of-field teaching performance is compared, such as when an E nglish teacher is assigned to teach algebra. Out-of-field teaching can be viewed as te aching without the appropriate certification to do so. The second sub-area is concerned with the e ffects of alternatively certified teachers in comparison to traditionally certified teachers. Pr esent government policy has decided that alternate means to certification are appropriate, w ith officials claiming that: [T]here is no evidence that lengthy preparation pro grams achieve [their] goals any better than streamlined programs that quickly get t alented teachers into the classroom....Requiring excessive numbers of pedagog y or education theory courses acts as an unnecessary barrier for those wishing to pursue a teaching career (Mathews, 2002). Our evaluation of this literature, reviewed in more detail below, is that there is sufficient evidence to conclude that 1) subject-matter knowled ge is an important, but not sufficient, factor in a certified teacher's success with mathematics a nd science students in the upper grades; 2) that teachers who have training in pedagogy outperf orm teachers without such training; and 3) that traditionally certified teachers teaching in t heir area of certification outperform both certified teachers teaching out-of-field and altern atively certified teachers. The data on these issues, however, is certainly not unequivocal, and dissenters to all these conclusions exist (see Ballou & Podgursky, 1999; Peck, 1989; Miller, McKen na, & McKenna, 1998). We look at these literatures next.Teacher Subject-matter Knowledge. Studies related to teacher subject matter often either evaluate 1) whether a major or minor in a subject area, e.g., mathematics, effects student ac hievement (Hawk, Coble, & Swanson, 1985; Goldhaber & Brewer, 1996; Monk & King, 1994); 2) wh ether a passing score on a certification exam provides evidence that certain subject matter has been mastered or that certification affects student achievement (Ashton & Crocker, 1987 ; Byrne, 1983; Strauss & Sawyer, 1986; Glass, 2002); or 3) whether advanced degrees, e.g., master's degree, or professional development increase student achievement (Goldhaber & Brewer, 1996, 2000; Fetler, 1999; Ehrenberg & Brewer, 1994). Each of these areas of evidence will be reviewed separately. Researchers have usually, though not always shown t hat having a major or minor in mathematics or science is beneficial to student ach ievement in those content areas. Hawk,


10 of 53Coble, and Swanson (1985) provide research on that issue obliquely, by comparing in-field and out-of-field teaching; concluding that student achi evement, for general mathematics as well as algebra, is greater for students who are taught by teachers certified in mathematics (in-field teachers, possessing a major or minor) than is the achievement of students taught by teachers certified in some other content area (out-of-field teachers, neither a major or minor in the area). The researchers hypothesize that the greater succes s of these in-field teachers' appears to be their greater ability to successfully impart conten t specific knowledge to students, as compared to their out-of-field counterparts. It is importan t to note that these studies compare teachers who hold a standard teaching certificate in their s ubject area (indicating specialized content knowledge training) with teachers who also hold a s tandard teaching certificate in another subject area (indicating a lack of specialized cont ent knowledge training). The study supports the case for certification in a content area, and s uggests deficiencies can be expected among those who are teaching in areas for which they are not prepared. From their research Goldhaber and Brewer (1996) con clude that "in mathematics and science, teacher subject-specific training has a significant impact on student test scores in those subjects" (p. 206). These same researchers go on t o say that their results suggest that it is subject-specific training, not teacher ability that leads to such findings. These authors believe that achievement in technical subjects can be impro ved by a cessation of out-of-field teaching. The generalizability of these results to the humani ties and for teachers in the primary grades is unknown.Monk and King (1994) also evaluated subject-matter preparation and student performance. In an earlier analysis Monk (1994) had found that ther e was a "positive relationship between the number of subject-related courses in a teacher's ba ckground and subsequent performance gains of these teachers students within the indicated sub ject area" (as cited in Monk & King, 1994, p. 36). Continued investigation of this phenomena rev ealed interactions, among them, that, "low-pretest students' performance gains in mathema tics were more sensitive to the mean level of their previous teachers preparation than were th e high-pretest students" (p. 56). This suggests that lower achieving students will profit more from teachers who are well prepared in their subject matter, than might better achieving s tudents. Thus policies that promote uncertified teachers as the instructors of the poor est and the lowest achieving students, which is the way those policies are usually realized, may be particularly harmful. Byrne (1983) provides a review of thirty studies th at relate teachers' subject matter knowledge, measured by subject knowledge exams or coursework t aken, to student achievement. These results were contradictory. A majority of the stud ies showed a positive relationship (17), while a large number (14) showed that no relationship exi sted. Byrne does not provide more than a tally analysis of the studies included, which is in sufficient given the capabilities of meta-analytic research. A re-analysis using meta-a nalysis would be helpful. The National Teacher Exam (NTE) was once used as a measure of subject matter knowledge and was extensively studied. It measured both subj ect matter content knowledge and pedagogical knowledge. Quirk, Witte, and Weinberg (1973) found only a single study, Lins (1946), in which NTE scores were correlated positiv ely with students' average gain in performance from pretest to posttest. But this fin ding must be reconsidered in light of the work of Strauss and Sawyer (1986). They analyzed the de terminants of average student performance on standardized exams and found that a "1% increase in teacher quality, ceteris paribus, as measured by standardized test scores [NTE], is acco mpanied by a 3-5 % decline in the level of failure or rate of failure of students on high scho ol competency examinations" (p. 41). Simply put, increased scores by teachers on the NTE exams, reflecting increased subject matter


11 of 53knowledge and increased pedagogical knowledge, decr eased student rates of failure. Research on the PRAXIS tests, successor to the NTE, has been conducted by Gitomer, Latham, and Ziomek (1999), and will be reported below. Teachers' advanced degrees are another indicator of subject matter competency. Goldhaber and Brewer (1996), as part of the study reported above, also found that teachers' degree level is significantly related to student achievement in the area in which the degree was obtained. However, when a general production function model i s used, teachers with master's degrees appeared to be no more effective than teachers with out advanced degrees. Results varied depending on the statistical models that were used to analyze teacher effects. Goldhaber and Brewer (2000) report that mathematics students who have teachers with bachelors or masters degrees in mathematics have higher test scores than students of teachers without these degrees. They report, however, that there is no comparable i mpact of degree in science. Fetler (1999) confirms the findings in mathematics, noting that Schools with more experienced and more highly educated mathematics teachers tend to have h igher achieving students" (p. 13). But of course, higher achieving students have access to be tter schools, and thus these kinds of studies require caution when interpreting them.Ehrenberg and Brewer (1994), however, report unambi guously that teachers' degree level does matter. "The greater the percentage of teachers wi th at least a masters degree...the higher black students' scores are [on measures of mathematics, r eading, and vocabulary that are associated with the High School and Beyond study]" (p. 10). O n the other hand, with a Texas sample, Rivkin, Hanushek, and Kain (2000), note just the op posite, namely, that there "is no evidence that a masters degree raises teacher effectiveness" (p. 3). However, these researchers also report "teacher quality is a very important determi nant of the quality of education" (p. 30). Thus the conclusions in this report appear contradi ctory. Kain (1998) also studied this issue. He reports th at in his earlier research, teachers accounted for at least 7% of total variation in student achie vement, indicating that a set of teacher characteristics, including certification and traini ng, affect student achievement. To analyze the affect of teacher degree on student achievement, Goldhaber and Brewer (2000) used data from the National Educational Longitudina l Study of 1998 (NELS: 88), which provided information about students and their teach ers in 10th and 12th grades. Students were surveyed as well as tested on one or more mathemati cs, science, English/writing and history exams. The researchers found that "having a degree in education has no impact on student science test scores and, in mathematics, having a B A in education actually has a statistically significant (at the 10% level) negative impact on m athematics scores of students" (pp. 138-139). Goldhaber and Brewer's research focused on secondary grades. The complexity of the content taught in secondary mathematics classes is undoubtedly greater than that taught in elementary school, so advanced training in mathemat ics may be required to effectively transmit that content. That same depth of subject matter kn owledge may not be required for elementary teaching.There is some support for this hypothesis, although research on the effects of advanced degrees and subject matter majors for primary grade student s is scanty. One such study, however, was published by the National Center for Educational St atistics (NCES) (as cited in Hawkins, Stancavage, & Dorsey, 1998). These results concern ed fourth-grade mathematics students who took the National Assessment of Educational Progres s (NAEP). The researchers note that "fourth-grade students who were taught by teachers with an undergraduate or graduate minor in mathematics or mathematics education did not perfor m better on the 1996 mathematics


12 of 53assessment than students whose teachers had an unde rgraduate or graduate major in education" (p. 12). But the NCES report also states that, unl ike fourthgrade, eighth-grade students who were taught by a teacher with a major in mathematic s outperformed students taught by teachers with majors in education.One conclusion to be drawn from these data is that a teacher's depth of knowledge in a subject matter influences students' achievement more in the upper grades than the primary grades. But the broader conclusion might be that as the content in a subject matter area becomes more complex, teachers need a much deeper knowledge of t hat subject matter area to help their students learn at high levels. It is not grade lev el per se, we think, but the complexity of the ideas to be taught that requires the specialized su bject matter knowledge of a teacher. Thus the claim made by TFA, that an individual with a subjec t matter major from an elite college who elects to teach elementary school without certifica tion is bringing great strength to the schools, may be questioned. It seems probable that after ba sic college level content is mastered, after some threshold of competency in a subject matter do main is crossed, as it is for most college majors, then pedagogical training for teaching in t he elementary grades is more important to success than is content knowledge. Support for thi s interpretation comes from Rowan, Correnti and Miller (2002), in their study of teacher subjec t matter competency in the early grades. We look at this issue next.Professional Knowledge/Pedagogical Content Knowledg e The findings from research that examines a teachers level of education related coursework and their effectiveness with students is extensive, but often contradictory. We believe, however, that some interpretations of this literature are po ssible, though we note that better research in this area is possible and badly needed.Ferguson and Womack (1993) found that the amount of education coursework teachers completed explained about 16% of the variance in te aching performance, as measured by supervisor evaluations; this was more variance acco unted for than with teachers' content knowledge, as measured by NTE specialty scores. Th is research suggests that education coursework is a strong predictor of teaching effect iveness, over and above grade point average in a teachers' major and their NTE specialty scores In their review of research on this same issue Ashton, Crocker, and Olejnik (1986) also foun d education coursework to have a significant relationship to teacher performance.More recent research by Wenglinsky (2002), on the l ink between teacher quality and student performance, supports the belief that teacher input s do influence student performance. He notes that the greatest influence on student's achievemen t comes from classroom practices and the professional development that supports them. Wengl insky's research indicates that "regardless of the level of preparation students bring into the classroom, decisions that teachers make about classroom practices can either greatly facilitate s tudent learning or serve as an obstacle to it" (p. 7). That is, teacher pedagogical decisions and act ivities (which are separate from but not unrelated to teacher subject matter knowledge) inde pendently make a difference in student achievement. Rowan and colleagues (Rowan, Correnti & Miller, in press) reached similar conclusions. These researchers found relatively large effects on young students that could be attributed to teachers, independent of school, social class, previous achie vement, and so forth. For any given year, looking at a single score, at a single point in tim e, teachers accounted for 4% to 18% of the


13 of 53variance in student's reading and mathematics achie vement. This yielded effect sizes of .21 to .42. Across years, looking at student growth, the effects of teachers on students were magnified. Analysis of the teachers' effects on st udent growth in reading yielded effect sizes of from .77 to .88. The teachers' effects on growth i n mathematics achievement were equally impressive, yielding effect sizes of between .72 an d .85. The effect of teachers' characteristics on student achievement growth, across time, is roug hly three times greater than they are on student achievement status measured at only one poi nt in time. When searching for which teacher characteristics make a difference, these in vestigators found that the most consistent predictor of young children's achievement was teach er experience. Experience was found to be a much better predictor of student achievement than was subject matter competency. Here again we see the relative importance of pedagogy ov er content knowledge in influencing the achievement of young children.What is often not discussed in research reports con necting some teacher quality variable and student achievement is that the great bulk of a tea cher's pedagogical training and understanding of beneficial classroom practices is provided in th eir teacher training programs. Clearly experience matters; but that means that preparation to profit from experience must matter as well. And that suggests that the experience gained from intensive student teaching, over a sufficient time period, might also matter. Such ex periences are provided as a matter of course in most traditional teacher certification programs, and are missing from most alternative and emergency certification programs. Without adequate teacher training, then, emergency certified teachers and other under-certified teachers could r etard student learning as they engage in teacher learning. We examine teacher experience in more detail next. Teacher Experience Teacher experience is another teacher quality var iable that influences student learning and is indirectly related to issues of certification. Haw kins, Stancavage, and Dorsey (1998) report that in the 1996 AEP analysis, students who were taught by teachers with less than 5 years of teaching experience performed below the level of th ose students whose teachers had 6-10 years or 25 or more years of experience (p. 22). Fetler (1999) also supports the finding that number of years teaching is positively related to student test scores. Lopez (1995), using a large data set from Texas, reports that teachers require about 7 y ears of experience in order to be able to maximize their students' test performance. Similar ly, Rivkin, Hanushek, and Kain (2000) report that there are small but significant relatio nships between student achievement gains and teacher experience: "The teacher experience investi gation concentrates on entering teachers and supports the notion that those in the first two yea rs of experience do worse than more experienced teachers. New teachers' average studen t gains are lower by roughly 20 percent of a standard deviation in both 4th and 5th grades" (p. 27). They also report that 4th and 5th grade student achievement on the Texas Assessment of Acad emic Skills (TAAS) is effected by overall teacher experience. The results indicate a small but significant relationship between achievement gains and teacher experience. The stud y of Rowan et al., (in press) cited above, supports this conclusion as well.The point of this discussion about experience is th at virtually all university teacher certification programs include both early field experiences and s tudent teaching in their curriculum to provide experience to novice teachers. While we ha ve little empirical evidence to determine what kinds and amounts of experience are the most b eneficial, it seems likely that teacher induction programs that offer little or no experien ce will be deficient. This is a criticism of the TFA program and any other programs supporting emerg ency or alternative certification that


14 of 53allow un-experienced and inexperienced teachers to become classroom instructors. Reviewing similar literature for the Department of Education, Wilson, Floden and Ferrini-Mundy (2002) conclude that the parts of the teacher education ex perience that make a difference are unknown, but that "the research suggests that there is value added by teacher preparation (p. 194)." They also state that clinical experiences and field-work such as that provided through student teaching, are seen as powerful forces—maybe the mos t powerful force—in programs of teacher preparation.Interestingly, if a state policy provides for emerg ency certification to teach for only a short period of time, they may do a disservice to student s, since it is through experience that teachers acquire their competency. The logic is this: It ma y be wrong to employ emergency certified teachers, but to dismiss them solely on the basis t hat they served two years, the maximum for an emergency certificate in some states, is to negate and reject how much they may have learned in that time. On a case-by-case basis, it may be bett er to decide if an emergency teacher has been reflective about his or her experience and thereby learned enough to be effective. It may compound the original error to dismiss them after a short period of time. The review of research on content knowledge, pedago gical knowledge, and experience, given above, focuses on where these abilities and charact eristics of teachers are to be learned, and in what mix, but there is no major dissent about their importance for student learning. Wenglinsky (2002) makes this case best using data f rom the National Assessment of Educational Progress (NAEP) to examine the role tea chers and their instructional practices play in influencing student achievement. He summarizes his findings as follows: "The study finds that the effects of classroom practices, when added to those of other teacher characteristics, are comparable in size to those of student background, suggesting that teachers can contribute as much to student learning as the students themselves ." Appropriate Assignment of Certified Teachers Some research on certification status supports th e fact that teachers who are certified and teaching in the area in which they are certified ou tperform teachers who are less than fully certified and teachers who have no certification (D arling-Hammond, Berry, & Thoreson, 2001; Darling-Hammond, 1992; Fenstermacher, 1992; Evertso n, 1984). Unfortunately, however, not all certified teachers are assigned to teach in the areas for which they have been trained (Goldhaber & Brewer, 1996; 2000; Ehrenberg & Brewer 1994). In fact, large numbers of the certified teaching staff are assigned duties for wh ich their certification is irrelevant. This kind of out-of-field teaching is exemplified in the elem entary grades when a fifth grade teacher is assigned to teach a second grade class, or in high school, when an English teacher is assigned to teach an American History or a biology class. The most cited reason for out-of-field teaching is a lack of fit between the teachers on the staff and the teaching assignments that an administrator must make (Ingersoll, 1999a).In some curricula areas such as bilingual and speci al education, science, and mathematics, there is a teacher shortage. This appears to be caused b y increased student enrollments, retirements due to the "graying" of the teaching profession, i ncreased turnover by teachers due to increased difficulties in carrying out their jobs, and the in creased remuneration for mathematicians and scientists for work in other fields. If faced with difficulty filling positions, school boards and administrators think of three solutions: they hire less qualified teachers, they assign teachers trained in another subject area to teach in the und erstaffed areas, or they make extensive use of substitute teachers.


15 of 53There are many problems with the teacher shortage e xplanation for out-of-field teaching. First of all, this explanation does not provide an answer for why large numbers of out-of-field teachers exist in fields like English and social st udies, two areas of teaching that have long had a surplus of teachers. Another problem with this e xplanation is that it has only been within the past few years that schools have had trouble fillin g teaching openings, while the problem of out-of-field teaching has been with us for a signif icantly longer period of time. Finally, the teacher shortage is based on the assumption that th ere are too few teacher candidates. But in fact, the biggest difficulty is that teachers are c hoosing to leave their profession for other jobs (Ingersoll, 1999a). Ingersoll (1999a) comments, "O ut-of-field teaching is common, I believe, because it is not only legal but also more convenie nt, less expensive and less time consuming than the alternatives" (p. 7). Ingersoll (1996) evaluated data from the School and Staffing Survey to determine the proportion of teachers who teach out-of-field. He provides data indicating that one-fifth of public school students enrolled in 7th-12th grade E nglish courses are taught by teachers who did not have at least a minor in English or some ot her closely related field. Of the students enrolled in 7th-12th grade mathematics courses, mor e than a quarter are taught by teachers without a minor in mathematics or mathematics educa tion. The results are less drastic in other areas of course work. In Arizona, 35% of teachers in English, 39% of teachers in Math, 35% of teachers in Social Studies and 27% of teachers in S cience were assigned to teach secondary courses without a major in that subject area ("Qual ity Counts," 2000). Research also supports the belief that out-of-field teaching is related to levels of school poverty (Ingersoll, 1996; 1999b; Haycock, 2001). Ingersoll (1996) reports, in no fields did high-poverty schools have less out-of-field teachin g than did low poverty schools, while in several fields, students in high poverty schools re ceived distinctively more out-of-field teaching than in low poverty schools" (p. 5). This trend is similar for students who are placed within different educational tracks in their courses. Hig h track students are exposed to less out-of-field teaching than low track students (Goldhaber & Brewe r, 2000; Rivkin, et al, 2000, Ingersoll, 1996), while "minority and poor students are dispro portionately placed in lower track and lower achievement courses, [which] critics claim are taug ht by the least qualified" (Ingersoll, 1996, p. 1). Darling-Hammond (1997b) reports that in the m ost heavily minority schools and inner cities less than 50% of the teachers in mathematics and science are licensed and have a degree in the subject they teach. Darling-Hammond remonstr ates that throughout the country we have the least qualified teachers teaching the most disa dvantaged students, while the most qualified teachers are teaching the most advantaged students.At the secondary level the relationship between infield teaching and student achievement is stated forcefully by Hawk, Coble, and Swanson (1985 ). They conclude that: In field certified math teachers know more math and show evidence of using more effective teaching practices than their out-of-fiel d counterparts. Further, and most important, students of in field certified math teac hers achieve at a higher level than do students taught by out-of-field teachers (p. 15) In short, a certified teacher teaching in the field for which they were prepared performs better than when assigned to areas for which they were not prepared. Preparation matters. Alternative Routes to Teaching


16 of 53Much of the research that draws attention to altern ative certification programs does not adequately address the issue that many such program s are similar in both the level and rigor of training provided by traditional certification prog rams (Buck, Polloway, & Robb, 1995; Miller, McKenna, & McKenna, 1998; McKibbin, 1988; Bliss, 19 92; Stoddard, 1992; Darling-Hammond, Berry, & Thoreson, 2001). On the other hand, many of the alternative teacher training programs are poorly designed and a dministered, providing little in the way of appropriate training (Wilson, Floden, Ferrini-Mundy 2002). Although increasing dramatically in number, there are currently no standards for ass essing alternative certification programs. The large variability in alternative certification prog rams makes research on this phenomenon difficult. (Of course, to be equally fair, we must note the variability in traditional programs of teacher education, whose design and administration have also been noted by many to be equally slipshod. Even accredited programs have, in our op inion, some embarrassing design characteristics.)Advocates of alternative certification, however, cl aim that they provide teachers for urban and rural schools and in specific shortage areas, i.e., mathematics and science. Zumwalt (1991) summarizes research on several alternative certific ation programs and reports that they do attract teachers who are more willing to work in ru ral or urban poor districts than traditionally trained teachers. McKibbin and Ray (1994) also rep ort that alternative certification programs attract people with subject matter majors like math ematics and science who are interested in teaching, but not interested in traditional teacher certification. Alternative certification is also seen as a cost-ef fective way to train people who did not or will not enroll in conventional undergraduate or graduat e education programs. Such programs are cheaper (Zumwalt, 1991), as might be expected from programs that are shorter in duration and provide less instruction, supervision and assessmen t of their students. Proponents of alternative certification claim that these programs attract better quality candidates who are more academically able than those who atten d traditional certification programs (Kanstroom & Finn, 1999). Participants of these pr ograms are generally people who have majored in traditional academic subjects rather tha n education. It is a major assumption of alternative programs that subject matter content kn owledge is more important to teaching than is education related coursework (Jelmberg, 1996). But some studies show that the teachers in alternative routes to certification have high dropout rates from both the programs of instruction and from actual teaching. (Wilson, Floden & Ferrini -Mundy, 2002). Other studies show that alternative certification recruits in mathematics a nd science have lower grade point averages than recruits in traditional teacher education prog rams (Stoddart, 1992). Moreover, to date, these alternatively certified teachers have not dem onstrated strong skills in their content area. Furthermore, teachers from alternative routes to ed ucation, including TFA teachers, when compared to those trained in more traditional teach er education programs, report many more problems with their preparation programs. For exam ple, on 39 of 40 different questions the TFA teachers rated their preparation more poorly th an did those who were trained in more traditional programs. The self-confidence and sense of efficacy of those prepared in traditional programs was higher than for those who came to teac hing through alternative programs (Darling-Hammond, Chung, & Frelow, 2002).When these facts about alternative routes to teachi ng are added to research that debunks the belief that subject matter knowledge is more import ant than education related coursework (Darling-Hammond, 2000; Gitomer, Latham, and Ziomek 1999; Monk and King, 1994; Wilson, Floden & Ferrini-Mundy, 2002), we see evide nce of an educational policy that must be seriously questioned.


17 of 53Proponents of alternative certification also make t he claim that traditional certification programs attract mostly twenty-something, white wom en, a problem because the students of the public schools in urban areas are not often white, and some commonality and similarity in life experience is probably a better background for teac hing positions in these more difficult settings. Proponents of alternative certification correctly note that they often attract a more diverse group of candidates, specifically men, olde r adults, minorities and retried military personnel (Bradshaw & Hawk, 1996; Eifler & Potthoff 1998; Hawley, 1992; Houston, Marshall, & McDavid, 1993; Keltner, 1994; Kennedy, 1991; Kwiatkowski, 1999); Natriello & Zumwalt, 1993; MacDonald, Manning, & Gable, 1994; a nd Stoddart, 1993). Wendy Kopp (1994), Teach for America founder, notes that short alternative certification programs allow young adults who are unsure of their career options an opportunity to serve students. Alternative certification may actually be an improv ement over simple emergency certification, which allows almost anyone with a bachelors degree to teach without any preparation to speak of. But some researchers (Bradshaw & Hawk, 1996; B erry, 2001) are critical of the level of professional knowledge demonstrated by alternativel y certified teachers. Alternatively certified teachers tend to have a limited view of curriculum and a lack of understanding of student ability as well as motivation; they experience difficulty t ranslating content knowledge into meaningful information for students to understand; they are le ss effective planners of instruction; and they tend not to learn about teaching through their expe riences. Research is also critical of the supervision and mentoring support that is given to preservice teachers in most alternative certification programs (Smith, 1991; Darling-Hammon d, 1992; McKibbin & Ray, 1994; Bradshaw & Hawk, 1996; Jelmberg, 1996; Miles-Nixon & Holloway, 1997; Berry, 2001). It appears likely, from the extant research and critic ism, that poorly run and short alternative certification programs, particularly those that do not provide much classroom experience and supervision, may actually not be any better than si mply hiring emergency certified teachers with no teacher education experiences. Teach for AmericaThe most familiar of the alternative certification programs is Teach for America (TFA). This ambitious program recruits graduates from top unive rsities, provides them minimal training, and places them in public school classrooms across the nation to teach. The public schools, however, are all in either rural or poor urban dist ricts (Darling-Hammond, 1994). Research conducted on TFA has been less than encouraging. Four separate evaluations found that TFA's training program did not prepare candidates to succeed with students, despite the no ticeable intelligence and enthusiasm of many of the recruits. Most criticism of a corps member's teaching behavior (classroom management was the greatest are a of concern, followed by insufficient knowledge of the fundamentals of teach ing and learning) was qualified by the cooperating teachers' perceptions of limitat ions of the program in providing the corps member with adequate practice or theory t o be successful (Darling-Hammond, 1997a, p. 310). From an interview study by Stevens and Dial (1993), TFA teachers apparently decide to teach because they like working with children; they didn' t have other options; and they felt that TFA was their best alternative given their "circumstanc es and indecisiveness at the time" (p. 70).


18 of 53Jonathan Schorr (1993), a former TFA teacher, descr ibes the inadequate training and preparation that he and other TFA teachers received prior to being placed into schools. He notes, "just eight weeks of training ... is not eno ugh for teachers" (p. 316). Schorr admits, "I was not a successful teacher, and the loss to the s tudents was real and large" (p. 318). Schorr offers the first-hand experience that makes Darling -Hammond (1994; 1997a; 2001) quite critical of TFA, specifically due to the program's limited training of candidates, lack of evaluation, and the fact that such a program perpet uates the placement of poorly trained teachers with the most needy students in the nation Raymond, Fletcher, and Luque (2001) conducting rese arch for the Center for Research in Education Outcomes (CREDO), released a report evalu ating the Teach for American program in Texas. The report compares scores on the Texas Assessment of Academic Skills (TAAS) of students taught by TFA teachers and non-TFA teacher s, and lauded the performance of TFA teachers. However, the National Commission on Teac hing and America's Future released a response outlining serious concerns with the resear ch (see Darling-Hammond, 2001 and 2002). The most important of the criticisms is that the pe rformance of the TFA teachers was never compared to the performance of regularly certified teachers. The comparison used to assess the TFA teachers was other uncertified teachers, some o f whom didn't even have four-year college degrees. It should also be noted that when we tried to acces s the data for this report, we were informed from both the primary researcher and the Texas scho ol district responsible for the data that it was not available for independent review. We were told that the data was not the property of the researchers who reported the study, nor did it belong to the district, and that neither had a complete data set to provide for independent analys is. In our opinion, therefore, it is appropriate to regard this report as irrelevant, gi ven that the comparison used to assess TFA teachers was faulty, the data are not available for verification or replication by other scientists, and the report has not been published in a peer-rev iewed journal. Traditional Teacher CertificationIn this section we discuss traditional programs of teacher certification, with the understanding that they vary enormously, as do the alternative ce rtification programs. The Holmes Group (1986), active in teacher education reforms, report s that competent teaching consists of three elements: subject matter knowledge, systematic know ledge of teaching, and reflection on experience. Members of the Holmes group defended t eacher education in the universities by arguing that they do, in fact, prepare people to su ccessfully integrate these three elements into their professional lives. In their defense we note that teachers entering the field from university teacher education programs are generally more acade mically able than the average college student, while unlicensed entrants into teaching ha ve significantly lower levels of academic achievement than most college students and were low er also then those who were prepared by the university to teach (Gitomer, Latham, & Ziomek 1999). In fact, "reviews of research over the past thirty years, summarizing hundreds of stud ies, have concluded that even with the shortcomings of current teacher education and licen sing, fully prepared and certified teaches are better rated and more successful with students than teachers without this preparation" (Darling-Hammond, 1997a, p. 308; Evertson 1984). A pparently disregarding this research, Former Secretary of Education Chester Finn proposes that the common sense route to boosting teacher quality is to simplify entry and hiring. F undamentally, he argues, we should let anyone who wants to teach do so, and simply deregulate the teacher certification process (Kanstroom &


19 of 53Finn, 1999; Finn & Kanstroom, 2000). Finn is also the president of the foundation that helped support the unverifiable Raymond, Fletcher, and Luq ue (2001) study that is so supportive of uncertified teachers. Nationally syndicated conser vative columnist Thomas Sowell supports Finn's position. Sowell (2002) says bluntly that c ollege of education courses are "drivel" and falsely reports that the academic record of those w ho enter teaching through the university route is deficient in comparison to almost all other coll ege majors. The research martialed in support of prepared and c ertified teachers includes research demonstrating that teachers hired without preparati on or only partial training experience difficulty meeting the needs of the students in the ir classrooms. Such individuals have more difficulties than fully prepared teachers do in acc omplishing their day-to-day job requirements (Darling-Hammond, 1997a). Prospective teachers app arently perform better if they have completed a teacher preparation program focused on content knowledge, pedagogical course work (including learning theories, developmental th eories, theories of motivation and issues of student assessment) and practice teaching. Althoug h variations in the philosophy, implementation and quality of teacher education pro grams are enormous, research nevertheless suggests that many versions of this form of prepara tion are successful in providing adequately trained teachers for the complexity of classroom in struction (Ashton & Crocker, 1987; Darling-Hammond, 1992; Wilson, Floden, & Ferrini-Mu ndy, 2002). McDiarmid and Wilson's (1991) research is relevant to this point. They demonstrated that subject matter knowledge is not sufficient to prepa re teachers for teaching the concepts in these fields to students. They did this by evaluating ma thematics majors in alternative certification programs that stressed subject matter knowledge and found that those teachers had strongly held misconceptions about the subject matter and th e appropriate ways to teach the content. Their results indicated that these teachers were un able to integrate their subject matter knowledge with teaching practices to allow for effe ctive instruction. In effect, because they were lacking in education coursework, they were una ble to provide the appropriate instruction to students.Another aspect of good quality certification progra ms is that they provide experiences for the preservice teachers in classrooms both under direct supervision and with continued mentoring. Darling-Hammond (1992) notes that skills need to be learned in context, where they can be practiced under supervision. The student teaching experience allows the preservice teacher to transform information from coursework in order to l earn its character in the context of the real world of teaching in classrooms. Jelmberg (1996), cited above, compared traditionally certified teachers with alternatively certified teachers' per formance based on their experience. His results showed that experienced teachers from tradi tional certification programs are rated higher on instructional skills and planning by their princ ipals, and perform better, than did experienced teachers who came from alternative certification pr ograms. Emergency certificationThe reviews of research, above, compared alternativ e certification programs, some of which provide enough preparation for graduates to receive full certification, while others provide minimal training resulting in graduates receiving e ither a provisional or emergency certificate. Research reviewed above also compared fully certifi ed teachers to one another, distinguishing between teachers who taught in-field or out-of-fiel d. There is little research comparing emergency certified teachers and regularly certifie d teachers.


20 of 53Emergency certificates are issued to prospective te achers who have met some, but not all of the requirements for state certification. Minimum requ irements are often a bachelor's degree and a passing score on a certification exam. Emergency c ertificates are issued for a limited time period, usually one to two years. Some states allo w for these to be renewed, while others states issue a one time only, nonrenewable certificate. I n 1998 data about certification waiver rates were available from 39 states (U.S. Department of E ducation, 1999). Sixteen of the 39 states had waiver rates greater than 2% of their teaching population, eight had rates over 5% while some had rates as high as 17%.Emergency certified teachers are more likely to be hired in already low performing schools, schools that serve low SES students, schools in rur al and inner city areas, and for positions that are hard to fill (Darling-Hammond, 1997a; Darling-H ammond, 2001; Ingersoll, 2001; Gitomer, Latham, & Ziomek, 1999). Since high poverty school s are more likely to have high rates of out-of-field teaching as well as more emergency cer tified teachers, it may that some states are failing to provide the "adequate education" that mo st state constitutions require (Ingersoll, 2001; Hirsh, Koppich & Knapp, 2001; Rivkin, et. al. 2000). And now that federal government has waded in on this issue, requiring a highly qual ified teacher in every classroom, the competency of teachers with emergency certification is sure to be questioned further. Emergency certification is justified on the basis o f three arguments. First, it is argued, that there is a teacher shortage requiring that states e mergency certify teachers to provide enough teachers for every classroom. But the shortage may not be as severe as many claim (Feistritzer, 1994; Ingersoll, 1997, 2001; Ballou, 1996; Hirsh, K oppich, & Knapp, 2001; Hardy, 1998). The National Center for Education Statistics indicates that the teaching force will grow to more than 3 million teachers by the year 2008 (US Department of Education, 1999). But it is a misconception that colleges of education will need to train millions of new teachers to meet the needs of school districts. Darling-Hammond (cited in Hardy, 1998) believes that this potential shortage is not universal, claiming that "there are districts that experience difficulty hiring qualified teachers, but overall, we have a surplus of teachers" (p.20). The teacher shortages are seen in subject areas like mathematics and science; in the service areas for special needs populations, such as special education and bilingua l education; and shortages exists in rural areas and in inner city school districts (Wayne, 20 00; Natriello & Zumwalt, 1993; Hardy, 1998; Hirsch, Koppich, & Knapp, 2001). The projected teacher shortage is also based on ass umptions of increased student enrollments and an aging workforce. But these assumptions have similarly been questioned. Research evaluating the Survey of Recent College Graduates ( Ballou, 1996) has shown that: In every year there were at least twice as many [qu alified] applicants as there were persons hired in full-time public school positions. Far from indicating that the nation faces a teacher shortage, these data show th at the teacher labor market as a whole has been in a chronic state of excess supply, though shortages may arise in specific locations and subject areas (p. 101). Research also indicates that regularly certified te achers are in short supply because of poor pay; low levels of job satisfaction, particularly when w orking with disadvantaged minority students (Hanushek, Kain & Rivkin, 2001); and because limite d faculty input about the management of schools discourages college graduates from teaching and drives current teachers out of the profession. Ingersoll (2001) suggests that:


21 of 53The imbalance of teacher supply and demand at the r oot of school staffing problems is neither synonymous with, or primarily d ue to, teacher shortages in the technical sense of a deficit in the quantity of qua lified candidates. Rather than insufficient supply, the data indicate that school staffing problems are primarily due to excess demand, resulting from a 'revolving d oor'—where large numbers of teachers depart their jobs for reasons other than r etirement. Thus the solution...does not primarily lie in increasing sup ply, but rather in decreasing demand (p. 501). A second argument for emergency certification is th at there are many people who would teach, but do not, because standard certification requirem ents prohibit them from doing so. Thus opponents of traditional teacher education programs call for the removal of certification requirements, claiming that there is no "special bo dy of knowledge" that teachers need to know in order to be successful. These advocates for the abolition of requirements claim that what needs to be learned by new teachers can be learned in the first year of teaching (Roth, 1994; Kanstroom & Finn, 1999; Finn & Kanstroom, 2000). I n fact, anecdotal evidence claiming that teachers learn more in the first year of teaching t han they do in their education courses is quite strong. Armed with this knowledge, it is then argu ed that a person who holds a college degree, in possession of some level of content knowledge, a nd with some limited experience teaching youth, is competent enough to begin to teach. Such beliefs drive the movement against certification despite research that argues against this position (McDiarmid & Wilson, 1991). Traditional certification programs are rejected by many adults who would be interested in teaching as a second career but who will not or can not engage in time consuming and expensive regular teacher certification programs. Proponents of alternative certification believe that these adults, called career transitioners, have skills th at they have learned in their other employment that could be used to teach children (MacDonald, Ma nning, & Gable, 1994). Additionally, some believe that adults have unique life skills an d experiences that can be useful to students (Zumwalt, 1991). Research does indicate that alter native certification programs attract an older and more diverse population, through their more fle xible schedules, less stringent requirements, and so forth; however, it is unclear from this rese arch that certification needs to be waived in order to recruit a more diverse teaching population with many life skills and employment experiences (Natriello & Zumwalt, 1993; Bradshaw & Hawk, 1996; Hawley, 1992). The third argument for emergency certification make s use of the long-standing lack of confidence by state officials and the general publi c in the quality of the teachers who graduate from colleges of education. Too often colleges of education are perceived to attract less able students, thus producing under qualified teachers. This is simply not true. Research supports the assertion that the academic quality of students entering colleges of education is quite good. For example, Gitomer, Latham, and Ziomek (1999) sho wed in their analysis of ACT and SAT scores that students from colleges of education wer e as academically skilled as students with other college majors. They also reported that trad itionally certified teachers have the highest passing rates on certification exams (PRAXIS I and II) compared to alternatively certified and emergency certified teachers, even though they appe ared to be similar in initial achievement, based on SAT scores. They concluded that traditiona l certification (having training in teaching methods, pedagogy and practice in teaching) makes a difference on licensure. They attribute the better performance of traditionally certified t eachers to the training and instruction, provided by colleges of education.


22 of 53Summary of Related ResearchWith regard to teacher subject matter the current r esearch suggests to us that in mathematics, especially, and at the upper grades, in particular, subject matter competency as assessed by college majors, courses taken, and degrees held, le ads to higher student achievement. Professional pedagogical knowledge appears equally important a contributor to student achievement at the upper grades, and may even be mo re important than content knowledge in the elementary grades.With regard to experience through teacher education course work and by means of learning on the job, the research suggests that student achieve ment is affected in positive ways. The powerful effects of content knowledge, pedagogical knowledge, and experience, contribute to success in teaching in one's own field. When teach ing out-of-field, such knowledge and skill is of less use and teacher effectiveness is compromise d. When we examined alternative routes to teaching we found them to be quite variable. Still, in comparison to traditional teacher education program s (that are also quite variable) the alternative programs take less time, are less costl y, attract more diverse students, but also record higher drop-out rates. Because they usually take m uch less time, alternative programs may suffer from under-preparing students for the classr oom, a problem compounded by the lack of supervision and support given by the hiring distric ts. Teach for America, as one alternative route to teaching, appears to suffer from the ordin ary and typical problems inherent in the designs of such programs. While criticisms abound, there is a substantial body of literature suggesting that traditional teacher education progr ams, warts and all, seems to provide more competent appearing teachers whose students perform better. The research suggests that emergency certified te achers are probably the least prepared to do well. Unfortunately, such teachers are typically h ired into districts with the hardest to teach students.After reviewing this literature it seemed as if the conditions necessitating out-of-field teaching and the hiring of alternate and emergency certified teachers could easily be modified, eliminating the need for these practices to exist. In the past, however, neither politicians nor school boards had the will to do that. But under t he new federal No Child left behind Act of 2001, school districts will have to have a "highly quali fied" teacher in every classroom or loose federal funding. It will be an interesting few yea rs as ways to define "highly qualified" and the related term "competent" are worked on to meet the letter of the law. The research in these areas is also likely to be reinterpreted in light of that law. In the near future, surely both the definitions of, and the research associated with th e idea of teacher quality, will find their ways into courtrooms of America. This study may help th e courts in thinking about what it means to have a highly qualified teacher in the classroom.MethodResearch DesignAn ex-post-facto archival research design was used to study the performance of students in the classes of the under-certified and certified teache rs in selected districts of the state of Arizona. Districts provided both the information about the t eachers participating in this study and their


23 of 53Stanford Nine (SAT 9) class means. The SAT 9 data provided by districts were also compared with the same data obtained from the Arizona Depart ment of Education. SAT 9 data were available from 1997-2000, but was not available for the 2000-2001 academic year. Sample In Arizona under-certified teachers have three labe ls: "emergency" (for holders of bachelor degrees from accredited institutions, with little o r no educational coursework, who can get clearance of criminal background through fingerprin t analysis); "temporary" (a rarely used designation similar to "emergency"); and "provision al" (for those with some, or even considerable teacher education training, who are sh ort certain units or requirements that could earn them a standard certificate). In opposition t o the under-certifed teachers are the regularly certified teachers who meet all of the state requir ements for certification. These teachers hold a bachelor's degree from an accredited institution, h ave completed 45 semester hours of elementary or secondary education course work, obta ined a passing scores on the AEPA, demonstrated that they understand both the Arizona and US constitution, and been vetted for any criminal background through analysis of their f ingerprints. (The definitions of under-certified and certified teachers, given above were accurate through the year 2000, when the Arizona Department of Education made changes to the certification laws. Certification requirements are still in flux, and so current Ariz ona certification policies may not be the same as those reported in this study.)Among the under-certified teachers in this study ar e some from an alternative route to traditional teacher training, the program "Teach fo r America (TFA)." Teach for America is a popular alternative certification program. Its mis sion calls for placing energetic, bright, but unqualified teachers into poor, urban school distri cts (Darling-Hammond, 1994). The instructional effectiveness of under-certified teac hers in general, and the TFA teachers in particular, is of considerable interest to the poli cy community. District Selection Arizona school district superintendents listed by the department of education as participating in the Federal Teacher S hortage Loan Deferment Program (Arizona Department of Education, policy/ts) were invited to participate in this study. This federal program requires the State Dep artment of Education to generate a ranking of school districts in the state by the percent of under-certified teachers in each district. This list of school districts provided a convenient popu lation from which to obtain a sample. Of the school districts receiving a request to participate (N=24 for the 1998-1999 dataset, N=12 for the 1999-2000 dataset) five school districts responded positively. These five school districts represent 20.8% and 16.6%, respectively, of the sch ool districts designated by the state as having severe teacher shortages. All five school d istricts chosen for this study were included in the Department of Education's 1998-1999 classificat ion of school districts. Only two of these school districts were also included in the Departme nt of Education's 1999-2000 classification of school districts with severe shortages. All of the participating school districts shared similar characteristics. They all serve inner city student populations, largely minority, and all participate in the Teach for America (TFA) program. All the schools in these districts have difficulty filling teaching positions.Methods of data collection. The five participating school districts provided l ists of new hires for the 1998-1999 and 1999-2000 school years. In s ome cases this list contained information about the teachers' certification status, while in other cases further research was necessary to obtain this information. We were granted permissio n to access individual teacher personnel


24 of 53files in order to collect the necessary data on the school where the teacher was employed, the grade level taught, the teachers' certification sta tus, their highest degree earned, the date and institution where there degree was earned, their ag e (determined from year of their birth), and teaching experience. Of the teachers included in the dataset, 64% had no prior teaching experience, a judgment based on their hire date, employment history, resume and application. The majority of newly hired teachers were recent graduates from college. Some of the new teachers, however, had delayed their entrance into teaching for many years after g raduation, but they had no prior teaching experience indicated in their personnel files. Tea chers were removed from the sample if they taught kindergarten, first grade, art, music or spe cial education, grades and subjects not assessed by the Stanford Nine (SAT 9).The assessment departments of each school district provided test scores aggregated at the classroom level. Included were the teachers' class totals as raw scores, scaled scores, grade equivalent, national percentile rank, stanine, nati onal normal curve equivalent, as well as class percentile rank and class stanine. Individual stud ent scores were not needed or provided. Additionally, state level SAT 9 data was later obta ined from the Research and Policy division of the State Department of Education in order to co nfirm the accuracy of the SAT 9 data collected from the school districts. State level d ata was aggregated, by teacher name, for each of the school districts. In the event of a discrep ancy between the two sets of data, we opted to use the state level data. This discrepancy occurre d with only one of the school districts data files.Matching ProcedureIn the five districts studied, 293 teachers were hi red in 1998-1999 and 1999-2000 who met the requirements for inclusion in this study. In order to participate teachers personnel files were required to contain the demographic data necessary for analysis, as well as classroom level SAT 9 scores. Teachers' data files were matched using SPSS procedures to sort the files. Teachers in each district were first categorized on the basi s of their certification status, under-certified teachers (labeled emergency, provisional or tempora ry certified teachers by their districts) constituted one group, certified teachers made up t he other group. Teachers from each group were then matched based on grade level and highest degree attained. Teachers for whom no matches could be found were removed from the analys is. Matches were made using the following rules: 1) mat ches were first made within the same school, 2) matches were made within the same school district, and 3) matches were made between similar school districts. The first and se cond matching rules serve to minimize exogenous variables associated with student achieve ment scores, e.g., socio-economic status, school characteristics, curriculum, etc. It is ass umed that teachers in the same school teach similar students, an imperfect but reasonable assum ption. The identical assumption can be made about schools within the same district boundar ies, since Arizona school district boundaries are based on relatively homogenous geogr aphic areas. Conversations with district personnel, in the course of collecting the data at the district offices, provided no evidence that the certified or under-certified teachers were "tra cked" in any way. The assignment of teachers to schools, and to classrooms within schools, appea rs to have been unbiased. Similarly, we have no reason to believe that class size or studen t ability was different in any way for the certified or under-certified teachers in our sample


25 of 53 The cross-district matching of teachers, however, i s more problematic than the within district matching. We made these matches based on the "same ness" of the two districts. Sameness was determined using data collected from the Education Finance Statistics Center, a subdivision of the National Center for Education Statistics (U.S. Department of Education, 2001). Using public school district financial records for the la test year available, 1996-1997, data about each of the participating school districts were collecte d for: student teacher ratio, administrative ratio, median income, percent of householders with high school graduates, percent of non-white children, percent of limited English proficiency, a nd percent of children in poverty. School districts that shared similar characteristics were matched to one another. This procedure matched the Osborn Elementary School District to th e Creighton Elementary School District, and matched the Roosevelt Elementary School Distric t to both the Nogales Elementary School District and the Murphy Elementary School District. These are not pseudonyms, but the actual names of the Arizona school districts that generous ly helped us in this study. Synthesizing data from the various state department s of education, also provided data that were used to match schools havin g similar characteristics. Sameness matches were also made based on: AIMS reading and m ath scores, SAT 9 reading and math scores, teacher experience, percent of teachers wit h masters degrees, attendance rate, open enrollment, percent free and reduced lunch, and eth nic breakdown within the school district. These data support the matches that were made using the NCES finance data with the exception that the Murphy School District was found to more c losely match the Creighton School District. Teacher matching across districts, there fore, was accomplished by finding similar school districts serving similar student population s with similar economic bases. The assertion that teachers in these different school districts a re sensibly matched is well substantiated, but we acknowledge that the nature of the matching procedu res used constitutes a potential flaw in studies such as this. Random assignment of under-c ertified and certified teachers to classes within districts would have provided a stronger des ign, but this was not possible. It is important to note, however, that the matches in thi s study were made with out any knowledge of the teachers' class scores on the SAT 9.Descriptive analyses were conducted on the complete sample of teachers in the data set (N= 293). These results are provided in Table 1.Table 1 Descriptive Statistics for 1998-1999 and 1999-2000 Datasets Certified TeachersN=159 Emergency Certified TeachersN=89 Temporary Certified TeachersN=19 Provisionally Certified TeachersN=26 District Murphy2113101 Creighton643400 Osborn21222 Roosevelt41351919 Nogales12544 Grade


26 of 53 225937 319825 429724 5241312 6212245 7221432 8191641 College ASU741369 U of A9612 NAU16533 U of Phoenix0400 Ottawa8021 Grand Canyon0101 Out-of-State (largepublic) 142524 Out-of-State (smallpublic) 13811 Out-of-State (largeprivate) 4700 Out-of-State (smallprivate) 6824 Foreign Education91000 Prescott College1220 Other4000 Certified TeachersN=159 Emergency Certified TeachersN=89 Temporary Certified TeachersN=19 Provisionally Certified TeachersN=26 Major Education9710 Elem. Ed.7471419 Second. Ed.5100 Early Child.4000 C & I6000 Ed. Admin.6000 Bilingual Ed.8020 Spec. Ed.2000 Phys. Ed.1010 Liberal Arts101200


27 of 53 Soc. Sciences2325114 Sciences41703 Business4700 Languages11100 Other2200 Degree BA 126771725 MA 321221 PhD 1000 TFA Status Yes 03202 No 159571924Note: N = 293 From this population, teachers for whom no matches were found were removed from the analysis. The initial matching procedures produced 116 pairs of teachers (N=232 individual teachers) out of the total teacher pool of 293, the reby using 79% of the original dataset. We undertook analyses of the quality of the matching p rocedures that were used. These are reported on, below, and each analysis is based eith er on the entire sample drawn (N = 293), or on the pairs that were created by the matching proc edure that was used (N = 232). However, the process of matching produced 28 mismat ched pairs consisting of teachers who did not share class score data for the same test ad ministrations. This occurred, for example, when one teacher in a pair had scores for 1998 and the other teacher had scores for 1999. Additional matches were then made based on all of t he above matching rules, but eliminating cross year matches. This finally resulted in N=109 matched pairs, using 74% of the original data set. The appendix to this study contains descriptive inf ormation about the 109 pairs of teachers who comprise the sample for this study. The data are also available as a Microsoft Excell spreadsheet. Matching in primary schools across grades 3-8, howe ver, created a problem inherent in archival studies such as this one. To design this study sen sibly we needed teachers of self-contained classes. If departmentalization (more than one tea cher working with the class) were occurring we would have problems inferring a teacher's affect on student learning. But we have no knowledge of what went on in every school at these upper grade levels. We were told, however, that these schools used little departmenta lization, and that the teachers for whom we had files were the classroom teachers of record for the district and the state. The 218 teachers in our sample were, therefore, the teachers designa ted by administrators as those responsible for their student 's achievements on the SAT 9 tests. Since these were the responsible teachers we included all the matches from grades 3 to 8. Never theless, because we worried about the issue of departmentalization in the upper two grade level s, we ran separate analyses. One set of analyses was done with the full sample of 109 pairs of certified and under-certified teachers, and another set of analyses was done with a reduced sample, eliminating all the pairs of teachers in the 7th and 8th grade. The appendix to this report describes all 109 pairs of teachers by grade level and thus identifies which pairs were eliminated from the second analysis. In the


28 of 53discussion that follows, when the second analysis u sing the smaller sample of pairs of teachers from grades 3-6 produces results different from the analysis of the entire sample, we will note these differences.InstrumentsThe Stanford Achievement Test, Ninth Edition (SAT 9 ), a nationally norm referenced standardized test is used by all districts in the s tate of Arizona and was used, as well, in this study. The test assesses student achievement in re ading, math and language arts (Harcourt Brace, 1997). The SAT 9 is believed by the State D epartment of Education to relate to Arizona academic standards, which teachers use as a guide t o instruction. It is claimed that The SAT 9 tests between 70-80% of the material outlined in th e state's academic standards (ADE personal communication, 2001). This relationship is stronge st for the reading and mathematics subtests, and in grades 2 through 8. The language subtest of the SAT 9 is not as well related to state standards because it does not require a writing sam ple of students, an ability that is promoted in the standards. The analysis of the State Departmen t of Education suggests that the SAT 9 is a reasonable indicator of student achievement, perhap s more for reading and mathematics than for language. Furthermore, in Arizona the test is often used as an indicator of teacher and school effectiveness.Scoring Once teachers were matched to one another their Stanford Achievement Test, Ninth Edition (SAT 9) scores were aggregated at the class level. For each pair of teachers their mean National Normal Curve Equivalent (NCE) scores for S AT 9 reading, math and language were analyzed. The NCE scores are a type of normalized standard score resulting form the division of the normal curve into 99 equal units. This scor e is traditionally used for research purposes, enabling researchers to interpret differences in NC Es more readily because of the equal-interval nature of the NCE scores. Differences between NCEs obtained by different groups have the same meaning regardless of what part of the scale i s referenced. For the purposes of this study individual student scores were not collected and th us cannot be reported. Analyses of Matching ProceduresThe credibility of the matching procedure for pairi ng uncertified and certified teachers is important in the interpretation of the results of t his study. Therefore, we undertook some analyses to explore that issue. We began by lookin g at the similarity of the SAT 9 test scores in each school and district to determine their compara bility to each other, a check on the level of "sameness" of each school and district to one anoth er. These analyses were conducted using NCE scores on the mathematics, reading and language sub-tests of the SAT 9, for both the 1998-1999 and 1999-2000 data sets. Alpha levels we re set at p = .05. Matching Analysis 1 To answer the question about whether students' t est scores on the SAT 9 are different as a function of which school they at tend, a one-way analysis of variance (ANOVA) was conducted on the entire data set (N=293 ) to evaluate the relationship between the school assignments and student achievement scor es. The independent variable was teachers' school assignment; the dependent variable was mean student achievement scores of these teachers as measured in Normal Curve Equivale nts (NCE) for reading, mathematics and language in 1998-1999 and 1999-2000. The ANOVA was not significant with the exception of mathematics scores in the 1999 sample. These resul ts are provided in Table 2. The results indicate that, overall, the schools from whom teach ers in the sample came showed no statistically significant differences from each oth er in terms of mean NCE scores on the SAT 9,


29 of 53 except for the special case of mathematics scores i n the 1999 sample, F (41,190) = 1.65, p = .01. It is unlikely that there are inherent differ ences in the schools that could bias the findings of subsequent analyses.Table 2 ANOVA Results for School SamenessSAT 9 Subtest and Year Sum of SquaresdfMean SquareFSig. Reading 1998Between Groups2853.473679.261.000.487 Within Groups7389.239379.45 Total10242.70129 Math 1998Between Groups2801.623677.821.070.385 Within Groups6753.209372.62 Total9554.82129 Language 1998Between Groups3887.7336107.991.200.237 Within Groups8341.169389.69 Total12228.89129 Reading 1999Between Groups4621.2641112.711.420.060 Within Groups15037.2319079.14 Total19658.49231 Math 1999Between Groups6176.9541150.661.650.014* Within Groups17398.9319091.57 Total23575.88231 Language 1999Between Groups4831.1341117.831.370.083 Within Groups16363.2119086.12 Total21194.3404231 Note. Indicates significance p = .05 To determine df for 1998-1999 sample:BG= each school with cases N= 37, df = 37-1= 36WG= each case (130) – total groups (37) df = 130-37 =93 Total df = N-1, 130-1 = 129To determine df for 1999-2000 sample:BG = each school with cases N = 42, df = 42-1= 41WG= each case (232) – total groups (42) df = 232-42 = 190 Total df = N-1, 232-1 = 231Matching Analysis 2 To answer the question about whether the test sc ores of students whose teachers might be paired in later analyses differed as a function of which district they attend, a one-way analysis of variance (ANOVA) was conducted on the entire data set (N=293) to evaluate the relationship between the district assi gnments and student achievement scores. The independent variable was teachers' district assignm ent; the dependent variable was the classroom mean student achievement scores of these teachers as measured in Normal Curve


30 of 53 Equivalents (NCE) for reading, mathematics and lang uage in 1998-1999 and 1999-2000. The results indicate significant differences between th e mean NCE scores for all subtests and for both years of data. For the 1998-1999 data set the ANOVA was significant, F (4,126) = 3.20, p = .02 for reading, F (4,126) = 2.81, p = .03 for mathematics, and F (4,126) = 3.38, p = .01 for language. For the 1999-2000 data set, F (4, 228) = 8.19, p = .01 for reading, F (4, 228) = 8.75, p = .01 for mathematics, and F (4, 228) = 6.93, p = .01 for language. These results are provided below in Table 3.Table 3 ANOVA Results for District SamenessSAT 9 Subtest and Year Sum of SquaresdfMean SquareFSig. Reading 1998Between Groups957.574239.393.200.015* Within Groups9413.5312674.71 Total10371.10130 Math 1998Between Groups799.174199.792.810.028* Within Groups8964.8812671.15 Total9764.05130 Language 1998Between Groups1183.684295.923.380.011* Within Groups11045.5612687.66 Total12229.24130 Reading 1999Between Groups2471.104617.788.19.000* Within Groups17200.5022875.44 Total19671.60232 Math 1999Between Groups3144.954786.248.75.000* Within Groups20486.5122889.85 Total23631.46232 Language 1999Between Groups2297.244574.316.93.000* Within Groups18902.6122882.91 Total21199.85232 Note. Indicates significance p = .05To determine df for 1998-1999 sample:BG= each district N= 5, df= 5-1= 4WG= each case (131) – total groups (5) df= 131-5=12 6 Total df= N-1, 131-1= 130To determine df for 1999-2000 sample:BG= each district N= 5, df= 5-1= 4WG= each case (233) – total groups (5) df= 233-5=22 8 Total df= N-1, 233-1= 232The results of this ANOVA indicate that at the dist rict level the mean student NCE scores were statistically different from one another for the te achers who comprise the population from


31 of 53 which our sample would be analyzed. This suggests that the procedures we used to match teachers across districts were not faultless. But analysis of the mean student scores across districts suggests that only one district may have been an outlier, with slightly higher SAT 9 scores than the others. Since only 38% of all tea chers had to be matched with teachers from another district, it is likely, therefore, that onl y a small percent of those matches could have been problematic, totaling less than 10% of all the matches that were made. In addition, the matching of teachers across district lines was base d on multiple measures of district sameness; NCE scores provide only one such measure. Because of that, we believe that the matching of teachers across district lines can still be defende d as a reasonable way to obtain a sample for analysis of the student achievement of certified an d uncertified teachers. The discrepancy between the results for the ANOVA o n the participating schools and the participating districts is curious and remains an i ssue to be resolved. This is, of course, one of the reasons that hierarchical designs have become n ecessary in the analysis of classroom, school and district level data. But for the purposes of t his study, it is not clear that this discrepancy would cause any systematic bias in the data analyse s to follow. Matching Analysis 3 After the 109 pairs of matched teachers were ide ntified we then inquired whether the average SAT 9 scores of certified teach ers differed as a function of whether they were matched with teachers within their same school district or with teachers from another participating school district. A one-way analysis of variance (ANOVA) was used in this analysis. The independent variable was teachers' d istrict assignment, either within or between school district. The dependent variable was the st udent achievement scores of these teachers as measured in Normal Curve Equivalents (NCE) for read ing, mathematics and language in 1998-1999 and 1999-2000. ANOVA results indicate no significant differences between the mean NCE scores for the certified teachers matched within the same district as compared to certified teachers matched between districts. Thes e results are provided below in Table 4.Table 4 ANOVA Results for Certified Teacher Matches Within and Between DistrictsSAT 9 Subtest and Year Sum of SquaresdfMean SquareFSig. Reading 1998Between Groups66.66166.660.840.36 Within Groups4195.265379.16 Total4261.9254 Math 1998Between Groups56.29156.290.760.39 Within Groups3932.175374.19 Total3988.4654 Language 1998Between Groups8.8818.880.110.74 Within Groups4251.575380.22 Total4260.4554 Reading 1999Between Groups30.45130.450.360.55 Within Groups7566.448985.02


32 of 53 Total7596.8990 Math 1999Between Groups4.6814.680.050.82 Within Groups8336.888993.67 Total8341.5690 Language 1999Between Groups23.20123.200.330.57 Within Groups6279.828970.56 Total6303.0290 To determine df for 1998-1999 sample:BG= match type N= 2, df= 2-1= 1WG= each case (55) – total groups (2) df= 55-2=53Total df= N-1, 55-1= 54To determine df for 1999-2000 sample:BG= match type N= 2, df= 2-1= 1WG= each case (91) – total groups (2) df= 91-2=89Total df= N-1, 91-1= 90Matching Analysis 4 After the 109 pairs of matched teachers were ide ntified we then inquired whether the average SAT 9 scores of under-certified (emergency, temporary or provisional certified) teachers' differed as a function of whet her they are matched within the same school district or with teachers from another participatin g school district. A one-way analysis of variance (ANOVA) was conducted. The independent va riable was teachers' district assignment, either within or between school district. The depe ndent variable was the student achievement scores of these teachers, as measured in Normal Cur ve Equivalents (NCE) for reading, mathematics and language in 1998-1999 and 1999-2000 In general, the ANOVA results indicate no significant differences between the sco res of under-certified teachers matched within district as compared to under-certified teac hers matched between districts. But that was not true for all tests. Significant differences in NCE scores were found for mathematics in the 1999-2000 data set. For mathematics, F (1, 93)= 8.08, p = .01. The exclusion of 7th and 8th grade teachers yielded similar results; F (1, 67)= 4.12, p = .047. These results are provided below in Table 5.Table 5 ANOVA Results for Under-Certified Teacher Matches Within and Between DistrictsSAT 9 Subtest and Year Sum of SquaresdfMean SquareFSig.Reading 1998Between Groups0.9310.930.020.90 Within Groups2214.403563.27 Total2215.3336 Math 1998Between Groups19.52119.520.450.51 Within Groups1503.553542.96 Total1523.0736 Language 1998Between Groups14.09114.090.170.68


33 of 53 Within Groups2936.853583.91 Total2950.9436 Reading 1999Between Groups69.52169.520.820.37 Within Groups7849.489384.40 Total7919.0094 Math 1999Between Groups688.591688.598.080.01* Within Groups7928.949385.26 Total8617.5394 Language 1999Between Groups246.321246.323.270.07 Within Groups6856.389175.35 Total7102.7192 Note. Indicates significance p = .05To determine df for 1998-1999 sample:BG= match type N= 2, df= 2-1= 1WG= each case (37) – total groups (2) df= 37-2=35Total df= N-1, 37-1= 36To determine df for 1999-2000 sample:BG= match type N= 2, df= 2-1= 1WG= each case (95) – total groups (2) df= 95-2=93Total df= N-1, 95-1= 94For 1999-2000 Language: Total cases= 93It has been argued, above, that the matching proced ures used in this study were sensible. The four statistical analyses intended to evaluate the matching procedures provide evidence that they were not perfect, but that evidence does not lead t o the conclusion that the approach taken in this study was unreasonable or would lead to faulty conclusions. The matching of the pairs of teachers, one certified with one under-certified, w ithin and across district lines, took place before the SAT scores of the teachers in each pair were scrutinized. Thus the matching procedures appear to be unbiased with regard to the research questions that are of interest. The results of the analyses appropriate to these resear ch questions are considered next.ResultsWe first chose to look at whether the three kinds o f under-certified teachers differed among themselves. We believed that subsequent analyses w ould be simpler if the SAT 9 NCE scores of the students of these three groups of teachers w ere not statistically different from each other. If that were the case, we could treat the three sub -groups of under-certified teachers as a single group. A one-way analysis of variance (ANOVA) was conducte d in which the independent variable was teachers' certification, while the dependent va riable was the student achievement scores of these teachers as measured in Normal Curve Equivale nts (NCE) for reading, mathematics and language in 1998-1999 and 1999-2000. Results indica te that the NCE scores for all under-certified teachers (emergency, temporary and provisional) were not statistically different from one another. These results are provided below in Table 6. Because of this finding we subsequently treated all uncertified teachers as a homogeneous group. In all subsequent


34 of 53 analyses we will compare certified teachers to the total group of under-certified teachers (emergency, temporary and provisional).Table 6 ANOVA Results for Emergency, Temporary & Provisiona l Certified TeachersSAT 9 Subtest and Year Sum of SquaresdfMean SquareFSig. Reading 1998Between Groups16.9128.460.130.88 Within Groups2198.423464.66 Total2215.3336 Math 1998Between Groups68.48234.240.800.46 Within Groups1454.593442.78 Total1523.0736 Language 1998Between Groups176.61288.311.080.35 Within Groups2774.333481.60 Total2950.9436 Reading 1999Between Groups116.51258.250.690.51 Within Groups7802.499284.81 Total7919.0094 Math 1999Between Groups131.56265.780.710.49 Within Groups8485.979292.24 Total8617.5394 Language 1999Between Groups120.69260.350.780.46 Within Groups6982.029077.58 Total7102.7192 To determine df for 1998-1999 sample:BG= match type N= 3, df= 3-1= 2WG= each case (37) – total groups (3) df= 37-3=34Total df= N-1, 37-1= 36To determine df for 1999-2000 sample:BG= match type N= 3, df= 3-1= 2WG= each case (95) – total groups (3) df= 95-3=92Total df= N-1, 95-1= 94For 1999-2000 Language: Total cases= 93To answer the key question in this study, whether s tudents taught by certified teachers outscore students taught by under-certified teachers, a corr elated t -test was conducted to evaluate the difference in student achievement scores of the cer tified and under-certified teachers. The results indicate that for 1998-1999, students taugh t by certified teachers outperformed students taught by under-certified teachers. More specifica lly, the reading scores of the students of certified teachers were significantly higher ( M =36.52, SD = 9.59) than were the reading scores


35 of 53 obtained by the students of under-certified teacher s ( M =30.67, SD = 8.02), t (27)= 2.36, p = .01. In this same year, on the language test, the scores of the students taught by certified teachers were significantly higher ( M =34.33, SD = 9.17) than were the scores of the students taught by under-certified teachers ( M =29.89, SD = 9.82), t (27)= 1.81, p = .04. While the difference between the certified and the under-certified teach ers on the mathematics test were not found to be significant, the results were in the same direct ion as they were for the reading and language tests. Students taught by certified teachers score d higher ( M =38.80, SD = 8.77) than did the students taught by under-certified teachers ( M =35.82, SD = 7.32). Results for 1999-2000 replicated the results of the data from 1998-1999. Students taught by certified teachers significantly outperformed stude nts taught by under-certified teachers on every test. In reading, the scores of the students of certified teachers were significantly higher ( M =35.62, SD = 9.31) than were the scores of students instructed by under-certified teachers ( M =32.48, SD = 9.43), t (86)= 2.43, p = .01. In mathematics, the scores of students of c ertified teachers were significantly higher ( M =39.75, SD = 9.52) than were the scores obtained by students of under-certified teachers ( M =35.22, SD =9.77), t (86)= 2.95 p = .001. And in the area of language, the scores of the students of certifie d teachers were significantly higher ( M =35.60, SD = 8.57) than were the scores of the students instru cted by under-certified teachers ( M =33.47, SD = 8.90), t (84)= 1.71, p = .05. These results are provided below in Table 7 The exclusion of 7th and 8th grade teachers yielded similar, and mor e dramatic results; the average difference between these two groups increased. Moreover, in t his analysis of only grades 3-6, all subtests across both years were found to be significantly di fferent.Table 7 Correlated ttest Results Comparing Certified and Under-Certified TeachersSAT 9 Subtestand Year Mean ofDifferencesSt. Dev. St. Error of Mean 95% Conf. Int.Lower Limit 95% Conf. Int.Upper LimittdfSig. Reading 19985.8513.112.480.7710.932.36270.01*Math 19982.9711.442.16-1.477.411.37270.09Language 19984.4413.012.46-0.609.491.81270.04*Reading 19993.1412.071.290.575.712.43860.01*Math 19994.5314.311.531.487.582.95860.00*Language 19992.1311.491.25-.354.611.71840.05* Note. Indicates significance p = .05 To determine df for 1998-1999 sample:Number of matches N=28Total df= N-1, 28-1= 27To determine df for 1999-2000 sample:Number of matches N=87Total df= N-1, 87-1= 86For 1999-2000 Language: Total cases= 85The NCE scale provides a metric for evaluating the differences between certified teachers and under-certified teachers. Students taught by certi fied teachers outscored their counterparts by 6 NCE points in reading, 3 NCE points in mathematics and nearly 5 NCE points in language in


36 of 53 1998-1999. The results are similar for 1999-2000. Students taught by certified teachers outscored their counterparts by 3 NCE points in rea ding, 5 NCE points in mathematics, and 2NCE points in language. Expressed as a proportion of the standard deviation of the NCE scale represented as an effect size, these differences ra nge across the two years from .14 to .28 in reading, 14 to .24 in mathematics and .09 to .19 in language. These results are provided below in Table 8. The exclusion of 7th and 8th grade teac hers from this analysis yielded similar, yet more dramatic results in terms of effect size. Acr oss the two years the range of the effect sizes were from .19 to .38 in reading, .24 to .28 in math ematics and .14 to.33 in language. For ease of discussion it is appropriate to choose a summary statistic to represent these data. A reasonable way to do that is to conclude that the a verage ES across all sub-tests of the SAT 9, across both years of testing, and across analyses, is around .20. Because of the relationship between effect size (ES) and yearly progress on sta ndardized tests (Glass, 2002), one could expect that during one academic year in the primary grades, the students of certified teachers would make approximately 2 months more academic gro wth than would the students of under-certified teachers. The academic year is a 1 0-month year so the loss of two months or 2/10ths of a year is the loss incurred by students placed with under-certified teachers. That is, students pay approximately a 20% penalty in academi c growth for each year of placement with under-certified teachers.Table 8 NCE Differences between Certified and Under-Certifi ed Teachers & Effect Size (ES) Ranges for 1998-1999 and 1999-20 00SAT Sub-Test1998 – 19991999 – 2000ES RangeReading63.14 – .28Math35.14 – .24Language42.09 – .19Note. *Effect sizes (ES) when using normal curve eq uivalencies (NCE) must be calculated with a standar d deviation of 21.06 NCE units.To answer the question whether the test scores of s tudents of teachers in the Teach for America program are different from the scores of students w ho studied with other under-certified teachers, a one-way analysis of variance (ANOVA) wa s conducted. The independent variable was TFA trained vs. all other forms of training amo ng the under-certified teachers. The dependent variable was the student achievement scor es of these teachers as measured in Normal Curve Equivalents (NCE) for reading, mathematics an d language in 1998-1999 and 1999-2000. ANOVA results indicate that the NCE scores of TFA t eachers were not statistically different from the NCE scores for other under-certified teach ers. These results are provided below in Table 9.Table 9 ANOVA Results Teach for America Teachers & Other Under-Certified TeachersSAT 9 Subtest and Year Sum of SquaresdfMean SquareFSig. Reading 1998Between Groups34.00134.000.550.47


37 of 53 Within Groups2181.333562.32 Total2215.3336 Math 1998Between Groups21.30121.300.500.49 Within Groups1501.783542.91 Total1523.0836 Language 1998Between Groups261.641261.643.410.07 Within Groups2689.303576.84 Total2950.9436 Reading 1999Between Groups92.74192.741.100.30 Within Groups7826.269384.15 Total7919.0094 Math 1999Between Groups1.3111.31.01.91 Within Groups8616.229392.65 Total8617.5394 Language 1999Between Groups19.94119.940.260.61 Within Groups7082.779177.83 Total7102.7192 To determine df for 1998-1999 sample:BG= TFA or under-certified N= 2, df= 2-1= 1WG= each case (37) – total groups (2) df= 37-2=35Total df= N-1, 37-1= 36To determine df for 1999-2000 sample:BG= TFA or under-certified N= 2, df= 2-1= 1WG= each case (95) – total groups (2) df= 95-2=93Total df= N-1, 95-1= 93For 1999-2000 Language: Total cases= 93Given the previous analyses, in which the students of certified teachers outperformed the students of under-certified teachers and the studen ts of TFA teachers scored no different than did the students of other under-certified teachers, it may be that the next analysis is superfluous. Nevertheless, because of the intensit y of the debate about the performance of TFA teachers described in the literature reviewed above we chose to inquire whether students taught by certified teachers outperformed students taught by Teach for America (TFA) teachers. A correlated t -test was used for this analysis and it indicated t hat for the 1999-2000 data set, in reading, the scores of the students of the certifie d teachers were significantly higher ( M =35.53, SD = 9.87) than were the scores of the students of TFA teachers ( M =30.51, SD = 6.85), t (21)= 1.87, p = .04. In mathematics, the scores of the students of the certified teachers were significantly higher ( M =41.15, SD =9.21) than were those obtained by students of teac hers from the TFA program ( M =35.34, SD = 7.67), t (21)=2.13, p = .02. Finally, in language, the scores of the students in the classes of the certified teache rs were significantly higher ( M =36.43, SD = 9.48) than were the scores of the students of teach ers trained by TFA ( M =32.11, SD = 8.71), t (21)=1.79, p = .04.


38 of 53 Although the same pattern of results were found for the 1998-1999 sample, the differences between the two groups were not statistically signi ficant. We believe this occurred because of the smaller sample size in the 1998-1999 analysis. These results are provided below in Table 10. The exclusion of 7th and 8th grade teachers yi elded similar, and more dramatic results; the average difference between these two groups increas ed. In this analysis the differences were found to be significant in both years, in all subte sts, except for math in 1998-1999.Table 10 Correlated t-test Results Comparing Certified Teachers and Teach for America TeachersSAT 9 Subtestand Year Mean ofDifferencesSt. Dev. St. Error of Mean 95% Conf. Int.Lower Limit 95% Conf. Int.Upper LimittdfSig. Reading 19984.2810.103.57-4.1712.731.2070.13Math 19982.2510.023.54-6.1310.620.6370.27Language 19982.577.152.53-3.418.541.0270.17Reading 19995.0212.582.68-0.5610.601.87210.04*Math 19995.8112.812.730.1311.492.13210.02*Language 19994.3111.292.41-0.699.321.79210.04*Note. Indicates significance p = .05To determine df for 1998-1999 sample:Number of matches N=8Total df= N-1, 8-1= 7To determine df for 1999-2000 sample:Number of matches N=22Total df= N-1, 22-1= 21The data set was also examined to gain information about the role of experience in developing teacher competency. For teachers that were in both the 1998-1999 and the 1999-2000 set of data, we had hoped to look at whether teacher exper ience effects student SAT 9 scores, and more particularly, whether the differences in perfo rmance between the certified teachers and the uncertified teachers was moderated as a function of the increased experience of the uncertified teachers. But the sample of teachers for whom we h ad data across two years was very small (six pairs across the two years), so no confident a nswers to these questions can be offered. One of our analyses was a one-way within-subjects A NOVA, with the factor being experience, as measured in time from 1998-1999 to 1999-2000. T he dependent variable was the student's achievement scores for these teachers as measured i n Normal Curve Equivalents (NCE) for reading, mathematics and language in 1998-1999 and 1999-2000. The results indicate that there is no significant difference in NCE scores fr om the first year to the second year. Nevertheless, the scores for each subtest of the SA T increased from the first year to the second, indicating that teacher experience may affect the a chievement test scores of their students. The means and standard deviations are provided below, i n Table 11. The scores increased from one to two NCE points in each of the three subtests, with the increase in mathematics being the greatest. The di fference in the scores between the first year and second year are provided in Table 12. We also ran an ANOVA on these changes over time,


39 of 53 and those results are given in Table 13.Table 11 Means and Standard Deviations for Matched Teachers with Two Years of DataSAT 9 Subtest and YearMeanStd. DeviationNReading 1998-199936.5014.2512Reading 1999-200037.797.2612Math 1998-199939.0311.4912 Math 1999-200041.078.3812Language 1998-199935.8513.0012Language 1999-200037.057.3812Table 12 Difference in Mean SAT 9 Scores for Matched Teachers with Two Years of DataSAT 9 Subtestand Year Mean ofDifferencesSt. Dev. St. Error of Mean 95% Conf. Int.Lower Limit 95% Conf. Int.Upper LimittdfSig. Reading-1.2914.544.20-10.527.95-0.31110.38Math- 13 ANOVA for Teachers with Two Years of DataSAT 9 Subtest FdfError dfSig. Reading.09111.77Mathematics.58111.46Language.13111.73In order to evaluate whether the differences between certified teachers and under-certified teachers, with two years of data, remained similar, grew or decreased from the first year to the second, mean NCE scores for each group were analyze d. Results indicate that the difference between the scores of certified teachers and the sc ores of under-certified teachers for the 1998-1999 to 1999-2000 data set, as measured in NCE scores, decreased in reading and language, but increased in mathematics. These resu lts are provided below, in Table 14.Table 14 Difference between Certified and Under-Certified Te achers' NCE Scores from 1998-1999 to 1999-2000 for


40 of 53 Teachers with Two Years of DataAcademic YearReadingMathematicsLanguage1998-199912.937.809.511999-20003.189.474.02 Note. All scores favor certified teachers over unde r-certified teachers.Discussion and ConclusionMany different values necessarily come into play wh en making educational policy about the qualifications that are needed to become a beginnin g teacher. So much is riding on the performance of these individuals, trusted with educ ating our nation's young. So many skills are needed to do that job well. Thus, a single empiric al study of this kind cannot provide answers to complex policy questions about the relative bene fits and liabilities of allowing certified and under-certified teachers to teach our young. Never theless, there is every reason to think that the results of this study are generalizable and worth c onsidering when educational policies on these issues are debated. As we understand the national situation it appears not to be very different from that in Arizona. From New York, through Chicago, and on to Los Angel es, teachers in schools that serve the poor are often under-certified, inexperienced, and may be teaching out-of-field. Teachers who serve wealthier students overwhelmingly hold regula r certification, have accumulated considerably more teaching experience, and are less often required to teach out-of-field. (Darling-Hammond, 1997a, 1997b; 2001; Ingersoll, 20 01; Gitomer, Latham, & Ziomek, 1999; Lankford, Loeb, & Wycoff, 2002). This study addressed one of these factors—the effec tiveness of certification on student achievement. We found what might be expected of th ose who choose to do complex work, namely, that those who trained longer and harder to do that work do it better. Common sense and empirical data agree. Despite our lack of under standing of how it is accomplished, and despite the extreme variability in the programs of instruction (surely masking both excellent and dreadful programs), the present research study supp orts the assertion that university prepared teachers are of higher quality than those prepared without an approved program of preparation (see also Evertson, 1984; Darling-Hammond, 1997a). In this study regularly certified teachers signific antly outperformed under-certified teachers with children who are most at risk of school failure and school dropout. These already low achieving children, when assigned to the classrooms of under-certified teachers made gains that were approximately 2 months less per school year on three different subtests of the SAT 9. This is about 20% less academic growth than they would h ave made had they been assigned to a teacher with regular state certification. The Rowan et al. (in press) study, cited above, sta tes that the relationship between measures of student growth and measures of teacher competency a re much stronger than are the relationships found when a single years measure of achievement is used as the dependent variable, as in this study. Since the districts we studied had relatively large percentages of under-certified teachers the odds of a student gett ing more than one such teacher during their primary grades is high. If the magnitude of the ef fects on student achievement growth over


41 of 53time were as high as Rowan et al. believe, then it is likely that exposure to just two under-certified teachers would result in intractabl e deficits in academic growth in reading, mathematics and language. Although their research methods are hard to follow, Sanders and Rivers (1996) reach similar conclusions: the effect s of poor quality teachers are cumulative. In this era of accountability for schools and stude nts, low test performance can mean the loss of employment for teachers and administrators, while f or students, such results can lead to retention in grade or denial of a high school degre e. But there are school systems throughout the nation that make regular use of large numbers o f under-certified teachers and thus, through their hiring practices, virtually guarantee that th eir students will achieve relatively low levels of performance on norm-referenced standardized tests. Students, teachers and administrators will each be made to pay for a policy that assures less then desirable outcomes from the school system.This situation raises broad questions of policy, su ch as, what are the causes of, and who is accountable for, the placement of the under-certifi ed teachers in the classrooms of our most challenging students? Who should accept responsibi lity for an educational policy that appears harmful and that clearly handicaps students in the lower social classes? Will the school districts that make heavy use of under-certified teachers all violate the new federal guidelines, since under-certified teachers seem not to be highly qualified to teach? And if these dis tricts will not be in compliance with the new federal regulations b ecause they cannot attract qualified teachers to their classrooms what can they do differently to receive funding and change the working conditions so that they can attract and keep qualif ied teachers? Policy makers should take the results of this study seriously, perhaps also funding more research of this kind to ensure the validity of our findings. But meanwhile, on the basis of our findings, we see evidence of a harmful educational policy. We believe that those in authority need to attend to the legal and moral issues that a rise from our data. It appears that we are systematically providing an inferior education to t he children of the poor. They start with academic difficulties and then through the policies we adopt we handicap them 20% more per year when we assign them to classrooms staffed by u nder-certified teachers. The data we have collected also inform us that ther e is no difference between the performance of new teachers from Teach for America and that of all other under-certified teachers. On all tests, and in both years, the certified teachers ou t-performed the under-certified novice teachers from Teach for America. Our results contradict cla ims made by TFA advocates that the enthusiasm and subject-matter knowledge, as well as a general education in a prestigious university, prepare these recruits to teach adequat ely in America's classrooms. The TFA teachers are no better able to teach than any other under-prepared teacher. In general, research on Teach for America has been limited and the results are often contradictory (Darling-Hammond, 1994; 1997a; 2001; Stevens & Dial, 1993; Schorr, 1993; Kopp, 1994; Raymond, Fletcher, & Luque, 2001). Our findings do, however, directly contradict those reported by Raymond, Fletcher & Lu que (2001). We find no evidence to support their claim that TFA teachers produce a pos itive effect on their students' achievement relative to teachers recruited in other ways. In o ur view, the preponderance of the available literature raises serious concerns about the TFA pr ogram. Although new TFA teachers are required to take a six-week summer training program before their school year begins, and they receive support throughout the school year from TFA personnel, the performance of their students is indistinguishable from that of student' s taught by other under-certified teachers. More important for policy makers is that the level of performance of the students of the TFA


42 of 53teachers was lower than that of the students taught by equally inexperienced but fully certified teachers. That is the more important finding.TFA may be a meaningful way for young college gradu ates to make some money and take a few years out of the ordinary path their careers de mand. But they are hurting our young, vulnerable, inner-city students. (We expect that T FA teachers are faring no better in rural communities, but our data does not address that pop ulation.) Because an overwhelmingly high percent of the TFA students also leave the professi on after their two years of service, their hard earned teaching experience will never be put to use with future generations of students. While the TFA program appears to be a failure, it i s simply part of the larger pattern of failure that attends to the policy of hiring under-certifie d teachers. The policy of hiring under prepared teachers for the schools that serve America's poor looks like an act of class warfare, a concept that Americans find hard to accept. But states hav e adopted, or allowed policies to continue unchallenged, that prevent poor and rural American children from receiving the education they need for citizenship or to compete in the economy o f the 21st century. These are policies to be ashamed of and abandoned. We hope that the new fed eral legislation will change things, for if it is taken literally, we might eventually have hig hly qualified teachers for all of our nations' students to learn from. Note 1 This article is based on the first author's disser tation titled Teacher certification does matter: The effects of certification status on student achi evement completed Spring, 2002, in the College of Education, Arizona State University. Th e second author received partial funding for helping with this research from the Rockefeller Fou ndation, to whom we are grateful. The views expressed in this report, however, are the so le responsibility of its authors and may not reflect the views of The Rockefeller Foundation or the Arizona Department of Education.ReferencesAngus, D. L. (2001). Professionalism and the public good: A brief histor y of teacher certification Washington, DC: Fordham Foundaton. Retrieved Ju ly 25, 2002 from l Arizona Department of Education Web Page. (2002). A vailable: Ashton, P., & Crocker, L. (1987). Systematic study of planned variations: The essential focus of teacher education reform. Journal of Teacher Education, 38 (1), 2-8. Ashton, P., Crocker, L., & Olejnik, S. (1986). Does teacher education make a difference? A literature review and planning study Executive summary and technical monograph prepar ed for the Institute on Student Assessment and Evaluat ion, Florida Department of Education. Ballou, D. (1996). Do public schools hire the best applicants? Quarterly Journal of Economics, 3 (2), 97-134. Ballou, D., & Podgursky, M. (1999). Reforming teac her preparation and licensing: What is the evidence? Teachers College Record Retrieved November 1999 from Berry, B. (2001). No shortcuts to preparing good t eachers. Educational Leadership, 58 (8).


43 of 53Retrieved January 2002 from room/edlead/ 0105/berry.html. Bliss, T. (1992). Alternative certification in Con necticut: Reshaping the profession. Peabody Journal of Education, 67 (3), 35-54. Bradshaw, L., & Hawk, P. (1996). Teacher certification: Does it really make a differ ence in student achievement? Greenville, NC: Eastern North Carolina Consortium for Assistance and Research in Education.Buck, G. H., Polloway, E. A., & Robb, S. M. (1995). Alternative certification programs: A national survey. Teacher Education and Special Education, 18 (1), 39-48. Burbules, N. C., & Densmore, K. (1991). The limits of making teaching a profession. Educational Policy, 5 (1), 44-63. Byrne, C. J. (1983). Teacher knowledge and teacher effectiveness: A lite rature review, theoretical analysis and discussion of research str ategy Paper presented at the meeting of the Northeastern Educational Research Association, Elle nville, NY. Cochran-Smith, M. & Fries, M. K. (2001). Sticks, s tones and ideology: The discourse of reform in teacher education. Educational Researcher 30(8), 3-15. Crehan, K. D., Hess, R., Lawrence, A., & Sabers, D. (2000). Validity issues in teacher certification testing. Paper presented at the annual meeting of Arizona E ducational Research Organization, Tucson Arizona.Darling-Hammond, L. (1992). Teaching and knowledge: Policy issues posed by alternate certification for teachers. Peabody Journal of Education, 67 (3), 123-154. Darling-Hammond, L. (1994). Who will speak for the children? How "Teach for America" hurts urban schools and students. Phi Delta Kappan, 76 (1), 21-34. Darling-Hammond, L. (1997a). The right to learn: A blueprint for creating school s that work San Francisco, CA. Jossey-Bass.Darling-Hammond, L. (1997b). Doing what matters most: Investing in quality teach ing NY: National Commission on Teaching and America's Futur e, Teachers College, Columbia University. Retrieved July 27, 2002 from Darling-Hammond, L. (2000). Teacher quality and stu dent achievement: A review of state policy evidence. Education Policy Analysis Archives, 8 (1). Retrieved January 2001 from Darling-Hammond, L. (2001). The research and rhetoric on teacher certification: A response to "Teacher Certification Reconsidered." NY: National Commission on Teaching and America's Future, Teachers College, Columbia Univer sity Retrieved July 26, 2002 from Darling-Hammond, L. (2002). The research and rhetor ic on teacher certification: A response to "Teacher Certification Reconsidered." Educational Policy Analysis Archives, 10 (36). Retreived September 5, 2002 from n36.html


44 of 53Darling-Hammond, L., Berry, B. & Thoreson, A. (2001 ). Does teacher certification matter? Evaluating the evidence. Education Evaluation and Policy Analysis, 23 (1), 57-77. Darling-Hammond, L., Chung, R., & Frelow, F. (2002) Variation in teacher preparation: How well do different pathways prepare teachers to teac h? Journal of Teacher Education, 53 (4), 286-302.Eifler, K., & Potthoff, D. (1998). Nontraditional teacher education students: A synthesis of the literature. Journal of Teacher Education, 49 (3), 187-195. Educational Testing Service (2002). PRAXIS information web page Retrieved July 25, 2002 from prxtest.html Ehrenberg, R. G., & Brewer, D. J. (1994). Do schoo l and teacher characteristics matter? Evidence from High School and Beyond. Economics of Educational Review, 13 (1), 1-17. Evertson, C. (1984). The characteristics of effective preparation progra ms: A review of research Washington, DC: National Commission on Excellenc e in Teacher Education. Everston, C., Hawley, W., & Zlotnik, M. (1985). Ma king a difference in educational quality through teacher education. Journal of Teacher Education, 36 (3), 2-12. Feistritzer, E. (1993). National overview of alter native teacher certification. Education and Urban Society, 26 (1), 18-28. Feistritzer, E. (1994). Evolution of alternative t eacher certification. Educational Forum, 58 132-138.Fenstermacher, G. (1992). The place of alternative certification in the education of teachers. Peabody Journal of Education, 67 (3), 155-185. Ferguson, R. F. (1991). Paying for public educatio n: New evidence on how and why money matters. Harvard Journal on Legislation, 28 (2), 465-498. Ferguson, R. F., & Womack, S. T. (1993). The impac t of subject matter and education coursework on teaching performance. Journal of Teacher Education, 44 (1), 55-63. Fetler, M. (1999). High school staff characteristi cs and mathematics test results. Education Policy Analysis Archives, 7 (9). Retrieved January 2000 from Finn, C., & Kanstroom, M. (2000). Improving, empow ering, dismantling. The Public Interest Manhattan Institute: Authors. Retrieved December 2 001 from ing.html. Galambos, E. C. (1984). Testing teachers for certification and recertificat ion Washington, DC: National Commission on Excellence in Teacher Ed ucation. Gitomer, D., Latham, A. S., & Ziomek, R. (1999). The academic quality of prospective teachers: The impact of admissions and licensure te sting Princeton, NJ: Teaching and Learning Division, Educational Testing Service. Re trieved July 27, 2002, from


45 of 53 l#impact Glass, G.V (1978). Standards and criteria. Journal of Educational Measurement, 15 (4), 237-61.Glass, G.V (2002). Teacher characteristics. In A. Molnar (Ed.), School Reform Proposals: the Research Evidence (Chapter 8). Retrieved March 2002 from Go, K. (2002, July 31). 1 out of 6 teachers in Ariz ona lacks certification. Arizona Republic Retrieved August 2 from ergency31.html Goldhaber, D. D., & Brewer, D. J. (1996). Evaluating the effect of teacher degree level Rockville, MD: Westat, Inc.Goldhaber, D. D. & Brewer, D. J. (2000). Does cert ification matter? High school teacher certification status and student achievement. Education Evaluation and Policy Analysis, 22 (2), 129-145.Greenwald, R., Hedges, L. V., & Laine, R. D. (1996) The effect of school resources on student achievement. Review of Educational Research, 66 361-396. Hanushek, E. A., Kain, J. F., & Rivkin, S. G. (2001 November). Why public schools lose teachers (Working Paper 8599). Washington, DC: National Bu reau of Economic Research. Harcourt Brace Educational Measurement. (1997). Stanford Achievement Test, Ninth Edition Spring norms book San Antonio, TX: Author. Harcourt Brace Educational Measurement. (2001). Stanford Achievement Test, Ninth Edition Administrator's interpretive guide: Arizona student achievement program San Antonio, TX: Author.Hardy, L. (1998). A good teacher is hard to find. American School Board Journal, 185 (9), 20-23.Hawk, P., Coble, C. R., & Swanson, M. (1985). Cert ification: It does matter. Journal of Teacher Education, 36 (3), 13-15. Hawkins, E. F., Stancavage, F. B., & Dorsey, J. A. (1998). School policies affecting instruction in mathematics Washington, D.C.: National Center for Education S tatistics. Hawley, W. (1992). The theory and practice of alte rnative certification: Implications for the improvement of teaching. Peabody Journal of Education, 67 (3), 3-34. Haycock, K. (2001). Closing the achievement gap. Educational Leadership, 58 (6). Retrieved February, 2002, from .html Hernandez, R. (1998, December 4). Suit says minori ty students receive an inferior education. The New York Times Retrieved, May 2002, from Higher Education Reauthorization Act. (1998). Pub. L. No. 105-244, 20 U.S.C. § 1021-1030. Hirsh, E., Koppich, J. E., & Knapp, M. S. (2001). Revisiting what states are doing to improve


46 of 53the quality of teaching: An update on patterns and trends Seattle: University of Washington, Center for the Study of Teaching and Policy.Holmes Group. (1986). Tomorrow's teachers: A report of the Holmes Group East Lansing, MI: Author.Houston, R. W., Marshall, F., & McDavid, T. (1993). Problems of traditionally prepared and alternatively certified first-year teachers. Education and Urban Society, 26 (1), 78-89. Howey, K., & Zimpher, N. (1994). Nontraditional co ntexts for learning to teach. The Educational Forum, 58 156-161. Hundreds of teachers in city lack qualifications. ( 2002, January 31) Philadelphia Inquirer Retrieved July 26, 2002 from rticle2 Ingersoll, R. (1996). Teacher quality and inequality Paper presented at the annual meeting of the American Statistical Association, Chicago, IL.Ingersoll, R. (1997). Teacher turnover and teacher quality: The recurring myth of teacher shortages. Teachers College Record, 91 (1), 41-44. Ingersoll, R. (1999a). Invited commentary: Underst anding the problem of teacher quality in American schools. Education Statistics Quarterly : National Center for Education Statistics. Retrieved October 2001 from e/2-esq11-c.html. Ingersoll, R. (1999b). The problem of under-qualif ied teachers in American secondary schools. Educational Researcher, 28 (2), 26-37. Ingersoll, R. (2001). Teacher turnover and teacher shortages: An organizational analysis. American Educational Research Journal, 38 (3), 499-534. Jaeger, R. (1990). Establishing standards for teac her certification tests. Educational Measurement: Issues and Practice, 9 (4), 15-20. Jelmberg, J. (1996). College-based teacher educati on versus state-sponsored alternative programs. Journal of Teacher Education, 47 (1), 60-66. Kain, J. F. (1998). Impact of individual teachers and peers on individu al student achievement Paper prepared for the annual meeting of the Associ ation for Public Policy Analysis and Management, New York, NY.Kanstroom, M., & Finn, C. (Eds.)(1999). Better teachers better schools Washington, DC: Education Leaders Council, Thomas B. Fordham Founda tion. Retrieved July 26, 2002, from Kearns, J. (1984). State competency testing for te acher certification. Journal of Teacher Education, 35 (2), 58-61. Keltner, D. (1994). Troops to teachers: Alternativ e certification and the military. The Educational Forum, 58 182-186.


47 of 53Kennedy, M. (1991). Policy issues in teacher educa tion. Phi Delta Kappan, 72 (9), 659-665. Kopp, W. (1994). Teach for America: Moving beyond the debate. The Educational Forum, 58 187-192. Kwiatkowski, M. K. (1999). Debating alternative teacher certification: A trial by achievement. In M. Kanstroom & C. Finn (Eds.)(1999 ). Better teachers better schools (pp. 215-237). Washington, DC: Education Leaders Counc il, Thomas B. Fordham Foundation. Retrieved July 26, 2002, from Labaree, D. (1992). Power, knowledge, and the rati onalization of teaching: A genealogy of the movement to professionalize teaching. Harvard Educational Review, 62 (2), 123-154. Lankford, H., Loeb, S., & Wycoff, J. (2002). Teach er sorting and the plight of urban schools: A descriptive analysis. Educational Evaluation and Policy Analysis 24 (1), 37-62. Lopez, O. S. (1995). Classroom diversification: An alternative paradigm for research in educational productivity Unpublished doctoral dissertation, University of Texas, Austin. Mathews, J. (2002, July 18). Teacher training: Too much or not enough? Washington Post Retrieved July 20 from 1-2002jul18.html MacDonald, R., Manning, M. L., & Gable, R. (1994). An alternative certification program: Career transition. Action in Teacher Education, 16 (2), 19-27. McDiarmid, G.W., & Wilson, S. (1991). An exploratio n of the subject matter knowledge of alternative route teachers: Can we assume they know their subject? Journal of Teacher Education, 42 (2), 93-103. McKibbin, M. (1988). Alternative teacher certifica tion programs. Educational Leadership, 46 (3), 32-35. McKibbin, M., & Ray, L. (1994). A guide for altern ative certification program improvement. The Educational Forum, 58 201-208. Miles-Nixon, C., & Holloway, P. (1997). Alternativ e certification: The good, the bad and the ugly. Delta Kappa Gamma Bulletin, 63 (4), 40-47. Miller, J. W., McKenna, M. C., & McKenna, B. A. (19 98). A comparison of alternatively and traditionally prepared teachers. Journal of Teacher Education, 49 (3), 165-176. Monk, D. (1994). Subject area preparation of secon dary mathematics and science teachers and student achievement. Economics of Educational Review, 12 (2), 125-142. Monk, D., & King, J. (1994). Multi-level teacher r esource effects on pupil performance in secondary mathematics and science. In R.G. Eherenb erg (Ed.), Choices and Consequences (pp. 29-58) Ithaca, NY: ILR Press. Murnane, R. J. (1983). Understanding the sources o f teaching competence: Choices, skills and the limits of training. Teachers College Record, 83 (3), 564-569.


48 of 53Murnane, R. J. (1991). The case for performance ba sed licensing. Phi Delta Kappan, 73 (2), 137-142.Mitchell, K. J., Robinson, D. Z., Plake, B. S., & K nowles K. T. (Eds). (2001). Testing teacher candidates: The role of licensure tests in improvin g teacher quality National Research Council, Washington, D.C: National Academy Press. Natriello, G., & Zumwalt, K. (1993). New teachers for urban schools? The contribution of the provisional teacher program in New Jersey. Education and Urban Society, 26 (1), 49-62. No Child Left Behind (NCLB). (2001). Conference rep ort to accompany H.R. Rep. No. 107-334, 107th Cong., 1st sess. (2001).Olson, L. (2000). Finding and keeping competent te achers. Education Week, 19 (18), 12-18. Pearce, K. (2000, August 18). Substitutes can teac h without college degree. The Arizona Republic pp. B1, B2. Peck, H. (1989). The effect of certification status on the performan ce of mathematics teachers: A pilot study Paper presented at the Annual Meeting of the Ame rican Educational Research Association, San Francisco, CA.Pratte, R. & Rury, J. L. (1991). Teachers, profess ionalism, and craft. Teachers College Record, 93 (1), 59-72. Pyburn, K. (1990). Legal challenges to licensing e xaminations. Educational Measurement: Issues and Practice, 9 (4), 5-6, 14. Quality counts 2000: Who should teach? The states d ecide. (2000). Education Week, 19 (18). Quirk, T. J., Witten, B. J., & Weinberg, S. F. (197 3). Review of studies of the concurrent and predictive validity of the National Teacher Examina tions. Review of Educational Research, 43 (1), 89-113. Raymond, M., Fletcher, S., & Luque, J. (2001) Teach For America: An evaluation of teacher differences and student outcomes in Houston, Texas Palo Alto, CA: CREDO, The Hoover Institution, Stanford University. Retrieved Septe mber 1, 2002 from, S. G., Hanushek, E. A., & Kain, J. F. (2000 ). Teachers, schools and academic achievement Working Paper. Retrieved March 2001 from Roth, R. (1994). The university can't train teache rs? Transformation of a profession. Journal of Teacher Education, 45 (4), 261-268. Rowan, B., Correntti, R., & Miller, R. J. (in press ). What large-scale, survey research tells us about teacher effects on student achievement: insig hts from the Prospects study of elementary schools. Teachers College Record Sanders, W. L. (1997). Teacher and classroom context effects on student ac hievement:


49 of 53Implications for teacher evaluation Retrieved March 2001 from Sanders, W. L., & Rivers, J. C. (1996). Cumulative and residual effects of teachers on futu re student academic achievement Knoxville, TN: University of Tennessee Value-Add ed Research and Assessment Center.Schaeffer, B. (1996). Standardized tests and teacher competence Fair Test. Retrieved January 2000 from Schorr, J. (1993). Class Action. Phi Delta Kappan, 7 (4), 315-318. Schalock, D. (1979). Research on teacher selection In D. C. Berliner (Ed.), Review of research in education (Vol. 7) Washington, D.C: American Educational Research Association. Scores needed to pass Minnesota teacher tests among lowest in U. S. (2002, July 21). Minneapolis Star Tribune Retrieved July 26, 2002 from Shimberg, B. (1990). Social considerations in the validation of licensing and certification exams. Educational Measurement: Issues and Practice, 9 (4), 11-14. Smith, I. L., & Hambleton, R. K. (1990). Content v alidity studies of licensing examinations. Educational Measurement: Issues and Practice, 9 (4), 7-10. Smith, J. (1991). The alternate route: Flaws in th e New Jersey plan. Educational Leadership, 49 32-36. Soar, R. S., Medley, D. M., & Coker, H. (1983). Te acher evaluation: A critique of currently used methods. Phi Delta Kappan, 64 (4), 239-246. Sowell, T. (2002, July 31). Unsettling school data Washington Times. Retrieved on August 2, 2002, from Stevens, C. J., & Dial, M. (1993). A qualitative s tudy of alternatively certified teachers. Education and Urban Society, 26 (1), 63-77. Stoddart, T. (1992). Los Angeles Unified School Di strict intern program: Recruiting and preparing teachers for the urban context. Peabody Journal of Education, 26 29-48. Stoddart, T. (1993). Who is prepared to teach in u rban schools? Education and Urban Society, 26 (1), 29-48. Strauss, R. P., & Sawyer, E. A. (1986). Some new e vidence on teacher and student competencies. Economics of Education Review, 5 (1), 41-48. Students of certified teachers more likely to pass skills test (1999, April 12). Abilene Reporter Retrieved on July 26, 2002 from l Summers, A., & Wolfe, B. (1975). Which school resources help learning? Efficiency an d equity in Philadelphia public schools Business Review. Federal Bank of Philadelphia, P A.


50 of 53 U.S. Department of Education, National Center for E ducation Statistics. (2001). Education finance statistics center peer search detail Washington, DC: Author. Retrieved August 2001 from U.S. Department of Education, National Center for E ducation Statistics. (1999). (Lewis, L., Parsad, B., Carey, N., Bartfai, N., Farris, E., & S merdon, B.) Teacher quality: A report on the preparation of public school teachers Washington, DC: Author. Wayne, A. (2000). Teacher supply and demand: Surpr ises from primary research. Educational Policy Analysis Archives, 8 (47). Retrieved December 2000 from Wenglinsky, H. (2002). How schools matter: The link between teacher classroom practices and student academic performance. Education Policy Analysis Archives, 10 (12). Retrieved March 2002 from Wilson, S. M., Floden, R. E., & Ferrini-Mundy, J. ( 2002). Teacher preparation research: An insider's view from the outside. Journal of Teacher Education 53(3), 190-204. Wise, A. (1994a). The coming revolution in teacher licensure: Redefining teacher preparation. Action in Teacher Education, 16 (2), 1-13. Wise, A. (1994b). Choosing between professionalism and amateurism. The Educational Forum, 58 139-146. Wong, E. (2000, August 13). Poorest schools lack t eachers and computers. The New York Times Retrieved August, 2000, from Wyatt, E. (2000, July 14). Regents support plan to close some Roosevelt schools if standards are not met. The New York Times Retrieved July, 2000 from Zumwalt, K. (1991). Alternative routes to teaching : Three alternative approaches. Journal of Teacher Education, 42 (2), 83-92 AppendixIn HTML In Excell Spreadsheet About the AuthorsIldiko Laczko-Kerr Arizona Department of EducationPhoenix, AZIldiko Laczko-Kerr is an Education Research Associa te in the Research and Policy Unit at the Arizona Department of Education. She received her P h.D. in Educational Psychology in 2002


51 of 53 from the College of Education at Arizona State Univ ersity. She holds a BA in Psychology and a MA in Educational Psychology from the University of Arizona. Her current research at the department of education involves working with state assessment data, the development and implementation of the state’s achievement profiles, and research relating to Arizona’s teaching force. She can be reached by email at ilaczko@ade.a David C. BerlinerRegents' Professor of EducationCollege of EducationArizona State UniversityTempe, AZ 85287-2411Email: berliner@asu.eduDavid C. Berliner is Regents' Professor of Educatio n at the College of Education of Arizona State University, in Tempe, AZ. He received his Ph. D. in 1968 from Stanford University in educational psychology, and has hled positions at t he University of Massachusetts, WestEd, and the University of Arizona. He has served as preside nt of the American Educational Research Association (AERA), president of the Division of Ed ucational Psychology of the American Psychological Association (APA), and as a Fellow of the Center for Advanced Study in the Behavioral Sciences. Berliner is a member of the Na tional Academy of Education. His publications include The Manufactured Crisis Addison-Wesley, 1995 (with B.J. Biddle) and the Handbook of Educational Psychology Macmillan, 1996 (Edited with R.C. Calfee). Specia l awards include the Research into Practice Award and the lifetime achievement award from AERA, the E. L. Thorndike award from APA, and the 2 003 Brock international award for educational achievements. His scholarly interests i nclude research on teaching and education policy analysis.Copyright 2002 by the Education Policy Analysis ArchivesThe World Wide Web address for the Education Policy Analysis Archives is General questions about appropriateness of topics o r particular articles may be addressed to the Editor, Gene V Glass, or reach him at College of Education, Arizona State University, Tempe, AZ 8 5287-2411. The Commentary Editor is Casey D. Cobb: .EPAA Editorial Board Michael W. Apple University of Wisconsin Greg Camilli Rutgers University John Covaleskie Northern Michigan University Alan Davis University of Colorado, Denver Sherman Dorn University of South Florida Mark E. Fetler California Commission on Teacher Credentialing Richard Garlikov Thomas F. Green Syracuse University


52 of 53 Alison I. Griffith York University Arlen Gullickson Western Michigan University Ernest R. House University of Colorado Aimee Howley Ohio University Craig B. Howley Appalachia Educational Laboratory William Hunter University of Ontario Institute of Technology Daniel Kalls Ume University Benjamin Levin University of Manitoba Thomas Mauhs-Pugh Green Mountain College Dewayne Matthews Education Commission of the States William McInerney Purdue University Mary McKeown-Moak MGT of America (Austin, TX) Les McLean University of Toronto Susan Bobbitt Nolen University of Washington Anne L. Pemberton Hugh G. Petrie SUNY Buffalo Richard C. Richardson New York University Anthony G. Rud Jr. Purdue University Dennis Sayers California State University—Stanislaus Jay D. Scribner University of Texas at Austin Michael Scriven Robert E. Stake University of Illinois—UC Robert Stonehill U.S. Department of Education David D. Williams Brigham Young University EPAA Spanish Language Editorial BoardAssociate Editor for Spanish Language Roberto Rodrguez Gmez Universidad Nacional Autnoma de Mxico Adrin Acosta (Mxico) Universidad de J. Flix Angulo Rasco (Spain) Universidad de Teresa Bracho (Mxico) Centro de Investigacin y DocenciaEconmica-CIDEbracho Alejandro Canales (Mxico) Universidad Nacional Autnoma Ursula Casanova (U.S.A.) Arizona State Jos Contreras Domingo Universitat de Barcelona Erwin Epstein (U.S.A.) Loyola University of Chicago Josu Gonzlez (U.S.A.) Arizona State University


53 of 53 Rollin Kent (Mxico)Departamento de InvestigacinEducativa-DIE/ Mara Beatriz Luce (Brazil)Universidad Federal de Rio Grande do Sul-UFRGSlucemb@orion.ufrgs.brJavier Mendoza Rojas (Mxico)Universidad Nacional Autnoma deMxicojaviermr@servidor.unam.mxMarcela Mollis (Argentina)Universidad de Buenos Humberto Muoz Garca (Mxico) Universidad Nacional Autnoma deMxicohumberto@servidor.unam.mxAngel Ignacio Prez Gmez (Spain)Universidad de Daniel Schugurensky (Argentina-Canad)OISE/UT, Simon Schwartzman (Brazil)American Institutes for Resesarch–Brazil(AIRBrasil) Jurjo Torres Santom (Spain)Universidad de A Carlos Alberto Torres (U.S.A.)University of California, Los