USF Libraries
USF Digital Collections

Educational policy analysis archives

MISSING IMAGE

Material Information

Title:
Educational policy analysis archives
Physical Description:
Serial
Language:
English
Creator:
Arizona State University
University of South Florida
Publisher:
Arizona State University
University of South Florida.
Place of Publication:
Tempe, Ariz
Tampa, Fla
Publication Date:

Subjects

Subjects / Keywords:
Education -- Research -- Periodicals   ( lcsh )
Genre:
non-fiction   ( marcgt )
serial   ( sobekcm )

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
usfldc doi - E11-00121
usfldc handle - e11.121
System ID:
SFS0024511:00121


This item is only available as the following downloads:


Full Text
xml version 1.0 encoding UTF-8 standalone no
mods:mods xmlns:mods http:www.loc.govmodsv3 xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govmodsv3mods-3-1.xsd
mods:relatedItem type host
mods:identifier issn 1068-2341mods:part
mods:detail volume mods:number 7issue 9series Year mods:caption 19991999Month March3Day 2626mods:originInfo mods:dateIssued iso8601 1999-03-26



PAGE 1

1 of 23 Education Policy Analysis Archives Volume 7 Number 9March 26, 1999ISSN 1068-2341 A peer-reviewed scholarly electronic journal Editor: Gene V Glass, College of Education Arizona State University Copyright 1999, the EDUCATION POLICY ANALYSIS ARCHIVES. Permission is hereby granted to copy any article if EPAA is credited and copies are not sold. Articles appearing in EPAA are abstracted in the Current Index to Journals in Education by the ERIC Clearinghouse on Assessment and Evaluation and are permanently archived in Resources in Education High School Staff Characteristics and Mathematics T est Results Mark Fetler California Department of EducationAbstractThis study investigates the relationship between me asures of mathematics teacher skill and student achievement i n California high schools. Test scores are analyzed in relation to te acher experience and education and student demographics. The results are consistent with the hypotheses that there is a shortage of qualified ma thematics teachers in California and that this shortage is associated wit h low student scores in mathematics. After controlling for poverty, teacher experience and preparation significantly predict test scores. Shor t-term strategies to increase the supply of qualified mathematics teache rs could include staff development, and recruitment incentives. A long-ter m strategy addressing root causes of the shortage requires mor e emphasis on mathematics in high school and undergraduate progra ms.Introduction Debate on how best to teach mathematics has a long history, dating at least to Plato's Greece and the methods illustrated in the M eno. That dialogue presented mathematics instruction as an instance of a general method of teaching based on inquiry. More recent authors frame education as a system, de scribed by indicators of instructional context, processes, and outcomes. (Levin, 1974; Mur nane, 1987; Office of Educational

PAGE 2

2 of 23Research and Improvement, 1988; Shavelson, McDonnel l and Oakes, 1989; and Porter, 1991) Martin (1996) describes an influential framew ork for the study of mathematics achievement adopted by The Third International Math ematics and Science Study (TIMSS). The TIMSS framework focused on an explanat ory system with three factors: what is intended to be taught; what teachers actual ly do in the classroom; and what students learn. Curriculum standards and frameworks—"what i s intended to be taught"—are teaching tools that can help students to learn, dep ending on the skill of the teacher. Perhaps because direct measures of teaching skill a re difficult to define and obtain, researchers and policymakers use teacher education and experience as plausible proxy measures. Individuals with the same amount of exper ience and similar teaching credentials can vary in actual skill. Even so, in a n aggregate consisting of many teachers in many schools, it is not unreasonable to believe that more highly educated and experienced teachers possess greater skill. Additio nally, it is likely that the presence in schools of more educated and experienced teachers i s associated with better student achievement. The departmentalized instruction found in m ost high schools offers an opportunity to study the linkage between teaching and learning of specific subjects, such as mathematics. Secondary school teachers typically po ssess single-subject credentials that require proof of specialized subject matter knowled ge, and authorize teaching in specific areas. The possession of a valid credential and aut horization to teach mathematics is one indicator of teacher education and experience. By c ontrast, at the elementary level, teachers possess multiple-subject credentials. At t he elementary level there is no specific indicator of teacher skill in mathematics instructi on.Mathematics Curriculum and Achievement Official statements of high school course r equirements and curriculum standards permit an inference about the importance of high sc hool mathematics. A national survey of states, conducted by the Council of Chief State School Officers (1998) found that 23 states require more than two credits in math, compa red to 13 states in 1989. Forty-two states, including California, have mathematics cont ent standards ready for implementation. Table 1 summarizes the minimum acad emic coursework expected from California students during a traditional four-year high school career. The California State University System and the University of California publish minimum subject requirements for freshman admission, including elec tives. Based on these recommendations, mathematics should be considered s econd only to English, and should occupy 20 percent or more of a student's curriculum Table1 California High School Core Academic Course Requirements High School Graduation CaliforniaStateUniversity University of California Mathematics 233

PAGE 3

3 of 23English 344 Science 212 History/Social Science 312 Other/Electives 364 Total 131515 The National Assessment of Educational Prog ress (NAEP) (See http://nces.ed.gov/naep/ .) has demonstrated influential models of standards -based assessment, and has focused attention on student ac hievement in mathematics by providing state-by-state summaries of student perfo rmance. The NAEP 1992 and 1996 assessments (Reese, et al., 1997) used a framework related to the National Council of Teachers of Mathematics (NCTM) "Curriculum and Eval uation Standards for School Mathematics," originally published in 1989. (See http://www.nctm.org .) These standards include five mathematics strands: number sense, pro perties, and operations; measurement; geometry and spatial sense; data analy sis, statistics, and probability; and algebra and functions. In addition to the five stra nds, the NAEP assessment examined mathematical abilities (conceptual understanding, p rocedural knowledge, and problem solving) and mathematical power (reasoning, connect ions, and communication). Mathematical abilities relate to the knowledge or p rocesses involved in successfully handling tasks. Mathematical power refers to the ab ility to reason, to communicate, and to make connections of concepts and skills across s trands, or from mathematics to other areas. Table 2 displays percentages of students att aining mathematics achievement levels for California, the western region, and the nation. Although California's percentage is one point lower for the most advanced students in 1 996, the state is ten points lower for students at or above a basic level. Table 2 NAEP Grade 8 Percentages of Students at Achievement Levels At or AboveAdvanced At or AboveProficient At or AboveBasic Below Basic1996 Nation 4236139 Western Region 3225941 California 3175149

PAGE 4

4 of 231992 Nation 3205644 Western Region 3215842 California 2165050 Teacher characteristics may explain some of the variation in NAEP scores. Hawkins (1998) used eighth grade data from the 1996 assessment to show that students taught by teachers with an undergraduate or graduat e major in mathematics scored higher than students taught by teachers with majors in education or some other field. Moreover, students taught by teachers with certific ates in mathematics outperformed students taught by teachers with certificates in ot her areas. Finally, students of teachers who rated themselves as knowledgeable or very knowl edgeable about the NCTM curriculum and evaluation standards scored higher t han students whose teachers reported little or no knowledge of the standards.Achievement of College Entrants The California Postsecondary Education Comm ission (1998) estimates that thirty percent of public high school graduates are eligibl e for freshman admission at the California State University (CSU). Admission requir ements in mathematics include three years of college preparatory coursework, norm ally Algebra I, Algebra II, and Geometry. CSU requires all entering freshmen to dem onstrate proficiency in mathematics, either by taking the Entry Level Mathe matics (ELM) examination or by presenting proof of adequate performance on an appr opriate Advanced Placement, SAT, or ACT mathematics test. Those who cannot demonstra te proficiency must take remedial courses. California public high schools pr oduced 269,071 graduates in 1997 and California State University enrolled 26,781 of them in fall 1998, for a college going rate of ten percent. Fifty-five percent of these fi rst time freshmen required remedial mathematics instruction. (See http://www.co.calstate.edu/asd/index.html .)Supply and Preparation of Teachers Supply and demand for mathematics faculty i s one part of a larger system that prepares and employs teachers. Growing student enro llment is driving the demand for qualified teachers. The National Center for Educati onal Statistics (NCES, 1996) estimates that total K-12 enrollment will grow abou t 10 percent from 49.8 million in 1994 to 54.6 million by 2006. California public sch ool enrollment will rise over 11 percent from 5.6 million students in 1997 to 6.2 mi llion ten years later. (California Department of Finance, 1998) California public scho ols employed 270,000 teachers in 1997. (California Department of Education, 1998). O ther factors remaining equal, the enrollment growth should create about 30,000 new te aching positions over the next decade. This growth, combined with turnover related to retirement and attrition, and with efforts reduce class size, have resulted in es timates of a need to hire more than 300,000 teachers in California over the next ten ye ars. Given that teaching skill is associated wit h student achievement, school districts and policymakers are interested in how teachers are prepared. (Darling-Hammond and Hudson, 1990; National Commission on Teaching and A merica's Future, 1996; Ashton,

PAGE 5

5 of 231996; Education Week, 1997) While teaching skill is a goal of preparation, usually a credential only requires an academic degree and cou rsework. Virtually all public school teachers in the United States have at least a bache lor's degree, and a majority possess an advanced degree. (NCES 1995a) The trend is toward h igher levels of education. In 1971, 28 percent of public school teachers possessed a ma ster's, specialist, or doctoral degree. Twenty years later 53 percent of teachers had an ad vanced degree. Although high demand for teachers is prompt ing reforms, California's degree and coursework requirements tend to resemble those of m any other states. (National Association of State Directors of Teacher Education and Certification, 1998). Historically, a California preliminary credential r equired a Bachelor's degree in a subject other than professional education, and a one-year p reparation program with training in educational principles and teaching strategies. Tho se seeking a clear credential fulfill additional course requirements and a year of educat ionally related study. Career changers with at least a Bachelor's degree and comp etence in their subject of instruction may work as paid teaching interns while they receiv e support and training in pedagogy from school districts or universities. Nationally, schools are filling an increasi ng proportion of vacancies with inexperienced applicants. (NCES 1995a) From 1988 to 1991 public schools hired more first-time teachers and fewer reentrants or transfe rs. Teachers who transfer from other schools or return to a school have more experience, but receive higher salaries than first-time teachers. First-time teachers earn less, but are more likely to leave the profession. Teacher retirement and migration into o ther occupations influence turnover in schools. (NCES 1995b) Nationwide, between 1990-9 1 and 199192 about 5 percent of teachers left teaching, including retirees. Teac hers with less full-time teaching experience were more likely to leave. Smaller schoo ls experience higher teacher attrition. Lower salaries and benefits may be a fac tor in this relationship. Small schools offer teachers less compensation than larger school s. School with more student poverty have higher turnover than other schools. Credential requirements restrict access to the teaching profession. One way to meet increased demand is to relax the requirements, redu cing the time and cost to become a teacher. For example, when there are too few fully qualified applicants, California school districts use emergency permits to hire indi viduals who lack some requirements for a credential, usually proof of competence in th eir subject(s) of instruction or pedagogy. (Hart and Burr, 1996) In recent years eme rgency permits have become more popular. A risk of this increased popularity is tha t less well prepared teachers may be less effective in their jobs or more prone to attri tion. States have sought to increase the supply o f teachers by setting up alternatives to traditional training programs. Zumwalt (1996) descr ibes alternative certification as easing entry requirements, minimizing preparation n eeded prior to paid teaching, and emphasizing on-the-job training. Proponents portray these programs as attracting higher-ability, more diverse, experienced people wi th subject matter majors. (Ashton, 1991; Dill, 1996; Feistritzer, 1994; Haberman, 1992 ) Zumwalt cautions that it is difficult to generalize about the success of alternative prog rams. Alternative approaches assume that school staffs have the resources to support un prepared novice teachers. The success of alternative approaches may actually depend on th e extent to which novice teachers actually receive needed support and obtain classroo m assignments appropriate to their abilities.Mathematics Teachers

PAGE 6

6 of 23 Nationwide, 90 percent of teachers in grade s 9 through 12 for whom mathematics is their main assignment report having a major or mino r in that subject. (NCES; 1998, 1999) Students at public secondary schools with a h igher poverty level or with a higher percentage minority enrollment were more likely to be taught any of the core subjects, including mathematics, by a teacher who had not maj ored in that subject. Brunsman (1997) describes California's requ irements for a single-subject credential in mathematics and the number of qualified mathemat ics teachers. In addition to the requirements that apply to all credentials, high sc hool mathematics teachers must demonstrate their competence in the subject either by completing a subject matter program, or they can demonstrate their competence t hrough an examination. Approved subject matter programs include a core with at least 30 semester units of mathematics coursework that is related to subjects that are commonly taught in departmentalized mathematics classes. This core inc ludes courses in first and second year algebra, geometry, first and second year calcu lus, number theory, mathematics systems, statistics and probability, discrete mathe matics, and the history of mathematics. Programs also include a minimum of 15 semester unit s of supplemental coursework to provide breadth. Optionally, a credential candidate can demo nstrate subject matter competence in mathematics through examination. California has ado pted a standardized subject matter test in mathematics that includes two hours of mult iple-choice questions and a two-hour performance assessment. Historically, less than hal f of the examinees pass the examination. School districts can assign less than fully qualified teachers to mathematics classes by several methods. An emergency permit requires a Bachelor's degree, passing a basic skills test, and completing a minimum of 18 semeste r hours or 9 upper division/graduate semester units of course work in mathematics. In or der to renew the permit, the teacher must complete six semester units toward earning a c redential in mathematics. A limited-assignment emergency permit requires that t he teacher have a valid teaching credential in another subject. A waiver requires on ly that the teacher pass or not ever have taken the mathematics portion of a basic skill s test. Table 3 shows the numbers of single subject credentials, emergency permits, and waivers issued 1993-94 to 1996-97 in mathematics. T he number of emergency/waiver teachers in mathematics far outpaces the number of fully qualified new teachers. Over the four year period California granted credentials to 2,689 fully qualified new mathematics teachers, and granted other permits or assignments to 6,339 less well-qualified teachers. Unfortunately, it is not k nown how many fully qualified teachers actually applied for and accepted jobs in public sc hools. Virtually every waiver and emergency permit represents an employed teacher. Th ese figures suggest that the supply of fully qualified teachers does not meet current d emand. There is a downward trend in the number of fully qualified teachers prepared and possibly hired, and an upward trend in the number of less than fully qualified people a ctually hired on waivers or permits. Table 3 First Time or New Type Single Subject Credentials, Emergency Permits, and Waivers in Mathematics

PAGE 7

7 of 23Credentials1993-41994-51995-61996-7Credentials Via Completed Program 470475431449 Credentials Via Passed Examination 278218242126 Total Credentials748693673575Total Emergency Permits and Waivers 1,4801,3801,4651,617Student Performance Some research suggests that teacher skills and ability influence student achievement. Greenwald, Hedges and Lane (1996) revi ewed a number of studies of the relationship between school inputs and student outc omes. Some school resources, i.e., teacher ability, teacher education, and teacher exp erience were strongly related to student achievement. On the other hand, Hanushek's (1996) synthesis of research studies found mixed support for a relationship between scho ol resources and achievement. Although Hanushek did not detect a clear pattern, m easures of teacher experience were more consistently related to achievement than measu res of teacher education. Ashton (1996) notes that teachers with regular state certi fication receive higher supervisor ratings and student achievement than teachers who d o not meet standards. Teachers without preparation have trouble anticipating and o vercoming barriers to student learning, and are likely to hold low expectations f or low-income children. Ashton suggests that reducing certification requirements a nd hiring of teachers who do not meet certification standards, worsens the quality of edu cation of low income children.Method The 795 regular California high schools in this study typically serve 1.3 million students per year. About 93 percent of regular high schools offer instruction in grades 9 through 12, although various other grade configurat ions are represented, most commonly 10-12, or 7-12. These schools reported employing 56 ,571 full-time equivalent (FTE) teachers in fall 1998, with 14.1 percent of the FTE dedicated to mathematics instruction. Approximately 600 non-traditional high schools serv ing about 100,000 students per year were excluded from the study. Generally, non-tradit ional schools have small enrollments and do not offer the academic curriculum needed to attend California's public universities. Reasons for referral to a non-traditi onal school could include an unstable home environment, emotional difficulties, pregnancy etc. Non-traditional schools diverge from regular schools in serving a populatio n of students with different needs and providing different kinds of services. The web site for California's Standardized Testing and Reporting (STAR) program ( http://star.cde.ca.gov/ ) provided school average mathematics achievement t est scores. The Stanford Achievement Test Series, Ninth Edition (Stanford 9), was administered to all students in grades 2 through 11 between March 1 5, 1998 and May 25, 1998. Obtaining direct measures of mathematics skill is p robably no easier than obtaining

PAGE 8

8 of 23direct measures of teaching skill. Multiple-choice test scores are commonly used as expedient indicators of student skill. The Stanford 9 high school mathematics tests require 45 minutes of examination time and include 48 questions. The content of the tests is oriented towards basic skills and is based on the NCTM framework. Scaled scores were derived using Item Response Theory Rasc h model techniques. (Harcourt Brace, 1997). Results for schools testing fewer tha n 10 students were not available. Overall, 2.5 percent of students were legal ly exempted either by parent request or by means of an Individual Education Plan or Section 504 Plan for students with disabilities. Possible effects of selective testing were examined with the help of an estimate of student participation in the assessment Grade level participation rates were estimated using fall 1998 grade enrollment as the d enominator and the number tested as the numerator. Given California's increasing enroll ment, this statistic slightly underestimates actual participation statewide. Teachers with instructional assignments in mathematics were identified from the results of the 1998 Professional Assignment Informa tion Form (PAIF), an annual survey conducted as a part of California Basic Educational Data System (CBEDS). The information requested on the PAIF is required of ea ch certificated staff person, and includes demographics, assignments, and position/cr edentials. The educational level of teachers with instructional assignments in mathemat ics was coded as: (1) Doctorate; (2) Master's degree plus 30 or more semester hours; (3) Master's degree; (4) Bachelor's degree plus 30 or more semester hours; (5) Bachelor 's degree; and (6) Less than Bachelor's degree. Very few teachers possess less t han a Bachelor's degree, so these individuals were aggregated with those who did poss ess the degree. Years of educational service included service in the current district, o ther states, and countries, but did not include substitute teaching. School summary statist ics for staff with mathematics assignments included the numbers with emergency per mits, teaching credentials, and mathematics authorizations. The percent of emergenc y permits was computed using the headcount of staff with one or more mathematics ass ignments as denominator and the number of staff with emergency permits as numerator AFDC is the percentage of students in the school's attendance area who are enrolled in either public or private schools and who are from f amilies receiving aid. As an indicator of poverty AFDC often correlates with student achie vement (White, 1982), and functions in this study as a control variable. The descriptive and correlational statistic s used in this study permit informed speculation about relationships among the phenomena measured by the study variables. Of course, these techniques by themselves do not ju stify conclusions regarding cause and effect.Results Table 4 displays the percent of teachers wi th one or more mathematics assignments by educational level and possession of an emergency permit. There is a discrepancy between the number of teachers with emergency permi ts reported by districts on the fall CBEDS census for regular high schools, and a larger number of permits actually issued by the California Commission on Teacher Credentiali ng (CTC). The CTC number reflects a yearcumulative total for all schools. Differences in the scope and method of data collection likely account for much of the disc repancy. Table 4

PAGE 9

9 of 23Educational Level and Emergency Permits of Mathematics Teachers DegreeEmergency Permit YesNo Ph.D. 1.3%1.4% MA+ 5.2%23.3% MA 7.7%16.6% BA+ 26.3%46.7% BA or Less 59.6%12.0% Total Percent 100.0%100.0% Total Head Count 1,0098,516 The results indicate that 10.5 percent of m athematics teachers in regular high schools have emergency permits. Although some emerg ency permit teachers have advanced degrees, a majority possessed only a bacca laureate degree. By contrast, the majority of mathematics teachers with credentials c ompleted post baccalaureate work, and about one-fourth of them completed work beyond the masters degree. Table 5 displays the percent of teachers wi th one or more mathematics assignments by number of years of service and authorization to teach mathematics. The distributions are bimodal with relatively higher percentages of m athematics teachers with five or less years of experience, consistent with the hypothesis that mathematics teachers tend leave the education profession after several years. At th e same time, more than half of all mathematics teachers report ten or more years of te aching experience. Results indicate that one-fourth of those who teach mathematics lack authorization. Although the data do not track teaching assignments over time, about 60 percent of those lacking authorization have ten or more years of experience. Table 5 Years of Service and Authorization of Mathematics Teachers Years of ServiceMathematics Authorization YesNo 0 0.7%2.3%

PAGE 10

10 of 231 7.6%9.8% 2 5.9%6.6% 3 5.2%4.5% 4 5.1%3.4% 5 5.0%4.0% 6 4.2%2.9% 7 3.5%2.5% 8 3.9%2.5% 9 3.9%2.1% 10+ 55.3%59.5% Total Percent 100.0%100.0% Total Head Count 7,2282,383 Table 6 displays mean test scores, proporti on of students participating in the assessment, AFDC, years of experience of mathematic s teachers, and education level of mathematics teachers. Grade-level mean test scores and participation rates were weighted by the number of students tested. AFDC and the teac her statistics reflect the entire school and were weighted by total school enrollment. An in creasing trend in test scores may indicate improvement in achievement from grade 9 to 11. However, the declining number of students enrolled in higher grades is consistent with student attrition from dropping out. Moreover, the decreasing trend in student part icipation is consistent with more selective testing in grade 11. Higher scores could be accounted for either by attrition or selective testing. Table 6 Means of Selected Variables SchoolGrade 9Grade 10Grade 11 Test Score n/a690697703 Participation n/a.86.85.81 AFDC 15.7%N/an/an/a

PAGE 11

11 of 23Years Teaching 14.4N/an/an/a Education Level 3.6N/an/an/a Number Tested n/a347,201313,303260,933 Number Enrolled n/a405,516370,080321,896 Number of Schools n/a785794789 Table 7 displays selected correlations of s chool mean scaled scores and other variables, weighted by the appropriate grade enroll ment. The results appear to be consistent across grades. All correlations are stat istically significant, (p < .001). Appendix A includes all pairwise correlations of study varia bles. The empirical correlations probably underestimate the relationships between th e study variables for several reasons. The measures of mathematics teacher characteristics are based on a relatively small proportion of all teachers at a school. The student outcome measure, a narrowly defined indicator of mathematics achievement, focuses on on e of many areas of study expected of students. More broadly defined measures could produ ce stronger correlations. Differences in the level of aggregation could also limit the correlations. AFDC and the teacher characteristics are school-wide measures. T est scores and student participation are grade specific. Greater consistency in aggregation, not possible with the available data, could also produce stronger correlations. The strong relationship often found between poverty and achievement is replicated in this study. AFDC correlates more strongly with t est scores than do the other study variables. Correlations with AFDC are largest for n inth grade test scores and smallest for eleventh grade. An investigation of this trend is b eyond the scope of this study. However, it is possible that lower achieving students are le ss likely to be tested in higher grades, possibly the result of attrition or selective testi ng. If true, the absence of lower achieving students may have resulted in a restriction of rang e of the variables and lower correlations. The positive relationship between student p articipation and test scores is counter-intuitive and seems inconsistent with the h ypothesis that lower participation rates are associated with widespread exclusion of lower a chieving students. However, school participation rates are negatively related to pover ty. Schools with more poverty tend to have lower participation rates and lower test score s. Schools with less poverty tend to have higher participation rates and higher scores. Of course, school characteristics other than poverty could be related to student participat ion. For example, participation rates might reflect administrative competence. Students a t better run schools might have better opportunities to learn and might be more likely to take and do well on tests. Teaching experience, measured by the averag e number of years in service, is positively related to test results. Schools with we ll-prepared teachers tend to have higher mathematics scores, whether preparation is measured as percent of mathematics teachers with emergency permits or as an education level ind ex. To some extent, the effect of teaching experience is mediated by poverty. That is schools with more poverty tend to have both less wellprepared teachers and lower te st scores. One way to assess the influence variables independently is to include all of them in a multiple regression

PAGE 12

12 of 23analysis. Table 7 Correlations for Selected Variables Variable9th Grade Test Score 10th Grade Test Score 11th Grade Test Score Percent AFDC -0.64-0.61-0.59 Percent Participation 0.450.480.35 Years Teaching 0.240.260.27 Education Level -0.24-0.23-0.22 Percent Emergencies -0.39-0.36-0.36 Table 8 displays the results of three multi ple regression analysis for grades 9, 10, and 11. Achievement test scores were the dependent variables and the analyses were weighted by the number of students tested. The raw weights reflect the variables as originally measured. The beta weights reflect the p redictors after scaling to standard deviation units and aid comparisons of the importan ce of predictors within and across grades. Table 8 Multiple Regression Analyses by Grade Level Grade 9 WeightsGrade 10 WeightsGrade 11 Weights RawBetaRawBetaRawBeta Intercept 671.10667.50686.30 AFDC -0.6-10.7-0.5-9.3-0.6-9.0 Participation 30.73.739.44.524.62.9 Years Teaching 0.31.70.42.10.52.4 Percent Emergencies -27.8-4.1-19.8-3.1-24.4-3.2

PAGE 13

13 of 23R-Square 0.500.470.44 The three multiple regressions yield simila r patterns of results. Student poverty, measured by AFDC, demonstrates the strongest relati onship with test scores. Student participation, following the pattern of related sim ple correlations, is positive related to test scores, even taking poverty into account. The percent of mathematics teachers on emergency permits predicted test scores about as we ll as student participation. Higher percents of emergencies were associated with lower scores. Finally, the average number of years of teaching experience was positively rela ted to scores. Schools with more experienced mathematics teachers tend to have highe r mathematics achievement. The values of R-square, a measure of how we ll a combination of the independent variables predicts test scores, appear to trend dow n as the grade levels increase. This downward trend parallels a similar downward trend i n the importance of AFDC as a predictor. One explanation for the trend could be i ncreasing homogeneity of students at higher grade levels. As more disadvantaged students either drop out or find placement in alternative schools, those remaining in regular hig h schools become more similar socially and demographically. If this hypothesis is true, it could account for some of the increase in test scores in higher grades.Discussion The results of this study are consistent wi th the hypothesis that there is a shortage of qualified mathematics teachers and that this shorta ge is associated with weak student achievement in mathematics. Student poverty strongl y predicts mathematics achievement in this study, as in many others. After factoring o ut the effects of poverty, teacher experience and preparation are significantly relate d to achievement. Several California state policies communica te the importance of learning mathematics. Long standing high school graduation c ourse requirements oblige students to commit a significant amount of time to mathemati cs instruction. Similar course requirements for college entrance reinforce the mes sage. A state curriculum framework for mathematics appeared in 1985, and the state col leges and universities published a statement of desired competencies in 1982. More rec ently, the State Board has adopted mathematics curriculum standards for what students are expected to know and be able to do at each grade level. Finally, the current and pa st statewide assessment programs include mathematics tests. Historically, California policymakers and educators have consistently proclaimed the importance of teaching and learning mathematics. To what extent has the setting of priorities and goals resu lted in desired student outcomes? There are several indications that high sch ool student performance in mathematics does not rise to expectations, for those who are co llege bound or for others. The 1992 and 1996 NAEP mathematics results are troubling for sev eral reasons. In general, relatively large percentages of students exhibit "below basic" skill levels. Compared to the nation, California has lower percentages of students that a re "at or above basic." The NAEP results are consistent the 1998 findings from the S TAR assessment program that suggest lagging performance of California high school stude nts on the basis of national norms. Additionally, the California NAEP results do not fo llow improvements nationwide from 1992 to 1996. Another negative indicator is the fin ding that over half of 1998 first time freshman at California State University required re medial classes in mathematics. One explanation for the lower than desired results in mathematics relates to student

PAGE 14

14 of 23demographics. Traditionally, student poverty correl ates with low achievement. Possibly, disadvantaged students enjoy less support for acade mic pursuits from their families and peers, and are more focused on meeting needs relate d to safety and survival. California has a growing number of students whose primary lang uage is not English. These students do not have the same degree of access to the curric ulum or assessment as native English speakers. On the other hand, many believe that teac hing and learning mathematics depends less on mastery of English than other subje cts. Although language skills are important for assessments of writing or reading com prehension, they probably play a lesser role in understanding mathematical notation, solving equations, etc. Of course, public schools do not control th e demographics of their students. Except for those students exhibiting serious disciplinary problems, a public school must serve all who live in its attendance area. There is little th at schools can do to change the social and economic circumstances of students. However, the la ck of power to alter demographics does not justify complacency towards the education of disadvantaged students. Leverage to improve student outcomes exists at other points in the educational system. Ideally, schools will provide a safe and positive learning e nvironment along with programs and resources to compensate for particular disadvantage s. Beyond such compensatory programs, outcomes for disadvantaged students likel y depend on sound curriculum standards and quality teaching. Despite the powerful effect of poverty, the experience and education of mathematics teachers predicts student achievement. Schools with more experienced and more highly educated mathematics teachers tended to have higher achieving students. Schools with higher percentages of teachers on emergency permits tended to have lower achieving students. Unfortunately, teacher credential informati on indicates a declining trend in the number of newly-prepared, fullyqualified, high sc hool mathematics teachers, and increases in the number of those who are teaching o ut of their area or on emergency permits. One reason advanced for these trends is th at college students with an interest in mathematics avoid teaching in favor of more lucrati ve career pathways found in science or engineering. There appears to be a shortage of m athematically able students to meet the overall demand in teaching and other profession s. Given growing K-12 enrollments, a strong policy commitment to learning mathematics, a nd likely growth in technical professions that compete with education, this short age is likely to persist and grow more severe. One way to mitigate the effects of this sho rtage would be to provide training in mathematics to teachers who lack subject matter pre paration. One difficulty with such staff development is that the amount of training ne eded to develop the necessary skills is likely to be great, and that limited staff time and resources will result in long, sustained periods of training. The challenge, in some cases, will be to provide the equivalent of an undergraduate minor spanning multiple courses over a period of years, to a teacher who is already employed full-time. Some under-prepared tea chers may not have taken the necessary courses in college because they lacked pr erequisite skills from high school. This in-service challenge will be difficult to reco ncile with the limited time and resources usually provided for staff development. Given the d ifficulties, it would be prudent to evaluate the effectiveness of such in-service progr ams, and to consider other ways of easing the shortage. Financial incentives might induce more peop le to take up teaching mathematics. There is abundant anecdotal evidence that higher st arting salaries in other fields have drawn people with technical skills away from teachi ng. One drawback of financial incentives is the potential for inequality and divi siveness that it might create in the

PAGE 15

15 of 23teaching profession. An additional issue is whether policy makers could make available sufficient additional funds for an incentive progra m effective enough to meet the needs of schools. An alternative long-term strategy to addres s a shortage would be to require higher levels of mathematical skills of all undergraduate students, possibly by increasing the rigor and number of required lower division mathema tics courses, and by requiring more upper division mathematics courses. The general edu cation breadth requirement at the California State University only calls for "a minim um of twelve semester units or eighteen quarter units into the physical universe a nd its life forms, with some immediate participation in laboratory activity, and into math ematical concepts and quantitative reasoning and their applications." (California Stat e University, 1993) This policy often translates into a requirement for one mathematics c ourse at specific state universities. The general education requirement for students who tran sfer from a community college only calls for three semester units in "mathematical con cepts and quantitative reasoning." Considering the weight given to mathematics in the CSU entrance requirements, the general education mathematics requirement appears i nconsequential. Although little has been published on general education requirements, C SU's policies probably resemble those of many other colleges and universities. Stre ngthening the mathematics requirements could increase the numbers of students who major or minor in the subject, and could help to meet the growing demand for such expertise in teaching and technical professions. A change in course requirements will face a number of challenges. Some believe that there has been a trend over the last several d ecades to weaken undergraduate mathematics requirements. One reason sometimes adva nced for this trend is that many entering freshmen are not prepared to handle colleg e mathematics. Increasing the rigor and number of required mathematics courses might ad versely impact student retention and degree attainment. There may also be difficulty in providing sufficient faculty and resources to support additional requirements in mat hematics. High school student ability in mathematics should be seen as one outcome of a larger system that includes both K-12 schools and h igher education. It would be unfortunate if weakened undergraduate requirements are related to poor high school preparation. This pattern could be a symptom of a d ownward spiral in mathematics literacy in the population. As collegiate requireme nts are weakened, resources for undergraduate mathematics programs lessen, the math ematics skills of teachers decrease, and the students of these teachers are less well pr epared. Expectations of faculty and administrators in high school and college could dri ft lower, making it more difficult to provide the resources and leadership needed to crea te and implement high standards. In the short run a pattern of low expectations and low performance is the path of least resistance. Rigorous mathematics courses are not po pular with students and are unrewarding for faculty. Easing the requirements pr ovides short term relief. In the long run the path of least resistance results in lowered student ability and decreased capacity to make improvements. Public discontent with school pe rformance will grow unless teaching and learning improve. Schools and teacher preparation programs ne ed to coordinate their programs more closely in preparing, recruiting, and hiring teache rs. One basis for cooperation would be to set policy goals at both the state and local lev els to eliminate the use of less than fully qualified teachers within a given time frame, for e xample, within five years. At the state level it would be useful to reduce the options for hiring less than fully qualified teachers and simplify the procedures for obtaining an author ization to teach mathematics. Subject matter preparation programs are approved partly on the basis of course titles and

PAGE 16

16 of 23descriptions. Unfortunately, titles and description s permit considerable latitude in the rigor of such programs and there is little assuranc e as to the skills that prospective teachers actually develop. Given the apparent short age of mathematically inclined undergraduates there may be an incentive to lessen the rigor of preparation programs in order to keep the "pipeline" full and meet school d istrict demands. One way to cope with variation in rigor is to establish a uniform assess ment of subject matter knowledge needed to teach high school mathematics. The effectiveness of setting goals is reduc ed and the implementation of well-designed programs is undermined without timely and accurate data that describe how faithfully the programs are implemented and the extent to which outcomes are attained. In particular, monitoring of the supply a nd demand of teachers is severely hampered when information about credentials, teachi ng assignments, and employment is scattered across separate agencies or administrativ e units and is not easily linked. Although one state agency tracks credentials, it do es not know how many credential holders are employed in public schools, or elsewher e. Another state agency conducts an annual staff census of teaching assignments, but la cks detailed information about credential status and does not track employment of individuals across time or schools. The agency responsible for teacher retirement maint ains some employment history information, but does not follow credentials or ass ignments. Finally, student outcome data is not readily associated with information abo ut teachers. Employment history, credentials, assignment information and student out comes should be combined to provide more useful information for policy makers and progr am administrators. The possibility of a unified data system in order to guide and evaluate education programs raises legitimate concerns about confident iality and conditions of employment. Reasonable protection for the rights of individual teachers should be built into any such system. Balancing these concerns for privacy is the need to design, implement, and evaluate high quality programs that work for studen ts. Beyond a responsibility to spend public money wisely there is a moral obligation to prepare students well for success in work and higher education. The use of timely and re levant information is one way to improve the odds for success.ReferencesAshton, P. (Ed.) (1991). Alternative approaches to teacher education. Journal of Teacher Education 42 (2), 82. Ashton, P. (1996). Improving the preparation of tea chers. Educational Researcher Vol. 25, No. 9, pp. 21-22.Brunsman, B. (1997). Recruitment and Preparation of Teachers for Mathematics Instruction: Issues of Quality and Quantity in Cali fornia. Sacramento: California Commission on Teacher Credentialing. ( http://www.ctc.ca.gov ) California Department of Education. (1998). Public school summary statistics: 1997-98. Sacramento: author. http://www.cde.ca.gov/ftpbranch/sbsdiv/demographic s/reports/ California Department of Finance. (1998). Californi a public K-12 projections by ethnicity: 1998 series. Sacramento: Author. http://www.dof.ca.gov/html/Demograp/k12ethtb.htm California Postsecondary Education Commission. (199 6). Performance Indicators of

PAGE 17

17 of 23Higher Education. Sacramento: Author. ( http://www.cpec.ca.gov ) California Postsecondary Education Commission. (199 8). Eligibility of California's1996 High School Graduates for Admission to the State's Public Universities. Sacramento: Author. ( http://www.cpec.ca.gov ) California State University. (1993). General educat ion breadth requirements: Executive Order 595. Long Beach, CA: Author. ( http://www.calstate.edu/ ) Council of Chief State School Officers. (December, 1998). Key State Education Policies on K-12 Education. Washington, DC: Author. ( http://www.ccsso.org ) Darling-Hammond, L., and Hudson, L. (1990). Precoll ege science and mathematics teachers: Supply, Demand, and Quality. In Review of Research in Education (Vol. 16). Washington, DC: American Educational Research Assoc iation. ( http://www.aera.net ) Dill, V. (1996). Alternative teacher certification. In J. Sikula (Ed.), Handbook of Research on Teacher Education (2nd edition, pp. 932-957). New York: Macmillan. Education Week. (1997). Quality Counts: A Report Ca rd on the Condition of Public Education in the 50 States. Author, Vol. XVI, Janua ry 22, 1997. ( http://www.edweek.com/ ) Feistritzer. C. (1994). The evolution of alternativ e teacher certification. The Educational Forum 58 (2), 132-138. Greenwald, R., Hedges, L., and Laine, R. (1996). Th e effect of school resources on student achievement. Review of Educational Research Vol. 66, No. 3, pp. 361-396. ( http://www.aera.net ) Haberman, M. 1992). Alternative certification: Can the real problems of urban education be resolved by traditional teacher education? Teacher Education and Practice 8 (1), 13-27.Hanushek, E. (1996). A more complete picture of sch ool resource policies. Review of Educational Research Vol. 66, No. 3, pp. 397-409. ( http://www.aera.net ) Harcourt Brace. (1997). Stanford Achievement Test S eries Ninth Edition Technical Data Report. San Antonio: author. ( http://www.startest.com/ ) Hart, G. And Burr, S. (1996). A State of Emergency ... In a State of Emergency Teachers. Sacramento: California State University Institute f or Education Reform. ( http://www.csus.edu/ier/ ) Hawkins, E., Stancavage, F., and Dossey, J. (1998). School Policies and Practices Affecting Instruction in Mathematics: Findings from the National Assessment of Educational Progress (NCES 98-495). Washington, D.C .: U.S. Department of Education, Office of Educational Research and Improvement.Levin, H. (1974). A conceptual framework for accoun tability in education. School Review 82, (3). pp. 363-392.

PAGE 18

18 of 23Martin, M.O. (1996) Third International Mathematics and Science Study: An Overview. In M. O. Martin and D. L. Kelly (Eds.), Third International Mathematics and Science Study Technical Report, Volume I: Design and Develo pment Chestnut Hill, MA: Boston College.Murnane, R. (1987). Improving education indicators and economic indicators: The same problems? Educational Evaluation and Policy Analysis 9, (2), pp. 101116. National Association of State Directors of Teacher Education and Certification (1998). Manual on Certification and Preparation of Educatio nal Personnel in the United States. Dubuque, Iowa: Kendall Hunt. ( http://www2.nasdtec.org/nasdtec/ ) National Commission on Teaching and America's Futur e (1996 ). What Matters Most: Teaching for America's Future New York, Teachers College, Columbia University. National Center for Education Statistics. (1995a). America's Teachers Ten Years After "A Nation at Risk." Washington, D.C.: U. S. Departm ent of Education Office of Educational Research and Improvement, NCES 95-766. ( http://nces.ed.gov ) National Center for Education Statistics. (1995b). Which Types of Schools Have the Highest Teacher Turnover? Washington, D.C.: U. S. D epartment of Education Office of Educational Research and Improvement, NCES 95-778. ( http://nces.ed.gov ) National Center for Education Statistics. (1996). P rojections of Education Statistics to 2006. Washington, DC: U. S. Department of Education Office of Educational Research and Improvement. ( http://nces.ed.gov ) National Center for Education Statistics. (1998). T he condition of education: Indicator 58. Education and certification of secondary school teachers. Washington, D.C.: U. S. Department of Education Office of Educational Resea rch and Improvement. ( http://nces.ed.gov ) National Center for Education Statistics. (1999). T eacher quality: A report on the preparation and qualifications of public school tea chers. Washington, D.C.: U. S. Department of Education Office of Educational Resea rch and Improvement. ( http://nces.ed.gov ) National Council of Teachers of Mathematics. (1989) Curriculum and Evaluation Standards for School Mathematics. Reston, VA: Natio nal Council of Teachers of Mathematics. ( http://www.nctm.org .) Office of Educational Research and Improvement. (19 88). Creating Responsible and Responsive Accountability Systems: Report of the OE RI State Accountability Study Group. Washington, DC: U. S. Department of Educatio n. ( http://www.ed.gov/offices/OERI/ ) Porter, A. (1991). Creating a system of school proc ess indicators. Educational Evaluation and Policy Analysis 13, (1), pp. 13-29. ( http://www.aera.net ) Reese C., Miller, K., Mazzeo, J., and Dossey, J. (1 997). National Assessment of Educational Progress 1996 Mathematics Report Card f or the Nation and the States.

PAGE 19

19 of 23Washington, DC: U.S. Department of Education, Offic e of Educational Research and Improvement. ( http://www.ed.gov/offices/OERI/ ) Shavelson, R., McDonnell, L., and Oakes, J. (Eds.) (1989). Indicators for Monitoring Mathematics and Science Education: A Sourcebook. Sa nta Monica, CA: The RAND Corporation.White, K. (1982). The relation between socioeconomi c status and achievement. Psychological Bulletin, 91, 461481. ( http://www.aera.net ) Zumwalt, K. (1996). Simple Answers: Alternative Tea cher Certification. Educational Researcher Vol. 25, No. 8, 40-42. ( http://www.aera.net )NoteA significant portion of the research reported here was completed while the author was a staff member of the Califonria Commission on Teache r Credentialing.About the AuthorMark FetlerCalifornia Department of Education515 L Street, #270Sacramento, CA 95814916/322-0373 (voice) 916/327-3516 (fax)Email: mfetler@cde.ca.gov Mark Fetler earned a doctorate in Psychology in 197 8 from the University of Colorado, with training in cognition, measurement, and statis tics. He currently works for the California Department of Education Special Educatio n Division in the area of stateand district wide assessment. Past assignments include the California Department of Education Assessment Program, the California Commun ity College Chancellor's Office, the California Commission on Teacher Credentialing, the Northwest Regional Education Laboratory, and the Western Interstate Commission o n Higher Education. He serves on the boards of various professional and community or ganizations.Appendix A Pairwise CorrelationsPairwise CorrelationsVariableby VariableCorrelationCountSignif ProbScaled Score 10Scaled Score 90.94877730Scaled Score 11Scaled Score 90.93817700Scaled Score 11Scaled Score 100.96297780

PAGE 20

20 of 23Participation 9Scaled Score 90.38737760Participation 9Scaled Score 100.37337740Participation 9Scaled Score 110.36957700Participation 10Scaled Score 90.38837750Participation 10Scaled Score 100.39167820Participation 10Scaled Score 110.38977780Participation 10Participation 90.58767820Participation 11Scaled Score 90.28827730Participation 11Scaled Score 100.29427800Participation 11Scaled Score 110.29137780Participation 11Participation 90.52147790Participation 11Participation 100.73837870AFDCScaled Score 9-0.61827710AFDCScaled Score 10-0.58637770AFDCScaled Score 11-0.58377750AFDCParticipation 9-0.39477770AFDCParticipation 10-0.31457870AFDCParticipation 11-0.27917840Years TeachingScaled Score 90.17117740Years TeachingScaled Score 100.20457800Years TeachingScaled Score 110.22047760Years TeachingParticipation 90.05887800.1009Years TeachingParticipation 100.08177900.0216Years TeachingParticipation 110.0777850.031Years TeachingAFDC-0.08437870.018Education LevelScaled Score 9-0.18527740Education LevelScaled Score 10-0.19017800

PAGE 21

21 of 23 Education LevelScaled Score 11-0.20167760Education LevelParticipation 9-0.06637800.0642Education LevelParticipation 10-0.12257900.0006Education LevelParticipation 11-0.10047850.0049Education LevelAFDC0.1217870.0007Education LevelYears Teaching-0.29197920Percent EmergenciesScaled Score 9-0.30587740Percent EmergenciesScaled Score 10-0.2857800Percent EmergenciesScaled Score 11-0.30177760Percent EmergenciesParticipation 9-0.09537800.007 7 Percent EmergenciesParticipation 10-0.11457900.00 13 Percent EmergenciesParticipation 11-0.09697850.00 66 Percent EmergenciesAFDC0.19457870Percent EmergenciesYears Teaching-0.40567920Percent EmergenciesEducation Level0.20717920Copyright 1999 by the Education Policy Analysis ArchivesThe World Wide Web address for the Education Policy Analysis Archives is http://epaa.asu.edu General questions about appropriateness of topics o r particular articles may be addressed to the Editor, Gene V Glass, glass@asu.edu or reach him at College of Education, Arizona State University, Tempe, AZ 85287-0211. (602-965-96 44). The Book Review Editor is Walter E. Shepherd: shepherd@asu.edu The Commentary Editor is Casey D. Cobb: casey.cobb@unh.edu .EPAA Editorial Board Michael W. Apple University of Wisconsin Greg Camilli Rutgers University John Covaleskie Northern Michigan University Andrew Coulson a_coulson@msn.com Alan Davis University of Colorado, Denver Sherman Dorn University of South Florida

PAGE 22

22 of 23 Mark E. Fetler California Department of Education Richard Garlikov hmwkhelp@scott.net Thomas F. Green Syracuse University Alison I. Griffith York University Arlen Gullickson Western Michigan University Ernest R. House University of Colorado Aimee Howley Ohio University Craig B. Howley Appalachia Educational Laboratory William Hunter University of Calgary Richard M. Jaeger University of North Carolina—Greensboro Daniel Kalls Ume University Benjamin Levin University of Manitoba Thomas MauhsPugh Green Mountain College Dewayne Matthews Western Interstate Commission for HigherEducation William McInerney Purdue University Mary McKeown-Moak MGT of America (Austin, TX) Les McLean University of Toronto Susan Bobbitt Nolen University of Washington Anne L. Pemberton apembert@pen.k12.va.us Hugh G. Petrie SUNY Buffalo Richard C. Richardson Arizona State University Anthony G. Rud Jr. Purdue University Dennis Sayers Ann Leavenworth Centerfor Accelerated Learning Jay D. Scribner University of Texas at Austin Michael Scriven scriven@aol.com Robert E. Stake University of Illinois—UC Robert Stonehill U.S. Department of Education Robert T. Stout Arizona State University David D. Williams Brigham Young University EPAA Spanish Language Editorial BoardAssociate Editor for Spanish Language Roberto Rodrguez Gmez Universidad Nacional Autnoma de Mxico roberto@servidor.unam.mx Adrin Acosta (Mxico) Universidad de Guadalajaraadrianacosta@compuserve.com J. Flix Angulo Rasco (Spain) Universidad de Cdizfelix.angulo@uca.es

PAGE 23

23 of 23 Teresa Bracho (Mxico) Centro de Investigacin y DocenciaEconmica-CIDEbracho dis1.cide.mx Alejandro Canales (Mxico) Universidad Nacional Autnoma deMxicocanalesa@servidor.unam.mx Ursula Casanova (U.S.A.) Arizona State Universitycasanova@asu.edu Jos Contreras Domingo Universitat de Barcelona Jose.Contreras@doe.d5.ub.es Erwin Epstein (U.S.A.) Loyola University of ChicagoEepstein@luc.edu Josu Gonzlez (U.S.A.) Arizona State Universityjosue@asu.edu Rollin Kent (Mxico)Departamento de InvestigacinEducativaDIE/CINVESTAVrkent@gemtel.com.mx kentr@data.net.mx Mara Beatriz Luce (Brazil)Universidad Federal de Rio Grande do SulUFRGSlucemb@orion.ufrgs.brJavier Mendoza Rojas (Mxico)Universidad Nacional Autnoma deMxicojaviermr@servidor.unam.mxMarcela Mollis (Argentina)Universidad de Buenos Airesmmollis@filo.uba.ar Humberto Muoz Garca (Mxico) Universidad Nacional Autnoma deMxicohumberto@servidor.unam.mxAngel Ignacio Prez Gmez (Spain)Universidad de Mlagaaiperez@uma.es Daniel Schugurensky (Argentina-Canad)OISE/UT, Canadadschugurensky@oise.utoronto.ca Simon Schwartzman (Brazil)Fundao Instituto Brasileiro e Geografiae Estatstica simon@openlink.com.br Jurjo Torres Santom (Spain)Universidad de A Coruajurjo@udc.es Carlos Alberto Torres (U.S.A.)University of California, Los Angelestorres@gseisucla.edu


xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam a22 u 4500
controlfield tag 008 c19999999azu 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E11-00121
0 245
Educational policy analysis archives.
n Vol. 7, no. 9 (March 26, 1999).
260
Tempe, Ariz. :
b Arizona State University ;
Tampa, Fla. :
University of South Florida.
c March 26, 1999
505
High school staff characteristics and mathematics test results / Mark Fetler.
650
Education
x Research
v Periodicals.
2 710
Arizona State University.
University of South Florida.
1 773
t Education Policy Analysis Archives (EPAA)
4 856
u http://digital.lib.usf.edu/?e11.121