USF Libraries
USF Digital Collections

Educational policy analysis archives

MISSING IMAGE

Material Information

Title:
Educational policy analysis archives
Physical Description:
Serial
Language:
English
Creator:
Arizona State University
University of South Florida
Publisher:
Arizona State University
University of South Florida.
Place of Publication:
Tempe, Ariz
Tampa, Fla
Publication Date:

Subjects

Subjects / Keywords:
Education -- Research -- Periodicals   ( lcsh )
Genre:
non-fiction   ( marcgt )
serial   ( sobekcm )

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
usfldc doi - E11-00338
usfldc handle - e11.338
System ID:
SFS0024511:00338


This item is only available as the following downloads:


Full Text

PAGE 1

1 of 26 A peer-reviewed scholarly journal Editor: Gene V Glass College of Education Arizona State University Copyright is retained by the first or sole author, who grants right of first publication to the EDUCATION POLICY ANALYSIS ARCHIVES EPAA is a project of the Education Policy Studies Laboratory. Articles appearing in EPAA are abstracted in the Current Index to Journals in Education by the ERIC Clearinghouse on Assessment and Evaluation and are permanently archived in Resources in Education Volume 11 Number 40November 10, 2003ISSN 1068-2341The Relationship between Exposure to Class Size Reduction and Student Achievement in California Brian M. Stecher Daniel F. McCaffrey Delia Bugliari RAND Santa Monica, CaliforniaCitation: Stecher, B. M., McCaffrey, D.F. & Bugliar i, D. (2003, November 10). The relationship between exposure to class size reduction and studen t achievement in California. Education Policy Analysis Archives, 11 (40). Retrieved [Date] from http://epaa.asu.edu/epa a/v11n40/.AbstractThe CSR Research Consortium has been evaluating theimplementation of the Class Size Reduction initiati ve in California since 1998. Initial reports documented the implemen tation of the program and its impact on the teacher workforce, th e teaching of mathematics and Language Arts, parental involvement and student achievement. This study examines the relati onship between student achievement and the number of years students have been exposed to CSR in grades K-3. The analysi s was

PAGE 2

2 of 26 conducted at the grade level within schools using s tudent achievement data collected in 1998-2001. Archival d ata collected by the state were used to establish CSR participati on by grade for each school in the state. Most students had one of two patterns of exposure to CSR, which differed by only one year du ring grade K-3. The analysis found no strong association betwe en achievement and exposure to CSR for these groups, a fter controlling for pre-existing differences in the gro ups.IntroductionIn 2002, Florida voters passed a comprehensive, cla ss size reduction amendment, making Florida the most recent state to adopt this popular, but expensive, educational improvement strategy. During the 1990s, class size reduction (CSR) policies were proposed or adopted b y more than a score of states. California was the most dramatic example. I n 1996, California enacted SB 1777, providing a substantial incentive for school districts to reduce their class sizes from an average of roughly 30 students per cl ass to 20 or fewer. With the signing of this bill, districts in 1996-97 were pro vided with nearly $1 billion in education funds to reduce class size in grades K-3. The funding then increased to roughly $1.5 billion in the second year (1997-98), and it has continued at this level in subsequent years. In addition to the state initi atives, the federal government invested more than $1 billion annually in the reduc tion of class size during the Clinton administration.Despite the continuing enthusiasm among educational policymakers, the value of large-scale CSR efforts remains unproven. The relat ionship of class size to student performance has been studied for over 30 ye ars with mixed results. (See Bohrnstedt and Stecher, 1999, for a comprehensive r eview of the literature.) Earlier findings regarding the efficacy of class si ze reduction were mixed, but recent high-profile studies, especially those relat ed to the Tennessee STAR (Student/Teacher Achievement Ratio) project, have t ipped the policy scales firmly in favor of smaller classes (Mosteller, 1995; Finn, 1998; Finn and Achilles, 1999). In this controlled experiment, researchers have fou nd both short-term and long-term achievement gains associated with smaller class sizes in grade K-3 (Nye, Hedges, and Konstantopoulos, 1999). In fact, a recent study by Krueger and Whitmore (1999) shows that students who were in sma ller classes in K–3 as part of the Tennessee STAR project were more likely to t ake high school courses known to lead to college attendance and to take col lege entrance examinations. Importantly, in all the STAR-related studies the ga ins were larger for minority and lower socio-economic students than for others.Can these effects be achieved on a large scale? The experience of California offers important insights into class size reduction as a statewide policy. The size and complexity of initiating a class size reduction program in the nation’s most populous state and the diversity of California’s cl assrooms represent an important, real-world, test of the effectiveness of CSR as a b road-based policy. This paper presents the results of the most recent analysis of the relationship between the level of exposure to CSR and student achievement in California.

PAGE 3

3 of 26 Summary of Previous Findings from CaliforniaThe CSR Research Consortium, a group of California research and policy organizations, (Note 1) evaluated California’s CSR program beginning in 19 98. In the first two Class Size Reduction (CSR) evaluation reports (see Bohrnstedt and Stecher 1999; Stecher and Bohrnstedt, 2000), resear chers estimated the impact of CSR on student achievement by comparing the Stan ford Achievement Test, 9th Edition (SAT-9) test scores of third-grade students taught in reduced size classes with those of third-grade students taught in non-re duced size classes. (Note 2) Pre-existing differences between the CSR and non-CS R students were adjusted for statistically using student and teacher backgro und characteristics as well as scores from fourthand fifth-grade students who ha d little or no exposure to CSR. Stecher, McCaffrey and Burroughs (1999) and Stecher McCaffrey, Burroughs, Wiley and Bohrnstedt (2000) found that students who were exposed to CSR in third grade performed better than those who were no t. This was true in 1997-98, when both groups of third grade students had little or no prior exposure to CSR, and it was true again in 1998-99, when both groups had one to two years of prior exposure. The differences in scores were equivalent to effect sizes of about 0.04 to 0.1 standard deviation units. In 1998-99, the di fferences were larger for mathematics and language than for reading and spell ing. The researchers found that t he effects of such “one-year” differences in CSR ex posure were similar regardless of a school’s population demographics, i .e., regardless of a school’s percentage of minority, (Note 3) low-income, (Note 4) or English learner (EL) students. (Note 5) In 1998-99 the effects were somewhat larger in sch ools with the highest percentages of minority, low-income, or EL students, but the differences in scores were not statistically signif icant. There was evidence that CSR effects persisted after students had returned to non-reduced classes for one year. Restricting their attention to students enrolled in the same school for three or more years, Stecher et al., (2000) found that third graders who were in reduced classes in 1997-98 scor ed higher than their counterparts in non-reduced classes. Then, in 1999, after both of these groups had been in non-reduced fourth-grade classes, the f irst group again outperformed the second, and the difference was 0.04 standard de viation units. These fourth-grade effects were observed for students exp osed to CSR solely in third grade and for students exposed to CSR in both secon d and third grade. There were no such effects, however, for students whose e xposure was in second grade only.In those selected cases where the California result s could be directly compared with the results of the Tennessee STAR project., th e findings were similar. The important exception is that there was no interactio n between class size effects and demographic factors in California, while in Tenness ee it was found that class size reduction had roughly twice as great an effect for minority students as for non-minority students. Unfortunately, because the r esearchers did not have achievement data prior to the introduction of CSR a nd did not have student achievement data from kindergarten and first grade students, they were unable to estimate the cumulative effects of four years of ex posure to CSR in California’s schools. The size of this effect was one of the chi ef findings from the Tennessee

PAGE 4

4 of 26 STAR study.For a number of reasons, it was not possible to use the same approach to judging the impact of CSR on achievement in subsequent eval uation reports. By 2000-01, CSR had been implemented in over 95 percent of the third-grade classes in California, leaving too few untreated students to s erve as a comparison group. Furthermore, some or all of the upper-grade (i.e., fourthand fifth-grade) students in most schools had participated in reduced size cl asses in earlier years, so their test results could not be used to control for pre-e xisting differences. Thus, the analytic strategies used in the first two evaluatio ns of the California CSR program were no longer applicable in subsequent years.However, the large but uneven growth in participati on in CSR over time provided an opportunity to look at the impact of CSR on achi evement in a different manner. From 1996-97 to 2000-01, CSR went from partially im plemented in two grade levels to almost fully implemented in four grade le vels (kindergarten through third grade). In the third evaluation report, Stecher, Bu gliari, and McCaffrey (2002) used statewide test results to compare achievement results among cohorts of students who had different patterns of exposure to CSR. Trends in achievement that corresponded to patterns of exposure provide e vidence in support of the hypothesis that CSR improves achievement; trends th at have no relationship to CSR participation offered no such support.Focusing on statewide average achievement scores du ring the period 1997-98 to 2000-01, the researchers compared the average achie vement of successive cohorts of students as they moved through the syste m with their average exposure to CSR. Successive cohorts of students had higher achievement during this period, which suggests that one or more of the state educational reforms (which include CSR, new curriculum standards, a sta tewide standardized testing program, the end of bilingual education, and high s takes accountability) had a positive effect. However, the trend in test scores over this period was unrelated to the trend in CSR exposure, so the researchers could not make a strong case that CSR was chiefly responsible for achievement gains.Yet, aggregate analyses do not tell the whole story For example, the state level analysis could not control for external effects, su ch as student mobility. Neither did it permit the researchers to examine the influence of student or teacher background characteristics. The present study addre sses these limitations by analyzing trends in exposure and achievement at the school level, where more data are available to refine the comparisons and co ntrol potentially confounding factors. MethodsAchievement DataBeginning in 1998, California students in grades 211 have been required to complete the SAT-9 annually in the spring. The test results are reported in the summer and fall, and they are made available for re search purposes in the public release California Standardized Testing and Reporti ng (STAR) data files. All analyses reported below use the public release STAR data

PAGE 5

5 of 26 (http://www.cde.ca.gov/statetests/).As part of STAR testing, students complete standard ized multiple-choice tests in mathematics, reading, language and spelling. We foc us here on mathematics, reading and language. We use SAT-9 scale scores (ra ther than raw scores, percentile ranks, or normal curve equivalents) as m easures of achievement in these analyses because scale scores are designed so that score differences are comparable for the entire range of scores. In addit ion, the scales are equated across grade levels, facilitating cross grade compa risons. School SampleThe initial school sample included 4,961 elementary schools in the STAR data files from school years 1997-98 through 2000-01. We excluded those schools for which the STAR file in any year contained scores fo r 10 or fewer students and those schools for which the STAR files were missing basic demographic data (gender, ethnicity, English language fluency status ) on all students. These criteria excluded 2,069 school (42 percent), leaving 2,892 s chools in our analysis file. Despite the exclusions, the schools in our sample c losely resemble the schools in the state as a whole in terms of student demographi c characteristics. Table 1 shows the comparison between the sample schools and the whole state in terms of participation in CALWORKS, eligibility for free or reduced priced lunches, race/ethnicity, and language status for the 1999-20 00 school year. The mean values for sample schools are within one to three p ercentage points of the mean values for the state as a whole on all variables, s o the generalizability of the results from our analyses are not limited by the po pulations served by sampled schools.Table 1. Demographic Characteristics of Sample Scho ols and All Elementary Schools Demographic feature All elementary schoolsaAnalysis sample schools Percent CALWORKS participants13.59 (12.88)14.64 (12 .97) Percent free or reduced price lunch eligible51.99 ( 30.27)53.76 (30.17) Percent white 38.39 (29.38)34.86 (28.66) Percent Hispanic 40.98 (29.31)43.19 (29.60) Percent African American 8.13 (12.60)9.28 (14.00) Percent Asian 7.76 (12.01)7.88 (11.40) Percent minority 61.61 (29.38)65.15 (28.66) Percent ELL 27.14 (24.10)29.29 (24.30) Total enrollment 609.94 (282.39)660.40 (276.38)aState sample includes 4,761 elementary schools open since 1996 with CDS codes.Class Size Reduction ParticipationClass size reduction began with the 1996-97 school year, one year prior to STAR testing. By the 1999-2000 school year over 90 perce nt of all students in

PAGE 6

6 of 26 kindergarten through third grade were participating in CSR. However, for earlier cohorts, CSR participation varied across schools. T his variation provided an opportunity to compare achievement with CSR exposur e. The first step in our analysis, therefore, was to determine CSR participa tion by grade and school year for each of the 2,892 schools in the analysis file. We focused on CSR participation for three cohorts of student--those who entered kin dergarten in 1995-96 (K95), 1996-97 (K96) or 1997-98 (K97). These three cohorts of students reached the third grade in 1999, 2000, and 2001 and they are th e only cohorts with exposure to CSR for whom we have SAT-9 scores in both second and third grade. For each elementary school in California we develop ed an indicator of CSR participation by grade level by year. Unfortunately the state did not collect comparable information about CSR participation ever y year, so we had to use multiple data sources to infer CSR status. The prim ary data for assessing CSR status were the individual student SAT-9 answer fil es, which included indicator variables for CSR participation for every student. We also used teacher reports of classroom enrollment from the CBEDS Professional As signment Information Form (PAIF). A third source of information was the distr ict level J-7 CSR report, which describes district participation in CSR for the 199 6-97 and 1997-98 school years (http://cde.ca.gov/csr/). The J-7 information was o nly useful when participation was uniform across the district. Finally, the CBEDS School Information File (SIF) data contain school and grade level CSR indicators for the 1998-99, 1999-2000, and 2000-01 school years.The CSR indicator development process began with th e student-level STAR data file. If 10 percent or fewer students within a grad e at a school were coded as participating in the CSR program (either option 1 o r 2), we classified that grade as not reduced. If 90 percent or more students within a grade at a school were indicated as in the CSR program, we classified that grade as reduced. We classified a grade as undetermined by STAR if betwe en 10 percent and 90 percent of students were indicated as CSR. Let Cgjt,STAR denote the CSR status for grade g = kindergarten, 1, 2 or 3, in school j and school-year t = 1996-97, 1997-98, 1998-99, 1999-00 or 2000-01. Cgjt,STAR equals “R” if we determine the school had reduced classes for grade g in year t ; Cgjt,STAR equals “N” if not reduced and “U” if undetermined.Because the STAR data did not permit clear classifi cation for every school, grade level, or school year, i.e., in some instances Cgjt,STAR equals “U,” we turned to other sources to make our final determination of CS R participation. The PAIF data provide the number of students in each teacher's cl assroom and the number of teaching assignments. The distribution of students across classrooms for teachers with multiple assignments cannot be determined from the PAIF. Therefore, for determining CSR participation we used only teachers with a single teaching assignment. Also, some teachers report over 50 stud ents or fewer than 14 students in their classroom. We excluded these teac hers from the classification process, arguing that they represented data errors or nontraditional education assignments.A school was judged to have reduced size classes fo r a given grade in a given year if over 65 percent of included teachers in tha t grade reported 21 or fewer students. If fewer than 35 percent of included teac hers in a grade reported 21 or

PAGE 7

7 of 26 fewer students, we classified that grade as not red uced. We classified a grade as undetermined by PAIF if between 35 percent and 65 p ercent of the classes were reported as having 21 or fewer students. We let Cgjt,PAIF denote the CSR status as determined by the PAIF where the variable again takes on the values of “R," “N,” and “U” for reduced, not reduced or undetermin ed. We also created variables for the CSR participation as determined by the SIF ( Cgjt,SIF) and the J-7 data ( Cgjt,J7). Cgjt,SIF equals “U” for the 1996-97 and 1997-98 school years for all grades and schools bec ause grade-level CSR indicators were not added to SIF until 1998-99. Fin ally, Cgjt,J7 takes on values “R” and “N” only if the district had uniform CSR practi ces at a grade level across all schools.For final CSR classification, we compared the CSR i ndicators based on STAR, PAIF, SIF and J-7. In the majority of cases, all de terminable sources agreed, Cgjt,STAR = Cgjt,PAIF = Cgjt,SIF = Cgjt,J7 or some variables equaled “U” and the remaining variables agreed. In these cases we assig ned the common value to the CSR indicator. In the cases of disagreement, we exa mined the longitudinal trend in CSR indicators before making a final determinati on. For example, if Cgjt,STAR= R and Cgjt,PAIF= N for year t we checked the data for the previous year ( t 1). If Cgjt-1,STAR = Cgjt-1,PAIF = R, then we decided that the school probably had reduced class size in year t as well. Schools for which we were unable to resol ve data conflicts confidently were excluded from the f inal analytic file. We excluded 543 schools because of indeterminate CSR status, le aving a sample of 2,349 schools. The excluded schools constituted 19 percen t of the 2,892 schools that met the data and size conditions described above. T he remaining schools constituted 47 percent of the original sample.CSR Exposure by CohortFor each of the three focal cohorts, K95, K96 and K 97, Tables 2, 3 and 4 present the distribution of CSR exposure across the final s ample of schools. Table 2 shows that nearly 90 percent of the schools in the sample had one of two patterns of CSR exposure for the K95 student cohorts: CSR in grades 2 and 3 only (22.3 percent) or CSR in grades 1, 2 and 3 (66.8 percent) For the K96 cohort there was even less variation in CSR exposure. Table 3 shows that these students participated in CSR for grades 1, 2 and 3 in almost every school (89.9 percent). By the K97 cohort, Table 4 shows that more schools introduced CSR in kindergarten, and the schools fell, almost exclusiv ely, into one of two patterns of CSR exposure: kindergarten through grade 3 (38.8 pe rcent) or grades 1, 2 and 3 (59.9 percent).Table 2. Distribution of CSR Exposure for Cohort K9 5 Exposure patternNumber of schoolsPercent of sample Indeterminate 200.9 None 251.1 Grade 3 only 70.3 Grade 2 only 662.8

PAGE 8

8 of 26 Grades 2 and 3 52522.3 Grade 1 only 100.4 Grades 1 and 3 50.2 Grades 1 and 2 1054.5 Grades, 1, 2 and 3 1,56966.8 Kindergarten and grade 3 10.0 Kindergarten, grades 2 and 3 10.0 Kindergarten, grades 1, 2 and 3 150.6 Table 3. Distribution of CSR Exposure for Cohort K9 6 Exposure patternNumber of schoolsPercent of sample Indeterminate 120.5 None 10.0 Grade 3 only 10.0 Grades 2 and 3 120.5 Grade 1 only 40.2 Grades 1 and 3 10.0 Grades 1 and 2 502.1 Grades, 1, 2 and 3 2,11289.9 Kindergarten, grades 2 and 3 10.0 Kindergarten, grades 1, 2 and 3 1556.6 Table 4. Distribution of CSR Exposure for Cohort K9 7 Exposure patternNumber of schoolsPercent of sample Indeterminate 70.3 Grades 2 and 3 10.0 Grades 1 and 2 200.9 Grades, 1, 2 and 3 1,40659.9 Kindergarten, grades 2 and 3 10.0 Kindergarten, grades 1 and 2 30.1 Kindergarten, grades 1, 2 and 3 91138.8Grouping Schools by CSR ExposureWe focused our analyses on four groups of schools w ith distinctive patterns of CSR exposure. These 1,918 schools constitute 82 per cent of the schools in the final analysis sample and 40 percent of the schools in the original sample. Table 5 shows these four patterns. Because few schools had any of the remaining exposure patterns, we restrict the study to schools in these four groups.Table 5. Distribution of CSR Exposure for All Three Cohorts

PAGE 9

9 of 26 GroupK95K96K97Number of schools A1, 2, 31, 2, 31, 2, 3877B2, 31, 2, 31, 2, 3348 C2, 31, 2, 3K, 1, 2, 3152D1, 2, 31, 2, 3K, 1, 2, 3541Demographic differences across groups (described be low) led us to focus our primary comparisons of outcomes on Group A and Grou p B. These two groups contain 1,225 schools. In Group A, students who ent ered kindergarten in 1995-96, 1996-97 or 1997-98 had reduced-size classes in grad es 1, 2 and 3 (but not kindergarten). Group B schools serve a similar popu lation of students, but the three cohorts had different exposure to CSR. Studen ts entering kindergarten in 1995-96 had two years of exposure to CSR in second and third grade, those entering in subsequent years had an additional year of CSR in first grade. Student SampleAs noted above, our analyses are restricted to stud ents in the K95, K96 and K97 cohorts. From these cohorts we included only those students who: 1) attended the same school for kindergarten through second or thir d grade, depending on the grade of the outcome used in the analysis; 2) did n ot have a test identified as “Out of Level”; and 3) were not identified as receiving Special Education services. We also excluded students when their STAR data CSR fla g was inconsistent with the data from the vast majority (over 90 percent) of th eir fellow students in the same grade and school. For example if the STAR student d ata file indicated that for a particular school over 90 percent of third graders in a cohort were in reduced size classes, then we excluded any third graders from th at school and cohort for whom the STAR data indicated they were not in reduced si ze classes. Table 6 contains summaries of the student demograph ic characteristics and teacher qualifications of the identified cohorts of students in schools in the four groups. Groups A and B are similar in terms of stud ents and teacher characteristics, while Groups C and D are distinctl y different. Schools in Groups A and B have greater percentages of minority students EL students, and students from families receiving public assistance than scho ols in Groups C and D. Groups A and B also are similar in terms of teacher charac teristics, and they have fewer teachers who are fully-credentialed than schools in Groups C and D. These differences make comparisons between Groups C and D and the other groups difficult because such comparisons would confound s tudent demographics and teacher qualifications with CSR effects. Therefore we focus only on Groups A and B.There is one instance in which schools in Groups A and B differ with respect to teacher credentials that only is apparent when the data are disaggregated by cohort. Group B schools have more uncredentialed fi rst-grade teachers than Group A schools for cohorts K96 and K97. This diffe rence appeared when Group B introduced CSR at first grade, and it probably is a result of these schools hiring new teachers in the tight teacher labor market that followed the introduction of CSR. (See Tables 7-12 for student and teacher chara cteristics disaggregated by

PAGE 10

10 of 26 cohort and grade level.)Table 6. Average Student and Teacher Characteristic s for Cohorts K95, K96 and K97, by Group Group Student characteristics a Teacher characteristics b Minority %EL %AFDC %ExperienceCredential A66.8433.2320.4013.3089.13B69.2332.0621.0913.2588.51 C57.6625.9118.3813.4693.10D51.9920.6718.2613.5294.71aAverage for the three cohorts during their kinderga rten, first, second, and third grades.bAverage years of experience for teachers of the ide ntified cohorts of students; percentage of teachers of the identified cohorts of students with full credentials.Table 7. Percentage of Students in Cohort Whose Fam ilies Receive AFDC During Four Years, by Group GroupCohortKindergartenFirst gradeSecond gradeThird grade AK95624.8423.7120.6019.12 K96723.7122.0619.1217.43 K97822.0619.1217.4315.62 BK95625.3624.3422.0120.06 K96724.3422.9920.0617.74 K97822.9920.0617.7415.39 CK95623.1122.6717.8217.12 K96722.6720.1217.1215.00 K97820.1217.1215.0012.64 DK95621.3920.9721.7017.44 K96720.9719.1617.4415.13 K97819.1617.4415.12713.16 Table 8. Percentage of Minority Students in Cohort During Four Years, by Group GroupCohortKindergartenFirst gradeSecond gradeThird grade AK95664.7965.6963.7867.55 K96765.6966.5767.5568.42 K97866.5767.5568.4269.57 BK95666.5767.9862.6870.36 K96767.9869.2070.3671.57 K97869.2070.3671.5772.92 CK95655.1056.4552.8858.64 K96756.4557.0858.6459.86 K97857.0858.6459.8661.21

PAGE 11

11 of 26 DK95649.1149.8456.7852.05 K96749.8450.8052.0553.14 K97850.8052.0553.1454.32 Table 9. Percentage of EL Students in Cohort During Four Years, by Group GroupCohortKindergartenFirst gradeSecond gradeThird grade AK95632.5132.9233.2633.37 K96732.9233.3333.3733.48 K97833.3333.3733.4833.40 BK95630.7931.8129.5932.50 K96731.8132.2432.5032.86 K97832.2432.5032.8632.97 CK95624.5025.4523.5826.26 K96725.4526.4926.2626.52 K97826.4926.2626.5227.15 DK95619.0019.7224.8820.35 K96719.7220.5020.3520.83 K97820.5020.3520.8320.96 Table 10. Average Years of Teaching Experience for Teachers of Cohort During Four Years, by Group GroupCohortKindergartenFirst gradeSecond gradeThird grade AK95616.0813.2412.8512.51 K96716.8711.1512.6112.94 K97814.5810.7212.5913.45 BK95615.5413.3313.1112.18 K96716.4511.2512.6912.70 K97815.1410.7512.8512.99 CK95616.3013.0412.3813.86 K96715.5811.1712.6214.44 K97813.2411.5312.9614.42 DK95616.4211.9913.0413.75 K96715.3311.9312.9814.03 K97812.8812.2713.3414.24 Table 11. Percentage of Teachers of Cohort with Ful l Credentials During Four Years, by Group GroupCohortKindergartenFirst gradeSecond gradeThird grade AK95698.0695.2187.3485.21 K96796.2285.5687.1087.05 K97888.0985.1186.6588.01 BK95698.7895.7486.5684.18

PAGE 12

12 of 26 K96796.2683.1186.0986.28 K97888.9682.2985.7588.11 CK95698.9997.3192.6690.27 K96797.8592.2489.4792.27 K97891.9889.2191.6793.27 DK95698.4596.5893.1794.05 K96797.0193.6394.4194.28 K97892.3994.0294.1994.34 Table 12. Parameter Estimates (Standard Errors) for Model 1 Grade 2Grade 3 MathReadingLanguageMathReadingLanguageMean Group A, K95 569.5 (0.9) 571.5 (1) 583.3 (0.9) 603.3 (1) 608.8 (1.1) 607.6 (1) Difference, Group A K96 less K95 7.3 (0.3)5 (0.3)4.4 (0.3)6.9 (0.3)4.6 (0.3)5.8 (0.3 ) Difference, Group A K97 lessK95 13.2 (0.4) 10.9 (0.4) 9.1 (0.4) 12.3 (0.4) 9.4 (0.3)10.4 (0.4) Difference between Groups K95 -8.6 (1.7)-4.5 (1.8)-5.4 (1.7) -5.8 (1.8) -6.7 (2)-7.3 (1.8) Group B linear trend1.9 (0.5)-0.1 (0.5)0.5 (0.5)0.7 (0.5)0.7 (0.5)0.8 (0.5) Effect of additional year CSR atgrade 1 -0.9 (0.7)1.7 (0.7)0.9 (0.7)0.7 (0.7)-1.1 (0.7)-0.8 (0.7)Note: The difference parameter estimates of the Dif ference, Group A K96 less K95 and the Difference, Group A K97 less K95 contai n the Group A linear trend and the common (across Groups) cohort deviations fr om the linear trend. Group A schools had between 53,000 and 59,000 stude nts per cohort when the cohorts reached grade 2, and between 46,000 and 48, 000 students per cohort when the cohorts reached third grade. For Group B, the numbers of second graders per cohort ranged from 23,000 to 25,000 and the number of third graders per cohort ranged from 19,000 to 21,000. The sample s are smaller in third grade than in second grade because they are restricted to students who attended the same school for one additional year.AnalysisOur goal is to determine if cohort-to-cohort variat ion in CSR exposure predicts cohort-to-cohort variation in test scores. On the b asis of the exposure patterns presented in Table 5, we note that a comparison of schools across years, groups and cohorts can only provide data on the effects of a one-year variation in exposure to CSR. Larger differences in exposure do not exist among comparable groups of schools. In addition, other reforms and c hanges were taking place during this period that might have affected test sc ores. As a result, a simple comparison of scores for students in the K95 cohort with scores for students in the K96 or K97 cohorts might confound CSR effects with these other changes. More

PAGE 13

13 of 26 complex comparisons, however, can isolate the effec ts of CSR with less confounding of alternative effects. For example, be cause the exposure to CSR was the same for all three cohorts in Group A, thes e schools provide a measure of the effects of factors unrelated to CSR on the tren d in scores over these three years. Similarly, differences between K96 and K97 s cores in Group B schools also are unrelated to CSR because exposure was the same for these two cohorts (but not for the K95 cohort). Thus, differences among th ese five cohorts in Groups A and B can be used to estimate the effects of other programs and the effects of cohort-to-cohort variation.On the other hand, the students in the Group B-K95 cohort had one year less CSR exposure during first grade than the students i n the two later Group B cohorts and than students in all three cohorts in G roup A. By comparing scores for the Group B-K95 students to those of other students we can observe differences between groups with varying exposure to CSR. Howeve r, we must make judicious use of the data from the other students to limit th e confounding effects of other programs and cohort-to-cohort variation in scores. The following list of comparisons with Group B-K95 highlights the assumpt ions about groups and time trends that are required for the comparisons to pro vide unconfounded estimates of the CSR effect. It also points out the compariso ns that we believe provide the best estimates of the CSR effect.Comparison 1 : Compare Group B-K95 scores to Group B-K96 or Grou p B-K97 scores. The comparison yields unconfounded estimate s of the CSR effect if we assume that, in the absence of CSR, scores do not c hange systematically over time. However, research has consistently shown that score gains occur in the years following the introduction of a new, high-sta kes testing program even in the absence of other initiatives. Thus, this assumption seems unwarranted, i.e., scores are likely to change over time even in the a bsence of CSR. In fact, this change is evident in Group A where CSR exposure is constant. As a result, we will not use these within-Group B comparisons as an esti mate of the CSR effect. Comparison 2 : Compare Group B-K95 scores to Group A-K95 scores. This comparison yields unconfounded estimates of the CSR effect if we assume that, in the absence of CSR, the groups would have the sa me scores on average. At first this assumption seems reasonable because the schools in the two groups are very similar on student demographic and teacher cha racteristics. However, the schools in Group A implemented CSR more quickly tha n schools in Group B, and the factors that led to this alternative behavior m ight be related to average scores. Thus, we do not think this assumption is warranted. (Alternatively, comparison of Group B-K95 to Group A-K96 or K97 would be affected both by time trends and cross group differences. The required assumptions f or unconfounded estimation are not tenable in these comparisons either.)Comparison 3 : Compare the difference between Group B-K96 and Gr oup B-K95 to the difference between Group B-K97 and Group B-K 96. This comparison attempts to remove the time trend by using the diff erence between Group B-K97 and Group B-K96 scores as an estimate of the time t rend between K95 and K96. The comparison yields unconfounded estimates if we assume that the time trend in scores is linear across the three cohorts. This is one of the estimates that will be presented in the Results section. (In Table 13, Comparison 3 is found in the

PAGE 14

14 of 26 row labeled Difference and the column labeled Group B.) Comparison 4 : Compare the difference between Group B-K95 and Gr oup A-K95 to the difference between Group B-K96 and Group A-K 96. (This is equivalent to comparing the difference between K96 and K95 for Gr oup B to the difference between K96 and K95 in Group A.) This comparison us es differences across Groups in K96 to estimate differences across groups in K95. Alternatively, we can view this estimate as using Group A to estimate the time trend from K95 to K96. This estimate is unconfounded if we assume that, in the absence of CSR, group differences would be constant over time. (We could also include the K97 cohorts in these comparisons.) We also present this compari son in the Results section. (In Table 13, Comparison 4 is found in the row labeled K96 less K95 and the column labeled Difference.)Comparison 5 : Compare the difference in differences for Group B (i.e., compare the difference between K96 and K95 and the differen ce between K97 and K96) to the difference in differences for Group A. This mod el uses Group A to estimate the size of cohort-to-cohort deviations from a line ar time trend in Group B. This model produces unconfounded estimates of the CSR ef fect if we assume that no interactions would exist in between groups and devi ations from time trends in the absence of CSR. (In Table 13, Comparison 5 is found in the row labeled Difference and column labeled Difference.)Because scores for students within the same school might be positively correlated and because schools vary in size, the simple averag e estimators described above might not be efficient. Therefore, we also fit a hi erarchical linear model to estimate Comparison 5 while allowing for possible intra-scho ol correlation. Model 1 for a score for the k th student in cohort t ( t = 1 for K95, 2 for K96 and 3 for K97), school j of group i, yijtk, is given by The functions I( t = 1) and I( t = 2) equal one if t = 1 or 2 respectively and zero otherwise. SAS Proc Mixed provided estimates of the coefficients of the random effects model. We also used fixed school effects mo dels and the results were nearly identical. Sensitivity analyses were conduct ed to explore the effects of teacher credentials, and the results were essential ly unchanged. We fit Model 1 separately for grades 2 and 3. Indiv idual student scores are not linkable over time in the STAR data, so growth mode ling was not possible. Models of change for cohorts within school were feasible, but because we had no hypotheses on the effects of a year's delay in CSR for growth in the following two years, we looked only at the effects within each gr ade.Results

PAGE 15

15 of 26 CSR Effects on Math, Reading and Language Test Scor e There is an upward trend in scores across cohorts K 95, K96, and K97 in both Group A and Group B schools (see Figure 1). The top panel of the figure shows the box and whisker plots of the distribution of sc hool mean math scores for the three cohorts of students from Group A schools. The dot corresponds to the median score, the upper and lower sides of the rect angle correspond to the 25th and 75th percentiles of the distribution, and the b rackets at the ends of the whiskers correspond to the 5th and 95th percentiles of the distribution of scores. Dots beyond the whiskers are extreme outliers.There is an obvious upward trend in scores across c ohorts over time, as the distribution shifts to the right for each successiv e cohort. However, in Group A schools, all three cohorts experienced exactly the same pattern of CSR exposure (grades 1 through 3). Thus, the trend in scores is not related to changes in the level of CSR exposure. (Note 6) During the time period that our three study cohort s were in kindergarten through grade 3, California en acted several other statewide education initiatives, including the introduction o f demanding new curriculum standards, a statewide standardized testing program with high-stakes accountability, and the end of bilingual education. All of these programs might contribute to rising test scores across cohorts, ev en if differences in CSR have no effect.The lower panel in Figure 1 shows box and whisker p lots for the cohorts in the Group B schools. The plots for Group B show a nearl y identical trend to the plots for Group A, even though students in cohort K95 in Group B had one year less exposure to CSR than students in the other two coho rts in Group B. Figures for reading and language scores show similar patterns ( see Figures 2 and 3). On the basis of this figure, it seems clear that the addit ional year of CSR in first grade did not have large effects on mathematics scores.

PAGE 16

16 of 26 Figure 1. Third grade SAT-9 score distributions in mathemati cs for successive cohorts of students with constant vs. increasing CS R exposure.

PAGE 17

17 of 26 Figure 2. Third grade SAT-9 score distributions in reading f or successive cohorts of students with constant vs. increasing CSR exposu re. Figure 3. Third grade SAT-9 score distributions in language for successive cohorts of students with constant vs. increasing CS R exposure. Table 13 provides further evidence that, for studen ts in these cohorts and schools, the effects of an additional year of CSR were small In Table 13a, the first row presents the differences between mean second-grade math scores for K96 and K95 for Groups A and B, and the difference between these differences. The second row contains the differences between mean se cond-grade math scores for K97 and K96 for the two groups and the difference b etween the differences. In the third row we have the difference of these two cohor t-to-cohort differences in each Group and between the groups. Tables 13b-13f contai n similar differences for grade 3 mathematics scores and for grades 2 and 3 r eading and language scores. The table contains the results of comparisons 3, 4 and 5 among cohort means by group, grade, and subject. For example, for Group B the difference in mean scores for K96 and K95 is the difference between a cohort of students that participated in CSR in grades 1, 2 and 3 and a coho rt that participated only in grade 2 and 3. Thus, the value of 6.49 from Table 1 3a represents in part an effect of one additional year of CSR when students were te sted in second grade. It also includes other effects occurring during this time. Comparison 3 attempts to remove the time trend in this comparison by using t he difference between K97

PAGE 18

18 of 26 and K96 in Group B, which is found in Table 13a to be 8.05. Under the assumptions listed above, the difference between th ese two values produces an unconfounded estimate of the CSR effect as 1.57 ( the last row of Table 13a in the Group B column).Comparison 4 uses the difference between K96 and K9 5 in Group A to estimate the natural trend in scores, and adjusts the Group B differences accordingly. This produces an estimate of the CSR effect as 1.15 (the last column in the first row of Table 13a). As noted above, each estimate makes dif ferent assumptions about what has remained constant across time or groups. T he estimate in the Group B column assumes that changes from cohort to cohort i n Group B are constant except for CSR. The estimate in the K96 less K95 ro w assumes that changes from K95 to K96 are constant across Groups A and B except for CSR. Comparison 5 assumes that, except for the effects o f CSR, cohort-specific deviations from a linear trend are constant across groups. This difference of differences approach provides an estimate of the CS R effect equal to 0.52. This value is computed as the difference of the values f or Groups B and A in the last row of Table 13a. (The estimate is given in the Dif ference column of the Difference row of Table 13a.)Table 13a. Second Grade Math Group AGroup BDifference K96 less K955.346.491.15K97 less K966.398.051.67Difference-1.05-1.57-0.52 Table 13b. Third Grade Math Group AGroup BDifference K96 less K956.798.171.38K97 less K966.537.290.71Difference0.260.930.67 Table 13c. Second Grade Reading Group AGroup BDifferenceK96 less K952.053.661.61K97 less K966.266.05-0.21Difference-4.21-2.391.82 Table 13d. Third Grade Reading Group AGroup BDifferenceK96 less K954.634.04-0.59K97 less K966.236.770.54Difference-1.59-2.72-1.13 Table 13e. Second Grade Language Group AGroup BDifferenceK96 less K952.003.351.35K97 less K965.255.540.29Difference-3.26-2.201.06 Table 13f. Third Grade Language Group AGroup BDifferenceK96 less K956.035.83-0.20K97 less K965.786.550.76Difference0.24-0.71-0.96The estimates in Table 13 ignore random school effe cts that are included in Model 1 to produce efficient estimates and test the null hypothesis that the effect is zero. The results of this model are reported in Table 14, and the full model estimates

PAGE 19

19 of 26 are included in Table 15. The estimated effects are uniformly small in absolute value ranging from 1.1 to 1.7; these estimates ar e also small relative to the standard deviation in SAT-9 scores (about 40 scale score points). In addition, the effects across grades are offset--the negative esti mate for math in grade 2 is followed by a positive estimate at grade 3, and the positive estimates for reading and language at grade 2 are followed negative estim ates at grade 3. Overall, the estimates from Table 13 and Table 14 are very simil ar and suggest little CSR effect. We also explored school fixed effects model s and the results were nearly identical to those in Table 15.Table 14. Estimates of 95% Confidence Intervals of CSR Effects from Model 1 Grade 2Grade 3 Math0.9 (2.3, 0.5)0.7 (0.7, 2.2)Reading1.7 (0.3, 3.1)1.1 (2.6, 0.3)Language0.9 (0.4, 2.2)0.8 (2.3, 0.6)We also conducted some sensitivity analyses to see whether these results were consistent for across student and teacher character istics. We found similar results when we restricted the analysis to schools with mor e than 65 percent minority students, suggesting that the CSR effect was not la rger for minority students. (This analysis included about one-half of the schoo ls.) To address the possible bias introduced by the difference between Groups A and B in the change in the percentage of fully-credentialed first grade teache rs, we restricted the analysis to schools with no change in the percentage of fully-c redentialed teachers during this time period. The results of this analysis were simi lar, as well. Finally, we ran the analyses with both restrictions, and although the s ample of schools was small, we saw no substantial differences in the results.CaveatsThese school-level analyses were less susceptible t o confounding from external sources than the statewide analyses presented by St echer, Bugliari and McCaffrey (2002). For example, we were able to cont rol for student mobility by only including students who attended the same schoo l from kindergarten through second or third grade. Yet, there are still limitat ions in these analyses. The greatest limitation comes from the lack of variatio n that existed in exposure to CSR. Our comparisons were limited to a one-year dif ference in exposure to reduced size classes among students whose total exp osure was two or three years. The one-year difference occurred in first gr ade, and all students subsequently participated in reduced size classes i n second and third grade--the points at which their achievement was measured. The Tennessee STAR experiment compared students who attended reduced s ize classes for four consecutive years with students who attended normal size classes for four consecutive years. They found that at least two yea rs of exposure were needed to produce lasting differences. Those conditions for c omparison did not exist in California.There have also been modest changes in the demograp hic characteristics of students during this period that might have affecte d achievement trends. Table 15

PAGE 20

20 of 26 shows selected demographic characteristics of Calif ornia public school students during this time period. There has been a modest in crease in the percentage of Hispanic students during this time period, but our differencing approach should have minimized the impact of this gradual change. Y et, our models were simple and did not adjust for demographic or other student background variables. Given the small size of effects and the general similarit y of the comparison groups we used a simple analysis rather than complex models. However, small differences among the groups might have affected our results, a nd more complex models might have removed some of these differences.Table 15. Demographic Characteristics of California Students, 1995-2000 (percentages) Race/Ethnicity School year Total enrollment Limited English Proficient (LEP) Asian Hispanic or Latino African American White (not Hispanic) Other 1995–965,467,22423.68.238.78.840.43.91996–975,612,96524.28.239.78.739.53.91997–985,727,30324.68.140.58.838.83.91998–995,844,11124.68.141.38.737.84.21999–005,951,61224.78.042.28.636.94.32000–016,050,89524.98.043.28.435.94.5Note. Starting in 1998–99, all figures include Cali fornia Youth Authority (CYA) schools. “Other” inclu des American Indian or Alaskan Native, Pacific Islander, Filipino, and, be ginning in 1998, Multiple or No Response. Note. Source: California Department of Education, E ducation Demographics Unit.There have been significant policy and program chan ges during this period that also affected student achievement. These changes in clude new state standards and curricula, revised grade-level promotion polici es, a new test-based school-level accountability system with large rewar ds for increases in scores, and the elimination of traditional bilingual education programs. Because they occurred simultaneously, we used various forms of differenci ng to disentangle their separate effects and to isolate the unique contribu tion of CSR to score improvement during this period. However, the differ encing requires many assumptions about the equivalence of groups and coh orts in the absence of CSR and the large of number of changes in other program s calls into question the validity of those assumptions.In addition, there is some reason to doubt the vali dity of the score gains we used as the basis for these analyses. The California sch ool accountability system has created a high-stakes atmosphere that may lead to c hanges in test scores that are independent of actual changes in achievement. The g ains in SAT-9 scores observed in California are well within the range th at might be associated with such score inflation. Again, differencing removes genera l trends due to score inflation but cannot account for differential inflation.Another limitation is the restricted sample of the schools and students used in our study. Many schools did not have complete student d emographic data, and they were eliminated from our sample. Others had too few valid test scores and were eliminated for this reason. Still other schools wer e dropped because of indeterminacy in CSR exposure. In addition our anal yses focus on students who

PAGE 21

21 of 26 did not change schools during the K-3 years. The ef fects of CSR might be different for the schools and students we excluded from our analysis, but we do not have the data to determine the effects of these restrictions on our results. We do not have any good hypotheses about the likely di rection of differences between the CSR effects in our sample and those for the ent ire state. Finally, the available data do not allow us to judg e the impact of the entire CSR program and its effects on students for the last fi ve years. Rather, we look for evidence that reduced size classes can make a diffe rence by testing whether additional exposure yields greater achievement. A p ositive result would be encouraging evidence that small classes are benefic ial and that offering them to students in California could have positive effects. Our null finding, however, cannot be interpreted as evidence that the CSR prog ram is not effective. Our results are consistent with at least two possible i nferences: a.) reduced size classes have no effect, or b.) two, three or four y ears of exposure to reduced size classes do have a positive effect compared to no ex posure but the difference between two years of exposure and three years of ex posure is negligible. One should not make the most pessimistic interpretation of our results (e.g., that reduced size classes have no effect and therefore t he entire CSR program is a failure). Rather we should make the most cautious i nterpretation that, in the context of a K-3 program of reduced size classes, a one-year incremental difference in exposure has no effect. K-3 CSR might have large positive effects on students but differential gains among students with small differences in exposure cannot be used as evidence of those larger effects.ConclusionsThe goal of this investigation was to determine the extent to which changes in achievement correspond to the implementation of the CSR program. The analyses show that scores at the elementary level h ave been rising at the same time that increasing percentages of students have b een taught in reduced size classes. However, many other educational reforms we re enacted during this period that might have contributed to the achieveme nt gains, and it is impossible for us to determine how much the various factors ma y have influenced trends in overall student achievement. Our analyses that used differences in group means to control for the other factors showed that a oneyear difference in exposure occurring in first grade is not associated with gre ater gains in achievement. Due to the rapidity of CSR implementation, we could not te st the cumulative effects of two or three years of exposure. Thus, while the ana lyses presented in this chapter find no association between one year's difference i n exposure and differences in achievement, we cannot draw any conclusions about t he effects of CSR in larger doses.NotesThis research was conducted under the auspices of t he CSR Research Consortium, including RAND, the American Institutes for Research, Policy Analysis for California Education (PACE), WestEd, a nd EdSource. Findings were reported previously as a Technical Appendix to the Consortium’s final report What Have We Learned About Class Size Reduction in Calif ornia (Bohrnstedt and

PAGE 22

22 of 26 Stecher, 2002). The research was funded by the Cali fornia Department of Education, the Walter and Elise Haas Fund, the Will iam and Flora Hewlett Foundation, the Walter S. Johnson Foundation, the S an Francisco Foundation, and the Stuart Foundation. The opinions expressed h ere are the authors’. Endnotes The CSR Research Consortium includes the American I nstitutes for Research (AIR), RAND Corporation, Policy Analysis f or California Education (PACE), WestEd, and EdSource. 1. The Consortium’s analyses were limited by the fact that there were no achievement data for kindergarten students or first grade students in any year, and there were no achievement data for any st udents prior to 1998. 2. Minority students are any students not classified a s Caucasian. The largest groups of minority students are, in order of group size, Hispanics, Asian/Pacific Islanders, and African Americans. 3. Students are referred to as low-income or as being from low-income families in this report if state records classify them as re ceiving public assistance in the form of Aid to Families with Dependent Children (AFDC) or its successor in California, CalWORKS. 4. Students for whom English is a second language and who are not fully proficient in English are often referred to as limi ted English proficient (LEP), English language learners (ELL), and English learne rs (EL). We use EL throughout this report to reflect the usage in the California law that implemented proposition 227, a proposition passed b y California's voters in 1998 that banned the implementation of bilingual ed ucation except under special parental waiver conditions. 5. Although the trend in scores is not related to leve l of CSR exposure, the size of gains might be sensitive to class size reduction overall. For example, the achievement gains for primary grades were larger th an for upper elementary, where classes remained large. Small cla sses might allow teachers to better implement reforms or to respond more quickly to the incentives of the accountability system. However, w e do not have adequate data to test for effects between grades; we can onl y compare differential amounts of CSR among students in the same grades. 6.ReferencesBohrnstedt, G. W. & Stecher, B. M. (Eds.) (1999). Class size reduction in California: Early evaluatio n findings, 1996–1998. Palo Alto, CA: CSR Research Consortium. Bohrnstedt, G. W. & Stecher, B. M. (Eds.) (2002). What have we learned about class size reduction in California Sacramento: California Department of Education. Finn, J. (1998). Class size and students at risk: What is known? Wha t is next? (No. AR 98-7104). Washington DC: Office of Educational Research and I mprovement. U.S. Department of Education. Finn J. and Achilles, C. (1999). Tennessee’ class s ize study: Findings, implications, misconceptions. Educational Evaluation and Policy Analysis 21, 97-109. Krueger, A. B., and Whitmore, D. M. (1999). The eff ect of attending a small class in the early grades on college-test taking and middle school test results: Evidence from Project STAR. Economic Journal Mosteller, F. (1995, Summer/Fall). The Tennessee st udy of class size in the early school grades. The

PAGE 23

23 of 26 Future of Children 5, 113-127. Nye, B., Hedges, L. V., and Konstantopoulos, S, (19 99). The long-term effects of small classes: A five-year follow-up of the Tennessee Class Size Exp eriment. Educational Evaluation and Policy Analysis 212, 127-142. Stecher, B. M. & Bohrnstedt, G. W. (Eds.) (2000). Class size reduction in California: The 1998-99 evaluation findings Sacramento, CA: California Department of Educatio n. Stecher, B. M. & Bohrnstedt, G. W. (Eds.) (2002). Class size reduction in California: Findings from 1999-00 and 2000-01. Sacramento, CA: California Department of Education Stecher, B. M., Bugliari, D., and McCaffrey, D. F. (2002, February). Achievement. In B. M. Stecher & G. W. Bohrnstedt (Eds.) Class size reduction in California: Findings from 1 999-00 and 2000-01 Sacramento, CA: California Department of Education. Stecher, B. M., McCaffrey, D. M., and Burroughs, D. (1999). Achievement In G. W. Bohrnstedt & B. M. Stecher (Eds.). Class size reduction in California: Early evaluatio n findings, 1996–1998. Palo Alto, CA: CSR Research Consortium. Stecher, B. M., McCaffrey, D. F., Burroughs, D. Wil ey, E., and Bohrnstedt, G. W. (2000). Achievement. In B. M. Stecher & G. W. Bohrnstedt (Eds.) Class size reduction in California: The 1998-99 evaluation findings Sacramento, CA: California Department of Educatio n.About the AuthorsBrian StecherSenior Social ScientistRAND1700 Main StreetPO Box 2138Santa Monica, CA 90407-2138Brian Stecher’s research emphasis is applied educat ional measurement, including the implementation, quality, and impact of state as sessment and accountability systems; the cost, quality, and feasibility of perf ormance-based assessments, and the development and validation of licensing and cer tification examinations. Daniel McCaffreySenior StatisticianRAND201 North Craig Street, Suite 202Pittsburgh, PA 15213-1516Email: daniel_mccaffrey@rand.org Dan McCaffrey's research includes studies related t o education and health policy. His current projects involve value-added modeling f or estimating teacher effects and propensity score methods for comparing nonequiv alent groups in quasi-experiments. He is also interested in nonpara metric methods for estimating the standard errors for models fit to clustered dat a. Delia BugliariSenior Programmer/AnalystRAND1700 Main StPO Box 2138

PAGE 24

24 of 26 Santa Monica CA 90407-2138Delia Bugliari specializes in analysis of education data on student achievement, school demographics, and teacher surveys. Her resea rch interest includes missing data imputation in education and economic d ata. The World Wide Web address for the Education Policy Analysis Archives is epaa.asu.edu Editor: Gene V Glass, Arizona State UniversityProduction Assistant: Chris Murrell, Arizona State University General questions about appropriateness of topics o r particular articles may be addressed to the Editor, Gene V Glass, glass@asu.edu or reach him at College of Education, Arizona State Un iversity, Tempe, AZ 85287-2411. The Commentary Editor is Casey D. Cobb: casey.cobb@unh.edu .EPAA Editorial Board Michael W. Apple University of Wisconsin David C. Berliner Arizona State University Greg Camilli Rutgers University Linda Darling-Hammond Stanford University Sherman Dorn University of South Florida Mark E. Fetler California Commission on TeacherCredentialing Gustavo E. Fischman Arizona State Univeristy Richard Garlikov Birmingham, Alabama Thomas F. Green Syracuse University Aimee Howley Ohio University Craig B. Howley Appalachia Educational Laboratory William Hunter University of Ontario Institute ofTechnology Patricia Fey Jarvis Seattle, Washington Daniel Kalls Ume University Benjamin Levin University of Manitoba Thomas Mauhs-Pugh Green Mountain College Les McLean University of Toronto Heinrich Mintrop University of California, Los Angeles Michele Moses Arizona State University Gary Orfield Harvard University Anthony G. Rud Jr. Purdue University Jay Paredes Scribner University of Missouri Michael Scriven University of Auckland Lorrie A. Shepard University of Colorado, Boulder

PAGE 25

25 of 26 Robert E. Stake University of Illinois—UC Kevin Welner University of Colorado, Boulder Terrence G. Wiley Arizona State University John Willinsky University of British ColumbiaEPAA Spanish and Portuguese Language Editorial BoardAssociate Editors for Spanish & Portuguese Gustavo E. Fischman Arizona State Universityfischman@asu.eduPablo Gentili Laboratrio de Polticas Pblicas Universidade do Estado do Rio de Janeiro pablo@lpp-uerj.netFounding Associate Editor for Spanish Language (199 8-2003) Roberto Rodrguez Gmez Universidad Nacional Autnoma de Mxico Adrin Acosta (Mxico) Universidad de Guadalajaraadrianacosta@compuserve.com J. Flix Angulo Rasco (Spain) Universidad de Cdizfelix.angulo@uca.es Teresa Bracho (Mxico) Centro de Investigacin y DocenciaEconmica-CIDEbracho dis1.cide.mx Alejandro Canales (Mxico) Universidad Nacional Autnoma deMxicocanalesa@servidor.unam.mx Ursula Casanova (U.S.A.) Arizona State Universitycasanova@asu.edu Jos Contreras Domingo Universitat de Barcelona Jose.Contreras@doe.d5.ub.es Erwin Epstein (U.S.A.) Loyola University of ChicagoEepstein@luc.edu Josu Gonzlez (U.S.A.) Arizona State Universityjosue@asu.edu Rollin Kent (Mxico) Universidad Autnoma de Puebla rkent@puebla.megared.net.mx Mara Beatriz Luce (Brazil) Universidad Federal de Rio Grande do Sul-UFRGSlucemb@orion.ufrgs.br Javier Mendoza Rojas (Mxico)Universidad Nacional Autnoma deMxicojaviermr@servidor.unam.mx Marcela Mollis (Argentina)Universidad de Buenos Airesmmollis@filo.uba.ar Humberto Muoz Garca (Mxico) Universidad Nacional Autnoma deMxico Angel Ignacio Prez Gmez (Spain)Universidad de Mlagaaiperez@uma.es

PAGE 26

26 of 26 humberto@servidor.unam.mx DanielSchugurensky (Argentina-Canad) OISE/UT, Canadadschugurensky@oise.utoronto.ca Simon Schwartzman (Brazil) American Institutes forResesarch–Brazil (AIRBrasil) simon@sman.com.br Jurjo Torres Santom (Spain) Universidad de A Coruajurjo@udc.es Carlos Alberto Torres (U.S.A.) University of California, Los Angelestorres@gseisucla.edu EPAA is published by the Education Policy Studies Laboratory, Arizona State University


xml version 1.0 encoding UTF-8 standalone no
mods:mods xmlns:mods http:www.loc.govmodsv3 xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govmodsv3mods-3-1.xsd
mods:relatedItem type host
mods:identifier issn 1068-2341mods:part
mods:detail volume mods:number 11issue 40series Year mods:caption 20032003Month November11Day 1010mods:originInfo mods:dateIssued iso8601 2003-11-10


xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam a22 u 4500
controlfield tag 008 c20039999azu 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E11-00338
0 245
Educational policy analysis archives.
n Vol. 11, no. 40 (November 10, 2003).
260
Tempe, Ariz. :
b Arizona State University ;
Tampa, Fla. :
University of South Florida.
c November 10, 2003
505
Relationship between exposure to class size reduction and student achievement in California / Brian M. Stecher, Daniel F. McCaffrey [and] Delia Bugliari.
650
Education
x Research
v Periodicals.
2 710
Arizona State University.
University of South Florida.
1 773
t Education Policy Analysis Archives (EPAA)
4 856
u http://digital.lib.usf.edu/?e11.338