xml version 1.0 encoding UTF8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchemainstance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam a22 u 4500
controlfield tag 008 c20059999azu 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E1100468
0 245
Educational policy analysis archives.
n Vol. 13, no. 46 (November 23, 2005).
260
Tempe, Ariz. :
b Arizona State University ;
Tampa, Fla. :
University of South Florida.
c November 23, 2005
505
Achievement gaps and correlates of early mathematics achievement : evidence from the ECLS Kfirst grade sample / Madhabi Chatterji.
650
Education
x Research
v Periodicals.
2 710
Arizona State University.
University of South Florida.
1 773
t Education Policy Analysis Archives (EPAA)
4 856
u http://digital.lib.usf.edu/?e11.468
xml version 1.0 encoding UTF8 standalone no
mods:mods xmlns:mods http:www.loc.govmodsv3 xmlns:xsi http:www.w3.org2001XMLSchemainstance xsi:schemaLocation http:www.loc.govmodsv3mods31.xsd
mods:relatedItem type host
mods:identifier issn 10682341mods:part
mods:detail volume mods:number 13issue 46series Year mods:caption 20052005Month November11Day 2323mods:originInfo mods:dateIssued iso8601 20051123
PAGE 1
Readers are free to copy, display, and distribute this article, as long as the work is attributed to the author(s) and Education Policy An alysis Archives, it is distributed for noncommercial purposes only, and no alteration or transformation is made in the work. More details of this Creative Commons license are available at http://creativecommons.org/ licenses/byncnd/2.5/. All other uses must be approved by the author(s) or EPAA EPAA is published jointly by the Colleges of Education at Arizona State University and the University of South Florida. Articles are indexed by H.W. Wilson & Co. Send commentary to Casey Cobb (cas ey.cobb@uconn.edu) and errata notes to Sherman Dorn (epaaeditor@shermandorn.com). EDUCATION POLICY ANALYSIS ARCHIVES A peerreviewed scholarly journal Editor: Sherman Dorn College of Education University of South Florida Volume 13 Number 46 Novemb er 23, 2005 I SSN 1068Â–2341 Achievement Gaps and Correlates of Early Mathematics Achievement: Evidence from the ECLS KÂ–First Grade Sample1 Madhabi Chatterji Teachers College, Co lumbia University Citation: Chatterji, M. (2005) Achievement gaps and correlates of early mathematics achievement: Evidence from the ECLS KÂ–first grade sample. Education Policy Analysis Archives, 13 (46). Retrieved [date] from ht tp://epaa.asu.edu/epaa/v13n46/. Abstract In light of the NCLB Act of 2001, this study estima ted mathematics achievement gaps in different subgroups of kindergartners and first gr aders, and identified childand schoollevel correlates and moderators of early mathematics achievement. A subset of 2300 students nested in 182 sc hools from the Early Childhood Longitudinal Study KÂ–First Grade data set was analyzed with hierarch ical linear models. Relative to school mean estimates at the end of kindergarten, significant mathematics achievement gaps were foun d in Hispanics, African Americans and high poverty students. At the end of Grade 1, mathem atics gaps were sign ificant in African American, high poverty, and female subgroups, but not in Hispanics. Schoollevel correlates of Grade 1 Mathematics achievem ent were class size (with a small negative main effect), athome reading time by pare nts (with a large posit ive main effect) and school size (with a small pos itive main effect). Crosslevel interactions in Grade 1 indicated that schools with larger class and school size s had a negative effect on 1 An early version of this study was supported by the Institute for Urban and Minority Education (IUME), Teachers College, Columbia University during the summer of 2002. The author acknowle dges Professor Edmund W. Gordon, Director of IUME, for the opportunity to begin wo rk on problems and issues on mi nority education. Earlier papers were presented at th e Annual Meetings of the American Evalua tion Association in 2002, and the American Educational Research Association, in 2004. My thanks to A lissa Peltzman and Nancy Koh, currently in the M.A. program in Policy Studies at Teachers College, for their assistance with the paper.
PAGE 2
Education Policy Analysis Archives Vol. 13 No. 46 2 African American childrenÂ’s ma th scores; schools giving mo re instructional time to reading and math had a positive effect on high poverty studentsÂ’ scores, and schools with higher elementary teacher certificati on rates had a positi ve effect on boysÂ’ mathematics achievement. Keywords: achievement gap; early childhood; primary grades; Early Childhood Longitudinal Study (ECLS); correlates and moderators of mathematics achievement. Purpose The passage of the No Child Left Behind (NCLB) Act of 2001 has drawn attention to achievement differentials in diverse U.S students, commonly referred to as the Â“achievement gapÂ”. By law, public schools are now held accountable for equitable achievement outcomes in subgroups of minority versus nonminority, normally achieving versus exceptional, as well as socioeconomically advantaged versus disadvantaged students (P.L 107Â– 110, 115 Stat. 1425, 2002). As a consequence of such school reform legislation, disparities among ch ildrenÂ’s mathematics achievement as well as factors that influence the observed differences, are now of cen tral concern to researchers, practitioners, policymakers and the public alike. Although much research now concludes that gender gaps in mathematics are declining in large scale examinations of adol escents and adults (see for example, Friedman, 1989), documentation is sparse on the mathematics gender gap in early school years. Questions surrounding the time at which mathematics achievement gaps fi rst develop, groups in which they are consistently manifested, and circumstances under which they redu ce or are sustained over time, is of particular relevance today. The isolation of schoolversus ch ildlevel characteristics that potentially narrow observed differences in early mathematics achievement is a needed area of research. This study estimates achievement gaps and corre lates of early mathematics achievement with a particular focus on male versus female, poor versus welltodo and various ethnic subgroups in the U.S. A main aim of the research was to estimate th e size of mathematics achievement gaps manifested in early school years in the contex t of recently set reform agendas for schools to close gaps by 2014. As formal instruction in mathematics tends to be more consistently distributed in first grade rather than in kindergarten, the study focuses mainly on first grade achievement outcomes. At the same time, kindergarten gaps are estimated so that changes in achievement patterns can be meaningfully compared and interpreted. A subset of data from the Longitudinal Kinderga rtenFirst Grade Public Use file of the Early Childhood Longitudinal Study (ECLS) was analyzed. The ECLS, consisting of data collected between kindergarten entry (Fall 1998) and the end of first grade (Spring 2000), uses a nationally representative sample. Particular research objectives that guided th e present study were to determine: (1) the extent to which variability in childrenÂ’s mathematics achievemen t in first grade is explained by childversus schoollevel factors; (2) whether childrenÂ’s membership in a specific subgroup resulted in significant withinschool achievement gaps in kindergarten and subsequently, in first grade, controlling for particular child background characteristics; (3) the degree to which selected, theoreticallysupported or policyrelevant school factors significantly influenced first grade mathematics achievement, when some child background characteristics are controlled while others allowed to vary randomly by school; and (4) whether childrenÂ’s membership in particular eth nic, gender or poverty subgroups interacted with their school membership to affect mathematics scor es, and school factors accounting for the variance in school slopes. Kindergarten education varied based on length of day (half versus full day) in the ECLS sample (Walston & West, 2004). To assay the cumulative effects of KÂ–1 years on endoffirst grade mathematics achievement in selected analytic mo dels, child mathematics measures taken at
PAGE 3
Achievement Gaps and Correlates of Early Mathematics Achievement 3 kindergarten entry (Kentry) were used as indicators of prior achievement. In other models, endofkindergarten mathematics achievement measures (Ke nd) were used to control for differences in prior mathematics learning. The latter effects were interpreted as indicative mainly of childrenÂ’s first grade school experiences on endoffirst gr ade achievement, assuming all children entered with similar levels of prior preparation in mathematics. Results of both series of models are compared. Theoretical Framework Predictors with which to build explanatory mo dels for early mathematics achievement were drawn from a review of a broad literature base th at focused on four areas: the national push for reducing achievement gaps in mathematics; the documented demographic shifts in the U.S. and achievement trends in diverse students; early find ings of the Kindergarten year ECLS sample; and existing evidence on childand schoollevel correlates of achievement. That literature is now discussed, with specific attention devoted to research on early care, gender differences, and particular school practice variables, such as reduced class size, school size and teacher qualifications, and their expected influences on early mathematics achievement. The NCLB ActÂ’s Emphasis on the Achievement Gap The most recent legislative action on the standardsbased reform movement in the U.S., the No Child Left Behind Act of 2001 (NCLB), uses challenging academic standards, coupled with highstakes standardsbased testing and accountability, as its main strategy for fostering school improvement (P.L 107Â–110, 115 Stat 1425, 2002). For the first time in the history of national school reforms, the law places an emphasis on achievement by all groups of students, particularly those who are historically lowachieving, such as ethnic minor ities, socioeconomically disadvantaged or special needs students. To monitor the status of achievemen t gaps and ensure that historically lowachieving students receive much needed attention, NCLB mand ates disaggregated reporting of state test scores. That is, schools must break down results on student performance by relevant subgroups in key subject areas such as reading and mathematics, and attempt to close achievement gaps evidenced in various subcategories by 2014. NCLB provides states with some flexibility in selecting appropriate standardsbased assessments, as well as in defining the mean ing of Â“Adequate Yearly ProgressÂ” (AYP) using performance standards (cutscores) on state tests. However, the Act requires that schools monitor and reduce achievement lags in students belonging in high risk groups by the preset deadline. The lack of comparability in state standards and current practices for monitoring achievement gaps have raised concerns among researchers and policyanalysts (Linn, 2003; Linn, Baker & Betebenner, 2002). The recruitment and retention of qualified teachers in schools is another NCLB strategy for enhancing school and student outcomes (see Section 1119, P.L 107Â–110, 115 St at. 1425, 2002). Each local educational agency supported by NCLB funds must ensure that hired teachers are Â“highly qualifiedÂ”. Proof of teacher qualifications can be in the form of full state licensure or certification in the relevant area of specialization, a Bachelor Â’s degree, professional development, classroom experience, and knowledge of a subject garnered over time. States are allowed freedom in devising ways by which teachers might demonstrate their competency and subject matter knowledge. Nevertheless, schools are expected to be in complia nce by 2005Â–06. Schools in rural areas, where the same teacher might teach more than one subject, an d secondary schools, where higher levels of subject
PAGE 4
Education Policy Analysis Archives Vol. 13 No. 46 4 area knowledge are needed, have been accorded some flexibility in meeting the NCLB requirements ( http://www.ed.gov/nclb/methods/teachers/hqtflexibility.html ). Student Background Characteristic s and Scholastic Achievement Outside legislative mandates, what factors are lik ely to diminish or exacerbate the academic deficits in children in high risk subgroups at school? Academic risk factors that are frequently documented in the clinical and child development literature include ethnicity age, gender, poverty and lack of parental education, family mobility, limi ted English proficiency, along with nutritional and health factors (for examples, see Garmezy, 1993; Garmezy & Masten, 1986; Werner, 1993). It is thus not unreasonable to think that children who are at ac ademic risk due to multiple risk factors, such as membership in particular ethnic groups combined wi th high poverty levels, are likely to start school at a greater disadvantage. However, the same body of literature also recognizes protective factors in childrenÂ’s personal, family, and school backgrounds that foster resilience despite observed child or family risk levels. For example, parenting and family socioeconomic fact ors are viewed as major predictors of childrenÂ’s cognitive and social development because families play a central role in childrenÂ’s lives and both genetic and environmental influences are known to have a combined effect on their scholastic success (see Collins, Maccoby, Steinberg, Hetherington & Borns tein, 2000). Public initiatives, such as Head Start and Title 1 programs were founded to counter the welldocumented negative relationship of childrenÂ’s poverty levels with their cognitive, social and physical development. Influences of prior learning and poverty A fairly recent metaanalysis of 60 studies determined that cognitive and social skills measured in late pr eschool years were predictive of performance in the same domains in the early school years (Laparo & Pianta, 2000). Effect sizes were positive (+0.49) for cognitive and academic predictors. The National Institute of Child Health and Human Development (NICHD Early Child Care Research Network, 2002) also reported results from studies on the positive influences of parenting and childcare centers on childrenÂ’s preacademic skills. A recent longitudinal study of 1000 children showed that higher quality of child care and experience in preschool centertype activities were positive correlates of preacademic skills and language (NICHD Early Child Care Research Network, 2002), with gender, ethnicity, family income, parenting and other background factors controlled. Higher quality of child care pr edicted better preacademic and language outcomes, irrespective of the hours and type of care. Contrary to expectations, however, these researchers were not able to document that better quality child care would be advantageous to children from disadvantaged environments, such as low income households. That is, they did not find significant interaction effects between poverty and quality of care. They concluded that their findings were limited by their use of multiple regression mo dels to test for interactions (p. 159). Influences of ethnic differences A 1994 report published by the U.S. Department of Education (NCES, 1995a) concluded that African American child ren were less likely to be enrolled in preprimary education relative to Whites, and were more likely to be below modal grade for their age in school. The same report showed that WhiteAfrican American gaps in mathematics, reading, and science appeared as early as age 9, and did not narrow with age. Over the years, Hispanic children have also been found to start school with less preschool experience than White children and the achievement gap between these two groups was found to widen considerably by 1993 (17% to 38%) (NCES, 1995b). The HispanicWhite gap was also found to appear at age 9 and persisted through age 17, but there were areas in which the gap seemed to narrow. Evidence from other longitudinal studies of ad olescents suggests that certain subgroups among particular ethnic minorities tend to perform better than other students of the same ethnic origin.
PAGE 5
Achievement Gaps and Correlates of Early Mathematics Achievement 5 Grissmer and colleagues (1994), for instance, foun d that Hispanic White youth demonstrate different achievement gains over time relative to their Hisp anic Black counterparts. Systematic studies on Asian students in the U.S. are rare, possibly because they are perceived to perform well, or as well as the majority ethnic group in U.S. schools. Are there any signs of the ethnic achievement gaps reducing? As the literature suggests, manifestation of achievement gaps in early school years is predicated on background differences in young children, which affect their school readiness levels as well as social and cognitive potential (Boethel, 2004). There is some evidence that earl y care programs such as the Head Start give a significant boost to children from low income environments, particularly African American children (see Lee, BrooksGunn, Schnur, 198 8; HubbsTait et al, 2002). Such ev idence would suggest that with appropriate parenting and preschool care, achiev ement gaps could narrow before children start kindergarten. Lee (2002) observed an overall reducti on in ethnic achievement gaps on the National Assessment of Ed ucational Progress (NAEP) and the Scholastic Aptitude Test (SAT) through the 1980s, and concluded that schools had made Â“great progress toward equityÂ”. African AmericanWhite and HispanicWhite gaps narrowed in the early 19 80s. In the late 1980s and 1990s, however, the performance gaps either stabilized or widened agai n, corresponding with the onset of standardsbased reforms. Using recent NAEP results in reading, mathematic s and writing, some states are demonstrating that the achievement gap is reducing after the pa ssage of NCLB, according to National GovernorsÂ’ Association (NGA) Clearinghouse ( www.subnet.nga.org ). Despite high mobility rates, Department of Defense schools show the smallest gaps between Whi te and African American students, even in fourth grade. Multifaceted approaches to building educ ational programs, use of Â“bestÂ” practices and improvement of teacher quality are identified by the NGA as key factors that alter and influence student achievement outcomes. Gender differences. Gender differences in mathematics are now documented to be small and decreasing over time, but they exist in college en trance examinations (Friedman, 1989; 1995; Hyde, Fennema, & Lamon, 1990). Whether due to socializ ation differences, innate genetic differences, or different spatial aptitudes influenced by combined geneticenvironmental factors, the gender gap in mathematics is one that seems to interest both rese archers and policymakers. FriedmanÂ’s (1995) metaanalysis examining correlational studies of spatial an d mathematics skills in ma les versus females led to the conclusion that the relationship was stronger in males than females in many studies, showing genderspecific relationship patterns that favored ma les. In select samples, however, the relationship between spatial concepts and math ematics ability was stronger in femalesÂ—leading to the conclusion that motivation, career goals, and factors such as socialization in the environment moderate such relationships. The jury is thus still out as to gender differences and moderators of mathematics skills in younger schoolgoing children from developed countries such as the U.S. Schoollevel Correlates of Student Achievement Reviews of literature on standardsbased school reforms (Chatterji, 2002; DarlingHammond, 1998; Knapp, 1997) as well as empirical studies (s ee for examples, Ferguson, 1991; Ferguson & Ladd, 1996; Sirotnik & Kimball, 1999) conclude that high student achievement on standardsbased tests is more likely when teachers are certified to teach and both teachers and school leaders are appropriately trained on the new content standards. In light of NCLBÂ’s emphasis on teacher qualifications and professional development, these facto rs merit continuing evaluation.
PAGE 6
Education Policy Analysis Archives Vol. 13 No. 46 6 Efficient allocation of resources to support particular reforms, such as through the infusion of technology, reduced school and class sizes, alignm ent of classroom instructional practices with new content standards, longer blocks of time dedi cated to subjectspecific instruction, alignment of practices to meet needs of diverse student populati ons, higher levels of parent involvement in childrenÂ’s education, is also expected to influen ce achievement at all levels. School context factors, such as overall poverty levels in the student body, have also been shown to affect leadership behaviors, schooling practices, and organizational culture (H annaway & Talbert, 1993; Hannaway & Kimball, 1998), and in turn, academic outcomes. Discussions of class size as a reform initiative often center on cost effectiveness issues rather than on what schools and teachers actually do with the added resources that could moderate student outcomes when class sizes are small. The experi mental Tennessee STAR project provided evidence that classes with under 20 pupils have positive effects on studentsÂ’ academic performance in early school years (Finn & Achilles, 1999). Molnar et al. (199 9) also examined classroom processes in smaller classes in Wisconsin, documenting that smaller classe s lead to more individualized instruction, with greater variety in instructional activities. A large scal e study in the U.K, conducted by Blatchford and colleagues (2002) also supports to the latter contentionÂ—more individualized attention is evident in smallersize classes, and potentially facilitates an environment in which studentsÂ’ early academic difficulties can be tackled. Does school size matter at the elementary level? Based on existing research from the past decade, effective school sizes at the elementary level were determined to lie at 300Â–400 students (Williams, 1990), making schools of 500 or more st udents too large for educating younger learners. Some recent research syntheses concluded that sma ll schools are better at raising student achievement, especially for minority and low income students. Small schools were also found to increase attendance, teacher satisfaction, and parent and community in volvement (see Williams, 1990; Ayers, Bracey, & Smith, 2000). Although seemingly dated, the Â“effective school sÂ” literature identified a strong school leadership, a focus on academic outcomes (success or ientation and academic press values), a positive and orderly school climate, and teachersÂ’ positive ex pectations for students as educationally effective factors for disadvantaged students (see, for ex ample, Brookover, Beady, Flood, Schweitzer, & Wisenbaker, 1979; Clark, Lotto, & Mc Carthy, 1980; Edmonds, 1979; Rowan, Bossert, & Dwyer, 1983. While this work faced criticisms from methodolog ical and substantive perspectives because studies often failed to arrive at common factors supported by solid statistical evidence (see for example, Purkey & Smith, 1983, Rosenholtz, 1985), some Â“effecti ve schoolsÂ” factors have surfaced in the recent literature on standardsbased reforms, and merit furt her empirical testing at the elementary level. Early Findings from the ECLS The ECLS data, the focus of the present study, permits a unique look at subsets of a nationally representative sample of kindergartners on a range of cognitive, behavioral, and health indices. Zill and West (2001) reported descriptive statistics from the ECLSK data from the 1998Â–99 kindergarten class. Their initial analyses show considerable variation in childrenÂ’s knowledge and skills as they enter school. Descriptive breakdowns of the first year samp le suggest that the variations in achievement were partially related to age, gender, and family risk factors. Family risk was defined based on low maternal education, welfare dependency or poverty st atus, having one parent at home, and having parents whose native language is not English. Child renÂ’s health and behaviors were, in turn, found to be dependent on family risk factors. Disturbingly, 33% of Hispanic students were found to have two or more family risk factors; 27% of African Americ ans fell in the same category; while 17% of Asians
PAGE 7
Achievement Gaps and Correlates of Early Mathematics Achievement 7 had the same profile. In contrast, 6% of White stud ents were categorized in the same family risk category in the ECLSK sample. Consistent with literature in clinical and child development fields, Zill and West (2001) also reported that sociodemographic risk factors, such as si ngle parent families, high levels of poverty, and transiency, were more common among kindergartners from ethnic minorities than among those from White families in the ECLSK sample, and that the atrisk children started with lower scores on cognitive assessments in kindergarten. Given the policy directions set by the standardsbased reform movement and NCLB, the need is high for additional and replicated evidence on di rect and moderating effects of particular school practices and policies in the early school years. New information has begun to emerge with data on the ECLS, but very little is available on schoollevel correlates and moderators of early mathematics achievement in high risk groups emphasized in the NCLB Further, it is not clear as to whether stable achievement gaps in mathematics form as early as in first grade, how large they are, and whether gaps are comparable in magnitude in all high risk subgrou ps identified in the legislation. The present study addressed these unresolved issues. Method Procedures used for identification or constructi on of childand schoollevel variable measures, for setting up a data set with complete informati on on chosen variables, and for testing particular analytic models, are described next. Schooland Childlevel Factors Selected for Study Separate sets of factors were selected at the ch ildversus schoollevel for analysis. At the child level, variables included child development factors (age of the child in months and gender), sociodemographic factors (ethnicity, familyÂ’s socioeconomic status), and cognitive measures taken prior to formal schooling (mathematics achievement scores in the beginning and at end of kindergartenÂ—Kentry and Kend scores). The ethnic minority subsamples of interest in this study consisted of children coded as AfricanAmerican (nonHispanic), Asian, Hispanic (race specified), and Hispanic (race unspecified) students in the ECLS database. The term Â“povertyÂ” refers to students falling at or below the second quintile on the continuous measure of family socioeconomic status used in the ECLS. At the school level, variables included appropriately aggregated context factors (poverty rate, school size), school inputs (mean class size, teacher certification ra tes), organizational practice/policy factors (success orientation and academic press va lues, levels of teacher support for planning and professional development, student attendance rate, inci dence of individualized educational plans (IEPs, an index of exceptional education services), an d several parent involvement factorsÂ–such as, on average, how long parents read to children at home every week; overall ed ucation support at home, parent satisfaction with school activities, and pare ntal involvement in school functions. School means on the child poverty index and prior achievement were built into the models as controls in some HLMs. Initially, all theoretically relevant variables wer e incorporated in the models; in the final models, variables with negligible and/or statistically nonsi gnificant effects or excessive missing data were dropped from the analyses.
PAGE 8
Education Policy Analysis Archives Vol. 13 No. 46 8 Variables/Measures: Operational Definitions Table 1 provides descriptive statistics on all va riables used in the analyses at the child and school levels. The operational definitions of differe nt constructs, their theoretical and psychometric bases, and coding methods are described briefly next, and detailed in Appendix A. Table 1 Descriptive Statistics on Level 1 and Level 2 Variables Variable name N Mean SD Minimum Maximum Level 1: Child Characteristics Age in months 2300 79.89 4.10 70.13 94.77 Math score at Kentry, C1RMSCAL 2300 20.75 7.18 7.18 56.91 Math score at Kend, C2RMSCAL 2300 29.26 8.51 9.14 57.25 Math score at Grade 1, C4RMSCAL 2300 45.06 8.37 12.20 60.50 African American (%) 2300 0.12 0.33 0 1 Hispanic1 (%) 2300 0.06 0.25 0 1 Hispanic2 (%) 2300 0.05 0.22 0 1 Asian (%) 2300 0.03 0.18 0 1 Poverty (%) 2300 0.29 0.46 0 1 Gender (Male=1; %) 2300 0.49 0.50 0 1 Level 2: School Characteristics Number of children nested in schools182 16.74 4.76 10 33 Teacher support 182 14.30 2.90 4 20 School success orientation 182 12.84 1.64 8 15 Student attendance rate 182 97.46 1.14 93.02 99.73 Education support at home 182 10.85 0.94 7.92 13.90 Parent satisfaction 182 6.23 0.77 4.84 10 Class size 182 21.50 4.74 11.84 52 Class time to reading and math 182 3.82 1.06 2 8 School size 182 0.85 0.36 0 1 Public versus private sector 182 0.89 0.31 0 1 Individualized Educational Plans 182 5.45 9.65 0 67.94 At home reading time 182 0.58 0.17 0.20 1 Parent involvement 182 6.65 0.80 5.10 9 Teacher certification rateEl ementary 182 0.79 0.28 0 1 Schoollevel controls Kend Math 182 33.26 5.55 19.80 48.02 Kentry Math 182 22.83 4.55 12.81 38.26 Poverty rate 182 38.57 29.16 0 100 Minority rate 182 0.33 0.34 0 1
PAGE 9
Achievement Gaps and Correlates of Early Mathematics Achievement 9 The ECLS mathematics assessment The ECLS mathematics assessment is a multilevel, interviewbased assessment that was administered in two stages, using an adaptive design. First, children received a 12Â–20 item routing test. Th eir performance on the routing test determined the second stage form that would be at the appropriate di fficulty level for different children. For individual children, scores on the full set of test items wer e estimated using Item Response Theory (IRT) procedures. IRT equating methodology was used to place all items on a common scale, making scale scores comparable across children at different leve ls, regardless of the items to which they responded. The item content of the ECLS mathematics assessment is as follows: number and shape (identifying onedigit numerals, recognizing geometr ic shapes, and onetoone counting of up to 10 objects); relative size ( reading singledigit numbers, counting beyond 10, recognizing a sequence of patterns, and using nonstandard units of length to compare objects); ordinality and sequence (reading 2digit numerals, recognizing the next number in a sequence, identifying ordinal position in a sequence, and solving a simple word problem); addition and subtraction (solving simple addition and subtraction problems); and multiplication and division (solving simple multiplication and division problems and recognizing more complex number patterns). Reliability of scores from the ECLS mathematics assessments are reported to be in the following ranges: .88Â–.95, for ability estimates on IRT scale scores; and .78Â–.88, for the routing tests (NCES et al., 1999). First grade mathematics ou tcome measure (C4RMSCAL ). IRTscaled scores from the ECLS mathematics assessment, taken at the end of firs t grade, were used a dependent variables in HLM equations after their distribution properties were examined in the larger ECLS sample. Prior achievement in mathematics (C1RMSCAL, C2RMSCAL). IR Tscaled scores from the ECLS mathematics assessment, taken either at the be ginning (Kentry) or end of kindergarten (Kend), were used as childlevel predictors or as dependen t variables after their distribution properties were examined in the overall ECLS sample with all available cases. Both the kindergarten measures were preferred over prior mathematics measures taken at the beginning of first grade (C3RMSCAL), as about 2/3 of the original sample had missing data on this measure. A possible summer lag between the end of kindergarten and first grade was anticipated to have been made up by the end of the Grade 1 year. Survey constructs Composite measures were created with selected item sets from different ECLS questionnaires to serve as predictors. Indices created with childlevel data were aggregated by school, and treated as schoollevel factors. The desc riptions of each survey construct, data sources from which they were extracted, and results of vario us psychometric analyses on variables, are detailed in Appendix A. Exploratory factor analytic work (principal components analysis followed by varimax rotation) was done to extract relatively independent but th eoretically meaningful factors. Internal consistency reliability estimates of factorsupported composites were obtained using Cron bachÂ’s alpha prior to incorporating the index scores into HLM equations as childor schoollevel factors. Itemfactor loadings of .40 and above, and CronbachÂ’s alpha esti mates of .70 were used as criteria for psychometric defensibility of indices. All available cases in th e ECLS database were used for various psychometric analysesÂ—the range of cases varied from 4,637 to 11,379 for these examinations. Unique variable definitions In this study, the Â“special educationÂ” service variable refers to the incidence or percent of IEPs, aggregated by school. That is, the percents of children in the study sample were those identified by their schools as havi ng IEPs on file. Appendix B, Table B3 provides the frequency breakdown on the ECLS survey item th at served as the data source (U4IEP) for this variable. The unique variable definition yields lower percents than national estimates of students with disabilities (typical range from 10Â–15%). Elsewher e, Walston and West (2004) reported the percent
PAGE 10
Education Policy Analysis Archives Vol. 13 No. 46 10 representation of students with disabilities in th e larger ECLS sample was 12% (2568 out of 21260), close to the national estimate. Likewise, the Â“povertyÂ” variable is not based on student enrollment in free or reduced lunch programs at schools (as found in many policy studies), but rather the ECLS composite for socioeconomic status computed for individual children ba sed on parent education, occupation and income (see Appendix A). The lower two quintile ranges in this continuous SES measure were selected as a cut to identify Â“high povertyÂ” students. The percents in the original ECLS sample and initially screened study sample were thus at or around 40%; the final 2300 cases had a lower poverty rate (29%) by this definition (see Table B1 in Appendix B). However, when schoollevel aggregates were computed, the mean poverty rate across schools was abou t 39% (see Table 1, lower panel). Finally, Â“Reading TimeÂ” given by parents at hom e was based on an individual item that was separate from the items used to construct remaining pare nt involvement composites from the Parent Questionnaire. This item asked how long parents read to their child at home (more than 30 minutes per week versus less, a (1/0) binary variable). The data were incorporated in the models as a schoollevel aggregateÂ–a proxy for family care/parenting variable shown to influence future academic performance of children. Prior to running the HLMs, the relationship of th is variable with other schoollevel aggregates, such as a schoolÂ’s aggregated poverty rate across sampled children, was examined to identify possible suppressor effects (see Appendix B, Table B2, for a correlation table). The magnitude of these overlaps was judged to be too small to cloud subsequent interpr etations of results with multiple predictors. As is evident, Reading Time, averaged by school, correlated .087 (n.s.) with the Poverty rate, and .18 ( p < .05) with the Educational Support variable. The latt er correlation suggested that schools with higher levels of parental support for education also had hi gher athome reading time values. Several of the correlations in Table B2 were similarly meaningful and were formally tested with HLMs. Sample Size and Composition Utilization of multiple data elements and composite variables necessitated a careful examination of missing data in the data set prep ared for analysis. Some decisions were made on how best to retain an optimal sample size with as much complete data as possible on variables that were relevant to the research objectives. Normalized sa mple design weights using the C124PW0 variable were applied, as recommended by ECLS staff, to retain original degrees of freedom. To start, all graderetained children were scr eened out by including only those who were firsttime kindergartners (P1FIRKDG=1) and firsttime first graders in traditional classrooms and schools (T4GLVL=4) with data available on the weighting variable. Next, new variable measures were constructed as described previously and a childlevel file was created using all cases with complete data on gender, poverty, race, and prior achievement and ou tcome indicators. Lastly, a schoollevel file was created, aggregating all school factors to be tested in HLMs, including at least 10 students within each school. Missing values on 6 constructs were impute d with the mean substitution procedure in SPSS (detailed at the end of Appendix A). The final data set for the present analysis yielded 182 schools with 2300 cases. The composition of the screened study sample with weighted and unweighted cases, compared to the original ECLS sample and the final data set, is described in Appendix B, Table B1 on key demographic variables.
PAGE 11
Achievement Gaps and Correlates of Early Mathematics Achievement 11 Descriptive Statistics on Child and Schoollevel Variables The top panel in Table 1 shows descriptive statistics on childlevel (Level 1) variables in the dataset ( N =2300). As is evident, 12% of the sample were Blacks, 6% belonged to the Hispanic1 group (race specified), 5% were Hispanic2 (ra ce unspecified), and 3% were Asian children. The children had a mean age of 79.89 months at the be ginning of first grade; 29% were from high poverty households; and about half (49%) wer e male. At the school level, the number of children nested in schools varied from 10Â–33, with a mean of 17. This number is different from the Â“class sizeÂ” variable and reflects children sampled by school per the ECLS sample design, where on average, 5 children and 3 classrooms were sampled from each school. Analytic Models A series of twolevel models, with childrenÂ’s achievement modeled at the childlevel, nested under schools at the second level, were run with the HLM Version 5.02 program (Raudenbush, Bryk, Cheong & Congdon, 2001). The research ra tionale for each HLM was as follows. Unconditional (null) model The first analysis involved the use of a oneway random effects ANOVA, also called the unconditional or null model. This analysis was motivated by the need to partition the total variance in achievement into withinand betweenschool components. The variance estimates were obtained by fitting an HLM where each childÂ’s endoff irst grade achievement score, yij, is explained by the estimated school mean, j0 and unique error associated with that child, rij. School means were explained by the grand mean, G00 and unique error for each school, uj. yij = j0+ rij. j0 = G00 + uj. The analysis with the null model yielded answer s to three questions: How much do individual students vary around their sc hool means ? How much of to tal variance in mathematics achievement is attributab le to schools? and How precise an esti mate of the population mean is the school mean, j0? These questions were answered by ex amining the variance estimates within schools ( 2) and between schools ( ), and the size of the intraclass correlation (proportion of total variance that is be tween schools). ICC values greater than .10 indicate that there are sufficient withinschool dependencies to justify multilevel analysis. A reliability es timate for school mean estimates (intercepts, j0), is reported in the HL M output. A criterion of .60 was set for the reliability of the intercepts. Subsequent models incorporating predictors at both levels were evaluated against these initial variance estimates. Random intercepts model with only childlevel predictors The next series of HLMs was specified to answer two main questions: Given the estimated withinschool variance, what proportion of that variance in achievement can be accounted for by child background characteristics, such as their age, prior achievement in math gender, poverty, and ethnicity ? Compared to school mean estimates and controlling for other child background characte ristics, how large are the achievement gaps in selected ethnic, poverty, and gender groups? The sc hoollevel model remained as in the null model; predictors were all entered in the child level equation. All predictors were centered around their school means; thus, the estimated coefficients for each risk group showed the withinschool achiev ement differential, controlling for the other child
PAGE 12
Education Policy Analysis Archives Vol. 13 No. 46 12 background characteristics serving as predictors in the models but allowing variability between schoolsÂ—a primary research objective for the study. yij = j0+ j1 ( African American)+ j2 ( HispanicRace specified) + j3 ( HispanicRace unspecified) + j4( Asian) + j5( Prior Achievement) + j6( Poverty) + j7( Male) + j8( Age)+ rij. j0 = G00 + uj.. j1 = G10 j2 through j8 ( all slopes fixed as in j1). Gaps were estimated at the end of kindergarten and in Grade 1, using both the Kentry and Kend mathematics measures as prior achievement indicators in separate models. Findings were compared. The models were evaluated against the null model by examining the proportion of unexplained withinschool variance that was accounted for, after all the childlevel predictors were included in the model. Random intercepts model with child leve l covariates and schoollevel predictors In the third series of HLMs, theoreticallysupported school variables were modeled to explain betweenschool variability in achievement, with childlevel predictors also entered. Effects of childlevel predictors that were main focus of the study, ethnicity, gender, and poverty were again allowed to vary randomly between schoolsÂ—that is, they were group (school) meancentered. School aggregates on poverty and prior achievement were reentered in second level eq uations as context controls to study effects of other school factors, adjusted for these average eff ects. Remaining childlevel factors, such as age, were now centered around the grand mean, and served as a constant across schools. Again both the Kentry and Kend mathematics measures were used as covari ates at the childlevel in different runs to compare findings. yij = j0+ j1 ( African American)+ j2 ( HispanicRace specified) + j3 ( HispanicRace unspecified) + j4( Asian) + j5( Prior Achievement)+ j6( Poverty) + j7( Male) + j8( Age)+ rij. j0 = G00 + G01( Mean Poverty) + G02(Mean Prior Achievement)+ G03 (School Size) + G04 (Mean Class Size)+ G05(Mean Teacher Variables) + G06 (Mean Organizational Variables) + G07 ( Mean Reading Time) + *G08 (Mean Parent Involvem ent Variables) + uj, where j1 = G10 j2 through j8 (all slopes fixed as in j1). (* indicates that there is more than on e coefficient in this predictor category.) This series of HLMs revealed the degree to wh ich the selected school factors significantly and positively influenced first grade mathematics achiev ement in schools, as main, additive effects. The reduction in the variance estimate of uj enabled a calculation of the pr oportion of originally estimated betweenschool variance (from the Null Model) that could be explained by the school factors chosen, and an evaluation of the usefulness of models. A co mparison of models examining Kentry to Grade 1 versus Kend to Grade 1 effects was also made. Slope parameters for predictors at the child level were again fixed; that is, the influence of individual child level characteristics was not set to vary by school.
PAGE 13
Achievement Gaps and Correlates of Early Mathematics Achievement 13 Random intercepts and slopes models, with explanatory variables to examine crosslevel interactions Finally, to answer questions as to whet her mathematics achievement varied due to interaction of a childÂ’s risk group membership mo deled in the childlevel equation and schools in which they belonged, modeled at the secondlevel, slopes for these predictors were next allowed to vary randomly. Here, only the Kend mathematics me asure was used as a covariate. When statistically significant slope variance was found, the equation for that slope parameter was modeled with schoollevel predictors to identify significant explanatory variables. For example, with the poverty variable, the questions were: Does mathematics achievement vary significantly in poor versus wellto do children who belong in different schools? If so, what schoo l variables significantly explain the achievement variance in the slopes? The equations were: j6( Poverty) = G60+ u6, to examine if the poverty slope, j6, had significant variance when modeled as a random variable. To identify significant explanatory variables, the subsequent equation was built as follows: j6 = G60+ +G61 (Mean Minority)+ G62( Mean Poverty) + G63 (Mean Prior Achievement) +G64( School Size) +G65( Mean Class Size) + *G66 (Mean Teacher Variables) + *G67 (Mean Organizational Variables) *G68 (Mean Parent Involv ement Variables)+ u6. These analyses were pursued with one slope mo deled at a time. The number of schools with necessary data at the school leve l dropped in the crosslevel models and are reported in tables with results. The reliability of the slopes and intercepts was checked at each stage of the analysis and are also reported. Results The results of the final models are presented in Tables 2Â–10. A few predictors that did not significantly contribute to the variance of achiev ement (such as early childhood certification rate) were dropped from reduced models. Child versus School Variance Comp onents in First Grade Mathematics How much variability in first grade mathematics achievement could be attributed to children versus schools? The results with the unconditional model for mathematics are presented in Table 2. Children were found to vary significantly around thei r school means, as evidenced in the statistically significant t value. The withinschools variability in achi evement was estimated at 66.06; the betweenschools variability was estimated at 16.37. This yiel ded an intraclass coefficient (ICC) of .198, showing that about 20% of the total variability in mathematic s achievement could be attr ibuted to schools (i.e., betweenschool variance). The estimated school mean was 44.28 ( SE =0.34). The reliability of this estimate was .76.
PAGE 14
Education Policy Analysis Archives Vol. 13 No. 46 14 Table 2 First Grade Mathematics Va riance Partitioned to Students versus Schools: Results from Null Model Variable Parameter estimates d.f. p Fixed Effect Coefficient SE t School meana, G00 44.28 0.34 129.30 181 < .01 Random Effect Variance Component 2 School level effect, uoj 16.37 831.95181 < .01 Child level effect, rij 66.06 a Reliability of intercept (school mathematics mean estimates at end of Grade 1) = .76 Variance Attributable to Schools (ICC) = [16.37/(16.37+66.06)]= .20 Mathematics Achievement Gaps in Kindergarten and First Grade What were the withinschool achievement different ials in kindergarten and first grade of different ethnic, gender and poverty groups, adjusting for Kentry versus Kend variability in childrenÂ’s mathematics achievement? How much of the withinschool variance in first grade achievement was accounted for by the chosen predictors? Tables 3Â–5 provide the estimates on the size and significance of mathematics achievement gaps and childlevel pred ictors. Figures 13 present a pictorial view of the achievement gaps. Table 3 Kindergarten Achievement Gaps: Results from Random Intercepts Model with Level 1 Predictors and KEntry Measures of Prior Achievement Variable Parameter estimates d.f. p Fixed effects Coefficient SE t School meana, G00 28.20 0.35 80.58 181 < .01 Withinschool effects Age in months 0.02 0.03 0.66 2291 .51 Poverty Status Gap 0.56 0.25 2.23 2291 .03 African American Gap 1.55 0.45 3.40 2291 < .01 Hispanic1 (Race sp.) Gap 0.14 0.49 0.28 2291 .78 Hispanic2 (Race Not sp.) Gap 0.90 0.46 1.95 2291 .05 Asian Gap 0.01 0.54 0.02 2291 .99 Gender Gap (Male vs. Female) 0.08 0.24 0.36 2291 .72 Achievement at Kentry 0.92 0.02 42.00 2291 < .01 Random effects Variance Component 2 School level effect, uoj 20.43 2276.48 181 < .01 Child level effect, rij 24.61 a Reliability estimate of intercept = .91 Withinschool variance accounted for by Leve l 1 predictors: [(66.0824.61/66.08)]= .64. It should be noted that Â“achievement gapÂ” in Tables 3Â–5 are the multilevel regression coefficients. They are interpreted as the achievem ent difference of a selected subgroup (e.g., African American vs. others) as compared to the estima ted school mean (the intercept), controlling for prior achievement and other demographic characteristics.
PAGE 15
Achievement Gaps and Correlates of Early Mathematics Achievement 15 Table 3 show that achievement gaps at the end of kindergarten, controlling for Kentry mathematics achievement, are significant in high poverty, African American, and Hispanic2 (race not specified) subgroups. The school mean is estimated at 28.20. A high poverty child is estimated to score lower than the school mean by 0.56 units ( p < .05); an African American child by 1.55 units ( p < .001), and Hispanic2 children by 0.90 ( p =.05). There is no gender gap evident. Children who have higher mathematics scores at the start are predicted to do si gnificantly better at the en d of kindergarten with a coefficient of +.92 ( p < .01), controlling for other factors within schools. About 64% of the variability in childrenÂ’s mathematics achievement at the end of kindergarten is explained by the predictors modeled. Table 4 coefficients show that significant mathematics gaps are manifested again in first grade in African American children versus other ethnic groups (2.01 points, p < .01) and in high poverty versus welltodo children (1.72, p < .01), but not for the Hispanic subgroups. There is now a small but significant gender gap, with males s howing an advantage over females (+0.53, p < .10). Kentry mathematics scores are modeled with other predicto rs as covariates. The withinschool mean estimate is close to the estimate in Table 1, 44.96 ( SE =0.36). About 43% of the variability in childrenÂ’s mathematics achievement in Grade 1 is explained by the predictors modeled, a lower proportion than that in Table 3 possibly because schooling and other background factors begin to influence mathematics outcomes more heavily in Grade 1 than at Kend. Table 4 First Grade Mathematics Achievem ent Gaps: Results from Random Intercepts Model with Level 1 Predictors and Kindergarten Entr y Measures of Prior Achievement Variable Parameter estimates d.f. p Fixed effect Coefficient SE t School meana, G00 44.96 0.36 122.15 181 < .01 Withinschool effects Age in months 0.05 0.03 1.33 2291 .18 Poverty Status Gap 1.72 0.38 4.49 2291 < .01 African American Gap 2.01 0.54 3.70 2291 < .01 Hispanic1 gap (Race specified) 0.64 0.49 1.30 2291 .19 Hispanic2 gap (Race not specified 1.00 0.69 1.43 2291 .15 Asian Gap 0.48 0.65 0.74 2291 .45 Gender Gap (Male vs. Female) 0.53 0.27 1.89 2291 .05 Achievement at Kentry 0.75 0.03 23.05 2291 < .01 Random Effect Variance Component 2 School level effect, uoj 14.63 1178.73 181 < .01 Child level effect, r ij 37.77 a Reliability estimate of intercept = .83 Withinschool variance accounted for by Leve l 1 predictors: [(66.0637.77)/66.06] = .43. With Kend mathematics scores used as prior achievement measures in Table 5, the gaps reduce marginally in size but remain statistica lly significant at the .05 alpha level in the same groups of children. The observed change in gap estimates is small, but the effects of the kindergarten yearÂ’s experiences on Grade 1 mathematics achievement are evidenced in the 10% difference in achievement
PAGE 16
Education Policy Analysis Archives Vol. 13 No. 46 16 variance explained. In all, 53% of the withinschool variance in achievement is accounted for by the childlevel predictors in the second model. Th e schoollevel mean in first grade mathematics achievement is again close at 44.71 ( SE =0.36). The reliability of the school mean estimates in both first grade models is high at .83 and .87, respectively. Table 5 First Grade Mathematics Achievem ent Gaps: Results from Random Intercepts Model with Level 1 Predictors and Kend Measur es of Prior Achievement Variable Parameter values d.f. p Fixed effect Coefficient SE t School meana, G00 44.71 0.36 121.92 181 < .01 Withinschool effects Age in months 0.03 0.03 0.90 2291 .37 Poverty Status gap 1.40 0.33 4.19 2291 < .01 African American gap 1.24 0.50 2.48 2291 .01 Hispanic1 gap (race specified) 0.39 0.52 0.76 2291 .44 Hispanic2 gap (race not specified 0.53 0.66 0.79 2291 .42 Asian gap 0.53 0.54 0.99 2291 .32 Gender gap (M ale =1) 0.49 0.25 1.95 2291 .05 Achievement at Kend 0.70 0.01 36.36 2291 < .01 Random Effect Variance Component 2 School level effect, uoj 16.19 1524.32 181 < .01 Child level effect, r ij 30.84 a Reliability estimate of intercept = .87 Withinschool variance accounted for by Leve l 1 predictors: [(66.0630.84)/66.06] = .53. As expected, gaps are slightly smaller in magnitude in Table 5 than those presented in Table 4. This is because childrenÂ’s initial mathematics variability is now statistically controlled using mathematics measures taken at Kend, just prior to their entry into first grade. The withinschool Grade 1 results indicate the following (Table 5): an increase in childrenÂ’s age by one month marginally drops the school mathematics mean by 0.03 units ( p =.37), childrenÂ’s membership in the high poverty group (poverty status=1, versus others) signific antly decreases the mean by 1.40 units ( p < .01). A male child, versus females, scores 0.49 unit s above the estimated school mean ( p =.05) in mathematics. An African American child, versus others, is estimated at scoring 1.24 units below the estimated school mean ( p < .01). Hispanic1 children (race specified) score 0.39 units above; Hispanic2 (race unspecified) children score 0.53 units below; while an Asian child is estimated to score 0.53 units below the school meanÂ—but none of the achievement di fferentials for the last three ethnic groups are statistically significant at the 10% error level once Kend mathematics variability is controlled. Finally, for every unit increase in the mathematics achievemen t score at the end of kind ergarten, a first grader scored 0.70 units higher on the firs t grade measure of mathematics ( p < .01).
PAGE 17
Achievement Gaps and Correlates of Early Mathematics Achievement 17 28.2 26.65 28.34 27.3 28.21 25.5 26 26.5 27 27.5 28 28.5 School MeanAfrican American Hispanic (Race sp.) Hispanic (Race Not sp.) Asian Ethnic Group PerformanceMath Score Figure 1. End of Kindergart en Math Achievement Gaps Contr olling for KEntr y Achievement. 44.96 42.95 45.6 43.96 44.48 41.5 42 42.5 43 43.5 44 44.5 45 45.5 46 School MeanAfrican American Hispanic (Race sp.) Hispanic (Race Not sp.) Asian Ethnic Group PerformanceMath Score Figure 2. First Grade Math Achievement Gaps Controlling for KEntry Achievement. 44.71 43.47 45.11 44.1844.17 42.5 43 43.5 44 44.5 45 45.5 School MeanAfrican American Hispanic (Race sp.) Hispanic (Race Not sp.) Asian Ethnic Group PerformanceMath Score Figure 3. First Grade Math Achievement Gaps Controlli ng for KEnd Achievement. SchoolÂ–Level Correlates of Mathematics Achievement Which school practice or policy variables influe nce mathematics achievement of first graders? The results are presented separately in Tables 6Â–7 for models that used Kentry versus Kend measures of mathematics as covariates in the Level 1 models, and show consistent results, with one exception. In
PAGE 18
Education Policy Analysis Archives Vol. 13 No. 46 18 all, 75Â–79 % of the estimated betweenschool variance reported in Table 2 (16.37), was explained by the modeled school level predictors, showing the utility of the models (see table notes). Table 6 Main Effects of School Variables in Level 2: Results of HLM with Level 1 Covariates and KEntry Measures as Controls Variable Parameter estimates d.f. p Fixed school effect Coefficient SE t School meana, G00 44.47 0.28 154.62 166 < .01 Fixed school effects Class size 0.05 0.03 1.64 166 .09 Frequency of Individualized educational plans 0.02 0.02 0.95 166 .34 Student attendance rate 0.00 0.23 0.01 166 .99 Parental support for educat ion 0.28 0.23 1.23 166 .21 Parental satisfacti on with school 0.05 0.25 0.23 166 .81 School size 0.95 0.55 1.71 166 .08 Public school 0.36 0.64 0.57 166 .56 Time parents read to children at home 1.73 1.04 1.65 166 .09 Parent Involvement leve ls 0.25 0.37 0.69 166 .48 Teacher support 0.08 0.07 1.24 166 .21 School success orientation 0.16 0.10 1.57 166 .11 % Teachers with Elementary Certification 0.51 0.61 0.84 166 .40 Class time dedicated to reading and math 0.24 0.18 1.28 166 .19 Context Controls at Sc hoollevel % Low SES students 0.002 0.01 0.19 .84 KEntry Achievement 0.68 0.07 9.48 166 < .01 a Reliability estimate for intercept = .55 Betweenschool variance in achievement accounted fo r by school predictors: [(16.373.41)/16.37)]=.79. In Table 6, effects are estimated following childrenÂ’s exposure to kindergarten and first grade experiences cumulatively, equalized across school s on Kentry mathematics scores but varying on ethnicity, poverty and gender. Consistent with prior research, class size has a small negative and significant effect at the 10% error level (0.053, p < .10), showing that with average increases in teacherreported numbers of children in classrooms, ther e is a small drop in school mathematics means. Because the effects are estimated on school aggregates the influences are smaller in magnitude than may be obtained for individual childrenÂ’s scores. A large positive correlate is the schoolÂ’s average on parentreported athome reading time. Here, a unit change (30 minutes or more versus less) in the predictor results in gains of 1.73 unit s on school mathematics achievement means ( p < .10). School size is the third statistically significant correlate at the 10% level, but in an unexpected direction. Operationalized as 500+ versus less than 500 students (a binary variable), a value of 1 resulted in 0.95 point increase in school mathematics means, indi cating that larger school sizes yielded higher mathematics scores in schools.
PAGE 19
Achievement Gaps and Correlates of Early Mathematics Achievement 19 Table 7 Effects of School Variables: Resu lts of HLM with Level 1 Covariat es and Kend Measures of Prior Achievement Variable Coefficient SE t d.f. p Fixed effect School meana, G00 44.30 0.26 165.97 166 < .01 School effects Class size 0.05 0.03 1.65 166 .09 Frequency of Individualized educational plans 0.01 0.02 0.92 166 .35 Student attendance rate 0.00 0.23 0.03 166 .97 Parental support for educat ion 0.30 0.23 1.31 166 .19 Parental satisfacti on with school 0.05 0.25 0.20 166 .84 School size 0.89 0.54 1.64 166 .09 Public school 0.45 0.63 0.72 166 .47 Time parents read to children at home 1.71 1.06 1.61 166 .10 Parent Involvemen t levels 0.34 0.37 0.90 166 .36 Teacher support sy stems 0.09 0.07 1.27 166 .20 School success orientation 0.17 0.10 1.66 166 .09 % Teachers with Elementary Certification 0.57 0.60 0.95 166 .34 Class time dedicated to reading and math 0.24 0.18 1.28 166 .19 Context Controls in schoollevel equations: % Low SES students (school aggregate) 0.00 0.01 0.40 .68 Kend Achievement 0.69 0.07 9.33 166 < .01 a Reliability estimate for intercept = .63 Betweenschool variance in achievement accounted fo r by Level 2 predictors: [(16.674.00)/16.67)]=.76. In Table 7, children vary in terms of gender, ethnicity, and poverty, and are now equalized on prior mathematics preparation at Kend. Thus, pupils can be expected to be less variable at the end of Grade 1 than the model in Table 6 and only Grade 1 effects are estimated. Again, class size, reading time at home and school size variables have similar and significant effects, in the same directions. In addition, a counterintuitive result is obtained with the school success variable, where unit increases on the composite resulted in small drops in school means by 0.18 points ( p < .10). The other school predictors do not have significant main effects. Interaction of ChildÂ’s African A merican Status with Schools Because African American children were found to have statistically significant gaps, a further question dealt with whether their performance varied by school when Kend measures were controlled. Other ethnic group interaction analyses were prohi bited by a reduction in the number of schools available for analysis that included children from Asian and Hispanic subgroups.
PAGE 20
Education Policy Analysis Archives Vol. 13 No. 46 20 Table 8 Crosslevel Interaction Effects of African Americ an Group Membership and School on First Grade Mathematics Variable a Parameter values d.f. p Random effects Variance Component 2 School level intercepts, uoj 3.51 138.5057 < .01 African American slopes, u3j 4.36 82.6272 .18 Child level effect, r ij 37.44 Fixed effects Coefficient SE t Intercept for Af. Am. slope 0.59 0.43 1.38 166 .16 Class size 0.18 0.11 1.62 166 .10 % Low SES students 0.06 0.02 2.85166 < .01 School size 2.13 1.101.93166 .05 Parent involvement 1.81 0.802.25166 .05 a Reliability estimate of intercept = .46; Reliability estimatee of slope = .19 Variance explained in slopes = [ (4.361.29)/4.36] = .70. Table 8 shows that there was no significan t crosslevel interaction between a childÂ’s membership in African American versus other ethni c group and their school, as evidenced in the results on the slope parameter. The intercept fo r the African American slope was 0.599 ( p =.16), a statistically nonsignificant and somewhat lower value than the initial gap estimates in Tables 35. The initial variance estimate for school slopes was 4.36 ( p =.18). The lack of a significant interaction in this model was based on 73 schools only (see d.f. and chi squared value), and suggested that while African American childrenÂ’s mathematics achievement varied, it was not significantly different based on school membership. Because initial deficit estimates were significan t in this subgroup, an explanatory model was built to isolate school factors that accounted for wh atever school slope variance was manifested in the data with 73 schools. These results showed that wi th increased class sizes, African American children scored significantly lower by 0.18 units, over an d above the initial deficit estimate of 0.599 units ( p =.10.); with increased school sizes, likewise, th ey scored significantly lower by 2.13 units ( p =.05); in high versus low poverty schools their performance remained much the same (.06, p < .01); in schools with higher parent involvement levels, thei r performance was significantly lower (1.81, p =.06)Â—a counterintuitive finding. These variables dropped th e initial variance estimate to 1.29, accounting for 70% of the variability evidenced in school slopes. Interaction of ChildÂ’s Poverty Status with Schools Table 9 presents the results on the second interaction question showing that mathematics performance of high and low poverty children varied significantly by school (variance estimate of slope, u7j =5.51, p =.000). The intercept indicated that acro ss all schools, high poverty children were scoring 1.01 units lower compared to low poverty children (compare with gap estimates in Tables 35). Significant explanatory variables for slope vari ability were school averages on total time teachers gave per day to reading and mathematics instru ction (30Â–60 minutes per day versus less) with a positive coefficient of +0.54 ( p =.08); IEPs in school, with a small negative coefficient of 0.06 ( p < .10); and public versus private sector with a larger negative effect of 2.60 ( p =.03). The last result showed that public schools fare worse than pr ivate/other schools in affecting mathematics performance of high poverty students. In this analys is, schools with higher poverty rates scored as well
PAGE 21
Achievement Gaps and Correlates of Early Mathematics Achievement 21 as those with lower poverty rates (+0.03, p =.03). The intercept reliability was found to be .47, and initial slope reliability was 0.31, with 10% of vari ance in slopes accounted for by the school factors modeled (final variance estimate=4.97). Table 9 Crosslevel Interaction Effects of Poverty Gr oup Membership and Sc hool on First Grade Mathematics Variablea Parameter estimates d.f. p Random Effect Variance component 2 School level intercepts, uoj 3.54 258.56 137 < .01 School poverty slopes, u8j 5.51 222.14 152 < .01 Child level effect, rij 29.88 Fixed Effect Coefficient SE t Intercept for Poverty Slope 1.01 0.34 2.94 166 < .01 Class time to Reading and Math 0.54 0.31 1.72 166 .08 Individualized Educational Plans 0.06 0.03 1.69 166 .08 Public school 2.60 1.23 2.14 166 .03 % Low SES Students 0.03 0.02 1.75 166 .07 a Reliability estimate of intercept = .47; Reliability estimate of slope = .31 Variance explained in slop es= (5.514.97)/5.51= .10. Interaction Effects of ChildÂ’s Gender with Schools Finally, Table 10 shows that the effect of a ch ildÂ’s gender on mathematics achievement varied by school (variance estimate of slope=2.11, p =.005). Overall, the intercept, reflecting the mean achievement of boys, was 0.30 units higher than for girls in first grade (the earlier gap estimates were around 0.49, about 0.19 units higher). One schoollevel factor, teacher certification rates in elementary education, yielded a statistically significant an d large positive effect for boys versus girls (+2.49, p =.009). Statistically significant explanatory variab les that had negative effects for boys versus girls were the following: schoolsÂ’ teacher support levels (0.17, p =.05) and school averages on parent involvement (0.87, p < .05). To interpret these negative findings for teacher support, as an example, boys scored 0.17 units lower in ma thematics than girls in schools where teacher support was reportedly a unit higher. The reliability of intercepts in the full analytic model was .54, and the reliability of the slope was .19. A total of 24% of the variance in sl opes was explained by the modeled school variables (final variance estimate for slopes=1.61).
PAGE 22
Education Policy Analysis Archives Vol. 13 No. 46 22 Table 10 Crosslevel Interaction Effects of Gender Gr oup Membership and Sc hool on First Grade Mathematics Variablea Parameter estimates d.f. p Random effects Variance Component 2 School level intercepts, uoj 5.32 375.99 162 < .01 School level slopes, u8j 2.11 229.98 177 < .01 Child level effect, r ij 30.24 Fixed effects Coefficient SE t Intercept for Male Slope 0.30 0.25 1.20 167 0.22 Elementary Teacher Certificatio n 2.49 0.95 2.61 167 < .01 Teacher Support 0.17 0.09 1.95 167 0.05 Parent Involvement 0.87 0.521.66 167 0.09 a Reliability estimate of intercept = .54; Reliability estimate of slope = .19 Variance explained in slopes = [(2.111.61)/2.11] =.24. Discussion The ECLS data allowed a deeper examination of achievement gaps and school factors that directly affect or moderate childrenÂ’s mathematics achievement levels in isolated subgroupsÂ—thus addressing a void in the literature. The merit of pa rticular school practices and policies, such as class size reduction and higher teacher qualifications, coul d also be evaluated in the context of mandatory school reforms ensuing from legislation such as the NCLB Act. Several findings in the present study support current educational policy directions; ot hers contradict conventional or theoretical expectations. In conclusion, these findings are discussed along with limitations and areas for further research. Representativeness of the ECLS Data Set Table B1 (in Appendix B) and Table 1 show that the present sample was generally representative of the national ECLS sample in terms of gender and ethnicity, but somewhat underrepresented on the poverty status variable. The smaller number of cases ( N =2300 in 182 schools) resulted from a search for an optimal data set with complete data on all school and childlevel variables of interest in the present investigation. In the final data set, enough cases were present in each of the subgroups of interest and on variables selected for study to enable examination of main effects at each level and crosslevel interactions on school practice/policy issues discussed in the review of literature. Mathematics Achievement Gaps In interpreting the achievement gaps in the present study, it should again be borne in mind that gap estimates are regression coefficients that repr esent differences of a defined group on the outcome measure, as compared to the school means and controlling for other modeled child background factors. This definition is different from the NCLB approach, which uses mean differences computed with a specified reference group in mi nd (e.g., African American versus White).
PAGE 23
Achievement Gaps and Correlates of Early Mathematics Achievement 23 The results show that the pattern of ethnic and gender gaps in mathematics changes from kindergarten to first grade. However the influence of higher poverty levels is consistently negative on achievement, even when prior knowledge, gend er and ethnicity are controlled. Likewise, prior mathematics preparation has a strong positive and significant effect on subsequent achievement measures, with other factors held constant. In kindergarten, African Americans and one Hispanic group show signs of emerging gaps, but the gap in Hispanics declines to a nonsignificant level in Grade 1. Also, there is no gender gap evident until formal mathematics instruction begins in Grade 1. The statistically significant advantages estima ted for children who start with higher initial mathematics measures (measures taken either at Ken try or Kend) are consistent with prior research discussed in this paper. Institution of statistical controls on preGrade 1 ma thematics achievement in the analytic models wipes out potential differences in quality of early childhood care, parenting, and known differences in full and halfday kindergarten exposure in different children in the ECLS sample. All these variables have been shown in the literatur e to be factors that influence school readiness of entering first graders, particularly those who start with academic challenges. The quality of educational experiences before and during the kindergarten year may be critical in influencing subsequent mathematics achievement patterns in diverse children. From a school practice and policy perspective it is clear that children of different minority groups exhibit slightly different patterns of mathematics achievement in Grade 1. Contrary to expectations set by prior research on ethnic achievem ent gaps in both older and younger pupils (Lee, 2002; NCES, 1995a, 1995b; Zill & West, 2001), Hispanic children did not show statistically significant mathematics achievement gaps in first grade, irre spective of whether Kentry or Kend mathematics variability was controlled. Asians were estimated to sc ore slightly lower, although this last coefficient was again not better than chance. That significant ethnic achievement gaps are manifested in African American, but not in other ethnic subgroups, when Kend or Kentry mathematics achievement variability is controlled in first grade children should be noted. Likewise, that mathematics gender differences are small but significant as early as in first grade should be also noted by educators and policymakers. The finding on mathematics gaps in children from economically disadvantaged families is consisten t with results on national tests at all levels of schooling. It is imperative that such gaps are follo wed through elementary school and as children start their middle and high school years. Shifts in the size of the mathematics gaps should be closely monitored. More importantly, schools should find ways to address domainspecific needs of learners as they arise at the classroom level. Timely detecti on and diagnosis of curriculumspecific needs in learners would give schools and teachers the opportunity to prevent mathematics gaps from developing in later years. School Level Correlates of Mathematics Achievement When partitioned, about 20% of the total variab ility in childrenÂ’s mathematics achievement was betweenschools variance. The main eff ects models explained about 75Â–79% of that betweenschools variance estimate, showing class size, reading time at home, school size, and school success (academic press) orientation as significant school level corre lates at the 10% error levelÂ—consistent outcomes on three counts controlling for Kentry or Kend vari ability (Tables 6Â–7). Alimitation is that these correlates were significant at the 10% error level. However, the finding that class size negative ly affected school achievement means is also consistent with results from other research efforts, such as the Tennessee STAR investigations (Finn & Achilles, 1999), which showed positive effects of reduced class size in early grades. The effects reported here are of unit increases in mean teac herreported class size on school mathematics score
PAGE 24
Education Policy Analysis Archives Vol. 13 No. 46 24 averages; hence they are small in size. Each unit in the class size variable stands for an additional child. The descriptive data (Table 1) suggest that class sizes in ECLS schools went up to as many as 52 pupils in a class (Mean= 21.50, SD =4.74). The literature shows that with smaller classes, teachers are able to individualize their instruction more and employ a ra nge of instructional practices (Molnar et al, 1999) that may be more developmentally appropriate. More reading time given by parents, on average, had a large positive effect on school mathematics achievement means, controlling for child renÂ’s background characteristics. Although the study did not establish causal links, the association of mathematics achievement with increased reading activities at home is encouraging. In early years cognitive development in children may not be subjectspecific, hence increased athome reading activities could potentially result in gains in both mathematics and reading. A comment on some of the counterintuitive findin gs. The positive rather than negative effects of larger school size on early mathematics achiev ement may have to do with larger schools having more resources. Another explanation may be the la ck of representativeness of the sample on this variable. The breakdown on school size in the init ially screened 12,710 ECLS cases had shown that 31% of the children were in schools with 300Â–499 st udents, with 19% in even smaller schools. This indicated that the remaining 50% belonged in schools that exceeded 500 studentsÂ—a less than desirable school size for elementary schools. A 500 cu t was thus used to define the school size variable. However, Table 1 shows that following selection of the study sample, just 15% of the 182 schools were Â“largeÂ” (with more than 500 students), wh ile the remaining 85% were Â“smallÂ” schools. This distribution might have tilted results. On the success orientation variable, schools dea ling with challenging student populations often tend to have organizational cultures with high acad emic press values. Yet, they may show relatively poor academic outcomes because they serve struggling students. That administrator reports of school success orientation had a negative bu t statistically significant influe nce on mathematics achievement in schools might have resulted from this last reality. The low correlations in Table B2 (Appendix B) suggest that results were likely not affected by suppressor effects of other schoollevel variables in the models. Crosslevel Interaction Effects The more interesting results of this study were on moderators of Grade 1 mathematics achievement gaps. Several crosslevel interaction re sults were in expected directions and may point to some school policy/practice actions, keeping in mind that the evidence is correlational. With the achievement variability that was found in African Am erican children by school, increased class and school sizes affected mathematics achievement nega tively, and athome reading time was positively associated with mathematics achievement. For poor versus welltodo children, more instructional time per day to mathematics and reading had a posit ive and significant effect on school achievement meansÂ—another affirming finding. Further, higher tea cher certification rates in elementary education affected boysÂ’ mathematics achievement in a clearly positive direction. As much as a 2point gain in achievement was estimated to re sult for boys in schools with unit increases in schoolsÂ’ teacher certification rates. This last finding, although show ing differential effects by gender on mathematics achievement, is consistent with the results of the companion ECLS study by this author, where the school certification rate showed a large positive main effect on first grade reading achievement. The need for improving qualified teacher recruitment and retention is currently emphasized in the NCLB legislation; however, policy implementation varies in some regions when certified teachers are in short supply.
PAGE 25
Achievement Gaps and Correlates of Early Mathematics Achievement 25 Finally, increased IEPs in schools had a signif icant but small negative effect on mathematics school means of high poverty students. NCLB re quires that schools set common benchmarks for achievement of special versus nor mallyachieving student populations; this may not be a reasonable expectation. It should be noted again in evaluating the implications of these findings that the IEP indicator in this study did not correspond to the percent of students with disabilities in the ECLS sample, and does not reflect national estimates of special education enrollments (Table B3). Limitations and Future Research Because secondary analysis of large scale national surveys does not permit causal interpretations of various school effectsÂ—the eviden ce presented here is correlational. The controls instituted were statistical rather than experime ntal. The composition of the data set, variable definitions, missing data on surveys, and particular va riables selected for modeling, all affected findings obtained. Some of the significant negative coefficien ts obtained with parent, administrator, and teacher survey indices need further confirmation. Some eth nic group interactions could not be examined due to reduction in available cases. Future studies shou ld thus attempt to further validate and replicate the findings reported here with multiple ECLS data sets Direct and moderating effects of other variables, such as methods of mathematics instruction and childrenÂ’s cognitive development on mathematics achievement, should also be examined to bett er inform future school policies and practices. Despite the limitations noted, several results were replicated across analytic models tested. Monitoring of achievement gaps and a search for si gnificant correlates and moderators of achievement should thus continue in other similar populations, and especially with longitudinal data from the ECLS sample in Grades 3 and 5.
PAGE 26
Education Policy Analysis Archives Vol. 13 No. 45 26 References Ayers, W., Bracey, G., & Smith, G. (2000). The ultimate educational reform? Make schools smaller. Tempe, AZ: EPSL, Educational Po licy Research Unit. CERAIÂ–00Â–35. Blatchford, P., Moriarty, V., Edmonds, S., & Mart in, C.(2002). Relationsh ips between class size and teaching: A multimethod analysis of English infant schools. American Educational Research Journal, 39 (1), 101Â–132. Boethel, M. (2004). Readiness: School, family & community (Annual Synthesi s 2004). Austin, TX: National Center for Fa mily and Community Connectio ns with School, Southwest Educational Development Lab. Brookover, W, Beady, C., Flood, P., Sc hweitzer, J, & Wisenbaker, J. (1979). School social systems and student achievement: Schools can make a difference New York: Prae ger Publishers. Chatterji, M. (2002). Models and methods for examining standard sbased reforms and accountability initiatives: Have the tools of inquiry answered pressing questions on improving schools. Review of Educat ional Research, 72 (3), 345Â–386. Clark, D.L., Lotto, L.S., & McCa rthy, M.M. (1980). Factors associated with success in urban elementary schools. Phi Delta Kappan, 61 467Â–470. Collins, W. A., Macoby, E., Steinberg, L., He therington, E.M., & Bo rnstein, M. (2000). Contemporary research on pa renting: The case of nature versus nurture. American Psychologist, 55 218Â–232. DarlingHammond, L. (1998). Teac hers and teaching: Testing polic y hypotheses from a national commission report. Educational Researcher, 27 (1), 5Â–15. Ferguson, R. F. (1991). Paying for public education: New evidence on how and why money matters. Harvard Journal on Legislation, 28 (465), 465Â–498. Ferguson, R. F., & Ladd, H. F. (1996). How an d why money matters: An analysis of Alabama schools. In H. Ladd (Ed.), Holding Schools Accountable (pp. 265Â–298 ). Washington, D.C.: Brookings Institution. Finn, J. D., & Achilles, C. M. (1999). TennesseeÂ’s class size study: Findings, implications, misconceptions. Educational Evaluation and Policy Analysis, 21 (2), 97Â–109. Friedman, L. (1995). The space factor in mathematics: Ge nder differences. Review of Educational Research, 65 (1), 22Â–50. Friedman, L. (1989). Mathematics and the gender gap: A metaanalysis of recent studies on sex differences in quantitative tasks. Review of Educat ional Research, 59 185Â–213.
PAGE 27
Achievement Gaps and Correlates of Early Mathematics Achievement 27 Garmezy, N. (1993). Children in po verty: Resilience despite risk. Psychiatry, 56 127Â–136. Gramezy, N. & Masten, A.S. (1 986). Stress, competence, and re silience: Common frontiers for therapists and ps ychopathologists. Behavior Therapy, 17 500Â–521. Grissmer, D., W, Kirby, S. N., Berends, M., & Williamson, S. (1994). Student achievement and the changing American family. Santa Monica, CA: RAND. Hannaway, J., & Talbert, J.E. (1993). Bringing context into effective schools research: Urbansuburban differences. Education Administration Quarterly, 29 (2), 164Â–186. Hannaway, J., & Kimball, K. (1998). Big isnÂ’t always bad: School district size, poverty, and standardsbased reform Washington, D.C.: Urban Institute. HubbsTait, L., Culp, A.M., Erron, H., Culp, R. et al ( 2002). Relation of He ad Start attendance to childrenÂ’s cognitive and social outcomes: Moderation by family risk. Early Childhood Research Quarterly, 17 539Â–558. Hyde, J.S., Fennema, E., & Lamon, S.J. (1990). Gender differences in ma thematics performance: A metaanalysis. Psychological Bulletin, 106 139Â–155. Knapp, Michael, S. (1997). Between systemic reforms and the mathematics and science classroom: The dynamics of innovation, im plementation, and professional learning. Review of Educational Research, 67 (2), 227Â–266. Laparo, K. & Pianta, R.C. (2000) Predicting childrenÂ’s competen ce in early sc hool years: A metaanalytic review. Review of Educational Research, 70 (4), 443Â–484. Lee, J. (2002). Racial and ethnic achievement ga p trends: Reversing the progress towards equity. Educational Researcher, 31 (1), 3Â–12. Lee, V.E., BrooksGunn, J., & Schnur, E. (1988). Does Head Start Work? A 1year followup of comparison of disadvantaged children atte nding Head Start, no preschool and other preschool programs. Developmental Psychology, 24 (2), 210Â–222. Linn, Robert L. (2003). Ac countability: Respons ibility and Reasonab le Expectations. Educational Researcher, 32 (7), 3Â–13. Linn, R. L., Baker, E.L., & Betebenner, D.W. (2 002). Accountability Sys tems: Implications of requirements of the No Child Left Behind Act of 2001. Educational Researcher, 31 (6), 3Â–16. Molnar, A., Smith, P. Zahorik, J., Palmer, A., Ha lbach, A., & Erle, K. (1999). Evaluating the SAGE program: A pilot program in target ed teacherpupil redu ction inWisconsin. Educational Evaluation and Policy Analysis, 21 (2), 165Â–177. National Center for Educationa l Statistics (N CES) (1995a). Findings from the condition of education 1995: The educ ational progress of Af rican American students Washington, D.C.: U.S. Dept. of Education/OERI.
PAGE 28
Education Policy Analysis Archives Vol. 13 No. 45 28 National Center for Educationa l Statistics (N CES) (1995b). Findings from the condition of education 1995: The ed ucational progress of Hispanic students Washington, D.C.: U.S. Dept. of Education/OERI. National Center for Educational Statistics (NCES) et al. (1999). ECLSK base year public use data files and electronic code book .Washington, D.C.: U.S. Dept. of Education/OERI. National Governors Association (2005). Closing the achievement gap Washington, D.C.: NGA Center for best Practices. No Child Left Behind Ac t of 2001, P.L. No. 10 7Â–110, 115 Stat. 1425 (2002). NICHD Early Child Care Research Network (2002). Early child care and childrenÂ’s development prior to school entry: Results from the NI CHD study of early child care. American Educational Research Journal, 39 (1), 133Â–164. Purkey, S.C. & Smith, M.S. (1983). Effective schools: A review. Elementary School Journal, 84 (4), 427Â–452. Raudenbush, S., Bryk, A. Cheong Y.F., & Congdon, R. (2001). HLM 5: Hierchical linear and nonlinear modeling. Lincolnwood, IL: SSI. Rosenholtz, S. (1985). Effective schools: Interpreting the evidence. American Journal of Education, 93, 352Â–388. Rowan, B., Bossert, S.T. & Dwyer, D.C. (1983). Research on effective schools: A cautionary note. Educational Researcher, 12 (4), 24Â–31. Sirotnik, K. A., & Kimball, K. (1999). Standards fo r standardsbased acco untability systems. Phi Delta Kappan, 81 (3), 209Â–214. Walston, J. & West, J. (2004). Fullday and halfday kindergarten in the United States: Findings from the ECLS, Kindergart en class of 1988Â–1989 Washington, D.C.: U.S. Dept. of Education. Werner, E.E. (1993). Risk, resilie nce, and recovery: Perspectives from the Kauai Longitudinal Study. Developmental and Psychopathology, 5 503Â–515. Williams, D.T. (1990). The dimensions of educ ation: Recent resear ch on school size. Working Paper Series. Clemson, SC: Clemson Univ ersity, Strom Thurmond Institute of Government and Publ ic Affairs (ERIC Re production Document No. ED 347 006). Zill, N. & West, J. (2001). Entering kindergarten: Findings from the condition of education 2000 Washington, D.C.: U.S. Dept. of Education/OERI.
PAGE 29
Achievement Gaps and Correlates of Early Mathematics Achievement 29 About the Author Madhabi Chatterji Teachers College, Columbia University Email: mb1434@columbia.edu Madhabi Chatterji Associate Professor of Measuremen t, Evaluation, and Education at Teachers College, Columbia University, has research interests in designing classroomand schoolbased assessment system s; development and validation of construct measures with classical and Rasch measurement methods; and in evaluating standa rdsbased educational reforms and smalland largescale interventions with systemic models. Her most recentlyinitiated research, supported by the National Science Foundation in 2004, de als with addressing K12 achievement gaps in mathematics using a teachermediated model of "proximal" assessment and data use. Another line of inquiry deals with gathering researchbased evidence on field interventions using mixe dmethod research designs.
PAGE 30
Education Policy Analysis Archives Vol. 13 No. 45 30 Appendix A Description and Psychometric Pr operties of ECLS/Survey Indices Child Variables ChildÂ’s socioeconomic statuspoverty : This index was based on a categorical measure of socioeconomic status (W1SESQ5) provided by NCES that broke down a continuous measure of socioeconomic status built on parent education, o ccupation, and income, into five categories by quintile. The first and second quintile categories we re coded as 1 (low SES, High Poverty), and quintiles 35 were coded as 0 (high SES, Low Poverty). This measure yielded much the same results as the continuous SES composite and was used because of its interpretability. In different models, this measure was used as a Level 1 (based on child quintile category) as well as a Level 2 predictor (based on school means). Teacher Variables Teacher certification variables ECERT, ECCERT (Source: Teacher Questionnaire BB4ELEMCT, B4ERLYCT). These variables were dummy coded, with a 1 to indicate if the teacher had elementary certification (B4ELEMCT) ve rsus not, 0; or early childhood certification (B4ERLYCT) versus not. Only the former was used in the final models based on preliminary findings. School aggregates on certifi cation rates were entered in HLMs. Teacher support composite TSUP_1 (Source: Administrator Questionnaire). This index was composed of four selfreport items giving admini strator reports on whether the school had an active professional development program and gave teacher s planning time, time for professional growth and incentives for improvement (S2PRODEV, S2ACTSTF, S2ADEQTE, S2INCENT). The index was supported by results of a principal components analysis with a prerotation eigenvalue of 2.01, with component loadings following varimax rotation of .63 to .83. The CronbachÂ’s alpha reliability estimate of composite scores was .71. This index is interpreted as the extent to which a child was exposed to a school with high levels of teacher supports. A schoollevel aggregate of TSUP was entered into HLMs at Level 2. School/Organizational Indices School success orientation S_SUC1 (Source: School Administrator Questionnaire). The S_SUC1 composite was based on five selfreport i tems indicating leader reports of the degree of success schools have in emphasizing childrensÂ’ acad emic learning, namely, raising test performance, providing challenges to high achievers, added he lp for low achievers, and being open to new ideas (S2SUCC67, 1011). The factor yielded a prero tation eigenvalue of 1.19 using principal components analysis, and factor loadings of .53 to .77 following varimax rotation. The alpha reliability estimate of the composite was .73. Mean s at the school level were utilized for HLMs. Class size CSIZE: (Source: Teacher Questionnaire, Part A). This was a numeric index computed as follows with responses to two items on the teacher questionnaire asking separately for the # of boys versus # of girls in their cla ssroom. CSIZE=A1BOYS+A1GIRLS. Again, means at the school level were used for HLMs.
PAGE 31
Achievement Gaps and Correlates of Early Mathematics Achievement 31 Class time to reading and math TIM_1: (Source: Teacher Questionnaire Part A). This was a numeric index computed as follows with respons es to two items on the teacher questionnaire, dealing with how much time in minutes, teachers de dicated to reading and math activities in their classrooms per day: TIM=A2MINRD +A2MINMTH. These two items loaded on a principal component with an eigenvalue of 1.57 before ro tation, and had varimax rotated factor loadings of .88 and .83. The alpha reliability was .87. Schoollev el means on the composite were used in HLMs. Parent Involvement Indices Educational support EDSUP (Source: Parent Questionnaire). This index was based on parent/guardian reports of whether or not they did math, writing, and mathematics with their child at home. This 3item set had a prerotation eigenvalue of 2.5, and factors loadings from .31 to.78 following varimax rotation. The Cr onbachÂ’s alpha estimate was .65. Parent satisfaction with school activities PAR_S (Source: Parent Questionnaire). This 4item index was dealt with whether the school pr ovides opportunities for parent and community involvement, and for tracking how their child is doing. PARSCHL=HOWCHD+P2CHILDR+P2CHANCV+P2COMMUN. PAR_S was the school mean. Following satisfactory factor extraction, the obtai ned alpha reliability of the composite was .75. Parent involvement PARINV (Source: Parent Questionnaire). COMPUTE PARINV=P2ATTENB+ P2PARGRP +P2ATTENS +P2VOLUNT+ P2FUNDRS. This index showed the degree to which parents reported invo lvement in school based on attending events and functions, volunteering, fundraising etc. As in th e others, principal components analysis helped identify this subset of items; however, it yiel ded a lower alpha reliability of .56. P_INV was the school mean. Other Childlevel Demographic/Background Variables At the child level, several background variab les were dummycoded for analysis. These were gender (Male=1, Female=0), ethnicity in grou ps of interest (Asian=1, Others=0; African American=1, Others=0; Hispanic (race specified)=1, Others=0; and Hispanic (race unspecified)=1, Others=0). In addition, age of the child in months in grade 1, a continuous measure, was included in the preliminary analysis. Other SchoolLevel Variables At the school level, urbanicity (urban=1, othe r (rural/ suburban)=0), school size(greater than or equal to 500=1, less than 500=0), and school sector (public=1, other=0) were dummy coded. Each schoolÂ’s total IEPs, an index of excep tional education services (SPCED) provided by the school, was computed. Likewise, student attenda nce was computed as a percent of total days attended in the school year. This last index was ba sed on the number of absences reported for each child in the ECLS database. All school level indi ces were aggregated across children within each school.
PAGE 32
Education Policy Analysis Archives Vol. 13 No. 45 32 Imputed Values: The number of values imputed with the mean substitution procedure of SPSS was as follows: EDSUP_1, 1 case; TSUP_1, 23 cases; S_ SUCC, 33 cases; S_ATT, 38 cases; CSIZ_1, 15 cases; TIM_1, 55 cases.
PAGE 33
Achievement Gaps and Correlates of Early Mathematics Achievement 33 Appendix B Supplementary Tables Table B1 Representiveness of Fi nal Data Set as compar ed with ECLS Sample Variable ECLSK Sample Initially Screened Study Sample (weighted) Final Data Set Number of cases 17212 14742 2300 Poverty Status (Low SES,) 6958 (40.4%) 5885 (39.9%) 667(29%) Gender (Males) 7429 (50 .4%) 8867 (49%) 1127 (51.6%) EthnicityBlack 2441 (14.2%) 2374 (16%) 276 (12%) EthnicityHispanic 1 1388 (8.1%) 1346 (9%) 138 (6%) EthnicityHispanic 2 1560 (6.4%) 1430 (9.7%) 115 (5%) EthnicityAsian 1093 (6.4%) 448 (3%) 69 (3%)
PAGE 34
Education Policy Analysis Archives Vol. 13 No. 45 34 Table B2 Correlations of Schoollevel Variables Variable Teacher Support School Succes SchoolAttend. Educ. Suppt. Parent Satis. Class size Class time to reading/math Poverty School size IEPs At home reading time Teacher Support Â— School Success .165** Â— School Attendance .042 .057 Â— Educational support .010 .028 .002 Â— Parent Satisfaction .001 .125* .064 .083 Â— Class size .004 .041 .184** .009 .012 Â— Class time to reading/math .132* .144* .152* .028 .112 .128* Â— Poverty .097* .200** .035 .033 .013 .117* .278** Â— School size .126** .031 .061 .023 .095*.150** .073 .078 Â— IEPs .097* .002 .003 .006 .025 .068 .036 .083 .025 Â— At home reading time .121* .008 .037 .180**.014 .114* .117 .087 .012 .067Â— Parent Involvement .114* .169** .022 .003 .206**.103* .186** .484** .228**.071.004 *p < .05; **p < .01
PAGE 35
Achievement Gaps and Correlates of Early Mathematics Achievement 35 Table B3 Frequency distributions on Selected Variables in Initially Screened Sample Category ECLS sample within category Total school enrollment (used in study) 0149 445 150299 1915 300499 3996 500749 3620 800+ 2734 Individualized Education Plans (Variable: U4IEP, used in study) Does not have IEP 9012 Has IEP 775 Diagnosed Learning Program (Variable: P3 HEQ0 20; not used in study due to missing data) No 213 Yes 270 Missing 12436
PAGE 36
Education Policy Analysis Archives Vol. 13 No. 45EDUCATION POLICY ANALYSIS ARCHIVES http://epaa.asu.edu Editor: Sherman Dorn, University of South Florida Production Assistant: Chris Murre ll, Arizona State University General questions about ap propriateness of topics or particular articles may be addressed to the Editor, Sherman Dorn, epaaeditor@shermandorn.com. Editorial Board Michael W. Apple University of Wisconsin David C. Berliner Arizona State University Greg Camilli Rutgers University Casey Cobb University of Connecticut Linda DarlingHammond Stanford University Mark E. Fetler California Commission on Teacher Credentialing Gustavo E. Fischman Arizona State Univeristy Richard Garlikov Birmingham, Alabama Gene V Glass Arizona State University Thomas F. Green Syracuse University Aimee Howley Ohio University Craig B. Howley Appalachia Educational Laboratory William Hunter University of Ontario Institute of Technology Patricia Fey Jarvis Seattle, Washington Daniel Kalls Ume University Benjamin Levin University of Manitoba Thomas MauhsPugh Green Mountain College Les McLean University of Toronto Heinrich Mintrop University of California, Berkeley Michele Moses Arizona State University Anthony G. Rud Jr. Purdue University Michael Scriven Western Michigan University Terrence G. Wiley Arizona State University John Willinsky University of British Columbia
PAGE 37
Achievement Gaps and Correlates of Early Mathematics Achievement 37 EDUCATION POLICY ANALYSIS ARCHIVES Englishlanguage Graduate Student Editorial Board Noga Admon New York University Jessica Allen University of Colorado Cheryl Aman University of British Columbia Anne Black University of Connecticut Marisa Cannata Michigan State University Chad d'Entremont Teachers College Columbia University Carol Da Silva Harvard University Tara Donahue Michigan State University Camille Farrington University of Illinois Chicago Chris Frey Indiana University Amy Garrett Dikkers University of Minnesota Misty Ginicola Yale University Jake Gross Indiana University Hee Kyung Hong Loyola University Chicago Jennifer Lloyd University of British Columbia Heather Lord Yale University Shereeza Mohammed Florida Atlantic University Ben Superfine University of Michigan John Weathers University of Pennsylvania Kyo Yamashiro University of California Los Angeles
PAGE 38
Education Policy Analysis Archives Vol. 13 No. 45 Archivos Analticos de Polticas Educativas Associate Editors Gustavo E. Fischman & Pablo Gentili Arizona State University & Universidade do Estado do Rio de Janeiro Founding Associate Editor for Spanish Language (1998Â—2003) Roberto Rodrguez Gmez Editorial Board Hugo Aboites Universidad Autnoma MetropolitanaXochimilco Adrin Acosta Universidad de Guadalajara Mxico Claudio Almonacid Avila Universidad Metropolitana de Ciencias de la Educacin, Chile Dalila Andrade de Oliveira Universidade Federal de Minas Gerais, Belo Horizonte, Brasil Alejandra Birgin Ministerio de Educacin, Argentina Teresa Bracho Centro de Investigacin y Docencia EconmicaCIDE Alejandro Canales Universidad Nacional Autnoma de Mxico Ursula Casanova Arizona State University, Tempe, Arizona Sigfredo Chiroque Instituto de Pedagoga Popular, Per Erwin Epstein Loyola University, Chicago, Illinois Mariano Fernndez Enguita Universidad de Salamanca. Espaa Gaudncio Frigotto Universidade Estadual do Rio de Janeiro, Brasil Rollin Kent Universidad Autnoma de Puebla. Puebla, Mxico Walter Kohan Universidade Estadual do Rio de Janeiro, Brasil Roberto Leher Universidade Estadual do Rio de Janeiro, Brasil Daniel C. Levy University at Albany, SUNY, Albany, New York Nilma Limo Gomes Universidade Federal de Minas Gerais, Belo Horizonte Pia Lindquist Wong California State University, Sacramento, California Mara Loreto Egaa Programa Interdisciplinario de Investigacin en Educacin Mariano Narodowski Universidad To rcuato Di Tella, Argentina Iolanda de Oliveira Universidade Federal Fluminense, Brasil Grover Pango Foro Latinoamericano de Polticas Educativas, Per Vanilda Paiva Universidade Estadual Do Rio De Janeiro, Brasil Miguel Pereira Catedratico Un iversidad de Granada, Espaa Angel Ignacio Prez Gmez Universidad de Mlaga Mnica Pini Universidad Nacional de San Martin, Argentina Romualdo Portella do Oliveira Universidade de So Paulo Diana Rhoten Social Science Research Council, New York, New York Jos Gimeno Sacristn Universidad de Valencia, Espaa Daniel Schugurensky Ontario Institute for Studies in Education, Canada Susan Street Centro de Investigaciones y Estudios Superiores en Antropologia Social Occidente, Guadalajara, Mxico Nelly P. Stromquist University of Southern California, Los Angeles, California Daniel Suarez Laboratorio de Politicas PublicasUniversidad de Buenos Aires, Argentina Antonio Teodoro Universidade Lusfona Lisboa, Carlos A. Torres UCLA Jurjo Torres Santom Universidad de la Corua, Espaa
