xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam a22 u 4500
controlfield tag 008 c20059999azu 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E11-00468
Educational policy analysis archives.
n Vol. 13, no. 46 (November 23, 2005).
Tempe, Ariz. :
b Arizona State University ;
Tampa, Fla. :
University of South Florida.
c November 23, 2005
Achievement gaps and correlates of early mathematics achievement : evidence from the ECLS Kfirst grade sample / Madhabi Chatterji.
Arizona State University.
University of South Florida.
t Education Policy Analysis Archives (EPAA)
xml version 1.0 encoding UTF-8 standalone no
mods:mods xmlns:mods http:www.loc.govmodsv3 xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govmodsv3mods-3-1.xsd
mods:relatedItem type host
mods:identifier issn 1068-2341mods:part
mods:detail volume mods:number 13issue 46series Year mods:caption 20052005Month November11Day 2323mods:originInfo mods:dateIssued iso8601 2005-11-23
Readers are free to copy, display, and distribute this article, as long as the work is attributed to the author(s) and Education Policy An alysis Archives, it is distributed for non-commercial purposes only, and no alteration or transformation is made in the work. More details of this Creative Commons license are available at http://creativecommons.org/ licenses/by-nc-nd/2.5/. All other uses must be approved by the author(s) or EPAA EPAA is published jointly by the Colleges of Education at Arizona State University and the University of South Florida. Articles are indexed by H.W. Wilson & Co. Send commentary to Casey Cobb (cas firstname.lastname@example.org) and errata notes to Sherman Dorn (email@example.com). EDUCATION POLICY ANALYSIS ARCHIVES A peer-reviewed scholarly journal Editor: Sherman Dorn College of Education University of South Florida Volume 13 Number 46 Novemb er 23, 2005 I SSN 1068Â–2341 Achievement Gaps and Correlates of Early Mathematics Achievement: Evidence from the ECLS KÂ–First Grade Sample1 Madhabi Chatterji Teachers College, Co lumbia University Citation: Chatterji, M. (2005) Achievement gaps and correlates of early mathematics achievement: Evidence from the ECLS KÂ–first grade sample. Education Policy Analysis Archives, 13 (46). Retrieved [date] from ht tp://epaa.asu.edu/epaa/v13n46/. Abstract In light of the NCLB Act of 2001, this study estima ted mathematics achievement gaps in different subgroups of kindergartners and first gr aders, and identified childand school-level correlates and moderators of early mathematics achievement. A subset of 2300 students nested in 182 sc hools from the Early Childhood Longitudinal Study KÂ–First Grade data set was analyzed with hierarch ical linear models. Relative to school mean estimates at the end of kindergarten, significant mathematics achievement gaps were foun d in Hispanics, African Americans and high poverty students. At the end of Grade 1, mathem atics gaps were sign ificant in African American, high poverty, and female subgroups, but not in Hispanics. School-level correlates of Grade 1 Mathematics achievem ent were class size (with a small negative main effect), at-home reading time by pare nts (with a large posit ive main effect) and school size (with a small pos itive main effect). Cross-level interactions in Grade 1 indicated that schools with larger class and school size s had a negative effect on 1 An early version of this study was supported by the Institute for Urban and Minority Education (IUME), Teachers College, Columbia University during the summer of 2002. The author acknowle dges Professor Edmund W. Gordon, Director of IUME, for the opportunity to begin wo rk on problems and issues on mi nority education. Earlier papers were presented at th e Annual Meetings of the American Evalua tion Association in 2002, and the American Educational Research Association, in 2004. My thanks to A lissa Peltzman and Nancy Koh, currently in the M.A. program in Policy Studies at Teachers College, for their assistance with the paper.
Education Policy Analysis Archives Vol. 13 No. 46 2 African American childrenÂ’s ma th scores; schools giving mo re instructional time to reading and math had a positive effect on high poverty studentsÂ’ scores, and schools with higher elementary teacher certificati on rates had a positi ve effect on boysÂ’ mathematics achievement. Keywords: achievement gap; early childhood; primary grades; Early Childhood Longitudinal Study (ECLS); correlates and moderators of mathematics achievement. Purpose The passage of the No Child Left Behind (NCLB) Act of 2001 has drawn attention to achievement differentials in diverse U.S students, commonly referred to as the Â“achievement gapÂ”. By law, public schools are now held accountable for equitable achievement outcomes in subgroups of minority versus non-minority, normally -achieving versus exceptional, as well as socio-economically advantaged versus disadvantaged students (P.L 107Â– 110, 115 Stat. 1425, 2002). As a consequence of such school reform legislation, disparities among ch ildrenÂ’s mathematics achievement as well as factors that influence the observed differences, are now of cen tral concern to researchers, practitioners, policymakers and the public alike. Although much research now concludes that gender gaps in mathematics are declining in large scale examinations of adol escents and adults (see for example, Friedman, 1989), documentation is sparse on the mathematics gender gap in early school years. Questions surrounding the time at which mathematics achievement gaps fi rst develop, groups in which they are consistently manifested, and circumstances under which they redu ce or are sustained over time, is of particular relevance today. The isolation of schoolversus ch ild-level characteristics that potentially narrow observed differences in early mathematics achievement is a needed area of research. This study estimates achievement gaps and corre lates of early mathematics achievement with a particular focus on male versus female, poor versus well-to-do and various ethnic subgroups in the U.S. A main aim of the research was to estimate th e size of mathematics achievement gaps manifested in early school years in the contex t of recently set reform agendas for schools to close gaps by 2014. As formal instruction in mathematics tends to be more consistently distributed in first grade rather than in kindergarten, the study focuses mainly on first grade achievement outcomes. At the same time, kindergarten gaps are estimated so that changes in achievement patterns can be meaningfully compared and interpreted. A subset of data from the Longitudinal Kinderga rten-First Grade Public Use file of the Early Childhood Longitudinal Study (ECLS) was analyzed. The ECLS, consisting of data collected between kindergarten entry (Fall 1998) and the end of first grade (Spring 2000), uses a nationally representative sample. Particular research objectives that guided th e present study were to determine: (1) the extent to which variability in childrenÂ’s mathematics achievemen t in first grade is explained by childversus school-level factors; (2) whether childrenÂ’s membership in a specific subgroup resulted in significant within-school achievement gaps in kindergarten and subsequently, in first grade, controlling for particular child background characteristics; (3) the degree to which selected, theoretically-supported or policy-relevant school factors significantly influenced first grade mathematics achievement, when some child background characteristics are controlled while others allowed to vary randomly by school; and (4) whether childrenÂ’s membership in particular eth nic, gender or poverty subgroups interacted with their school membership to affect mathematics scor es, and school factors accounting for the variance in school slopes. Kindergarten education varied based on length of day (half versus full day) in the ECLS sample (Walston & West, 2004). To assay the cumulative effects of KÂ–1 years on end-of-first grade mathematics achievement in selected analytic mo dels, child mathematics measures taken at
Achievement Gaps and Correlates of Early Mathematics Achievement 3 kindergarten entry (K-entry) were used as indicators of prior achievement. In other models, end-ofkindergarten mathematics achievement measures (K-e nd) were used to control for differences in prior mathematics learning. The latter effects were interpreted as indicative mainly of childrenÂ’s first grade school experiences on end-of-first gr ade achievement, assuming all children entered with similar levels of prior preparation in mathematics. Results of both series of models are compared. Theoretical Framework Predictors with which to build explanatory mo dels for early mathematics achievement were drawn from a review of a broad literature base th at focused on four areas: the national push for reducing achievement gaps in mathematics; the documented demographic shifts in the U.S. and achievement trends in diverse students; early find ings of the Kindergarten year ECLS sample; and existing evidence on childand school-level correlates of achievement. That literature is now discussed, with specific attention devoted to research on early care, gender differences, and particular school practice variables, such as reduced class size, school size and teacher qualifications, and their expected influences on early mathematics achievement. The NCLB ActÂ’s Emphasis on the Achievement Gap The most recent legislative action on the standards-based reform movement in the U.S., the No Child Left Behind Act of 2001 (NCLB), uses challenging academic standards, coupled with highstakes standards-based testing and accountability, as its main strategy for fostering school improvement (P.L 107Â–110, 115 Stat 1425, 2002). For the first time in the history of national school reforms, the law places an emphasis on achievement by all groups of students, particularly those who are historically low-achieving, such as ethnic minor ities, socio-economically disadvantaged or special needs students. To monitor the status of achievemen t gaps and ensure that historically low-achieving students receive much needed attention, NCLB mand ates disaggregated reporting of state test scores. That is, schools must break down results on student performance by relevant subgroups in key subject areas such as reading and mathematics, and attempt to close achievement gaps evidenced in various sub-categories by 2014. NCLB provides states with some flexibility in selecting appropriate standards-based assessments, as well as in defining the mean ing of Â“Adequate Yearly ProgressÂ” (AYP) using performance standards (cut-scores) on state tests. However, the Act requires that schools monitor and reduce achievement lags in students belonging in high risk groups by the pre-set deadline. The lack of comparability in state standards and current practices for monitoring achievement gaps have raised concerns among researchers and policy-analysts (Linn, 2003; Linn, Baker & Betebenner, 2002). The recruitment and retention of qualified teachers in schools is another NCLB strategy for enhancing school and student outcomes (see Section 1119, P.L 107Â–110, 115 St at. 1425, 2002). Each local educational agency supported by NCLB funds must ensure that hired teachers are Â“highly qualifiedÂ”. Proof of teacher qualifications can be in the form of full state licensure or certification in the relevant area of specialization, a Bachelor Â’s degree, professional development, classroom experience, and knowledge of a subject garnered over time. States are allowed freedom in devising ways by which teachers might demonstrate their competency and subject matter knowledge. Nevertheless, schools are expected to be in complia nce by 2005Â–06. Schools in rural areas, where the same teacher might teach more than one subject, an d secondary schools, where higher levels of subject
Education Policy Analysis Archives Vol. 13 No. 46 4 area knowledge are needed, have been accorded some flexibility in meeting the NCLB requirements ( http://www.ed.gov/nclb/methods/teachers/hqtflexibility.html ). Student Background Characteristic s and Scholastic Achievement Outside legislative mandates, what factors are lik ely to diminish or exacerbate the academic deficits in children in high risk subgroups at school? Academic risk factors that are frequently documented in the clinical and child development literature include ethnicity age, gender, poverty and lack of parental education, family mobility, limi ted English proficiency, along with nutritional and health factors (for examples, see Garmezy, 1993; Garmezy & Masten, 1986; Werner, 1993). It is thus not unreasonable to think that children who are at ac ademic risk due to multiple risk factors, such as membership in particular ethnic groups combined wi th high poverty levels, are likely to start school at a greater disadvantage. However, the same body of literature also recognizes protective factors in childrenÂ’s personal, family, and school backgrounds that foster resilience despite observed child or family risk levels. For example, parenting and family socio-economic fact ors are viewed as major predictors of childrenÂ’s cognitive and social development because families play a central role in childrenÂ’s lives and both genetic and environmental influences are known to have a combined effect on their scholastic success (see Collins, Maccoby, Steinberg, Hetherington & Borns tein, 2000). Public initiatives, such as Head Start and Title 1 programs were founded to counter the well-documented negative relationship of childrenÂ’s poverty levels with their cognitive, social and physical development. Influences of prior learning and poverty A fairly recent meta-analysis of 60 studies determined that cognitive and social skills measured in late pr eschool years were predictive of performance in the same domains in the early school years (Laparo & Pianta, 2000). Effect sizes were positive (+0.49) for cognitive and academic predictors. The National Institute of Child Health and Human Development (NICHD Early Child Care Research Network, 2002) also reported results from studies on the positive influences of parenting and child-care centers on childrenÂ’s pre-academic skills. A recent longitudinal study of 1000 children showed that higher quality of child care and experience in preschool center-type activities were positive correlates of pre-academic skills and language (NICHD Early Child Care Research Network, 2002), with gender, ethnicity, family income, parenting and other background factors controlled. Higher quality of child care pr edicted better pre-academic and language outcomes, irrespective of the hours and type of care. Contrary to expectations, however, these researchers were not able to document that better quality child care would be advantageous to children from disadvantaged environments, such as low income households. That is, they did not find significant interaction effects between poverty and quality of care. They concluded that their findings were limited by their use of multiple regression mo dels to test for interactions (p. 159). Influences of ethnic differences A 1994 report published by the U.S. Department of Education (NCES, 1995a) concluded that African American child ren were less likely to be enrolled in pre-primary education relative to Whites, and were more likely to be below modal grade for their age in school. The same report showed that White-African American gaps in mathematics, reading, and science appeared as early as age 9, and did not narrow with age. Over the years, Hispanic children have also been found to start school with less preschool experience than White children and the achievement gap between these two groups was found to widen considerably by 1993 (17% to 38%) (NCES, 1995b). The Hispanic-White gap was also found to appear at age 9 and persisted through age 17, but there were areas in which the gap seemed to narrow. Evidence from other longitudinal studies of ad olescents suggests that certain subgroups among particular ethnic minorities tend to perform better than other students of the same ethnic origin.
Achievement Gaps and Correlates of Early Mathematics Achievement 5 Grissmer and colleagues (1994), for instance, foun d that Hispanic White youth demonstrate different achievement gains over time relative to their Hisp anic Black counterparts. Systematic studies on Asian students in the U.S. are rare, possibly because they are perceived to perform well, or as well as the majority ethnic group in U.S. schools. Are there any signs of the ethnic achievement gaps reducing? As the literature suggests, manifestation of achievement gaps in early school years is predicated on background differences in young children, which affect their school readiness levels as well as social and cognitive potential (Boethel, 2004). There is some evidence that earl y care programs such as the Head Start give a significant boost to children from low income environments, particularly African American children (see Lee, Brooks-Gunn, Schnur, 198 8; Hubbs-Tait et al, 2002). Such ev idence would suggest that with appropriate parenting and pre-school care, achiev ement gaps could narrow before children start kindergarten. Lee (2002) observed an overall reducti on in ethnic achievement gaps on the National Assessment of Ed ucational Progress (NAEP) and the Scholastic Aptitude Test (SAT) through the 1980s, and concluded that schools had made Â“great progress toward equityÂ”. African American-White and Hispanic-White gaps narrowed in the early 19 80s. In the late 1980s and 1990s, however, the performance gaps either stabilized or widened agai n, corresponding with the onset of standards-based reforms. Using recent NAEP results in reading, mathematic s and writing, some states are demonstrating that the achievement gap is reducing after the pa ssage of NCLB, according to National GovernorsÂ’ Association (NGA) Clearinghouse ( www.subnet.nga.org ). Despite high mobility rates, Department of Defense schools show the smallest gaps between Whi te and African American students, even in fourth grade. Multi-faceted approaches to building educ ational programs, use of Â“bestÂ” practices and improvement of teacher quality are identified by the NGA as key factors that alter and influence student achievement outcomes. Gender differences. Gender differences in mathematics are now documented to be small and decreasing over time, but they exist in college en trance examinations (Friedman, 1989; 1995; Hyde, Fennema, & Lamon, 1990). Whether due to socializ ation differences, innate genetic differences, or different spatial aptitudes influenced by combined genetic-environmental factors, the gender gap in mathematics is one that seems to interest both rese archers and policy-makers. FriedmanÂ’s (1995) metaanalysis examining correlational studies of spatial an d mathematics skills in ma les versus females led to the conclusion that the relationship was stronger in males than females in many studies, showing gender-specific relationship patterns that favored ma les. In select samples, however, the relationship between spatial concepts and math ematics ability was stronger in femalesÂ—leading to the conclusion that motivation, career goals, and factors such as socialization in the environment moderate such relationships. The jury is thus still out as to gender differences and moderators of mathematics skills in younger school-going children from developed countries such as the U.S. School-level Correlates of Student Achievement Reviews of literature on standards-based school reforms (Chatterji, 2002; Darling-Hammond, 1998; Knapp, 1997) as well as empirical studies (s ee for examples, Ferguson, 1991; Ferguson & Ladd, 1996; Sirotnik & Kimball, 1999) conclude that high student achievement on standards-based tests is more likely when teachers are certified to teach and both teachers and school leaders are appropriately trained on the new content standards. In light of NCLBÂ’s emphasis on teacher qualifications and professional development, these facto rs merit continuing evaluation.
Education Policy Analysis Archives Vol. 13 No. 46 6 Efficient allocation of resources to support particular reforms, such as through the infusion of technology, reduced school and class sizes, alignm ent of classroom instructional practices with new content standards, longer blocks of time dedi cated to subject-specific instruction, alignment of practices to meet needs of diverse student populati ons, higher levels of parent involvement in childrenÂ’s education, is also expected to influen ce achievement at all levels. School context factors, such as overall poverty levels in the student body, have also been shown to affect leadership behaviors, schooling practices, and organizational culture (H annaway & Talbert, 1993; Hannaway & Kimball, 1998), and in turn, academic outcomes. Discussions of class size as a reform initiative often center on cost effectiveness issues rather than on what schools and teachers actually do with the added resources that could moderate student outcomes when class sizes are small. The experi mental Tennessee STAR project provided evidence that classes with under 20 pupils have positive effects on studentsÂ’ academic performance in early school years (Finn & Achilles, 1999). Molnar et al. (199 9) also examined classroom processes in smaller classes in Wisconsin, documenting that smaller classe s lead to more individualized instruction, with greater variety in instructional activities. A large scal e study in the U.K, conducted by Blatchford and colleagues (2002) also supports to the latter contentionÂ—more individualized attention is evident in smaller-size classes, and potentially facilitates an environment in which studentsÂ’ early academic difficulties can be tackled. Does school size matter at the elementary level? Based on existing research from the past decade, effective school sizes at the elementary level were determined to lie at 300Â–400 students (Williams, 1990), making schools of 500 or more st udents too large for educating younger learners. Some recent research syntheses concluded that sma ll schools are better at raising student achievement, especially for minority and low income students. Small schools were also found to increase attendance, teacher satisfaction, and parent and community in volvement (see Williams, 1990; Ayers, Bracey, & Smith, 2000). Although seemingly dated, the Â“effective school sÂ” literature identified a strong school leadership, a focus on academic outcomes (success or ientation and academic press values), a positive and orderly school climate, and teachersÂ’ positive ex pectations for students as educationally effective factors for disadvantaged students (see, for ex ample, Brookover, Beady, Flood, Schweitzer, & Wisenbaker, 1979; Clark, Lotto, & Mc Carthy, 1980; Edmonds, 1979; Rowan, Bossert, & Dwyer, 1983. While this work faced criticisms from methodolog ical and substantive perspectives because studies often failed to arrive at common factors supported by solid statistical evidence (see for example, Purkey & Smith, 1983, Rosenholtz, 1985), some Â“effecti ve schoolsÂ” factors have surfaced in the recent literature on standards-based reforms, and merit furt her empirical testing at the elementary level. Early Findings from the ECLS The ECLS data, the focus of the present study, permits a unique look at subsets of a nationally representative sample of kindergartners on a range of cognitive, behavioral, and health indices. Zill and West (2001) reported descriptive statistics from the ECLS-K data from the 1998Â–99 kindergarten class. Their initial analyses show considerable variation in childrenÂ’s knowledge and skills as they enter school. Descriptive breakdowns of the first year samp le suggest that the variations in achievement were partially related to age, gender, and family risk factors. Family risk was defined based on low maternal education, welfare dependency or poverty st atus, having one parent at home, and having parents whose native language is not English. Child renÂ’s health and behaviors were, in turn, found to be dependent on family risk factors. Disturbingly, 33% of Hispanic students were found to have two or more family risk factors; 27% of African Americ ans fell in the same category; while 17% of Asians
Achievement Gaps and Correlates of Early Mathematics Achievement 7 had the same profile. In contrast, 6% of White stud ents were categorized in the same family risk category in the ECLS-K sample. Consistent with literature in clinical and child development fields, Zill and West (2001) also reported that socio-demographic risk factors, such as si ngle parent families, high levels of poverty, and transiency, were more common among kindergartners from ethnic minorities than among those from White families in the ECLS-K sample, and that the at-risk children started with lower scores on cognitive assessments in kindergarten. Given the policy directions set by the standards-based reform movement and NCLB, the need is high for additional and replicated evidence on di rect and moderating effects of particular school practices and policies in the early school years. New information has begun to emerge with data on the ECLS, but very little is available on school-level correlates and moderators of early mathematics achievement in high risk groups emphasized in the NCLB Further, it is not clear as to whether stable achievement gaps in mathematics form as early as in first grade, how large they are, and whether gaps are comparable in magnitude in all high risk subgrou ps identified in the legislation. The present study addressed these unresolved issues. Method Procedures used for identification or constructi on of childand school-level variable measures, for setting up a data set with complete informati on on chosen variables, and for testing particular analytic models, are described next. Schooland Child-level Factors Selected for Study Separate sets of factors were selected at the ch ildversus school-level for analysis. At the child level, variables included child development factors (age of the child in months and gender), sociodemographic factors (ethnicity, familyÂ’s socio-economic status), and cognitive measures taken prior to formal schooling (mathematics achievement scores in the beginning and at end of kindergartenÂ—Kentry and K-end scores). The ethnic minority sub-samples of interest in this study consisted of children coded as African-American (non-Hispanic), Asian, Hispanic (race specified), and Hispanic (race unspecified) students in the ECLS database. The term Â“povertyÂ” refers to students falling at or below the second quintile on the continuous measure of family socio-economic status used in the ECLS. At the school level, variables included appropriately aggregated context factors (poverty rate, school size), school inputs (mean class size, teacher certification ra tes), organizational practice/policy factors (success orientation and academic press va lues, levels of teacher support for planning and professional development, student attendance rate, inci dence of individualized educational plans (IEPs, an index of exceptional education services), an d several parent involvement factorsÂ–such as, on average, how long parents read to children at home every week; overall ed ucation support at home, parent satisfaction with school activities, and pare ntal involvement in school functions. School means on the child poverty index and prior achievement were built into the models as controls in some HLMs. Initially, all theoretically relevant variables wer e incorporated in the models; in the final models, variables with negligible and/or statistically non-si gnificant effects or excessive missing data were dropped from the analyses.
Education Policy Analysis Archives Vol. 13 No. 46 8 Variables/Measures: Operational Definitions Table 1 provides descriptive statistics on all va riables used in the analyses at the child and school levels. The operational definitions of differe nt constructs, their theoretical and psychometric bases, and coding methods are described briefly next, and detailed in Appendix A. Table 1 Descriptive Statistics on Level 1 and Level 2 Variables Variable name N Mean SD Minimum Maximum Level 1: Child Characteristics Age in months 2300 79.89 4.10 70.13 94.77 Math score at K-entry, C1RMSCAL 2300 20.75 7.18 7.18 56.91 Math score at K-end, C2RMSCAL 2300 29.26 8.51 9.14 57.25 Math score at Grade 1, C4RMSCAL 2300 45.06 8.37 12.20 60.50 African American (%) 2300 0.12 0.33 0 1 Hispanic-1 (%) 2300 0.06 0.25 0 1 Hispanic-2 (%) 2300 0.05 0.22 0 1 Asian (%) 2300 0.03 0.18 0 1 Poverty (%) 2300 0.29 0.46 0 1 Gender (Male=1; %) 2300 0.49 0.50 0 1 Level 2: School Characteristics Number of children nested in schools182 16.74 4.76 10 33 Teacher support 182 14.30 2.90 4 20 School success orientation 182 12.84 1.64 8 15 Student attendance rate 182 97.46 1.14 93.02 99.73 Education support at home 182 10.85 0.94 7.92 13.90 Parent satisfaction 182 6.23 0.77 4.84 10 Class size 182 21.50 4.74 11.84 52 Class time to reading and math 182 3.82 1.06 2 8 School size 182 0.85 0.36 0 1 Public versus private sector 182 0.89 0.31 0 1 Individualized Educational Plans 182 5.45 9.65 0 67.94 At home reading time 182 0.58 0.17 0.20 1 Parent involvement 182 6.65 0.80 5.10 9 Teacher certification rate-El ementary 182 0.79 0.28 0 1 School-level controls K-end Math 182 33.26 5.55 19.80 48.02 K-entry Math 182 22.83 4.55 12.81 38.26 Poverty rate 182 38.57 29.16 0 100 Minority rate 182 0.33 0.34 0 1
Achievement Gaps and Correlates of Early Mathematics Achievement 9 The ECLS mathematics assessment The ECLS mathematics assessment is a multi-level, interview-based assessment that was administered in two stages, using an adaptive design. First, children received a 12Â–20 item routing test. Th eir performance on the routing test determined the second stage form that would be at the appropriate di fficulty level for different children. For individual children, scores on the full set of test items wer e estimated using Item Response Theory (IRT) procedures. IRT equating methodology was used to place all items on a common scale, making scale scores comparable across children at different leve ls, regardless of the items to which they responded. The item content of the ECLS mathematics assessment is as follows: number and shape (identifying one-digit numerals, recognizing geometr ic shapes, and one-to-one counting of up to 10 objects); relative size ( reading single-digit numbers, counting beyond 10, recognizing a sequence of patterns, and using non-standard units of length to compare objects); ordinality and sequence (reading 2-digit numerals, recognizing the next number in a sequence, identifying ordinal position in a sequence, and solving a simple word problem); addition and subtraction (solving simple addition and subtraction problems); and multiplication and division (solving simple multiplication and division problems and recognizing more complex number patterns). Reliability of scores from the ECLS mathematics assessments are reported to be in the following ranges: .88Â–.95, for ability estimates on IRT scale scores; and .78Â–.88, for the routing tests (NCES et al., 1999). First grade mathematics ou tcome measure (C4RMSCAL ). IRT-scaled scores from the ECLS mathematics assessment, taken at the end of firs t grade, were used a dependent variables in HLM equations after their distribution properties were examined in the larger ECLS sample. Prior achievement in mathematics (C1RMSCAL, C2RMSCAL). IR T-scaled scores from the ECLS mathematics assessment, taken either at the be ginning (K-entry) or end of kindergarten (K-end), were used as child-level predictors or as dependen t variables after their distribution properties were examined in the overall ECLS sample with all available cases. Both the kindergarten measures were preferred over prior mathematics measures taken at the beginning of first grade (C3RMSCAL), as about 2/3 of the original sample had missing data on this measure. A possible summer lag between the end of kindergarten and first grade was anticipated to have been made up by the end of the Grade 1 year. Survey constructs Composite measures were created with selected item sets from different ECLS questionnaires to serve as predictors. Indices created with child-level data were aggregated by school, and treated as school-level factors. The desc riptions of each survey construct, data sources from which they were extracted, and results of vario us psychometric analyses on variables, are detailed in Appendix A. Exploratory factor analytic work (principal components analysis followed by varimax rotation) was done to extract relatively independent but th eoretically meaningful factors. Internal consistency reliability estimates of factor-supported composites were obtained using Cron bachÂ’s alpha prior to incorporating the index scores into HLM equations as childor school-level factors. Item-factor loadings of .40 and above, and CronbachÂ’s alpha esti mates of .70 were used as criteria for psychometric defensibility of indices. All available cases in th e ECLS database were used for various psychometric analysesÂ—the range of cases varied from 4,637 to 11,379 for these examinations. Unique variable definitions In this study, the Â“special educationÂ” service variable refers to the incidence or percent of IEPs, aggregated by school. That is, the percents of children in the study sample were those identified by their schools as havi ng IEPs on file. Appendix B, Table B3 provides the frequency breakdown on the ECLS survey item th at served as the data source (U4IEP) for this variable. The unique variable definition yields lower percents than national estimates of students with disabilities (typical range from 10Â–15%). Elsewher e, Walston and West (2004) reported the percent
Education Policy Analysis Archives Vol. 13 No. 46 10 representation of students with disabilities in th e larger ECLS sample was 12% (2568 out of 21260), close to the national estimate. Likewise, the Â“povertyÂ” variable is not based on student enrollment in free or reduced lunch programs at schools (as found in many policy studies), but rather the ECLS composite for socioeconomic status computed for individual children ba sed on parent education, occupation and income (see Appendix A). The lower two quintile ranges in this continuous SES measure were selected as a cut to identify Â“high povertyÂ” students. The percents in the original ECLS sample and initially screened study sample were thus at or around 40%; the final 2300 cases had a lower poverty rate (29%) by this definition (see Table B1 in Appendix B). However, when school-level aggregates were computed, the mean poverty rate across schools was abou t 39% (see Table 1, lower panel). Finally, Â“Reading TimeÂ” given by parents at hom e was based on an individual item that was separate from the items used to construct remaining pare nt involvement composites from the Parent Questionnaire. This item asked how long parents read to their child at home (more than 30 minutes per week versus less, a (1/0) binary variable). The data were incorporated in the models as a schoollevel aggregateÂ–a proxy for family care/parenting variable shown to influence future academic performance of children. Prior to running the HLMs, the relationship of th is variable with other school-level aggregates, such as a schoolÂ’s aggregated poverty rate across sampled children, was examined to identify possible suppressor effects (see Appendix B, Table B2, for a correlation table). The magnitude of these overlaps was judged to be too small to cloud subsequent interpr etations of results with multiple predictors. As is evident, Reading Time, averaged by school, correlated -.087 (n.s.) with the Poverty rate, and .18 ( p < .05) with the Educational Support variable. The latt er correlation suggested that schools with higher levels of parental support for education also had hi gher at-home reading time values. Several of the correlations in Table B2 were similarly meaningful and were formally tested with HLMs. Sample Size and Composition Utilization of multiple data elements and composite variables necessitated a careful examination of missing data in the data set prep ared for analysis. Some decisions were made on how best to retain an optimal sample size with as much complete data as possible on variables that were relevant to the research objectives. Normalized sa mple design weights using the C124PW0 variable were applied, as recommended by ECLS staff, to retain original degrees of freedom. To start, all grade-retained children were scr eened out by including only those who were firsttime kindergartners (P1FIRKDG=1) and first-time first graders in traditional classrooms and schools (T4GLVL=4) with data available on the weighting variable. Next, new variable measures were constructed as described previously and a child-level file was created using all cases with complete data on gender, poverty, race, and prior achievement and ou tcome indicators. Lastly, a school-level file was created, aggregating all school factors to be tested in HLMs, including at least 10 students within each school. Missing values on 6 constructs were impute d with the mean substitution procedure in SPSS (detailed at the end of Appendix A). The final data set for the present analysis yielded 182 schools with 2300 cases. The composition of the screened study sample with weighted and unweighted cases, compared to the original ECLS sample and the final data set, is described in Appendix B, Table B1 on key demographic variables.
Achievement Gaps and Correlates of Early Mathematics Achievement 11 Descriptive Statistics on Child and School-level Variables The top panel in Table 1 shows descriptive statistics on child-level (Level 1) variables in the dataset ( N =2300). As is evident, 12% of the sample were Blacks, 6% belonged to the Hispanic-1 group (race specified), 5% were Hispanic-2 (ra ce unspecified), and 3% were Asian children. The children had a mean age of 79.89 months at the be ginning of first grade; 29% were from high poverty households; and about half (49%) wer e male. At the school level, the number of children nested in schools varied from 10Â–33, with a mean of 17. This number is different from the Â“class sizeÂ” variable and reflects children sampled by school per the ECLS sample design, where on average, 5 children and 3 classrooms were sampled from each school. Analytic Models A series of two-level models, with childrenÂ’s achievement modeled at the child-level, nested under schools at the second level, were run with the HLM Version 5.02 program (Raudenbush, Bryk, Cheong & Congdon, 2001). The research ra tionale for each HLM was as follows. Unconditional (null) model The first analysis involved the use of a one-way random effects ANOVA, also called the unconditional or null model. This analysis was motivated by the need to partition the total variance in achievement into withinand between-school components. The variance estimates were obtained by fitting an HLM where each childÂ’s end-of-f irst grade achievement score, yij, is explained by the estimated school mean, j0 and unique error associated with that child, rij. School means were explained by the grand mean, G00 and unique error for each school, uj. yij = j0+ rij. j0 = G00 + uj. The analysis with the null model yielded answer s to three questions: How much do individual students vary around their sc hool means ? How much of to tal variance in mathematics achievement is attributab le to schools? and How precise an esti mate of the population mean is the school mean, j0? These questions were answered by ex amining the variance estimates within schools ( 2) and between schools ( ), and the size of the intraclass correlation (proportion of total variance that is be tween schools). ICC values greater than .10 indicate that there are sufficient within-school dependencies to justify multi-level analysis. A reliability es timate for school mean estimates (intercepts, j0), is reported in the HL M output. A criterion of .60 was set for the reliability of the intercepts. Subsequent models incorporating predictors at both levels were evaluated against these initial variance estimates. Random intercepts model with only child-level predictors The next series of HLMs was specified to answer two main questions: Given the estimated withinschool variance, what proportion of that variance in achievement can be accounted for by child background characteristics, such as their age, prior achievement in math gender, poverty, and ethnicity ? Compared to school mean estimates and controlling for other child background characte ristics, how large are the achievement gaps in selected ethnic, poverty, and gender groups? The sc hool-level model remained as in the null model; predictors were all entered in the child level equation. All predictors were centered around their school means; thus, the estimated coefficients for each risk group showed the within-school achiev ement differential, controlling for the other child
Education Policy Analysis Archives Vol. 13 No. 46 12 background characteristics serving as predictors in the models but allowing variability between schoolsÂ—a primary research objective for the study. yij = j0+ j1 ( African American)+ j2 ( Hispanic-Race specified) + j3 ( Hispanic-Race unspecified) + j4( Asian) + j5( Prior Achievement) + j6( Poverty) + j7( Male) + j8( Age)+ rij. j0 = G00 + uj.. j1 = G10 j2 through j8 ( all slopes fixed as in j1). Gaps were estimated at the end of kindergarten and in Grade 1, using both the K-entry and Kend mathematics measures as prior achievement indicators in separate models. Findings were compared. The models were evaluated against the null model by examining the proportion of unexplained within-school variance that was accounted for, after all the child-level predictors were included in the model. Random intercepts model with child leve l covariates and school-level predictors In the third series of HLMs, theoretically-supported school variables were modeled to explain between-school variability in achievement, with child-level predictors also entered. Effects of child-level predictors that were main focus of the study, ethnicity, gender, and poverty were again allowed to vary randomly between schoolsÂ—that is, they were group (school) mean-centered. School aggregates on poverty and prior achievement were reentered in second level eq uations as context controls to study effects of other school factors, adjusted for these average eff ects. Remaining child-level factors, such as age, were now centered around the grand mean, and served as a constant across schools. Again both the K-entry and K-end mathematics measures were used as covari ates at the child-level in different runs to compare findings. yij = j0+ j1 ( African American)+ j2 ( Hispanic-Race specified) + j3 ( HispanicRace unspecified) + j4( Asian) + j5( Prior Achievement)+ j6( Poverty) + j7( Male) + j8( Age)+ rij. j0 = G00 + G01( Mean Poverty) + G02(Mean Prior Achievement)+ G03 (School Size) + G04 (Mean Class Size)+ G05(Mean Teacher Variables) + G06 (Mean Organizational Variables) + G07 ( Mean Reading Time) + *G08 (Mean Parent Involvem ent Variables) + uj, where j1 = G10 j2 through j8 (all slopes fixed as in j1). (* indicates that there is more than on e coefficient in this predictor category.) This series of HLMs revealed the degree to wh ich the selected school factors significantly and positively influenced first grade mathematics achiev ement in schools, as main, additive effects. The reduction in the variance estimate of uj enabled a calculation of the pr oportion of originally estimated between-school variance (from the Null Model) that could be explained by the school factors chosen, and an evaluation of the usefulness of models. A co mparison of models examining K-entry to Grade 1 versus K-end to Grade 1 effects was also made. Slope parameters for predictors at the child level were again fixed; that is, the influence of individual child -level characteristics was not set to vary by school.
Achievement Gaps and Correlates of Early Mathematics Achievement 13 Random intercepts and slopes models, with explanatory variables to examine cross-level interactions Finally, to answer questions as to whet her mathematics achievement varied due to interaction of a childÂ’s risk group membership mo deled in the child-level equation and schools in which they belonged, modeled at the second-level, slopes for these predictors were next allowed to vary randomly. Here, only the K-end mathematics me asure was used as a covariate. When statistically significant slope variance was found, the equation for that slope parameter was modeled with schoollevel predictors to identify significant explanatory variables. For example, with the poverty variable, the questions were: Does mathematics achievement vary significantly in poor versus well-to do children who belong in different schools? If so, what schoo l variables significantly explain the achievement variance in the slopes? The equations were: j6( Poverty) = G60+ u6, to examine if the poverty slope, j6, had significant variance when modeled as a random variable. To identify significant explanatory variables, the subsequent equation was built as follows: j6 = G60+ +G61 (Mean Minority)+ G62( Mean Poverty) + G63 (Mean Prior Achievement) +G64( School Size) +G65( Mean Class Size) + *G66 (Mean Teacher Variables) + *G67 (Mean Organizational Variables) *G68 (Mean Parent Involv ement Variables)+ u6. These analyses were pursued with one slope mo deled at a time. The number of schools with necessary data at the school leve l dropped in the crosslevel models and are reported in tables with results. The reliability of the slopes and intercepts was checked at each stage of the analysis and are also reported. Results The results of the final models are presented in Tables 2Â–10. A few predictors that did not significantly contribute to the variance of achiev ement (such as early childhood certification rate) were dropped from reduced models. Child versus School Variance Comp onents in First Grade Mathematics How much variability in first grade mathematics achievement could be attributed to children versus schools? The results with the unconditional model for mathematics are presented in Table 2. Children were found to vary significantly around thei r school means, as evidenced in the statistically significant t value. The within-schools variability in achi evement was estimated at 66.06; the betweenschools variability was estimated at 16.37. This yiel ded an intraclass coefficient (ICC) of .198, showing that about 20% of the total variability in mathematic s achievement could be attr ibuted to schools (i.e., between-school variance). The estimated school mean was 44.28 ( SE =0.34). The reliability of this estimate was .76.
Education Policy Analysis Archives Vol. 13 No. 46 14 Table 2 First Grade Mathematics Va riance Partitioned to Students versus Schools: Results from Null Model Variable Parameter estimates d.f. p Fixed Effect Coefficient SE t School meana, G00 44.28 0.34 129.30 181 < .01 Random Effect Variance Component 2 School level effect, uoj 16.37 831.95181 < .01 Child level effect, rij 66.06 a Reliability of intercept (school mathematics mean estimates at end of Grade 1) = .76 Variance Attributable to Schools (ICC) = [16.37/(16.37+66.06)]= .20 Mathematics Achievement Gaps in Kindergarten and First Grade What were the within-school achievement different ials in kindergarten and first grade of different ethnic, gender and poverty groups, adjusting for K-entry versus K-end variability in childrenÂ’s mathematics achievement? How much of the within-school variance in first grade achievement was accounted for by the chosen predictors? Tables 3Â–5 provide the estimates on the size and significance of mathematics achievement gaps and child-level pred ictors. Figures 1-3 present a pictorial view of the achievement gaps. Table 3 Kindergarten Achievement Gaps: Results from Random Intercepts Model with Level 1 Predictors and K-Entry Measures of Prior Achievement Variable Parameter estimates d.f. p Fixed effects Coefficient SE t School meana, G00 28.20 0.35 80.58 181 < .01 Within-school effects Age in months 0.02 0.03 0.66 2291 .51 Poverty Status Gap -0.56 0.25 -2.23 2291 .03 African American Gap -1.55 0.45 -3.40 2291 < .01 Hispanic-1 (Race sp.) Gap 0.14 0.49 0.28 2291 .78 Hispanic-2 (Race Not sp.) Gap -0.90 0.46 -1.95 2291 .05 Asian Gap 0.01 0.54 0.02 2291 .99 Gender Gap (Male vs. Female) 0.08 0.24 0.36 2291 .72 Achievement at K-entry 0.92 0.02 42.00 2291 < .01 Random effects Variance Component 2 School level effect, uoj 20.43 2276.48 181 < .01 Child level effect, rij 24.61 a Reliability estimate of intercept = .91 Within-school variance accounted for by Leve l 1 predictors: [(66.08-24.61/66.08)]= .64. It should be noted that Â“achievement gapÂ” in Tables 3Â–5 are the multilevel regression coefficients. They are interpreted as the achievem ent difference of a selected subgroup (e.g., African American vs. others) as compared to the estima ted school mean (the intercept), controlling for prior achievement and other demographic characteristics.
Achievement Gaps and Correlates of Early Mathematics Achievement 15 Table 3 show that achievement gaps at the end of kindergarten, controlling for K-entry mathematics achievement, are significant in high poverty, African American, and Hispanic-2 (race not specified) subgroups. The school mean is estimated at 28.20. A high poverty child is estimated to score lower than the school mean by -0.56 units ( p < .05); an African American child by -1.55 units ( p < .001), and Hispanic-2 children by -0.90 ( p =.05). There is no gender gap evident. Children who have higher mathematics scores at the start are predicted to do si gnificantly better at the en d of kindergarten with a coefficient of +.92 ( p < .01), controlling for other factors within schools. About 64% of the variability in childrenÂ’s mathematics achievement at the end of kindergarten is explained by the predictors modeled. Table 4 coefficients show that significant mathematics gaps are manifested again in first grade in African American children versus other ethnic groups (-2.01 points, p < .01) and in high poverty versus well-to-do children (-1.72, p < .01), but not for the Hispanic subgroups. There is now a small but significant gender gap, with males s howing an advantage over females (+0.53, p < .10). K-entry mathematics scores are modeled with other predicto rs as covariates. The within-school mean estimate is close to the estimate in Table 1, 44.96 ( SE =0.36). About 43% of the variability in childrenÂ’s mathematics achievement in Grade 1 is explained by the predictors modeled, a lower proportion than that in Table 3 possibly because schooling and other background factors begin to influence mathematics outcomes more heavily in Grade 1 than at K-end. Table 4 First Grade Mathematics Achievem ent Gaps: Results from Random Intercepts Model with Level 1 Predictors and Kindergarten Entr y Measures of Prior Achievement Variable Parameter estimates d.f. p Fixed effect Coefficient SE t School meana, G00 44.96 0.36 122.15 181 < .01 Within-school effects Age in months -0.05 0.03 -1.33 2291 .18 Poverty Status Gap -1.72 0.38 -4.49 2291 < .01 African American Gap -2.01 0.54 -3.70 2291 < .01 Hispanic-1 gap (Race specified) 0.64 0.49 1.30 2291 .19 Hispanic-2 gap (Race not specified -1.00 0.69 -1.43 2291 .15 Asian Gap -0.48 0.65 -0.74 2291 .45 Gender Gap (Male vs. Female) 0.53 0.27 1.89 2291 .05 Achievement at K-entry 0.75 0.03 23.05 2291 < .01 Random Effect Variance Component 2 School level effect, uoj 14.63 1178.73 181 < .01 Child level effect, r ij 37.77 a Reliability estimate of intercept = .83 Within-school variance accounted for by Leve l 1 predictors: [(66.06-37.77)/66.06] = .43. With K-end mathematics scores used as prior achievement measures in Table 5, the gaps reduce marginally in size but remain statistica lly significant at the .05 alpha level in the same groups of children. The observed change in gap estimates is small, but the effects of the kindergarten yearÂ’s experiences on Grade 1 mathematics achievement are evidenced in the 10% difference in achievement
Education Policy Analysis Archives Vol. 13 No. 46 16 variance explained. In all, 53% of the within-school variance in achievement is accounted for by the child-level predictors in the second model. Th e school-level mean in first grade mathematics achievement is again close at 44.71 ( SE =0.36). The reliability of the school mean estimates in both first grade models is high at .83 and .87, respectively. Table 5 First Grade Mathematics Achievem ent Gaps: Results from Random Intercepts Model with Level 1 Predictors and K-end Measur es of Prior Achievement Variable Parameter values d.f. p Fixed effect Coefficient SE t School meana, G00 44.71 0.36 121.92 181 < .01 Within-school effects Age in months -0.03 0.03 -0.90 2291 .37 Poverty Status gap -1.40 0.33 -4.19 2291 < .01 African American gap -1.24 0.50 -2.48 2291 .01 Hispanic-1 gap (race specified) 0.39 0.52 0.76 2291 .44 Hispanic-2 gap (race not specified -0.53 0.66 -0.79 2291 .42 Asian gap -0.53 0.54 -0.99 2291 .32 Gender gap (M ale =1) 0.49 0.25 1.95 2291 .05 Achievement at K-end 0.70 0.01 36.36 2291 < .01 Random Effect Variance Component 2 School level effect, uoj 16.19 1524.32 181 < .01 Child level effect, r ij 30.84 a Reliability estimate of intercept = .87 Within-school variance accounted for by Leve l 1 predictors: [(66.06-30.84)/66.06] = .53. As expected, gaps are slightly smaller in magnitude in Table 5 than those presented in Table 4. This is because childrenÂ’s initial mathematics variability is now statistically controlled using mathematics measures taken at K-end, just prior to their entry into first grade. The within-school Grade 1 results indicate the following (Table 5): an increase in childrenÂ’s age by one month marginally drops the school mathematics mean by 0.03 units ( p =.37), childrenÂ’s membership in the high poverty group (poverty status=1, versus others) signific antly decreases the mean by -1.40 units ( p < .01). A male child, versus females, scores 0.49 unit s above the estimated school mean ( p =.05) in mathematics. An African American child, versus others, is estimated at scoring -1.24 units below the estimated school mean ( p < .01). Hispanic-1 children (race specified) score 0.39 units above; Hispanic-2 (race unspecified) children score -0.53 units below; while an Asian child is estimated to score -0.53 units below the school meanÂ—but none of the achievement di fferentials for the last three ethnic groups are statistically significant at the 10% error level once K-end mathematics variability is controlled. Finally, for every unit increase in the mathematics achievemen t score at the end of kind ergarten, a first grader scored 0.70 units higher on the firs t grade measure of mathematics ( p < .01).
Achievement Gaps and Correlates of Early Mathematics Achievement 17 28.2 26.65 28.34 27.3 28.21 25.5 26 26.5 27 27.5 28 28.5 School MeanAfrican American Hispanic (Race sp.) Hispanic (Race Not sp.) Asian Ethnic Group PerformanceMath Score Figure 1. End of Kindergart en Math Achievement Gaps Contr olling for K-Entr y Achievement. 44.96 42.95 45.6 43.96 44.48 41.5 42 42.5 43 43.5 44 44.5 45 45.5 46 School MeanAfrican American Hispanic (Race sp.) Hispanic (Race Not sp.) Asian Ethnic Group PerformanceMath Score Figure 2. First Grade Math Achievement Gaps Controlling for K-Entry Achievement. 44.71 43.47 45.11 44.1844.17 42.5 43 43.5 44 44.5 45 45.5 School MeanAfrican American Hispanic (Race sp.) Hispanic (Race Not sp.) Asian Ethnic Group PerformanceMath Score Figure 3. First Grade Math Achievement Gaps Controlli ng for K-End Achievement. SchoolÂ–Level Correlates of Mathematics Achievement Which school practice or policy variables influe nce mathematics achievement of first graders? The results are presented separately in Tables 6Â–7 for models that used K-entry versus K-end measures of mathematics as covariates in the Level 1 models, and show consistent results, with one exception. In
Education Policy Analysis Archives Vol. 13 No. 46 18 all, 75Â–79 % of the estimated between-school variance reported in Table 2 (16.37), was explained by the modeled school level predictors, showing the utility of the models (see table notes). Table 6 Main Effects of School Variables in Level 2: Results of HLM with Level 1 Covariates and KEntry Measures as Controls Variable Parameter estimates d.f. p Fixed school effect Coefficient SE t School meana, G00 44.47 0.28 154.62 166 < .01 Fixed school effects Class size -0.05 0.03 -1.64 166 .09 Frequency of Individualized educational plans -0.02 0.02 -0.95 166 .34 Student attendance rate 0.00 0.23 0.01 166 .99 Parental support for educat ion -0.28 0.23 -1.23 166 .21 Parental satisfacti on with school -0.05 0.25 -0.23 166 .81 School size 0.95 0.55 1.71 166 .08 Public school 0.36 0.64 0.57 166 .56 Time parents read to children at home 1.73 1.04 1.65 166 .09 Parent Involvement leve ls -0.25 0.37 0.69 166 .48 Teacher support -0.08 0.07 -1.24 166 .21 School success orientation -0.16 0.10 -1.57 166 .11 % Teachers with Elementary Certification 0.51 0.61 0.84 166 .40 Class time dedicated to reading and math 0.24 0.18 1.28 166 .19 Context Controls at Sc hool-level % Low SES students -0.002 0.01 -0.19 .84 K-Entry Achievement 0.68 0.07 9.48 166 < .01 a Reliability estimate for intercept = .55 Between-school variance in achievement accounted fo r by school predictors: [(16.37-3.41)/16.37)]=.79. In Table 6, effects are estimated following childrenÂ’s exposure to kindergarten and first grade experiences cumulatively, equalized across school s on K-entry mathematics scores but varying on ethnicity, poverty and gender. Consistent with prior research, class size has a small negative and significant effect at the 10% error level (-0.053, p < .10), showing that with average increases in teacherreported numbers of children in classrooms, ther e is a small drop in school mathematics means. Because the effects are estimated on school aggregates the influences are smaller in magnitude than may be obtained for individual childrenÂ’s scores. A large positive correlate is the schoolÂ’s average on parent-reported at-home reading time. Here, a unit change (30 minutes or more versus less) in the predictor results in gains of 1.73 unit s on school mathematics achievement means ( p < .10). School size is the third statistically significant correlate at the 10% level, but in an unexpected direction. Operationalized as 500+ versus less than 500 students (a binary variable), a value of 1 resulted in 0.95 point increase in school mathematics means, indi cating that larger school sizes yielded higher mathematics scores in schools.
Achievement Gaps and Correlates of Early Mathematics Achievement 19 Table 7 Effects of School Variables: Resu lts of HLM with Level 1 Covariat es and K-end Measures of Prior Achievement Variable Coefficient SE t d.f. p Fixed effect School meana, G00 44.30 0.26 165.97 166 < .01 School effects Class size -0.05 0.03 -1.65 166 .09 Frequency of Individualized educational plans -0.01 0.02 -0.92 166 .35 Student attendance rate 0.00 0.23 0.03 166 .97 Parental support for educat ion -0.30 0.23 -1.31 166 .19 Parental satisfacti on with school -0.05 0.25 -0.20 166 .84 School size 0.89 0.54 1.64 166 .09 Public school 0.45 0.63 0.72 166 .47 Time parents read to children at home 1.71 1.06 1.61 166 .10 Parent Involvemen t levels -0.34 0.37 -0.90 166 .36 Teacher support sy stems -0.09 0.07 -1.27 166 .20 School success orientation -0.17 0.10 -1.66 166 .09 % Teachers with Elementary Certification 0.57 0.60 0.95 166 .34 Class time dedicated to reading and math 0.24 0.18 1.28 166 .19 Context Controls in school-level equations: % Low SES students (school aggregate) -0.00 0.01 -0.40 .68 K-end Achievement 0.69 0.07 9.33 166 < .01 a Reliability estimate for intercept = .63 Between-school variance in achievement accounted fo r by Level 2 predictors: [(16.67-4.00)/16.67)]=.76. In Table 7, children vary in terms of gender, ethnicity, and poverty, and are now equalized on prior mathematics preparation at K-end. Thus, pupils can be expected to be less variable at the end of Grade 1 than the model in Table 6 and only Grade 1 effects are estimated. Again, class size, reading time at home and school size variables have similar and significant effects, in the same directions. In addition, a counter-intuitive result is obtained with the school success variable, where unit increases on the composite resulted in small drops in school means by -0.18 points ( p < .10). The other school predictors do not have significant main effects. Interaction of ChildÂ’s African A merican Status with Schools Because African American children were found to have statistically significant gaps, a further question dealt with whether their performance varied by school when K-end measures were controlled. Other ethnic group interaction analyses were prohi bited by a reduction in the number of schools available for analysis that included children from Asian and Hispanic subgroups.
Education Policy Analysis Archives Vol. 13 No. 46 20 Table 8 Cross-level Interaction Effects of African Americ an Group Membership and School on First Grade Mathematics Variable a Parameter values d.f. p Random effects Variance Component 2 School level intercepts, uoj 3.51 138.5057 < .01 African American slopes, u3j 4.36 82.6272 .18 Child level effect, r ij 37.44 Fixed effects Coefficient SE t Intercept for Af. Am. slope -0.59 0.43 -1.38 166 .16 Class size -0.18 0.11 1.62 166 .10 % Low SES students 0.06 0.02 2.85166 < .01 School size -2.13 1.10-1.93166 .05 Parent involvement 1.81 0.80-2.25166 .05 a Reliability estimate of intercept = .46; Reliability estimatee of slope = .19 Variance explained in slopes = [ (4.36-1.29)/4.36] = .70. Table 8 shows that there was no significan t cross-level interaction between a childÂ’s membership in African American versus other ethni c group and their school, as evidenced in the results on the slope parameter. The intercept fo r the African American slope was -0.599 ( p =.16), a statistically non-significant and somewhat lower value than the initial gap estimates in Tables 3-5. The initial variance estimate for school slopes was 4.36 ( p =.18). The lack of a significant interaction in this model was based on 73 schools only (see d.f. and chi squared value), and suggested that while African American childrenÂ’s mathematics achievement varied, it was not significantly different based on school membership. Because initial deficit estimates were significan t in this subgroup, an explanatory model was built to isolate school factors that accounted for wh atever school slope variance was manifested in the data with 73 schools. These results showed that wi th increased class sizes, African American children scored significantly lower by -0.18 units, over an d above the initial deficit estimate of -0.599 units ( p =.10.); with increased school sizes, likewise, th ey scored significantly lower by -2.13 units ( p =.05); in high versus low poverty schools their performance remained much the same (.06, p < .01); in schools with higher parent involvement levels, thei r performance was significantly lower (-1.81, p =.06)Â—a counter-intuitive finding. These variables dropped th e initial variance estimate to 1.29, accounting for 70% of the variability evidenced in school slopes. Interaction of ChildÂ’s Poverty Status with Schools Table 9 presents the results on the second interaction question showing that mathematics performance of high and low poverty children varied significantly by school (variance estimate of slope, u7j =5.51, p =.000). The intercept indicated that acro ss all schools, high poverty children were scoring -1.01 units lower compared to low poverty children (compare with gap estimates in Tables 35). Significant explanatory variables for slope vari ability were school averages on total time teachers gave per day to reading and mathematics instru ction (30Â–60 minutes per day versus less) with a positive coefficient of +0.54 ( p =.08); IEPs in school, with a small negative coefficient of -0.06 ( p < .10); and public versus private sector with a larger negative effect of -2.60 ( p =.03). The last result showed that public schools fare worse than pr ivate/other schools in affecting mathematics performance of high poverty students. In this analys is, schools with higher poverty rates scored as well
Achievement Gaps and Correlates of Early Mathematics Achievement 21 as those with lower poverty rates (+0.03, p =.03). The intercept reliability was found to be .47, and initial slope reliability was 0.31, with 10% of vari ance in slopes accounted for by the school factors modeled (final variance estimate=4.97). Table 9 Cross-level Interaction Effects of Poverty Gr oup Membership and Sc hool on First Grade Mathematics Variablea Parameter estimates d.f. p Random Effect Variance component 2 School level intercepts, uoj 3.54 258.56 137 < .01 School poverty slopes, u8j 5.51 222.14 152 < .01 Child level effect, rij 29.88 Fixed Effect Coefficient SE t Intercept for Poverty Slope -1.01 0.34 -2.94 166 < .01 Class time to Reading and Math 0.54 0.31 1.72 166 .08 Individualized Educational Plans -0.06 0.03 -1.69 166 .08 Public school -2.60 1.23 -2.14 166 .03 % Low SES Students 0.03 0.02 1.75 166 .07 a Reliability estimate of intercept = .47; Reliability estimate of slope = .31 Variance explained in slop es= (5.51-4.97)/5.51= .10. Interaction Effects of ChildÂ’s Gender with Schools Finally, Table 10 shows that the effect of a ch ildÂ’s gender on mathematics achievement varied by school (variance estimate of slope=2.11, p =.005). Overall, the intercept, reflecting the mean achievement of boys, was 0.30 units higher than for girls in first grade (the earlier gap estimates were around 0.49, about 0.19 units higher). One school-level factor, teacher certification rates in elementary education, yielded a statistically significant an d large positive effect for boys versus girls (+2.49, p =.009). Statistically significant explanatory variab les that had negative effects for boys versus girls were the following: schoolsÂ’ teacher support levels (-0.17, p =.05) and school averages on parent involvement (-0.87, p < .05). To interpret these negative findings for teacher support, as an example, boys scored 0.17 units lower in ma thematics than girls in schools where teacher support was reportedly a unit higher. The reliability of intercepts in the full analytic model was .54, and the reliability of the slope was .19. A total of 24% of the variance in sl opes was explained by the modeled school variables (final variance estimate for slopes=1.61).
Education Policy Analysis Archives Vol. 13 No. 46 22 Table 10 Cross-level Interaction Effects of Gender Gr oup Membership and Sc hool on First Grade Mathematics Variablea Parameter estimates d.f. p Random effects Variance Component 2 School level intercepts, uoj 5.32 375.99 162 < .01 School level slopes, u8j 2.11 229.98 177 < .01 Child level effect, r ij 30.24 Fixed effects Coefficient SE t Intercept for Male Slope 0.30 0.25 1.20 167 0.22 Elementary Teacher Certificatio n 2.49 0.95 2.61 167 < .01 Teacher Support -0.17 0.09 -1.95 167 0.05 Parent Involvement -0.87 0.52-1.66 167 0.09 a Reliability estimate of intercept = .54; Reliability estimate of slope = .19 Variance explained in slopes = [(2.11-1.61)/2.11] =.24. Discussion The ECLS data allowed a deeper examination of achievement gaps and school factors that directly affect or moderate childrenÂ’s mathematics achievement levels in isolated subgroupsÂ—thus addressing a void in the literature. The merit of pa rticular school practices and policies, such as class size reduction and higher teacher qualifications, coul d also be evaluated in the context of mandatory school reforms ensuing from legislation such as the NCLB Act. Several findings in the present study support current educational policy directions; ot hers contradict conventional or theoretical expectations. In conclusion, these findings are discussed along with limitations and areas for further research. Representativeness of the ECLS Data Set Table B1 (in Appendix B) and Table 1 show that the present sample was generally representative of the national ECLS sample in terms of gender and ethnicity, but somewhat underrepresented on the poverty status variable. The smaller number of cases ( N =2300 in 182 schools) resulted from a search for an optimal data set with complete data on all school and child-level variables of interest in the present investigation. In the final data set, enough cases were present in each of the subgroups of interest and on variables selected for study to enable examination of main effects at each level and cross-level interactions on school practice/policy issues discussed in the review of literature. Mathematics Achievement Gaps In interpreting the achievement gaps in the present study, it should again be borne in mind that gap estimates are regression coefficients that repr esent differences of a defined group on the outcome measure, as compared to the school means and controlling for other modeled child background factors. This definition is different from the NCLB approach, which uses mean differences computed with a specified reference group in mi nd (e.g., African American versus White).
Achievement Gaps and Correlates of Early Mathematics Achievement 23 The results show that the pattern of ethnic and gender gaps in mathematics changes from kindergarten to first grade. However the influence of higher poverty levels is consistently negative on achievement, even when prior knowledge, gend er and ethnicity are controlled. Likewise, prior mathematics preparation has a strong positive and significant effect on subsequent achievement measures, with other factors held constant. In kindergarten, African Americans and one Hispanic group show signs of emerging gaps, but the gap in Hispanics declines to a non-significant level in Grade 1. Also, there is no gender gap evident until formal mathematics instruction begins in Grade 1. The statistically significant advantages estima ted for children who start with higher initial mathematics measures (measures taken either at K-en try or K-end) are consistent with prior research discussed in this paper. Institution of statistical controls on pre-Grade 1 ma thematics achievement in the analytic models wipes out potential differences in quality of early childhood care, parenting, and known differences in full and half-day kindergarten exposure in different children in the ECLS sample. All these variables have been shown in the literatur e to be factors that influence school readiness of entering first graders, particularly those who start with academic challenges. The quality of educational experiences before and during the kindergarten year may be critical in influencing subsequent mathematics achievement patterns in diverse children. From a school practice and policy perspective it is clear that children of different minority groups exhibit slightly different patterns of mathematics achievement in Grade 1. Contrary to expectations set by prior research on ethnic achievem ent gaps in both older and younger pupils (Lee, 2002; NCES, 1995a, 1995b; Zill & West, 2001), Hispanic children did not show statistically significant mathematics achievement gaps in first grade, irre spective of whether K-entry or K-end mathematics variability was controlled. Asians were estimated to sc ore slightly lower, although this last coefficient was again not better than chance. That significant ethnic achievement gaps are manifested in African American, but not in other ethnic subgroups, when K-end or K-entry mathematics achievement variability is controlled in first grade children should be noted. Likewise, that mathematics gender differences are small but significant as early as in first grade should be also noted by educators and policy-makers. The finding on mathematics gaps in children from economically disadvantaged families is consisten t with results on national tests at all levels of schooling. It is imperative that such gaps are follo wed through elementary school and as children start their middle and high school years. Shifts in the size of the mathematics gaps should be closely monitored. More importantly, schools should find ways to address domain-specific needs of learners as they arise at the classroom level. Timely detecti on and diagnosis of curriculum-specific needs in learners would give schools and teachers the opportunity to prevent mathematics gaps from developing in later years. School Level Correlates of Mathematics Achievement When partitioned, about 20% of the total variab ility in childrenÂ’s mathematics achievement was between-schools variance. The main eff ects models explained about 75Â–79% of that between-schools variance estimate, showing class size, reading time at home, school size, and school success (academic press) orientation as significant school level corre lates at the 10% error levelÂ—consistent outcomes on three counts controlling for K-entry or K-end vari ability (Tables 6Â–7). Alimitation is that these correlates were significant at the 10% error level. However, the finding that class size negative ly affected school achievement means is also consistent with results from other research efforts, such as the Tennessee STAR investigations (Finn & Achilles, 1999), which showed positive effects of reduced class size in early grades. The effects reported here are of unit increases in mean teac her-reported class size on school mathematics score
Education Policy Analysis Archives Vol. 13 No. 46 24 averages; hence they are small in size. Each unit in the class size variable stands for an additional child. The descriptive data (Table 1) suggest that class sizes in ECLS schools went up to as many as 52 pupils in a class (Mean= 21.50, SD =4.74). The literature shows that with smaller classes, teachers are able to individualize their instruction more and employ a ra nge of instructional practices (Molnar et al, 1999) that may be more developmentally appropriate. More reading time given by parents, on average, had a large positive effect on school mathematics achievement means, controlling for child renÂ’s background characteristics. Although the study did not establish causal links, the association of mathematics achievement with increased reading activities at home is encouraging. In early years cognitive development in children may not be subjectspecific, hence increased at-home reading activities could potentially result in gains in both mathematics and reading. A comment on some of the counter-intuitive findin gs. The positive rather than negative effects of larger school size on early mathematics achiev ement may have to do with larger schools having more resources. Another explanation may be the la ck of representativeness of the sample on this variable. The breakdown on school size in the init ially screened 12,710 ECLS cases had shown that 31% of the children were in schools with 300Â–499 st udents, with 19% in even smaller schools. This indicated that the remaining 50% belonged in schools that exceeded 500 studentsÂ—a less than desirable school size for elementary schools. A 500 cu t was thus used to define the school size variable. However, Table 1 shows that following selection of the study sample, just 15% of the 182 schools were Â“largeÂ” (with more than 500 students), wh ile the remaining 85% were Â“smallÂ” schools. This distribution might have tilted results. On the success orientation variable, schools dea ling with challenging student populations often tend to have organizational cultures with high acad emic press values. Yet, they may show relatively poor academic outcomes because they serve struggling students. That administrator reports of school success orientation had a negative bu t statistically significant influe nce on mathematics achievement in schools might have resulted from this last reality. The low correlations in Table B2 (Appendix B) suggest that results were likely not affected by suppressor effects of other school-level variables in the models. Cross-level Interaction Effects The more interesting results of this study were on moderators of Grade 1 mathematics achievement gaps. Several cross-level interaction re sults were in expected directions and may point to some school policy/practice actions, keeping in mind that the evidence is correlational. With the achievement variability that was found in African Am erican children by school, increased class and school sizes affected mathematics achievement nega tively, and at-home reading time was positively associated with mathematics achievement. For poor versus well-to-do children, more instructional time per day to mathematics and reading had a posit ive and significant effect on school achievement meansÂ—another affirming finding. Further, higher tea cher certification rates in elementary education affected boysÂ’ mathematics achievement in a clearly positive direction. As much as a 2-point gain in achievement was estimated to re sult for boys in schools with unit increases in schoolsÂ’ teacher certification rates. This last finding, although show ing differential effects by gender on mathematics achievement, is consistent with the results of the companion ECLS study by this author, where the school certification rate showed a large positive main effect on first grade reading achievement. The need for improving qualified teacher recruitment and retention is currently emphasized in the NCLB legislation; however, policy implementation varies in some regions when certified teachers are in short supply.
Achievement Gaps and Correlates of Early Mathematics Achievement 25 Finally, increased IEPs in schools had a signif icant but small negative effect on mathematics school means of high poverty students. NCLB re quires that schools set common benchmarks for achievement of special versus nor mally-achieving student populations; this may not be a reasonable expectation. It should be noted again in evaluating the implications of these findings that the IEP indicator in this study did not correspond to the percent of students with disabilities in the ECLS sample, and does not reflect national estimates of special education enrollments (Table B3). Limitations and Future Research Because secondary analysis of large scale national surveys does not permit causal interpretations of various school effectsÂ—the eviden ce presented here is correlational. The controls instituted were statistical rather than experime ntal. The composition of the data set, variable definitions, missing data on surveys, and particular va riables selected for modeling, all affected findings obtained. Some of the significant negative coefficien ts obtained with parent, administrator, and teacher survey indices need further confirmation. Some eth nic group interactions could not be examined due to reduction in available cases. Future studies shou ld thus attempt to further validate and replicate the findings reported here with multiple ECLS data sets Direct and moderating effects of other variables, such as methods of mathematics instruction and childrenÂ’s cognitive development on mathematics achievement, should also be examined to bett er inform future school policies and practices. Despite the limitations noted, several results were replicated across analytic models tested. Monitoring of achievement gaps and a search for si gnificant correlates and moderators of achievement should thus continue in other similar populations, and especially with longitudinal data from the ECLS sample in Grades 3 and 5.
Education Policy Analysis Archives Vol. 13 No. 45 26 References Ayers, W., Bracey, G., & Smith, G. (2000). The ultimate educational reform? Make schools smaller. Tempe, AZ: EPSL, Educational Po licy Research Unit. CERAIÂ–00Â–35. Blatchford, P., Moriarty, V., Edmonds, S., & Mart in, C.(2002). Relationsh ips between class size and teaching: A multimethod analysis of English infant schools. American Educational Research Journal, 39 (1), 101Â–132. Boethel, M. (2004). Readiness: School, family & community (Annual Synthesi s 2004). Austin, TX: National Center for Fa mily and Community Connectio ns with School, Southwest Educational Development Lab. Brookover, W, Beady, C., Flood, P., Sc hweitzer, J, & Wisenbaker, J. (1979). School social systems and student achievement: Schools can make a difference New York: Prae ger Publishers. Chatterji, M. (2002). Models and methods for examining standard s-based reforms and accountability initiatives: Have the tools of inquiry answered pressing questions on improving schools. Review of Educat ional Research, 72 (3), 345Â–386. Clark, D.L., Lotto, L.S., & McCa rthy, M.M. (1980). Factors associated with success in urban elementary schools. Phi Delta Kappan, 61 467Â–470. Collins, W. A., Macoby, E., Steinberg, L., He therington, E.M., & Bo rnstein, M. (2000). Contemporary research on pa renting: The case of nature versus nurture. American Psychologist, 55 218Â–232. Darling-Hammond, L. (1998). Teac hers and teaching: Testing polic y hypotheses from a national commission report. Educational Researcher, 27 (1), 5Â–15. Ferguson, R. F. (1991). Paying for public education: New evidence on how and why money matters. Harvard Journal on Legislation, 28 (465), 465Â–498. Ferguson, R. F., & Ladd, H. F. (1996). How an d why money matters: An analysis of Alabama schools. In H. Ladd (Ed.), Holding Schools Accountable (pp. 265Â–298 ). Washington, D.C.: Brookings Institution. Finn, J. D., & Achilles, C. M. (1999). TennesseeÂ’s class size study: Findings, implications, misconceptions. Educational Evaluation and Policy Analysis, 21 (2), 97Â–109. Friedman, L. (1995). The space factor in mathematics: Ge nder differences. Review of Educational Research, 65 (1), 22Â–50. Friedman, L. (1989). Mathematics and the gender gap: A meta-analysis of recent studies on sex differences in quantitative tasks. Review of Educat ional Research, 59 185Â–213.
Achievement Gaps and Correlates of Early Mathematics Achievement 27 Garmezy, N. (1993). Children in po verty: Resilience despite risk. Psychiatry, 56 127Â–136. Gramezy, N. & Masten, A.S. (1 986). Stress, competence, and re silience: Common frontiers for therapists and ps ychopathologists. Behavior Therapy, 17 500Â–521. Grissmer, D., W, Kirby, S. N., Berends, M., & Williamson, S. (1994). Student achievement and the changing American family. Santa Monica, CA: RAND. Hannaway, J., & Talbert, J.E. (1993). Bringing context into effective schools research: Urbansuburban differences. Education Administration Quarterly, 29 (2), 164Â–186. Hannaway, J., & Kimball, K. (1998). Big isnÂ’t always bad: School district size, poverty, and standards-based reform Washington, D.C.: Urban Institute. Hubbs-Tait, L., Culp, A.M., Erron, H., Culp, R. et al ( 2002). Relation of He ad Start attendance to childrenÂ’s cognitive and social outcomes: Moderation by family risk. Early Childhood Research Quarterly, 17 539Â–558. Hyde, J.S., Fennema, E., & Lamon, S.J. (1990). Gender differences in ma thematics performance: A meta-analysis. Psychological Bulletin, 106 139Â–155. Knapp, Michael, S. (1997). Between systemic reforms and the mathematics and science classroom: The dynamics of innovation, im plementation, and professional learning. Review of Educational Research, 67 (2), 227Â–266. Laparo, K. & Pianta, R.C. (2000) Predicting childrenÂ’s competen ce in early sc hool years: A meta-analytic review. Review of Educational Research, 70 (4), 443Â–484. Lee, J. (2002). Racial and ethnic achievement ga p trends: Reversing the progress towards equity. Educational Researcher, 31 (1), 3Â–12. Lee, V.E., Brooks-Gunn, J., & Schnur, E. (1988). Does Head Start Work? A 1-year follow-up of comparison of disadvantaged children atte nding Head Start, no pre-school and other pre-school programs. Developmental Psychology, 24 (2), 210Â–222. Linn, Robert L. (2003). Ac countability: Respons ibility and Reasonab le Expectations. Educational Researcher, 32 (7), 3Â–13. Linn, R. L., Baker, E.L., & Betebenner, D.W. (2 002). Accountability Sys tems: Implications of requirements of the No Child Left Behind Act of 2001. Educational Researcher, 31 (6), 3Â–16. Molnar, A., Smith, P. Zahorik, J., Palmer, A., Ha lbach, A., & Erle, K. (1999). Evaluating the SAGE program: A pilot program in target ed teacher-pupil redu ction inWisconsin. Educational Evaluation and Policy Analysis, 21 (2), 165Â–177. National Center for Educationa l Statistics (N CES) (1995a). Findings from the condition of education 1995: The educ ational progress of Af rican American students Washington, D.C.: U.S. Dept. of Education/OERI.
Education Policy Analysis Archives Vol. 13 No. 45 28 National Center for Educationa l Statistics (N CES) (1995b). Findings from the condition of education 1995: The ed ucational progress of Hispanic students Washington, D.C.: U.S. Dept. of Education/OERI. National Center for Educational Statistics (NCES) et al. (1999). ECLS-K base year public use data files and electronic code book .Washington, D.C.: U.S. Dept. of Education/OERI. National Governors Association (2005). Closing the achievement gap Washington, D.C.: NGA Center for best Practices. No Child Left Behind Ac t of 2001, P.L. No. 10 7Â–110, 115 Stat. 1425 (2002). NICHD Early Child Care Research Network (2002). Early child care and childrenÂ’s development prior to school entry: Results from the NI CHD study of early child care. American Educational Research Journal, 39 (1), 133Â–164. Purkey, S.C. & Smith, M.S. (1983). Effective schools: A review. Elementary School Journal, 84 (4), 427Â–452. Raudenbush, S., Bryk, A. Cheong Y.F., & Congdon, R. (2001). HLM 5: Hierchical linear and nonlinear modeling. Lincolnwood, IL: SSI. Rosenholtz, S. (1985). Effective schools: Interpreting the evidence. American Journal of Education, 93, 352Â–388. Rowan, B., Bossert, S.T. & Dwyer, D.C. (1983). Research on effective schools: A cautionary note. Educational Researcher, 12 (4), 24Â–31. Sirotnik, K. A., & Kimball, K. (1999). Standards fo r standards-based acco untability systems. Phi Delta Kappan, 81 (3), 209Â–214. Walston, J. & West, J. (2004). Full-day and half-day kindergarten in the United States: Findings from the ECLS, Kindergart en class of 1988Â–1989 Washington, D.C.: U.S. Dept. of Education. Werner, E.E. (1993). Risk, resilie nce, and recovery: Perspectives from the Kauai Longitudinal Study. Developmental and Psychopathology, 5 503Â–515. Williams, D.T. (1990). The dimensions of educ ation: Recent resear ch on school size. Working Paper Series. Clemson, SC: Clemson Univ ersity, Strom Thurmond Institute of Government and Publ ic Affairs (ERIC Re production Document No. ED 347 006). Zill, N. & West, J. (2001). Entering kindergarten: Findings from the condition of education 2000 Washington, D.C.: U.S. Dept. of Education/OERI.
Achievement Gaps and Correlates of Early Mathematics Achievement 29 About the Author Madhabi Chatterji Teachers College, Columbia University Email: firstname.lastname@example.org Madhabi Chatterji Associate Professor of Measuremen t, Evaluation, and Education at Teachers College, Columbia University, has research interests in designing classroomand school-based assessment system s; development and validation of construct measures with classical and Rasch measurement methods; and in evaluating standa rds-based educational reforms and smalland large-scale interventions with systemic models. Her most recentlyinitiated research, supported by the National Science Foundation in 2004, de als with addressing K-12 achievement gaps in mathematics using a teacher-mediated model of "proximal" assessment and data use. Another line of inquiry deals with gathering research-based evidence on field interventions using mixe d-method research designs.
Education Policy Analysis Archives Vol. 13 No. 45 30 Appendix A Description and Psychometric Pr operties of ECLS/Survey Indices Child Variables ChildÂ’s socioeconomic status-poverty : This index was based on a categorical measure of socio-economic status (W1SESQ5) provided by NCES that broke down a continuous measure of socio-economic status built on parent education, o ccupation, and income, into five categories by quintile. The first and second quintile categories we re coded as 1 (low SES, High Poverty), and quintiles 3-5 were coded as 0 (high SES, Low Poverty). This measure yielded much the same results as the continuous SES composite and was used because of its interpretability. In different models, this measure was used as a Level 1 (based on child quintile category) as well as a Level 2 predictor (based on school means). Teacher Variables Teacher certification variables -ECERT, ECCERT (Source: Teacher Questionnaire BB4ELEMCT, B4ERLYCT). These variables were dummy coded, with a 1 to indicate if the teacher had elementary certification (B4ELEMCT) ve rsus not, 0; or early childhood certification (B4ERLYCT) versus not. Only the former was used in the final models based on preliminary findings. School aggregates on certifi cation rates were entered in HLMs. Teacher support composite -TSUP_1 (Source: Administrator Questionnaire). This index was composed of four self-report items giving admini strator reports on whether the school had an active professional development program and gave teacher s planning time, time for professional growth and incentives for improvement (S2PRODEV, S2ACTSTF, S2ADEQTE, S2INCENT). The index was supported by results of a principal components analysis with a pre-rotation eigenvalue of 2.01, with component loadings following varimax rotation of .63 to .83. The CronbachÂ’s alpha reliability estimate of composite scores was .71. This index is interpreted as the extent to which a child was exposed to a school with high levels of teacher supports. A school-level aggregate of TSUP was entered into HLMs at Level 2. School/Organizational Indices School success orientation S_SUC1 (Source: School Administrator Questionnaire). The S_SUC1 composite was based on five self-report i tems indicating leader reports of the degree of success schools have in emphasizing childrensÂ’ acad emic learning, namely, raising test performance, providing challenges to high achievers, added he lp for low achievers, and being open to new ideas (S2SUCC6-7, 10-11). The factor yielded a pre-ro tation eigenvalue of 1.19 using principal components analysis, and factor loadings of .53 to .77 following varimax rotation. The alpha reliability estimate of the composite was .73. Mean s at the school level were utilized for HLMs. Class size CSIZE: (Source: Teacher Questionnaire, Part A). This was a numeric index computed as follows with responses to two items on the teacher questionnaire asking separately for the # of boys versus # of girls in their cla ssroom. CSIZE=A1BOYS+A1GIRLS. Again, means at the school level were used for HLMs.
Achievement Gaps and Correlates of Early Mathematics Achievement 31 Class time to reading and math TIM_1: (Source: Teacher Questionnaire Part A). This was a numeric index computed as follows with respons es to two items on the teacher questionnaire, dealing with how much time in minutes, teachers de dicated to reading and math activities in their classrooms per day: TIM=A2MINRD +A2MINMTH. These two items loaded on a principal component with an eigenvalue of 1.57 before ro tation, and had varimax rotated factor loadings of .88 and .83. The alpha reliability was .87. School-lev el means on the composite were used in HLMs. Parent Involvement Indices Educational support -EDSUP (Source: Parent Questionnaire). This index was based on parent/guardian reports of whether or not they did math, writing, and mathematics with their child at home. This 3-item set had a pre-rotation eigenvalue of 2.5, and factors loadings from .31 to.78 following varimax rotation. The Cr onbachÂ’s alpha estimate was .65. Parent satisfaction with school activities PAR_S (Source: Parent Questionnaire). This 4item index was dealt with whether the school pr ovides opportunities for parent and community involvement, and for tracking how their child is doing. PARSCHL=HOWCHD+P2CHILDR+P2CHANCV+P2COMMUN. PAR_S was the school mean. Following satisfactory factor extraction, the obtai ned alpha reliability of the composite was .75. Parent involvement PARINV (Source: Parent Questionnaire). COMPUTE PARINV=P2ATTENB+ P2PARGRP +P2ATTENS +P2VOLUNT+ P2FUNDRS. This index showed the degree to which parents reported invo lvement in school based on attending events and functions, volunteering, fundraising etc. As in th e others, principal components analysis helped identify this subset of items; however, it yiel ded a lower alpha reliability of .56. P_INV was the school mean. Other Child-level Demographic/Background Variables At the child level, several background variab les were dummy-coded for analysis. These were gender (Male=1, Female=0), ethnicity in grou ps of interest (Asian=1, Others=0; African American=1, Others=0; Hispanic (race specified)=1, Others=0; and Hispanic (race unspecified)=1, Others=0). In addition, age of the child in months in grade 1, a continuous measure, was included in the preliminary analysis. Other School-Level Variables At the school level, urbanicity (urban=1, othe r (rural/ suburban)=0), school size(greater than or equal to 500=1, less than 500=0), and school sector (public=1, other=0) were dummy coded. Each schoolÂ’s total IEPs, an index of excep tional education services (SPCED) provided by the school, was computed. Likewise, student attenda nce was computed as a percent of total days attended in the school year. This last index was ba sed on the number of absences reported for each child in the ECLS database. All school level indi ces were aggregated across children within each school.
Education Policy Analysis Archives Vol. 13 No. 45 32 Imputed Values: The number of values imputed with the mean substitution procedure of SPSS was as follows: EDSUP_1, 1 case; TSUP_1, 23 cases; S_ SUCC, 33 cases; S_ATT, 38 cases; CSIZ_1, 15 cases; TIM_1, 55 cases.
Achievement Gaps and Correlates of Early Mathematics Achievement 33 Appendix B Supplementary Tables Table B1 Representiveness of Fi nal Data Set as compar ed with ECLS Sample Variable ECLS-K Sample Initially Screened Study Sample (weighted) Final Data Set Number of cases 17212 14742 2300 Poverty Status (Low SES,) 6958 (40.4%) 5885 (39.9%) 667(29%) Gender (Males) 7429 (50 .4%) 8867 (49%) 1127 (51.6%) Ethnicity-Black 2441 (14.2%) 2374 (16%) 276 (12%) Ethnicity-Hispanic 1 1388 (8.1%) 1346 (9%) 138 (6%) Ethnicity-Hispanic 2 1560 (6.4%) 1430 (9.7%) 115 (5%) Ethnicity-Asian 1093 (6.4%) 448 (3%) 69 (3%)
Education Policy Analysis Archives Vol. 13 No. 45 34 Table B2 Correlations of School-level Variables Variable Teacher Support School Succes School-Attend. Educ. Suppt. Parent Satis. Class size Class time to reading/math Poverty School size IEPs At home reading time Teacher Support Â— School Success .165** Â— School Attendance -.042 -.057 Â— Educational support -.010 .028 .002 Â— Parent Satisfaction .001 -.125* -.064 -.083 Â— Class size -.004 -.041 .184** -.009 -.012 Â— Class time to reading/math .132* -.144* -.152* -.028 -.112 .128* Â— Poverty .097* -.200** .035 -.033 -.013 .117* .278** Â— School size .126** .031 .061 -.023 -.095*.150** -.073 .078 Â— IEPs .097* -.002 .003 -.006 .025 -.068 .036 .083 .025 Â— At home reading time .121* .008 -.037 .180**.014 -.114* .117 -.087 .012 .067Â— Parent Involvement .114* -.169** .022 -.003 .206**.103* .186** .484** .228**.071.004 *p < .05; **p < .01
Achievement Gaps and Correlates of Early Mathematics Achievement 35 Table B3 Frequency distributions on Selected Variables in Initially Screened Sample Category ECLS sample within category Total school enrollment (used in study) 0-149 445 150-299 1915 300-499 3996 500-749 3620 800+ 2734 Individualized Education Plans (Variable: U4IEP, used in study) Does not have IEP 9012 Has IEP 775 Diagnosed Learning Program (Variable: P3 HEQ0 20; not used in study due to missing data) No 213 Yes 270 Missing 12436
Education Policy Analysis Archives Vol. 13 No. 45EDUCATION POLICY ANALYSIS ARCHIVES http://epaa.asu.edu Editor: Sherman Dorn, University of South Florida Production Assistant: Chris Murre ll, Arizona State University General questions about ap propriateness of topics or particular articles may be addressed to the Editor, Sherman Dorn, email@example.com. Editorial Board Michael W. Apple University of Wisconsin David C. Berliner Arizona State University Greg Camilli Rutgers University Casey Cobb University of Connecticut Linda Darling-Hammond Stanford University Mark E. Fetler California Commission on Teacher Credentialing Gustavo E. Fischman Arizona State Univeristy Richard Garlikov Birmingham, Alabama Gene V Glass Arizona State University Thomas F. Green Syracuse University Aimee Howley Ohio University Craig B. Howley Appalachia Educational Laboratory William Hunter University of Ontario Institute of Technology Patricia Fey Jarvis Seattle, Washington Daniel Kalls Ume University Benjamin Levin University of Manitoba Thomas Mauhs-Pugh Green Mountain College Les McLean University of Toronto Heinrich Mintrop University of California, Berkeley Michele Moses Arizona State University Anthony G. Rud Jr. Purdue University Michael Scriven Western Michigan University Terrence G. Wiley Arizona State University John Willinsky University of British Columbia
Achievement Gaps and Correlates of Early Mathematics Achievement 37 EDUCATION POLICY ANALYSIS ARCHIVES English-language Graduate -Student Editorial Board Noga Admon New York University Jessica Allen University of Colorado Cheryl Aman University of British Columbia Anne Black University of Connecticut Marisa Cannata Michigan State University Chad d'Entremont Teachers College Columbia University Carol Da Silva Harvard University Tara Donahue Michigan State University Camille Farrington University of Illinois Chicago Chris Frey Indiana University Amy Garrett Dikkers University of Minnesota Misty Ginicola Yale University Jake Gross Indiana University Hee Kyung Hong Loyola University Chicago Jennifer Lloyd University of British Columbia Heather Lord Yale University Shereeza Mohammed Florida Atlantic University Ben Superfine University of Michigan John Weathers University of Pennsylvania Kyo Yamashiro University of California Los Angeles
Education Policy Analysis Archives Vol. 13 No. 45 Archivos Analticos de Polticas Educativas Associate Editors Gustavo E. Fischman & Pablo Gentili Arizona State University & Universidade do Estado do Rio de Janeiro Founding Associate Editor for Spanish Language (1998Â—2003) Roberto Rodrguez Gmez Editorial Board Hugo Aboites Universidad Autnoma Metropolitana-Xochimilco Adrin Acosta Universidad de Guadalajara Mxico Claudio Almonacid Avila Universidad Metropolitana de Ciencias de la Educacin, Chile Dalila Andrade de Oliveira Universidade Federal de Minas Gerais, Belo Horizonte, Brasil Alejandra Birgin Ministerio de Educacin, Argentina Teresa Bracho Centro de Investigacin y Docencia Econmica-CIDE Alejandro Canales Universidad Nacional Autnoma de Mxico Ursula Casanova Arizona State University, Tempe, Arizona Sigfredo Chiroque Instituto de Pedagoga Popular, Per Erwin Epstein Loyola University, Chicago, Illinois Mariano Fernndez Enguita Universidad de Salamanca. Espaa Gaudncio Frigotto Universidade Estadual do Rio de Janeiro, Brasil Rollin Kent Universidad Autnoma de Puebla. Puebla, Mxico Walter Kohan Universidade Estadual do Rio de Janeiro, Brasil Roberto Leher Universidade Estadual do Rio de Janeiro, Brasil Daniel C. Levy University at Albany, SUNY, Albany, New York Nilma Limo Gomes Universidade Federal de Minas Gerais, Belo Horizonte Pia Lindquist Wong California State University, Sacramento, California Mara Loreto Egaa Programa Interdisciplinario de Investigacin en Educacin Mariano Narodowski Universidad To rcuato Di Tella, Argentina Iolanda de Oliveira Universidade Federal Fluminense, Brasil Grover Pango Foro Latinoamericano de Polticas Educativas, Per Vanilda Paiva Universidade Estadual Do Rio De Janeiro, Brasil Miguel Pereira Catedratico Un iversidad de Granada, Espaa Angel Ignacio Prez Gmez Universidad de Mlaga Mnica Pini Universidad Nacional de San Martin, Argentina Romualdo Portella do Oliveira Universidade de So Paulo Diana Rhoten Social Science Research Council, New York, New York Jos Gimeno Sacristn Universidad de Valencia, Espaa Daniel Schugurensky Ontario Institute for Studies in Education, Canada Susan Street Centro de Investigaciones y Estudios Superiores en Antropologia Social Occidente, Guadalajara, Mxico Nelly P. Stromquist University of Southern California, Los Angeles, California Daniel Suarez Laboratorio de Politicas Publicas-Universidad de Buenos Aires, Argentina Antonio Teodoro Universidade Lusfona Lisboa, Carlos A. Torres UCLA Jurjo Torres Santom Universidad de la Corua, Espaa