Educational policy analysis archives

Educational policy analysis archives

Material Information

Educational policy analysis archives
Arizona State University
University of South Florida
Place of Publication:
Tempe, Ariz
Tampa, Fla
Arizona State University
University of South Florida.
Publication Date:


Subjects / Keywords:
Education -- Research -- Periodicals ( lcsh )
non-fiction ( marcgt )
serial ( sobekcm )

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
E11-00353 ( USFLDC DOI )
e11.353 ( USFLDC Handle )

Postcard Information



This item has the following downloads:

Full Text
xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam a22 u 4500
controlfield tag 008 c20049999azu 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E11-00353
0 245
Educational policy analysis archives.
n Vol. 12, no. 4 (January 28, 2004).
Tempe, Ariz. :
b Arizona State University ;
Tampa, Fla. :
University of South Florida.
c January 28, 2004
Group and interaction effects with no child left behind : gender and reading in a poor, Appalachian district / Robert Bickel [and] A. Stan Maynard.
x Research
v Periodicals.
2 710
Arizona State University.
University of South Florida.
1 773
t Education Policy Analysis Archives (EPAA)
4 856

xml version 1.0 encoding UTF-8 standalone no
mods:mods xmlns:mods http:www.loc.govmodsv3 xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govmodsv3mods-3-1.xsd
mods:relatedItem type host
mods:identifier issn 1068-2341mods:part
mods:detail volume mods:number 12issue 4series Year mods:caption 20042004Month January1Day 2828mods:originInfo mods:dateIssued iso8601 2004-01-28


1 of 22 A peer-reviewed scholarly journal Editor: Gene V Glass College of Education Arizona State University Copyright is retained by the first or sole author, who grants right of first publication to the EDUCATION POLICY ANALYSIS ARCHIVES EPAA is a project of the Education Policy Studies Laboratory. Articles appearing in EPAA are abstracted in the Current Index to Journals in Education by the ERIC Clearinghouse on Assessment and Evaluation and are permanently archived in Resources in Education Volume 12 Number 4January 28, 2004ISSN 1068-2341Group and Interaction Effects with “No Child Left B ehind”: Gender and Reading in a Poor, Appalachian District Robert Bickel Marshall University A. Stan Maynard Marshall UniversityCitation: Bickel, R., Maynard, A.S., (2004, January 28). Group and interaction effects with “No Child Left Behind”: Gender and reading in a poor, A ppalachian district, Education Policy Analysis Archives, 12 (4). Retrieved [Date] from /v12n4/.Critics of “No Child Left Behind” judge that it ove rsimplifies the influence of social context and the place of social ly ascribed traits, such as social class, race, and gender, in determin ing achievement. We hold that this is especially likely to be true with regard to gender-related group effects and gender-i mplicated interaction effects. We make our concerns concrete in a multilevel, repeated measures analysis of reading a chievement in a poor, rural school district located in the southe rn coalfields of Appalachian West Virginia. Our results suggest that as the percentage of students who are male increases, scho ol mean scores in reading achievement decline for three rea sons: individual males do less well than females; the gre ater the percentage of males, the lower the scores for all s tudents; added to that, the greater the percentage of males, the l ower the scores


2 of 22 for males specifically. Given the accountability me asures and sanctions proposed by “No Child Left Behind,” havin g a large percentage of males in a school could be disastrous We conclude that gender effects in reading achievement are complex, easily overlooked, and have no obvious remedy. As s uch, they lend credence to the view that “No Child Left Behin d” oversimplifies the social context of schooling and underestimates the importance of social ascription. “No Child Left Behind” is the first re-authorizatio n of the Elementary and Secondary Education Act since 1994 (U.S. Department of Education, 2002a). An oft-noted consequence of the revised version of the Act is expansion of the role of the federal government in public education (Seldon, 2001; Rebora, 2002). The controversial nature of the Act is refle cted in the Bush Administration’s counter assertion that “No Child L eft Behind” actually increases flexibility and control at the local level. In this view, what some take to be expansion of federal authority is better construed as redefinition (U.S. Department of Education, 2002b). The primary purpose of the redefined federal role, as explained by the current Secretary of Education, is to employ federal educat ion funds to close the achievement gap between disadvantaged and minority students and their peers, raising all students to a proficient level ( U.S. Department of Education, 2003). Broadly, this is to be accomplished through more rigorous accountability measures, through enabling students to transfer fro m schools that do not meet prescribed performance levels, and by upgrading req uired qualifications for teachers and paraprofessional aides (White House, 2 003). Persistent failure to move students toward acceptab le performance levels forces schools to invoke a variety of costly correc tives. These include providing vouchers to facilitate transfer from poorly perform ing schools to public alternatives, complemented with supplemental servic es, including private tutoring (White House, 2001).Expectations“No Child Left Behind,” is premised on the assumpti on that effective schools need not be constrained by contextual factors or by students’ socially ascribed characteristics. The rationale for this rejection o f conventional educational wisdom is often couched in terms of expectations: raise expectations for less-advantaged and minority students, and they wil l rise to the occasion (White House, 2001; U.S. Department of Education, 2003). O therwise, students become victims of what the Secretary has termed “th e soft bigotry of low expectations” (quoted in Huston, 2003).School Context and Socially Ascribed Traits“No Child Left Behind,” thus, constitutes an emphat ic dismissal of the inevitable intrusiveness of the social context of schooling. Much the same is true of students’ socially ascribed traits. If context and social ascription interfere with


3 of 22 student achievement, it is because schools are dysf unctional. Otherwise, these extraneous intrusions would be deflected by proper procedures, best practices, and effective school organization (cf. Bush, 2002).Consistent with this view, the Act mandates that pe rformance measures be disaggregated, reporting separately scores for spec ified categories of students. Categories include economic disadvantage, ethnicity gender, English language proficiency, and disability. This permits group com parisons to determine if the achievement gap between members of less-advantaged and socially devalued groups and other students is being closed (White Ho use, 2002). Whatever its merits, “No Child Left Behind” seems d isarmingly straightforward and modern. Scientifically validated methods of acc omplishing education, coupled with high expectations for all, enables eac h student to shake off the constraints of class, race, gender, and other non-m eritocratic factors. Deficiencies in curriculum, organization, or person nel that interfere with this process can and must be remedied.To many professional educators, however, “No Child Left Behind” represents a dangerous oversimplification of the social circumst ances of education (Coles, 2001; Bianchini, 2002; Denlinger, 2002; Huston, 200 3; Bailey, 2003; Hardy, 2003). In this view, the effects of class, race, ge nder, and context cannot be explained and remedied with the ease the Act implie s.Group Effects and Interaction EffectsComplicating matters further, group effects and int eraction effects are not reducible to readily identifiable individual charac teristics or easy-to-see organizational factors (Aiken & West, 1991; Kreft & DeLeeuw, 1998; Raudenbush and Bryk, 2002). In the absence of welldeveloped theory, such effects are difficult to anticipate and often go un detected (Velicer, 1972; Baron & Kennedy, 1986; Jaccard, Turrisi, and Wan, 1990; I versen, 1991; Snijders and Bosker, 1999). Nevertheless, group effects and inte raction effects which bear on determining measured school performance are ubiq uitous and consequential (Heck and Thomas, 2000).For example, neighborhood effects at the group leve l suggest that students, in the aggregate, can imbue an entire school with a sh ared ethos which they jointly import from their out-of-school context (Vartanian and Gleason, 1999; Solon, Page, and Duncan, 2000; Bickel, Smith, and Eagle, 2 002: Bickel and Howley, 2003). Depending on neighborhood quality, and net the influence of social class, neighborhood effects may enhance or diminish achievement. Intervening in neighborhoods, however, is beyond th e scope of research-based practices and procedures, and raised expectations. As a result, the consequences of such powerful group effects are ign ored by “No Child Left Behind.”As another example, the frequently reported finding that, with class size held constant, the negative association between poverty and achievement is exacerbated as schools get larger represents an int eraction effect which has no known remedy, other than to make schools smaller. A s such, there is no good


4 of 22 reason to believe that the reforms proposed by “No Child Left Behind” will diminish its pernicious consequences (Bickel and Ho wley, 2000; Bickel, Howley, Glascock, and Williams, 2001).Research ObjectivesIn the following we use a small data set collected from all eight elementary schools in an impoverished, rural county in the coa lfields of southern West Virginia. Our objective is to focus on one socially ascribed trait, gender, and to assess the plausibility of claims that such extrane ous characteristics need not interfere with educational attainment. We do this b y examining the group effects of gender and gender-implicated interaction effects in a multilevel, repeated measures analysis.If we find gender-based group effects or gender-imp licated interaction effects which have no available remedy, we will tentatively conclude that “No Child Left Behind” is premised on an unduly simplified view of the social circumstances of education. As a result, efforts to accomplish schoo l reform through focusing on characteristics of individual students and readily manipulable organizational factors will yield, at best, limited success, becau se group effects and interaction effects will still be at work.Why Gender?Economic disadvantage and minority group status are more conspicuous in discussions of “No Child Left Behind” than gender c ategory. Nevertheless, “No Child Left Behind” highlights gender by explicitly permitting use of federal funds for single-sex schools, something that administrato rs and policy makers had assumed to be inconsistent with Title IX of the Edu cation Amendments of 1972 (Otterbourg, 2001). Proponents of “No Child Left Be hind” cite funding for single-sex schools as one means of providing greate r flexibility and control at the state and local levels (White House, 2002).More to the point, while the effects on achievement of economic disadvantage and devalued minority group status are consistent a nd well known, gender effects are much more difficult to predict and expl ain (see, for example, Cloer and Dalton, 2001; Lynch, 2002; Phillips, Norris, Os mond, and Maynard, 2002). Sometimes they occur, and sometimes they do not. Th e same uncertainty applies to their direction, to the advantage of mal es or females (see, for example, High, 1996). Gender effects seem, therefor e, less likely to be detected, especially if they take the form of group effects or interaction effects. Lack of sensitivity to the importance of group-leve l effects of gender and gender-implicated interaction effects may lead us t o misunderstand the real complexity of the social organization of school ach ievement. The consequences of gender effects for schools faced with the accoun tability demands and sanctions promulgated by “No Child Left Behind” may be disguised and damaging.The County


5 of 22 The poor, rural county which was the source of our data is located in southern West Virginia, bordering on eastern Kentucky. Its population, 26,253, has declined by 24.3 percent since 1980 (U.S. Census Bu reau, 2001). The county is 87.7 percent rural, in a state that is 63.9 perc ent rural; the same figure for the entire U.S. is 24.8 percent. The median family inco me is $21,347, well below the state median of $29,696 and little more than ha lf the national median of $41,994. Of families with children, 21.4 percent ha d incomes below the federal poverty level in a state where 17.9 percent of all families with children were below that income level; the same figure for the en tire U.S. is 12.4 percent. Among elementary school students in the county, 74. 9 percent are eligible for free/reduced cost lunch (U.S. Census Bureau, 2001).Data“No Child Left Behind” gives priority to literacy, reflecting the educational priorities of President Bush (International Reading Association, 2003). It posits the existence of research-based, scientifically val idated practices and procedures to promote reading achievement, and prov ides competitive Reading First grants to assist states in implementing readi ng improvement programs for children in the early elementary grades.Given the conspicuous role of reading in “No Child Left Behind,” it is useful that our multilevel repeated measures analyses are based on successive administrations of the widely used Woodcock-Johnson 22 letter-word identification test and Woodcock-Johnson 23 passage comprehension test as standardized measures of reading achievement (Woodc ock and Johnson, 1990). All variables are described in Table 1, and descriptive statistics by gender are reported in Table 2.Data were originally collected for use in a local, unpublished evaluation of a program designed to provide training for parents an d other volunteers to tutor low-achieving students in the lower elementary grad es in this poor, rural, Appalachian county. Tutors were paired with student s identified by teachers as in danger of being retained because of reading defi ciencies. One hundred-five students from the county’s eight e lementary schools were referred and tutored. Achievement tests were admini stered to forty-four first grade students and sixty-one second grade students at the beginning and end of the 1996-97 school year. The number of test take rs was constant from the first test administration to the second. TABLE 1 VARIABLES W-J 22Woodcock-Johnson 22: Letter-Word Identificati on Reading Achievement Test; Internal ConsistencyReliability = .92. W-J 23 Woodcock-Johnson 23: Passage ComprehensionReading Achievement Test; Internal Consistency


6 of 22 Reliability = .90. TIME1Test Administered Twice: Beginning of Grade 1 or 2 and End of Grade 1 or 2 Level 1, Within Subjects; Coded 0 and 1. GENDER2Gender Level 2, Between Subjects; Coded 1 (M ale) or 0 (Female). GENDER3Gender (Aggregated) Level 3, Between Schools GRADE2First or Second Grade, Level 2, Between Subje cts. AGE2Age in Years Level 2, Between Subjects.AGE3Age in Years (Aggregated) Level 3, Between Scho ols. SCHLSIZE3Total School Enrollment Level 3, Between S chools. CLASSIZE3Mean Class Size Level 3, Between Schools.LUNCH3Percent Eligible for Free/Reduced Cost Lunch, Between Schools. SPAN3Grade-Span Configuration, Between Schools. TABLE 2 DESCRIPTIVE STATISTICS: MALES MeansStandard DeviationsMinimumMaximumW-J 2221.766.158.0035.00W-J 238.794.980.0019.00TIME10.500.501.001.00GENDER21.000.001.00GENDER30.650.170.360.90GRADE21.60 0.491.002.00AGE27.480.816.179.00AGE37.520.257.107.98SCHLSIZE3296.8784.09151.00381.00CLASSIZE321.541.9218.9024.50LUNCH374.9510.9855.0095.00SPAN35.190.865.009.00 N = 63 DESCRIPTIVE STATISTICS: FEMALES


7 of 22 MeansStandard DeviationsMinimumMaximumW-J 2222.455.1314.0035.00W-J 239.564.170.0017.00TIME10.500.501.001.00GENDER20. N = 42Data AnalysisOur analysis was done with SPSS 11.0 Mixed Models, using variables measured at three levels: within subjects for repea ted measures, between subjects, and between schools (SPSS, 2001). The eig ht schools in which the one hundred five respondents were located ranged in size from one hundred fifty-one to three hundred eighty-one students. The number of test-takers per school varied from twelve to thirty-eight. This rep resents approximately twenty percent of the students in first and second grades in each school for 1996-97. In addition to representing eight schools, the stud ents in our secondary analysis were distributed among an undocumented number of cl assrooms. Since students were not identified by classroom, this can not be used as another level in our multilevel analysis.Reading Achievement Growth as a Linear ProcessWith only two test administrations, we represent re ading achievement growth as a linear process (Raudenbush and Bryk, 2002: 163-16 9). Moreover, with a small number of observations at the second and third leve ls, we have sought to be parsimoniously selective in specifying our model (K reft and De Leeuw, 1998: 58-60). Independent variables are limited to time, to represent movement from the beginning to the end of the school year in our repeated measures analysis; gender at levels two and three, reflecting our inte rest in reading achievement as a function of gender differences among poor, rural elementary school students; age at levels two and three; grade level at level t wo; mean classroom size at level three; school size at level three; percent of students eligible for


8 of 22 free/reduced cost lunch at level three; and grade-s pan configuration at level three.Independent Variables DefinedTime (TIME1) is a first-level, within-subjects meas ure which corresponds to the two dates of test administration. TIME1 has a rando m coefficient. This means that the relationship between TIME1 and the repeate d measures dependent variable has been permitted to vary from student to student, with the regression coefficient corresponding to TIME1 treated as funct ion of cross-level interactions of TIME1 with second-level and third-l evel independent variables. Second-level variables include gender (GENDER2), gr ade level (GRADE2), and age (AGE2). All second-level variables have fixed c oefficients, except GENDER2. The random coefficient corresponding to GE NDER2 is permitted to vary from school to school, and is treated as a fun ction of cross-level interactions with third-level variables.Random coefficients are used with TIME1 and GENDER2 because of the importance of these variables in our analysis: we a re working with a growth model, and our primary substantive interest is in t he relationship between gender and achievement.Random coefficients might have been used with other level two independent variables, acknowledging that their regression coef ficients may vary from school to school. In addition, use of a random intercept i s commonplace, reflecting differences among mean achievement level from schoo l to school. However, use of random coefficients and a random intercept i s a case-intensive process, and we are constrained by the small number of stude nts and schools in our secondary analysis. In addition, the primary purpos e of second level and third level variables which do not measure gender effects is to serve as controls. We are less concerned with accurately gauging the nume rical magnitude and statistical significance of regression coefficients for control variables than for variables gauging gender effects.Third-level, between-school, variables used in our analysis are gender composition (GENDER3), school size as measured by t otal enrollment (SCHLSIZE3), mean class size (CLASSIZE3), percent e ligible for free or reduced cost lunch (LUNCH3), and grade span configu ration (SPAN3). Each of these explanatory factors has a fixed coefficient.The Absence of EthnicityCertainly, ethnicity or race, with their predictabl y non-meritocratic consequences, could rightly be construed as variabl es which demand inclusion in any discussion of the relationship between socia lly ascribed traits and achievement. However, this poor, aging, rural Appa lachian county, is 96.4 percent white, and none of the students in our samp le was reported to be non-white.The Absence of Individual StudentsÂ’ Social Class


9 of 22 Information which would enable us to estimate each studentÂ’s social class or socioeconomic status was not included in the data s et used in our secondary analysis. Among our eight elementary schools, howev er, the percentage of students eligible for free or reduced cost lunch ra nges from fifty-five percent to ninety five-percent, with a median of seventy-four percent. This information, in the form of the level three variable LUNCH3, is use d as a between-schools explanatory factor.The Absence of Grade-Level CompositionOur analysis includes a variable which assigns a gr ade level, first or second, to each student. This is an essential control. Howeve r, efforts to aggregate this information to the school level and incorporate it as a level three explanatory factor produced serious multicollinearity problems. When the aggregated grade-level variable is deleted, however, all Varia nce Inflation Factors and the Condition Index are well within normal limits.Cross-Level InteractionsCross-level interaction terms are a staple of multi level modeling. They are essential in defining the mathematical character of multilevel models (Snijders and Bosker, 1999: 72-83; Angeles and Mroz, 2001), a ccounting for variability in random regression coefficients (Kreft and De Leeuw, 1998: 72-105), and are of substantive value, as well.However, as product terms, cross-level interactions proliferate rapidly as the number of independent variables increases. Therefo re, cross-level interactions must be selected judiciously (Snijders and Bosker, 1999: 77; Heck and Thomas, 200: 188-89). Because of the substantive importanc e of gender in our analysis, we have limited our cross-level interaction terms t o those which can be created with GENDER2 or GENDER3 and another independent var iable. Use of grand mean and group mean centering helps to avoid intractable multicollinearity problems by rendering multiplicat ive interaction terms orthogonal to the variables from which they were cr eated. In the present instance, when we use all of the selected independe nt variables and interaction terms in an ordinary least squares multiple regress ion equation, collinearity diagnostics yield fourteen variance inflation facto rs less than 2.00, with the remaining three ranging from 2.16 to 3.00. The valu e of the condition index is 3.38. All measures are well within acceptable limit s (Chatterjee, Hadi, & Price, 2000: 238-241; Kmenta, 1997: 438-439).Woodcock-Johnson 22 Results: Within SubjectsWith the Woodcock-Johnson 22 letter-word identifica tion reading achievement test as our outcome measure, we see in Table 3 that TIME1, the first-level (between-subjects) independent variable with a rand om coefficient, is statistically significant and positive. Since we ar e estimating a growth model, this comes as no surprise. Since TIME1 has two leve ls, coded 0 and 1, the


10 of 22 regression coefficient tells us that the passage of time from the first test administration to the second results in an increase in measured math achievement equal, on the average, to 4.08 points. Since the repeated measures dependent variable has a mean of 22.05 and a standard deviation of 5.77 for the entire sample, this is substantial gro wth, equal to 0.71 standard deviation units in just one school year. TABLE 3 MAIN EFFECTS: WOODCOCK-JOHNSON 22 LEVEL 1: WITHIN STUDENTS PARAMETERESTIMATEt VALUESIG. TIME14.0812.22.000 LEVEL 2: BETWEEN STUDENTS PARAMETERESTIMATEt VALUESIG. GENDER2-1.27-2.40.025 GRADE29.1213.14.000 AGE2-1.67-4.05.000 LEVEL 3: BETWEEN SCHOOLS PARAMETERESTIMATEt VALUESIG. GENDER3-1.06-0.67.509 AGE30.690.71.484 SCHLSIZE30.020.70.486 CLASSIZE30.151.03.313 LUNCH3-0.24-9.18.000 SPAN3-0.31-1.40.172 LEVEL 1 INTERCEPT TERM PARAMETERESTIMATEt VALUESIG. INTERCEPT22.03110.35.000Woodcock-Johnson 22 Results: Between SubjectsThree of the second-level, between-individuals inde pendent variables, have statistically significant regression coefficients: GENDER2, with a random coefficient, and GRADE2 and AGE2, with fixed coeffi cients. The regression coefficient corresponding to gender tells us that m ale students, on average, score 1.27 points below female students. This disad vantage for males holds with a reasonable complement of controls in place, including the level two variables GRADE2 and AGE2. As one would expect, our results for GRADE2 tell us that second graders, on average, do better than first graders, with the


11 of 22 statistically significant regression coefficient sh owing a 9.12 test score advantage for students in the higher grade. Further more, when controlling for GRADE2 and a variety of less closely related factor s, our results show that older students, on average, score 1.67 points per y ear lower than younger students. This reflects the fact that studentsÂ’ age is positively correlated with retention, and those who are retained tend to do le ss well on standardized tests than those who do not repeat one or more grades (Th ompson & Cunningham, 2000).Woodcock-Johnson 22 Results: Between SchoolsAt the third level, between schools, there is one a ggregated variable, LUNCH3, with a statistically significant regression coeffic ient. In this instance, we see that for each one percent increase in our free/reduced c ost lunch variable, the Woodcock-Johnson 22 score decreases, on average, by 0.24 points. Since our social class proxy, LUNCH3, can be construed as a school-level measure of the incidence of poverty, this statistically signif icant and negative relationship is not surprising.Woodcock-Johnson 22 Results: Cross-Level Interactio nsIn Table 4, we see that one cross-level interaction term, GENDER2byGENDER3, has a statistically significant r egression coefficient. This means that, in addition to the positive main e ffect relationship due to gender differences at the between-subjects level, i t is also the case that males do less well than females as the percentage male in a school increases. TABLE 4 CROSS-LEVEL INTERACTIONS: WOODCOCK-JOHNSON 22 PARAMETERESTIMATEt VALUESIG.TIME1byGENDER2-0.62-0.86.406TIME1byGENDER3-1.62-0.79.442GENDER2byGENDER3-8.45-2.16.043GENDER2bySCHLSIZE3-0.05-0.79.437GENDER2byCLASSSIZE30.251.40.447GENDER2byLUNCH30.010.12.910GENDER2bySPAN30.150.31.759Woodcock-Johnson 22 Results: The Influence of Gende r in the Complete ModelBy way of summarizing our results for the WoodcockJohnson 22, Table 5 reports values of the -2 log likelihood summary sta tistic for the empty model and the complete model. With a smaller-is-better summar y statistic, when


12 of 22 explanatory factors are introduced, the numerical v alue of the -2 log likelihood measure decreases, and the decrement is statistical ly significant, meaning an improved model fit (see Snijders & Bosker, 1999: 82 -83). For the full model, we also report the R2 L summary measure. R2 L is the proportional reduction in the -2 log likelihood statistic due to the independent variables (Menard, 2002: 24), here equal to 14.6 percent. TABLE 5 Empty Model Variance Components Error Structure -2 Log Likelihood1324.9 Complete Model Variance Components Error Structure -2 Log Likelihood1132.1 R2 L = 14.6% Of primary importance with regard to the influence of gender, however, are the results already reported in Tables 3 and 4: gender has a between-individuals main effect and a level-two-by-level-three interact ion effect, the product of gender composition at the school level and gender a t the individual level. In both instances, with the Woodcock-Johnson 22 letter -word identification test as the outcome measure, gender works to the disadvanta ge of males. It is useful to emphasize, moreover, that malesÂ’ di sadvantage is, in part, due to the gender composition of the school they attend. A s the percentage of males increases, the male disadvantage is made worse.Woodcock-Johnson 22 Results: Random CoefficientParametersWhen a simplified analysis is run using TIME1 and G ENDER2 with random coefficients as the only independent variables, the variance of the regression coefficient corresponding to GENDER2 is statistical ly significant. However, in Table 6 we see that when all specified third-level variables and cross-level interactions are included, the variance of the GEND ER2 regression coefficient is no longer statistically significant. This means that variability in the random coefficient for GENDER2 has been accounted for by c ross-level interaction effects. TABLE 6 COVARIANCE PARAMETERS: RANDOM EFFECTS PARAMETERESTIMATEWALD ZSIG.TIME10.000.001.000


13 of 22 GENDER24.351.85.065 Intraclass Correlation, Levels1&2 = .616 Intracla ss Correlation, Levels 2&3 = .162 COVARIANCE PARAMETERS: REPEATED MEASURES PARAMETERESTIMATEWALD ZSIG. BEGIN SCHOOL YEAR6.216.15.000 END SCHOOL YEAR5.465.26.000First-Level Error Covariance StructureWith repeated measures analysis, the Mixed Models p rocedure for SPSS 11.0 provides a range of choices for the repeated measur e error structure, including scaled identity, compound symmetry, first-order aut ocorrelation, variance components, and unstructured (SPSS, 2001). The vari ances of the two scores which make up the linear growth measure are substan tially different, 6.21 and 5.46, which is consistent with using variance compo nents in modeling our error covariance structure (Schineller, 1997; Bickel and Howley, 2003). Furthermore, running the analysis with the alternatives yields a smaller-is-better -2 log likelihood statistic larger than that obtained with variance components (see Angeles and Mroz, 2001). Table 6 shows us, moreover that both of the repeated measures covariance parameter estimates ar e statistically significant.Woodcock-Johnson 23 Results: Within SubjectsIn Table 7 we see that, much as with our Woodcock-J ohnson 22 results, TIME1, the first-level (between-subjects) independe nt variable with a random coefficient, is statistically significant and posit ive when using the Woodcock-Johnson 23 passage comprehension reading a chievement test as the dependent variable. The regression coefficient corresponding to the TIME1 within-subjects variable tells us that, from the fi rst test administration to the second, the test score has increased, on average, b y 3.46 points. With a repeated measures dependent variable which has a me an of 9.10 and a standard deviation of 4.68, this is a substantial i ncrease, equal to 0.74 standard deviation units, and comparable to our findings wit h the Woodcock-Johnson 22. TABLE 7 MAIN EFFECTS: WOODCOCK-JOHNSON 23 LEVEL 1: WITHIN STUDENTS PARAMETERESTIMATEt VALUESIG.TIME1 3.4611.49.000 LEVEL 2: BETWEEN STUDENTS


14 of 22 PARAMETERESTIMATEt VALUESIG.GENDER2-1.15-2.26.043GRADE27.5011.65.000AGE2-1.00-3.17.007 LEVEL 3: BETWEEN SCHOOLS PARAMETERESTIMATEt VALUESIG.GENDER3-5.35-3.63.002AGE3 3.423.13.007 SCHLSIZE3-0.05-1.96.070CLASSIZE30.241.79.095LUNCH3-0.13-5.53.000SPAN30.110.53.604 LEVEL 1 INTERCEPT TERM PARAMETERESTIMATEt VALUESIG.INTERCEPT9.2149.13.000Woodcock-Johnson 23 Results: Between SubjectsThree of the second-level, between-individuals inde pendent variables, have statistically significant regression coefficients. As with the Woodcock-Johnson 22 results, these are GENDER2, with a random coeffi cient, and GRADE2 and AGE2, with fixed coefficients. The coefficient corr esponding to gender tells us that male students, on average, score 1.15 points l ower than female students. This disadvantage for males holds with a reasonable complement of controls in place, including the level two variables GRADE2 and AGE2. As before, our results for GRADE2 tell us that second graders, on average, do better than first graders, with the statistically significant regress ion coefficient showing a 7.50 test score point advantage for students in the high er grade. Furthermore, when controlling for GRADE2 and a variety of less closel y related factors, our results show that older students, on average, score 1.00 po int per year lower than younger students. Again, age is correlated with ret ention, with older students more likely to be the retained, and students who ar e retained tending to do less well on standardized achievement tests (Thompson an d Cunningham, 2000).Woodcock-Johnson 23 Results: Between SchoolsAt the third level, between schools, Table 7 shows us that GENDER3, AGE3, and LUNCH3 have statistically significant regressio n coefficients with the Woodcock-Johnson 23 score as the dependent variable In this instance, we


15 of 22 see that for each one percent increase in the perce ntage of students who are male, the Woodcock-Johnson 23 score decreases, on a verage, by 5.35 points. Furthermore, for each one year increase in the aver age age at the school level, average test score increases by 3.42 points. Final ly, for each one percent increase in the free/reduced cost lunch variable, t he Woodcock-Johnson 23 score decreases, on the average, by 0.13 points.Woodcock-Johnson 23 Results: Cross-Level Interactio nsIn Table 8, we see that one cross-level interaction term, GENDER2byGENDER3 has a statistically significant re gression coefficient. This means that, in addition to the negative main effect relationships due to gender differences at the between-subjects and between-sch ools levels, it is also the case that males do less well than females as the pe rcentage male in a school increases. TABLE 8 CROSS-LEVEL INTERACTIONS: WOODCOCK-JOHNSON 23 PARAMETERESTIMATEt VALUESIG.TIME1byGENDER2-0.92-1.41.195TIME1byGENDER3-3.34-1.81.108GENDER2byGENDER3-8.07-2.11.040GENDER2bySCHLSIZE30.020.32.751GENDER2byCLASSIZE3-0.21-0.67.515GENDER2byLUNCH30.030.41.691GENDER2bySPAN3-0.31-0.69.498Woodcock-Johnson 23 Results: The Influence of Gende r in the Complete ModelBy way of summarizing our results for the WoodcockJohnson 23, Table 9 reports values of the -2 log likelihood summary sta tistic for the empty model and the complete model. Again, with the smaller-is-bett er summary statistic, when explanatory factors are introduced, the numerical v alue of the -2 log likelihood measure decreases, and the model-to-model decrement is statistically significant. TABLE 9 Empty Model Variance Components Error Structure -2 Log Likelihood 1238.2 Complete Model


16 of 22 Variance Components Error Structure -2 Log Likelihood 1051.5 R2 L = 15.1% Since gender effects are our primary concern, howev er, the findings already reported in Tables 7 and 8 are of special interest: gender has both between-individuals and between-schools main effect s, as well as a level-two-by-level-three interaction effect. In all three instances, with the Woodcock-Johnson 23 passage comprehension test as t he outcome measure, gender works to the disadvantage of males.As with the Woodcock-Johnson 22, it is useful to em phasize that gender effects on the Woodcock-Johnson 23 are not limited to the i ndividual level. Instead, as the percentage of students who are male increases, the scores of all students are, on average, diminished, and the scores of male students specifically are diminished still more.Woodcock-Johnson 23 Results: Random Coefficient ParametersWhen the analysis is run using just TIME1 and GENDE R2 as independent variables with random coefficients, the variance of neither coefficient is statistically significant. The same is true for res ults based on the full model, reported in Table 10. This means that the coeffici ents corresponding to these two explanatory factors do not vary from one higher level unit to another. TABLE 10 COVARIANCE PARAMETERS: RANDOM EFFECTS PARAMETERESTIMATEWALD ZSIG. TIME10.000.001.000 GENDER24.881.64.101 Intraclass Correlation, Levels1&2 = .471 Intracl ass Correlation, Levels 2&3 = .311 COVARIANCE PARAMETERS: REPEATED MEASURES PARAMETERESTIMATEWALD ZSIG. BEGIN SCHOOL YEAR5.684.65.000 END SCHOOL YEAR3.773.04.002First-Level Error Covariance Structure


17 of 22 As with the Woodcock-Johnson 22, when using repeate d measures analysis with Woodcock-Johnson 23, the variances of the two scores which make up the linear growth measure differ substantially, having values of 5.74 and 3.68. Again, this is consistent with using variance compo nents error structure. As before, variance components error structure yielded the smallest value for the smaller-is-better -2 log likelihood summary statist ic, and Table 11 shows us that both repeated measures covariance parameter estimat es are statistically significant.DiscussionWith unusual consistency across two widely used mea sures of reading achievement, we have found that first and second gr ade males in a poor, rural, Appalachian school district do less well than femal es. For the both Woodcock-Johnson 22 and 23, individual male student s, on average, do less well than female students. In addition, for the Woo dcock-Johnson 23, as the percentage of students in a school who are male inc reases, the scores of all students tend to decline. Furthermore, for the Wood cock-Johnson 22 and 23, as the percentage of a school’s students who are ma le increases, the scores of male students specifically are further diminished.Of special importance for our research objectives a re the group effect of gender with the Woodcock-Johnson 23, and the interaction e ffects involving gender with both the Woodcock-Johnson 22 and the WoodcockJohnson 23. Both sets of effects make clear that the role of the socially ascribed trait gender in determining reading achievement is not limited to t he individual level. As such, gender effects take forms that may be difficult to anticipate. How one remedies group-level gender group effects and gender-implica ted interaction effects, moreover, is not clear. It does seem clear, however that “No Child Left Behind” presumes a social world wherein schooling is less c omplex, and easier to understand and reform, than is actually the case.Imagine, for example, a distribution of schools whi ch vary with regard to gender composition. Our results suggest that as the percen tage of students who are male increases, school mean scores in reading achie vement may decline for three reasons: individual males do less well than f emales; the greater the percentage of males, the lower the scores for all s tudents; and the greater the percentage of males, the lower the scores for males specifically. Given the accountability measures and sanctions proposed by “ No Child Left Behind,” having a large percentage of males in a school coul d be disastrous.ConclusionAt the outset, we noted that the disarmingly straig htforward and science-focused character of “No Child Left Behind” is judged by many professional educators to be misleading. In their v iew, the effects of class, race, gender, and context cannot be explained and remedie d with the ease the Act implies. We added that the involvement of social as cription in group effects and interaction effects could further complicate matter s with regard to both substance and method. We have now demonstrated that gender effects for


18 of 22 elementary reading can be complex, indeed, taking t he form of individual effects, group effects, and interaction effects. Th is makes it likely that the socially ascribed trait gender will intrude in unan ticipated and undetected ways in determining the achievement objectives and accou ntability measures mandated by “No Child Left Behind.” Our findings le nd credence to the view that “No Child Left Behind” oversimplifies the social co ntext of schooling and underestimates the importance of socially ascribed traits.ReferencesAiken, L. and West, G. (1991) Multiple Regression: Testing and Interpreting Interactions. Newbury Park, CA: Sage. Angeles, A. and Mroz, T. (2001) A Guide to Using Mu ltilevel Models for the Evaluation of Program Impact. Chapel Hill, NC: Carolina Population Cente r, University of North Carolina at Chapel Hill. Baron, R. and Kenny, D. (1986) The Moderator-Mediat or Variable Distinction in Social Psychological Research: Conceptual, Strategic, and Statistical Co nsiderations. Journal of Personality and Social Psychology. 51: 1173-1182. Bianchini, L (2002) NCTE Resolution on the Reading First Initiative. News from Bickel, R. and Howley, C. (2000) The Influence of S cale on School Performance: A Multilevel Extension of the Matthew Principle. Education Polic y Analysis Archives. 8. Bickel, R. and Howley, C. (2003) Elementary Math Ac hievement and Rural Development: Effects of Contextual Factors Intrinsic to the Modern World. A thens, OH: ACCLAIM Working Paper No. 15, Appalachian Collaborative for Learning, Assessment, and Instruction in Mathematics. Bickel, H., Howley, C., Glascock, C., and Williams, T. (2001) High School Size, Achievement Equity, and Cost: Robust Interaction Effects and Tentative Results. Education Policy Analysis Archives. 9. Bickel, R., Smith, C., and Eagle, T. (2002) Poor, R ural Neighborhoods and Early School Achievement. Journal of Poverty. 6: 89-108. Bracey, G. (2003) NCLB – A Plan for the Destruction of Public Education. NoChildLeft.Com. Februray. Bush, G. (2002) President Launches Quality Teacher Initiative. Washington, D.C.: Executive Office of the President. Chatterjee, S., Hadi, A., and Price, B. (2000) Regr ession Analysis by Example. New York: Wiley. Cloer, T. and Dalton, S. (2001) Gender and Grade Di fferences in Reading Achievement and in Self-Concept as Readers. Journal of Reading Achieve ment. 26: 31-36. Coles, G. (2001) Learning to Read “Scientifically.” Rethinking Schools Online. Summer. d154.htm Denlinger, S. (2002) Teaching as a Profession: A L ook at the Problem of Teacher Deficits. Clearing House. 75: 116-117. Hardy, L. (2003) Overburdened and Overwhelmed. ASBJ .Com. April. Heck, R. and Thomas, L. (2000) An Introduction to M ultilevel Modeling Techniques. Mahwah, NJ: Lawrence Earlbaum. High, C. (1996) The Texas Study: A Regression Analy sis of Selected Factors that Influence the Scores of Students on the TASP Test. Houston: Texas Association of College Testing


19 of 22 Personnel. Huston, P. (2003) The Bigotry of Expectations. Scho ol Administrator. January. ht tp:// 01/execeper. htm International Reading Association (2003) IRA Survey Examines Process for Reading First Applications. Reading Today. February/March. http://www/ feb_survey.html Iversen, G. (1991) Contextual Analysis. Newbury Par k, CA: Sage. Jaccard, J., Turrisi, T., and Choi, K. (1990) Inter action Effects in Multiple Regression. Newbury Park CA: Sage. Kmenta, J. (1997) Elements of Econometrics. Ann Arb or, MI: University of Michigan. Kreft, I. and De Leeuw (1998) Introducing Multileve l Modeling. Thousand Oaks, CA: Sage. Lynch, J. (2002) Parents’ Self-Efficacy Beliefs, Pa rents’ Gender, Children’s Reader Self-Perceptions, Reading Achievement, and Gender. Journal of Researc h in Reading. 25: 54-67. Menard, S. (2002) Applied Logistic Regression Analy sis. Newbury Park, CA: Sage. Otterbourg, S. (2001) The Partnership for Family In volvement in Education: Who We and What We Do. Jessup, MD: The Partnership for Family Involvem ent in Education. Phillips, L., Norris, S., Osmond, W. & Maynard, A. (2002) Relative Reading Achievement of 187 Children from First through Sixth Grades. Journal o f Educational Psychology. 94: 3-13. Raudenbush, S. and Bryk, A. (2002) Hierarchical Lin ear Models. Thousand Oaks, CA: Sage. Rebora, A. (2002) No Child Left Behind. Education W eek on the Web, April 2. ?id=59 Schineller, L. (1997) An Econometric Model of Capit al Flight from Developing >Countries. Washington, D.C.: International Finance Discussion Paper, Number 579, Board of Governors of the Federal Reserve System. Seldon, R (2001) Parent Power: Why National Standar ds Won’t Improve Education. Washington, D.C.: The Cato Institute. Snijders, T. and Bosker, R. (1999) Multilevel Analy sis. Thousand Oaks, CA: Sage. Solon, G., Page, M., and Duncan, G. (2000) Correlat ions Between Neighboring Children in Their Subsequent Educational Attainment. Review of Econo mics and Statistics. 82: 383-393. SPSS (2001) SPSS Advanced Models 11.0. Chicago, IL: SPSS. Thompson, C. and Cunningham, E. (2000) Retention an d Social Promotion: Implications for Policy. New York: Teachers College, Columbia University, ER IC Clearinghouse on Urban Education. U.S. Census Bureau (2001) QuickFacts. Washington, D .C.: U.S. Census Bureau, Department of Health and Human Services. U.S. Department of Education (2002a) The “No Child Left Behind Act of 2001,” Executive Summary (Updated). Washington, D.C.: U.S. Government Printi ng Office. U.S. Department of Education (2002b) The “No Child Left Behind” Act: Reauthorization of the Elementary and Secondary Act Legislation and Polici es Website. July 11. < E/esea/ > U.S. Department of Education (2003) Comments by Sec retary Paige to the Commonwealth Club of California. March 12. /03122003a.html Velicer, W. (1972) The Moderator Variable Viewed as Heterogeneous Regression. Journal of Applied Psychology. 56: 266-269.


20 of 22 Vartanian, T., and Gleason, P. (1999) Do Neighborho od Conditions Affect High School Dropout and College Graduation Rates. Journal of Socioeconomics 28: 21-42. White House (2002) Fact Sheet: No Child Left Behind January 8. www.whitehouse/gov/news/releases/2002/01/20020108.h tml White House (2001) Transforming the Federal Role in Education so that No Child No Child is Left Behind. December 12. d.html Woodcock, R. and Johnson, M. (1990) Woodcock-Johnso n Tests of Achievement. Allen, TX: DLM Teaching Services.About the AuthorsRobert Bickel is Professor of Advanced Educational Studies at Ma rshall University. His recent research is concerned with c orrelates of crime on school property, the limits of educational reform in promo ting rural development, and adverse consequences of schools’ efforts to meet th e requirements of “No Child Left Behind.” He has recently completed a monograph on multi-level analysis for education policy analysts.Stan Maynard is Professor of Secondary Education at Marshall Un iversity. He is also Executive Director of the June Harless Cent er for Rural Educational Research and Development. His primary interest has long been development of innovative, cost-effective ways to assure high-qual ity public education for less-advantaged students living in isolated rural a reas The World Wide Web address for the Education Policy Analysis Archives is Editor: Gene V Glass, Arizona State UniversityProduction Assistant: Chris Murrell, Arizona State University General questions about appropriateness of topics o r particular articles may be addressed to the Editor, Gene V Glass, or reach him at College of Education, Arizona State Un iversity, Tempe, AZ 85287-2411. The Commentary Editor is Casey D. Cobb: .EPAA Editorial Board Michael W. Apple University of Wisconsin David C. Berliner Arizona State University Greg Camilli Rutgers University Linda Darling-Hammond Stanford University Sherman Dorn University of South Florida Mark E. Fetler California Commission on TeacherCredentialing


21 of 22 Gustavo E. Fischman Arizona State Univeristy Richard Garlikov Birmingham, Alabama Thomas F. Green Syracuse University Aimee Howley Ohio University Craig B. Howley Appalachia Educational Laboratory William Hunter University of Ontario Institute ofTechnology Patricia Fey Jarvis Seattle, Washington Daniel Kalls Ume University Benjamin Levin University of Manitoba Thomas Mauhs-Pugh Green Mountain College Les McLean University of Toronto Heinrich Mintrop University of California, Los Angeles Michele Moses Arizona State University Gary Orfield Harvard University Anthony G. Rud Jr. Purdue University Jay Paredes Scribner University of Missouri Michael Scriven University of Auckland Lorrie A. Shepard University of Colorado, Boulder Robert E. Stake University of Illinois—UC Kevin Welner University of Colorado, Boulder Terrence G. Wiley Arizona State University John Willinsky University of British ColumbiaEPAA Spanish and Portuguese Language Editorial BoardAssociate Editors for Spanish & Portuguese Gustavo E. Fischman Arizona State Universityfischman@asu.eduPablo Gentili Laboratrio de Polticas Pblicas Universidade do Estado do Rio de Janeiro pablo@lpp-uerj.netFounding Associate Editor for Spanish Language (199 8-2003) Roberto Rodrguez Gmez Universidad Nacional Autnoma de Mxico Adrin Acosta (Mxico) Universidad de J. Flix Angulo Rasco (Spain) Universidad de


22 of 22 Teresa Bracho (Mxico) Centro de Investigacin y DocenciaEconmica-CIDEbracho Alejandro Canales (Mxico) Universidad Nacional Autnoma Ursula Casanova (U.S.A.) Arizona State Jos Contreras Domingo Universitat de Barcelona Erwin Epstein (U.S.A.) Loyola University of Josu Gonzlez (U.S.A.) Arizona State Rollin Kent (Mxico) Universidad Autnoma de Puebla Mara Beatriz Luce (Brazil) Universidad Federal de Rio Grande do Javier Mendoza Rojas (Mxico)Universidad Nacional Autnoma Marcela Mollis (Argentina)Universidad de Buenos Humberto Muoz Garca (Mxico) Universidad Nacional Autnoma Angel Ignacio Prez Gmez (Spain)Universidad de DanielSchugurensky (Argentina-Canad) OISE/UT, Simon Schwartzman (Brazil) American Institutes forResesarch–Brazil (AIRBrasil) Jurjo Torres Santom (Spain) Universidad de A Carlos Alberto Torres (U.S.A.) University of California, Los EPAA is published by the Education Policy Studies Laboratory, Arizona State University


Download Options

Choose Size
Choose file type
Cite this item close


Cras ut cursus ante, a fringilla nunc. Mauris lorem nunc, cursus sit amet enim ac, vehicula vestibulum mi. Mauris viverra nisl vel enim faucibus porta. Praesent sit amet ornare diam, non finibus nulla.


Cras efficitur magna et sapien varius, luctus ullamcorper dolor convallis. Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Fusce sit amet justo ut erat laoreet congue sed a ante.


Phasellus ornare in augue eu imperdiet. Donec malesuada sapien ante, at vehicula orci tempor molestie. Proin vitae urna elit. Pellentesque vitae nisi et diam euismod malesuada aliquet non erat.


Nunc fringilla dolor ut dictum placerat. Proin ac neque rutrum, consectetur ligula id, laoreet ligula. Nulla lorem massa, consectetur vitae consequat in, lobortis at dolor. Nunc sed leo odio.