|USFDC Home||| RSS|
This item is only available as the following downloads:
xml version 1.0 encoding UTF-8 standalone no
mods:mods xmlns:mods http:www.loc.govmodsv3 xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govmodsv3mods-3-1.xsd
mods:relatedItem type host
mods:identifier issn 1068-2341mods:part
mods:detail volume mods:number 10issue 38series Year mods:caption 20022002Month September9Day 2222mods:originInfo mods:dateIssued iso8601 2002-09-22
xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam a22 u 4500
controlfield tag 008 c20029999azu 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E11-00286
Educational policy analysis archives.
n Vol. 10, no. 38 (September 22, 2002).
Tempe, Ariz. :
b Arizona State University ;
Tampa, Fla. :
University of South Florida.
c September 22, 2002
District fiscal policy and student achievement : evidence from combined NAEP-CCD data / Gary G. Huang [and] Binbing Yu.
Arizona State University.
University of South Florida.
t Education Policy Analysis Archives (EPAA)
1 of 39 Education Policy Analysis Archives Volume 10 Number 38September 22, 2002ISSN 1068-2341 A peer-reviewed scholarly journal Editor: Gene V Glass College of Education Arizona State University Copyright 2002, the EDUCATION POLICY ANALYSIS ARCHIVES .Permission is hereby granted to copy any article if EPAA is credited and copies are not sold. EPAA is a project of the Education Policy Studies Laboratory. Articles appearing in EPAA are abstracted in the Current Index to Journals in Education by the ERIC Clearinghouse on Assessment and Evaluation and are permanently archived in Resources in Education .District Fiscal Policy and Student Achievement: Evidence from Combined NAEP-CCD Data Gary G. Huang Binbing Yu Synectics for Management Decisions, Inc.Citation: Huang, G. & Yu, B. (2002, September 22). District fiscal policy and student achievement: Evidence from combined NAEP-CCD data, Education Policy Analysis Archives 10 (38). Retrieved [date] from http://epaa.asu.edu/epa a/v10n38/.Abstract School restructuring raises questions about the rol e of school districts in improving student learning. Centralization by state governments and decentralization to individual schools as proposed in systemic reform leave districts' role unsettled. Empirical research on the district role in the context of ongoing reform is inadequate. This a nalysis of combined data from the NAEP and the Common Core of Data (CCD ) was intended to address the issue. We analyzed 1990, 1992, and 1 996 NAEP 8th grade mathematics national assessment data in combination with CCD data of
2 of 39corresponding years to examine the extent to which student achievement was related to districts' control over instructiona l expenditure, adjusting for relevant key factors at both district and stude nt levels. Upon sample modification, we used hierarchical linear modeling (HLM) to estimate the relationships of student achievement to two dis trict fiscal policy indictors, current expenditure per pupil (CEPP) and districts' discretionary rates for instructional expenditure ( DDR). Net of relevant district factors, DDR was found unrelated to distri cts' average 8th grade math performance. The null effect was consistent in the analysis of the combined NAEP-CCD data for 1990, 1992, and 1996. In contrast, CEPP was found related to higher math performance in a m odest yet fairly consistent way. Future research may be productive t o separately study individual states and integrate the findings onto t he national level. The role of the local school district is problemati c in the on-going school restructuring that focuses on student learning. Centralization by state govern ments and decentralization to individual schools as proposed in systemic reform leaves districts' ro le unsettled (Elmore 1993). Research is needed to examine the impact of district policymaking on stud ent learning. Such research entails linking standardized achievement measures at the student le vel to district-level policy information comparable across jurisdictions.This study explored policy and methodological issue s relevant to these concerns. We analyzed NAEP data in combination with the Common Core of Da ta (CCD) to examine the extent to which student achievement was related to districts' contr ol over expenditure, adjusting for relevant key factors at both district and student levels.Research BackgroundEducation reformers face a dilemma in trying to red efine the local school district role in relation to state government. On the one hand, the local distri ct, as an independent political entity for local control of public schools, is supposed to buffer th e schools against external political influences. On the other hand, as a legal creation of the state, t he district is expected to work as an instrument of state government and to implement state policies an d regulations with minimal change. The notion of simultaneous decentralization and centralization gives states and individual schools more policymaking authority, but leaves the local distri cts in an ambiguous position (Hannaway 1992; Clune 1993; Keedy 1994; Marsh 1997). As the authori ty and responsibility of the state governments continue to expand in public education, districts' role as a governmental unit seems to be diminishing (e.g., see Walberg & Walberg 1994; E lmore 1993). State governments are playing an increasingly impor tant role in fiscal control and prescriptive policymaking in a wide range of areas that were his torically the domains of local district decisionmaking. States are contributing a large and growing share to local school revenue, specifying requirements on staffing and instruction (General Accounting Office 1998). State aid formulas have often incorporated specific mechanism s for district inefficiency control, linking spending to performance (Duncombe & Yinger 1998; Ma rsh 1997). Frequently, state governments threaten to and occasionally do take over local sch ools when persistent failure by local administration is confirmed (e.g., Iannaccone & Lut z 1994; Guskey 1993). The districts' power is also threatened by decentra lization in forms of site-based management and
3 of 39the school as a professional community (Marsh 1997; Elmore 1993; Clune 1993; Porter 1994; Newmann & Wahlage 1995). State education department s now often select individual schools as administrative units for deregulation and interact directly with them to encourage local initiatives (Fuhrman & Elmore 1995). Local schools are expandin g their options in mobilizing community support, making innovations in instruction and curr iculum, and making decisions on spending and staffing (Elmore 1993; Hannaway 1993). Research app ears to support such restructuring as recent studies focus on school-level professional autonomy and its link to authentic instruction and achievement outcomes (e.g., Lee, Smith & Croninger 1995; Newmann & Wahlage 1995). One implication is that decentralization is reducing bu reaucratic influenceincluding district influencein local school decisionmaking.The 16,000 local districts in the U.S. are, however more than a historical heritage. National data reports describe local districts as solid administr ative units with diverse conditions (e.g., Levine & Christenson 1998; Protheroe 1997). The question for reformers is how to adjust the system to the changing environment, since the underlying logic of American federalism requires multiple jurisdictions that compromise different interests a nd generate productive tension and dependence among jurisdictions (Elmore 1993). Available resear ch suggests that it is possible for local districts to push reform further by building up strong local constituencies and developing policy initiatives (Fuhrman & Elmore 1990; Admundson 1993). This resea rch, however, has been conducted largely with local or state information sources and has rar ely used national data. To conduct research at national scale requires nationwide data on both stu dent achievement and local fiscal conditions. Synthetic analysis of the National Assessment of Ed ucational Progress (NAEP) and the Common Core of Data (CCD), explored in this study, seems a n efficient approach to this goal.Conceptual FrameworkOur basic assumption is that although state governm ents are constitutionally responsible for public education, local districts have an important role i n improving instruction. An approach districts take in playing that role is to adjust fiscal policy to increase the proportion of expenditure for instruct ion above the state average.Instructional expenditure as a district policySchool districts differ in instructional expenditur e. A recent study used the Bureau of the Census Annual Survey of Local Government data from 1980 to 1994 to calculate multiple indicators of cross-district disparity in instructional expenditu re (Hussar & Sonnenburg 1999). This study found that while disparity in instructional expenditures across districts seemed to decline in many states, disparity measures and the pattern of decline varie d substantially. Furthermore, in a fairly large number of states, the disparity measures were incon sistent, and, in a small number of states, disparity increased over the same period of time. W hile these disparity measures may imply inequity they also reflect local fiscal policy diff erences because instructional expenditures, relativ e to current expenditures, are more subject to govern ment control (Hussar & Sonnenburg 1999). Research is unsettled about the relationship betwee n school expenditure and student achievement (for reviews and debates, see Conn 1995; Lockwood 1 994; Hanushek 1994, 1996; Hedges & Greenwald 1996; Hough 1993). An emerging consensus, however, is that overall funding and per-pupil expenditure may be overly simplistic as a predictor of learning outcomes, since schooling is conditioned by complicated factors of administra tion, family, and community (e.g., Wainer 1993; Hough 1993). While recent research has provided som e evidence supporting the view that increased funding relates to better performance (e. g., Payne & Biddle 1999; Wenglinsky 1997),
4 of 39whether overall revenue or average per-pupil expend iture substantially affects academic outcomes remains controversial (e.g., Hanushek 1996; Hedges & Greenwald 1996). Research needs to relate performance outcomes to variations in resource allo cation in general and teaching resources devoted to specific program areas in particular (Br ent, Roellke & Monk 1997; Monk & Hussain 2000). How the money is spent may be crucial to imp roving achievement (Childs & Shakeshaft 1986; Conn 1995). Research examining the impact of public school spending, particularly spending for disadvantaged schools, suggests that learning c an be improved if investment targets improving teacher quality (e.g., Kazal-Thresher 1993; Ferguso n 1991), core curriculum, and standardized achievement (e.g., Lockwood 1994).Policy analyses in systemic reform highlight the im portance of fiscal control as an incentive/sanction mechanism directly linking to ac ademic performance (Lockwood 1994; Elmore 1994). Under this mechanism, schools and teachers a re encouraged to develop pedagogical initiatives for reaching high-standard curricular g oals and are held accountable for improving student performance. In this line of thinking, the amount of money spent directly on improving instruction/learning relative to the amount spent on administrative operations a nd support services should be a crucial determinant of achievement outc omes (Childs & Shakeshaft 1986; Elmore 1993). On the other hand, current expenditure per p upil (CEPP), which covers spending on broad and immediate local needs, to some extent indicates local initiatives and priorities in fiscal control Encompassing obligatory capital outlay and district discretionary spending, CEPP may also be a valid predictor of student achievement.In this study, we ask how districts change state pr escribed patterns of expenditure and how such changes relate to academic achievement. The departu re of a district's proportional spending on instruction from the state baseline proportion may imply local autonomy in fiscal policy, which has been advocated as a fundamental mechanism for maint aining local stake and accountability (Murphy 1994; Strang 1987). Conceptualizing fiscal control as an essential element of local autonomy, we attempted to pinpoint a key issue in r edefining the district role. We hypothesized that, other conditions being equal, a higher distri ct instructional spending rate relative to the stat e average is associated with higher district average math achievement. In addition, we also examined CEPP, a more generic indicator of local financial c ontrol, by assessing its relationship to student performance.Factors related to district spendingCommunities are often constrained from spending mor e on instruction by local conditions. For example, poor communities have to spend more on bas ic social services, and sparsely populated areas must pay more for transportation. A large sta te contribution to district revenue probably constrains local discretion. The larger the share o f the state contribution, the more likely the recipient districts fund instruction at a level clo se to the state baseline. Local cost of living is another factor that needs to be considered when exa mining school resources as it obviously affects the spending for teacher hiring and instruction.The power of districts varies depending on local so cioeconomic conditions, enrollment size, and geographic locale. Local socioeconomic conditions a s a determinant of local autonomy may alter the intended consequence of district policy. Wealth y districts with their well educated populations and organized political support may push reform far ther and benefit more from focusing on instruction and learning (Elmore 1993). In contrast poor districts with populations of low education and income but high mobility often cannot come up with powerful political support for reform. Increasing spending on instruction may comp romise desperately needed social services and
5 of 39ultimately undermine student learning. The influenc e of district policy probably also varies across urban-rural areas. Traditionally, rural residents a re demographically homogeneous within communities but diverse across communities (Nachtig al 1982), have strong ties to local schools, and are skeptical about external governance (DeYoun g 1990). Within different sociodemographic contexts, districts' investment in instruction may result in quite different achievement outcomes. Prior studies have explored school funding in conne ction to standard achievement by synthetic analyses of data from CCD and NAEP (Wenglinsky 1997 1998) or international standardized tests (Payne & Biddle 1999). In particular, Wenglinsky (1 997) aggregated achievement measures to the district level and used both school-level and distr ict-level variables to predict achievement. Taking the district as the unit of analysis, single-level analyses of aggregated performance show that average performance is positively associated with s chool resources. While such aggregated studies deal with the broad issues linking school resources to performance, they were not meant to and did not address specific questions concerning district fiscal policy in relation to individual student achievement. Furthermore, there are methodological problems in converting NAPE's multi-level stratified sample design into a single-level distri ct sample design. The district sample resulting fro m attaching aggregated NAEP performance data to CCD d istrict records does not necessarily represent the national population of school distric ts. In a second study, Wenglinsky used a hierarchical l inear model (HLM) technique to analyze merged 1992 NAEP and CCD data on 12th grade mathematics (Wenglinsky 1998). Again, this analysis did not focus on district fiscal control. It addressed the broad concern of school resources in connection to social distribution of academic ac hievement. Further, the study did not distinguish district and school spending measures in the analys is. It did not deal with the methodological difficulty stemming from the data merge, namely, th e possible unreliability of the estimates resulting from the merged data. (Note 1) Our study was designed to continue this line of re search by synthesizing national data on student achievemen t and district spending, using sampling adjustment procedures and the technique of two-leve l hierarchical linear modeling. Research questionsThe first question in our study was whether or not student achievement varies across districts. The answer to this question sets the basis for addressi ng the substantive concern of district-level effect and for statistical testing of two-level models. Th e second question was how student achievement relates to district instructional spendingand CEPP As the central issue in this analysis, we asked how district mean achievement related to district f iscal policy, adjusting for other variables at the district level and sociodemographics at the student level. District fiscal policy was indicated by two variables, CEPP and District Discretionary Rate of instructional spending. The latter was simply the difference of the district instructional spending r ate from the state instructional spending rate. Compatible to the concept underlying such disparity measures as the coefficient of variation and the Gini coefficient (see Hussar & Sonnenberg 1999), DD R should make it straightforward to interpret the HLM estimates of the district fiscal policy eff ect. The third question was: to what extent did DDR and CEPP, together with other district factors, account for the achievement outcome variance after adjusting for relevant district-level factors (e.g., enrollment size, state contributed share to district revenue, minority rate, poverty rate, geographic locale). In addition, we examined the po ssible interaction effects of DDR with three district conditions: the proportion of district rev enue that came from the state, the average socioeconomic status (SES), and urban-rural locale. This would allow us to ask whether DDR had different effects on achievement under different co nditions.
6 of 39 A final question about the achievement gaps: did hi gher district instructional spending help reduce math achievement gaps associated with race and SES? In other words, did increased instruction spending above the state average not only work to p romote academic excellence, but also equity?Data Sources and MethodologyFor this study, we combined data from NAEP 1990, 19 92, and 1996 National Comparison Grade 8 Files and the Common Core of Data (CCD) in school y ears 198990, 199192, and 199596. As the most comprehensive and reliable national dat a source on academic achievement, the NAEP math tests in these three years shared a framework supported by the National Council of Teachers of Mathematics (Reese, Miller, Mazzeo & Dossey 1997 ). CCD, covering the universe of U.S. public school di stricts, provides district-level itemized revenue/expenditures on an annual basis (National C enter for Education Statistics 1995). It also contains a state file that gives spending data at t he state level, which can be used to compute DDR. CCD offers information for examining district fisca l policies, including core expenditures per pupil, current expenditures per pupil, total expend itures per pupil, percent of total instruction expenditures, and percent of total salary expenditu res, as well as related state spending measures. Additionally information on local sociodemographic conditions is available, including extensive 1990 Census data that were incorporated into distri ct records. A linkage file is available for linking NAEP national and state assessments files with CCD data for the years between 1990 and 1998 (Westat 1998).File MergeFirst, we extracted data from the 1990, 1992, and 1 996 NAEP National Comparison of math in 8th grade and CCD district files for school years 1989 90, 199192, and 199596. Using district identification code in both the NAEP and CCD files, we merged the two datasets for these years. With assistance from Westat and ETS, the three year s' data were matched reasonably well. Most districts contained adequate numbers of students fo r two-level analysis (see Table 1).Table 1 Sample size at district and within-district levels: Combined NAEP-CCD data (1990, 1992, and 1996, unwei ghted)Year District sample size Mean within-district student sample size Standard deviationMinimumMaximum 1990144184.108.40.206 157.0 1992177220.127.116.11 185.0 199616034.918.610.0113.01 For the 1990 data one district (LEAID 3701530) had two cases and was excluded from the
7 of 39 analysis.2 For the 1992 data two districts (with LEAIDs 53049 20 and 1713970) had fewer than three cases and were excluded from the HLM analysis.The resulting files contain NAEP 8th grader records with affiliated district variables attached. Additionally, district-level cross-product terms we re constructed to represent the interaction effects of DDR and the child poverty rate, DDR and urban lo cale, and DDR and the percentage of district revenue from the state (see Tables 2.12.3 for unwe ighted descriptive statistics at the two levels).Table 2-1 Descriptive statistics at studentand district-lev els: 1996 NAEP-CCD dataStudent Level 1 Variable nameVariable labelNMeanSDMinimumMaximumDSEXStudent sex 5,5901.500.501.002.00 MRPCM1Plausible value 1 5,590268.9736.46127.98388.50 MRPCM2Plausible value 2 5,590269.1736.03120.35393.46 MRPCM3Plausible value 3 5,590268.8736.52124.06384.47 MRPCM4Plausible value 4 5,590268.8936.01138.54378.93 MRPCM5Plausible value 55,590269.1435.87121.28381.77MINORITYNon-Asian minorities 5,5900.330.470.001.00 PARHI_EDParent education 5,5900.580.490.001.00 District level 2 ASIERAverage state instruction expenditure rate 16060.996.4850.0877.85 DDRDistrict Discretionary instruction spending Rate 160-0.246.78-24.9917.35 URBAN2Urban district1600.270.430.001.00RURALRural district 1600.210.410.001.00 ENROLL_KDistrict total enrollment in thousand 16041.84110.320.141,049.04 BLACK_PBlack student rate 16019.0024.750.0099.00 P7118TPDistrict poverty rate 16017.0111.190.0058.50 PC30ETPDistrict at-risk student rate 1603.683.400.0015.80 C_STREVPDistrict revenue percentage from state 16047.6118.961.8078.90 CURPPE_KCurrent per pupil expenditure in $K 1605.361.433.3811.27 LEV2WTDistrict weight 1601.001.370.0610.29 DDR_RULInteraction DDR by rural 160-1.114.18-24.996.35
8 of 39 DDR_URBInteraction DDR by urban 1600.402.94-18.9613.44 DDR_STPInteraction DDR by revenue percentage from state 160-27.66355.46-1404.46921.62 DDR_SESInteraction DDR by poverty rate 160-20.10153.74-753.19529.19 DDR_RSKInteraction DDR by at-risk student rate 160-2.1635.07-189.98112.78Table 2-2 Descriptive statistics of at studentand districtlevels: 1992 NAEP-CCD dataStudent level 1 Variable name Variable labelNMeanSDMinimumMaximum DSEXStudent sex8,0141.480.501.002.00MRPCM1Plausible value 1 8,014258.9437.53130.75372.33 MRPCM2Plausible value 2 8,014259.0737.93127.82380.17 MRPCM3Plausible value 38,014259.2937.91110.48389.45MRPCM4Plausible value 4 8,014259.3337.65126.88380.38 MRPCM5Plausible value 5 8,014259.3037.74123.39389.55 MINORITYNon-Asian minorities8,0140.360.480.001.00PARHI_EDParent education 8,0140.540.500.001.00 District level 2 ASIERAverage state instruction expenditure rate17760.736.6950.0877.85 DDRDistrict Discretionary instruction spending Rate1770.315.20-13.5416.26 URBAN2Urban district 1770.300.450.001.00 RURALRural district 1770.150.350.001.00 ENROLL_KDistrict total enrollment in thousand 17743.9398.040.17962.27 BLACK_PBlack student rate17715.8224.580.0099.00P7118TPDistrict poverty rate 17715.9411.570.3068.70 PC30ETPDistrict at-risk student rate 1774.034.090.0023.60 C_STREVPDistrict revenue percentage from state 17742.4020.980.0084.60
9 of 39 CURPPE_KCurrent per pupil expenditure in $K 1774.921.342.9810.07 LEV2WTDistrict weight 1771.001.270.068.55 DDR_RULInteraction DDR by rural 177-0.592.40-13.546.42 DDR_URBInteraction DDR by urban 1770.382.59-11.1616.26 DDR_STPInteraction DDR by revenue percentage from state 1775.36266.05-1063.18757.29 DDR_SESInteraction DDR by poverty rate 177-3.26100.04-472.35306.73 DDR_RSKInteraction DDR by at-risk student rate 177-0.0527.16-162.2698.59Table 2-3 Descriptive statistics at studentand district-lev els: 1990 NAEP-CCD dataStudent level 1 Variable name Variable labelNMeanSDMinimumMaximum DSEXStudent sex6,2131.480.501.002.00MRPCM1Plausible value 16,213256.2433.43149.28370.23MRPCM2Plausible value 26,213256.2833.21138.77375.86MRPCM3Plausible value 36,213256.6133.11159.70367.46MRPCM4Plausible value 46,213256.1433.44139.74377.80MRPCM5Plausible value 56,213256.3233.17145.25352.46MINORITYNon-Asian minorities6,2130.320.470.001.00PARHI_EDParent education6,2130.520.500.001.00 District Level 2 ASIERAverage state instruction expenditure rate 14359.615.7149.1676.82 DDRDistrict Discretionary instruction spending Rate 1430.197.59-20.7022.36 URBAN2Urban district1430.260.440.001.00RURALRural district 1430.370.480.001.00 ENROLL_KDistrict total enrollment in thousand 14337.28103.390.09918.01 BLACK_PBlack student rate 14314.9220.480.0093.00 P7118TPDistrict poverty rate14319.5512.351.0060.40
10 of 39 PC30ETPDistrict at-risk student rate 1434.504.550.0025.30 C_STREVPDistrict revenue percentage from state 14348.4318.391.6082.40 CURPPE_KCurrent per pupil expenditure in $K 1434.311.152.418.69 LEV2WTDistrict weight 1430.991.710.018.44 DDR_RULInteraction DDR by rural 143-1.174.42-20.709.21 DDR_URBInteraction DDR by urban 1430.853.10-7.2415.37 DDR_STPInteraction DDR by revenue percentage from state 14310.84392.55-1,340.281,551.92 DDR_SESInteraction DDR by poverty rate 1432.70169.59-494.14673.09 DDR_RSKInteraction DDR by at-risk student rate 1433.0140.59-142.15153.75 Sample Modification and Overall WeightThe resulting student subsamples within districts m ight not have been reliable in presenting the student populations of the given districts, because schools, not districts, were a sampling stage in the original NAEP design. Therefore, we reweighted and poststratified the data to improve its statistical reliability. The purpose was to shift t he sampling stage from the school to the district i n order to examine differences across groups of stude nts who were hypothetically influenced by local districts' instructional.The modification of the NAEP school-student sample into a district-student sample entailed reweighting the original sample and establishing th e formal statistical status of the district-student sample. Within a sampled PSU, the NAEP school sampl e via "post-allocation" induced a district sample that included districts with which the sampl e schools were affiliated. With modification the representativeness of the school sample to PSUs wou ld lead to the representativeness of the district sample to PSUs. The procedure is highlighted below; see Appendix I for details. Student sample weightingWe assigned weights to each sample district within a PSU according to the district's post-inclusion probability, which was defined as the inclusion pro bability of the union of sample schools in the district. The calculation of this probability is de scribed in Appendix I. The post-inclusion probabilities were used as a reference scale for as signing the district weights. The weight assigned to a sample district was proportionate to the recip rocal of its post-inclusion probability. The students within a district were weighted by the reciprocal of the student sampling rate within the district. To calculate the weight we used the s tudent population size of the sample district at a given grade. The CCD school file provided student e nrollment size for each grade of the school, which was aggregated to the district level. We made a ratio adjustment (Deville & Sarndal 1992; Deville, Sarndal & Sautory 1993; Little 1993) of th e student weights within the districts via poststratification according to important geographi c/demographic features. We used this procedure to improve the representativeness of the student sa mples within districts to the district student
11 of 39populations.The PSU weights remained intact. The obtained stude nt weights were further adjusted at the national level in the same way as the poststratific ation conducted for the 1992 NAEP (see Wallace & Rust 1994, section 5.1.4). The resulting student sample preserved the goal of the original NAEP sampling design; namely, the targeted number of stu dents at the given grades for the assessment were selected at a uniform probability nationwide ( Wallace and Rust 1994, chapter 2). District sample weighting Multilevel linear modeling requires the use of leve l-2 unit (district) weights in analysis to assure that the level-2 sample represents the specified po pulation (Bryk & Raudenbush 1992). We weighted the district sample to make it represent t he national district population using the national district population information from CCD. To improv e the representativeness of the district sample calibration (ratio adjustment), poststratification was carried out along race and Census region. These demographic and geographic characteristics we re selected after a comparison of the sample and population distributions of those characteristi cs (Deville, Sarndal & Sautory 1993; Little 1993) (see Appendix II for the modified sampling distribu tion of race and Census region).Analytical ApproachBefore data analysis, we edited the data, examined missing data patterns, and constructed indicators to represent concepts to be analyzed (see Tables 2. 12.3 for descriptive statistics of the variables at the studentand district levels).Expenditure measuresDistrict instructional spending measures and other fiscal variables were made comparable across locations by adjusting for local cost of living and inflation. We used the Teacher Cost Index (TCI) for states and districts available from NCES for th e adjustment (National Center for Education Statistics 1995; also see Fowler & Monk 2001). TCI, an index of costs for hiring teachers, was developed through a regression analysis that estima ted the effects of multiple factors, including the cost of living and quality of life for each state a nd district. The TCI score for states were centered by the national average of 100 (National Center for Education Statistics 1995). We multiplied each state's total capital outlay per pupil by the TCI score for the given state and each district's by the district's TCI. Instructiona l expenditure rates at both district and state were calculated by dividing instructional expenditure pe r pupil with the adjusted total capital outlay per pupil. The resulting district rate was centered aro und the given state rate to generate the DDR in instructional expenditure. This TCI adjustment did not change the values of DDRs within a state, but it adjusted the difference in DDR for districts in different states. Rescaling dataWe recoded student race/ethnicity into a binary var iable (White/Asian vs. minorities) and parent's education into two binary variables (with some post secondary education vs. without). All per-pupil total expenditure measures were rescaled in thousan ds of dollars, total expenditure in millions of dollars, and total enrollment in thousands of stude nts. In HLM modeling, each of the district-level variables was further centered around the grand mea n, whereas each student variable was centered
12 of 39around the district mean.Missing dataWe examined nonresponse patterns to assure there we re no systematic missing data to bias statistics. Approximately 12 percent of the 1992 NA EP student records and 3 percent of the 1996 records had missing data on one or more district va riables due to unmatched records. There were no such missing data for 1990. We flagged the cases th at had no district fiscal data in each file with a single missing indicator. One indicator was suffici ent because missing data occurred quite consistently for most of the district fiscal measur es. We then imputed data with grand mean on each missing variable and examined the missingness in a series of single-level regular regression analyses that included all the predictor variables of math achievement (for rationale see Cohen & Huang 2000; Little & Rubin 1986). We found that the missing data were largely random as the regression coefficient for the missing flag was not statistically significant in the three years. Thus in the HLM analysis, we simply used the mean-impute d values for the missing data without using the missing flag.Two-level hierarchical modelingAfter reweighting the samples, we conducted univari ate and bivariate analyses to examine data quality. Single-level multiple regression analyses were run with conceptually important variables to explore the general pattern of relationships betwee n district variables and math achievement. We also examined the data to make sure that no obvious anomalies existed. The central research question was whether student a chievement was related to district discretionary rates in instructional spending (DDR). To address t his question with data for each specified year, the two-level modeling took the math composite plau sible values as the independent variables and DDRs as the predictor variables controlling for dis trictand student-level variables. See appendix III for a formal discussion of the HLM procedure. W e used software package HLM Windows 5.03, which has a component to run the repeated procedure with multiple plausible values and to average the estimated coefficients (see Bryk, Raudenbush & Congdon 1996). We conducted the two-level analysis in four steps. First, we examine the extent to which achievement varied across districts and the proport ion of variability attributed to student effects an d to district effects. With a one-way random effects ANOVA model, we separated the total variance of math achievement into betweenand within-district components. By assessing the intraclass correlation and reliability of district means, we d etermined that district-level variance was substantial and statistically significant for furth er modeling. Second, we estimated the relationships of a distric t's average math score to DDR and CEPP, adjusting for the effects of total enrollment size, minority student rate, poverty rate, state contribution to the district revenue, and geographi c locale. With a random-interception model, we examined the extent to which district average achie vement related to DDR and CEPP, controlling for the district-level variables specified above.Next, we explored the interaction effects between D DR and district factors that might have potential combined effect on achievement. We asked of the DDR effect on achievement differentiated by some district characteristics. Fo r example, DDR was probably more influential to achievement in disadvantaged districts; or, DDR mig ht affect achievement in districts that relied more on local revenue than on state contributions. We specifically estimated the effect of the
13 of 39cross-product terms of DDR and poverty, DDR and rur al-urban locale, and DDR and the state share in district's revenue.Finally, we tested three important student-level va riablessex, SES, and minority statusin a random coefficient regression model We assess the extent to which these three individ ual effects varied between districts. Given that these gaps did vary across districts, we then tested full models that included these student variables. The full mod el accounted for both district average achievement and district achievement gaps. We first determine whether DDR related to district average math score, adjusting for both districtan d student-level variables. Then, we asked whether achievement gaps due to sociodemographic background s related to DDR (or, whether DDR helped reduce math achievement gaps).Including student variables in the model was also m ethodologically important. Students were not randomly associated with districts; the district-le vel estimates might be biased if we did not control for student background effects. Second, sex, SES, a nd minority status at student level were established background factors that strongly relate d to achievement. Controlling for these variables could reduce unexplained level-1 error and thus imp rove the precision of the estimates of district spending rates as well as the power of hypothesis t ests (Bryk & Raudenbush 1992). Problems and limitationsIt is typically more difficult to model slopes than means with HLM techniques. Prior analyses of the NAEP data have reported unreliable estimation with slope equations (for example, Arnold 1995). While concentrated in intercept models, we did test slope models with limited number of variables at the two levels. The resulting slope estimates we re considerably less reliable than those of intercept models.FindingsData across the three years appeared reasonably com patible in univariate statistics. At the student level, sex (with male coded 1 female coded 2) and m inority (with the Asian and White coded zero, all the other groups coded 1) distributions were qu ite steady with data of the three years. Parents' educational levels seemed slightly rising. The rate of sample students whose parents had at least some college education was 52 percent in 1990, 54 p ercent in 1992, and 58 percent in 1996. The NAEP math achievement on average was higher in 1996 than in the other two years (approximately 268 in 1996, 259 in 1992, and 256 in 1990) with com patible standard deviations and ranges (see Tables 2-1 to 2-3).At the district level, the descriptive statistics w ere reasonably compatible across years as well. The average state instructional spending rate (ASIER) b arely differed over the years, ranging from 59.61 to 60.99 percent. DDR averaged very close to zero b ecause by definition the scores were centered by the state average. This measure's standard devia tion was similar across years though the range shifted slightly. CEPP on average rose from $4,310 in 1990 to $5,360 in 1996. The average proportion of district revenue that came from the s tate (C_STREVP) also seemed acceptable, with estimates ranging from 42.40 percent in 1992 to 48. 43 percent in 1990. These sample estimates from the NAEP-CCD combined data were quite consiste nt with the released national statistics (e.g., National Center for Education Statistics 1996). Lik ewise, district level demographic statistics (total enrollment, Black student rate, poverty rate, and a t-risk student rate) appeared reasonable for the three years. Tables 2.12.3 also present descriptiv e statistics for the interaction terms between DDR and district demographic variables.
14 of 39 A possible anomaly was with the geographic locale. While the rate of urban districts ranged 26 to 30 percent for the yearsagain reasonable estimates the rate of rural districts shifted somewhat excessively from 15 percent in 1996 to 37 percent i n 1990. This problem could be indicative of the unreliability of the merged NAEP and CCD data.Bivariate StatisticsA number of patterns revealed in the bivariate corr elation analysis must be noted. First, fiscal measures at district level were correlated, often i n large magnitude. The total expenditure and current expenditure, for example, had a correlation coefficient around 0.98 in all three years (the full matrix of the correlation coefficients is avai lable upon request to the first author). Per pupil spending items (for example, CEPP, per pupil total expenditure, and per pupil instruction expenditure) were correlated to each other. These m easures were substantially correlated with the total expenditure measures, albeit to a moderate ex tent. Expenditure measures were also closely related to enrollment size. Table 3 shows the estim ates of bivariate correlation among the selected district-level variables.Note that even in this selected group, there were p airs of variables that were well correlated. Proportion of district revenue from the state was s trongly associated with CEPP (0.68 in 1996, 0.70 in 1992, and 0.66 in 1990). Poverty rate and at-ris k student rate were also strongly related (0.62, 0.75, and 0.40, respectively, for the three years). We excluded at-risk student rate from the equation because it presumably overlapped with poverty rate to indicate the disadvantaged condition of a district, yet its range was smaller than the povert y rate. For similar reasons, we dropped rural local e and retained urban locale to make the model parsimo nious. (Note 2) The high multicollinearity required model specifica tion with high selectivity of independent variables at the district level such that the inclu ded variables were not highly correlated to each other and adequately represented our conceptual mod el. After testing OLS regression analyses with SAS and an initial run of two-level models with HLM we decided to specify in the final model the following independent variables at the district lev el: District discretionary rate in instructional spendi ng, State average instructional spending rate, CEPP (in thousands of dollars), Proportion of revenue from the state, Total enrollment (in thousands of students), Black student enrollment rate, Child poverty rate, and Urban locale (in contrast with suburban). Table 3 Correlation coefficients of the selected district l evel variables: NEAP-CCD district level data 1996 1992 and 1990 (weighted wi th district weight)1996 (N=160) State average instruction spending rate(1)(2)(3)(4)(5)(6)(7)(8)
15 of 39 DDS (1)*-0.19Total enrollment (thousand) (2)-0.05*0.26Black student rate (3)*-0.300.030.15Current expenditure per pupil in $k (4)*0.68-0.12-0 .03-0.15 Poverty rate (5)*-0.28*-0.220.000.15-0.11At-risk student rate (6)*-0.340.020.05*0.28-0.14*0. 62 Percent of revenue from state (7)*-0.29*-0.19-0.140 .07*-0.240.120.08 Urban district (8)-0.06*0.22*0.24*0.230.020.02*0.18 -0.10 Rural district (9)0.04*-0.50-0.16*-0.18-0.080.10-0. 16*0.29*-0.31 1992 (N=177)DDS (1)*0.16Total enrollment (thousand) (2)-0.03*0.18Black student rate (3)0.070.08-0.10Current expenditure per pupil in $k (4)*0.70-0.01-0 .06-0.02 Poverty rate (5)*-0.30*-0.28-0.07-0.10*-0.20At-risk student rate (6)*-0.31-0.13-0.10-0.12*-0.21 0.75 Percent of revenue from state (7)*-0.27*-0.180.06-0 .14*-0.40*0.35*0.38 Urban district (8)0.000.11*0.32*-0.19-0.04*0.160.10 0.08 Rural district (9)*-0.28*-0.47-0.11-0.02*-0.17*0.16 0.090.13*-0.22 1990 (N=143)DDS (1)*-0.34Total enrollment (thousand) (2)0.000.14Black student rate (3)*-0.24*0.270.14Current expenditure per pupil in $k (4)*0.66*-0.250 .03*-0.20
16 of 39 Poverty rate (5)*-0.430.030.02*0.32*-0.29At-risk student rate (6)*-0.290.110.080.55**-0.30*0 .40 Percent of revenue from state (7)-0.120.09-0.040.12 -0.05*0.310.01 Urban district (8)-0.010.12*0.340.140.080.020.08-0. 13 Rural district (9)*-0.24-0.15-0.130.05*-0.17*0.41*0 .17*0.32*-0.21* p < 0.05HLM Modeling ResultsUnconditional models were tested with each year's d ata. Without any independent variables, the models separately estimated the variance of the stu dent math achievement at individual and district levels. Table 4 presents the estimates of the uncon ditional models. For each year, a substantial proportion of variance occurred at the district lev el, meaning that districts differed in their averag e math achievement. For 1996, 26 percent of the varia nce was at the district level, as indicated by the intraclass correlation coefficient. A chi-square st atistic of 2401 with 159 degrees of freedom was highly significant for the district variance estima te. The reliability of school means (0.71) was also acceptable for further HLM analysis, as these sampl ed school means on average represented the true school means reasonably well. (Note 3) The two-level variance distribution pattern was si milar in the 1992 and 1990 data. These statistics strongl y suggested that HLM modeling with random effect at the both levels was necessary for analyzi ng student achievement in relation to district fisc al variables.Table 4 Distribution of variance at the student and distric t levels: Unconditional models with the 1996, 1992, and 1990 NAEP-CCD dataEstimates 1996 NAEP-CCD 1992 NAEP-CCD 1990 NAEP-CCD Fixed effectCoefficientAverage district mean 00(standard error) 270.06(1.47) 260.39(1.66) 256.29(1.65) Random effectDistrict level variance u0jdf 245.00159 373.39176 262.38143
17 of 39 chi-squarep value 2401.180.000 2950.850.000 2950.490.000 Student level variance component rij975.561160.06681.59 Intraclass correlation rho0.200.240.28Reliability of the district meanj0.710.780.71 District average math achievement in relation to DD R Using district variables to explain district averag e math achievement, we tested a series of means-as-outcomes models. At each level, the model specified a random effect. Fixed effects, however, were only specified at the district level, including state average instructional spending rate, DDR, total enrollment in thousands of student s, Black student enrollment rate, CEPP in $1000, child poverty rate, at-risk student rate, ur ban and rural locale (both coded in contrast to suburban), and percent of revenue from the state. T he resulting estimates are presented in Table 5.1. Entering district-level independent variables in th e model helped account for approximately a half of the district-level variance (49 percent in 1996, 62 percent in 1992, and 53 percent in 1990). Controlling for other variables, several district v ariables were found related to district average mat h achievement in one or more years. District total en rollment was related to low average math achievement in 1992, but not in 1990 and 1996. The minority enrollment rate was strongly related to low average achievement in all three years, with high statistical significance. The child poverty rate was related to low achievement mean, with a si gnificant estimate for each year. These findings were compatible with prior research in school organ ization and achievement. The effect of DDR was not substantiated with data f or the three years. Net of the effects of other district variables, DDR was not statistically signi ficantly related to district mean math achievement in any year. The estimate in the three years was es sentially zero since it was trivial in size and not statistically significant. CEPP, on the other hand, was related to high average achievement, although the estimate for 1992 was not statisticall y significant (see Table 5). With the NAEP-CCD combined data for the three years we failed to find evidence to support our central hypothesis that high district instructional spending relative to the state average would increase district average performance level, holdin g other things constant.Table 5 District average achievement and DDR and other dist rict level variables: Means-as-outcomes model (standard error in parentheses)Estimates 1996aNAEP-CCD 1992 NAEP-CCD 1990 NAEP-CCD
18 of 39 Fixed effectsDistrict mean 00268.53 (1.18)259.23 (1.19)254.86 (1.28) State average instructional spendingrate 01-0.21 (0.24)-0.19 (0.28)-0.64 (0.31) District discretionary rate ofinstructional spending (DDR) 02-0.21 (0.17)0.42 (0.26)-0.20 (0.16) Total enrollment in thousands 03-0.01 (0.02)*-0.03 (0.01)-0.01 (0.01) Minority enrollment rate 04**-0.45 (0.06)**-0.54 (0.07)**-0.36 (0.09) CEPP $1000 05**2.74 (1.06)1.49 (1.13)**4.16 (1.59) Child poverty rate 07*-0.21 (0.11)*-0.32 (0.14)**-0.59 (0.13) Urban district 08-0.82 (3.82)-1.27 (3.01)1.10 (3.50) State revenue percentage 010-0.03 (0.05)*-0.16 (0.06)-0.09 (0.06) Random effectsDistrict mean u0j 125.21150.41121.57 chi-square1303.421288.031420.38District level variance explained0.490.620.53Student level variance974.561158.68671.91 p < 0.05 ** p < 0.01a A missing value flag was included in the model but not presented because its coefficient was not statistically significant.Joint effects between DDR and other district variab les Exploring the possible joint effects between fiscal and demographic variables at the district level, we tested a number of interaction t erms and presented selected estimates from the model (see Table 6). We found no evidence of a joint effect between DDR and the state contribution to the district revenue in r elation to math achievement. The three years' estimates were all trivial and not statistic ally significant. In other words, the state funding for local districts and districts' autonomy in instructional spending did not jointly influence achievement in some peculiar way as we mi ght suspect.Table 6 District average achievement accounted for by DDR a nd other district-level variables and interaction terms: Mea ns-as-outcomes model (standard error in parentheses)
19 of 39 Estimates 1996aNAEP-CCD 1992 NAEP-CCD 1990 NAEP-CCD Fixed effectsDistrict mean 00268.72 (1.22)259.17 (1.31)254.95 (1.17) State average instructional spendingrate 01-0.26 (0.23)-0.15 (0.27)-0.44 (0.30) District discretionary rate ofinstructional spending (DDR) 02*-1.03 (0.52)0.87 (0.65)0.41 (0.35) Total enrollment in thousand 03-0.02 (0.01)*-0.03 (0.01)-0.02 (0.01) Minority enrollment rate 04**-0.47 (0.06)**-0.53 (0.08)**-0.36 (0.10) CEPP $1000 05**2.89 (1.05)1.57 (1.13)*3.72 (1.58) Child poverty rate 06-0.10 (0.13)*-0.41 (0.16)**-0.58 (0.15) State revenue percentage 07-0.02 (0.06)*-0.15 (0.06)-0.09 (0.06) Urban district 08-1.64 (3.69)-0.11 (3.06)-0.27 (2.74) Interaction termsDDR x State revenue percentage 090.01 (0.01)0.00 (0.01)-0.00 (0.00) DDR x Urban 010-0.36 (0.51)0.25 (0.63)**1.47 (0.41) DDR x Child poverty 011*0.03 (0.01)-0.05 (0.03)-0.02 (0.01) Random effects District mean u0j 125.47146.52116.91 chi-square1247.821306.151284.82District level variance explained0.490.640.55Student level variance974.181158.68671.91 p < 0.05 ** p < 0.01a A missing value flag was included in the model but not presented because its coefficient was not statistically significant.The combined effect of DDR and urban locale seemed more complicated. It was substantial and statistically significant with the 1990 data (the fixed effect estimate 010 = 1.47 with p < 0.001), but virtually nil with the 19 92 and 1996 data. The positive 1990 estimate implied that higher spending on instructio n in a district (relative to the state average) was related to higher average achievement for urban districts and the effect size was fairly large. The finding, were it substantiate d with multiple years' data, would offer important policy implications.
20 of 39 Another interaction term was between DDR and the ch ild poverty rate, which also resulted in inconsistent estimates across years. On ly in 1996 was the estimate meaningful (0.03 at p < 0.05 level), implying that, among high poverty districts, higher instructional spending than the state average was related to slig ht better average math achievement; but among low poverty districts, there was no such rela tionship. Again, this finding would be potentially important should it be confirmed. It wa s not, however, substantiated with data for 1992 and 1990. In fact, the effect disappeared even with the 1996 data in subsequent analysis where additional variables were statistica lly controlled for (see Table 8.1). Though the findings on combined effects between dis trict discretionary instructional spending were highly tentative, they posed question s for further studies. Note that the DDR estimate, again, was trivial afte r the interaction terms entered into the equation, except for 1996, when the estimate was si gnificant at p < 0.05 level, modest, and negative (1.03). This exception, as we see, should not ch ange the overall finding that DDR was largely unrelated to 8th graders' math achievement. On the other hand, net of the additional interaction effects, CEPP estimat e remained largely substantial and significant for 1996 and 1990, indicating a positiv e relationship with math achievement. The estimate was not statistically significant for 1992. Math achievement gaps associated with sex, race/eth nicity, and SES To examine the relationships of DDR to math achieve ment gaps associated with social and demographic categories, it was necessary first to assess the magnitude of the gaps and their variance; that is, how much the gaps vari ed across districts. We specified a random-coefficient regression model wherein individ ual student sex, race/ethnicity, and parental educational attainment were estimated as b oth fixed and random effects. The estimates are presented in Table 7.Table 7 Math achievement differences relating to sex race/e thnicity and parents educational attainment: Random coefficient regression model (standard error in parentheses)Estimates 1996aNAEP-CCD 1992 NAEP-CCD 1990 NAEP-CCD Fixed effects coefficientdistrict mean 00269.77 (1.46)260.20 (1.65)256.05 (1.62) sex difference 100.01 (1.26)0.78 (1.26)0.57 (1.30) Race-ethnic difference 20**-23.33 (2.15)**-24.09 (2.00) **-20.16 (1.90) Parents' educational attainmentdifference 30**16.75 (1.45)**15.38 (1.38)**17.89 (1.31) Random effects **256.42**383.49**267.97
21 of 39 District mean u0j(df; chi-square) (136; 2557.89)(153; 3106.93)(117; 2853.21) sex difference u1j(df; chi-square) **82.69(136; 212.05) **80.35(153; 220.76) **91.67(117; 195.57) Race-ethnic difference u2j(df; chi-square) **168.08(136; 202.07) **191.28(153; 233.29) **105.77(117; 238.53) Parents educational attainmentdifference u3j (df; chi-square) **69.07(136; 219.91) **44.75(153; 210.52) **68.04(117; 227.77) Student level variance778.71962.97530.74Correlation among district effectsDistrict mean t00Sex difference t11Race/ethnicity difference t22Parents education difference t33t00 t11 t221.00-0.22 0.25 -0.14 0.16 -0.180.44 t00 t11 t221.00-0.01-0.17 -0.330.21 -0.29 0.23 t00 t11 t221.00-0.030.02 -0.030.11 -0.22 -0.11 Reliability of regression coefficient estimatesDistrict mean0.730.790.71Sex difference0.310.330.34Race/ethnic difference0.310.320.25Parents education difference0.260.230.28 p < 0.05 ** p < 0.01a A missing value flag was included in the model but not presented because its coefficient was not statistically significant.There was no evidence from each year's data that 8t h grade math achievement differed by sex within districts as the fixed effect coefficient was close to zero and was not statistically significant. However, the random vari ation associated with sex was substantial and statistically significant across th e three years (e.g., for 1996 u1j = 82.69, df = 136, c2 = 212.05), suggesting that sex difference in achie vement varied considerably across districts around a mean of zero. This situat ion was similar to findings from some earlier studies (e.g., Raudenbush, Kidchanapanish, & Kang 1991). Not only the magnitude of the gap, but also the direction of the association, could differ across
22 of 39districts. It is possible to sort out factors that were responsible for the variance in future analysis. However, limited by the scope of this stu dy, we did not include sex as an achievement gap in the subsequent modeling.Race/ethnicity was clearly a strong predictor of ma th achievement for all three years. The fixed effect estimate 20 was large (-23.33 for 1996, -24.09 for 1992, and 20.16 for 1990) and highly significant (p < 0.001 each year), confirming that minority students (other than Asian American) on average tended to ac hieve low on the NAEP math test within districts. The random effect for race/ethnic ity was substantial and significant, revealing that districts differed in the racial/eth nic gap and that the variance needed further explanation.As a socioeconomic status indicator, parents' educa tional attainment was specified in the model. The fixed effect estimate 30 was positive and large for each year (16.75, 15.38 and 17.89 respectively for 1996, 1992, and 1990, al l significant at p < 0.01 level). The estimate suggested that within districts, on averag e, students whose parents had college or more education performed better on the NAEP math te st. The estimate of the random effect was also substantial and highly significant over the years, revealing that districts differed in the math achievement gap associated wit h family social background. Obviously, the varying achievement gap relating to parents' education merited further modeling.Because of the specified student-level variables in the model, student-level variance became considerably small relative to the estimates from the prior means-as-outcomes model (778.71, 962.97 and 530.74 in Table 7, compar ed with the 974.56, 1158.68, and 681.08 shown in Table 5, respectively, for the thre e years). The covariance matrixes generated from the model wi th the three years' data provided information about the correlation between districtlevel residual random effects (Table 7, panel 3). One of the consistent findings for all th e years was that the district average achievement residual variance positively and modera tely related to the residual variance of parents' education-associated achievement gap (0 .16, 0.21, and 0.11). This finding implied that, holding other things constant, the hi gher the district average math test scores, the wider the district-level achievement ga p between students whose parents had higher education and students whose parents did not Another consistent estimate was the correlation between sex gap and parents' education gap (-0.18, -0.29, and 0.22 for the three years). These estimates suggested that, to so me modest extent, the wider a district's sex difference in math achievement, the smaller the district's math achievement gap relating to parents' education. The remaining estim ates of correlation changed substantially across years and it was impossible to interpret them without further analysis. A final note for the random coefficient model: For each year, while the reliability of the intercept (district average math achievement) was r easonably high, the reliability measures for slopes (differences associated with se x, race/ethnicity, and parental education) were fairly low. This finding pointed to a frequently encountered problem in HLM analysis of survey data; that is, the difficult y to model the slopes (see Bryk & Raudenbush, 1992, pp. 102103). Additional analysis was needed to sort out whether the lower reliability for slope statistics was due to the NAEP data collection design, small within-district sample, real differences across dis tricts in the statistics, or simply the error variance in the slope estimation.
23 of 39 Final modelTo control for variables at the student and distric t levels in examining the relationships of DDR and achievement, we integrated the previous mod eling into a full HLM model, with both means and slopes as outcomes in the regression analysis. The resulting estimates of fixed effects and random effects are presented sepa rately in Tables 8.1 and 8.2. The findings were consistent with those discussed earli er. Accounting for district's average math achievement, we did not observe any effect by DDR. However, CEPP and a number of district factors did relate to the achievement m ean, as interpreted below.Table 8-1 District mean, racial/ethnic gap, and SES gap in ac hievement accounted for by DDR and other district variables: Fixed effe cts (standard error in parentheses)Estimates 1996 NEAP-CCDa1992 NAEP-CCD 1990 NAEP-CCD District average mean 00268.64(1.15)259.12(1.17)254.89(1.22) State average instructionalspending rate 01-0.27(0.23)-0.16(0.27)-0.44(0.30) Total enrollment in thousand02-0.01(0.02)-0.03(0.02)-0.02(0.01) Minority enrollment rate 03-0.47**(0.06)-0.53**(0.08)-0.36**(0.10) CEPP in $K 042.85**(1.07)1.60(1.14)3.69*(1.56) Child poverty rate 05-0.09(0.13)-0.40*(0.17)-0.58**(0.15) Percent of revenue from state06-0.03(0.06)-0.17**(0.06)-0.09(0.06) Urban locale 07-1.97(3.61)-0.27(3.02)-0.54(2.79) DDR 08-1.02(0.52)0.52(0.64)0.38(0.35) DDR*urban 09-0.36(0.49)0.20(0.64)1.48**(0.43) DDR*state revenue percentage0100.01(0.01)0.01(0.01)0.00(0.01) DDR*poverty rate 0110.03(0.02)-0.05(0.03)-0.02(0.02) Race/ethnicity gap 10-23.64**(1.76)-24.35**(1.68)-21.55**(1.59) Average state instructionalspending rate 110.52(0.38)0.56(0.37)-0.24(0.55) Total enrollment in thousands120.00(0.01)-0.01(0.01)-0.03*(0.01) Minority enrollment rate 13-0.13(0.10)-0.07(0.11)0.01(0.15) CEPP in $K 140.71(1.74)0.99(2.28)4.08(2.47)
24 of 39 Child poverty rate 150.04(0.18)0.38*(0.17)-0.08(0.23) Urban locale 160.57(3.53)-4.08(3.91)-7.37(4.39) DDR 170.13(0.40)-0.16(0.54)0.43(0.36) SES (parent education) gap 2015.95**(1.22)14.81**(1.15)17.13**(1.12) Average state instructionalspending rate 21-0.24(0.25)0.00(0.21)-0.05(0.32) Total enrollment in thousands220.01(0.01)0.00(0.02)0.01(0.01) Minority enrollment rate 23-0.08(0.06)-0.05(0.05)-0.07(0.09) CEPP in $K 240.35(1.04)1.63(1.14)-0.47(1.70) Child poverty rate 25-0.31**(0.12)-0.10(0.09)-0.03(0.15) Urban locale 26-1.18(2.76)0.46(2.60)-1.03(2.27) DDR 27-0.42(0.25)-0.01(0.21)0.00(0.18) p < 0.05 ** p< 0.01a A missing value flag was included in the model but not presented because its coefficient was not statistically significant.Table 8-2 District mean, racial-ethnic gap, and SES gap in ac hievement accounted for by DDR and other district variables: Random effects (standard error in parentheses)Estimates 1996aNAEP-CCD 1992 NAEP-CCD 1990 NAEP-CCD Random effectsDistrict mean u0jdfchi-squarevariance explained:relative to means-as-outcomesmodelrelative to random coefficient regression model 132.181251325.27-0.05b0.48 151.911421341.32b-0.03 0.61 119.941051259.89b-0.02 0.55 Racial/ethnic gap u1jchi-square 152.28187.61 165.78212.61 81.67209.58
25 of 39 variance explained:relative to random coefficient regression model 0.100.140.23 SES gap u1jchi-squarevariance explained:relative to random coefficient regression model 60.80204.700.13 42.14199.030.05 64.82206.780.06 Student level variance804.03988.08545.27 p < 0.05 ** p< 0.01a A missing value flag was included in the model but not presented because its coefficient was not statistically significant.b The estimate implies that the final model accounte d for a smaller portion of the variance of this variable.Fixed effects: The fixed effect estimate for DDR in 1996 was 08 = -1.02, not statistically significant (p = 0.051, see Table 8.1). The other y ears' estimates were close to zero and statistically insignificant as well. This finding, again, showed no evidence that district discretionary instructional spending relative to th e state average spending would lead to higher average math achievement.The minority enrollment rate was a consistently str ong negative predictor of districts' mean math achievement, with estimates of -0.47, -0. 53, and -0.36 for the three years, all highly significant. CEPP was quite consistently est imated as a positive predictor of district achievement level, with estimates of 2.85 in 1996, 1.60 in 1992, and 3.69 in 1990, and statistically significant in 1996 and 1990. The child poverty rate was another predictor with fairly consistent estimates over the years (-0.09 for 1996, -0.40 for 1992, and -0.58 for 1990), and the statistic was signific ant in 1992 and 1990. Note that the state's share of revenue for the district yielded a statistically significant estimate only for 1992 (-0.17, p < 0.01). This suggested that holding other conditions constant, districts that received a higher portion of state contributio n tended to achieve somewhat lower in the NAEP test in that year, probably due to the fac t that states' contribution was typically prioritized for low-achieving local schools.Estimates of interaction effects from the full mode l confirmed those from the previous modeling. For one, the interaction between DDR and urban locale was quite strong and statistically significant for 1990 (1.48, at p < 0. 01 level). DDR seemed to connect to the somewhat higher average math score of urban distric ts relative to districts in suburban or rural areas. The 1996 and 1992 data, however, faile d to produce consistent estimates and therefore this finding is tentative as well. We not e that the estimate for the DDR-poverty interaction effect was no longer found to be differ ent from zero for 1996 data with the
26 of 39full model.Gaps associated with race/ethnicity and parents' ed ucation were substantiated with the full model. Controlling for both within and between district variables, both race/ethnicity and parents' education had sizable fixed effect on mean achievement across the three years. For race/ethnicity, the estimate was -23.64 in 1996, -24.35 in 1992, and -21.55 in 1990, significant at p < 0.01 level. Two variables showed some unstable effect on the gap, but only for one year. One was district total enrollment, related to a trivially narrowed gap (-0.03 at p < 0.05 level) for 1990. Th e child poverty rate, on the other hand, related to a smaller racial gap (0.38 at p < 0.05 l evel) for 1992, suggesting that in districts of high poverty, minority students' achievement was slightly closer to White and Asian students' than it was in districts of low poverty. Again, without across-year consistent estimates, this interpretation requires additional analyses to confirm. Accounting for the gap associated with SES as indic ated by parents' educational attainment, none of the fixed estimates were statis tically significant, except that of the child poverty rate(-0.31 at p < 0.01 level) estimated with 1996 data. This effect may be interpreted in a similar way to that for the poverty rate and race/e thnicity gap. It may imply that for students who attended high poverty districts, paren ts' education made a smaller difference in their math achievement than it did for students in other districts. Poverty at the district level thus seemed to "dampen" the racial and socioe conomic differences at the student level. We may speculate that this finding perhaps h ints of narrow ranges of both student background measures and achievement measures in a h igh poverty district. In other words, poverty-related social homogeneity might und erlie the reduced achievement gaps. Without consistent results across years, such a "da mpening" hypothesis about district-level poverty on the individual achievemen t gaps awaits additional analysis to substantiate.Random effects:The final model's goodness of fit estimates were qu ite comparable for the three years (see Table 8.2). In general, the model has been improved in regard to accounting for the slope but not for the district mean. For example, between -district variance as estimated in the final model was actually smaller than it was estima ted in the means-as-the-outcome model (see Table 6), which contained the same distr ict-level variables as did the final model. This pattern, similar across the three years may suggest that the multiple predictor variables of slopes caused the model to fit less we ll as most added variables were not related to the slopes as expected. Nevertheless, th e variance of the two slopes was substantially reduced. Relative to the random-coeff icient regression model which contained only random effect for the slopes, the fi nal estimates showed that the slope equations fit better.ConclusionsOur findings did not support the hypothesis that dr ove this study, namely, that local districts could improve academic performance by inc reasing instructional spending. Net of district factors known to affect student achieve ment, school district discretionary spending in instruction, defined as the difference between the district instructional spending rate and the average instructional spendin g rate in a state, did not relate to a
27 of 39district's 8th grade average math performance. The null effect was consistent in the analysis of the combined NAEP-CCD data for 1990, 19 92, and 1996. On the other hand, fairly consistently, the analysis has found that so me district characteristics were related to average achievement. Specifically, other conditions being equal, a district's current expenditure per pupil (CEPP) was found related to h igher math performance in a modest yet consistent way. Demographic attributes at the d istrict level, including the minority student enrollment rate and the child poverty rate, were strongly related to lower math achievement, net of other effects.The finding of the null effect of local districts' control over instructional spending as indicated by DDR may lead to different directions f or future research on the district role in reform. First, considering the complex procedure s of local decisionmaking on educational financing, research needs to deepen the inquiry about districts' fiscal policymaking, especially the changing patterns of i nstructional spending in connection to other capital outlays and revenue. Variation in ins tructional spending might reflect distinctive school conditions and policy concerns o ther than academic performance. For example, new technologies and equipment often deman d a large budget for upgrading and staff training; school security is a pressing i ssue calling for increasing funding in some areas; aging buildings incur extremely high co sts to renovate. Many continuing or lump sum expenditures may cause instructional spend ing to fluctuate considerably, which may or may not have an immediate impact on academic test results. On the revenue side, federal or state funding and related requirements m ay vary depending on changes in the government and the legislation as well as economic conditions in a larger context. Investigators must consider specific motives underl ying decisions on spending and the broad conditions leading to such motives, together with DDR in order to understand local fiscal policymaking in relation to academic perform ance. The finding that CEPP was positively related to mat h achievement leads to questions about the value of instructional spending as a key indicator of district fiscal control in explaining student achievement. Instructional spend ing is probably a concept too narrow to capture the complexities involved in a district' s fiscal decisionmaking. In contrast, current spending adjusted to student enrollment cou ld be more predictive of student performance as it appears encompassing such issues as diverse district and community context and various needs for funding. Therefore, f uture research may test alternative indicators of district fiscal policy control in con nection to student academic performance and other outcomes. CEPP definitely should be a use ful candidate for such research. Furthermore, we recognize that the district has lim ited instructional spending control as the range of DDR was quite narrow (over the years, approximately 20 percent below or above the state average, see Tables 2.12.3). Such control may have little impactrelative to district policymaking in other r espectson student achievement. Hence, we may want to shift research attention to o ther aspects of district policymaking and operation. For example, research may focus on d istrict differences in handling curriculum and instructional standard development; teacher recruitment, training, participation in decisionmaking, and accountability ; community involvement and support; effectiveness in managing technological up grading, school and class size, facilities maintenance, student services, and other administrative roles. This analysis did offer some interesting albeit uns ettled clues for learning about how district discretionary instructional spending might function jointly with other variables to influence achievement. The 1990 data showed that hi gh DDR in urban districts was
28 of 39related to substantially higher average math perfor mance, although the relationship was not evident in analysis of the 1996 and 1992 data. On the other hand, with the 1996 data, higher DDR in poor districts appeared to relate to slightly higher math achievement than it did in other districts. Limited reliability of t he reweighted NAEP-CCD data is a likely culprit for the inconsistence. But unknown or shift ing circumstances wherein instructional spending interacted with local condit ions may also have contributed to such inconsistent patterns. It may be promising to expan d the research to conceptualize and examine such joint effects between local instructio nal spending and local conditions. It is possible that, with more reliable data and precise measures, future inquiries could identify this sort of "equalizing" effect on district instru ctional spending. Another intriguing yet uncertain issue that emerged in the analysis was sex difference in math achievement. Within districts, sex difference in 8th grade math achievement was estimated to be virtually zero with the three years data. But between districts, the variance of sex effect was estimated high and stati stically significant for all of the three years. Factors at the institution levelincluding s chool and districtshould be part of the explanation. However, classroom instructional p ractice and other organizational environmental influences are probably more salient relative to fiscal policy in explaining the between-district variance of sex difference. Ag ain, this question awaits future investigation.Combining NAEP and CCD data into a synthetic analys is proved to be a challenging task. A series of difficulties emerged in the proce ss of data merging and data editing because of changes over the years in both data coll ections (for a detailed discussion, see the Technical Report). In addition to addressing te chnical problems, future research would call for re-conceptualizing district fiscal m anagement processes, examining district-state interaction regarding education fina ncing, pinpointing joint effects of fiscal policymaking and local/regional conditions, and imp roving the reliability of the measurement and data quality of district fiscal sta tus. Specifically, it may be fruitful to separately stud y individual states and to integrate the findings onto the national level. This approach wou ld require using data from NAEP State Tests and CCD for individual state analyses i n a much more extensive scale than that attempted in the current analysis. It would al so entail thoughtful interpretation and reconciliation of the possibly incongruent findings from the state analyses to generate cogent narratives about education financing at the state and district levels. Such narratives would probably involve in-depth qualitative analyse s with information sources other than the survey data. A study of this sort would be a la rge-scale, challenging undertaking. But it appears more likely to produce comprehensive and in-depth knowledge about the local district's role in performance-driven reform, parti cularly its fiscal control in connection to student achievement.AcknowledgmentWe are grateful to Keith Rust and his colleagues at Westat for advice and assistance on methodological and technical issues ranging from th e NAEP sample modification to the data file merge. We also want to thank Robert Alfre d and other ETS staff who assisted us in dealing with the NAEP and CCD file merge. A numb er of Synectics colleagues have contributed to this project: Maxime Bokossa, who re viewed our sample modification procedures; Hongwei Zhang, who helped manage data f iles and data editing; and Elizabeth Walter, who edited the manuscript of the report. We wish to thank Synectics management, especially Jeff Whitesell and Sameena S alvucci, for their continued support
29 of 39and managerial flexibility. Our thanks also go to D eborah Sedlacek and Steven Gorman at NCES for their understanding and patience regard ing this exploratory study. DisclaimerThis research is supported by a grant to Synectics for Management Decisions, Inc., from the NAEP Secondary Analysis program (Award No. R902 B000002), U.S. Department of Education, National Center for Education Statistics (NCES). The authors assume full responsibility for the content of the report. Neith er NCES nor Synectics necessarily approve the viewpoints expressed in the report.Notes 1. In fact, the two studies came up with quite inconsi stent results regarding the relationship between central administration per pup il expenditure and math achievement. The aggregated study indicated a positive strong re lationship between the two variables (Wenglinsky 1997 pp. 230231). The HLM study found no relation at all between the two measures; rather it found per pupil instruction al expenditure to be associated with a smaller socioeconomic status (SES)-related achievem ent gap (Wenglinsky 1998 p. 276). Such inconclusive results call for further research to clarify both the methodological and the substantive issues involved in synthetic analys is of NAEP and CCD data. 2. However, recognizing that rural conditions differ f rom urban districts, we plan to specifically examine the rural-urban difference in district spending patterns and achievement during another project. 3. With the original NAEP data, the reliability for sc hool means is substantially higher, around 0.90 for the three years. The lower reliabil ity was apparently a result of sample modification.ReferencesAdmundson, K. J. (1993). Restructuring Reform and Reality: What School Distr icts Are Really Doing. NSBA Best Practices Series 1993 Alexandria VA: National School Boards Association ERIC Document No. ED360699.Allen, N. L., Kline, D. L., & Zelenak, C. A. (1997) The NAEP 1994 Technical Report. NCES 97-897. Washington DC: National Center for Edu cation Statistics. Arnold, C. L. (1995). Using HLM and NAEP Data to Explore School Correlate s of 1990 Mathematics and Geometry Achievement in Grades 4 8 12: Methodology and Results Washington DC: National Center for Education Statis tics. Brent, B. O., Roellke, C. F., & Monk, D. H. (1997). Understanding Teacher Resource Allocation in New York State Secondary Schools: A C ase Study Approach. Journal of Education Finance 23 2 207-33. Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchic al Linear Models Applications and Data Analysis Methods. Newbury Park CA: Sage Public ations. Bryk, A. S., Raudenbush, S. W., & Congdon, R. T. (1 996). Hierarchical Linear and Nonlinear Modeling with the HLM/2L and HLM/3L Progr ams Chicago IL: Scientific
30 of 39Software International.Chambers, J., & Fowler, W. J., Jr. (1995). Public School Teacher Cost Differences Across the United States: An Analysis to Develop a Teacher Cost Index (TCI). Washington DC: National Center for Education Statis tics. Childs, T. S., & Shakeshaft, C. (1986). A Meta-Anal ysis of Research on the Relationship between Educational Expenditures and Student Achiev ement. Journal of Education Finance 12 2 249-63. Clune, W. H. (1993). Systemic educational policy: a conceptual framework. In Furhman S. H. (Ed.). Designing coherent education policy San Francisco CA: Jossey-Bass Inc. Cochran, W. G. (1977). Sampling Techniques third edition. New York: John Wiley. Cohen, M. P., & Huang, G.G. (2000). Analyzing Survey Data by Complete-Case and Available-Case Methods: Alternatives to Imputation Paper presented at the American Statistical Association Annual Conference Indianapo lis IN. Conn, W. L. (1995). Funding fundamentals: the cost/ quality debate in school reform. West's Education Law Quarterly 4 1 123-139. Deville, J. C., & Sarndal, C. E. (1992). Calibratio n Estimators in Survey Sampling. Journal of the American Statistical Association 87 376-382. Deville, J. C., Sarndal, C. E., & Sautory, O. (1993 ). Generalized Raking Procedures in Survey Sampling. Journal of the American Statistical Association 88 1013-1020. DeYoung, A. (1990). Introduction. In A. DeYoung (Ed .). Rural education: Issues and practice. New York: Garland Publishing Inc. Doyle, D., & Finn, C. (1984). American schools and the future of local control. Public Interest 77 77-95. Duncombe, W., & Yinger, J. (1998). School Finance R eform: Aid Formulas and Equity Objectives. National Tax Journal 51 2 239-62. Elmore, R. F. (1994). Thoughts on program equity: p roductivity and incentives for performance in education. Educational Policy 8 4 453-459. Elmore, R. F. (1993). The role of local school dist ricts in instructional improvement. Furhman S. H. (Ed.). Designing coherent education policy San Francisco CA: Jossey-Bass Inc. Fay, R. E. (1995). VPLX Washington DC: U.S. Bureau of the Census. Ferguson, R. F. (1991). Racial Patterns in How Scho ol and Teacher Quality Affect Achievement and Earnings. Challenge 2 1 1-35. Forsyth, R., Hambleton, R., Linn, R., Mislevy, R., & Yen, W. (1996). Design/Feasibility Team Report to the National Assessment Governing Bo ard. NAGB web site. Fowler, W. J., Jr., & Monk, D. H. (2001). A Primer for Making Cost Adjustments in
31 of 39Educatio n NCES 2001323 Washington DC: U.S. Department of E ducation National Center for Education Statistics.Fuhrman, S. H., & Elmore, R. F. (1995). Ruling Out Rules: The Evolution of Deregulation in State Education Policy. New Brunswick NJ: Consortium for Policy Research in Education ERIC Document No. ED383057.Fuhrman, S. H., & Elmore, R. F. (1990). Understandi ng local control in the wake of state educational reform. Educational evaluation and policy analysis 12 82-96. General Accounting Office. (1998). School Finance: State and Federal Efforts To Target Poor Students. Report to Congressional Requesters GAO/HEHS-98-36. General Accounting Office Washington DC.Guskey, T. R. (1993). Policy Issues and Options Whe n States Take Over Local School Districts. International Journal of Educational Reform 2 1 68-71. Hajek, J. (1981). Sampling from a Finite Population. New York: Marcel Dekker. Hannaway, J. (1993). Political Pressure and Decentr alization in Institutional Organizations: The Case of School Districts. Sociology of Education 66 3 147-63. Hannaway, J. (1992). School Districts: The Missing Link in Education Ref orm. New Brunswick NJ: Consortium for Policy Research in Edu cation. ERIC Document No. ED359644.Hanushek, E. A. (1996). School resources and studen t performance. In Burtless G. (Ed.). Does Money Matter? The Effect of School Resources o n Student Achievement and Adult Success. Washington DC: Brookings Institution Press Hanushek, E. A. (1994). A Jaundiced view of "adequa cy" in school finance reform. Educational Policy 8 4 460-469. Hedges, L. V., & Greenwald, R. (1996). Have times c hanged? The Relation between school resources and student performance. In Burtle ss G. (Ed.). Does Money Matter? The Effect of School Resources on Student Achievement a nd Adult Success. Washington DC: Brookings Institution Press.Hough, J. R. (1993). Educational Finance Issues in North America. Education Economics 1 1 35-42. Hussar, W., & Sonnenberg, W. (1999). Trends in disparities in school district level expenditures per pupil. NCES 1999-017. Washington DC: NCES. Iannaccone, L., & Lutz, F. W. (1994). The Crucible of Democracy: The Local Arena. Journal of Education Policy 9 5&6 39-52. Johnson, E. G. (1989). Considerations and Technique s for the Analysis of NAEP Data. Journal of Educational Statistics 14 303-334. Johnson, E. G., & Allen, N. L. (1992). The NAEP 1990 Technical Report. NCES 92-067. Washington DC: National Center for Education Statis tics.
32 of 39Johnson, E. G., Mazzeo, J., & Kline, D. L. (1994). Technical Report of the NAEP 1992 Trial State Assessment Program in Reading. NCES 94-472. Washington DC: National Center for Education Statistics.Johnson, E. G., & Rust, K. F. (1992). Population In ferences and Variance Estimation for NAEP Data. Journal of Educational Statistics 17 175-190. Kazal-Thresher, D. M. (1993). Educational expenditu res and school achievement: When and how money can make a difference. Educational Researcher 22 2 30-32. Keedy, J. L. (1994). The Twin Engines of School Ref orm for the 1990s: The School Sites and National Coalitions. Journal of School Leadership 4 1 94-111. Lee, V. E., Smith, J. B., & Croninger, R. G. (1995) Another look at high school restructuring. More evidence that it improves stude nt achievement and more insight into why. Issues in Restructuring Schools. No. 9 Levine, R., & Christenson, B. (1998). Public School Districts in the United States: A Statistical Profile 1987-88 to 1993-94. Statistical Analysis Report. NCES-98-203. Washington DC: American Institutes for Research.Little, J. A. (1993). Post-Stratification: A Modele r's Perspective. Journal of the American Statistical Association 88 1001-1012. Little, J. A., & Wu, M.-M. (1991). Models for Conti ngency Tables with Known Margins When Target and Sampled Populations Differ. Journal of the American Statistical Association 86 87-95. Little, R. J. A., & Rubin, D. B. (1986). Statistical Analysis with Missing Data. New York NY: John Wiley & Sons.Lockwood, A. T. (1994). The Future of school finance. Focus in Change no.13 Madison WI: Wisconsin Center for Educational Research.Marsh, D. D. (1997). School Reform and Decentralization in the United St ates: The National Alliance Experience ED409606. Paper presented at the Annual Meeting o f the American Educational Research Association (Chicago IL March 24-28 1997). Monk, D. H., & Hussain, S. (2000). Structural influ ences on the internal allocation of school district resources: Evidence from New York S tate. Educational Evaluation and Policy Analysis 22 1 1-26. Murphy, J. (1994). Principles of School-Based Management Chapel Hill NC: North Carolina Educational Policy Research Center. ERIC D ocument No. ED373414. Murphy, J., & Hallinger, P. (1988). Characteristics of instructionally effective school districts. Journal of Educational Research 81 175-181. Nachtigal, P. (1982). Education in rural America: a n overview. In P. Nachtigal (Ed.). Rural education: In search of a better way Boulder CO: Westview Press. National Assessment Governing Board. (1999). The National Assessment of Educational
33 of 39Progress: Design 2000-2010 Adopted Unanimously by The National Assessment Governing Board June 23 1999. Washington DC: NAGB.National Center for Education Statistics. (1995). Common Core of Data (CCD). Includes Data From 1987/88-1992/93. Installation Guide Washington DC: the author. National Center for Education Statistics (1996). The Condition of Education 1996. Washington DC: the author. New York State Education Department. (1992). Study on the Generation of Revenues for Education. New York State Board of Regents. Final R eport. ERIC ACCESSION No. ED383106.Newmann, F. M., & Wahlage, G. G. (1995). Successful School Restructuring: A Report to the Public and Educators by the Center on Organi zation and Restructuring of Schools. Madison WI: University of Wisconsin-Madison.Payne, K. J., & Biddle, R. J. (1999). Poor school f unding child poverty and mathematics achievement. Educational Researcher 28 6 4-13. Pellegrino, J. W., Jones, L. R., & Mitchell, K. J. (Eds.). (1999). Grading the Nation's Report Card: Evaluating NAEP and Transforming the A ssessment of Educational Progress Washington DC: National Academy Press. Porter, A. C. (1994). National equity and school au tonomy. Educational Policy 8 4 489-500.Protheroe, N. (1997). ERSLocal School Budget Profi le Study. School Business Affairs 63 10 42-29. Rasbash, J., & Woodhouse, G. (1995). MLn Command Reference Version 1.0 London UK: University of London.Raudenbush, S. W., Kidchanapanish, S., & Kang, S. J (1991). The effects of pre-primary access and quality on educational achievement in Th ailand. Comparative Education Review, 35 255-273. Reese, C. M., Miller, K. E., Mazzeo, J., & Dossey, J. A. (1997). NAEP 1996 Mathematics Report Card for the Nation and the Stat es.Washington DC: NCES. Rust, K., Burke, J., Fahimi, M., & Wallace, L. (199 2). 1990 National Assessment of Educational Progress Sampling and Weighting Procedu res Part 2National Assessment Rockville MD: Westat Inc.Shao, J., & Tu, D. (1995). The Jackknife and Bootstrap New York: Springer-Verlag. Sharp, W. L. (1993). School Spending: Is There a Relationship between Sp ending and Student Achievement? A Correlation Study of Illinoi s Schools Paper presented at the Annual Meeting of the American Education Finance As sociation (Albuquerque NM March 1993). ERIC Document No. ED357503.Strang, D. (1987). The Administrative transformatio n of American education: school
34 of 39district consolidation 1938-1980. Administrative Science Quarterly 43 352-366. Wainer, H. (1993). Does Spending Money on Education Help? A Reaction to the Heritage Foundation and the "Wall Street Journal." Educational Researcher 22 9 22-24. Walberg, H. J., & Walberg, H. J. III. (1994). Losin g local control Educational Researcher 23 6 19-26. Wallace, L., & Rust, K. (1994). 1992 National Assessment of Educational Progress Sampling and Weighting Procedures Final Report. Rockville MD: Westat Inc. Weng, S. S., Zhang, F., & Cohen, M. P. (1995). Vari ance Estimates Comparison by Statistical Software. ASA 1995 Proceedings of the Section on Survey Resea rch Methods. Alexandria VA: American Statistical Association.Wenglinsky, H. (1997). How money matters: the effec t of school district spending on academic achievement. Sociology of Education 70 3221-237. Wenglinsky, H. (1998). Finance equalization and wit hin-school equity: the relationship between education spending and the social distribut ion of achievement. Educational Evaluation and Policy Analysis 20 4 269-283. Westat Inc. (1998). ESSI Work Task 18.104.22.168 1990 Through 1990 NAEP Samp les: NCES Identifiers. (Contract report and memorandum to NCES). Rockvill e MD: Westat. Westat Inc. (1996). A User's Guide to WesVar PC Version 2.0. Rockville MD. Wolter, K. M. (1985) Introduction to Variance Estimation New York: Springer-Verlag. Woodhouse, G. (ed.) (1995). A Guide to MLn for New Users London UK: Institute of Education University of London.About the AuthorsGary G. Huang, Ph.D.Senior Research Analyst Synectics for Management Decisions, Inc.North Moore St. Arlington, VA 22209Phone: 703.807.2324 703.528.2857 Email: email@example.comGary Huang is a research sociologist working in edu cation and public policy related areas. His research interests include minority chil dren's socialization and schooling, institutional effects on learning, rural education, and cross-cultural research and communication. He works mainly with large-scale sur vey data analyses but has a broad interest in other approaches to research.Binbing Yu, Ph.D.Synectics for Management Decisions, Inc.1901 North Moore Street, Suite 900
35 of 39 Arlington, VA 22209Appendix I Modification of the Sample and Weights Appendix I is available for downloading as a Rich T ext Formatted file. Appendix II Race-Census Region with Modified SampleNAEP-CCD 1990 Frequencies of students by region-race using sum of Level-1 weight 10 Based on original weight (ORIGWT) in NAEP data Race-RegionFreq.%Cum. Freq.Cu. % White-Northeast877022.512.37 87702 2.5 12.37 White-Midwest150156521.18 2378587 33.55 White-South 1975736 27.87 4354323 61.42 White-West600752.98.47 495507669. 89 Non-hispanic black114732616.18 61 0240286.08 Hispanic771106 10.88 687350896.95 Non-hispanic other216087.63.05 70 89596100.00 Frequencies of students by region-race usi ng sum of Level-1 weight 11 Based on adjusted weight (ADJWT) for merged NAE P-CCD data Race-RegionFreq.%Cum. Freq.Cu. % White-Northeast959336.413.53 95933 6.4 13.53 White-Midwest1393679 19.66 23530 1533.19 White-South150102321.17 385403854 .36 White-West964005.813.60 481804467 .96 Non-hispanic black113772516.05 59 5576984.01 Hispanic 838602.611.83 679437195. 84 Non-hispanic other295224.54.16 708 9596100.00 NAEP-CCD 1992 Frequencies of students by region-race usi ng sum of Level-1 weight 19 Based on original weight (ORIGWT) in NAEP data Race-RegionFreq.%Cum. Freq.Cu. %
36 of 39 White-Northeast454207.711.27 45420 7.7 11.27 White-Midwest766619.719.01 122082 730.28 White-South982156.524.36 22029845 4.64 White-West523640.312.99 27266246 7.63 Non-hispanic black711623.617.65 3 43824885.28 Hispanic432971.2 10.74 387121996. 02 Non-hispanic other1606533.98 4031 872100.00 Frequencies of students by region-race usi ng sum of Level-1 weight 20 Based on adjusted weight (ADJWT) for merged NAEPCCD data Race-RegionFreq.%Cum. Freq.Cu. % White-Northeast446802.111.15 446802.1 11 .15 White-Midwest753990.918.81 120079329.96White-South965832.324.10 216662554.05White-West51504212.85 268166766.90Non-hispanic black635054.615.84 331672282.75Hispanic505862.212.62 3822584 95.37 Non-hispanic other185603.8 4.63 4008188100 .00 NAEP-CCD 1996 Frequencies of students by region-race usi ng sum of Level-1 weight 14 Based on original weight (ORIGWT) in NAEP data Race-RegionFreq.%Cum. Freq.Cu. % White-Northeast402880.712.65 402880.712.65 White-Midwest615135.119.31 101801631.95White-South70572422.15 172374054.11White-West437100.6 13.72 216084067.82Non-hispanic black477674.114.99 263851482.82Hispanic 402241.112.63 304075695.44Non-hispanic other145152.34.56 3185908100.00 Frequencies of students by region-race usi ng sum of Level-1 weight 15 Based on adjusted weight (ADJWT) for merged NAEPCCD data Race-RegionFreq.%Cum. Freq.Cu. %
37 of 39 White-Northeast 389907.8 12.24 389 907.8 12.24 White-Midwest 598667.2 18.79 988 575.1 31.03 White-South 684636.921.49 1673212 52.52 White-West387357.9 12.16 2060570 64.68 Non-hispanic black538638.316.91 2 599208 81.58 Hispanic 432200.413.57 303140995. 15 Non-hispanic other 154499.44.85 100Appendix III Analytic procedures with two-level HLM modeling Appendix III is available for downloading as a Rich Text Formatted file. Copyright 2002 by the Education Policy Analysis ArchivesThe World Wide Web address for the Education Policy Analysis Archives is epaa.asu.edu General questions about appropriateness of topics o r particular articles may be addressed to the Editor, Gene V Glass, firstname.lastname@example.org or reach him at College of Education, Arizona State University, Tempe, AZ 8 5287-2411. The Commentary Editor is Casey D. Cobb: email@example.com .EPAA Editorial Board Michael W. Apple University of Wisconsin Greg Camilli Rutgers University John Covaleskie Northern Michigan University Alan Davis University of Colorado, Denver Sherman Dorn University of South Florida Mark E. Fetler California Commission on Teacher Credentialing Richard Garlikov firstname.lastname@example.org Thomas F. Green Syracuse University Alison I. Griffith York University Arlen Gullickson Western Michigan University Ernest R. House University of Colorado Aimee Howley Ohio University Craig B. Howley Appalachia Educational Laboratory William Hunter University of Calgary Daniel Kalls Ume University Benjamin Levin University of Manitoba
38 of 39 Thomas Mauhs-Pugh Green Mountain College Dewayne Matthews Education Commission of the States William McInerney Purdue University Mary McKeown-Moak MGT of America (Austin, TX) Les McLean University of Toronto Susan Bobbitt Nolen University of Washington Anne L. Pemberton email@example.com Hugh G. Petrie SUNY Buffalo Richard C. Richardson New York University Anthony G. Rud Jr. Purdue University Dennis Sayers California State UniversityStanislaus Jay D. Scribner University of Texas at Austin Michael Scriven firstname.lastname@example.org Robert E. Stake University of IllinoisUC Robert Stonehill U.S. Department of Education David D. Williams Brigham Young University EPAA Spanish Language Editorial BoardAssociate Editor for Spanish Language Roberto Rodrguez Gmez Universidad Nacional Autnoma de Mxico email@example.com Adrin Acosta (Mxico) Universidad de Guadalajaraadrianacosta@compuserve.com J. Flix Angulo Rasco (Spain) Universidad de Cdizfelix.firstname.lastname@example.org Teresa Bracho (Mxico) Centro de Investigacin y DocenciaEconmica-CIDEbracho dis1.cide.mx Alejandro Canales (Mxico) Universidad Nacional Autnoma deMxicocanalesa@servidor.unam.mx Ursula Casanova (U.S.A.) Arizona State Universitycasanova@asu.edu Jos Contreras Domingo Universitat de Barcelona Jose.Contreras@doe.d5.ub.es Erwin Epstein (U.S.A.) Loyola University of ChicagoEepstein@luc.edu Josu Gonzlez (U.S.A.) Arizona State Universityjosue@asu.edu Rollin Kent (Mxico)Departamento de InvestigacinEducativa-DIE/CINVESTAVrkent@gemtel.com.mx email@example.com Mara Beatriz Luce (Brazil)Universidad Federal de Rio Grande do Sul-UFRGSlucemb@orion.ufrgs.brJavier Mendoza Rojas (Mxico)Universidad Nacional Autnoma deMxicojaviermr@servidor.unam.mxMarcela Mollis (Argentina)Universidad de Buenos Airesmmollis@filo.uba.ar
39 of 39 Humberto Muoz Garca (Mxico) Universidad Nacional Autnoma deMxicohumberto@servidor.unam.mxAngel Ignacio Prez Gmez (Spain)Universidad de Mlagaaiperez@uma.es Daniel Schugurensky (Argentina-Canad)OISE/UT, Canadadschugurensky@oise.utoronto.ca Simon Schwartzman (Brazil)American Institutes for ResesarchBrazil(AIRBrasil) firstname.lastname@example.org Jurjo Torres Santom (Spain)Universidad de A Coruajurjo@udc.es Carlos Alberto Torres (U.S.A.)University of California, Los Angelestorres@gseisucla.edu