xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam a22 u 4500
controlfield tag 008 c20059999azu 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E11-00462
Educational policy analysis archives.
n Vol. 13, no. 40 (September 28, 2005).
Tempe, Ariz. :
b Arizona State University ;
Tampa, Fla. :
University of South Florida.
c September 28, 2005
Forced march for failing schools : lessons from the New York City Chancellor's district / Deinya Phenix, Dorothy Siegel, Ariel Zaltsman [and] Norm Fruchter.
Arizona State University.
University of South Florida.
t Education Policy Analysis Archives (EPAA)
xml version 1.0 encoding UTF-8 standalone no
mods:mods xmlns:mods http:www.loc.govmodsv3 xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govmodsv3mods-3-1.xsd
mods:relatedItem type host
mods:identifier issn 1068-2341mods:part
mods:detail volume mods:number 13issue 40series Year mods:caption 20052005Month September9Day 2828mods:originInfo mods:dateIssued iso8601 2005-09-28
Readers are free to copy display, and distribute this article, as long as the work is attributed to the author(s) and Education Policy Analysis Archives, it is distributed for noncommercial purposes only, and no alte ration or transformation is made in the work. More details of this Creative Commons license are available at http://creativecommons.org/licen ses/by-nc-nd/2.5/. All other uses must be approved by the author(s) or EPAA EPAA is published jointly by the Colleges of Education at Arizona State University and the Universi ty of South Florida. Articles are indexed by H.W. Wilson & Co. Send formal remarks to EPAA Commentary Editor Casey Cobb (email@example.com) and errata requests to Sh erman Dorn (firstname.lastname@example.org). EDUCATION POLICY ANALYSIS ARCHIVES A peer-reviewed scholarly journal Editor: Sherman Dorn College of Education University of South Florida Volume 13 Number 40 September 28, 200 5 ISSN 1068 A Forced March for Failing Schools: Lessons from the New York City Chancellor's District1 Deinya Phenix, Dorothy Siegel, Ariel Zaltsman, Norm Fruchter New York University Citation: Phenix, D., Siegel, D., Zaltsman, A., & Fruchter, N. (2005). A forced march for failing schools: Lessons from the Ne w York City Chancellors District. Education Policy Analysis Archives, 13 (40). Retrieved [date] from http://epaa.asu.edu/epaa/v13n40/. Abstract In the mid-nineties, the New York City Schools Chancellor created a citywide improvement zone to take over a significant proporti on of the citys lowest performing schools whose local community school districts had failed to improve them. This Chancellors District defined centralized management, rather than local control, as the critical variable necessary to initiate, enforce and ensure the implementation of school improvement. Th is large-scale intervention involved both a governance change and a set of ca pacity-building interve ntions presumably unavailable under local sub-district control. Our study retrospectively examined the origins, structure and components of the Chancellors District, and analyzed the characteristics and outcomes of the elementary schools mandated to receive these interventions. Our longitudinal analysis compared Chancellors District schools to New York Citys other state-identified low performing schools, based on a schoollevel panel of performance, demographic, human resource, and expenditure data 1 Partial support for the research leading to this arti cle was provided by the J.P. Morgan Chase Foundation. The authors would also like to thank Bree Picower, Margaret Murphy, Ben Kennedy, Natasha Pchelintseva and Cameron Cole for their work on earlier formations of the analysis. We also thank Richard Arum, Amy Ellen Schwartz, Leanna Stiefel, Dae Yeop Kim, Hella Bel Hadj Amor, Colin Chellm an, Patrice Iatarola and three anonymous reviewers for their comments on an earlier version of this paper.
Education Policy Analysis Archives Vol. 13 No. 40 2 collected from district Annual School Report Cards and School Based Expenditure Reports from 1998 throug h 2001. The results sugge st that the Chancellors District intervention improved these school s instructional capa city and academic outcomes, both relative to where these schools would ha ve been and relative to comparable schools. Keywords: school reform; low performing schools; accountability; district intervention Introduction This article analyzes the results of the Chancello rs District, an initiative created to accelerate their improvement by remove state-identified low-performing schools from their local district authorities, imposing a uniform curriculum, intensi ve professional development, reduced class size, extended time and other reforms. The seven-year Chancellors District initiative represents both an unprecedented intervention into New York City school governance and a major challenge to several reigning theories about the relationship between ce ntralized administration and local school change. Consider, first, how the Chancellors District depa rted from the New York City school systems governance norms. From 1969 to 2003, New York Ci tys public elementary and midd le schools were governed by 32 decentralized community sc hool districts (hereafter sub-district s), administered by locally elected school boards and their appointees, the community su perintendents. These su b-districts were quite large, averaging more than 20,000 students, with several of the largest district s enrolling more than 40,000 students. Many of th ese sub-districts would ha ve ranked among the 50 largest school systems in the country had they been independent jurisdictions. During their thirty-four years of relative au tonomy, these sub-districts developed diverse, and differentially effective, patterns of operation. Consistently high performance characterized schools in some sub-districts, while poor manage ment and dismal student outcomes plagued schools in others. Though the grim correlations among race, poverty and student achievement that characterize most urban districts have also persisted in New York City, individual school outcomes varied widely, both across and within the community school sub-districts. Academic performance was especially poor, and particularly highly correlated with indicators of race and poverty, in those sub-districts whose governance was marked by pa tterns of corruption, patronage and, most importantly, a consistent failure to fo cus on improving teaching and learning. The school systems central administration, governed by an appointed citywide board of education and a chief administrative officer (the Chancellor), had possessed the authority to remove failing schools from their community school sub-distri cts since the city system was decentralized in 1969. But that power remained unexercised for al most three decades until 1996, when the reigning Chancellor created a new, geographically non-c ontiguous sub-district, and imposed the same improvement regimen on each school. The Chancello rs District became a new, non-geographic improvement zone that eventually removed some 58 elementary and middle schools from local subdistrict control. This effort to remove failing schools from thei r sub-district jurisdictions in order to improve them was a radical change in New York City school governance. From the onset of decentralization, central leadership had bemoaned sub-district failure but had refused to intervene, either to force subdistricts to take steps to improve their schools or to take failing sch ools away from local sub-district control. The Chancellors assertion of the power to take over failing schools, and his creation of a new
A Forced March for Failing Schools 3 district to force-feed their improvement represen ts an historic departure from three decades of unobstructed local sub-district control. This article describes the origins, structure, and components of the Chancellors District, and details our analysis of whether these particularly low performing schools were improved more than other state-identified low performing schools. To un derstand the nature of the intervention that the Chancellors District represented, we analyzed administrative documents, including budget allocation memoranda, and conducted numerous interviews with state and city administrators. To evaluate the impact of the Chancellors District as an intervention, we conducted a longitudinal analysis that compared the academic performance of Chancellor s District schools to New York Citys other state identified, low performing (SURR) schools. The next section situates the Chancellors District initiative in reform theory and reform efforts, and the following section details the results of interviews with district officials about how this special district worked as an intervention. The subsequent sections detail the quantitative data and methods used for this study, including its limitations, and changes in the Chancellors District schools and their implications about the districts overall effectiveness. The Chancellors District initiative ended in July 2003, with the implementation of a systemwide restructuring policy that reorganized th e entire New York City school system. The 32 elementary and middle schools in the Chancellors Di strict were transferred back to their local subdistricts, which were themselves subsumed into a new regional structure under the Chancellors direct control. Thus the Chancellors District init iative is now history, and each of the new administrative regions is now responsible for improving its failing schools. But the extent to which the Chancellors District initiative succeeded in improving student outcomes, particularly in the failing elementary schools whose outcomes we examin ed, directly challenges the reigning theories that link school improvement to decentralization, and has important implications for the variety of schooland district-level improvement efforts unde rway in urban districts across the country. Big Bureaucracy, District Capacity and School Improvement The Chancellors District initiative is unique among recent large-scale reform efforts. Given the scale and complexity of the New York City system the Chancellors District initiative is akin to state takeover efforts of poorly performing districts. Because the Cha ncellor is responsible for more schools (currently over 1,300) than many state chiefs the Chancellors administrative relationship to the 32 sub-districts was comparable to state commissioners relationships to their local school districts. Moreover, as most states takeover effo rts of local districts have been for financial mismanagement rather than instructional failure, fe w state takeovers have targeted as many schools for restructuring, redesign and instructional im provement as the Chancellors District effort. The Chancellors District initiative also poses a strong challenge to three important arguments in the research about the relationship between district administration and school change. Historically, many researchers and critics have inve ighed against the effects of district size and the resulting bureaucracies, contending that larg e urban systems have become ungovernable and impervious to reform efforts (Domanico, 1994). Seymour Sarasons (1996) classic analysis maintained that big-city schools are insulated, encapsulated, and in other ways immune from hierarchically imposed efforts to alter dysfunctional practice at the school level. In another classic study, Weick (1976) argued that the loose coupling within the various layers of complex urban systems stymies the efforts of centralized intervent ions to produce changes in school practices that might lead to school improvement.
Education Policy Analysis Archives Vol. 13 No. 40 4 This analysis of the inevitable barriers to change that scale and hierarchical complexity impose may not necessarily imply reform from below or one-school-at-a-time change. But the critiques prognosis of the likely effectiveness of centr alized administrative efforts to drive change is quite bleak. The Chancellors Districts forced-march efforts to improve the schools taken from their local sub-districts challenge this critical tradition. Second, the Chancellors District initiative cha llenges several influential currents of recent reform theory that link the necessity for decentralization with the need to provide maximum autonomy at the school level to achieve successful schools. In Politics, Markets and Americas Schools John Chubb and Terry Moe (1990) argued that the key characteristic that distinguishes academically effective private sc hools from less effective public schools is the extent of autonomy at the school level. Chubb and Moes influential arguments stressed the inevitability of bureaucratization and consequent poor school pe rformance unless schools are severe d from district control and governed by market principles. Recent theoretical efforts to establish the primacy of the school, rather than the district, as the locus of improvement have not been monopolized by conservative scholars or market advocates. The Cross City Campaign for Urban School Reform, an advocacy organization composed of city reform groups, education advocates and pa rent activists, published an influential report (Hallet, 1995) that urged radical decentralization, to the school level, of all essential instructional and administrative functions, leaving school districts with only vestigial governing roles. The authors of Reinventing Central Office argued that urban school districts had consistently failed to implement effective improvement efforts, and characterized their administrations as retarding forces that stifled school-based reform efforts. A third recent and influential reform stream stresses the necessity for bottom-up or schoolby-school reform efforts. Severa l national reform consortia, such as the Coalition of Essential Schools, the School Development Program, the Ac celerated Schools project and the New American Schools, all focus on the need to generate individual school improvement through the implementation of replicable programs. This reform stream was elevated into national prominence through federal legislation, the Comprehensive School Reform Demonstration program, popularly known as the ObeyPorter Act of 1997, which has allocated more than $300 million annually for grants to individual schools to implement supposedly research-validated improvement models. The role of the district in initiating, coordinating, or supporting these school-based efforts was, at best, subordinated to the role of the intermediary organi zations marketing the particular models or, at worst, essentially untheorized (Bodilly, 2001). These reform currents that target individual sc hools as key improvement sites have begun to be challenged by efforts to define the school di strict as the necessary locus of capacity-building initiatives. Ascher, Fruchter, and Ikeda (1999), fo r example, argued that the local district is the critical actor that can encourage or retard the schools development of the necessary capacity for self-improvement (Ascher, Fruc hter, & Ikeda, 1999, p. 43). The Council of Great City Schools Foundations for Success (2002) analyzed the efforts of three urban districts to improve student academic performance, and to narrow the achievement gap between white students and students of color. The Annenberg Institute for School Refo rm has created School Communities that Work: A National Task Force on the Future of Urban Districts, to help districts restructure themselves into effective support systems focused on improving inst ruction. The University of Pittsburghs Learning Research and Development Center has created the Institute for Learning to help urban districts reorganize and improve their capacities to help their schools, and themselves, become continuous learning organizations (Snipes, Doolittle, & Herlihy, 2002). But in 1996, the Chancellor was bucking several traditions of reform theory when he took over schools whose local sub-districts had failed to improve them. His theory of action defined
A Forced March for Failing Schools 5 centralized management, rather than decentralized loca l control, as the critical variable necessary to initiate, enforce and ensure the implementation of school improvement. Defining the core issues as the ability to mobilize the political will and instructional capacity necessary to improve schools, he asserted that the central administration could mandate the policies, implement the procedures and provide the resources necessary to transform failing schools. Given this premise, the Chancellors District involved both a governance change and a coherent set of capacity-building interventions presumably unavailable to low-performing schools under local sub-district control. The Design of the Chancellors District Since 1989, the New York State Education De partment (SED) has used the Schools Under Registration Review (SURR) process to identify lo w-performing schools, and place them on a list of schools under registration review. SED requires SURR schools to create a comprehensive education plan, and can order chronically low-performing sc hools to undergo school redesign. Schools that fail to improve may have their registration revoke d, which means they are effectively closedthe ultimate sanction of the SURR process. In October 1995, the New York State Education Commissioner informed New York Citys Schools Chancellor that he would revoke the regist rations of sixteen chronically low-performing New York City schools that had long languished on the SURR list if their student performance did not improve by June 1997. In response, the Cha ncellor met with the schools local sub-district superintendents, community school board members, principals, and parent leaders, and notified them that he was requiring the schools to devel op and implement instructional improvement plans. In half the schools, he removed the principals and mandated comprehensive school redesign. Unsatisfied with the sub-district and school responses to his actions, the Chancellor decided, in February 1996, to intervene directly in schools in which, in his determination, the local subdistricts had fail[ed] to demonstrate the capacity to redesign failing organizations (New York City Board of Education, 1999b, p. 1). On his recomm endation, the New York City Board of Education created the Chancellors District, with the mission t o develop and expand central, district, and local school capacity to transform failing school organizations into redesigned and revitalized schools that meet high educational standards for students (New York City Board of Education, n.d., p. vi). He immediately transferred ten schools into the new sub-district. In the seven years of its operation, the Chan cellors District included 58 elementary and middle schools.2 These schools entered the District in annual cohorts of various sizes. By the end of the 2002 school year, the District had closed el even schools and returned fifteen to their home sub-districts. The Chancellors District has take n over schools from every New York City borough except Staten Island, with a disproportionate number from the Bronx. There were 32 elementary and middle schools in the District at the time of its dissolution in June 2003. The 1999 school year was a seminal year for the Chancellors District. Between 1996 and 1999, the district had taken in only a small number of schools. But in 1999, af ter an extensive review of the patterns of failure across the citys low-performing SURR schools, the Chancellor decided to take 37 more of the citys lowest performing sc hools into the Chancellors District, and imposed a new, highly structured improvement plan, A Model of Excellence, on all the Districts schools. He also increased staff capacity in all the SURR schools, especially those in the Chancellors District. 2 Throughout its existence, the Chancellors District also includ ed ten high schools, which had already been under more centralized control. Thes e schools are not included in the analysis here.
Education Policy Analysis Archives Vol. 13 No. 40 6 His goal was to remove every New York City school from the SURR list within two years (New York City Board of Education, 1999b). Table 1 Schools in the New York City Chan cellor's District (CD), 19962003 Academic Year Entered the CD Returned to Home District Closed 1996 9 0 0 1997 3 0 0 1998 0 2 0 1999 37 0 2 2000 1 8 5 2001 5 5 3 2002 3 1 Total 58 15 11 Sources: New York State Education Departme nt; New York City Board of Education City officials divided the citys SURR schools into three groups. Category 1 schools were those assessed at highest risk of continued failure Category 2 schools were at the next highest risk of failure. Category 3 schools were schools that were improving enough to become candidates for removal from the SURR list in the following year. Ten of the elementary and middle schools were identified as Category 1 schools had already been placed in the Chancellors District in previous years. Thirty-seven more Category 1 schools were adde d to the Chancellors District for the 1999 school year. Though the remaining five Category 1 schools exhibited the same pattern of failure as the schools in the Chancellors District, they were allo wed to remain in their local districts because their districts engaged in other major reform init iatives and the Chancellor decided that their subdistricts had the capacity to support their schools improvement plans. The superintendents of these districts met monthly with the Chancellors Dist rict Supervising Superintendent to coordinate implementation of the intervention. Thus, in 1999, the Chancellors District consisted of 47 of the citys SURR schools deemed to be the lowest performing elementary and middle schools. The district was sub-divided into four regions, each with its own instructional superintendent. In that same year, Chancellors District schools began implementing the new Model of Excellence. Class size was reduced throughout the district. A maximum of 20 students were mandated for kindergarten through grade 3, and 25 students for grades 4 through 8. Instructional time was increased by extending both the school day and the school year. The school day was lengthened to 20 minutes longer than in other elementary and middle schools in New York City. The school calendar was also ex tended by one week. Instructional time was further enhanced by developing after-school programs, implemented in each Chancellors District school through a schedule of activities that extended th e school day to 6 pm. The after-school program was designed to enhance and enrich daily learning (New York City Board of Education, 2001a, p. 2). Tutoring was offered from 3 to 4 pm in small grou p settings for those students in grades 3 who required extra reading or math assistance. A prescribed instructional program, a manda ted daily schedule and a required curriculum were imposed throughout the district. In elementa ry schools, the schedule mandated two daily 90minute literacy blocks, the first using Success for A ll, and the second using the Balanced Literacy
A Forced March for Failing Schools 7 program. The daily schedule also included a 60-minute math block, using the Trailblazers math program; and a 30-minute skills block, alternati ng between math and literacy skills. Science and social studies were each taught once per week. Because the time devoted to literacy instruction in the elementary school schedule was almost three times that assigned to math, the Chancellors District was perceived as concentrating on literacy skills im provement at the elementary level much more intensively than on math. The district also provided intensive professiona l development. Each school was assigned at least four on-site staff developers focused on Eng lish Language Arts, mathematics, technology and Success for All. Extra time was provided for prof essional development designed to be intensive, systematic, structured and aligned with the curricu lum. Each school was provided with an on-site teacher center staffed by a teacher specialist w ho offered additional coaching and professional development. Assessments, integrated into the Su ccess for All and Trailblazers programs, were designed to provide regular feedback to classroom teachers. In kindergarten through grade 3, the schools used New York Citys Early Childhood Litera cy Assessment System (ECLAS) to assess and improve literacy growth. Specially developed benc hmark assessments in reading and mathematics were used to assess the performance of students from grades 3 on. Table 2 Model of Excellence Components NYC Low-Performing Schools Component Chancellor's District schools (N=49) Other SURR schools (N=53) Reduced class size Extended school day and year After-school program Mandated instru ctional program Mandated daily schedule Mandated curriculum Number of on site staff developers 4 2 Extra time for staff development Prescribed staff development Teacher center and teacher specialist Student assessment program Additional supervisory/district support Category 2 and 3 schools (other SURR schools ) received fewer intervention components than schools in the Chancellors District, as illust rated in Table 2. Central authorities allocated additional funds, initially $20,000,000 (New York City Board of Education, 1999a), targeted to implement the specific interventions prescribed in the Model of Excellence in the Chancellors District schools. By 2000, when the Chance llors District model was fully implemented, the Districts schools spent an average of $2,400 more per student than the other SURR schools.3 Most of the increased spending represented in creased teacher costs, including two programs designed to attract certified teachers to the Cha ncellors District schools. Certified teachers who 3 According to the New York City Board of Education School Based Expenditure Reports, Chancellors District elementary and middle schools in 2000-01 spent an average of $13,150 per student. Other SURR schools spent an average of $10,744. This compares to an overall av erage New York City per student expenditure of $9,679 for elementary and middle schools.
Education Policy Analysis Archives Vol. 13 No. 40 8 chose to work in Extended Time Schools (ETS) received 15% additional pay in exchange for additional work. The ETS program, developed in collaboration with the United Federation of Teachers, was implemented in 1999 in all bu t two Chancellors District schools. Certified, experienced private school teachers who chose to te ach in the Chancellors District received $10,000 bonuses (Iatarola, 2001). The Chancellors District also introduced several policies to improve the qualifications, quality, preparation, and stability of the leadership and staff in Chancellors District schools. Most of the Chancellors District schools were assigned new principals. Additional assistant principals were also assigned, and both principals and assistant principals received professional development focused on how to supervise implementation of the instructional plan. The District, through its four instructional regions, provided additional intensive instructional and supervisory support to the schools leadership and staff. On-site professional development specialists, including three full-time instructional specialists (one each in literacy, mathematics and technology), a Success for All facilitat or, and a teacher center specialist, provided consistent, intensive, highly structured professiona l development for all teachers in the districts elementary schools. Teachers attended a one-week professional development program every August, in addition to the training they received on citywide staff development days. Furthermore, many ineffective teachers were removed from the Chancellors District schools. According to one official, the Chancellors District absorbed the cost of approximately two dozen teachers salaries until their cases were adjudicated, rather than allow them to remain in the classroom. Finally, the Chancellor introduced two important teacher incentive initiatives for all SURR schools, including those in the Chancellors Distri ct. The immediate impetus was the state mandate that as of September 1, 2000, only certified tea chers would be assigned to SURR schools. In exchange for agreeing to work in SURR or other hard-to-staff schools for a full year, teachers received grants of up to $3,400 from the Teache rs for Tomorrow program, which they could use to repay educational loans or meet other qualified educational expenses. Beginning in the 1999 school year, candidates for teacher positions who lacked traditional certification could participate in a new alternative certification program called the New York City Teaching Fellows Program. This program met the cost of a masters degree in education and provided training during the summer, as well as mentors during the school year. New York State gave New York City Teaching Fellows provisional certification to allow them to teach while completing their degrees. Most Teaching Fellows were placed in the Chancellors District or other SURR schools. SURR schools often had a disproportionate number of full-time special education students, the result of previous sub-district placement decisions. City policy for all SURR schools, including Chancellors District Schools, was to examine the number of special education students and reduce disproportionate placements. As these program descriptions indicate, the Chancellors District mounted a comprehensive effort to improve the poorly performing schools that had been removed from their sub-district jurisdiction. The next section describes how we assessed the effectiveness of the Chancellors Districts efforts to improve these schools. Methods In addition to our analysis of New York City Board of Education budgets and interview data described above, we conducted a longitudinal analysis that compared the academic performance of
A Forced March for Failing Schools 9 Chancellors District elementary schools to New York Citys other state identified, low performing (SURR) schools. We constructed a school-level pa nel, based on data collected from Annual School Report Cards and School Based Expenditure Repor ts from 1998 through 2001. These two administrative databases offer a wealth of inform ation on student demographics, teacher credentials and experience, school organizational characteristi cs, and expenditures for all schools in the New York City school system. Our chief outcome variable was schools fourth -grade academic performance, expressed as average scale scores, the percent of students meet ing the standard (Levels 3 and 4), and the percent far below the standard (Level 1) on the states English Language Arts (hereafter reading) and Mathematics exams. During the years observed, fo urth grade test results were the primary criteria that the State Education Department used to determine elementary schools accountability status. We also examined differences in student, school, and teacher characteristics, as well as general education expenditures for Chancellors District schools, other SURR schools, and the citywide average, across all four years. Tables reporting changes over time on these variables are presented in the section below. Tables reporting cross-sectional me ans, standard deviations, and ranges for each variable analyzed are available from the authors. Study Sample As we indicate above, the Chancellors District as an intervention evolved over time, culminating in the implementation of the Model of Excellence in the 1999 school year. This evolution in design, and the extent of the change s in the Chancellors District as an intervention across time, posed analytical challenges for our evaluation. Not only were different schools inor outof the Chancellors District at different time s, but the instructional regimen imposed on those schools also varied across time. Our solution was to focus our analysis on the elementary schools that were state identified as low performing (SURR) in 1998 and entered the Chancellors District in 1999 and thus received the full intervention described in the pr evious section. We use their 1998 data as a baseline. As a comparison group, we use the othe r elementary schools on the SURR list in 1998. Because the number of middle schools that were on the SURR list in 1998 was too small to support an appropriate statistical analysis, our st udy focuses on elementary schools only. For the purpose of this analysis, elementary schools are all schools that included a fourth grade, regardless of their overall grade configuration.4 Category 1 schools outside the Chancellors District, described above, were not included in the analysis. Table 3 Sample Schools By Status, 1998 School Status 1998 1999 2000 2001 Chancellor's District schools Open 25 25 24 Closed 0 0 1 Other SURR schools Open 50 25 25 23 Closed 0 0 0 2 4 This includes nine schools spanning Kindergarten through Eighth Grade.
Education Policy Analysis Archives Vol. 13 No. 40 10 As Table 3 shows, half the elementary schools that were on the SURR list in 1998 entered the Chancellors District in 1999. Our univariate analysis presents data for the Chancellors District elementary schools that rema ined open for the four school years from 1998 99, the pre-implementation or baseline year, through 200102, and compares changes in performance and other variables in those schools to changes in other SURR schools.5 We present the citywide averages for elementary schools as an overall benchmark. Regression Models To assess the effect of the Chancellors District intervention on school performance in the context of other potential causal factors, we developed regression models that include controls for student characteristics (e.g., school-level English proficiency, poverty, attendance, student demographics), as well as school-level characteristi cs (e.g., school size, expenditures, certain teacher characteristics). Our dependent variables are school-level fourth-grade reading and math performance.6 Our basic analysis is outlined in the equation: Pst = a + 1Sst + 2CDst + 3Tt + SCHs + est (1) where Pst is the reading or math performa nce of the school s in year t, Sst is a vector of student and school characteristics, including school size, and CDst is the Chancellor s District dummy that takes a value of for the Chancellors District schools for years 1999, 2000 and 2001, and for the baseline year 1998. The comparison SU RR schools take for all years. Tt is a vector of year dummies, and SCH is a school fixed ef fect, essentiall y a vector of school dummies measuring the effects of unob served or unmeasured time-invariant school characteristics, such as school culture, leader ship, and other school-based factors affecting the implementation of the various SURR and Chancellors District components. est is an error term with the usual properties. To co rrect for heteroskedasti city, we employ robust standard errors. In order to estimate and control for the e ffect of additional resources on schools, we estimate a fuller model: Pst = a + 1Sst + 2CDst + 3Tt + 4TCHst + 5Rst +SCHs + est (2) where TCHst is a vector of human resour ces, including teacher characteristics and the number of teachers per 100 students, and Rst represents monetary resources (non-teacher per student 5 Schools that were closed during this time period were excluded from the univariate analysis because their inclusion draws the 1998-99 averages downwardmaking the Chancellors District appear artificially successful in the later years. Conversely, closed sch ools are included in the regression analysis, described below, because their exclusion, as demonstrated by pe rforming the analysis with and without these schools, would bias the Chancellors District coefficients upward. 6 Our models are based on ordinary least squares (OLS) analysis. A case could be made that, given the great variety in schools sizes, it would be more appropriate to employ weighted least squares (WLS) models and weight our estimates by school enrollment. However, we compared the findings that we report in this paper with those of a WLS regression analysis and fo und that, in spite of some differences in the size of some coefficients, the conclusions remained the sa me. Thus, and because our focus is the Chancellors Districts impact on schools rather than estimating the student populations performance, we rely on the OLS results for our assessment.
A Forced March for Failing Schools 11 expenditures). We analyze model (2) in reference to model (1), anticipating a change in the coefficient 2 once resources are intr oduced to the model. We included school fixed effects in the model to allow a difference-in-difference specification; that is, we take into account how sc hools differ from one another, and we estimate the impact of the Chancellors District over and above the general differences. This methodology offers a precise estimate of Chancellors District improvemen t, averaged across all four years, compared to improvement in other SURR schools, the closes t comparison group. SURR schools were also the beneficiaries of additional intervention and support. While these interventions were not as intensive as the many interventions of the Chancellors District, SURR schools thus present a moving target against which we measure the Chanc ellors District performance. Measurable Impact The goal of the Chancellors District was to increase the instructional capacity and the academic outcomes of the failing schools the district had incorporated. To assess whether, and to what extent, the Chancellors District achieved its goal, we compare the Chancellors District elementary schools to the other SURR elementary schoolsas well as to all New York City elementary schoolson school-level characteristics and fourth grade academic performance in the 1998 baseline year. We examine changes in Chancellors District, other SURR and the average New York City elementary schools between the 1998 and 2001 school years, after the target schools had spent three years in the Chancellors District. Finally, we report the results of our regression analyses that compare the academic pe rformance in the Chancellors District schools to the other SURR schools, while controlling for fa ctors other than the Chancellors District interventions, across all years. Table 4 Student Demographics in Sample and All NYC Schools, 1998 Demographic characteristics Chancellor's District Schools (N=25) Other SURR Schools (N=25) All NYC schools (N=666) % White 0.8 0.9 17.1 % Black 54.1 56.0 35.6 % Hispanic 43.2 41.6 36.9 % Asian/other 1.9 1.5 10.4 % Limited English proficient 16.9 15.4 14.7 % Recent immigrant 4.4 3.8 7.1 % Free lunch eligible 91.6 93.0 74.7 % Full time special education 8.0** 12.1 5.8 % In this school entire year 90.6** 87.7 91.5 % Days students attended 87.8 88.4 91.0 % Referrals to special education 4.3 4.2 3.6 % Part time special education 6.0 6.2 6.4 Student enrollment 715.9 760.1 795.9 p < .10; ** p < .05; *** p < .01.
Education Policy Analysis Archives Vol. 13 No. 40 12 Chancellors District and other SURR school s and the citywide average, 1998. In the 1998 baseline year, Chancellors District elementary schools and all other SURR elementary schools differed considerably from the average New York City elementary school across a variety of student and school characteristics. As Table 4 shows, both the Chancellors District schools and other SURR schools were somewhat smaller, much less white, considerably poorer, and had more special education students, but fewer i mmigrant students, than the average New York City elementary school. The Chancellors District schools and other SURR schools were quite similar to each other, with the important exceptions of the pe rcent of students in full time special education and the percent of students who remained in their school for the entire year. The other SURR schools had proportionally more students in full-time special education than the Chancellors District schools, and more th an twice as many as th e average New York City elementary school (12.1% vs. 5.8%). In addition, other SURR schools had a significantly lower proportion of students who remained in the school for the entire year than the Chancellors District schools; the latters percentage was much closer to the citywide average. Thus, Chancellors District schools had significantly fewer students who moved in or out of the school during the school year than the other SURR schools. In 1998, Chancellors District Scho ols and other SURR schools also differed considerably from the average New York City elem entary school in teacher resources and school expenditures. (See Table 5.) The Chancellors Di strict schools had the lowest level of teacher resources in the citylower percentages of fully licensed and experienced teachersand the least stable teaching force. Furthermore, Chancellors District schools spent less than other SURR schools, in terms of both teacher expenditures and total per student expenditures. As Table 5 indicates, the student performance outcomes of all Chancellors District schools and other SURR schools were considerably below the citywide average in the 1998 baseline year. Table 5 Mean teacher characteristics and school expenditures, 1998 Characteristics CD (N=25) Other SURR (N=25) All schools (N=666) % Licensed teachers 67.1* 72.6 81.5 % Taught 2+ yrs in this school 42.6 48.6 59.8 % Taught 5 or more years 49.1* 54.1 59.3 % Teachers with masters degrees 69.0 71.1 77.1 Teachers per 100 students 6.7 7.2 6.4 Per student expenditures $7,79 2.80** $8,537.20 $7,554.00 Per student spending on teachers $3,357.20** $3,777.30 $3,509.20 Non-teacher spending per student $4,435.70 $4,759.90 $4,044.70 p < .10; ** p < .05; *** p < .01. Expenditures are per student, for general education and part time special education students. Table 6 indicates that student performance in the Chancellors District schools did not differ much from performance in the other SURR school s. Chancellors District schools had a slightly lower percentage of students meeting the standard on the fourth grade reading test than the other SURR schools, and a slightly higher average scal e score on the fourth grade math tests. In both cases, the differences between the average scores were marginally significant. However, both reading
A Forced March for Failing Schools 13 and math performance in the Chancellors Di strict and other SURR schools were considerably below the average performance of a ll the citys elementary schools. Table 6 Mean fourth grade test scores, NYC, 1998 99 Characteristic CD schools (N=25) Other SURR (N = 25) All NYC (N=666) Mean reading scale score 606.5 607.5 628.0 % meeting reading std. 12.2* 15.2 33.3 % far below reading std. 38.6 38.5 20.8 Mean math scale score 614.2* 609.0 636.2 % meeting math std. 27.6 23.6 50.7 % far below math std. 34.0 37.7 18.5 p < .10; ** p < .05; *** p < .01. Thus, in the 1998 school year, Chancellors District and other SURR schools had much higher levels of student need, lower levels of teacher resources, and poorer student performance than the average elementary school in the New York City system. This pattern of high student need, poor teacher resources and poor student performance is what the Chancellor targeted for improvement through the takeover of failing schools and the imposition of the Model of Excellence. Changes in Chancellors Di strict and other SURR sch ools, 1998 to 2001 Student demographics in Chancellors District schools and other SURR schools remained fairly constant from the 1998 baseline year through the 2001 school years, with several important exceptions. (See Table 7.) During this period, the overall student population declined in both groups of schools, by 10% in Chancellors District schools and 7% in other SURR schools. The proportion of special education students declined as well. In 1998, the average percentage of students in full time special education in Chancellors District schools (8.1%) was higher than the citywide average (5.8%). By 2001 2, this percentage had decreased to 4.8%, very similar to the average New York City school (4 .6%). Other SURR schools experienced an even greater decline in the percentage of their students in full time special educationfrom 12.4% to 7.8%. However, even with that decline, the percenta ge of students in full time special education in the other SURR schools in 2002 was still much higher than the citywide average and the Chancellors District average. The difference between the percentage of full time special education students in Chancellors District and other SURR schools was highly significant in both 1998 and 2001. The percentage of students referred for special education evaluation in Chancellors District schools also declined by 1.3 percentage points, from 4.3% to 3.0%, between 1998 and 2001. By comparison, the referral rate in other SURR schools increased by 1.8 percentage points, from 4.0% to 5.8%. The difference between the change s in the two groups was highly significant.47 The citywide referral rate also increased, from 3.6% in 1998 to 4.1% in 2001. The proportion of students who were English language learners declin ed in both Chancellors District (4.9 percentage points) and other SURR schools (4.2 percentage poin ts), compared to a citywide decline of almost three percentage points.
Education Policy Analysis Archives Vol. 13 No. 40 14 Table 7 Change in mean student and school characteristics, 1998 to 2001 PreinterventionIntervention Difference 1998 1999 2000 2001 1999 Characteristic CD SURRCD SURRCD SURRCD SURR CD SURR% White 0.8 0.9 0.8 1.00.8 1.20 .8* 1.3 0.0***0.4 % Black 55.1 55.9 54.5 56.354.6 56.0 54.3 55.6 -0.8 -0.3% Hispanic 42.2 41.6 42.5 40.942.4 41. 242.7 41.3 0.5 -0.3% Asian/other 1.9 1.6 2.3 1.82.2 1. 72.3 1.8 0.4 0.3% Limited English 16.5 15.1 14.9 13.313.0 11.9 11.6 10.9 -4.9 -4.2% Recent immigrant 4.4 3.8 4.1 3.33.7 3.34.0 3.6 -0.4 -0.1% Free lunch eligible 91.6 92.8 89.3 91.887.2 90.0 87.2 90.0 -4.4 -2.8% Full time Special Ed 8.1** 12.4 7.9* 11.56.7 9.44.8**7.8 -3.3 -4.6% In school for year 90.5** 87.3 91.7**89.491.3***88. 390.6***87.1 0.1 -0.2% Attendance 87.9 88.3 88.5**89.489.3* 90.090.2 90.4 2.3 2.1% Special Ed. referrals 4.3 4.0 6.2 5.94.8 5.23.0***5.8 -1.3***1.8% Part time special Ed 6.1 6.2 6.3 6.15.8 5.75.0 5.6 -1.1 -0.7Mean enrollment 700.5 750.7 667.8 722.0660.6 713.0631.9 696.6 -68.7 -54.1 Differences between CD and other SURR schoo ls: p < .10; ** p < .05; *** p < .01. The most dramatic changes in the Chancellors District and other SURR schools occurred in resource provision. Table 8 shows a considerable im provement in the teacher resources of the other SURR schools, and an even more remarkable increase in the resources in Chancellors District schools. The formerly under-resourced Chancellors District schools were the beneficiaries of large increases in the number, quality, and stability of their teaching staffs. The Chancellors District schools also benefited from major increases in funding; their per student spending increased by $5,713 from 1998 to 2001, compared to an increase of $2,667 per student in other SURR schools during the same period. By contrast, the average New York City school saw a smaller $2,234 increase in per stud ent expenditures. The additional costs associated with the Chancellors Districts elementary schools reflect the implementation cost of the Model of Excellence in the Chancellors District schools. Most of this increased expenditure was for teachers. The implementation of the Model of Excellence in Chancellors District elementary schools not only reduced class size, but also provided at least four on-site staff developers in each school. Moreover, Chancellors District school expenditures also involved the cost of absorbing the salaries of ineffective teachers, as well as the 15% salary differential for the additional extende d time hours that teachers worked. These efforts brought Chancellors District schools, which had been lowest in expendit ures on teachers, well above all other schools. In the 1998 school year, there were 6.7 teachers for every 100 students in Chancellors District schools. This ratio increased by 1.9 teacher s, to 8.6 teachers per 100 students in the 2001 school year. By contrast, there were 7.1 teachers for every 100 students in other SURR schools in 1998, but that ratio increased by only 0.6t o 7.7 teachersin 2001. The increase in the number of teachers per 100 students in Chancellors District schools, probably a reflection of reduced class size and the increase in staff deve lopers, was highly significant, compared to the increase in the number of teachers per 100 studen ts in other SURR schools, as well as to the much smaller increase in the citywide average.
A Forced March for Failing Schools 15 Table 8 Change in mean teacher characteristics an d school expenditures, 1998 to 2001 Pre-interventionIntervention Difference 1998 1999 2000 2001 1999 Characteristic CD SURRCD SU RRCD SURR CD SURR CD SURR % Licensed teachers 67.2*73.071.5 69.291.1 86.893.4*89.7 26.2**16.7 % Taught 2+ years in this school 42.2*50.943.0*50.545.6** 53.054.8**62.6 12.6 11.7% Taught 5 or more years 49.054 49.0 47.545.2 44.142.8 44 -6.3 -10.0% Teachers with masters degrees 68.670.970.6 67.769.2 68.070.7 70.6 2.0 -0.3Teachers per 100 students 220.127.116.11 8.19.2 8.88.6**7.7 1.9**0.6Per pupil spending on teachers ($) 3,346***3,751 4,713***4,165 5,995*** 4,962 6,431***4,970 3,085***1,219 Per pupil spending ($) 7,808* 8,495 9,792 9,689 12,344*** 11,033 13,520***11,162 5,713***2,667 Differences between CD and other SURR schoo ls: p < .10; ** p < .05; *** p < .01. Expenditures are per student, for general education and PT special education students. Differences are calculated only for scho ols that existed in both 19989 and 20012002. In 1998, the percentage of licensed teachers in Chancellors District schools (67.2%) was significantly lower than in other SURR schools (73.0%). By 2001, the two groups relative positions reversed, and the percentage of licensed teachers in Chancellors District schools (93.4%) was significantly higher than in other SURR sc hools (89.7%). Chancellors District schools increased their lice nsed teachers by 26.2 percentage points in this three-year period, while other SURR schools increased their licensed teachers by 16. 7 percentage points. The increase in the percentage of licensed teachers in Chancellors District schools was highly significant, compared t o the increase in the percentage of licensed teachers in othe r SURR schools, as well to the citywide average. A third area of improvement was in the st ability of the teaching staff. In 1998, only 42.2% of teachers in Chancellors Dist rict schools, compared to 50.9% in other SURR schools, had been in th eir school for two or more years. While this statistic rose by 12.6 percentage points from 1998 to 2001 in Chancellors Dist rict schools, it rose by a similar amount (11.7 percentage points ) in other SURR schools. Although both Chancellors District and other SURR elementary sc hools experienced improvements in their overall funding and expenditure on teacher resources throughout the period, improvem ents in the Chancellors Distri ct schools were greater than in the other SURR schools. By 200102, Chancellors District schools total spendi ng and their spending on teach ers were much greater than o ther SURR schools, and Chancellors District schools had a higher number of teachers per student and a higher percentage of fully li censed teachers. This situation contrasted sharply with what had prevailed four years earlier.
Education Policy Analysis Archives Vol. 13 No. 40 16 There were also considerable changes in academic performance in the Chancellors District and other SURR schools from 1998 to 2001. As Table 9 shows, within those four school years, most of New York Citys SURR schools improved sufficiently to be removed from the states SURR list, a considerable achievement. Fifty-six pe rcent of Chancellors Dist rict schools and 60% of other SURR schools were remove d from the SURR list, a simila r pace of improvement for both groups of schools. Table 9 Change in SURR status in New York Ci tys SURR schools, 1998 to 2001 CD schools Other SURR schools NumberPercent NumberPercent Closed 1 4 2 8 Removed from the SURR list 14 56 15 60 Still on the SURR list 10 40 8 32 Total 25 100 25 100 Source: New York State Education Department Table 10 indicates that the percentage of fourth grade students in Chancellors District schools meeting the states reading standard increa sed significantly more than the percentage of fourth grade students in other SURR schools. In 19 98, a lower percentage of students met the reading standard in Chancellors District schools (12.3%) than in other SURR schools (15.3%). But by 2001, the two groups relative positions reve rsed; more students met the reading standard in Chancellors District schools (30%) than in othe r SURR schools (27.2%). The 18 percentage point improvement in the scores of Chancellors District schools is particularly strong. The citywide average for elementary schools across those years increased 14 percentage pointsfrom 33.4% of fourth grade students meeting the states readin g standard in 1998 to 47.8% in 2001. (See Figure 5.) Table 10 Change in mean fourth grade reading and math results, 1998 to 2001 Preintervention Intervention Difference 1998 1999 2000 2001 1999 Characteristic CD SURRCDSURRCD SURRCDSURR CD SURRMean reading scale score 606.8 607.4 613.2614.5619.6 615.5627.8 626.7 21.019.3 % meeting re ading std. 12.3 15.3 21.522.126.9**22.630.0 27.2 17.7**11.9% far below reading std. 38.3 39.0 32.634.326.3**32.121.7 21.8 -16.6-17.2Mean math scale score 614.3* 609.4 610.7612.1618.7 617.8626.0 623.5 11.814.1% meeting math std. 27.8 23.6 22.023.930.6 29.838.0 34.1 10.210.5% far below math std. 33.8 37.3 35.032.928.3 30 16.8 19.7 -17.0-17.7 Differences between CD and other SURR schoo ls: p < .10; ** p < .05; *** p < .01. Between the 1998 and 2001 school years, th ere were no significant differences in the change in math scores in Chancellors District schools compared to other SURR schools. However, the pattern of change in math performance is more complex. While both groups improved over these years, Chancellors District performance, which was significantly higher than the performance of SURR schools in 1998, declined in 1999 00 and recovered over the next two years. Math
A Forced March for Failing Schools 17 performance in other SURR schools, by contrast, had a continuously positive upward trajectory. The end result is that the math difference between the two groups remained essentially the same. The findings of our univariate analyses suggest that the Chancellors District schools improved their students reading skills more than the other SURR schools. They also show that, although both groups of schools experienced importa nt improvements in the number and quality of their teaching staff and their expenditure levels these changes were much more pronounced in Chancellors District schools than in other SURR schools. However, the univariate analysis does not provide insight into how much of this improvement can be attributed to the Chancellors District inte rvention as opposed to the possible effects of other factors, such as changes in the composition of th e schools student population s. It is particularly important to control for student characteristics such as special educat ion or limited English proficiency, because the proportions of students in these categories were sharply re duced across the years of the analysis. Similarly, gi ven the dramatic improvement in funding, teacher to student ratios and teacher quality, it is important to determine how much of the change in student performance can be attributed to improved fu nding and teacher resources. To examine these issues while disentangling the effects of the different factors involved, the next section presents the regression analyses carried out to determine if the patterns in school-level performance remain after controlling for differences in student, school and teacher characteristics, as well as school expenditures. Regression Analysis Table 11 displays the estimated differences in academic performance, controlling for student, school, and teacher characteristics, as well as perstudent expenditures, between Chancellors District schools and other SURR schools. A positive coefficien t in the regressions estimating effects on the percent of students scoring at or above the st ate reading and math standards (Levels 3 and 4) indicates a positive association with student perf ormance. Conversely, a positive coefficient in the regressions predicting the percent far below the state reading and math standards indicates a negative association with student performance. The coefficients suggest that, when we control for student and school characteristics, student performance on the fourth grade state read ing test is significantly better in Chancellors District schools than in other SURR schools. This is reflected in the higher average percent of students scoring at or above the standard, and in the lower percent of students scoring far below the standard. The influence of the other variables varies across models, depending on whether the model controls for resources or not. Moreover, the coefficient on the year dummies is in many cases significant, and becomes larger in the more recen t years, reflecting an overall pattern of increasing achievement in student performance ac ross the entire sample of schools. The Chancellors Districts effect on student performance on the fourth grade math test is not as encouraging. As the regression table indicates, Chancellors District schools do not differ significantly from other SURR schools on the percent of students scoring at Levels 3 and 4, or at Level 1. The failure to significantly improve math scores in the Chancellors District may be a direct result of the much more intensive curricular an d scheduling focus on improving reading skills. Or the reading skills, and scores, of the Chancellors District students may have improved at the expense of their math scores.
Education Policy Analysis Archives Vol. 13 No. 40 18 Table 11 Effects on fourth grade academic performance, without (Model 1) and with (Model 2) resources % meeting reading standard % reading far below standard % meeting math standard % far below math standard Characteristic Model 1 Model 2 Model 1Model 2Model 1Model 2 Model 1 Model 2 Chancellor's District 5.706*** (2.043) 5.898** (2.140) -4.299* (2.308) -3.944* (2.334) -1.842 (3.066) -2.314 (3.159) 1.615 (2.736) 2.273 (2.948) Student & school % Black -1.826 (1.517) -2.850* (1.607) -0.338 (1.483) 0.128 (1.566) 0.080 (1.814) -1.133 (1.718) -0.791 (2.064) -0.024 (2.224) % Hispanic -2.396 (1.529) -3.325** (1.608) 0.028 (1.516) 0.512 (1.555) -0.820 (1.885) -1.888 (1.798) -0.456 (2.089) 0.148 2.220) % Asian or other -1.150 (1.711) -2.550 (1.783) 0.608 (1.826) 1.129 (1.933) -0.004 (2.260) -1.803 (2.179) 0.350 (2.231) 1.198 (2.458) % Limited English 0.214 (0.260) 0.307 (0.285) -0.008 (0.277) -0.135 (0.307) -0.083 (0.326) 0.073 (0.353) 0.222 (0.290) 0.133 (0.312) % Recent immigrant -1.228* (0.636) -1.259** (0.629) 0.704 (0.610) 0.821 (0.625) 0.408 (0.782) 0.374 (0.776) -0.091 (0.799) 0.040 (0.772) % Free lunch eligible -0.043 (0.095) -0.077 (0.095) -0.132 (0.100) -0.136 (0.107) -0.302** (0.141) -0.288* (0.151) 0.114 (0.137) 0.087 (0.140) % Full-time special ed. -0.092 (0.210) -0.061 (0.213) 0.706*** (0.223) 0.639*** (0.240) -0.453* (0.262) -0.428 (0.282) 0.539** (0.226) 0.520** (0.234) % In school entire year -0.083 (0.218) -0.073 (0.228) 0.039 (0.227) 0.075 (0.238) 0.007 (0.306) 0.055 (0.306) 0.2404 (0.272) 0.301 (0.270) % Attendance -0.043 (0.737) -0.232 (0.715) 0.166 (0.771) 0.511 (0.795) 0.116 (1.001) -0.290 (0.941) -0.083 (0.967) 0.364 (0.907) % Special ed. referrals -0.146 (0.265) -0.050 (0.279) -0.624* (0.344) -0.708** (0.349) 0.190 (0.356) 0.273 (0.357) -0.235 (0.314) -0.384 (0.316) % Part-time special ed. -0.175 (0.521) -0.435 (0.553) -0.111 (0.561) -0.124 (0.588) 0.044 (0.612) -0.074 (0.621) 0.844 (0.653) 1.266* (0.652) Students -3.390 (5.469) 6.984 (7.729) -4.818 (6.797) -15.263 (11.053) -12.721* (7.093) 0.697 (10.701) 11.197* (5.867) -9.215 (8.894) Year 4.454** 4.200** -1.184 -1.067 -4.116* -4.927** -0.812 1.051 2000 (-1.75) (-1.906) (-1.991) (-2.061) (-2.191) (-2.44) (-2.213) (-2.386) 7.392*** 4.312 -6.203***-2.488 2.114 -4.203 -4.281 2.847 2001 (-2.056) (-2.932) (-2.207) (-2.899) (-2.721) (-3.382) (-2.598) (-3.029) 11.248** 7.523** -13.03***-8.825** *6.973**0.289 13.286*** -6.068 2002 (-2.633) (-3.414) (-2.503) (-3.221) (-3.266) (-3.986) (-3.236) (-3.75)
A Forced March for Failing Schools 19 % meeting reading standard % reading far below standard % meeting math standard % far below math standard Characteristic Model 1 Model 2 Model 1Model 2Model 1Model 2 Model 1 Model 2 Resources 0.052 -0.108 0.155 -0.102 % Licensed teachers (-0.101) (-0.111) (-0.125) (-0.12) 0.106** -0.028 0.155** -0.055 % Taught 2+ yrs in school (-0.05) (-0.068) (-0.067) (-0.071) 0.083 -0.176 0.207 % Taught 5+ years 0.008 (-0.115) (-0.122) (-0.144) (-0.148) -0.199* 0.111 -0.119 0.176 % with masters degrees (-0.109) (-0.136) (-0.147) (-0.136) 0.043 0.181 -1.05 -0.784 Teachers per 100 students (-0.617) (-0.701) (-0.721) (-0.753) 1.226 -1.597* 0.721 -2.178***Non-teacher expenditures (-0.77) (-0.85) (-0.953) (-0.792) 239.5 347.6** 46.1 -22.5 85.6 207.4 52.1 -42.9 Constant (-159.7) (-165.1) (-154.0) (-162.3) (-186. 8)(-176.5) (-213. 6) (-221.4) N 197 195 197 195 197 195 197 195 R-squared 0.70 0.72 0.69 0.71 0.64 0.68 0.67 0.70 F-stat (Resources) 1.69 (-0.129) 1.02 (-0.42) 2.31** (-0.04) 3.14** (-0.01) F-statistic for school fixed effects 2.044** (-0.001) 2.251*** (0.000) 1.793** (-0.005) 1.818** (-0.004) 2.043** (-0.001) 2.297*** (0.000) 1.434* (-0.056) 1.706** (-0.010) p < .10; ** p < .05; *** p < .01. Equally important, Model 2 assesses the e ffect of resources on reading and math performance. Teacher stability has a positive effect on the percent of students meeting the standard, suggesting the importance of a well-supported teaching staff. Non-teacher expenditures have a negative effect on the percent far below the stan dard, suggesting the positive effect of general resources. These results are intuitive. However, the fact that the effects of the other resource variables are not consistent across models, and the resource variables are jointly significant only in the two math models presented (F=2.31 and 3.14, probably for different reasons) reflect a generally weak, counterintuitive relationship between resources and performance. This finding is consistent with the reverse causality problem in education prod uction, in which resources tend to be negatively correlated with performance, due to the high correlation between categorical expenditures, such as title 1 funds, and student needwhich in turn are negatively correlated with performance. In comparing Model 1a basic model in which teacher characteristics and expenditures are excludedwith a more complete model controlling fo r teacher characteristics and expenditures, we
Education Policy Analysis Archives Vol. 13 No. 40 20 find that the results are essentially the same. In theory, in Model 1 the Chancellors District coefficient could be inflated by the districts hu ge resource advantage versus other SURR schools. If that were the case, we would expect the coefficients for Chancellors District dummyand other variablesto be radically different (i.e., smalle r) in Model 2, when we control for resources and teacher characteristics. That Model 2 shows a substantively unchanged Chancellors District coefficient suggests that the Chancellors Distri ct effect was not simply a matter of increased funding and teacher characteristics. The positive e ffect of the Chancellors District may be tied to enhanced administrative support, more efficient use of instructional resources, and other factors. However, without detailed implementation data, it is impossible to disentangle which parts of the intervention were more causal. Overall, these regression results reiterate the univariate findings in school-level performanceon average, the Chancellors District schools performed significantly higher in reading performance during the years these schools were under the centralized improvement regimen described above, but did not show much progress in math. The positive regression coefficients in reading suggest a significant improvement for the Chancellors District schools, relative to where these schools would have been and relative to comparable schools. These results are consistent when resources are added to the models. Revisiting the twin goals of our analysis we assess whether the Chancellors District intervention increased schools instructional capa city and academic outcomes. Across the 1998 through 2001 school years, the Chancellors Di strict schools sustained higher student stability rates, increased teacher resources, and substantially increased per student expenditures, compared to both other SURR schools and the citywide average. Moreover, holding student characteristics, teacher characteristics and expenditures constant the Chancellors District schools increased their fourth grade reading performance by considerably more than the other SURR schools. This finding suggests that, at the elementary school level, the Chancellors District as an intervention succeeded in improving the reading outcomes, though not the math outcomes, of its schools and students. Conclusion The Chancellors District, as a unique initiative in centrally-driven school improvement, represents a signal intervention into New York City school governance and administration. When the Chancellor, in 1996, invoked a previously unexercised power to take failing elementary and middle schools from their sub-district jurisdicti ons, he did what no other New York City schools chief had ever attempted. He proceeded to create a special, non-geographic district that eventually encompassed 58 failing schools, and developed a seri es of organizational, curricular, instructional, and personnel interventions, mandated for all the dist ricts schools, to jump-start their improvement. Thus the Chancellors District effort represents an historic departure from three decades of central school system tolerance of local sub-district instructional failure. The Chancellors District initiative challenges several traditions of policy analysis about the relationship between district administration and school change. Its theory of change counters reigning theories about the stultifying weight of urban education bureaucracies, the inability of loosely coupled systems to sustain centrally-dri ven change, and the dichotomy between what bureaucratic systems impose and the autonomy successful schools require. Thus the Chancellors District effort may represent a return to more tr aditional notions of centralized management, or a harbinger of the newly emerging emphasis on the di strict as the necessary locus of school change. But where it departs from this debate is as a demons tration that a top down approach, if attached to
A Forced March for Failing Schools 21 resources and a set of appropriate reforms rather th an the mere fact of greater centralization, could yield positive results. How the Chancellors District initiative is ultimately assessed in the history of urban education reform depends primarily on the outco mes of the effort. Our findings suggest two categories of results. First, our univariate analys is demonstrates that the Chancellors District intervention significantly increased teacher resources and per-student expenditure across the districts schools, and significantly increased the percentage of students meeting the standard on the fourth grade state reading tests, compared to the outcomes of other SURR schools. Second, our regression analyses demonstrate th at when the Chancellors District schools are compared to the other SURR schools (the schools most similar to those in the Chancellors District) and when the analyses control for student and school characteristics, teacher resources and perstudent expenditures, the Chancellors District sc hools do significantly better than other SURR schools in reading, but not in math. But given that the major curricular, instructional and organizational interventions of the Chancellors District focused intensively on improving student literacy, these outcomes suggest that the Chance llors District had begun to achieve one of its primary student achievement goals. The eventual impact of these gains in math performance is yet to be determined. It is important to note that the districts upward curve in reading outcomes still left the Chancellors District schools quite far below the citywi de average, though the initiative was clearly narrowing the gap. It is also important that the Chancellors District initiative we evaluated represents only those three academic years of effort, from 1999 to 2001, in which the components of the Model of Excellence were im plemented in the districts schools. Had the initiative not been terminated in 2003, would the upward curve of reading achievement have continued to rise? Would the math achievement th at began to accelerate in 1999 have continued upward? Our data do not allow us to speculate. Both the Chancellors District and other SURR schools seem to have benefited from increases in teacher resources as well as over all expenditures. The Chancellors District schools received significantly more resources than the other SURR schools, which in turn received significantly more than the city schools as a whole. But when we control for the effects of teacher resources and per student expenditures, the Chanc ellors Districts elementary schools still perform significantly better than the other SURR schools in reading. Thus, something was working to improve outcomes in the Chancellors District schools that is not explained by increases in teacher resources or school-level expenditures. We cannot define what that something is, other than to point to the set of components that comprised the Chancellors District intervention. Because our evaluation was retrospective, we cannot specify what components of the intervent ion helped to produce the reading gains our findings demonstrate. Future research, perhaps be nefiting from more intensive implementation data, can make the connection between gains for particul arly successful or unsuccessful schools and the specific levels of interventions imposed. But it is important to reiterate that the Chancellors District took over some of the citys least well-reso urced schools serving the citys poorest and lowest performing students. By developing, mandatin g and implementing a comprehensive set of organizational, curricular, instructional and personnel changes, the Chancellors District significantly improved the reading outcomes of the students in those schools, in three years of focused effort. This is not a small accomplishment. Whether th e additional resources expended, in both teacher resources and per student expenditures, were ulti mately worth the extent of improved achievement the Chancellors District initiative generated, is a complex but essential question that our subsequent research will attempt to answer.
Education Policy Analysis Archives Vol. 13 No. 40 22 References Ascher, C., Fruchter, N., & Ikeda, K. (1999). Schools in context: Final report to the New York State Education Department 1997 1998, An analysis of SURR schools and their districts New York: Institute for Education and Social Policy. Bodilly, S. J. (2001). New American Schools concept of break the mold designs: How designs evolved and why. Santa Monica, CA: RAND, MRNAS. Chubb, J. E. & Moe, T. M. (1990). Politics, markets and Americas schools Washington, D.C.: The Brookings Institution. Domanico, R. (1994). Undoing the failure of larg e school systems: Policy options for school autonomy. Journal of Negro Education, 63, 19. Hallett, A. (Ed.). (1995). Reinventing central office: A primer for successful schools. Chicago: Cross City Campaign for Urban School Reform. Iatarola, P. (2001). IESP policy brief: Distributing teacher qua lity equitably: The case of New York City. New York: Institute for Education and Social Policy. Iatarola, P., & Fruchter, N. (200 4). District effectiveness: A study of investment strategies in New York City public schools and districts. Educational Policy 18, 491. Mintrop, H. (2003). The limits of sanctions in low-performing sc hools: A study of Maryland and Kentucky schools on probation. Education Policy Analysis Archives, 11(3). Retrieved September 27, 2005, from http ://epaa.asu.ed u/epaa/v11n3/. New York City Board of Education. (1 998a). Annual school reports 1998 2001 New York: New York City Board of Education. New York City Board of Education. (1 998b). School-based expenditure reports, 1998 2001. New York: New York City Board of Education. New York City Board of Education. (1999a). Budget and operations review memorandum #1 FY00. New York: New York City Board of Education. New York City Board of Education. (1999b). Chan ce llors District: A model of excellence for extended time schools, 1999 2000. New York: New York City Board of Education. New York City Board of Education. (2000). Flash report #1. Analy ses of performance of extended-time and non-extended time SURR schools. New York: New York City Board of Education. New York City Board of Education. (2001a). Chancellors District: A model of excellence: 2001 2002. New York: New York City Board of Education. New York City Board of Education. (2001b). Fact sheet I, Teachers of tomorrow program, 2000 2001. New York: N ew York City Board of Education.
A Forced March for Failing Schools 23 New York City Board of Education. (2002). Flash report #7. Year two analyses of performance of extended-time and non-extend ed t ime SURR schools New York: New York City Board of Education. New York City Board of Education. (n.d.). Corrective action plan: A citywide implementation framework for redesign schools. New York: New York City Board of Education. Resnick, L.B. & Glennan, T. K. (2002). Leadersh ip for learning: A theory of action for urban school districts. In A.M. Hightower, M.S. Knapp, J.A. Marsh & M.W. McLaughlin (Eds.). School districts and instructional renewal (pp.160). New York: Teachers College Press. Sarason, S. (1996). Revisiting the culture of the sc hool and the problem of change. New York: Teachers College Press. Snipes, J., Doolittle, F. & Herlihy, C. (2002). Foundations for success: Ca se studies of how urban school systems improv e student achievement New York: MDRC for the Council of the Great City Schools. Weick, K. E. (1976). Educ ational organizations as loosely coupled systems. Administrative Science Quarterly, 21, 1.
Education Policy Analysis Archives Vol. 13 No. 40 24 About the Authors Deinya Phenix Dorothy Siegel Ariel Zaltsman Norm Fruchter New York University, Steinh ardt School of Education Email: email@example.com Deinya Phenix is a Research Scientist at the New York University Institute for Education and Social Policy. Her research interests include urban education policy and social geography. Dorothy Siegel is a Senior Project Director at th e New York University Institute for Education and Social Policy. She is currently faci litating the development of a citywide inclusion program for children with Autism Sp ectrum Disorders in New York City. Ariel Zaltsman is a Doctoral Candidate in Public Administration at the R.F. Wagner School of Public Service, NYU. His research interests program evalua tion, performance-based budgeting, and social policy. Norm Fruchter is Clinical Professor of Educati on Policy at New Yo rk Universitys Steinhardt School of Education and Director of the Institute for Educati on and Social Policy. He is co-author of Choosing Equality: The Case for Democrat ic Schooling and Hard Lessons: Public Schools and Privatization.
A Forced March for Failing Schools 25 EDUCATION POLICY ANALYSIS ARCHIVES http://epaa.asu.edu Editor: Sherman Dorn, University of South Florida Production Assistant: Chris Murre ll, Arizona State University General questions about ap propriateness of topics or particular articles may be addressed to the Editor, Sherman Dorn, firstname.lastname@example.org. Editorial Board Michael W. Apple University of Wisconsin David C. Berliner Arizona State University Greg Camilli Rutgers University Casey Cobb University of Connecticut Linda Darling-Hammond Stanford University Mark E. Fetler California Commission on Teacher Credentialing Gustavo E. Fischman Arizona State Univeristy Richard Garlikov Birmingham, Alabama Gene V Glass Arizona State University Thomas F. Green Syracuse University Aimee Howley Ohio University Craig B. Howley Appalachia Educational Laboratory William Hunter University of Ontario Institute of Technology Patricia Fey Jarvis Seattle, Washington Daniel Kalls Ume University Benjamin Levin University of Manitoba Thomas Mauhs-Pugh Green Mountain College Les McLean University of Toronto Heinrich Mintrop University of California, Berkeley Michele Moses Arizona State University Anthony G. Rud Jr. Purdue University Michael Scriven Western Michigan University Terrence G. Wiley Arizona State University John Willinsky University of British Columbia
Education Policy Analysis Archives Vol. 13 No. 40 26 EDUCATION POLICY ANALYSIS ARCHIVES English-language Graduate -Student Editorial Board Noga Admon New York University Jessica Allen University of Colorado Cheryl Aman University of British Columbia Anne Black University of Connecticut Marisa Burian-Fitzgerald Michigan State University Chad d'Entremont Teachers College Columbia University Carol Da Silva Harvard University Tara Donahue Michigan State University Camille Farrington University of Illinois Chicago Chris Frey Indiana University Amy Garrett Dikkers University of Minnesota Misty Ginicola Yale University Jake Gross Indiana University Hee Kyung Hong Loyola University Chicago Jennifer Lloyd University of British Columbia Heather Lord Yale University Shereeza Mohammed Florida Atlantic University Ben Superfine University of Michigan John Weathers University of Pennsylvania Kyo Yamashiro University of California Los Angeles
A Forced March for Failing Schools 27 Archivos Analticos de Polticas Educativas Associate Editors Gustavo E. Fischman & Pablo Gentili Arizona State University & Universidade do Estado do Rio de Janeiro Founding Associate Editor for Spanish Language (1998003) Roberto Rodrguez Gmez Editorial Board Hugo Aboites Universidad Autnoma Metropolitana-Xochimilco Adrin Acosta Universidad de Guadalajara Mxico Claudio Almonacid Avila Universidad Metropolitana de Ciencias de la Educacin, Chile Dalila Andrade de Oliveira Universidade Federal de Minas Gerais, Belo Horizonte, Brasil Alejandra Birgin Ministerio de Educacin, Argentina Teresa Bracho Centro de Investigacin y Docencia Econmica-CIDE Alejandro Canales Universidad Nacional Autnoma de Mxico Ursula Casanova Arizona State University, Tempe, Arizona Sigfredo Chiroque Instituto de Pedagoga Popular, Per Erwin Epstein Loyola University, Chicago, Illinois Mariano Fernndez Enguita Universidad de Salamanca. Espaa Gaudncio Frigotto Universidade Estadual do Rio de Janeiro, Brasil Rollin Kent Universidad Autnoma de Puebla. Puebla, Mxico Walter Kohan Universidade Estadual do Rio de Janeiro, Brasil Roberto Leher Universidade Estadual do Rio de Janeiro, Brasil Daniel C. Levy University at Albany, SUNY, Albany, New York Nilma Limo Gomes Universidade Federal de Minas Gerais, Belo Horizonte Pia Lindquist Wong California State University, Sacramento, California Mara Loreto Egaa Programa Interdisciplinario de Investigacin en Educacin Mariano Narodowski Universidad To rcuato Di Tella, Argentina Iolanda de Oliveira Universidade Federal Fluminense, Brasil Grover Pango Foro Latinoamericano de Polticas Educativas, Per Vanilda Paiva Universidade Estadual Do Rio De Janeiro, Brasil Miguel Pereira Catedratico Un iversidad de Granada, Espaa Angel Ignacio Prez Gmez Universidad de Mlaga Mnica Pini Universidad Nacional de San Martin, Argentina Romualdo Portella do Oliveira Universidade de So Paulo Diana Rhoten Social Science Research Council, New York, New York Jos Gimeno Sacristn Universidad de Valencia, Espaa Daniel Schugurensky Ontario Institute for Studies in Education, Canada Susan Street Centro de Investigaciones y Estudios Superiores en Antropologia Social Occidente, Guadalajara, Mxico Nelly P. Stromquist University of Southern California, Los Angeles, California Daniel Suarez Laboratorio de Politicas Publicas-Universidad de Buenos Aires, Argentina Antonio Teodoro Universidade Lusfona Lisboa, Carlos A. Torres UCLA Jurjo Torres Santom Universidad de la Corua, Espaa