USF Libraries
USF Digital Collections

Educational policy analysis archives


Material Information

Educational policy analysis archives
Physical Description:
Arizona State University
University of South Florida
Arizona State University
University of South Florida.
Place of Publication:
Tempe, Ariz
Tampa, Fla
Publication Date:


Subjects / Keywords:
Education -- Research -- Periodicals   ( lcsh )
non-fiction   ( marcgt )
serial   ( sobekcm )

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
usfldc doi - E11-00325
usfldc handle - e11.325
System ID:

This item is only available as the following downloads:

Full Text
xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam a22 u 4500
controlfield tag 008 c20039999azu 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E11-00325
0 245
Educational policy analysis archives.
n Vol. 11, no. 27 (August 07, 2003).
Tempe, Ariz. :
b Arizona State University ;
Tampa, Fla. :
University of South Florida.
c August 07, 2003
Research or cheerleading? Scholarship on Community School District 2, New York City / Lois Weiner.
x Research
v Periodicals.
2 710
Arizona State University.
University of South Florida.
1 773
t Education Policy Analysis Archives (EPAA)
4 856


1 of 20 A peer-reviewed scholarly journal Editor: Gene V Glass College of Education Arizona State University Copyright is retained by the first or sole author, who grants right of first publication to the EDUCATION POLICY ANALYSIS ARCHIVES EPAA is a project of the Education Policy Studies Laboratory. Articles appearing in EPAA are abstracted in the Current Index to Journals in Education by the ERIC Clearinghouse on Assessment and Evaluation and are permanently archived in Resources in Education Volume 11 Number 27August 7, 2003ISSN 1068-2341Research or “Cheerleading”? Scholarship on Community School District 2, New Yor k City Lois Weiner New Jersey City UniversityCitation: Weiner, L. (2003, August 7). Research or “cheerleading”? Scholarship on Community School District 2, New York City. Education Policy Analysis Archives, 11 (27). Retrieved [date] from a/v11n27/. Related article: Vol. 11 No. 28 AbstractThis article examines data on student achievement a nd school demographics not explored by the researchers who ha ve promoted Community School District 2 (CSD 2) as a m odel of urban school reform that should be replicated elsew here. Data on achievement indicate a remarkable degree of social and racial stratification among CSD 2's schools and levels of achievement that closely correlate with race, ethnicity, and po verty. In addition, when CSD 2's scores on state and city tests of math ematics are compared with results from CSD 25 in Queens, a scho ol district that serves a population demographically similar, t he superiority of its functioning becomes questionable. The articl e explains why the design of research on CSD 2 illustrates the per ils to both


2 of 20 research and policy when university-based researche rs assume the role of “cheerleader” (Cuban, 1988), promoting reforms they have aided in implementing and assessing. The direct assistance that university-based researc hers provide to school systems involved in reform is generally accepted as positiv e, strengthening the relationship between theory and practice and in the process impr oving both. However, as Cuban (1988) notes in his “word to the wise” about researchers advising policymakers, “the legacy of disappointment with re searchers who have been cheerleaders for this or that approach to be used i n classrooms is not one to be envied” (p. 293). In this study I explain how refor m in Community School District 2 in Manhattan and researchers’ roles in advising, as sessing, and promoting the reform model adopted by CSD 2 and teacher union off icials illustrate the perils to education and research when researchers become “che erleaders” for reforms. (Note 1) Background and Significance of the StudyIn debates about how to improve student achievement in urban school districts, Community School District 2's (CSD 2) strategy of u sing professional development to implement national standards has been put forwar d as exemplary, an illustration of public education's viability in the nation's cit ies (Elmore, 1999-2000; Elmore & Burney, 1997a; Elmore & Burney, 1997b; Elmore & Bur ney, 1999; Elmore & Burney, February1999). Research reports have herald ed CSD 2's focus on a centralized system of professional development link ed to national standards as being the key to improving achievement in urban sch ools (Fink & Resnick, 1999; Resnick & Harwell, 2000; Stein, D’Amico & Johnstone April 1999). CSD 2's model is described by its most recent Superintendent as “ delivering a world class education for every student through a redesigned la bor management system that supports high performance learning communities util izing the New Standards ‘performance standards’ along with city and state a ssessments” (Harwayne, 2000). Typical of the commendation of CSD 2 produced by re searchers who have aided the district is this description: Over an eleven-year period, Community School Distri ct Two in New York City has amassed a strong record of successful school improvement in a very diverse urban school setting. Not only have test scores risen, but there is also a remarkable profes sional spirit among the teachers, principals, and central staff members of the district, which has 22,000 students in 45 schools (Fink & Resnick, 1999, p 3). Publications ranging from the TheWall Street Journal to the monthly magazine of the American Federation of Teachers (AFT) have prai sed the model. In this article I scrutinize evidence for the claims that the model h as been successful and that it can and should be replicated in other urban school systems. Scholarship on CSD 2 that promotes its success identifies the district a nd key personnel by name; my discussion adopts the same protocol. My rationale f or naming the district and researchers is that commendations of the model have linked it explicitly to CSD 2 officials and have created a legitimacy, among poli ticians and in the media. The model has attracted attention even beyond urban sch ool districts. One high-ranking state education official in Vermont pressed for CSD 2's curricular practices and


3 of 20 professional development model to be adopted there (personal communication with Vermont college administrator, April 2001).Sources of the Data for the StudyI examine reports of researchers who have worked wi th CSD 2 officials to design, implement, and assess the district’s reforms. I als o draw on publicly available data on school achievement and demographics published in The New York Times and on the New York State Department of Education and N ew York City Board of Education websites To my knowledge although these data are easily accessible, they have not been used heretofore to compare stude nt achievement in CSD 2 with achievement in other districts with comparable demo graphics. Other data sources I use are personal correspondence with CSD 2 teachers copies of memos sent to faculty by school administrators in CSD 2, an unpub lished report on CSD 2's math curriculum produced by a group of parents and mathe matics professors at New York University, and field notes following conversa tions with principals, teachers union officials and teachers in CSD 2 schools.All teachers and school administrators employed in CSD 2 who spoke and corresponded with me were informed beforehand that I might use the information they provided in a published study. In each case th e people I interviewed or who provided me with memos did so on the condition that they remain anonymous as the source of information. All cited fear of repris als from supervisors as the reason for confidentiality. I gathered information from fo ur administrators (in three schools), ten teachers (in three schools), and four people ho lding elected positions in the teachers union, the United Federation of Teachers ( UFT). My access to informants in CSD 2 was facilitated by contacts I made in the course of my participation as a parent-activist in the Joh n Melser Charrette School, or as it is more commonly called, PS 3 (Weiner, 2002). PS 3 is an arts-based, alternative school started by parents, with a unique history th at makes it simultaneously a “school of choice” and a regular zoned school for t he neighborhood, Greenwich Village (Zuckerman, 2001). PS 3 is probably one of the “off the screen” schools that CSD 2 officials identify as “not working within the District #2 framework. While student achievement in some cases is fairly strong, the district leadership has concerns about the quality of instruction and or le adership in these schools” (D’Amico, van den Heuvel, & Harwell, 2000, p. 6). M y initial examination of research on achievement in CSD 2 stemmed from my in terest in understanding if PS 3 was considered “off the screen” and if so, why According to published test scores, PS 3 maintained the same level of achieveme nt as the other school serving Greenwich Village, and I wanted to know the source of CSD 2 officials’ concerns about its instruction or leadership.However, the focus and scope of my inquiry changed after my preliminary look at data on achievement in other schools in CSD 2 and N ew York City. The overwhelming presence of racial and social segregat ion in CSD 2 schools and the correlations between segregation and low achievemen t levels prompted me to examine the design and conduct of research about th e district’s success in boosting student achievement. Both investigations a re discussed in this article. The first section deals with the context in which refor ms and research were formulated and the reasons critical questions were ignored; th e second segment examines


4 of 20 data (presented in tables 1-5) that suggest why the CSD 2 model may not be as successful as it has been promoted as being, as wel l as reasons that it may not be replicable in most cities.Insular, Self-referential Research DesignFrom the start, critical perspectives on CSD 2 were omitted from research on its system of professional development, as this descrip tion indicates: We were trying to figure out which people in the di strict should be interviewed and observed in order to understand how the district functioned. Someone started to diagram the way in w hich teachers were expected to learn from principals and professional developers and each other within their school, while at the same time p rincipals were expected to learn from the Superintendent and Deput y and from each other how to be better at their instructional leade rship job. Someone else said, “It’s like those nesting dolls people li ke to bring back from their travels”—and the name was born. The image seems to work because the dolls are each independent, free-standing “peop le,” yet they share a common form—and you can’t decide which is the most “important” doll, the tiny one in the middle that establishes the sha pe for them all or the big one on the outside that encloses them all (Fink & Resnick, 1999: p. 6). The study design supports an analogy (of nesting do lls) that seems not to be seen as simultaneously hierarchical and exclusively self -referential. The analogy is also remarkable for being static and decontextualized; r elationships among the “dolls” are unaffected by “outside” influences such as pare nt feedback or critical perspectives that might be provided by teachers or principals who disagree with the superintendent and by other researchers, people who do not fit into the nest. Resnick observes that the doll in the middle is “ti ny” and “the big one on the outside...encloses them all.” One way to view the n ested dolls as Resnick suggests is that they share a common form so it is not clear where the power to shape the relationship resides. But another view is certainly possible, that the outer doll shapes the configuration, and size and power dimini sh as one moves to the inside of the “nest.” Support for this latter interpretati on comes from Resnick’s description of hierarchical power relations in CSD 2, that prin cipals learn from the Superintendent and Deputy, and teachers from princi pals and professional developers and peers. (The addition of “peers” in t his description is interesting because it does not fit the “nesting doll” analogy. ) However, the research design did not address the possibility that differences in pow er and status among the nesting dolls corresponded to historically derived bureaucr atic relations in urban school systems (Tyack, 1974) that have been identified by a considerable amount of research, for instance Freedman (1987) and Knapp (1 995), as engendering teacher dissatisfaction and “burnout.”One result of the self-referential nature of the re search design, clear from the “nesting doll” analogy that was adopted, was lack o f attention to the controversies that have roiled in CSD 2, especially centering on the math curriculum that is mandated in every school. An unpublished report of a group of parents and professors organized to oppose the District’s math curriculum, written by a


5 of 20 mathematics professor at New York University, noted that parent and teacher dissatisfaction with the curriculum and its inflexi ble implementation were significant. My interviews with teachers confirmed the NYU repor t’s finding that there was widespread fear that teachers would be disciplined if they supplemented the mandated curriculum with other materials to prepare students for city-wide tests. District officials refused to approve any orders fo r math workbooks to be used as supplements to the official materials. In some scho ols, teachers resorted to photocopying entire books for their classes, with t acit support of principals who turned a blind eye to the practice. (Note 2) Despite its reputation as a powerful political forc e, the United Federation of Teachers had not positioned itself as an advocate f or teachers, who expressed fears of being disciplined for not “toeing the line ” with regard to CSD 2 curricular and instructional mandates. On the contrary, the UF T’s parent organization, the American Federation of Teachers (AFT) promoted CSD 2's policies in the union’s national magazine ( American Educator 1999-2000). Two “chapter chairs,” (personal communication, May 2000), teachers who ar e elected to be the union representatives in the building, who hear teachers’ problems and begin the grievance process, noted that they viewed the UFT l eadership as reluctant to pursue teachers’ complaints, formally or informally Clear evidence of teacher dissatisfaction with the UFT’s stance towards CSD 2 surfaced in the Spring 2000 election for the UFT’s district representative. Since creation of the community school districts in New Y ork City as a response to the call for community control, the UFT has had a paral lel organizational structure. Each community school district in New York has a un ion representative, a “district rep” who negotiates issues of local concern with th e district administration. The “district rep” is elected by chapter chairs, but el ections are almost always pro forma because of the UFT leadership’s control of the unio n apparatus (Weiner, 1998). However, CSD 2 chapter chairs elected a teacher run ning against the “heir apparent” of the UFT leadership (personal communica tion, chapter chair in CSD 2, May 2000). The UFT’s position was that though CSD 2 's administration often had trouble “hearing what teachers had to say,” neverth eless CSD 2 officials had to be supported; their model of reform was not only super ior to others, it was the only one that could convince the public that city schools co uld be salvaged (private conversation with UFT President, Randi Weingarten, Oct. 2000). The vote for the CSD 2 “district rep” demonstrated that chapter chai rs rejected the UFT leadership’s stance toward CSD 2 officials. Their dissatisfactio n certainly raises questions about the extent to which the CSD 2's strategy for labormanagement relations will be supported by teachers elsewhere. Yet opposition amo ng CSD 2 teachers to district policies and the UFT’s support of them is absent in research on CSD 2. Problems arising from the insularity of the researc hers and of the design of the research itself appear in what was perhaps the key study. As Harwell, D’Amico, Stein, and Gatti (2000) note "A shortcoming shared by previous research done on the effectiveness of District #2's professional dev elopment system...was that the units of analysis used in these studies were school s. As a result, variation among students' performance and teachers' experiences wit hin schools was ignored"(p.7). Hence their study attempts to correlate achievement on tests with teachers' professional development by examining test scores o f individual students taught by teachers who described their professional developme nt experiences in


6 of 20 questionnaires. However, the questionnaire required teachers to provide their New York City Board of Education ID or “file” numbers ( often used in lieu of names to locate personnel in official records) and their sch ools. Readers of HPLC’s research reports will not know th is fact. The consent form included in the appendix (Harwell, D’Amico, Stein & Gatti, 2000) is not a duplicate of the form distributed to teachers, with which I w as provided by teachers in two different schools. (Note 3) A cover letter from Deputy Superintendent Bea Johnstone (2000), also omitted from the appendix (H arwell, D’Amico, Stein & Gatti, 2000), an appendix cited in the subsequent study (D ’Amico, Harwell, Stein, van den Heuvell, April 2001) refers to this request for tea chers’ file numbers. The letter states “The survey asks for your name and your teac her folder number, so that the information gained from it can be linked to other d ata collected in the course of the HPLC study” but that “individual responses will not [emphasis in the original] be seen by other members of the District #2 community. ” Another startling error in the formulation of the questionnaire, also absent from the appendix, is the consent form’s listing of Anthony Alvarado, CSD 2's former Superintendent, as a Principal Investigator, with his institutional affiliation gi ven as CSD 2's office ( “Consent to act as a participant in a research study” University of Pittsburgh IRB# 980136). Alvarado was not Superintendent when the questionna ire was distributed. Rather, Elaine Fink, a CSD 2 deputy superintendent with who m he continued to collaborate professionally, had replaced him (New York City Boa rd of Education, 2000). It seems apparent that Johnstone’s letter to CSD 2 tea chers was designed to address teacher apprehensions about the confidentia lity of the questionnaire. (Note 4) CSD 2 chapter chairs communicated informally about what they should tell teachers who feared that completed questionnaires w ould not be confidential (personal communication with two chapter leaders). The poor return rate and the subsequent offer of $500 to any school with a high return rate seem strong evidence that the two chapter chairs who told me th at they had informally advised teachers to avoid filling out the questionnaire wer e not alone. CSD 2: Not a Typical Urban DistrictAnother key fact about CSD 2 that is not fully addr essed in reports by researchers who promote it as a model is the district’s access to human and material resources that urban districts typically lack. The “variabili ty” and variation among schools and neighborhoods--the term used to describe CSD 2's de mographics (Elmore & Burney, 1999a; Fink & Resnick, 1999)--fails to conv ey the numerous advantages afforded CSD 2 by the sizable numbers of economical ly comfortable families who send their children to public school. One study ack nowledges that District #2 is "a fairly wealthy urban district... the fourth wealthi est community school district in New York City...and in the upper quartile for urban dis tricts nationally" (Harwell, D’Amico, Stein & Gatti, 2000, p. 9). Surprisingly the implic ations of this important characteristic are not explored. Reports mention th at CSD 2 encompasses a broad swath of Manhattan’s wealthiest real estate and mos t of its prosperous neighborhoods. However, what the reports do not exp lore is the extent to which the district’s concentration of wealthy neighborhoods m ay spare its central office and many of its schools the psychological and fiscal de mands present in most urban school systems, demands that might be summarized as “Keeping students in, gangs out, scores up, alienation down, and the copy machine in working order: Pressures that make urban schools in poverty differ ent” (Metz, 1997).


7 of 20 Research does not explain that CSD 2's showcase, th e elementary school touted as having the highest test scores in New York City, is PS 234, in the heart of Tribeca. A ZIP-code by ZIP-code analysis of the New York real estate market found that Tribeca “was the highest priced residential ne ighborhood in Manhattan last year” (Hevesi, 2002). A report on median sale price s for apartments in Manhattan (Hevesi, 2002) shows how affluent almost all of CSD 2's neighborhoods actually are. Chelsea registered the third highest average p rice ($1,024,850) for apartments in Manhattan, due to loft conversions that have tur ned warehouses into art galleries. CSD 2 now contains only one large area i nhabited primarily by families living in poverty, Chinatown, populated in great pa rt by newly arrived immigrants. Poverty in the rest of CSD 2 occurs primarily where there are housing projects amid gentrification.As Tables 1, 2, and 3 show, CSD 2 differs demograph ically from other New York City districts, especially those with low levels of student achievement. The largest minority population in CSD 2 schools is Asian, and the Asian and white population combined constitute 65% of the students served. In New York City schools, the combined Asian & white figure is 27% (New York City Board of Education, 2000). Data on the school report cards for each school in CSD 2 show that schools in Chinatown serve the highest proportions of students in CSD 2 who are English-language learners, the designation for stud ents who have been in the US for three years or less. (Note 5) In many other districts in New York City, the immigrant population is primarily Spanish-speaking.New York City designates elementary schools with a “need factor” from 1 to 12, based on the percentage of students categorized as English Language Learners, students identified as eligible for special educati on, and students eligible for free or reduced lunch. The higher the number, the greater t he need of schools for services. As shown in Table 4, more than a third of CSD 2 schools have a “need factor” of only one, two, or three. The “need facto r” in these schools indicates that they have a student composition that more closely a pproximates what teachers would find in the suburban school systems close to the city, where there is no teacher shortage (Institute for Education & Social Policy, 2001). Another important fact that research on CSD 2's professional developm ent fails to address, suggested by scholarship about how teachers’ social class inf luences their work (Metz 1990), is that many teachers attracted to and recently hir ed by CSD 2 may want to work with administrators, other teachers, and perhaps st udents, who share their social class origins, aspirations, and world-view. One vet eran African-American teacher in a CSD 2 school suggested this possibility to me, no ting that minority graduates of “City” (City College, of City University of New Yor k) feel that it is highly unlikely they will be hired by CSD 2 officials. Another factor is that minority graduates of “City” don’t want to teach in CSD 2 because they are commi tted to working in places they see as high need. (Note 6) Student Achievement in CSD 2What are the implications of these demographic char acteristics of CSD 2? In this section, I examine the extent of racial and economi c segregation in CSD 2 schools and compare this to achievement on standardized mat h tests. I compare data on CSD 2 to equivalent data for the only other school district in all of New York City


8 of 20 and New York State to be roughly equivalent in its demographics, District 25 in Queens.Data on test scores and family income in each schoo l in New York City’s public schools, published in The New York Times (Goodnough, 2000), reveal the extent to which CSD 2's demographics are un representative of other districts in New York City and of urban school districts in New York Stat e as well. Compare, for instance, test results published in October 2000 for CSD 2 in Manhattan and District 8 in the Bronx. Both were reported as enrolling approximatel y the same number of students, CSD 2 with 2,204, District 8 with 2,374. Yet CSD 2 had 12 elementary schools out of 26 with fewer than 50% of its studen ts qualifying for free lunch, whereas District 8 had one out of 20 (Goodnough, 20 00). To pursue the issue of District’ 2s representativeness, which I propose as a key consideration in evaluating whether it can indeed be a model for urban school d istricts elsewhere, I examined demographic data for each county on the New York St ate Department of Education website ( Table 1). (http: // Table 1 Comparison of Student Demographics in urban school districts in New York State, Fall 1997* DistrictCSD 25, QueensCSD 2, ManhattanYonkersTotal24,12722,21223,968 White28.2%31.2%23.3%Black9.1%13.8%30.4%Hispanic24.4%21.1%41.5%Other38.4%33.8%4.8%Free/Reduced lunch48.5%51.3%74.7%Limited English19.1%16.9%16.4% *Data from the NY State Education Department school district profiles, downloaded on Jan. 31,2002: 55_99/D662300.html. As Table 1 shows, the only district in New York Sta te outside of those in New York City that enrolls about the same number of students as CSD 2 is Yonkers. In this comparison two factors that distinguish CSD 2 are i ts relatively (for an urban district) low percentage of students who qualify fo r free/reduced lunch and its racial and ethnic mix of students. Examining demographics for each of the community school districts in New York City, I found only one CSD 25 in Queens, that closely resembles CSD 2 in the number of students served (b etween 20-24,000), the proportion of students reported as eligible to rece ive free/reduced lunch (between 50-60%), and the student body’s ethnic/racial compo sition (around 10% Black, 30% Hispanic, 30% White, 35% Asian). Note also how the demographics of CSD 2 differ from the characteristics of the New York City schoo l system as a whole: nearly 75%


9 of 20 of the children the city school system serves are e ligible for free/reduced lunch; more than a third are Black; close to 40% are Hispa nic; Asian students (identified as “other” on the school report cards) represent on ly a little over 10%. I wish to caution that a complete analysis of the i mplications of the demographic differences would require close examination of data disaggregated by race and ethnicity that were not publicly available from New York State and New York City until 2002 (email communication, New York State Dep t. of Education Data Analyst). The discussion that follows is, therefore, suggesti ve of the questions that the demographic differences ought to raise. I do not pr esent my discussion as being other than suggestive. Asian students in CSD 2, the largest minority proportionally, would be categorized in John Ogbu’s typology (Ogbu, Sept. 1995; Ogbu, Dec. 1995) as “voluntary” (as opposed to “involuntary”) minorities. Ogbu posits that minorities, who emigrate voluntari ly, are more likely to experience racism in school and society as barriers to overcom e. In contrast, because of their history of oppression by the dominant culture, invo luntary minorities are more likely to see racism as a permanent impediment to achievem ent. Unlike most other districts in New York City and urban districts in N ew York State, CSD 2's single largest minority population consists of voluntary m inorities (Table 2). Both Ogbu’s typology and the categories used to report demograp hics in New York schools (Black, White, Hispanic, OtherAsian and Pacific I slander) obscure very important differences among these populations. As Cooper and Denner (1998) noted, a limitation in the analytical framework Ogbu employs is its lack of emphasis on variation and change within communities, especially upwardly mobile ethnic minority families and children. Gibson (1997) argue d that Ogbu's typology fails to account for intragroup variability and is too “dich otomous, too deterministic, and in danger of contributing to stereotypical images..."( p. 322). She faults the theory for not taking into account generational and gender dif ferences, as well as lacking explanatory power to account for the experience of groups such as Mexican Americans, who share elements of both categories. M oreover, she contends, the theory fails to take into account school effects an d human agency. While acknowledging the limitations in Ogbu’s typology, I suggest that his theory helps illuminate why school and instructional practices t hat are successful with one group of students may not be equally effective with anoth er, and thus his typology is germane to discussions of CSD 2's achievements. Table 2 Comparison of Demographics of CSD 2 and CSD 25 NYCCSD 25, Queens CSD 2, Manhattan Total enrollmentNA on report card 24,49921,559 White15.3%26.3%31.9%Black34.2%8.4%13.8%


10 of 20 Hispanic38.9%25.3%20.1%Other -Asian, Pacific Islanders, Alaskan Natives, Native Americans 11.5%40.1%34.2% Free/Reduced lunch74.150.4%59.9Recent immigrants7.3%12.3%9.2%Limited EnglishNA on report card 4557 students 2940 students Another, thornier problem with applying Ogbu’s anal ysis is that New York City Board of Education’s demographic data do not parall el Ogbu’s categories. For example, the category “English Language Learners” d escribes recent immigrants both from Puerto Rico and Peru; “Hispanic” students are those with Spanish surnames; students from British Guayana, who are “v oluntary” minorities might be labeled “Black,” placing them with African American s, “involuntary” minorities. Although I group students categorized as “Hispanic” in with those categorized as “Black” in the analysis, I caution that some of the se students may share the cultural framework of reference of “voluntary” and not “invo luntary minorities.” I divided CSD 2's 25 elementary schools into two ca tegories: those serving a population more than 50% combined Black and Hispani c students with a “need” factor of 7 or greater, and those with fewer than 5 0% Black and Hispanic students and a “need” factor less than 7. In CSD 2 there are 11 schools with a “need” factor over 7. Five of these 11 schools are high-poverty s chools in Chinatown. Table 3 shows the breakdown of schools in both districts, a ccording to these criteria. I have eliminated the schools in Chinatown from this compa rison to look at achievement of majority Black and Hispanic schools in both distric ts. Elmore and Burney (1999b) note that 18 schools in C SD 2 have populations more than two-thirds African-American, Hispanic, and Asi an, while four have populations that are more than two-thirds white As noted earl ier, the implications of this finding of “variability” are not explored further, in parti cular the extent to which achievement in CSD 2 schools correlates with their racial and s ocial stratification. As is evident from Table 3, only one elementary school, PS 11, ha s a student population that mirrors the district’s demographics. In CSD 2, 5 of the 11 schools with a “need” factor 7 and above are in Chinatown and have a popu lation that is more than 70% Asian. For instance PS 42, with a “need” factor of 10 (94.2% of its students receive free/reduced lunch and 18.9% are ELL) enrolls 88% A sian students. What is not evident in statistical analyses is that PS 11 also has a very large “gifted and talented” program in which almost all of its White students are enrolled (personal communication, CSD 2 administrator). Hence, the one elementary school that is demographically representative of the district’s en rollment houses two different schools, one serving White students in its “gifted and talented” students, the other Black and Hispanic students. The school’s scores ar e reported in the aggregate. Table 3 Elementary Schools Serving a Population More Than 5 0% Combined Black


11 of 20 and Hispanic Students with a “Need” Factor of 7 or Greater School“Need”% Hispanic% Black CSD 25: PS 2011029%40% School“Need”% Hispanic% Black CSD 2: PS 11731%26%CSD 2: PS 33105227CSD 2: PS 51106118 CSD 2: PS 11196616CSD 2: PS 12674330CSD 2: PS 151104330CSD 2: PS 19885226 Note: Data from NYCBOE website, http://www.nycenet. edu/daa/Mrr/districts. What is consistently referred to as “variability” o r “variation” in school demographics in CSD 2 is actually a euphemism for a familiar phe nomenon in US schools: racial segregation (Orfield & Eaton, 2003). The high degre e of racial and social stratification is especially noteworthy in light of comparison to CSD 25. With approximately the same demographics as CSD 2, CSD 2 5 has only one school that is as racially segregated as eleven schools in CSD 2. “Student Need” is a ranking of elementary schools i nto one of 12 categories based on the percent of students eligible for free lunch, percent of tested students who are in full-time and part-time special education pr ograms, and the percent of tested students who are English Language Learners (ELL). T he higher the numbers of students in these categories, the higher the school ’s “need” factor. To understand the significance of “need,” two schools (in CSD 25 ) designated as having “need” factors of “3" and “10" are compared in Table 4. Mo st schools in CSD 25 have a “need” factor within the range of 3-7 (17 of 23 sch ools fall within this range; 6 schools are outliers). CSD 2 has only 9 of 24 that fall within this range. Seventeen of its schools fall outside this range. The compari son indicates that CSD 2's schools are far more stratified than those in CSD 2 5, a district with a student enrollment that is equivalent in terms of the demog raphic categories used by the state. Table 4 Examples from CSD 25 of “Student Need” Categories 3 and 10 for Two Elementary Schools School’s need% students receiving free/reduced % sp ecial %


12 of 20 factorlunched.ELLCSD 25 School A“Need” of “3" 40%4%8% CSD 25 School B “Need” of “10" 73%36%5% Comparing achievement between schools in both distr icts, I used scores on the New York State fourth grade math test in 2000, repo rted in the New York Times (Goodnough, 2000) I compared scores of only those schools serving a majority of Black and Hispanic students. In the New York State tests, scores of level 1 and 2 indicate that the student is “not meeting standards .” The results of this comparison are shown in Table 5. Table 5 Scores on the NYS Fourth-Grade Math Test Only in 20 00 (Elementary schools serving a population more than 50% combined Black and Hispanic students with a “need” factor of 7 or greater) School perflevel1 perflevel2 perflevels 3 &4 “Need”factor % Hispanic % Black CSD 25: PS 201 17%41%43% 102940 School perflevel1 perflevel2 perflevels 3 &4 “Need”factor % Hispanic % Black CSD 2: PS 11 16%16%68% 73126 CSD 2: PS 33 26%41%33% 105227 CSD 2: PS 51 19%59%27% 106118 CSD 2: PS 111 26%47%27% 96616 CSD 2: PS 126 9%45%45% 73920 CSD 2: PS 151 15%58%28% 104330 CSD 2: PS 198 4%20%76% 85226


13 of 20 Source: Reported in the NY Times 15 Oct. 2000, (City section, pp. 14-16) In Table 6, I continue the comparison of school-wid e test scores, using data from the school report cards published on the New York C ity Board of Education website and including the number of students tested. These test scores, unlike the others I have analyzed, are for math scores in grades 3-8. T wo of the schools in CSD 2 are K-8 schools, PS/IS 33 and PS/IS 111. With the excep tion of PS 11, which has a large “gifted and talented program” of mainly white students, only one school in CSD 2 has a significantly higher proportion of stud ents “meeting standards” in math than in the school with similar demographics in CSD 25 Indeed, several CSD 2 schools do not perform as well as PS 201 in CSD 25 Table 6 Percentages of Students “Meeting the Standard” (i.e ., Perf. Levels 3 and 4) on City-wide Math Tests in Grades 3,5,6,7 and State Ma th Tests in Grades 4 and 8 (including students taking the test in translatio n) School Meetingstandard “Need”factor % Hispanic% Black CSD 25: PS 201 31.5%(235 tested) 1029%40% School Meetingstandard “Need”factor % Hispanic% Black CSD 2: PS 11(34% white; 8% Other) 50.6%(233 tested) 731%26% CSD 2: PS/IS 33(White 4%; Asian 37%) 17.3%(191 tested) 105227 CSD 2: PS 51 22.5%(145 tested) 106118 CSD 2: PS/IS 111 22.5%(516 tested) 96616 CSD 2: PS 126 32.6%(285 tested) 74330 CSD 2: PS 151 25%(115 tested) 104330 CSD 2: PS 198 53.4%(131 tested) 85226 Several questions are posed by this comparison with CSD 25, one of the most critical being what data disaggregated by race and ethnicity reveal about achievement. With the exception of the studies base d on the questionnaire


14 of 20 requiring teachers to self-identify with file numbe rs (Harwell, D’Amico, Stein & Gatti, 2000), reports by researchers promoting CSD 2 as ex emplary do not address this question. (Note 7) But in the one study that does attempt to use data disaggregated by race and ethnicity, the investigators state thei r primary research question as this: "Are teachers with strong professional development participation patterns more likely to have closed achievement gaps?"(Harwell, D ’Amico, Stein & Gatti, 2000, p. 19). The answer: "In summary, engagement in profess ional development, as measured by this questionnaire and reported by the 62 respondents, does not appear to have significant influence on student ach ievement in either literacy or mathematics” (Harwell, D’Amico, Stein & Gatti, 2000 p. 22). ConclusionsIn spite of statements by researchers looking for e vidence that CSD 2's policies have indeed boosted achievement, there are no data to support such claims (Harwell, D’Amico, Stein & Gatti, 2000). Indeed, no data support even the more modest claim of the "generally positive picture" of systemic reform in CSD 2 (Elmore & Burney, 1999a, p. 3). How then could rese archers promote CSD 2's instructional development practices as unusually su ccessful or its investment in staff development tied to national standards as a m odel to be emulated? Research on CSD 2 exemplifies the problems that ari se when researchers fail to maintain the independence and critique that Bourdie u (1998) demanded of intellectuals. The inter-dependence of district lea ders and researchers, combined with the exclusion of dissenting perspectives, obsc ured key questions about CSD 2 practices that need to be explored before they can or should be replicated. From the formulation of research design, to data collect ion, to presentation of findings, research on CSD 2 appears to have shown a marked di sregard for alternative perspectives and local knowledge. As a result, insi ghts that might have contributed to district officials’ and researchers’ learning ha ve been ignored Researchers have published reports that have reinforced the belief a mong CSD 2 officials that their work is a model for the entire New York City system because it is “leading New York City in implementing Standards” (Harwayne, 199 9). But comparison of existing data for CSD 25, knowledge of the social-contextual factors such as CSD 2's access to human and material resources other distri cts in the city lack, and the inattention to disaggregation of individual achieve ment according to race and ethnicity, indicate that the representation of CSD 2's practices as exemplary by researchers is unsubstantiated. The “labor-manageme nt” strategy that resulted from close relationships and consensus between high level union officials and district administrators caused significant turmoil that was not reported in the research. The consensus may not be replicable elsew here, indeed, was probably disrupted in CSD 2 with the election of a new “dist rict rep.” The rise in achievement levels since 1989 may be due to changes in the dist rict’s demographics and not to a focus on instruction or professional development linked to national standards. Research that has attempted to link achievement to professional development has failed to find evidence of correlation, let alone e stablish causation. It may be that CSD 25 is, in fact, just as promisin g a model of school improvement as CSD 2’s. Its elementary schools are far less seg regated and stratified by income than are CSD 2's. Its test scores are equivalent to those in CSD 2. It is interesting to note that the statement of its Superintendent in the 1999-2000 suggests a stance


15 of 20 towards students, parents, and community sharply di fferent from that promulgated by CSD 2: ... We teach "children not merely subjects." To sup port this goal, the District and the Community School Board work closel y to provide an integrated, holistic, comprehensive educational pro gram which motivates and engages all students, and provides th e optimum opportunity for every child to achieve state and ci ty academic standards... Staff are supported by professional development activities designed to help them hone their instructional skil ls. Parents and community members are actively involved in all scho ols and are recognized as valuable resources. It may be that CSD 2 has pioneered practices that s hould be replicated, as the researchers who have promoted it have concluded. Ho wever, the opposite conclusion is equally plausible. It was the task of research to explore both possibilities, but the role of cheerleader seems to have over-ridden the demands of scholarship.Notes 1. As of 2003, the NYC Board of Education has been ren amed the “Department of Education” and control over the city schools given to Mayor Bloomberg. He and his appointed Chancellor have submitted a proposal to t he state legislature to merge the 32 community school districts into 10. CSD 2's present superintendent has been selected to head one of the 10 new districts. 2. Information about the group publishing the report o n the math curriculum is available from Elizabeth Carson, 3. I informed Richard Elmore and Lauren Resnick about the concerns raised to me by CSD 2 teachers about the use of file numbers and Alvarado’s presence as a PI. Lauren Resnick responded (letter, 17 July 2000), an d Stein and Resnick met with me in April 2001. According to notes I took after o ur meeting, the objections I raised to the conduct of research, namely that it deepened a climate of fear, were dismissed. Resnick acknowledged the possibility of “bad design.” Stein noted that their findings actually contradicted claims being m ade about CSD 2's success. However both Stein and Resnick rejected my proposal to clarify publicly that HPLC’s latest research told a different story about CSD 2 from the one that had been widely publicized in earlier work. Resnick exp lained that their role had ended with the study’s completion. 4. Another item omitted from the Appendix and not ment ioned in the reports was a notice distributed to teachers, signed “The HPLC Re search Team” with the HPLC address, phone, fax, and website. It announced a re ward of $500 to schools in which 90% of the teachers returned questionnaires. The notice also informs teachers that the extra consent form to serve as th eir personal copy, included in the original packet with Johnstone’s cover letter, “is the wrong version” and should be discarded. 5. I examined the 1999-2000 report cards for all eleme ntary schools in CSD 2.


16 of 20 They and the District report cards are available at 6. Questionnaires returned to researchers show a prepo nderance of White, female, middle-class respondents (D’Amico, Harwell, Stein, Van den Heuvel, April 2001). Curiously, researchers on CSD 2 have never investig ated the extent to which district hiring screens out teachers and principals whose professional beliefs differ from those of the district leadership, and the ways those beliefs correlate with social class or race. 7. The investigators secured achievement and demograph ic data for individual students in District #2 through the Division of Ass essment and Accountability of the Office of the Deputy Chancellor of Instruction of t he New York City Board of Education. Disaggregated data have heretofore been made available only to researchers working with the Division of Assessment and Accountability, as I learned when I attempted to secure it for this stud y (email message from data analyst at DAA). References American Educator. (1999-2000, Winter). It's all ab out teaching and learning. American Educator, pp. 14-19. Bourdieu, P. (1998). Acts of resistance. Against the tyranny of the mark et. New York: The New Press.Cooper, C. R., & Denner, J. (1998, April). Theories linking culture and psychology: Universal and community-specific processes. America n Educational Research Association. San Diego.Cuban, L. (1988). Researchers advising policymakers : A word to the wise. Educational Psychologist, 23 (3), 287-293. D'Amico, L., Harwell, M., Stein, M. K., & van den H euvel, J. (2001, April). Examining the implementation and effectiveness of a district-wide instructional improvement effort. American Educational Research A ssociation. Seattle. D'Amico, L., van den Heuvel, J., & Harwell, M. (200 0). Multiple Perspectives on Instructional Improvement in Literacy and Mathemati cs. High Performance Learning Communities Project, contract #RC-96-137002. Learni ng Research and Development Center, University of Pittsburg.Elmore, R. F. (1999-2000, Winter). Building a new s tructure for school leadership. American Educator, pp. 6-13. Elmore, R. F., & Burney, D. (1997a). Investing in teacher learning: Staff development and instructional improvement in Commun ity School District #2, New York City [ERIC Document Reproduction Service, ED416203]. Ne w York: National Commission on Teaching & America's Future. Teachers College, Columbia University (62 pages).Elmore, R. F., & Burney, D. (1997b). School variation and systemic instructional improvement in Community School CSD 2 [Http://].


17 of 20 Pittsburgh: University of Pittsburgh, High Performa nce Learning Communities Project, Learning Research and Development Center.Elmore, R. F., & Burney, D. (1999). School variation and systemic instructional improvement in Community School District #2, New Yo rk City [The CEIC Review. Volume 8, Number 1]. Philadelphia: National Center on Education in the Inner Cities. Temple University Center for Research in Hu man Development and Education.Elmore, R. F., & Burney, D. (1999, February). Schoo l variation and systemic instructional improvement in Community School Distr ict #2, New York City. In A. Grant & M. Calabazana (Eds.), The CEIC Review The CEIC Review. Philadelphia: The National Center on Education in the Inner Citie s. Temple University Center for Research in Human Development and Education.Fink, E., & Resnick, L. B. (1999). Developing princ ipals as instructional leaders. HPLC reports [Online]. Available: tml (Accessed 12 November 2001).Freedman, S. (1987). Teacher 'burnout' and institut ional stress. In J. Ozga (Ed.), Schoolwork. Approaches to the labour process of tea ching (pp. 133_145). Philadelphia: Open Univ. Press. Milton Keynes.Knapp, M. (1995). The teaching challenge in high_po verty classrooms. In M. Knapp & & associates (Eds.), Teaching for Meaning in High_Poverty Classrooms (pp. 160_182). New York: Teachers College Press.Gibson, M. (1997, September). Exploring and explain ing the variability: Cross-national perspectives on the social performan ce of minority students. Anthropology and education, 28 (3), 318-329. Goodnough, A. (2000, 15 October). How grades four a nd eight fared on tests. The New York Times, pp. CY 14-16. Harwell, M., D'Amico, L., Stein, M. K., & Gatti, G. (2000). Research contract #RC-96-137002 with OERI. Pittsburgh, PA: Learning R esearch and Development Center, University of Pittsburgh.Hevesi, D. ( 2002, 17 May). TriBeCa is priciest nei ghborhood. The New York Times p. B6. Institute for Education & Social Policy. (2001). Distributing teacher quality equitably: The case of New York City. New York: Institute for Education & Social Policy, New York University.Johnstone, B. (2000, 1 May). [Letter to District #2 teachers distributed by principals]. New York City.Metz, M. H. (1990). How social class differences sh ape teachers' work. In M. W. McLaughlin, J. E. Talbert & N. Bascia (Eds.), The contexts of teaching in secondary schools. Teachers Realities (pp. 40-107). New York: Teachers College Press.


18 of 20 Metz, M. H. (1997, March). "Keeping Students in, Ga ngs Out, Scores up, Alienation Down, and the Copy Machine in Working Order: Pressu res That Make Urban Schools in Poverty Different." American Educational Research Association. Chicago.New York City Board of Education. (2000). Second Annual Report: Evaluation of the Performance Driven Budgeting Initiative. New York City: Author. Ogbu, J. U. (1995, September). Cultural problems in minority education: Their interpretations and consequences-Part One: Theoreti cal background. The Urban Review, 27 (3), 189-205. Ogbu, J. U. (1995, December). Cultural problems in minority education: Their interpretations and consequences-Part two: Case stu dies. The Urban Review, 27 (4), 271-297. Orfield, G. & Eaton, S. E. (accessed 11 March 2003) Back to segregation articles/reseg.php Resnick, L., & Harwell, M. (1998). High performance learning communities: CSD 2 achievement [OERI research contract #RC-96-137002 deliverable] Pittsburgh: University of Pittsburgh, Learning Research and Dev elopment Center. Resnick, L. B., & Harwell, M. (2000). Instructional variation and student achievement in a standards-based education district Pittsburgh, PA: Learning Research and Development Center.Stein, M. K., D'Amico, L., & Johnstone, B. (1999, A pril). District as professional educator: Learning from and in New York City's Comm unity School District #2. American Educational Research Association. Montreal Weiner, L. (2002, May). Assessing systemic’s reform “learning by all”: Who evaluates learning by policy analysts? Educational Policy, 16 (2): 239-263. Weiner, L. (1998, Summer). Albert Shanker’s legacy. Contemporary Education 69 (4), 196-202.Zuckerman, J. B. (2001). Queering the life of a pro gressive, urban, elementary school: Genealogical ghost stories [Dissertation], New York: Teachers College, Columbia University.About the AuthorLois WeinerLois Weiner is a Professor of Elementary and Second ary Education at New Jersey City University, 2039 Kennedy Blvd., Jersey City, N J 07305. Email correspondence should be addressed to LWEINER@NJCU.EDU. The World Wide Web address for the Education Policy Analysis Archives is


19 of 20 Editor: Gene V Glass, Arizona State UniversityProduction Assistant: Chris Murrell, Arizona State University General questions about appropriateness of topics o r particular articles may be addressed to the Editor, Gene V Glass, or reach him at College of Education, Arizona State Un iversity, Tempe, AZ 85287-2411. The Commentary Editor is Casey D. Cobb: .EPAA Editorial Board Michael W. Apple University of Wisconsin David C. Berliner Arizona State University Greg Camilli Rutgers University Linda Darling-Hammond Stanford University Sherman Dorn University of South Florida Mark E. Fetler California Commission on Teacher Credentialing Gustavo E. Fischman Arizona State Univeristy–Los Angeles Richard Garlikov Birmingham, Alabama Thomas F. Green Syracuse University Aimee Howley Ohio University Craig B. Howley Appalachia Educational Laboratory William Hunter University of Ontario Institute ofTechnology Patricia Fey Jarvis University of WashingtonSeattle Pacific University Daniel Kalls Ume University Benjamin Levin University of Manitoba Thomas Mauhs-Pugh Green Mountain College Les McLean University of Toronto Heinrich Mintrop University of California, Los Angeles Michele Moses Arizona State University Gary Orfield Harvard University Anthony G. Rud Jr. Purdue University Jay Paredes Scribner University of Missouri Michael Scriven University of Auckland Lorrie A. Shepard University of Colorado, Boulder Robert E. Stake University of Illinois—UC Kevin Welner University of Colorado, Boulder Terrence G. Wiley Arizona State University John Willinsky University of British ColumbiaEPAA Spanish Language Editorial Board


20 of 20 Associate Editor for Spanish Language Roberto Rodrguez Gmez Universidad Nacional Autnoma de Mxico Adrin Acosta (Mxico) Universidad de J. Flix Angulo Rasco (Spain) Universidad de Teresa Bracho (Mxico) Centro de Investigacin y DocenciaEconmica-CIDEbracho Alejandro Canales (Mxico) Universidad Nacional Autnoma Ursula Casanova (U.S.A.) Arizona State Jos Contreras Domingo Universitat de Barcelona Erwin Epstein (U.S.A.) Loyola University of Josu Gonzlez (U.S.A.) Arizona State Rollin Kent (Mxico) Universidad Autnoma de Puebla Mara Beatriz Luce (Brazil) Universidad Federal de Rio Grande do Javier Mendoza Rojas (Mxico)Universidad Nacional Autnoma Marcela Mollis (Argentina)Universidad de Buenos Humberto Muoz Garca (Mxico) Universidad Nacional Autnoma Angel Ignacio Prez Gmez (Spain)Universidad de DanielSchugurensky (Argentina-Canad) OISE/UT, Simon Schwartzman (Brazil) American Institutes forResesarch–Brazil (AIRBrasil) Jurjo Torres Santom (Spain) Universidad de A Carlos Alberto Torres (U.S.A.) University of California, Los EPAA is published by the Education Policy Studies Laboratory, Arizona State University

xml version 1.0 encoding UTF-8 standalone no
mods:mods xmlns:mods http:www.loc.govmodsv3 xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govmodsv3mods-3-1.xsd
mods:relatedItem type host
mods:identifier issn 1068-2341mods:part
mods:detail volume mods:number 11issue 27series Year mods:caption 20032003Month August8Day 77mods:originInfo mods:dateIssued iso8601 2003-08-07