|USFDC Home||| RSS|
This item is only available as the following downloads:
xml version 1.0 encoding UTF-8 standalone no
mods:mods xmlns:mods http:www.loc.govmodsv3 xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govmodsv3mods-3-1.xsd
mods:relatedItem type host
mods:identifier issn 1068-2341mods:part
mods:detail volume mods:number 14issue 23series Year mods:caption 20062006Month September9Day 1313mods:originInfo mods:dateIssued iso8601 2006-09-13
xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam a22 u 4500
controlfield tag 008 c20069999azu 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E11-00496
Educational policy analysis archives.
n Vol. 14, no. 23 (September 13, 2006).
Tempe, Ariz. :
b Arizona State University ;
Tampa, Fla. :
University of South Florida.
c September 13, 2006
NCLB : local implementation and impact in southwest Washington state / Linda Mabry [and] Jason Margolis.
Arizona State University.
University of South Florida.
t Education Policy Analysis Archives (EPAA)
Readers are free to copy display, and distribute this article, as long as the work is attributed to the author(s) and Education Policy Analysis Archives, it is distributed for noncommercial purposes only, and no alte ration or transformation is made in the work. More details of this Creative Commons license are available at http://creativecommons.org/licen ses/by-nc-nd/2.5/. All other uses must be approved by the author(s) or EPAA EPAA is published jointly by the Mary Lou Fulton College of Education at Arizona State Universi ty and the College of Educ ation at University of South Florida. Articles are indexed by H.W. W ilson & Co. Please contribute commentary at http://epaa.info/wordpress/ and send errata notes to Sherman Dorn (email@example.com). EDUCATION POLICY ANALYSIS ARCHIVES A peer-reviewed scholarly journal Editor: Sherman Dorn College of Education University of South Florida Volume 14 Number 23 September 14, 20 06 ISSN 1068 NCLB: Local Implementation and Impa ct in Southwest Washington State Linda Mabry and Jason Margolis Washington State Un iversity Vancouver Citation: Mabry, L., & Margolis, J. (2006). NCLB: Local implementation and impact in southwest Washington state. Education Policy Analysis Archives, 14 (23). Retrieved [date] from http://epaa.asu.edu/epaa/v14n23/. Abstract The research reported here is from the first two years of an ongoi ng and largely qualitative study to examine the impact of the No Child Left Behind federal education policy on educational practice an d climate in elementa ry schools in two districts in southwest Washington. Based on systematic drop-in observations in classrooms and interviews with teachers and school and district administrators, data indicated that the policy had partia lly yielded the intended standards-based reforms but at considerable local cost. While most pa rticipating administrators described efforts to use NCLB to leverage needed change, most teachers described struggles to sustain best pra ctice and to avoid some nega tive consequences to their students and schools. Administrators antici pated that resistant teachers would be nudged from the profession, and the greatest attrition among participating teachers was from the fourth-grade level at which the states standards-based test was administered. Fourth-grade teachers particularly expr essed concern about testrelated stress and test-driven curricula in terfering with childrens individual needs and with their own ability to provide developmentally appropriate instruction adapted for their particular students. The validity and utility of test results was a local issue. Keywords: accountability; NCLB; policy impact; school reform; teacher change.
Education Policy Analysis Archives Vol. 14 No. 23 2 Sin abandonar ningn nio: Implementacin e impacto locales en el sudoeste del Estado de Washington. Resumen Esta investigacin presenta datos parciales obtenidos dura nte los primeros dos aos de un estudio en curso y mayormente cualitativo con el objetivo de examinar el impacto de la poltica federal de educacin Sin abandonar ningn nio (NCLB) sobre la prctica educativa y el clima en las escuelas primarias de dos distritos escolares en el sudoeste del esta do de Washington. A travs de visitas y observaciones sistemticas en salas de clase y entrevistas con docentes y administradores de las escuelas y del distrito escolar, los datos obtenidos muestran que la poltica federal de es tndares NCLB produjo las reformas previstas pero a un costo local considerable Mientras que la mayor a de los administradores escolares que participaron describieron los esfuerzos de utilizar NCLB para estimular cambios considerados necesarios, la mayora de los profesores describieron problemas para sostener las buenas prcticas recomendadas y evitar algunas consecuencias negativas pa ra sus estudiantes y escuelas. Los administradores anticiparon que los prof esores que opondran mayor resistencia serian gradualmente desplazados de la profesin, y que el ag otamiento ms grande se dara entre los docentes de cuarto-grado ya que es en ese grado donde se administra los exmenes estatales esta ndarizados de NCLB. Los docentes de cuarto-grado expresaron preocupacin por la tensin nerviosa que generaba los exmenes y los planes de estudios organizados de acuerdo a los exmenes estandarizados, que adems interferan co n las necesidades individuales de los estudiantes y con la capacidad de los docentes de proporcionar una instruccin apropiada al desarrollo y ad aptada a las particularidades de sus estudiantes. La validez y utilidad loca l de los exmenes estatales estandarizados fue considerada un problema. Having a country this size with this de gree of diversity an d sa ying, By golly, everybody will meet a standard, is a huge thing. It s wonderful, but its huge. It falls to a lot of big and little districts to somehow make that happen. (Highland district administrato r, June 23, 2005) The most forceful federal education policy in forty years, the No Child Left Behind statute (NCLB, 2001) mandates universal literacy and numera cy by 2 014 at third through eighth grade levels as measured by standards-based state tests,1 the scores calibrated against scores on the National Assessment of Educational Progress (NAEP) fo r credibility purposes. Determination of the effectiveness of the federal policy requires local in vestigation because the primary intended effects are local: the refocusing of educational delivery in schools and classrooms to ensure attention to all students, including the historically underserved, an d the raising of individual student achievement. Beginning one year after NCLBs implementation, the follow-along policy study reported here, 1 All states except Iowa include standards-based testing as primary components of their educational accountability policies, although many of these 49 states administ er norm-referenced r ather than standardsbased tests at some grade levels.
NCLB: Local Implementation and Impac t in Southwest Washington State 3 undertaken in two school districts in southwest Washington state, documents NCLBs local implementation and impact. In the past quarter-century, studies of educational reform initiatives have documented their increasing politicization (e.g., Cook, Fitton, Viat or, Phillips & Gertz, 2001; Cuban, 1988; Fuhrman & Elmore, 1990; Wise, 1979) and many disappoint ments (e.g., Booher-Jenni ngs, 2005; Goodson & Foote, 2001; Lee & Wong, 2004; Stringfield & Yakimowski-Srebnick, 2005). The exclusion of educators from higher level policy-making has b een shown to result in overestimations of local capacity (e.g., Borko, Wolf, Simone, & Uchiyama, 20 03), conflicts with local policy (e.g., Shipps, 2003, Tyack & Cuban, 1995), local unwillingness to implement external initiatives (e.g., DarlingHammond, 1990; Fullan, 1991; Sarason, 1982, 199 0; Sipple, Killeen, & Monk, 2004), and attempts by local administrators to buffer classrooms from anticipated consequences (e.g., Rossman, Corbett & Dawson, 1986). Districtand school-level implementation has often taken the form of superficial compliance (e.g., Burlingame, 1993; Raywid, 1990), uncomfortable conglomerations of old and new practices (e.g., Louis, Febey, & Schroeder, 20 05; Tyack & Tobin, 1994), and distortions or displacements of highly regarded curricula and local programs (e.g., Berliner & Biddle, 1995; Meier, 1995; Schaffer, Nesselrodt, & St ringfield, 1997; Smith, 1991). Nevertheless, in formulating the Readin g First provisions of NCLB, for example, practicing educators and professional education and educat ional measurement organizations again played marginal roles (Miskel & Song, 2004). Subseque ntly and perhaps unsurprisingly, measurement researchers have identified technical and conceptual flaws in NCLBs test-driven accountability plan (Haertel, 2002; Hill & DePascale, 2003; Linn, Bake r, & Betebenner, 2002; Shepard, 2002; Sirotnik, 2004). The capacity of flaws to inhibit the achiev ement of policy aims or to result in unintended negative consequences underscores the importance of determining whether NCLB is reaching its aim of improving education such that no child is, in fact, left behind. With NCLB slated for Congressional reauthorizatio n in 2007, it is especially im portant at this juncture for research to document impact in both promising and problematic contexts by investigating questions such as those which motivated the study reported here: Does local response to NCLB represent successful implementation of the policy? Is the education al impact of NCLB at the classroom level positive? This study began with the assumption that NCLB might have both positive and negative effects. In the realization that early implementa tion can be stressful even for policies that are ultimately smoothly effected and embraced, districts and schools that appeared likely to meet policy requirements, local best case scenarios, were selected as research sites in an effort, for purposes of policy analysis, to moderate the influence of initial strain. If schools in southwest Washington could successfully achieve NCLB goals, these sites appeared likely to do so. Method In situ qualitative study promotes discovery of the details of implementation and impact, revealing policy effects at the classroom and schoo l levels. This type of policy analysis bears a likeness to utilization-focused evaluation (Patton, 1997) and impact evaluation as does educational research overall in an era of focus on what wo rks (Chatterji, 2005). To determine whether local teaching and learning were improving as intende d, classroom observations were systematically undertaken, teachers and school and district administrators were interviewed, and relevant documents and test score statistics were analyzed.
Education Policy Analysis Archives Vol. 14 No. 23 4 Site Selection and Description The two focal districts varied along several dimensions (see Table 1). In each district, a school enrolling fourth-gradersone of the three grade levels at which Washingtons standardsbased test was administer edwas selected and a ssigned a pseudonym, one school and district referred to as Highland and the other as Riverside. Even in the state which registered the highest SAT scores in the nation (Wood, 2005), both school s matched or exceeded aggregated state scores on the Washington Assessment of Student Learning (WASL), the states standards-based test. 2The districts each enrolled fewer than 2000 students, and the two schools populations hovered around 500 both years of the study; enrollments grew slightly in year 2. Both districts and both schools enrolled very high percentages of Caucasian students (89%), very low percentages of special education students (less than 12%), and re latively low percentages of students eligible for free and reduced lunches (17%). One district received Title 1 funding and the other did not. Both districts faculties were considered stable, the average years of teaching of the faculty at the two schools ranging from 9 years. The number of participating educators in the study overall was 25. At both schools, additional teacher-participants were recruited in year 2, partly due to attriti on (e.g., between years 1 and 2, two-thirds of the participating teachers at one school left for positions elsewhere). The principal at one school retired at the end of year 2, and there was a change in the superintendency in the other district at the end of both years 1 and 2. 2 Washington had the highest SAT scores among states in which more than half of the students took the test in 2004 and 2005.
NCLB: Local Implementation and Impac t in Southwest Washington State 5 Table 1 Comparisons of Local School Sites with Distri cts and Washington State Riverside Highland Variable 2003-04 2004-05 2003-04 2004-05 School population Total enrollment 430 474 468 506 Percentage white 91% 90% 97% 97% Special education enrollment 7% 12% 3% 12% Free and reduced lunch 38% 31% 18% 17% District population Enrollment 1,843 1,890 1742 1952 Schools 4 4 4 4 Percentage white 90% 89% 96% 95% Special education enrollment 10% 9% 0% 10% Free and reduced lunch 28% 27% 18% 13% Washington State Total enrollment 1,020,959 1,020,959 Percentage white 71% 71% Special education enrollment 12% 12% Free and reduced lunch 36% 36% Stability indicators Principals experien ce (in years at end of school year) 4 5 3 4 Mean teacher experience, school 10 9 20 22 Participating te achers 3 4 3 2 Participants who left school during or after year 2 2 1 1 Superintendents experience (in years at end of school year) 2 3 3 1 (inter im) District funding supplement referenda? None Fall 2005, failed None Fall 2004, passed State superintendent st atus Continued Relecteda Continued Relecteda Percentage of school 4th graders meeting state standards Reading 75.8% 87.5% 89.9% 87.0% Math 66.1% 73.8% 75.3% 77.0% Writing 68.3% 57.5% 61.2% 64.6% Science 30.4% 36.0% 28.5% 43.3% Percentage of district 4th graders meeting state standards Reading 72.6% 77.9% 79.0% 86.7% Math 59.0% 62.4% 62.7% 69.5% Writing 70.8% 60.7% 70.6% 74.0% Science 40.6% 41.5% 43.5% 49.3% Percentage of state 4th graders meeting state standards Reading 74.% 79.5% 74.% 79.5% Math 59.9% 60.8% 59.9% 60.8% Writing 55.8% 57.7% 55.8% 57.7% Science N/A N/A N/A N/A
Education Policy Analysis Archives Vol. 14 No. 23 6 Riverside Highland Variable 2003-04 2004-05 2003-04 2004-05 Washington State NAEP 4th graders by achievement level Below basic 25% 25% Basic 41% 41% Proficient 31% 31% Advanced 3% 3% State SAT rankingb 1 1 1 1 Source: State assessment website. http://www.k12.wa.us/assessment All sites and participants are identified by pseudonym. a Washington Education Association opposed the reel ection of the state superintendent in 2004. b Ranking of scores among states where more than half of students take the SAT. Source : Wood, 2005. Data Collection The primary approach to the research was qualitativ e, subscribing to a view of human phenomena as socially constructed (Vygotsky, 1978) from individuals perceptions of reality. The research process adhered to interpretive research tr aditions respectful of emergent design, multiple perspectives, and inductive analysis (Denzin & Linc oln, 1994; Eisner, 1991; Erickson, 1986; Lincoln & Guba, 1985; Wolcott, 1994). Data collection included 47 observations, 45 of these in classrooms where dr op-in privileges were negotiated to minimize observer effects. Semi-structured interviews were conducted (Fontana & Frey, 1994; Rubin & Rubin, 1995; Krueger, 1994), individually with administrators (eight overall) and collectively in focus groups with teachers (four overall). On two occasions, district superintendents elected to be interviewed with either an assistant superintendent or a district assessment officer present. At the end of yea r 2, a continuing assistant superintendent was interviewed during a period of transition to a n ew superintendent. Quantitative data collected from documentary sources, primarily in the form of de mographics and statistical indicators, augmented observation and interview data. Data Analysis and Conceptual Frameworks Data analysis was emergent in character (Den zin & Lincoln, 1994; Erickson 1986; Maxwell, 1992; Wolcott, 1994). Ongoing analysis using cons tant-comparative method (Glaser & Strauss, 1967; Strauss & Corbin, 1990, 1994) yielded preliminary interpreta tions subject to further investigation. Following completion of fieldwork (Wolcott, 19 94), thematic content analysis (LeCompte & Preissle, 1993; Miles & Huberman, 1994) was undert aken with reference to three types of analytic frameworks: Bronfenbrenners (1979) stages of ecological analysis, a sociological model promoting identification of connections between values, orga nizations, relationships, and practices; Knapps (1997) articulation of the stages involved in progressing toward educational reform; and three theories of change specifically in response to test-driven educational accountability. Ecological analysis The first and most general type of analytic framework used in this study was Bronfenbrenners (1979) model of human social behavior. Ecological analysis promoted
NCLB: Local Implementation and Impac t in Southwest Washington State 7 macroanalysis of the ideologies of participants regarding education and educational practice as compared to policy objectives; exoanalysis of local organizational and management contexts; mesoanalysis of the perceptions and working relationshi ps among teachers and with administrators; and microanalysis of classroom activities and behaviors involving or affecting students. Among other things, this model encouraged recognition of inconsistencies between participants statements of values and observed behaviors or allocations of instructional time. Stages of reform The second type of analytic framework used in this study was Knapps (1997) conceptualization of reform implementation. A ccording to this model, successful reforms pass through four stages: incremental increase based on countable indicators suggesting movement toward the reform vision; professional learning including questioning of current classroom practice and getting professional development; grafting of reform on to familiar practices; and full embodiment of reform (pp. 258-59). This conceptualization offe red a continuum for identifying the progress toward reform achieved at the research sites. Theories of change The third type of analytic framework responded to Elmore et al.s (2001) charge that testdriven accountability lacks an explicit theory of change, working from mere presumption that highstakes testing will improve educational outcomes wi thout positing how or why. Data were analyzed from the perspectives of three theories of change which offered differing explanations of the blackbox mechanisms linking accountability policy and desired outcomes. Standards as effectors In the first, published shortly before enactment of N CLB, the National Research Council described standards as the effector of improved teaching and learning (NRC, 1999) and standards-based tests as documenting achievement gains. Washingtons content standards, the Essential Academic Learning Requirements (EALRs), served as the intended basis of the states test, the WASL, at grades 4, 7, an d 10. The EALRs were generally accepted by teachers but not so the WASL (Washingt on Education Association, 2004). Similar to the situation in 19 other states, Washington lacked standards-based tests for grades 3, 5, 6, and 8 and, to comply with NCLBs testing requirements for grades 3, had filled in the gaps with off the shelf / norm-referenced tes t[s] (Skinner, 2005, p. 86). This study offered opportunity to examine the actual role of standards in classrooms in a standards-based accountability context. Tests as effectors The year after enactment of NCLB, Mabry, Poole, Redmond, and Schultz (2003) described a second logic in which tests rather than stand ards, propel change. In this model, accurate test scores would lead to valid score interpretations (see AERA, APA, & NCME, 1999; Messick, 1989) manifested in appropriate reward s and sanctions which motivate teachers and students toward improved learning. In actuality, frequent scoring disputes or errors have shown that accurate scores cannot be presumed (e.g., Galley, 2003; Hall, McDonald, Scherich, Vickers, & Zebrowski, 2001; Lemire, 1998) and that score in terpretation and use may be contested or invalid (e.g., Haney, 2000; Toenjes & Dworkin, 2002). This study included opportunity to examine the function of tests and their consequences in schools and classrooms. Data as effectors A third theory of action proposed by Marion and Gong (2003) posited an information loop in which school scores reported to parents, the media, and the state gen erate expectations and pressures and influence resource allocations. Expectations and pressures filter through administrators to teachers, effecting im proved teaching and learning subsequently reflected
Education Policy Analysis Archives Vol. 14 No. 23 8 in improved scores. In practice, aggregation of student scores to the school level presents a measurement issue: few, if any, state tests of stud ent achievement have been validated for purposes other than measuring individual student achiev ement (Shepard, 2002). This study provided opportunity to reveal whether public reporting of aggregated scor es affected pressures on teachers and their classroom practices. Test score gains were considered in light of measurement literature regarding the tendency of scores to rise gradually and then to fall pr ecipitously with implementation of new tests (Linn, 2000), potentially a result of score pollution (Haladyna, Nolen & Haas, 1991) as school personnel administering the tests succumb to pressure to raise scores (e.g., Cannell, 1987; Haney, 2000; Sternberg, 2002). This research also took in to consideration the potential for intensified susceptibility to score inflation when policy outcomes are unattainableNCLBs goal of universal literacy and numeracy empirically counterindicated by NAEP scores over a 30-year period (Haertel, 2002) or beyond the control of those held acco untable (Harmon, 1995) NCLBs Annual Yearly Progress (AYP) requirements described as especia lly unrealistic for schools and districts where small enrollments of identified subgroups of students threaten reliability and statistical stability in aggregated test results (Haertel, 2002; Hill, 2002). Discussion of findings is organized by five themes that emerged from the data during analysis: local uncertainty during a period of fe deral policy adjustments and lawsuits challenging NCLB; local compliance and resistance regarding curriculum authority and alignment with standards and standards-based tests; professional impact of NCLB on local educators; disparate conceptions of education in the best interests of children; and validity concerns expressed by participants. Volatility and Uncertainty Test-driven accountability in general has been the subject of sustained criticism (Baker, Linn, Herman, & Koretz, 2002; Herman, Baker, & Linn, 2004; Koretz, 2001; McLaughlin, 1991; Rumberger & Palardy, 2005), as have practices derived from NCLB specifically (e.g., Hendrie, 2005b). The volatility of the national policy context du ring the period of this study is suggested by the requests from 47 states for waivers from thei r own federally approved AYP plans (Olson, 2005a, 2005c), by state legislation intended to overrule NCLB (Sack, 2005), and by a half-dozen lawsuits filed against NCLB (Archer, 2005a; Hendrie, 2005a; Keller & Sack, 2005; Olson, 2005b). Within the state of Washington, Superinten dent of Education Te rry Bergeson, narrowly reelected in 2004 over teacher oppositi on to state testing, joined a multi-state effort to urge federal policy modification (Bergeson, 2005). Locally, a superintendent in southwest Washington joined a regional consortium which publicly blasted NCLB as lacking common sense (Benson, et al., 2004).3As policy requirements were contested and modified, the administrators participating in this study struggled to understand and meet NCLBs proficiency requirements and to identify and allocate resources to ensure their compliance. During the period of the study, procedural adjustments to NCLB complicated local forecasting of nearand long-term ex pectations to be met. 3 The superintendent mentioned was not from one of the districts participating in this study.
NCLB: Local Implementation and Impac t in Southwest Washington State 9 Proficiency Requirements: Changes and Cautions National controversies were often reflected in locally expressed concerns. For example, in year 1 of the study, all participating district an d school administrators expressed anxiety a bout NCLBs unrealistic requirements regarding the academic proficiency of students. In year 2, following NCLB adjustments (Davis & Hoff, 2005; Hoff, 2005b; Olson, 2004), some of which had been correctly anticipated by local district administrators, they expressed less concern about reaching NCLBs literacy and numeracy goals: 100% [proficiency]? I dont th ink th ats real. But will we see more students coming closer to meeting standards? I th ink so. (Highland district administrator, June 23, 2005) [Because the state allo ws some deviations fr om policy, ] we dont have to have all kids performing at standard to say we have 100%. Is it a [foreseeable] reality? I think we can get pretty close. (Riverside district administra tor, June 29, 2005) One principal echoed this qualif ie d optimism: Youre never going to get 100%, but I wouldnt be surprised if we hit 90% proficiency (Riverside principal, June 2 2, 2005). The other principal expressed less confidence: We did really well in reading last year but th ere were some kids who, despite every opportunity, did not show growth on the WASL. When you look at all the resources and the wonderful teachers we have, if theres any place it could happen, it would be here. Bu t, for whatever reason, not every child can be at the same place. I dont know. I dont know (Highland principal, June 28, 2005) Teachers also expressed uncertai nty, one wondering, When will the test change to meet the kids? Or will we hav e to keep bending to meet the needs of the test? (Riverside teacher, grade 3, May 3, 2005). Teachers doub ts were related in part to their surprising lack of information about NCLB, which a Highland district administrator explained in this way: [Implementing NCLB] is proving to be fa irly diffic ult. Our issue has been and continues to be a lack of awareness of the law and a sense of urgency Teachers think it involves other school districts, bigger school districts. (June 23, 2005) Funding: No Easy Options Implementing procedures to meet external a ccountability typically requires new resources often not easily found or allocated (e.g., Cohen, Raudenbush, & Ball, 2003; Gillborn & Youdell, 2000). At the national level (e.g., Broder, 2005; Gewertz, 2005; Rotherham, 2005) and at the state level, NCLB-related expenditures were raising quest ions (e.g., Archer, 2005b). In the third year of policy implementation (year 2 of the study), seven lawsuits against NCLB were filed in state and federal courts, two of which challenged the law for violating its own prohibition against unfunded mandates (Anderson, 2005; Archer, 2005a; see also Olson, 2005b). Reflecting the national context, uncertainty rela ted to identifying or obtaining the funds needed to meet NCLB requirements was cause fo r local concern. Although both partici pating districts had recently succeeded in raising funds through elections, one during the timeframe of the study, finding sufficient resources to meet policy challenges remain ed daunting. A Highland district administrator reported: My concern is the unfundedness of it. Its unfunded in a very peculiar, particular way. A l ot of the [federal grant] funding, such as is available, is driven by
Education Policy Analysis Archives Vol. 14 No. 23 10 aggregate numbers. In a small district like this, we cant even begin to [apply for] competitive grants. We have kids here who need ad ded support, but we have three or six or twelve [eligi ble kids]. We cant meet th e needs test, but the needs are here. [Using funds from a private gr antor,] we put into place things that really helped studen ts and teachers with learning, but that money has gone. We dont know how we can keep on doing that. Its pretty worrisome. (June 23, 2005) As some states and districts ac ro ss the country pressed for more policy modifications (Hoff, 2005b), others risked funding cuts through pol icy defiance (e.g., Hoff, 2005a, 2005c) and opting out by refusing Title 1 money in order to avoid NCLB penalties (Davis, 2005; Zehr, 2004). A local district administra tor noted the financia l difficulties of opti ng out, even in a district which had successfu lly passed a funding levy just a few months previously: I dont know if people are ultimately goin g to lo ve me or curse me for this, but Ive said I think its time to take Title 1 dollars off the table. We did refuse it for two years, but we need the money, frankly. (Highland di strict administrator, June 23, 2005) Insufficient personnel resources were among the specific funding-related issues identified by administrators, including a principal who lamented: [NCLB] is top-downYou will do this, You will do thatand theres no mone y to do it. Its like trying to ge t blood out of a turnip. Were in a small district, so we have sma ll funds and fewer people to do [what NCLB requires]. Our district doesnt have the funds to pa y teachers for 23 days [of professional development] before the school year starts. (Riverside principal, June 22, 2005) Fiscal concerns sometimes led to conceptual questions about NCLB s policy framework. Ideally, an interviewee summarized, I would only hold people accounta ble for what could be adequately resourced (H ighland district administ rator, June 23, 2005). Compliance and Resistance Achievement of the states own educational re form initiative (Washington State Senate, 1992) had not been fully attained in the deca de since its legislation (Borko, Wolf, Simone, & Uchiyama, 2003; Mabry, Poole, Redmond, & Schultz, 2003), suggesting it would not be realistic to expect NCLBs reforms to be fully implemented two or three years after enactment. Evidence of the level of reform achieved locally included incre mental increase based on countable indicators (test scores) and increased teacher training and professional learning, corresponding to the three early levels of Knapps (1997) continuum of classroom implementation of reform: incremental increase based on countable indicators suggesting movement toward the reform vision; professional learning, including questioning of current classroom pr actice and getting professional development; grafting of reform onto familiar practices ; and full embodiment of reform (pp. 258-59). All but one administrator described NCLB as providing leverage to effect needed change without dama ging working relationships with fa culty by appearing heavy-handed. Although they described NCLBs expectations of universal litera cy and numeracy as unrealistic or unreasonable, most administrators indicated that the policys positive consequences outweighed the negative, one explaining: I keep calling it the moon sh ot in public education. You might as well go for a big goal. Were going to be b etter for having tried. Special education leads the list [of children Im worried about]. ELL probably comes second. Children of
NCLB: Local Implementation and Impac t in Southwest Washington State 11 generational poverty are, in this school district, particularly disadvantaged. The frame of reference here is, Who are thes e kids, and what are they doing here? I worry about those kids, who tend to be invi sible. We are now st arting to look at what it means to grow up in generational poverty. (Highland district administrator, June 23, 2005) Students who had been neglecte d or invisi ble to teachers were being better taught, most administrators said. Their testim ony suggested that they were trying to use NCLB to effect reforms they themselves had long considered wo rthy, reforms to which they considered some teachers resistant. Curricular Control Reform not only broadened focus to include prev iously neglected students but also involved changes to curriculum, pedagogy, and roles. Prior to NCLB, teachers had largely determined the delivered curriculum, often through classroom-l evel adaptation of district text adoptions or curriculum guides, but external author ity was exercising increasing control. Most administrators believed this shift represented educational improvement, and they approved; most teachers did not. The battle over fluff Most participating administrators reported that NCLB was having a positive impact on instruction through the elimination of fun projects of low educational utility. Such projects proved to be contested local ground, most administrators pressing for instruction that could lead directly to measurable gains while teachers defended activities they described as developmentally appropriate and motivational. Administrators reported: From my standpoint, the overall impact [of NCLB] i s positive. I dont think teachers would necessarily agree with that. My favorite line has been, Teach the stuff, forget the fluff. Its been ha rd to sit down with a teacher and talk about that when parents lik e it if the students are happy and having fun in school [But] if the only thing thats fun is stuff that doesnt have any relevance, were missing the boat. (Riverside di strict administrato r, June 29, 2005) Teachers are more accounta ble for teaching the curriculu m, which benefits kids. In years past, teachers ta ught their favorite thingstheir dinosaur unitswhich may have been fun for the kids, but it didnt really enhance reading and writing. What we found was that teachers were doing fluffy activities that really werent focused on learning. (River side principal, June 22, 2005) We are actually moving toward a curric ulum that is alig ned with our state standards. Thats defi nitely a positive. The tradition in this district was that teachers pretty much did their own thing in their own way Teachers have been getting more precise about setting targ ets Theyre making those targets obvious to students and then provid ing instruction that actually helps st udents to attain the targets. (Highland district administrator, June 23, 2005) Teachers almost unanimously disagreed, citing fu n proj ects as producti ve for motivation, for positive educational climates and, instrumentally, for learning.
Education Policy Analysis Archives Vol. 14 No. 23 12 Their ability to determine whether or not to in clude fluff affected teachers perceptions of their professional decision-making opportunities. Some teachers expressed fear of being caught engaging students in activities they considered be neficial but not necessarily standards-based, for example: Today, we were writing plays. We had to [res erve] that [until] after [taking the] WASL. For 25 minutes, [students] we re trying on costum es, showing each other Now theyre passionate about the plays. But theres no EALR for feather boas. If someone had come in, what was I supposed to say? (R iverside teacher, grade 3, May 19, 2004) In the struggle to retain such activities, te ac hers indicated they were losing ground: You get [students interested] when theres something that th ey care about. When we can t do that, one of our tools has been taken away. (Highland teacher, grade 4, May 2, 2005) The principals at both particip ating schools had previ ously been teachers, but the views of only one corresponded with that of th e teachers. The Highland principa l explained the utility of some fluff, saying that the pressure to eliminate it is not really coming from me. Howev er, they do h ear comments from the district level. I taught here for ten years before I became a princi pal, so Im aware of a lot of the projects. Even Im like, Oh, I dont want to stop doing hot air balloons, you guys! because I love that [project]. Because of the requirements of No Child Left Behind or the state standards, [teachers] feel they cant do those things any more without doing a disservice to kids. Should we throw out some pet projec ts? I ha ve a teacher who goes to Africa every summer. She brings in some very rich cultural things she can integrate with all differen t areas and do those things she is really passionate about. The kids can learn from that. (June 28, 2005) At the microsystem level of cla ssroom implementation (Bronfenbr enner, 1979), the inclusion or exclusion of fluff affected stud ents educational oppo rtunities and experien ces. Activities were observed which might indeed have been considered fluff, as illustrated by the following observations of two fourth-grade classrooms. Some observations appeared to support teachers views about the importance of such activities for building bases for learni ng, and some exhibited all the ambiguity suggested by the dichotom y of opinion regarding appropriate classroom practice. Observation, Riverside Elementary School, April 16, 2004 The classr oom was noisy and happy-sounding, chairs filled with fourth-graders and kindergartners reading aloud together. From a side table, the fourth-grade te acher offered snacks of raisins, fruit-filled cookies, and pretzels. Two fourth-grade girls knelt, holding a Dr. Seuss picture book open for the ki ndergartner seated before them, and took turns reading ea ch page to her. One read fluidly with expression, We sat in shock and disbelief. Oh, no! we moaned, Oh, no! A fourth-grade boy move d his finger across lines in a counting book, pointing to each word as the kindergartner read alou d. The younger boy needed help with most words but ga mely persisted, his older tutor readily assisting and correcting him: Then, the kindergartner read. No, ten, the fourth-grader corrected. Move.
NCLB: Local Implementation and Impac t in Southwest Washington State 13 No, more. Three girls balanced a tiny picture book open on a de sk, touching it only when the book threatened to topple forwar d, as their kindergartner read the final page confidently, Y ou can help me. You are not too little to play. The younger child snapped the book closed, saying, Goodbye. All done. How did she do? the kindergarten teacher asked. Was she a good reader? The fourth-graders no dded, affirming that she was. As the reading groups fi nished, five students were looking at the class crayfish, who obligingly waved its feelers as a boy gently took it out of its tank. Two girls allowed the class turtle to walk around the carpet. The fourth-grade teacher of this classroom expressed a ppreh ension that her students, acting as reading tutors, were not engaged in what externa l authorities might consider appropriate reading instruction at fourth-grade level. She herself, however, felt such activities were advantageous for her students, especially for promoting the comp etence and confidence of struggling fourthgrade readers, and supported by research (see Green, Alde rman, & Liechty, 2004; Kerr & Verhaeghe, 2005). She explained: If you have a child who feels socially an d academ ically incompetent, [She or he is] not going to be as succe ssful as a student who feels, Im going to share this, and Im not going to be laughed at. Ther es not a focus on that at all, except in what the research is saying but the state isnt. At least, I dont see it. (May 19, 2004) Another teacher presided over an activity with less clear connection to l earning or reform initiatives: Observation, Highland Elementary School, grade 4, March 23, 2004 Two boys carried car parts featuring speci alty painting into the classroom. Jeri, the guest speaker, followed them, greeted the chil dren, and explained that she had once attended their school and that their teacher had once been her teacher. How many of you like car s? she asked. Hands shot up. Anybody not like cars? Two hands were raised am id general laughter. Jeri exhibited posters, a paint gun, pa inted c ar parts, and an airbrushed mailbox with flames spreading backward from the mail open ing. She explained how she had sanded and painted each piece. There were Ooooohs of interest when she mentioned she also painted Harley-Davidson motorcycles. She described the training she had undertaken confiding, I was the only girl in 58 people. Wow! the teacher said. Arent we proud! The cl ass broke into spontaneous applause. The teacher encouraged the children to think about their future careers while she turned on a promotional video Jeri had brought. After showing the video, both women encouraged th e girls in th e class to consider traditionally male-d ominated careers, Jeri describing herself as living proof that women could succeed and enjoy such occupations. No clearly academic instruction took place duri ng th is observation. Ha d she been challenged, the teacher might have justified the time allocated to this activity as career education or a proactive countering of gender stereotypes. That she was feelin g less free to attend to such things was indicated in a later interview: Theres a rigidity Im feelin g now that I didnt feel ab out fo ur years ago. Its coming from the demands from the state as to whats required for us to accomplish with our students. When I came here, we had our own curriculum
Education Policy Analysis Archives Vol. 14 No. 23 14 and, as a team, we determined what was best for students. No w, guidelines are set. (Highland teacher, grade 4, May 2, 2005) Curricular alignment and other test prep For coherence, standards-based testing systems rely on alignment of both curricula and the tests to content standards. Aligning curricula with content standards was described by local educators as an appropriate route to improvin g scores. Few teachers opposed the EALRs, and several praised them, suggesting substantial local compliance with the content standards. One teacher said, for example: Its been valuable, the things weve been going th rough the last ten years. Weve improved the curriculum, which was very valuable. Weve really improved our writing instruction. The EALRs and all this stuff has been very helpful. (Highland teacher, gr. 3, May 2, 2005) A colleague offered a quick caveat: The guidelin es are quite good, bu t the pressure that comes on teachers and on students is not (Highland teacher, gr. 4, May 2, 2005). A few teachers indicated that the content emphasized by th e EALRs was appropriate, one describing the authenticity of the math curriculum: Im teaching connected math, which is ap plicabl e to problems the students will have in real life. There are no drill-andkill sheets saying, H eres the algorithm. Now, do it. Instead, they explore, li ke, Heres a strategy. Aha! (Riverside teacher, gr. 6, May 3, 2005) Most teachers, however, expr essed concern about the overwh elming numbers of content standards, the Grade Level Expe ctations (GLEs) that elaborated the EALRs, one reporting: [With] the sheer number of [GLEs] Im expected to teach if you dont start in the firs t week and just go for it, you simply cannot teach to all of them. Teaching this past year was like a zoo, a boring lecture with no art until after the WASL, working endlessly but getting nothing done. I feel like I taught the [GLEs] and worked on classroom management all year. (Riversi de teacher, gr. 4, May 19, 2004) This teachers reference to a lack of art instruct ion reflected a general wo rry among teachers that untest ed subjects were being squeezed out of the curriculum. The fine arts, social studies, and sometimes science were describe d as threatened, for example: I came from a school in Ma ssachus etts where we had a strong integrated social studies program that drove the whole year. Here, when the topic of social studies came up, it was [suggested that] I could do it after the WASL. Whoa! That just blew my mind! (Riverside te acher, gr. 4, May 19, 2004) Teaching to the test itself, rather than to the s tandards, was uniformly perceived to be inappropriate, although no interviewee suggested that aligning the curriculum to state content standards might have the effect of pervasive teach ing to the (presumably) standards-aligned test. Not all parts of the WASL (i.e., not all subjects at all grades) had been subjected to external examination to determine alignment to the EALRs, leaving open the possib ility that a teachers implementation of a curriculum aligned to the standards migh t not result in her students preparedness for the WASLs content. Given such unresolved tensions, classroom observation data unsurprisingly indicated neither total policy compliance (e.g., uninterrupted attent ion to tested topics) nor to tal resistance. Some test prep was clearly observed, as indicated in the following vignette.
NCLB: Local Implementation and Impac t in Southwest Washington State 15 Observation, Riverside Elementary School, grade 4, April 16, 2004 As the observation began, the teacher hurriedly ex plained, This is not our usual seating arrangement, referrin g to a grid of student desks all separated from each other by rows and aisles. Its just to get the kids ready for WASL testing next week. The daily agenda posted at the front of the room indicate d that the first four hours were to be devoted to acad emic areas related to the WASL and NCLB.75 hours to math (math and math journal), 1 hour to reading (independent reading and reading aloud), an d 1.25 hours to language arts (cursive writing, a spelling test, an d a writing assignment). Although no other academic areas (e.g ., sc ience, art) were listed, several maps on display indicated that the teacher did give attention to social studies. Half of the classroom was encircled by an historical timelin e near which 3x5 index cards indicated milestones. One card, for example, offered information in a childs handwriting: 32 George Wash ington is born with a drawing of a bottle-wielding baby in a crib. Anothe r read, Abner Doubleday invents baseball and featured a drawing of a man and a floating ball. In this setting, studen ts were worki ng on math worksheets with WASLlike fractions problems such as: Parts of a Whole 5 parts shaded 7 parts in all 5/7 is shaded number of shaded parts 5 numerator number of parts in all 7 denom inator The teacher fielded questions from the st udents, circulating am ong them to help individuals. Occasionally, a student asked a question aloud, and the teacher gave a general explanation, sometimes writing information at the board. After huddling at a desk near the front of the room, two girls excitedly brought their work on a math problem to the teachers desk. One danced in place, pumping her fists in the air. When the teacher confirmed their solution, they crowed, We did it! We did it! In other observations, odd-fittin g connections between classroom activities and the state test suggested Knapps (1997) grafting of new onto old practices, as indicated in the following vignette. Observation, Highland Elementary School, grade 4, April 16, 2004 The teacher appeared to be working with students on several th ings simultaneously. As flowers were projected on a screen at the front of the classroom, the teacher quizzed students, asking them to id entify the flowers in writing. Part two of the quiz followed, the teacher distributing to each student a para graph about wildflowers. The teacher directed students to insert needed punctuation into the paragraph, then noted to the ob server, This is one thing we do to work on writing mechanics. Although observations such as these revealed a mixed picture in terms of classr oom devoted to test preparation, interviews suggested that te achers were acutely cons cious of expectations related to the states st andards-based te st. One teacher worried aloud: I feel I have to be accountable for [hav ing] every minute of the day match some kind of EALR. The rush to get through the EALRstheres so much pressure!
Education Policy Analysis Archives Vol. 14 No. 23 16 Its like a checklist of all these things without any depth. Here, we talk about WASL all year long. Its not How ar e we making our kids passionate about learning? Its about whether theyre passing the WASL. Its What is your percent [of students scoring at the] profic ient [level]? (River side teacher, gr. 4, May 19, 2004) Washingtons hybrid testing system administered the Iowa Test of Basic Sk ills (ITBS) and Iowa Tes t of Educational Development (ITED) to stud ents in non-WASL grad es. Teachers indicated that pressure was most in tense at grade levels targeted for the WASL: A lot of the pressure is on the fourth-grade leve l. [In third grade,] I have a lot of freedom to teach in ways th at I think are best for the kids. For example, were using [a commercial] reading curriculum wh ich doesnt go with the best practices in literacy that Ive learned. I feel I ha ve the freedom to figure out ways I could use that curriculum. I wont as much next year because the th ird grade will be required to take the WASL. (River side teacher, gr. 3, May 3, 2005) As suggested by the NRC theory of change (NRC, 1999), state content standards were in deed influencing change in wh at was taught and learne d in classrooms in the two schools participating in this study. In large part, ho wever, the standards appeared to be influential because the states standards-based test functioned as an enforcement lever, as suggested by the Mabry, Poole, Redmond, and Schultz (2003) theory of change. That is, evidence from this research suggested that it was an inters ection of these two models that drove classroom change. Whether the change constituted reformin th e sense of radically improved teaching and learningremained a clouded issue with local proponents passionately ar guing each side. Professional Impact Data suggested that the reshaping of professi onal roles and responsibilities, begun with the enactment of Washingtons standards-based educat ion reform act (Washington State Senate, 1992), had intensified under NCLB. Teacher Autonomy Federal determination of subjects to be prio ritized (i.e., reading and math) and state determinatio n of content to be taught (i.e., the EALRs) were justified locally by a district administrator: We are actually moving toward a curric ulum that is alig ned with our state standards. Thats de fi nitely a positive. By doing so I think we ha ve taken steps toward making the educational experience in this district more equitable. The tradition in this dist rict was that te achers pretty much did their own thing in their own way, isolated from each other, an d it was possible for kids to have an extraordinarily fine and rich experience with a particular teacher [with an] inventive curriculum, but pity the kids whose parents did not come to the schoolhouse door and say, I want that teacher. (Highland district administrator, June 23, 2005) Other administrators explicitly approved NCLB s mo ves toward teache r accountability, one arguing that neither teacher professiona lism nor autonomy were threatened:
NCLB: Local Implementation and Impac t in Southwest Washington State 17 Ive been working in state schools si nce the 1970s. Throughout that time, teachers have looked at the profession as their profession, and they dont want somebody else controlling it Id argue NCLB doesnt have any effect on your teaching. No one is walking around from the federal office to check what youre teaching. I think they have a lot of acad emic freedom to teach, but it has to be put into perspective. Theyre employees, not individual contractors. We pay teachers a good wage, and th ey should be accountable, not doing something just because its fun. Theyre professionals. Being professiona l is also about taking pride in a job done well. I can cite thousands of examples where theyre not demonstrating the professionalism they say they deserve. Teachers here will tell you they dont have enough input. Im not buying that story because they have opportunities to get involved, [like] pa rticipat[ing] on our School Improvement Plan, but theyre not stepping forward or theyre not doing it unless they get paid for it. (Riverside prin cipal, June 22, 2005) Another approved the accountability systems power to strengthen teachers expertise: This district is all about achievement, what h appens with learning in the classroom. Teachers have to be acco untableIf a child isnt getting the concept, what are you going to do about it? It makes them be more precise. Their repertoires may have to grow; they may need to collabora te as a team; the team may have to act as a microc osm; they may have to do more communicatingand th ey are. (Riverside district administrator, June 29, 2005) One of the two principals, expr essing co nfidence in teachers, had reorganized the school week to promote collaborative decision-making by grad e-level teams. This prin cipal struggled with a desire to buffer teachers from stress as opposed to passing pressures and expectations on to teachers, as described in the theory of change by Marion and Gong (2003): The teachers in this building are, I think, a little uniq ue. Of 22 teachers, I think thr ee have less than six years experi ence. Our teachers [have always been] dedicated to the kids. We have staff meetings every week. [This year, each grade-level team also has a] weekly collaboration. I wanted each team to [have more time to] collaborate and be very intentionally strategic about how they use their time. I know how much stress th ey put upon themselves to make sure theyre reaching all the standards I am torn between feelin g I want to take stress off the teachers and do what I can to support them so they can do their best for teaching the kids [but] someti mes I feel like I shou ld be doing more, pressuring them more. (Highlan d principal, June 28, 2005) To teachers, the pressure was alre ady palpable. Pressure at fourth -grade level, where the WASL was administered, was great enough that Highlands fourth-grade teachers had petitioned their district for relief: The fourth-grade teachers me t with the s uperintendent and principals. Most of us cried and said, You cant do this to us any more. Not until we met with the administrators did we fe el we were getting support I think they finally understood what we were going through. They were lookin g at the test, not looking at the people. (Highl and teacher, gr. 5, May 2, 2005) In addition to individual stress es, some teachers conn ected pressu re with th e good of the school community, for example:
Education Policy Analysis Archives Vol. 14 No. 23 18 I know that theres funds attached to [WASL scores which] ar e meaningful to the whole school. So I feel like I have to teach to it. I guess there is a lot of pressure. (Riverside teacher, gr. 4, May 3, 2005) Some administrators anticipate d that NCLB would nudge from the profession teachers who resisted local policy implementation efforts, and data did suggest that stress was triggering flight responses. Each case of participant attrition am ong teachers bet ween years 1 and 2 of the study involved fourth-grade teachers: two left thei r schools, two changed grade levels, and one explained, I moved from grade 4 to grade 5 to escape the WASL, but [wit h NCLB,] Im back in it again (Highland teache r, gr. 5, May 2, 2005). Another fifth fourth-grade teache r retired earlier than planned at the end of year 2. In addition, a third-grade teacher had considered but rejected an opportunity to move to fourth grade, explai ning, I dont want to do it because of the pressure (Riverside teacher, gr. 3, May 19, 2004). Despite some personnel losses and other concerns, most participating administrators described NCLBs impact as basically positive. From a policy p erspective, the shifts they described suggested progress on Knapps (1997) scale of effective reform, change from incremental change and conglomerates of old and new practices toward full reform. On the other side of the personnel line, most teachers described the shift as infringing on classroom decision-making and adaptation, with disempowering and discouraging professional impact. Said one: [Im] a younger teacher with 25 years to go be fore r etirement. Th e pressure of the whole profession, the parents hammering at you, the kids so needyI cant do this for 25 years. Its hard. I see myself burnt out in 10 years, bailing completely. That first year, it was nice to have those [state] guidelines, so I knew what to do. Now, Ive gotten my self into a pattern. There are things I would like to do, but I cant. (Hig hland teacher, gr. 4, May 2, 2005) Most teachers resisted not the content standard s themselves but the reli quishment of authority to the stan dards that implemen tation of standards-based refo rm, as they experienced it, was effecting. Data-Driven Administrators NCLB had altered the responsibilities of district and school administrators. Most notably, local data coll ection, use, and reporting had increase d: While some of the data is not necessarily applicable, it has helped to create a new kind of environment in which we are actively seeking data (Highland district administrator, June 23, 2005). All but one administrator expressed satisfaction with school improvements resulting from stronger focus on data: The data collection [for NCLB] has been help ful in creating change. We have used the data to implement ou r goals each year. That data is dissected by our staff, me, and the school improvement pl anning team, to see how we are making a difference. We try to link teacher goal s to the school goals. Every trimester, each teacher sets goals in reading and math according to where we see gaps in the data. The accountability piece comes tomorrow when we report our goals and their measurement to the [School] Boar d. (Riverside prin cipal, June 22, 2005) In addition to coordinating scho ol and teacher go als, district ad ministrators reported that data promoted improved curriculu m and personnel decisions: The data helped me make the case on behalf of chil dren that, in this district, we need to adequately resource reading and writing instruction. For example, last
NCLB: Local Implementation and Impac t in Southwest Washington State 19 year, I was able to advocate successfully for the adoption of a core reading series plus a variety of leveled readers for take -homea great many materials. I also advocated for a literacy coach, actually two, one of whom has worked out very well. Those have been positive changes th at have occurred fairly directly from the new federal mandate. (Highland district admini strator, June 23, 2005) I never thought Id be th e type of person whod s a y I love data, but its helping us make decisions that are better for kids Its providing supporting information to plan programs for studen ts. Some of our decisions have been reaffirmed by the data [b ut not all.] Wed been using SSRSustained Silent Readingfor a long time [wit h] the idea that, if you get kids reading, theyll be better readers, [but] the data didnt support that. Now were more conscious about whether were giving th em skills, helping them pr actice appropriate skills. We are looking at the datascientifically based dataas we ma ke selections for the future. (Riverside district administrator, June 29, 2005) The teacher-oriented Highland pr inci pal appeared as an outlier in describing classroom-level data as more useful than test scores, explaining: [If I were to choose betwee n WASL data and teacher-ge nerated data to see how students are doing,] Id go to the teachers Were teaching a child. You take out that human piece at times wh en you just look at data. (Highland principal, June 28, 2005) One local administrator described NC LB-reported data as useful for clarifications to parents: Its especially helpful in communicating to parents, in help ing them look past their emotional feelings about their children, to share data showing measurements. NCLB has helped us communicate what your child is learning. (Riverside district administrator, June 29, 2005) Another administrator worried about data that was ov erwhelming or that eluded clear interpretations: I try to look at where we are with the WASL. I try to look at Why are we here? Why did we go down from l ast year? We had 89.9% meet standards, so we did fairly well in reading last year. We did try to look at why those 17 students who didnt meet standards didnt meet stan dards. When we pulled [their records], several of them were on an IEP or they were ELL. We tr ied to look at what kind of services we were giving to those kids. Its a struggle. I constantly think I should be reading more data. (Hig hland principal, June 28, 2005) All participating administrators, even those who extolled th e usefulness of NCLB-required data, reported that the benefits came at a cost. One described administ rator flight in advance of 2008, when Washingtons graduation test was slated to determine eligibility for high school graduation, and 2014, when NCLBs 100% proficiency in readin g and math requirements at grades 3 were to be achieved: If you get a group of senior administrators toge ther, often the first joke is, So, are you pre or post ? or Are you pre or post ? Its taking a big toll I believe in the standards, absolutely. I d ont want to go back, but it is wearying. (Highland distri ct administrator, June 23, 2005) The Highland superintendenc y did ch ange hands in both years of this study.
Education Policy Analysis Archives Vol. 14 No. 23 20 Public Pressure When it comes out in the paper, holy cow! a teacher exclaimed (Riverside teacher, kindergarten, May 3, 2005) in describing the personal impact of public reporting. In interviews, teachers iden tified media attention as sources of personal stress less frequently than conversations with parents, although the parents might partly have been responding to media alerts. An administrator confirmed a pervasive general effect: Teachers cant get away from knowing NC LB is out there because, every time you turn around, you hear it. At some point, it is to excess As a staff, I think the teachers are worried. Its always in the backs of their minds, especially the fourth-grade group. (Highland principal, June 28, 2005) Overall, observation and inte rview data suggested that sc ores, reported fo r government accountability purposes and disseminated by the state and districts to th e schools, were stronger influences on local practice than media reports. In this regard this research elaborates the theory of change articulated by Marion and Gong (2003) by suggesting that different components in accountab ility systems information loops are, in reality, differentially weighted in terms of their effects on outcomes. In addition, the trickling down of information from government sources to public outlets (e.g., news media, school report cards) suggested governments opportunity to shape information for public consumption, a mediating influence not clearly registered in that theory. In the Interests of Children Analyzed with the assumption that all parties shared a desire to provide whats best for kids (e.g., Beach, et al., 2003; Behuniak, DeVito, Rivera, & Fremer, 2001), the data revealed brightline ideological distinctions. As teachers and administrators described and rationalized different approaches to curriculum and pedagogy, their justifications exposed ideological conflict at the macrosystem level (Bronfenbrenner, 1997). Both tea chers and administrators invoked science (i.e., data or research) to support their views. While NCLB suggested policy-makers appreciation of generalizable systemic change teachers descriptions of their responsibilities su ggested appreciation of improvements resulting from individual adaptation Against NCLBs pursuit of large-scale research-based and scientifically based curricula and practices, te achers, in effect, argued for locally developed programming informed by knowledge of contex tual and individual student circumstances. Teachers expressed worry that developmentally appr opriate practice, allowing for tailored educational delivery to students, was increasingly threatened by systemic reform initiatives. The magnitude of the inappropriateness, in terms of diff iculty level that they perceived, was suggested by a teacher who complained: We have parents who couldnt answer the questions on the fourth-grade test. Theres a developmental inappropriatene ss with WASL-type questions. (Highland teacher, grade 4, May 2, 2005) Most but not all teachers who remarked on th e developm ental appropriateness of the WASL expressed negative opinions, especially those prep aring students to take the WASL as indicated in this exchange between colleagues: Students are expected to write a five-par agraph essay for the fourth-grade WASL. So, gradually, we work as a team, coming together to try to find a way to address
NCLB: Local Implementation and Impac t in Southwest Washington State 21 these developmentally inappropriate expectations. We have to make it work. (Riverside teacher, gr. 4, May 3, 2005) I disagree. They can rise to the five-par agra ph essay. (Riverside teacher, gr. 6, May 3, 2005) Teachers often articulated comp lex understandings of student success that were not necessarily measurable and that tolerated divergent outcomes, for example: I think of success [being defined] in two waysmeeting the EALRs and being wh ere you need to be grade-level-wise. But [a student] could [alternatively] be academically successful in a developmen tally appropriate way. I have a student who is below grade level in skills but has grown this year, and that s academic success, too. (Riverside teac her, grade 4, May 19, 2004) At these sites, teaching and lear ning wer e occurring within an ideological conflict involving scale, the accountabilit y system focused on large-scale improvement and educators focused on improvements within their spheres of professional influence. District administrators focused on district improvements, principals focused on school improvements, and teache rs focused on individual student growth. Interv iewees perspectives about whether educational accountability was serving the interests of chil dren was largely a function of th eir positions within the system. Each scale levelthat of the policy-maker, district administrator, school principal, and teachereng endered its own particular interest con centrations, each suggesting different values. At the same time, each scale level was interconnected with the others, such that pursuit of the values associated with one level could be inhibited or pr evented by pursuit of any of the others, suggesting the power of each ideology to threaten the system. Externally mandated and enforced through locally significant rewards and sanctions, the values implicit in NCLB were gaining precedence over those emanating from participating classrooms. If indeed NCLB was more than an actits an attitude, as declared by U.S. Secretary of Education Margaret Spellings (National Public Radio, January 31, 2005), it was an attitude more consistent with that of administrators than with that of participating teachers in this study. Student Anxiety A month before the WASL would be administered to fourth-graders, a third-grader surprised her teacher by describing her test anxiety in a show-and-tell session described in the following vignette. Observation, Highland Elementary School, grade 3, March 21, 2004 After lunch, children s eated themselves in a circle on the classroom rug and shared moments from their out-of-school lives: a family bu ilding project, an an nual camping trip, a ferry ride, a playground altercation. One boy said, Im not go ing to be here in April, but Im going to be really sad because Im still going to have to take the WASL. Although the WASL would not be administered to these third-graders until the following year, his comment sparked an excited response from a classmate. Its a really, big, big, big test! a diminutive blonde girl exclaimed. Her excitem ent was contagious, and a small hubbub of chatter erupted. One at a time, the teacher said. Its like the biggest test of the year! the girl went on emphatically. Big meaning important or big meanin g long ? the teacher prompted.
Education Policy Analysis Archives Vol. 14 No. 23 22 Its important and its long! the girl repor ted. It takes like two weeks or three weeks to finish itI dont remember how long, but not one week. It takes more than one week. The WASL test, she paused, dropping her hands in her lap. I mean, everybody in my class at my other sc hool was scared of the WASL test! And everybody didnt want to move up to the next grade where theyd have to take the WASL test! My sister had to take it three or fo ur times already, a boy said somberly. And did she survive? the teache r asked hopef ully. The boy nodded. My brother had to take it a bunch of times, anot her boy sa id. His brain is rotted. Many teachers reported that thei r st udents experienced test anxiety. In a group interview, two teachers broke down in tears as they reported: Boy, do they ever feel that pressure on the WASL They sit down before those blank pieces of paper. I always have a handful of ve ry anxious students. Last year and this year, a studen t was getting counseling and losing sleep. In college, they teach you to scaffold learning But you push them into the WASL, and [she weeps] youre not even allowed to say, Youll be OK. Thats what teachers do But, with the WASL, were asked to sit and watch them struggle. (Highland teacher, gr. 4, May 2, 2005) We know, in the long run, theyll do fi ne, [she weeps ] but were torturing them along the way when we co uld be encouraging them. (Highland teacher, gr. 3, May 2, 2005) Several described thei r efforts to mi nimize student anxiety, efforts they considered largely futile given the constraints of external directives. One said: This year, [my students] were pr etty stressed out. I tried to defuse it. I got them little incentives, little treats. We had a conversation about What does smart mean? Does it mean you are better pers on if you can pass a test? [But] I had kids in tears both last year and this year. [If they d ont understand one part of the test,] you cant really explain it to them. Theyre just sitting there, like, Help! and theres nothing you can do about it. (River side teacher, gr. 4, May 19, 2004) A fourth-grade teacher desc ribed how she had down-played the importance of the WASL to protect her son from negative psycho-emotional effects: As a parent, I chose not to let my child know his WASL scor e. He failed the writing part of the WASL. Of course, he asked, Howd I do? What should I tell him? Hes a straight A student at th e middle school, and its not an easy curriculum there. I didnt want to tell him he was a failure or that he didnt pass it. I felt he would have given up on writing. He writes to the point. He doesnt elaborate, and that doesnt give them what [the scorers] want. (Highland teacher, gr. 4, May 2, 2005) Although few administrators mentioned test pressure on st udents, teachers r eports were confirmed by a principal, who elaborated with information about how school resources had been redeployed specifically to combat test-related pressure: We had a couple of students this year really kind of break down. They were worried. One little boy thought his teacher was going to lose hi s job if he didnt do well on the WASL. There were a couple of fourth-grade classrooms where we
NCLB: Local Implementation and Impac t in Southwest Washington State 23 heard that. I dont kn ow if the parents had read so mething, but it didnt come from the teachers. These students were worried and upset. The pressure [on the teacher] transfers to the student, which concerns me. Let me give you an example [of how we tried to handle that]. [During test administration,] we pulled teache rs [from non-WASL classrooms] and [had them] as extra support persons in the [WASL] classrooms because the [fourthgrade] teachers are so fraz zled. Id rather get a sub for a third-grad e teacher or a fifth-grade teacher at this point. (Highland princi pal, June 28, 2005) This voluntary added expense represented a cons cious decision to temporarily risk the educational quality in non-fourth-grade classrooms to comb at WASL anxiety in fourth-grade classrooms. Some parents were reportedly concerned about the effects of test stress on th eir children, said teachers who shared their concern. One noted: One of the things I have heard a lot in the community is pa re nts being really upset about the stress their kids are unde rsecond grade, fi rst grade. I had a conversation with a woman at the coffe e shop, and she said her second-grader knows the term [WASL and] is scared a bout it. I had one kid who was really scared about taking the [state-required] ITBS. We talked about what are tests for, made the ITBS a game. That wouldnt work in fourth grade. They know the WASL is the big test. (Riverside teacher, gr. 3, May 19, 2004) Teachers indicated that some pare nts had become alarming ly over-focused on test scores, increasing the pressure on th eir children. One reported: I have a [third-grade] girl who cries if she doesnt ge t 100% on all her papers. Her pa rents make her practice so she can do well on the test. At pa rent conference time, we had another mom who was not in terested in talking about how her kid was doing. Instead, she wanted to know, Whats going to be on that ITBS? and how was I going to get him ready. (H ighland teacher, gr. 3, May 2, 2005) Teachers described their e ffort s to reduce parent anxiety as no more successful than their efforts to reduce student anxiety saying, for example: We still cant get parents [of our elementary st udents] to believe that, if [students] fail the WASL, they will not be retained [at grade level]. We tell them every year, but the parents think they will fail at every grade le vel if they dont pass the WASL and maybe the ITBS too. (Hig hland teacher, gr. 4, May 2, 2005) I just about died one day when a kid in my class, who wa s taking the ITED, asked, Is this going to go in my permanent record? Apparently, hed been talking to his mom, and they were scared (Highland teacher, gr. 3, May 2, 2005) It was the personal effects on children of th e WA SL, not yet a high-stakes test for fourthgraders, that worried teachers most, some to te arsthe impact on self-esteem, motivation, and aspirations. One te acher pointed out: Think of the child. Youre talking abou t him p ersonally: Heres your score and you failed. Maybe it was an unrealistic expectation set for him. Mayb e it was not even fair to ask that child to do what we asked him to do, bu t now he knows hes a failure. Ive given lots of standardized tests but, for fourth grade, there is something about [the WASL] that is more difficult. (H ighland teacher, gr. 4, May 2, 2005)
Education Policy Analysis Archives Vol. 14 No. 23 24 Teachers Goals for their Students Most teachers considered the stress they perceived in their students to be discontinuous with their own teaching goals and priorities. More th an their goals regarding student achievem ent, teachers goals regarding student self-esteem, motivation, and aspirations were at odds with the testfocused accountability system implemented by the state and reinforced by NCLB. They noted the conflict explicitly, one saying: If you have a child who is passionate about learning, you can build on that. I feel the EALRs put a little bit of a bloc k on being able to do that (Riverside teacher, gr. 3, May 19, 2004). Some teachers observed that so me of th eir goals and their work with children was beyond the measurement capacity of the stat e test, one teacher commenting: Im excited my students actually learned. I could see growth f rom beginning to end. I think [its because of] all the wo rk we did as a community. We worked really, really well. Were a very close gr oup. It was my main objective that we would treat each other with respect, help each other. They came in as a very needy classlots of home issues, not a lot of support at homeso I wanted to make sure that they supported each other. They came in very low but, when we ju st did the STAR reading test to see where they were, I had only four students below fourth-grade. When they came in, I think I had ten. Theyr e getting division, and they know all about Lewis and Clarknot that thats on the WASLand theyve matured as individuals, problem-solving their little problems. (Highland teacher, gr. 4, May 2, 2005) Others noted two chilling effects of testings limitations: narrowing of the curriculum to tested subjects and lowering of childrens self-esteem and motivation when their favorite subjects were lost. Teachers said, for example: My main concern is the emphasis our sc hool puts on academ ic success in math and readi ng. Everything else is secondary to that. That says academic success is more important in those tw o areas than the others, whereas my opinion would be that theyre all at th e same [level of impor tance]. (Riversi de teacher, gr. 4, May 19, 2004) Three of my kids had sort of checked ou t alre ady in fourth grade. Two of them are fabulous artists, but theres no art [spe cialist] in this school. And, with the worries of WASL and trying to get things done, I didnt do much art. But if I had, it would have been an area in which they could have been more successful, and other kids could have seen them as successful. (Riverside teacher, gr. 4, May 19, 2004) Validity Concerns According to the Mabry, Poole, Redmond, and Schultz (2003) theory of change, test scores would lead to valid score interpretations which result in appropriate rewards and sanctions and improved teaching and learning. However, local educators expressed concerns in non-technical language about the validity of actual uses and consequences of test scores (see AERA, APA, & NCME, 1999).
NCLB: Local Implementation and Impac t in Southwest Washington State 25 Teachers expressed concerned that testing was damaging the educational system by squeezing out some subjects and pedagogies, a matter of systemic validity (Frederiksen & Collins, 1989). Formative feedback for improving the system did not fully compensate for this damage, partly because of timing: Its frustrating partly because we dont get WASL results until Fall when were into another school year (Hig hland principal, June 28, 2005). Some teachers expressed ecological or instructional validity concerns regarding discrepancies between the ethos of instruction and the ethos of testing. At this age, writing really requires scaffolding, said one (Highland teacher, gr. 4, Ma y 2, 2005), noting that teachers were forbidden to provide scaffolding during test time. A colleag ue illustrated the discontinuity with mastery approaches and encouragement: In the classroom, they get more chances. You can say to a student, Ill work with you. Y oull get it. Oh look, youve got two of them right today! On the WASL, you dont have the chance to say that. (Highland teacher, gr. 4, May 2, 2005) There were local concerns regarding construct validity whether th e WASL was measuring what it was intended to measure. Some teachers expl icitly doubted whether the state tests really measured their intended constructs for example: The [new WASL] science test has nothing to do with science instruction. Its an intelligence test in disguise. Their level of psychological maturity, their attention problems there are lots of other things being measured. (Highland teacher, gr. 4, May 2, 2005) In addition to measuring intellig ence rather than achiev ement, teachers sugge sted that the WASL actually measured another rival construct student motivation to perform well on the test: There are students who know the content but just don t care and th ey are very capable of giving you nothing. (H ighland teacher, gr. 4, May 2, 2005) I have really, really good writers, bu t the pro mpt didnt motivate some of them. I have a girl who said she was sick and tired of doing this test so, on the math test, she wrote, I dont know how to do this for every question. She was just worn out. I observed one boy, who had just finished the sixth grade STAR test at the 6.0 grade level, looking at a WASL question that demanded inferences and then turning in an almost empty test booklet. I know hes not performing below grade level, but his sc ore will show that he is. (H ighland teacher, gr. 4, May 2, 2005) Teachers also expressed concern about test scor es that re ported status but not progress, one saying: I dont know what my WASL scores are going to sh ow. In writing, I d be surprised if any of my st udents passed. But, from what they were doing at the beginning of the year to wh at they can do now, thats OK with me. (Highland teacher, gr. 4, May 2, 2005) A principal who, when asked abou t valid data sources, said Id go to the teachers (June 28, 2005) rather than to test scores in order to understand a childs achievement, suggested that classroom assessmentsformal and informalprovided better info rmation than the state test. A teacher elaborated this point, simultaneously suggesting that the state test provoked suspicion: You can tell a lot with classr oom assessments, and theres no trickery. But not everything theyve learned shows up on the WASL (Highland teacher, gr. 4, May 2, 2005).
Education Policy Analysis Archives Vol. 14 No. 23 26 Issues of consequential validity the adequacy and appropriateness of inferences and actions based on test scores figured prominently in inter view data (Messick, 1989, p. 13; emphasis in original). Data from teachers strongly indicated that most felt the consequences of testing borne by children were excessive and inappropriate. The appropriateness of the consequences to schools was also challenged. The validity of inferences of school quality based on WASL scores a ggregated to the building level, as required by NCLB, was as much in doubt among teachers as a m ong many researchers (e.g., Shepard, 2002). One participating teacher described collective teacher re sistance to a district interpretation of her schools WASL scores: [We were told,] You went down in all three [sta te-tested] areas. We had actually made a tremendous jumpwe were up 14 point s in some areas, down just one point in one area We blew a gasket. They didnt listen or even acknowledge the improvement we had made the year before because we were down that one point. (Highland teacher, gr. 4, May 2, 2005) Teachers intuitions about the invalidity of judg me nts based on test results fed resistance to the test and desire to prot ect their students and scho ols from what they pe rceived to be unfair consequences. Conclusion Test data indicating that the two partic ipating schools were meeting state and federal achievement targets suggested successful policy implementation. However, the fuller picture of implementation provided by classroom observati ons and interviews of personnel was much less clear about whether local implementation of NCLB had been successful. Test scores did show incremental increase based on countable indicators, and observations did reveal that both the s tate content standards and the state test were influencing curriculum and pedagogy. However, observations indicated more g rafting of old onto ne w classroom practices than full embodiment of reform (Knapp, 1997), perhaps indicating a predictable movement through phases of policy acceptance or perhaps, al ternatively, indicating that most teachers were defying policy where they could and complying where they could not. Whether the grafting constituted improved teaching and learning was a ma tter of local contention, administrators often considering the movement toward a more standards-based curriculum beneficial and teachers almost unanimously worrying about lost subjects and pedagogies and the erosion of their opportunities to devise or adapt teaching to their particular students. The grafting was observed to produce uneven and o dd-fitting conglomerates of practices suggesting unresolved ideological conflicts (see Bronfenbrenner, 1979). While some curricular effects were praised, attention to untested subjects (e.g., art, social studies) was diminishing. Teachers reported that standards and testing were overwhelming what they considered to be best or developmentally appropriate practices. Interview data revealed that, for participants in both administrative and instructional positions, NCLBs impact was problematic. At the di strict level, worries about unrealistic student achievement targets were giving wa y to accep tance of policy adjustments that would probably allow continued demonstration of acceptable progress. Administrators expressed satisfaction that historically low-performing students, previousl y ignored, were beginning to receive appropriate educational services. At the same time, NCLB requirements and sanctions were sufficiently harsh that, despite constrained resources, one local ad ministrator had recommended refusing Title 1
NCLB: Local Implementation and Impac t in Southwest Washington State 27 funding to escape them. Small districts ineligibi lity for federal grant assistance exacerbated local financial burdens. At the school level, teachers registered more anxiety about the impact of test scores than about either content standards or the public reporting of test results, all three having been described as drivers of educational reform (NRC, 1999; Ma bry, Poole, Redmond & Schultz, 2003; Marion & Gong, 2003). Teachers test-related anxiety infr inged on their job satisfaction and drove a surprisingly high number of participants from the fourth-grade during the two years of this study. Teachers deepest concerns about the impact of current accountability initiatives, as indicated by both frequency and poignancy of expre ssion, centered on their students. As attention to test scores rose, teachers r eported that children wer e increasingly suffering from test stress, a few students in one school requiring therapeutic intervention. Their efforts to help students cope with the pressure, teachers said, were essentially futile While most administrators extolled data-driven improvements, one principal summarized the worry: Were not teaching robots. I guess I dont know if No Chi ld Left Behind is really talking about a chil d or a piece of data that isnt being left behind (Highland principal, June 28, 2005).
Education Policy Analysis Archives Vol. 14 No. 23 28 References Anderson, P. (2005, May 25). NCLB is not a mandate, says GAO in clarification [letter to the editor]. Education Week p. 33. American Educational Research Association, American Psychological Association & National Council on Measurement i n Education. (1985). Standards for educational and psychological testing. Washington, D. C.: Authors. Archer, J. (2005a, April 13). Con necticut pledges firs t st ate legal challenge to NCLB law. Education Week, p. 38. Archer, J. (2005b, May 25). States eyeing expense of hand-s cored tests in light of NCLB rules. Education Week, p. 1. Baker, E. L., Linn, R. L., He rm an, J. L., & Koretz, D. (2002 ). Standards for educational accountability systems. CRESST Line, Winter, 1. Beach, K. D., Gee, J. P., Greeno, J. G., Haertel, E. H., Lee, C. D., Mehan, H. B., Mislevy, R. J., Moss, P. A., & Pullin, D. C. (2003, April). Opportunity to learn and assessment: Psychometric and sociocultural perspectives Presentation to the American Educational Research Association, Chicago. Behuniak, P., DeVito, P., Rivera, C., & Fremer, J. (2001, June). In search of fair larg e-scale assessments Presentation to the CCSSO Large-Scale Assessment Conference, Houston. Benson, D., Moses, M., Saul, D., Erb, J., Ot to, D., Phillips, D., & Erickson, J. (2004). No Child Left Behind fails the tes ts of common sense Western States Benchmarking Consortium, unpublished memorandum. Bergeson, T. (2005). New Year's addr ess to Washington educators [video]. Olympia, WA: Office of the Superintendent of Public Instruction. Berliner, D. C. (2002). Educational research: The hardest science of all. Educational Researcher, 31(8), 18. Berliner, D.C. & Biddle, B.J. (1995). The manufactured crisis: Myths, fraud and the attack on America's public schools Reading, MA: Addison-Wesley. Booher-Jennings, J. (2005). Below the bubble: "E ducational triage" and the Texas Accountability System. American Educational Research Journal, 42 (2), 231. Borko, H., Wolf, S.A., Simone, G., & Uchiyama, K.P. (2003). Schools in transition: Refor m efforts and school capacity in Washington state. Educational Evaluation and Policy Analysis 25(2), 1711. Broder, D. S. (2005, February 28). T he unsustainable budget. Oregonian p. D7.
NCLB: Local Implementation and Impac t in Southwest Washington State 29 Bronfenbrenner, U. (1979). The ecology of human development Cambridge, MA: Harvard University Press. Burlingame, M. (1993). Compliance and local educational p olitics and policy. Planning and Changing, 24( 1.2), 86. Cannell, J. J. (1987). Nationally normed elemen tary achiev ement testing in America's public schools: How all 50 states are above the national average. Educational Measurement: Issues and Practice, 7 (2), 5. Chatterji, M. (2005). Evidence on "what works" : An argumen t for extended-term mixed-method (ETMM) evaluation designs. Educational Researcher, 33 (9), 3. Cohen, D. K., Raudenbush, S. W., & Ball, D. L. (2003). Resource s, instruction, and research. Educational Evaluation and Policy Analysis 25 (2), 119. Cook, H. G., Fitton, R., Viator, K., Phillip s, S. E., & Gertz, S. (2001, June). Politics and highstakes testing Presentation to the CCSSO Large-Scale Assessment Conference, Houston. Cuban, L. (1988). Why do some re forms persist? Educational Administration Quarterly, 24 (3), 329. Darling-Hammond, L. (1990). Instructional policy in to practice: The power of the bottom over the top. Educational Evaluation and Policy Analysis, 12(3), 339. Davis, M. R. (2005, Februar y 9). Utah is unlikely fly in Bush's school ointment. Education Week pp. 1, 21. Davis, M. R., & Hoff, D. J. (2005, April 20). Questions linger over NCLB policy shifts. Education Week, pp. 21. Denzin, N. K. & Lincoln, Y. S. (1994). Handbook of qualitative research Thousand Oaks, CA: Sage. Eisenhart, M., & Towne, L. (2003). Contestation and chan ge in national polic y on "scientifically based" education research. Educational Researcher, 32 (7), 31. Eisner, E. W. (1991). The enlightened eye: Qual itative inquiry and the en hancement of educational practice NY: Macmillan. E lmore, R., Fuhrman, S., Koretz, D., Herman, J. L. Linn, R. L., & Baker, E. L. (2001, April). Holding accountability s ystems accountable: Research-based standards Presentation to the annual meeting of the American Educatio nal Research Association, Seattle, WA. Erickson, F. (1986). Qualitative methods in research on teaching In M. C. Wittr ock (Ed.), Handbook of research on teaching (3rd ed., pp. 119161). New York: Macmillan. Fontana, A. & Frey, J. H. (1994). Interviewi ng: The art of science. In Denzin, N. K. & Lincoln, Y. S. (Eds.), Handbook of qualitative research (pp. 361). Thousa nd Oaks, CA: Sage.
Education Policy Analysis Archives Vol. 14 No. 23 30 Frederiksen, J. R. & Collins, A. (1989). A systems approa ch to educatioal testing. Educational Researcher, 18 (9), 27. Fuhrman, S. H. & Elmore, R. F. (1990). Unde rstandi ng local control in the wake of state education reform. Educational Evaluation and Policy Analysis, 12 (1), 8296. Fullan, M. (1991). The new meaning of educational change New York: Teachers Colle ge Press. Galley, M. (2003, June 18). More errors are seen in th e sco ring of tests, Boston researchers say. Education Week, 22(41), 10. Gewertz, C. (2005, April 20). Critics questi on use of offs hore firms for online tutoring. Education Week, p. 14. Gillborn, D., & Youdell, D. (2000). Rationing ed ucation: Policy, practice, reform and equity Buckingham, England: Open University Press. Glaser, B. G. & Strauss, A. I. (1967). The discovery of gr ounded theory. Chicago, IL: Aldine. Goodson, I. & Foote, M. (2001). Te sting times: A sch ool case study. Education Policy Analysis Archives, 9 (2). Re trieved July 23, 2006, from http://epaa.asu.edu/epaa/v9n2/ Green, S. K., Alderman, G., & Liechty, A. (2004). Peer tutoring, individu alized interv ention, and program monitoring wi th at-r isk second-grade readers. Preventing School Failure, 49 (1), 11. Haertel, E. H. (2002, April). Technica l considerations in the u se of NAEP to confirm states' achievement gains Paper presentation to the annual meeting of the American Educational Research Association, New Orleans, LA. Haladyna, T. M., Nolen, S. B., & Haas, N. S. (1991). Raising standard ized achievement test scor es and the origins of test score pollution. Educational Researcher, 20 (5), 27. Hall, B., McDonald, J., Sche rich, H., Vickers, D., & Ze browski, M. (2001, June). Performance scorin gThe challenges and opport unities in today's assessment world Panel presentation to the CCSSO Large-Scale Assessment Conference, Houston. Haney, W. (2000). The myth of the Tex as miracle in education. Education Policy Analysis Archives, 8 (41). Retrieved July 23, 2006, from http://epaa.asu.edu/epaa/v8n41/ Harmon, M. H. (1995). Responsibility as paradox: A critique of rational d iscourse on government. Thousand Oaks, CA: Sage. Hendrie, C. (2005a, May 4). NCLB cases fa ce hurdle s in the courts. Education Week pp. 1, 22. Hendrie, C. (2005b, April 20). NCLB transfer policy seen as flawed. Education Week, pp. 1, 14 15.
NCLB: Local Implementation and Impac t in Southwest Washington State 31 Herman, J. L., Baker, E. L., & Linn, R. L. (2004, Spring). Acco untability systems in support of student learning: Moving to the next generation. CRESST Line, 1. Hill, R. K. (2002, April). Examinin g the reliability of accountability systems Paper presentation to the annual meeting of the American Educat ional Research Associ ation, New Orleans, LA. Hill, R. K., & DePascale, C. A. (2003). Reliability of No Ch ild Left Behind accountability designs. Educational Measurement: Issues and Practice, 22 (3), 12. Hoff, D. J. (2005a, May 4). Education department fines Texas for NCLB violation. Education Week pp. 27, 31. Hoff, D. J. (2005b, April 13). States to get new opti ons on NCLB law. Education Week pp. 1 38. Hoff, D. J. (2005c, March 9). Te xas stands be hind own testing ru le; Move puts state at odds with NCLB policy. Education Week pp. 1, 23. Keller, B. & Sack, J. L. (2005, April 27). Un ion, states wage frontal attack on NCLB. Education Week, pp. 1, 18. Kerr, H. V. & Verhaeghe, J. P. (200 5). Effects of explicit reading strategies instruction and peer tutoring on second and fifth raders' reading comprehensio n and self-efficacy perceptions. Journal of Experime ntal Education, 73 (4), 291. Knapp, M. (1997). Between syst emic reforms and th e mathematics and sc ience cla ssroom: The dyna mics of innovation, implementa tion, and professional learning. Review of Educational Research, 67, 227. Koretz, D. (2001, April). Toward a framework for evaluatin g gains on high-stakes tests Paper presentation to the annual meeting of the National Council on Measurement in Education, Seattle, WA. Krueger, R. A. (1994). Focus groups: A practical guide for applied research (2nd ed.) Thousand Oaks, CA: Sage. LeCompte, M. D. & Preissle, J. (1993). Ethnography and qualitative design in education al research (2nd ed.). San Diego: Academic Press. Lee, J., & Wong, K. K. (2004). The impact of ac countability on racial and socioeconomic equity: Considering both school resources and achievement outcomes. American Educational Research Journal, 41( 4), 797. Lemire, C. (1998, May 14). Court rejects TAAS score dispute. Austin Am erican-Statesman Lincoln, Y. S. & Guba, E. G. (1985). Naturalistic inquiry Newbury Park, CA: Sage. Linn, R. L. (2000). Assess ments and accou ntability. Educational Researcher, 29(2), 416.
Education Policy Analysis Archives Vol. 14 No. 23 32 Linn, R. L., Baker, E. L., & Betebenner, D. W. (2002). Accountab ility systems: Implications of requirements of the No Child Left Behind Act of 2001. Educational Researcher, 31(6), 3 16. Louis, K. S., Febey, K., & Schr oeder, R. (2005 ). State-mandated accountability in high schools: Teachers' interpretations of a new era. Educational Evaluation and Policy Analysis, 27(2), 17704. Mabry, L., Poole, J., Redmond, L., & Schultz, A. (2003) Local impact of state-mandated testing. Education Policy Analysis Archives, 11 (22). Marion, S. & Gong, B. (2003, October) Evaluating the validity of state accountability systems. Paper presented to the a nnual Reidy Interactive Le cture Series, Nashua, NH. Maxwell, J. A. (1992). Un derstanding and validity in qualitative research. Harvard Educational Review, 62 (3), 279. McLaughlin, M. S. (1991). Test-based accountability as a re form strategy. Phi Delta Kappan, 73(3), 24852. Meier, T. (1995). Standardized tests: A clea r and pres ent danger. In Levine, D., Lowe, R., Peterson, B., & Tenorio, R. (Eds.) Rethinking Schools: An Agenda for Change New York: The New Press. Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13). New York: American Counc il on Education, Macmillan. Miles, M. B. & Huberm an, A. M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Thousand Oaks, CA: Sage. Miskel, C. & Song, M. (2004). Passing Reading First : Prominence and proce sses in an elite policy network. Educational Evaluation and Policy Analysis, 25 (2), 89. National Research Council (1999). Hi gh stakes: Testing for tra cking, promotion, and graduation Washington, DC: National Academy Press. National Research Council. (2002). Scientific research in educa tion Washington, DC: National Academy Press. No Child Left Behind Act. (2001). Public Law No. 107, 115 Stat. 1425, 2002. Olson, L. (2004, September 22). No Child Left Behind Act changes weighed. Education Week pp. 31, 34. Olson, L. (2005a, M ay 25). Florida ga ins flexibility on NCLB provisions. Educat ion Week, pp. 18, 20. Olson, L. (2005b, April 6, 2005 ). NCLB st udy: State support for schools' improvement varies. Education Week, p. 9.
NCLB: Local Implementation and Impac t in Southwest Washington State 33 Olson, L. (2005c, July 13, 2005). Re quests win more le eway under NCLB. Education Week, pp. 1, 20. Patton, M. Q. (1997). Utilizationfocused evaluation (3rd ed.). Thousand Oaks, CA: Sage. Pellegrino, J. W., & Goldman, S. R. (2002). Be careful what you wish foryou may get it: Ed ucational research in the spotlight. Educational Researcher, 31 (8), 15. Raywid, M. A. (1990). Pseudo-reform, incremental reform, and restructuring. Phi Delta Kappan 72, 139. Rossman, G. B., Corbett, H. D. & Dawson, J. A. (1984). Inten tions and impacts: A comparison of sources of influences on local school systems. Urban Education, 21 (1), 86. Rotherham, A. J. (2005, January 12). In educ ation, no pundit left behind. Oregonian, p. B9. Rubin, H. J. & Rubin, I. S. (1995). Qualitative int erviewing: The art of hearing data Thousand Oaks, CA: Sage. Rumberger, R. W., & Palardy, G. J. (2005). Test scores, dropout rates, and transfer rates as alternative indicators of high school performance. American Educational Research Journal, 42 (1), 342. Sack, J. L. (2005, April 27). U tah passes bill to trump 'No Child' law. Education Week pp. 22 25. Sarason, S. B. (1982). The culture of the school and the pr oblem of change (2nd ed.). Boston: Allyn and Bacon, Inc. Sarason, S. B. (1990). The predictable failure of educational re form: Can we change course before its too late? San Fran cisco: Jossey-Bass. Schaffer, E. C., Nesselrodt, P. S. & Stringfield, S. C. (1997 ). Impediments to reform: An analysis of destabilizing issues in ten promising programs Arlington, VA.: Educational Research Service. Shepard, L. A. (2002, April). Building bridges between classr oom and large-scale assessments Paper presented at the annual meet ing of the American Educati onal Research Association, New Orleans, LA. Shipps, D. (2003). Pulling together: Ci vic capacity and urban school refo rm. American Educational Research Journal 40 (4), 841. Sipple, J. W., Killeen, K., & Monk, D. H. (2004) Adoption and adapta tion: School district reponses to state imposed learni ng and graduation requirements. Educational Evaluation and Policy Analysis, 26 (2), 1438. Sirotnik, K. A. (Ed.) (2004). Holding accountability a ccountable: What ought to matter in public education. New York: Teachers College Press.
Education Policy Analysis Archives Vol. 14 No. 23 34 Skinner, R. A. (2005, January 6). The state of the states. Quality Counts 2005: No Small Change. Bethesda, MD: Education Week. Smith, M. L. (1991). Meanings of test preparation. American Educational Research Journal, 28 521. St. Pierre, E A. (2002). "Science" rejects postmodernism. Educational Researcher, 31 (8), 25. Sternb erg, R. J. (2002). The "Janus principle" in psyc hometric testin g: The example of the upcoming SAT-I. The Score newsletter of American Psychological Association Division 5, Evaluation, Measurement, and Statistics, 24(2), 3. Strauss, A. & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedur es and techniques Newbury Park, CA: Sage. Strauss, A. & Corbin, J. (1994). Grounded theory methodol ogy: An overview. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (pp. 273). Thousand Oaks, CA: Sage. Stringfield, S. C., & Yakimows ki-Srebnick, M. E. (2005). Pr om ise, progress, problems, and paradoxes of three phases of accountability: A longitudinal case st udy of the Baltimore City Public Schools. American Educational Research Journal, 42 (1), 435. Toenjes, L. A. & Dworkin, A. G. (2002). Are in creasing test scores in Texas really a myth, or is Haney's myth a myth? Educational Policy An alysis Archives, 10 (17), 1. Tyack, D. & Cuban, L. (1995). Tinkering toward Utop ia: A century of pu blic school reform Cambridge, MA: Harvar d Universi ty Press. Tyack, D. & Tobin, W. (1994). The grammar of schooling: Why has it been so hard to change? American Educational Research Journal, 31 453. Vygotsky, L. S. (1978). Mind in so ciety: T he development of hi gher mental process. Cambridge, MA: Harvard University Press. Washington State Senate. (1992). Washington Substitute Senate Bill 5953: Act relating to education. Olympia, WA: Authors. Washington Education Association. (2004). Washington Education Asso ciation testimony to the House Education Committee Retrieved Decemb er 5, 2005, from http://www.washingtonea.org/ staticconte nt/edreform/testimony/Hse92004.htm Wise, A. (1979). Legislated learning: The bureaucr at ization of the American classroom. Berkeley: University of California Press. Wolcott, H. (1994). Transforming qua litative data: Description, analysis, and interpretation Thousand Oaks, CA: Sage.
NCLB: Local Implementation and Impac t in Southwest Washington State 35 Wood, S. O. (2005, September 1). Wash ington outpaces Oregon on SATagain. Oregonian pp. B1, B3. Zehr, M. A. (2004, Septem ber 8) Wary districts shif t or forgo federal funds. Education Week p. 6. About the Authors Linda Mabry Washington State University Vancouver Jason Margolis Washington State University Vancouver Email: firstname.lastname@example.org Linda Mabry 's research interests relate to the assessment of the achievemen t of K students and to program evaluation. Jason Margolis 's research interests include educational reform and teacher development.
Education Policy Analysis Archives Vol. 14 No. 23 36 EDUCATION POLICY ANALYSIS ARCHIVES http://epaa.asu.edu Editor: Sherman Dorn, University of South Florida Production Assistant: Chris Murr ell, Arizona State University General questions about ap propriateness of topics or particular articles may be addressed to the Editor, Sherman Dorn, email@example.com. Editorial Board Michael W. Apple Univ ersity of Wisconsin David C. Berliner Arizona St ate University Robert Bickel Marshall U n iversity Gregory Camilli Ru tg ers University Casey Cobb Un iversity of Connecticut Linda Darling-Hammond Stanfo rd University Gunapala Edirisooriya Yo un gstown State University Mark E. Fetler Ca lifo rnia Commission on Teacher Credentialing Gustavo E. Fischman Arizon a St ate Univeristy Richard Garlikov Birm ingham, Alabama Gene V Glass Arizon a St ate University Thomas F. Green Syr a cuse University Aimee Howley Ohi o University Craig B. Howley Ohio University William Hunter Un iver sity of Ontario Institute of Technology Daniel Kalls Ume U niversity Benjamin Levin Uni v ersity of Manitoba Thomas Mauhs-Pugh Green Mo untain College Les McLean Uni v ersity of Toronto Heinrich Mintrop University of California, Berkeley Michele Moses Arizon a St ate University Anthony G. Rud Jr. Purdue University Michael Scriven We ster n Michigan University Terrence G. Wiley Arizona St ate University John Willinsky Un iver sity of British Columbia
NCLB: Local Implementation and Impac t in Southwest Washington State 37 EDUCATION POLICY ANALYSIS ARCHIVES English-language Graduate -Student Editorial Board Noga Admon New York University Jessica Allen University of Colorado Cheryl Aman University of British Columbia Anne Black University of Connecticut Marisa Cannata Michigan State University Chad d'Entremont Teachers College Columbia University Carol Da Silva Harvard University Tara Donahue Michigan State University Camille Farrington University of Illinois Chicago Chris Frey Indiana University Amy Garrett Dikkers University of Minnesota Misty Ginicola Yale University Jake Gross Indiana University Hee Kyung Hong Loyola University Chicago Jennifer Lloyd University of British Columbia Heather Lord Yale University Shereeza Mohammed Florida Atlantic University Ben Superfine University of Michigan John Weathers University of Pennsylvania Kyo Yamashiro University of California Los Angeles
Education Policy Analysis Archives Vol. 14 No. 23 38 Archivos Analticos de Polticas Educativas Associate Editors Gustavo E. Fischman & Pablo Gentili Arizona State University & Universidade do Estado do Rio de Janeiro Founding Associate Editor for Spanish Language (1998003) Roberto Rodrguez Gmez Editorial Board Hugo Aboites Universidad Autnoma Metropolitana-Xochimilco Adrin Acosta Universida d de Guadalajara Mxico Claudio Almonacid Avila Univ ersidad Metropolitana de Ciencias de la Educacin, Chile Dalila Andrade de Oliveira U n iversidade Federal de Minas Gerais, Belo Horizonte, Brasil Alejandra Birgin Ministerio de Educacin, Arg entina Teresa Bracho Centr o de Investigacin y Docencia Econmica-CIDE Alejandro Canales Un iversida d Nacional Autnoma de Mxico Ursula Casanova Arizona St ate University, Tempe, Arizona Sigfredo Chiroque Inst ituto de Pedagoga Popular, Per Erwin Epstein Loyo la University, Chicago, Illinois Mariano Fernndez Enguita Un iversidad de Salamanca. Espaa Gaudncio Frigotto Un iversidade Estadual do Rio de Janeiro, Brasil Rollin Kent Universida d Autnoma de Puebla. Puebla, Mxico Walter Kohan Un iversidade Estadual do Rio de Janeiro, Brasil Roberto Leher Universidade Estadual do Rio de Janeiro, B rasil Daniel C. Levy Un iversity at Albany, SUNY, Albany, New York Nilma Limo Gomes Un iversidade Federal de Minas Gerais, Belo Horizonte Pia Lindquist Wong Californi a State University, Sacramento, California Mara Loreto Egaa P ro grama Interdisciplinario de Investigacin en Educacin Mariano Narodowski Universida d To rcuato Di Tella, Argentina Iolanda de Oliveira Un iversidade Federal Fluminense, Brasil Grover Pango Foro Lat inoamericano de Polticas Educativas, Per Vanilda Paiva Universidade Estadual Do Rio De Janeiro, Brasil Miguel Pereira Catedratico Un iversidad de Granada, Espaa Angel Ignacio Prez Gmez Universid a d de Mlaga Mnica Pini Univer sidad Nacional de San Martin, Argentina Romualdo Portella do Olivei ra Universidade de So Paulo Diana Rhoten Soci al Science Research Council, New York, New York Jos Gimeno Sacristn Universidad de Va lencia, Espaa Daniel Schugurensky O n tario Institute for Studies in Education, Canada Susan Street Centro de Invest igaciones y Estudios Superiores en Antropologia Social Occidente, Guadalajara, Mxico Nelly P. Stromquist University of Sou t hern California, Los Angeles, California Daniel Suarez Laborato rio de Politicas Publicas-Universidad de Buenos Aires, Argentina Antonio Teodoro Universida de Lus fona Lisboa, Carlos A. Torres UCLA Jurjo Torres Santom Universida d de la Corua, Espaa