xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam a22 u 4500
controlfield tag 008 c19999999azu 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E11-00123
Educational policy analysis archives.
n Vol. 7, no. 11 (March 31, 1999).
Tempe, Ariz. :
b Arizona State University ;
Tampa, Fla. :
University of South Florida.
c March 31, 1999
First year implementation of the School to Work Opportunities Act policy : an effort at backward mapping / Arthur M. Recesso.
Arizona State University.
University of South Florida.
t Education Policy Analysis Archives (EPAA)
xml version 1.0 encoding UTF-8 standalone no
mods:mods xmlns:mods http:www.loc.govmodsv3 xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govmodsv3mods-3-1.xsd
mods:relatedItem type host
mods:identifier issn 1068-2341mods:part
mods:detail volume mods:number 7issue 11series Year mods:caption 19991999Month March3Day 3131mods:originInfo mods:dateIssued iso8601 1999-03-31
1 of 43 Education Policy Analysis Archives Volume 7 Number 11March 31, 1999ISSN 1068-2341 A peer-reviewed scholarly electronic journal Editor: Gene V Glass, College of Education Arizona State University Copyright 1999, the EDUCATION POLICY ANALYSIS ARCHIVES. Permission is hereby granted to copy any article if EPAA is credited and copies are not sold. Articles appearing in EPAA are abstracted in the Current Index to Journals in Education by the ERIC Clearinghouse on Assessment and Evaluation and are permanently archived in Resources in Education First Year Implementation of the School to Work Opportunities Act Policy: An Effort at Backward Mapping Arthur M. Recesso Valdosta State UniversityAbstractThis study examined the intent of federal policy an d the actual implementation within local school districts. Speci fically, the focus is on the Federal School to Work Opportunities Act of 199 4 and its implementation in 47 school districts in upstate Ne w York as part of a consortium during the 1995-96 school year. The purp ose of the study was to determine 1) the extent to which an agreemen t to participate in a consortium arrangement designed to facilitate the i mplementation of a specific Federal or state policy resulted in the ac tive implementation efforts by individual consortium members, and 2) ho w a high school setting where School to Work activities were percei ved by local stakeholders as having great specific and important effects differed from a high school setting where School to Work activiti es were perceived by local stakeholders as having some or no effect. A b ottom-up backward mapping policy analysis model was used for the purp oses of this study. Local level data was used to create performance, en vironment, technology implementation, and School to Work imple mentation profiles of local high schools. Regression and corr elation analyses were
2 of 43used to determine the relationship between stakehol der perceptions and local high school characteristics. Results of the s tudy were reported and interpreted with the aim of furthering research and knowledge of policy analysis and the use of local level data to determi ne the success of policy implementation. This study found that variation bet ween federal intent and local adaptation is explained by characteristic s of the high school and perceptions of stakeholders. School to Work pol icy implementation, perceived by the high school administrator as a sta keholder, varied significantly by high school student performance, e nvironment in which the high school operates, and level of technology i mplementation in the high school. Results indicated that the backward ma pping policy analysis model is effective in determining the actual levels of policy implementation. Backward mapping results in a defin itive explanation of the role of the local actor and the use of discreti on in decision making. The final analysis as a result of backward mapping goes beyond the measurement of policy objectives being met and expl ains the meaning of local level participation.Introduction This study compared the original intent o f a specific educational reform policy to its local adaptation. Specifically, it focused on t he implementation of the Federal School to Work Opportunities Act of 1994 in the forty-four public high schools participating in the New York State Mohawk Valley Workforce Preparat ion Consortium in the 1995-96 school year. Policy analysis assumes implementation ca n be assessed by describing the reported activities of the "local integration" and comparing such findings to the original intent of the policy (Elmore 1982, Ingram & Schneid er 1990). The difference between original intent and reported local integration or a daptation were interpreted as "discrepancies" in fulfilling the purpose of the pa rticular policy. This study argued a more inclusive approach. This study was designed to examine these discrepancies in local level implementation. To this end data were c ollected on both the perceptions of the local stakeholders implementing the policy and the characteristics of the schools in which the implementation took place. To meet the st ated objectives and to delineate the patterns of local characteristics and stakeholder p erceptions of School to Work implementation, a rank ordering of the forty-four s chool settings that participated in the Mohawk Valley Workforce Preparation Consortium duri ng the 1995-96 school year was undertaken. The rank ordering of perceptions of Sch ool to Work implementation ranges from sites where perceptions of having "little or n o effect" on the on-going activities in the high school to perceptions of "great influence" The rank ordering on the basis of local characteristics (performance, environment, an d technology implementation), showed real deficiencies among sites. The result of the rank ordering of districts on these characteristics served as the basis for the analysi s of the relationship between local characteristics and perceptions of implementation. A second phase of the study described the specific organization, curriculum, instruction, and the community actions related to l ocal implementation in four of the high schools under study. Such description focused on (a) the use of local discretion, (b) the intent of the original policy, and (c) the sign ificance of organizational characteristics. These four sites are described in some detail. The schools were selected on the basis of
3 of 43their ability to elaborate cases of little or no ef fect, great effect, and a mixed or midrange effect of School to Work as discovered in the first phase.Background A Nation at Risk (1983) broadcast a message that schools were not p reparing students adequately for the workplace. It sharpened the on-going policy debate about the most effective means for preparing students for the workplace and/or college. The United States was portrayed as uncompetitive in the global marketplace. Students were trained for 1940s and 50s workplaces, not for those of the new millenium. Other reports noted employer complaints that too many recent grad uates lacked even the most basic reading, writing, and communication skills (U.S. Co ngress 1997, America's Choice: High Skills/Low Wages 1983, Nation at Risk 1983, SC ANS Report 1983). These reports asked for a policy to address the needs of employer s and of students as future employees.The Role of the Federal Government The Congressional intent of the School to Work Opportunities Act of 1994 was expressed in fourteen parts to provide an overarchi ng guide for States plans for implementing a sustainable School to Work system. I t required the involvement of all key stakeholders, especially employers, in planning the system, implementing it, and supporting its continuation. The School to Work Opp ortunities Act, as written, was intended to be administered in a way that provided for State and local discretion in implementing the system, as long as the efforts add ressed local capacity building, minimized program overlap, used scarce resources ef fectively, established clear goals, and provided the flexibility to meet these goals (S chool to Work Opportunities Act of 1994, 108 Stat 568, p.6-7). In the School to Work Opportunities Act, each State was required to plan three general areas. First, Federal government selection of states to receive funding under the Act was based, in part, on how well this system imp lementation plan built upon past support for School to Work-type initiatives. (STWOA 1994) Each state's proposal was required to explain how components would be linked together within a five year time-frame to establish a statewide School to Work system. Each state's system implementation plan also was required to show timel ines outlining the implementation of all components of the system and the relationshi p of each to other reform efforts. Second the state must explain how all policy compon ents, including those not supported in the past, were to be supported throughout the st ate. Thirdly, the state agencies named in the Act by the Federal policy makers to be criti cal partners in School to Work (i.e., Department of Labor and State Education Agency) wer e required to be named as having an active role within the State's implementation pl an. States were to be funded that demonstrated past practice, a commitment to School to Work-type activities, and planned for the future successful statewide impleme ntation of a School to Work system that met Federal guidelines. In the first year of approving statewide implementation plans, eight states received funds to implement a statewide School to Work syste m. State governments decided how funds would be distributed, according to flexible F ederal guidelines. The state was responsible for evaluating local progress towards f ull implementation of the system, ensuring that all students were being served and al l stakeholders were involved in the
4 of 43process. New York State New York was one of the initial eight Sta tes to receive implementation funding. It also was the recipient of the largest grant, over e ight million dollars in year one, and more than $60 million over five years. The funds we re considered "seed money' and "venture capital" for the development of a School t o Work System. (STWOA 1994) New York State submitted a proposal for the impleme ntation of a School to Work System over a ten year period, with five years of f unding to be provided in part by the Federal Government (New York State Implementation G rant p.1). Commitment to school reform was established through documentation of workforce development prior to the School to Work Opportunities Act of 1994. Th ree-and-three-quarter ($3.75) million dollars had been dedicated to Workforce Pre paration sites (involving 129 school districts) in the previous year (1993-94). The Gove rnor's School and Business Alliance was created in 1987; twelve regional sites were pro vided funding by the Governor's Executive Budget. Both of these initiatives had bee n working towards building the system called for in the STWOA. New York State's approved proposal provid ed the federally mandated framework. Skill standards, skill assessments, and skill certi ficates were to be integrated into performance-based assessment. Teachers, counselors, and administrators were to be prepared for the implementation of new strategies f or curriculum, instruction, and assessment. And parents, employers and other commun ity members were required to be involved in the decision making process for determi ning appropriate preparation for the workplace or college (New York State Implementation Grant p.6-30). Efforts supported by allocated funds were required to provide opportu nities for all students. Local Partnership In 1995 the State of New York, through th e NYSED, selected fifty-two partnerships throughout the State for funding as pl anning or implementing sites for School to Work initiatives. These partnerships cons isted of school districts, employers, higher education institutions, community members an d organizations, and localized State agencies (e.g., County Departments of Social Services). Each partnership had to identify how many and what types of activities were taking place already in the schools, workplaces, and school/workplace projects. They wer e asked to explain how each activity was to be replicated in nonproject sites in the local area and how new activities were going to be developed and supported. Programs were to be designed and supporte d for all students, K-12+, including at-risk, gifted, and out of school youth. The progr am and activity design was to be based on the needs of the local population of students. T he needs of the local target population, in turn, was to be determined by student performanc e on statewide testing, dropout rates, attendance rates, discipline problems, college boun d rates, and also local unemployment rates. One result was staff development programs cr eated to address ways to improve student performance. For example, the Teacher Job S hadowing Program connected staff with local employers and employees. Teachers visite d business sites, spoke with employers and employees, and toured facilities. Sub sequently, each teacher designed a
5 of 43classroom lesson incorporating work-based learning. Some participants were teachers of Regents classes. Their students were to apply knowl edge gained through the Regents syllabus to an actual workplace problem. The premis e was that students would more thoroughly understand of how the classroom knowledg e was applicable to actual problems, resulting in better performance on Regent s exams. This was an important point because Regents exams, and the rigors of the courses, were often cited as barriers to change. Only one region, the Mohawk Valley Workfo rce Preparation Consortium, submitted a proposal on behalf of the entire region that had been originally designated for funding distribution by the statewide advisory council. Most of the other partnerships in the State were connected and funded through a lo cal BOCES. Other initiatives were funded as either one of the big five school distric ts (Buffalo, New York City, Syracuse, Rochester, and Yonkers), as a group of school distr icts participating as a non-BOCES consortium, or as specialized magnet schools like t hose found in New York City.Conceptual Framework Many policy studies focus on the role of information and analysis in decision making, others on the role of political bargaining and power in policy decisions (Elmore, 1979, Lindblom & Woodhouse 1993, Wildavsky & Pressm an 1971). The actions of local actors and their use of discretion in decisio n making is often a critical component of these studies (McLaughlin 1987). At any given ti me, individuals at the local level deal with a myriad of issues and policies related to sch ool reform. Mandates and limited resources force local actors to decide which polici es will be implemented. The role of the local actor in implementation may be overlooked in forward mapping approaches to policy analysis. Local actor perceptions and their relationship to decision making and implementation deserve attention that typically doe s not take place in the traditional top-down approach. If it is agreed that the policy making process yields unintended results (including the failure to implement), then it is desirable to utilize other methods that may aid in determining predictors of success. Important in this approach is the ability to discover forces influencing policy accep tance or integration (Lindblom & Woodhouse, 1993, p.1-32).Literature Review and Conceptual Framework Policy Analysis: Forward Mapping v. Backward Mappin g Policy development, implementation, and a nalysis are extremely complex processes. "As a 'science' policy analysis is conce rned with predicting cause and effect relationships implicit in policies" (Elmore 1987 p. 174). A traditional approach to policy analysis is the top-down process of forward mapping Through the traditional lens, the view is of a relationship between policy creator an d policy implementation and outcomes. Richard Elmore suggests an approach to po licy analysis that goes beyond the traditional top-down analyses used in many studies. Elmore's interest is not limited to determining success or failure based on the measure ment of objectives, but seeks to understand why and under what conditions policies a re adopted locally. Elmore
6 of 43introduces another vantage-point from which policy is viewed from the locality and by grassroots stakeholders. Both views tell parts of a complex story. Thus a case can be made for studying policy with both the traditional forward lens (from design to local implementation) and a backward lens (from implement ation to original intent). Forward Mapping The most widely recognized type of analys is is the "top down" approach. This begins with the objectives and goals of the policy and works its way towards the outcomes to be measured in determining success. Thi s process often begins with legislation that outlines the congressional intent of the policy and determines success or failure by measuring reported outcomes in terms of the original objectives. Supporters of the top down approach assum e the clear delineation of goals from policy makers will lead to more effective support, more effective implementation, and greater success of policies in addressing problems. But many authors have reservations about the policy maker's power to affect local impl ementation process (Ingram & Schneider 1990, McLaughlin 1987). The top down appr oach assumes the policy makers have control, that they can affect the implementati on process, and that their decisions have some bearing on local actors. Authors such as McLaughlin, Ingram, Schneider, and Sabatier fault the top-down approach for not consid ering the discretion of the local actor as an important component of policy analysis. Ingra m and Schneider (1990) cite the work of Elmore for its emphasis on the "organizatio nal structure and personal factors" (e.g., perceptions and discretion) in determining t he actual policy impacts and in explaining their success or failure. McLaughlin and others indicate that a need to understand that many factors are involved in the su ccess or failure of a policy, and many of them involve the actions of the "street level bu reaucrat" (McLaughlin 1987, Wildavsky & Pressman 1971). Pressman and Wildavsky's study of the Oak land Project in the 1970's is heralded as a pioneering study of implementation. Their asse ssment of the Federal implementation of employment programs for severely disadvantaged adults raised questions about why the programs failed. The Federa l Government had established very clear goals for the public works projects, the Oakl and government officials supported the initiatives, private companies wanted the projects, and the people of Oakland wanted the projects to be a success. Yet the policy failed for reasons that were beyond the control of the policymaker (Wildavsky p.87-124). In their fina l analysis, Pressman and Wildavsky determined that the relationship between the number of transactions required to implement the policy and the perception that the im plementation would be successful were inversely related (Wildavsky 1971). This event ually lead to the understanding that policy decisions are not selfexecuting (McLaughli n 1987). The policy is created and exists for the local actor to follow as a guide for implementation. The local actor has perceptions of the policy. Those perceptions have a relationship to decision making. Decision-making results in implementing or nonimp lementing. Therefore, the notion that clearer goals or objectives should have any im pact is thwarted and the stance in support of hierarchical procedures is challenged. P ressman and Wildavsky used a forward mapping approach and did not clearly determ ine the cause of the policy's failure. Their approach does not consider the local actor, the characteristics of the organizations, or the local environment beyond the obvious unemployment rate. Their final analysis does mention the need for a process that takes into account each one of these items. Backward mapping does consider interna l and external environment of the local environment and local discretion. And backwar d mapping completes the analysis
7 of 43by looking back at what was originally intended. There is no single rule for creating and implementing new policies that will be automatically effective at the local level. Pressma n and Wildavsky's study demonstrated that even the best of intentions and the clearest o f goals can prove to be ineffective in achieving desired or expected results (Ingram and S chneider,1990, p.69). Still, many analysts believe it is the policy maker and the cla rity of the goals and objectives of policy that make the difference at the point of imp lementation. Ingram and Schneider (1990, p.69) argue that if policy makers framed bet ter statutes, implementation would improve. Expanding on the work of others such as Sa batier, Pressman, and Wildavsky, this theory follows the belief that implementation itself is hierarchical in nature. The focus is on a sequence of events rather than a dete rmination of actual influence or impact (Elmore, Studying Implementation, p.24). Different processes are required if exist ing policy analysis approaches are not successful in determining the actual effects of pol icies (Lindblom & Woodhouse 1993). Often these different, even non-conventional, proce sses must go beyond the lens of the policymaker. Even though a state or federal governm ent official may play a pivotal role, local influences have an effect on the implementati on of the policy. Knowledge of this aspect of implementation may contribute greatly to improving future policy development. Providing data that policy implementat ion is not necessarily made up of leaders (policy makers) and followers (policy imple mentors) would be an important step towards establishing policy implementation as an in teractive process (Lindblom & Woodhouse p. 4-6). Just as important as following t he conventional government process is the investigation into the influences that disto rt these processes (Lindblom & Woodhouse p.11-12).Backward Mapping Backward mapping is a form of policy anal ysis that acknowledges that there may be variables that exist outside the treatment progr am, and that some of these variables may have something to do with the organization's ch aracteristics (Elmore 1979, p.601-616). Considering the organizational characte ristics of an organization (the school district in this study) could help to explain why p olicies fail or succeed. While policy makers define the arena in which the implementation process will take place, and determine the actors and the roles each may play, d etermine the level of resources to be allocated, the impact of environmental influences o n implementation cannot always be foreseen (Pressman and Wildavsky p.174, 126). These include organizational, political, and technical conditions (Elmore, Backward Mapping, p. 603). Backward mapping challenges some of the m ost basic assumptions of the top down or traditional approach. It does not assume th at all organizations are the same. Each has varying environments (e.g., local unemploy ment, student performance) that affect its decision making related to addressing lo cal problems. The approach does not assume that all organizations are even interested i n implementation. The backward lens places value on the role of the local actor and on the local organization that is focusing on resolution of the problem. The process determine s if the school district does not identify itself as having the problem, or has the p roblem but is not concerned about it. In either event, backward mapping sets the stage for f urther analysis to determine why either case occurred. Elmore provides the analysis of the Youth Employment Policy as an example of the shortfalls of using the top down approach (Elmo re, 1988, p.29-33). The analysis process begins with the goals and objectives of the policy and summarizes the level of
8 of 43success of the policy by measuring perceived attain ment of goals. Similar to the Oakland Project, the results of this analysis may raise con cerns that there may be variables outside the process that have a direct impact on th e success of the policy (Pressman & Wildavsky, p. 191). However, these other variables go undetected without the use of another approach. McLaughlin adds support for the n eed of added "bottom up" approach and calls this the micro-level of policy analysis ( McLaughlin, Learning from Experience, p.177-178). That is, the focus is on the local acto r and the utilization of discretion, acknowledging that the policy is not the only deter minant affecting the actions of people at the focal point of the problem (Elmore, Studying Implementation, p.21). The logic of the analysis used by Wildavs ky and Pressman in Oakland and Elmore's assessment of Youth Employment policy indi cate the need for further study. It is clear that policy analysis involves not only the measurement of outcomes based on policy goals, but also an understanding of the envi ronment in which the problem resides and within which the implementation takes place. At the local level where decisions are made, the discretion of local actors come into play It is also where organizational characteristics may have some role in determining t he success or failure of the policy. The variation of organizational characteristics, lo cal market conditions, local preferences and prior commitments, and other perceived impacts on the organization will help to determine the relationship between the policy itsel f, the nature of implementation in the school district, and the actual success of failure of the reform effort.Design of the Study The purpose of this study was to compare the original intent of a specific educational reform policy to its local adaptation. The focus was on the implementation of the Federal School to Work Opportunities Act of 199 4 in the 44 public high schools participating in the Mohawk Valley Workforce Prepar ation Consortium during the 1995-96 school year.Research Question OneTo what extent did agreement to participate in a co nsortium arrangement designed to facilitate the implementation of state or federa l policy result in the active implementation efforts by individual consortium mem bers? Research question one, and its subparts, were developed based on the work of several authors such as Pressman and Wildavsky (197 1), who raised questions about the meaning of participation and the relationship to im plementation. The question also addressed Elmore's (1983,1988) backward mapping mod el that explains the relationship between the organization's characteristics and impl ementation. The political bargaining, organizational process, and rational decision makin g models also partly contributed to the development of research question one. The work of S imon (1993), Allison (1991), Gieseck (1995), and others delineate the need to un derstand why and under what conditions decisions had been made while implementi ng the policy. Research Question TwoHow did a high school setting where School to Work activities were perceived by local stakeholders as having great and specific imp ortant effects differ from a high
9 of 43school setting where School to Work activities were perceived by local stakeholders as having some or no effect? Similar to research question one, researc h question two and its subparts were based on the backward mapping model of Richard Elmore (19 88). However, unlike research question one, this question more directly addressed the relationship between local characteristics and implementation of the policy. R esearch question two was based on the work of Elmore (1988), McLaughlin (1987), Ingram an d Schneider (1990), and Lindblom and Woodhouse (1993), all of whom addressed or iden tified the need to research the differences between local organizations and the rel ationship of those differences to the level of implementation. In addition, it was within the work of Pressman and Wildavsky (1971) that the question of the relationship betwee n local level characteristics and implementation arose. The research of Rogers (1963, 1988, and 1995) and Tornatzky and Klein (1982) addressed the issue of innovations and their implementation at the local level. Their attention to the organization's involv ement in multiple innovations and the relationship between organizational characteristics and implementation of innovations was also a basis for research question two.Methodology Forty-seven school districts entered into a partnership to implement a Federal School to Work policy. The question of the match be tween the federal policy intent and the actual "grassroots" adaptation rested with the level of specific and important changes that can be attributed to four characteristics of t he school. School to Work implementation, student performance, environment, a nd technology implementation as four characteristics of a high school were addresse d using a backward mapping policy analysis model. The characteristics of the schools, to be discussed for the purposes of this study as "profiles," were developed for this study to deline ate information for district comparisons, predict changes, profile system strengths and weakn esses, and identify policy improvement strategies (Oakes, 1986). To provide in formation about the high schools, the profile had to have a reference point, a measure by which to judge the indices used to compare the high schools. The reference points by w hich to measure the indices were the substantive and technical criteria (Cooley, 1992). Each profile did provide at least one of the following recommended substantive criteria: inf ormation that described student performance in terms of achieving outcomes, informa tion that described the central features of the high school, or information that wa s policy relevant (Oakes, 1986). In addition, each profile was developed having the fol lowing technical characteristics: it measured features that can be found in all schools being studied, the data collected can be traced over time, the profiles can be understood by practitioners and policymakers, and the profile indices were generally accepted in educatio nal policy analysis (Cooley, 1992; George, 1993; Oakes, 1986; Windham, 1990). The purp ose of developing these profiles was to delineate the local conditions of the high s chools implementing federal policy, provide a common means for comparing those schools, and provide a final analysis that improved policy decision making. The meaning of participant and non-partic ipant as defined by implementing and non-implementing schools was explained using the Sc hool to Work implementation profile. The profile described to what extent feder al policies were adapted. The School to Work implementation profile also described to what extent the high school implemented
10 of 43schoolbased learning, work-based learning, and co nnecting activities components of the School to Work system. The components were required by the federal STWOA for schools using funds to implementing School to Work. The following data were collected as indi ces of the level of School to Work implementation achieved in each district: if the sc hool described itself as being involved in School to Work during the 1995-96 school year; t he priority of staff development overall and specifically connected to school based learning, work based learning, and connecting activities; the number of partnerships e stablished locally; and the level of commitment to required activities as described in t he legislation and regional proposal (e.g., job shadowing, school based learning, tours) as demonstrated by implementation by grade levels. These data were used to compare schoo ls within the consortium. Correlation methods were used to determine the relationship bet ween implementation level, the performance of the school's students, the environme nt in which the school operates, and the level of technology that has been implemented i n the school. The performance profile of the students i n the high school was used to explain the school's educational quality and its effectiveness in meeting desired educational outcomes (Cooley, 1992; George, 1993; Windham, 1990). The Ne w York State Regents exams and diploma are benchmarks by which students and school s were measured to determine academic success. A broad audience of educators and policymakers understand and generally accept the Regents system to be a consist ent measure of student achievement (Wiles, 1996). In developing the performance profil e, the following data were used: attendance rate, dropout rate, suspension rate, per centage of students graduating with a Regents Diploma; percentage of students enrolled pa ssing the Regents exams in English, Math III, Chemistry, US History and Government; and the percentage of students passing the Occupational Education Proficiency exam. The pe rformance profile also was used to determine a relationship between past performance a nd present participation rates. The performance profile was designed to measure the sch ool's ability to prepare students to meet expected academic outcomes. This performance m easure was compared to the school's ability to meet policy implementation outc omes as described by the School to Work implementation profile. In this analysis, the school's effectiveness in meeting expected outcomes and the level of School to Work p olicy implemented by that school were tied to the economic viability of the student. Educational outcomes (employment, achievement in subsequent education and training, a nd admission to further training upon graduation) were all indicators of school effective ness in meeting educational outcomes (Windham, 1990). These outcomes matched goals of th e School to Work policy implemented within these schools (STWOA, 1994). The School to Work policy was developed to better prepared students for the workp lace so they could enter the workforce more easily and prepare for better jobs (Jennings, 1995). Jennings found the better the student preparation programs and the better student s perform academically, the more likely they are to be employed after high school. T his established a link between School to Work and student performance and substantiated the use of performance profile. The performance profile also explained the similarities and/or differences between schools in terms of their effectiveness, and it was used to ex plain the relationship between student performance and the School to Work implementation p rofile. The environment profile was a composite o f indices of the external environment and external forces acting upon the high school. Th ese environmental indicators describe the demographic, social, and economic influences of the local communities in which the high schools are located (Bryson, 1988; Cook, 1990) Influences such as types of employers and the number of businesses located in t he community were important because School to Work policy makes specific mentio n of the relationship between
11 of 43successful School to Work implementation and these influences (Grumman, 1994; STWOA, 1994). For example, the school was required to be supporting the work-based learning component. Therefore, employers were requi red to be involved as partners in School to Work activities. A locality with numerous employers was potentially able to support numerous work-based learning activities. The availability of resources in the comm unity also described the context in which the schools operate and were represented in the Env ironment Profile (George, 1993). Levels of per capita income and number of people em ployed directly influenced school budgets and the funding available to support innova tive programs (George, 1993). While School to Work did provide implementation resources to the schools, schools did incur expenses during implementation. Therefore, schools with more resources had the potential to support more School to Work activities. The environment profile was composed of t he following items: percentage of free and reduced lunch (a local wealth indicator), avera ge household income, percentage of people with bachelors degrees, and percentage of yo uth at risk. These indicators of school and community were then linked to desired outcomes. Each of the profile indicators could be measured for each school district in the School to Work consortium and throughout the state of New York (Oakes, 1986). They capture endur ing features that are recognizable to both practitioners and policymakers and generally a ccepted as being valid and reliable items (State Education Department, 1993; Wiles, 199 6). School to Work took place within the broa der context of school reform. A high school that was involved in School to Work was more likely to also be involved in another systemic reform initiative (Grumman, 1994). The School to Work policy was being implemented at the same time as at least one other innovative reform effort in all of the Mohawk Valley Consortium schools. Computer-rela ted technology was also infused in the Mohawk Valley Consortium schools, and its st udy was completed for most of the same schools during the 1996-97 school year (Mann, 1997). The Mohawk Valley region technology implementation study considered the poli cy implications for a large-scale commitment to and investment in computer technology Mann collected detailed information from superintendents, high school admin istrators, teachers, and students. Data was compiled and manipulated in an effort to determ ine the impact of technology on student performance (Mann, 1997, p.2). Technology was used in this study as a me ans of profiling and comparing schools for several reasons. The data set collected by Mann covered a majority of the same school districts as this study. Similarly, Mann's survey r esults also relate to level of participation, level of staff commitment, participation in staff d evelopment, and the intent of technology integration. And Mann's study sought to explain, in part, how stakeholders perceived the effect, importance, and level of technology impleme ntation. Like this study on School to Work, the Mann study took the approach of addressin g implementation at the grassroots level and compared it to the intent of policies in place. Therefore, the fourth and final characteristic was the technology implementation pr ofile. Technology implementation in this study was defined as the level of computer-rel ated technology that had been implemented for instructional use (Mann, 1997). As innovations, School to Work and technology have some distinctive similar qualities that are called the critical mass aspect (Rogers, 1988). Neither innovation reached its pote ntial for utility unless other people or organizations implemented the idea. Technology can operate stand-alone, but its' uses increased as it was connected to other people, the Internet, and networks of software and information. Similarly, School to Work could have b een implemented with just one employer or in just one school. But it reached more students and a wider variety of students' needs with multiple participants and supp orters. A collaborative effort was
12 of 43required to establish a system that serves all stud ents. School to Work and technology are similar in the critical mass aspect in many other ways. In order for other organizations and people, other than those directly involved in initial implementation, need to be networked into t he continuation of implementation and programs being supported in order to replicate new ideas and see that the innovation works (Rogers, 1988). In School to Work, the more e mployers and teachers that were involved, the more students that were being served by its initiatives. Similarly, the more technology that was made available to teachers and students, the more access students have leading to a higher level of integration into the learning episode (Mann, 1997). Therefore, the technology profile was an indicator of the extent to which the school had implemented another innovation. Determining the rel ationship between the technology implementation and School to Work implementation pr ofiles explained the variance attributed to the level of technology implementatio n. The technology profile was also combined with the environment and performance profi le to explain the level of variance in School to Work implementation. The specific items used to develop the te chnology implementation profile were as follows: perception of whether the technology impor tance was linked to improved curriculum or to changes in teaching; and the numbe r of computers installed, networked, accessible to teachers, accessible to students, and the number of Internet accounts in the classroom. These were exactly the same items select ed, in part, from the Mann study (1997) and replicated in eleven other high schools that had not participated in that study. (Note 1) All four profiles of the high school were completed within the framework of a backward mapping policy analysis that took into con sideration the conditions of the high school as it implemented the policy. The analysis o f the School to Work policy implementation compared the Act's original intent w ith local implementation that was supported by the organization. The variance between intent and local adaptation was to be explained by the characteristics of the local high school. The "mapping" process began at the local level and traced the actions back to the beginning. A backward mapping lens, defined as stak eholder perceptions of specific and important effects and local organizations' characte ristics within a community environment, was offered as an alternative to tradi tional top-down methods (Elmore, 1979; McLaughlin, 1987). Stakeholders were consider ed as any group or individual who was affected by or who could affect the future of t he organization (Bryson, 1990). For the purposes of this study, the focus was on local stak eholders with a vested interest in activities that impacted the school district and th e student population. These included superintendents, school district administrators, te achers, employers, students and community members. School district administrators p rovided leadership and decision making for the schools. It was their duty to see th at programs were preparing all students leaving school for college or the workplace. The te achers prepared students along a continuum of learning that culminated in graduation from high school and entrance into higher education of the workforce. Whichever path w as taken, the students must have had attained the skills and knowledge necessary to perf orm in their chosen environment. Community members that provided representation on l ocal partnership committees (e.g., private employers) had an interest in students beco ming contributing members of the community. The student's contribution could be as a future employee with a good work ethic or as a good citizen that values a role in co mmunity efforts. While the forward map would stop at the program administrator's response to the level of integration, the backward map looked for evidence of change in the o rganization, such as methods used in the classroom, and the interpretation of integratio n from the viewpoint of participants and
13 of 43stakeholders. Measuring the level of specific and impor tant changes is an element of the backward mapping model. Grassroots level investigat ive procedures were used to determine perceptions of School to Work policy impl ementation resulting in changes in the high schools of Mohawk Valley Workforce Prepara tion Consortium during the 1995-96 school year. Interpretation of stakeholder perceptions were interpreted to determine if the federal policy intent was similar or significantly different from local implementation. These perceptions were examined in the context of the on-going high school operations, organizational characteristics o f the school district, and the local community. The development of the performance, envir onment, technology implementation, and School to Work implementation profiles provided a means for describing groupings of data collected in support of the backward mappin g model. It enabled the backward map to factor in the impact of external forces such as the local community, other government agencies, and private employers. Local adaptation o f program intent, and the conditions within which it took place, was discussed in terms of how the School to Work System was implemented within the local school district. Repre sentative stakeholders of the groups involved in the implementation process described to what extent they perceive School to Work to have been integrated into the schools. The level of implementation sought by the federal government was delineated to guide the states and schools in their efforts to im plement a School to Work. Full implementation was defined in the School to Work Op portunities Act legislation as having these components: (STWOA, 1994) All students are participating in School to Work ac tivities, All levels of education are supporting an integrate d School to Work System, All staff have seamlessly integrated School to Work into their classroom, All stakeholder groups have been involved in the pr ocess, and All initiatives and partnerships are tied together in one cohesive system across the State and country. Schools were differentiated based on the varying degrees to which they were able to achieve full implementation. These levels of implem entation were to be determined by: the number of School to Work initiatives that were implemented grades nine through twelve; the number of staff development related act ivities that were in place; the number and types of partnerships that have been establishe d with local employers; and the number of school-based, workbased, and connecting activi ties that were supported. The Performance Profile as an independent variable cons truct was developed to represent the educational quality of the school and its effective ness in meeting state educational benchmarks. Schools accumulated points in the form of data items on student performance. The total number of points for each hi gh school established a ranking by which to compare the schools. The comparison discus sed the success of the school in meeting NYSED-established benchmarks as a member of the consortium and an implementor of School to Work. Higher profile ranki ngs indicated schools having greater success meeting the statewide student performance s tandards. The success of the school in meeting these standards and, therefore, attaining a higher ranked profile spoke to the educational effectiveness of the school. The Perfor mance Profile was utilized to determine this independent variable's ability to explain the variance in the dependent variable, School to Work implementation. The variance was mea sured directly and in combination
14 of 43with the two other independent variables. The Environment Profile was a second inde pendent variable. This profile was a representation of the external factors effecting th e school. Highly ranked schools were operating in a healthier (e.g., more opportunities for kids with more employers) environment than other schools participating in the consortium. The availability of employers and employees, community financial resour ces, and availability of people to serve as appropriate role models could impact the s chool's ability to support School to Work activities. Technology Implementation, a third indepe ndent variable, was developed based on the responses from high school principals on the le vel of computer-related technology and training present in each building. The highest rank ed district had the most involvement in implementing technology, demonstrating support and dedication of resources to another reform effort.Sample Selection The Mohawk Valley Workforce Preparation C onsortium consisted of a population of 47 school districts. Fortyfour school district s had full secondary programs, grades 9 through 12 inclusive. This was a study of the popul ation of school districts, as partners in the Mohawk Valley Workforce Preparation Consortium, which support high school-level implementation. Therefore, three districts without a grade 9 through 12 program configuration were not surveyed. One district provi ded schooling up to the ninth grade and transported students to a nearby district for t enth through twelfth grade. The two other districts did not provide a high school program at all. They provided K-8 and K-6 programming and transported students to other distr icts for further schooling. One high school administrator from 44 hig h schools in the Mohawk Valley Workforce Preparation Consortium was surveyed. High school administrators who were not employed by the school district for a full scho ol year during the first year of implementation (1995-96) were not sent surveys. Out of the 44 possible districts, 33 superintendents and 39 high school administrators w ere surveyed. Response Rate High school administrators returned 24 of the 39 surveys, a 62% rate of return. Summary of Data Collection To complete this study using backward map ping, data that describe the characteristics of the local organization, the perc eptions of local stakeholders, and the adaptation of intent of the policy at the local lev el was collected. Three types of data collection sources were used. Original surveys were developed and sent to stakeholder groups. Follow-up interviews with stakeholders prov ided more indepth information and explanation for the survey data. Existing data sets were readily available to build the profiles that characterize the high schools' studen t performance, environment, technology implementation, and School to Work implementation. All three data sources proved to be useful and effective in supporting the analyses to be used in this study using the backward mapping process.Data Analysis
15 of 43 The data collected were analyzed in stage s. In the first stage, the data were compiled into profiles, labeled as independent and dependent variables, and analyzed using regression and correlation analysis. In a second st age, the aggregate data from specific questions posed to a surveyed stakeholder group and specific items from the existing data sets were used for analysis. For example, specific questions in the survey were asked to determine if high school administrators responsible for policy implementation described School to Work as being specific and important. Cor relation analysis was used to determine the extent to which the administrators' p erceptions of importance were associated with the level of implementation. In a s econd step, regression analysis was applied to determine the extent to which the level of implementation in high schools varied by the perceptions of the high school admini strator with significance. The third stage of analysis involved selecting high schools f or follow-up interviews by placing the high schools on a quadrant grid. Based on the schoo l's placement on the grid the school was selected for a followup interview. Interviews provided more detailed explanations about perceptions, participation, and implementatio n. Compiling the Profiles The profiles (Performance Profile, Enviro nment Profile, Technology Implementation Profile, and the School to Work Impl ementation Profile) were developed to serve as a means for explaining how organization al characteristics in terms of student performance, external environment, and technology i mplementation predicted or were associated with the level of School to Work impleme ntation. The profiles developed in this study were essentially constructs. They served the purpose of grouping many different but related variables into one term for ease of ana lysis and subsequent discussion. The profiles were used to explain to what extent studen t performance, the environment in which the school operated, and whether the level of technology implemented had an effect on the adaptation of intent at the local level. The Performance Profile was developed for each high school participating in the Mohawk Valley Workforce Preparation Consortium to d etermine if school organizational effectiveness characteristics varied systematically with stakeholder perceptions of specific and important effects of School to Work policy. The Performance Profile was a construct consisting of the following variables: percentage o f graduates receiving Regents Diplomas, dropout rate, attendance rate, suspension rate, percentage of enrolled passing the Math III Regents Exam, percentage of enrolled p assing the English Regents Exam, percentage of enrolled passing the Chemistry Regent s Exam, percentage of enrolled passing the U.S. History and Government Regents Exa m, and the percentage of number tested that passed the Occupational Education Profi ciencyIntroduction to Occupations State Exam. All of the performance data was collected from the New York State 1995-96 Report Card on the Schools released in February of 1997 (R eport Card on the School, 1997). Developing the Performance Profile, the reported fi gures, as raw data, for each item (e.g., percentage of graduate receiving Regents Diplomas) were summed into a total of performance profile points for each high school (Se e Table 1). The dropout rate and suspension rates were added to the schools total by subtracting the reported rate (e.g., 5% drop-out rate) from 100% and resulting in a positiv e figure (e.g., 95% did not drop-out) to be added into the schools total (See Table 1). This process thus penalized schools for higher dropout or suspension rates in either catego ry. None of the variables were weighted because there was no evidence of one item having mo re importance than another (e.g., Math III Regents Exam compared to the Chemistry Reg ents Exam). The state reporting
16 of 43does not differentiate the items as being more or l ess important. The total for each high school was rank ordered. The high school with the h ighest number of total points was given the top ranking. This resulted in a rank orde r listing of the highestperforming to lowest-performing high schools in the Mohawk Valley Consortium. Table 1. Compiling the Performance Profile ITEM CALCULATIO N RESULT % Grads receiving Regents Diplomas Actual % (0-100%) Dropout Rate 100% rat e (0-100%) Attendance Rate 100% rat e (0-100%) Suspension Rate 100% rat e (0-100%) % of Enrolled Passing: Math III Regents Exam Actual % (0-100%) English Regents Exam Actual % (0-100%) Chemistry Regents Exam Actual % (0-100%) US History and Govern Reg. Exam Actual % (0-100%) % of Number Tested & Passed Occ Ed Intro to Occ State Exam Actual % (0-100%) TOTAL Sum of fig ures 0 to 900 To determine if community characteristics varied systematically with stakeholder perceptions of specific and important effects of Sc hool to Work policy, the Environment Profile for each high school participating in the M ohawk Valley Consortium was developed. The Environment Profile was a construct of the following variables: percentage eligible to receive free and reduced lun ch, percentage of households on public assistance, percentage of residents with bachelors degrees, and the percentage of youth at risk. All of the environmental data was collect ed from the United States Census Bureau: 1990 Census Data. The sole exception was the free a nd reduced lunch eligibility data, which was obtained from the 1995-96 Report Card on the Schools. The data were summed for a total of environment profile points fo r each high school (See Table 2). The totals for each school were then ranked. A lower to tal number of points indicated a healthier and more resourceful environment in which to operate. Therefore, schools with lower totals were ranked higher: the highest rankin g was number one, the lowest, forty-four. The average household income and the pe rcentage of residents with bachelor's degrees had to be resorted to fit the pattern of th e other variables. The percentage of residents with bachelors degrees was subtracted fro m the highest possible percentage (100%). The result was summed into the total enviro nment profile points (See Table 2). The data items were not weighted because none of th e data items carried more importance in characterizing the environment of the high schoo l. Data was available for all of the participating high schools. Table 2 Compilation of the Environment Profil e ITEM CALCULATI ON RESULT % Eligible for Free and Reduced Lunch Actual % (0-100%) % of Households on Public Assistance Actual % (0-100%) % of Residents with Bachelors Degrees 100 %w/ BS (0-100%) % of Youth at Risk Actual % (0-100%) TOTAL Sum of figur es 0 to 400
17 of 43 To determine if technology implementation varies systematically with stakeholder perceptions of specific and important effects of Sc hool to Work policy, a Technology Profile was developed for each high school particip ating in the Mohawk Valley Consortium. The Technology Profile consisted of the following variables: number of computers installed in the high school, number of c omputers with a 486 megahertz or faster processor, number of computers networked, nu mber of teachers with access to computers, level of demand from teachers, level of demand from administration, level of demand from community, importance of improving curr iculum, and the importance of improving teaching (See Table 3). Each item was sum med into a technology profile points total. Similar to the performance and environment p rofile, no weighting was used with the technology profile. No item was determined, based o n the literature reviewed, to be in need of weighting. All items were treated with equa l importance. The level of technology implementation wa s used as a predictor of involvement in and commitment to reform efforts. The assessment of each school's integration of computer-related technology was intended to explain the level of technology present in the district, the knowledge and attitude of school district staff towards innovations, and more significantly, how technology was being utiliz ed in the learning environment. The variance between the school's technology profile an d the level of implementation of School to Work was discussed to explain any pattern s or distinct occurrences. Technology implementation levels were bas ed on participation in the activities that were supported by the Mohawk Regional Information C enter (MORIC). The MORIC is the vehicle through which each district purchased c omputerrelated technology, integration strategies, planning services, network support services, and leveraged State Aid for purchases of equipment and services. High i mplementors had participated in a variety of activities, such as teacher training, cu rriculum integration of technology, and administrative use of technology. Low implementing schools had few activities or none at all. Table 3 Compilation of the Technology Implementation Pr ofile ITEM CA LCULATION Number of computers in the high school Act ual Number Number of computers with 486+ processor Act ual Number Number of computers networked Act ual Number Number of teachers with access to comp. Act ual Number Level of demand from teachers Sur vey Number Level of demand from administration Sur vey Number Level of demand from community Sur vey Number Importance of improving curriculum Sur vey Number Importance of improving teaching Sur vey Number TOTAL Sum of figures To determine how School to Work implement ation varied systematically with stakeholder perceptions and/or the performance, env ironment, and technology implementation profile, the School to Work Implemen tation Profile for each participating high school was developed. A School to Work impleme ntation profile was created based on the survey results of the high school administra tors. The High School Administrators' School to Work implementation profile was a construct of the high school administrators' percep tions of the extent to which School to Work implementation took place (See Table 4). It wa s developed from the survey of high school administrators and used the following variab les: provision of multiple learning opportunities (applied academics, career academies, work sites as classroom, work-site simulation, community service projects, internships /apprenticeships, and co-ops/youth-run
18 of 43enterprises), number of grade levels School to Work activities took place (employer visits, job shadowing, simulated work environment, school a ssignments in the workplace, youth-run enterprise, community service projects, i nternships, work-linked learning, work-study co-ops), and the level School to Work st aff development was supported (active/hands-on learning, multiple learning enviro nments, partnership strategies, development of world-class standards, integrating c areer development). The high school administrators' total points for School to Work Imp lementation was summative of all the variables for each high school. The legislation and literature available do not differentiate on the importance of each item. Therefore, no weigh ting was used in the calculation. Table 4 High School Administrators' School to Work Im plementation ITEM RE SPONSE RANGE Provided multiple learning opportunitiesApplied academics 0 to 4 Career academies 0 to 4 Work sites as classrooms 0 to 4 Community service projects 0 to 4 Internships/apprenticeships 0 to 4 Co-ops/youth run enterprise 0 to 4 Number of grade levels STW activities offered Employer visits 0 to 4 Job shadowing 0 to 4 Simulation of work environment 0 to 4 School assignments in the workplace 0 to 4 Youth-run enterprise 0 to 4 Community service projects 0 to 4 Internships 0 to 4 Work-linked learning 0 to 4 Work-study co-ops 0 to 4 School to Work Staff Development Supported Active/hands-on learning 0 to 10 Multiple learning environment 0 to 10 Partnerships strategies 0 to 10 Developing world-class standards 0 to 10 Integration of career development 0 to 10 TOTAL (sum of response range figures) 0 to 114Methods Summary This was not a traditional forward mappin g (top-down) analysis. If this study had used the forward mapping approach, the data collect ion methods would have used the objectives stated in the original policy to determi ne what data should be collected. This study used the backward mapping (bottom-up) approac h. Backward mapping is concerned with the situation in which the policy wa s being implemented. Therefore, methods different from the top-down approach were u sed. The data collection methods gathered information about the actual activities at the local level. A final analysis was completed of the perceptions of local stakeholders, the competition for resources, and the environment in which the school was operating t o determine their effect on implementation of the policy. The methods of analysis used in a forward mapping process could have measured the locality's level of implementation by the objec tives of the original policy. But that process would not describe the contributing factors or barriers to implementation. Nor would it describe actual implementation and what it meant to a high school. To follow-through on the backward mapping approach, a different method of analysis was used. The perceptions of the local actor as a stake holder and/or implementor were
19 of 43compared to the original intent of the policy. The extent of variance that can be explained by these perceptions was measured. This a nalysis of perceptions was in combination with the measurement of variance of Sch ool to Work implementation levels attributed to the environment in which the school o perates, the performance of its students, and the level to which it has been active in implementing technology.Results of Data Analysis Within the context of this section of the article descriptive, regression, and correlation statistics are used to analyze the data collected from the surveys and data sources described. The data include regional and lo cal level data collected from the New York State Education Department (NYSED) and the New York State Department of Labor (NYSDOL); the stakeholder survey responses; a nd the data collected to build the Environment Profile, Performance Profile, Technolog y Implementation Profile, and School to Work Implementation Profile.Findings High School Administrators: Description of Survey R esponses The high school administrators' informati on about their school's involvement in School to Work was used to create the second of thr ee subgroups for the School to Work Implementation Profile. The profile was created usi ng the same method as for the superintendents. Each high school received a total number of points based on the high school administrator's reporting of School to Work implementation activities (See Table 4). The distribution of high school for each point range is reported in Table 5. Table 5 HS Administrator: STW Implementation Points per High School TOTAL POINTS NUMBER of SCHOOLS 1995 1996 0 -11 5 3 11-20 6 6 21-30 3 2 31-40 5 5 41-50 3 4 51-60 0 2 61-74 2 2 TOTAL 24 24 High school administrators were asked to identify the number of grade levels involved in various types of multiple learning envi ronments during the 1995-96 and 1996-97 school years. On average, 2.06 grade levels per school were providing out of class experiences in 1995-96; 2.28 were in 1996-97 (See Table 6). Schools, on average, did not support a single grade level for work site simulations, internships/apprenticeships, or youth-run enterprises in either school year. Hig h schools averaged over two grade levels of employer visits, and over 1.5 grade level s of job shadowing and community service projects for each year of the program (1995 -96 and 1996-97). Simulated work environments, school assignments in the workplace, youth-run enterprises, internships,
20 of 43work-linked learning, and work study co-ops did not average one grade per school and did not take place at all four grade levels in any of the schools. On a 1 to 10 scale (lowest to highest), h igh school administrators were asked to estimate their commitment to staff development rela ted to School to Work. Active/handson learning averaged 4.33 for 1995-96 and 5.11 for 1996-97, with a range of 0 to 9 and 0 to 8, respectively. Developing multiple learning en vironments, partnership strategies, world class standards, and integrating career devel opment in 1995-96 and 1996-97 all had a mean score of less than 5. The exception was integrating career development, it did achieve a mean of 5.56 for the 1996-97 school year. Most administrators did not report that a majority of the activities increased in occurrence from year one to year two (See Table 6). The most common activities that took place were out of class experiences, employer visits, and job shadowing. Each individual activity had an average range of occurre nce of .5 to 2.44 grade levels. There was an average one grade level of support per Schoo l to Work activity for each high school. Each high school administrator was asked to estimate his/her perception of the importance of different types of staff development. All of the staff development areas shown in Table 7 were emphasized in the federal leg islation, as well as the regional partnership State-approved proposal, as necessary f or all implementation. On a scale of 1 to 10 (lowest to highest), the highest average comm itment to any single staff development component was the integration of career development, which had a mean level of commitment of 5.6. High school administrat ors demonstrated a low level of commitment to all staff development components rela ted to School to Work. Partnership strategies, an activity used to develop relationshi ps with local employers and organizations to get them involved in School to Wor k, had a level of commitment just above a 3. There was a high level of agreement among high school administrators that School to Work should be a part of curriculum and instruct ion (See Table 8). High school administrators on average perceived School to Work as a more important activity than did superintendents. In contrast, high school admin istrators and superintendents agreed that technology should be a major component of Scho ol to Work. On a scale of 1 to 10 (lowest to highest), high school administrators als o were asked to indicate their level of agreement with four statements about the 1997-98 sc hool year. On average, School to Work as a part of high school curricular and instru ctional efforts was ranked 8.22. The actual range for responses from administrators to t his statement was from 6 to 10. The statement that business and industry should be acti ve in the development and integration of School to Work resulted in the mean response of 6.28, with a range of 1 to 10. The statement that community and parents should be acti ve in the development and integration of School to Work had a response mean o f 5.83, with a range of 1 to 10. Technology and electronic learning as a major compo nent of School to Work received a response mean of 7.72, also with a range of 1 to 10 Table 6 High School Administrators: Responses to Su rvey Number of Gra de Levels Min. Max. Sum Mean Out of class experiences 1995-96 0 4 37 2.06 1996-97 0 4 41 2.28 Work-site used as classroom 1995-96 0 2 6 .33 1996-97 0 2 10 .56
21 of 43 Internships or apprenticeships 1995-96 0 2 12 .67 1996-97 0 2 12 .67 Co-ops of youth run enterprises 1995-96 0 4 10 .56 1996-97 0 4 10 .56 Employer visits to classroom 1995-96 0 4 39 2.17 1996-97 0 4 44 2.44 Job shadowing 1995-96 0 4 28 1.56 1996-97 0 4 35 1.94 Simulated work environment 1995-96 0 4 14 .78 1996-97 0 3 11 .61 School assignments in workplace 1995-96 0 2 5 .28 1996-97 0 3 9 .50 Youth-run enterprises 1995-96 0 2 6 .33 1996-97 0 2 6 .33 Community service projects 1995-96 0 4 27 1.50 1996-97 0 4 30 1.67 Internships 1995-96 0 2 11 .61 1996-97 0 2 10 .56 Work-linked learning 1995-96 0 4 9 .50 1996-97 0 4 9 .50 Work-study co-ops 1995-96 0 2 11 .61 1996-97 0 2 13 .72 Table 7 High School Administrators Responses: Commitment to STW Staff Development (1 to 10 scale) Maximum Mean Active/hands-on learning* 1995-96 9 4.3 1996-97 8 5.1 Development multiple learning environments 1995-96 8 2.9 1996-97 8 3.4 Partnership strategies 1995-96 9 3.2 1996-97 9 3.3 Develop world-class standards 1995-96 10 3.6 1996-97 10 4.2 Integration of career development 1995-96 9 4.4 1996-97 *Response was 1 for all items Table 8 High School Administrators' Response: Actio ns for 1997-98 Low Hi gh Mean
22 of 43 STW should be part of curriculum and instruction 6 1 0 8.2 Business and industry should be active in integration 1 1 0 6.3 Community and parents should be involved in integration 1 1 0 5.8 Technology should be a major component of STW 1 1 0 7.7 level of agreement 1=none 10=completely High School Administrators: Analysis of Responses a s a STW Implementation Profile As shown in Table 9, there is a strong po sitive association between the high school administrators' School to Work implementation profi le and the amount of job shadowing and simulated work environment activities. The more job shadowing that was taking place in the high school, the higher the overall le vel of implementation of School to Work. There is no association between the high scho ol administrator's and the superintendent's estimation of the level of impleme ntation of School to Work. High school administrators were asked to estimate their level of commitment to School to Work staff development activities as desc ribed in the Federal legislation and the State-approved regional partnership proposal fo r the implementation of School to Work policy. Based on high school administrators' r esponses, there is a high association between the level of implementation of School to Wo rk and the administrators' commitment to active/hands-on learning, development of multiple learning environments, utilization of partnership strategies development of world class standards, and integration of career development staff develop ment activities (See Table 10). Table 11 shows the positive high correlation between the High School Administrators' School to Work Implementation Profile and the percentage o f youth-at-risk, the number of years the superintendent has been employed in that title in the current district, and the total number of years the superintendent has held that ti tle. Table 9 High School Administrators: STW Profile & Specif ic Responses High School Supers Job Simul. Administrators Rank Shadow Environ. Admin Profile .320 .546* .537* Of STW '95 Admin Profile .451 .735* .575* Of STW '96 School assign Youth Co mm. Internin workplace Enterp. S erv. ships Admin Profile .305 .086 .5 70* .331 Of STW '95 Admin Profile .221 .029 .4 39 .335 Of STW '96
23 of 43 *p<.05 **p<.01 Table 10 High School Administrators: Commitment to STW St aff Development Active/ Develop PartnDevelo p Integrate hands-on multiple ership World Class Career learning learn Strat. Standa rds Develop. environ.Admin STW .828** .907** .765** .839* .758** Profile in '95Admin STW .774** .931** .698** .819* .461 Profile in '96 p< .05 ** p< .01 Table 11 High School Administrators Profile of S TW Implementation % YAR 1995 Superintendent: Sup erintendent: Yrs in district To tal # of yrs HS Admin STW .442* .623* .792** Profile 1996 p< .05 ** p< .01 Regression analysis, using the high schoo l administrators survey data, was used to determine the level of variance in high school admi nistrators' perceptions of School to Work implementation explained by the performance, e nvironment, and technology implementation profiles. The results are different from the analysis using the superintendents' responses. The level of variance i n the High School Administrators' School to Work Implementation Profile is explained significantly (p <.10) by the high school's environment, performance, and level of imp lementation of technology. Roughly 50% of the variance can be explained with less than ten percent chance for error (See Table 12). Table 12 Regression AnalysisPredictors: Environment Profile, Technology Profile Performance ProfileDepend. var.: H.S. Administrators Profile of STW Im plementation '95 R=.707 Rsquare=.499 F=2.99 si g=.088 Depend. var.: H.S. Administrators Profile of STW Im plementation '96 R=.718 Rsquare=.515 F=3.184 si g=.077 High School Administrators: Summary
24 of 43 According to these high school administra tors, the level of participation in School to Work increased from the 1995-96 to the 1996-97 s chool year. Each of the required School to Work activities was taking place in at le ast one grade level in every school in the region. In terms of staff development, integrat ion of career development had the highest level of commitment. The development of par tnership strategies, however, was among the lowest concerns. High school administrato rs saw a need for involvement in workforce development activities and the staff deve lopment for developing those activities in general. But they did not follow thro ugh with the training of staff or program implementation across all grades. High school administrators extended stron g support for the continuation of School to Work into the 1997-98 school year. In addition, it was important business and industry and community members remain involved in the implem entation process. They also wanted technology to be a major component of the ac tivities supported. The higher the percentage of youth-at-ris k in the school district, the higher the implementation of School to Work as described by th e high school administrator. The greater the length of time the superintendent was e mployed by the district in his/her current title, the greater the level of School to W ork implementation. The greater the total number of years the superintendent has held that ti tle, the greater the level of School to Work implementation that took place in the high sch ool. Student performance, the local environmen t, and the level of technology implementation reached in the high school helped to explain the variation in the level of School to Work implementation perceived by the high school administrators. Specifically, there was a strong and significant re lationship between the local conditions under which the school was operating while implemen ting the policy and the success of actual implementation. School Selection and Followup Interview Process To understand more completely what affect ed the school district participation in School to Work, follow-up interview methods were us ed. The intent of the interview was to gather data to determine why schools implemented the School to Work policy at varying degrees although they were part of the same consortium. High schools in the region were selected based on their performance pro file and School to Work implementation profile, determining the school's pl acement on a grid (See Figure 1). The performance and School to Work implementation profi les were chosen as selection criteria because of the availability of data.
25 of 43 Figure 1. Placement of School Districts on the GridSummary of the Interview Process The interviews with stakeholders provided information that could not have been collected through surveys. Interviewees shared what had happened during the decision making process when considering involvement in Scho ol to Work. All agreed the initiative was worthwhile, but there were common is sues keeping schools from becoming involved: lack of resources to become involved, lac k of resources for continuation, and competition with other efforts. Participating schoo ls saw the School to Work initiatives as a means for schools to prepare and engage studen ts in career preparation decisions. They also recognized the desirability of creating c onnections between the community and teachers, thus exposing them to new ideas.Summary of Three Stages Four profiles were created to determine t o what extent characteristics of the local organization explained the variance in implementati on of Federal policy. The performance, environment, and technology implementa tion profile of a local organization significantly predict the variation in School to Wo rk implementation, when the high school administrator described the implementation. According to the survey results of
26 of 43superintendents, high school administrators, high s chool teachers, and employers, School to Work implementation levels increased from 1995-9 6 to the 1996-97 school year. During the same period, commitment to School to Wor k activities also increased. The majority of stakeholders surveyed felt very strongl y that School to Work should continue during the 1997-98 school year.ConclusionsSummary of Findings This study compared the original intent o f a specific educational reform policy to its local adaptation. Specifically, it focused on t he implementation of the Federal School to Work Opportunities Act of 1994 in the 44 public high schools participating in the New York State Mohawk Valley Workforce Preparation Consortium during the 1995-96 school year. To compare the original intent of the School to Work policy to the local program adaptation and explain their differences tw o questions were answered. Participation in a consortium arrangement designed to facilitate the implementation of State or Federal policy resulted in active, if mini mal, implementation efforts by individual consortium members. Stakeholders from th e high schools studied described their schools as having been active in the implemen tation of School to Work during the 1995-96 school year. However, the level of implemen tation throughout the region was limited and minimal in some high schools. The organizational and community context descriptions of those high schools in which School to Work was perceived as having both s pecific and important policy effects differed significantly from those where the policy was perceived as having little or unimportant effects. The greater the perceived e ffect of School to Work on operations, the higher the level of School to Work implementation in both 1995-96 and 1996-97. The greater the perceived importance of Sc hool to Work, the higher the level of implementation. However, in no high school had t he program been implemented in all four high school grades. A high school setting where School to Work activities were perceived by local stakeholders as having great spe cific and important effects differs from a high school setting where School to Work act ivities were perceived as having little or no effect. The differences in organizatio nal demographics and community characteristics that contributed to the high school administrator's description of School to Work implementation are statistically significan t. Profiles of high school student performance, high school operating environment, and the level of technology implemented predicted the level of School to Work i mplementation described by the high school administrator. However, none of the ind ividual profiles explained the variance in School to Work implementation by themse lves. Therefore, the extent to which there were differences in perceptions of the enriched or impoverished status of technology did not contribute to perceptions of Sch ool to Work. Similarly, there were no significant differences in perceptions of curriculu m scope and sequencing or in instructional strategies seen as being influenced b y School to Work. Based on the findings of this study, impl ementation was assessed by describing the reported activities of the local integration an d comparing them to the original intent of the policy. The difference between original inte nt and reported local adaptation were interpreted through the backward mapping analysis a s discrepancies in fulfilling the purpose of the particular policy. This study argued a more inclusive approach. Backward mapping includes an explanation of local conditions and expands policy analysis beyond the determination of success or failure of a policy based on the
27 of 43measurement of the objectives. This bottom-up appro ach explained the conditions under which the policy was implemented. The process was m ore than simply measuring success by meeting policy goals; it explained the r elationship between characteristics of the organization and local level policy implementat ion. Thus, there was consideration for the context in which implementation took place. In terms of context, backward mapping delineated both the conditions prior to and during implementation and their relationship to the variance in the level of implem entation.Backward Mapping as a Method of Policy Analysis Backward mapping was effective in accompl ishing three important objectives: analyzing policies that have been created to have a n impact at the local level, providing contributions to the knowledge of the policy implem entation process, and enabling a determination and explanation of the success of the studied policy. The process takes into account the difference in localities. Therefor e, the analyst takes into consideration that the many differences between the localities--i ncluding leadership, environment, student performance, involvement in other innovatio ns, and stakeholder perceptions--contribute to the differences in polic y implementation among these localities. Consequently, when the implementation p rocess is to take place over multiple years, or if it is slated for replication in other localities, the analyzed policy can be adjusted or improved based upon the findings. Forwa rd mapping, as a comparison, explains to what extent the implementor has met est ablished benchmarks. Variation in implementation is only considered more or less succ essful compared to a set goal, which is frequently complete integration. Backward mappin g takes the analysis an additional step and explains the extent to which the locality' s perceptions influenced variations of implementation. By addressing perceptions and the c onditions of the environment, backward mapping removed the focus from the policy objectives in determining implementation success. Rather, success is expresse d in terms of the effect of the local implementing organization. Backward mapping was use ful for determining and explaining the relationship between the locality pr ior to policy implementation and how policy, as intended to be implemented by the federa l government, was adapted to those conditions. During the process the direct analysis of the policy was shifted to focus on the high school. The organization's impact on the p olicy guiding the implementation was explained through an analysis of its characteristic s. The result was the delineation of predictors associated with the successful implement ation of the policy. However, the comparison of intent and act ual implementation is only one step in the backward mapping process. Backward mapping is a lso useful for analyzing why the implementation did or did not take place. Answering this question requires a process which enables the researcher to examine the conditi ons surrounding the policy. Because schools do not operate in a static environment, the policies being implemented by them are subject to forces which can cause the school it self to change. Simply measuring the ability to meet objectives is often worthwhile and appropriate. But for explaining varying degrees of implementation and how mediating variables may be dealt with by future policies, the information compiled in the ba ckward mapping process is beneficial. Student performance, environment, and tec hnological characteristics of the high school explained the variation between Federal inte nt and local adaptation. Stakeholders were asked to provide their perceptions of the impl ementation process. Concurrently, the data collected about the organization explained wha t had been taking place at the local level, irrelevant of the policy implementation. For example, the organization may have been involved with other important and specific cha llenges that precluded an
28 of 43implementation at a pace similar to other organizat ions. Likewise, similar groupings of organizations may have differentiated levels of imp lementation due to a common factor, such as the participation in a particular event. Ba ckward mapping was an effective method for delineating that implementation variance was explained by characteristics of the local organization. Backward mapping ascertained that organiz ational leadership, past performance, stakeholder perceptions, external environment, and involvement in other efforts were all associated with policy implementation success. Both forward and backward mapping explained the level of implementation attained. But backward mapping also explained why the levels of implementation were reached. Back ward mapping, therefore, was an effective tool in explaining the relationship betwe en the original policy, the implementing locality, and the actual implementatio n. Backward Mapping and Local Level Characteristics Backward mapping elicited information whi ch supports an understanding that successful implementation is not based solely on we ll-written policy. Characteristics of the high school which impacted the implementation p rocess were identified and measured. Their association with variation of schoo l implementation from established goals was explained in the final analysis of the pr ocess.Perceptions and Participation It was found that the district and buildi ng level leadership had an impact on the level of policy implementation. Each leader, as a s takeholder, had a different perception of the policy's interpretation and level of importa nce. Participation in terms of the policy implementation meant signing as a consortium partne r. But the variation of implementation among schools could not be explained simply by agreement to join a consortium. All of the schools identified themselve s as partners in the submission of the grant to the State, but not all of them implemented the same level of School to Work initiatives. Instead, interpretation of the policy inf luenced actual participation. Some school leaders interpreted School to Work as being career exploration. Those schools implemented basic activities such as tours, speaker s, and job shadowing. But they were less likely to have implemented components of the p olicy that took a higher level of commitment of resources, such as work-based learnin g. At least two schools excluded themselves from initially participating because of their leaders' limited interpretation of the legislation. Not until there were definitive ex amples of the policy being implemented in other school buildings did these leaders "buy-in to" the concept.Decision Making The original School to Work policy provid ed the flexibility to adapt implementation to meet local needs. It was at the d iscretion of the consortium to determine how the framework of support and implemen tation would be provided. The local school determined how the policy was actually implemented. This study found that the way in which a superintendent perceived multipl e reform efforts, as well as the way the high school administrator perceived the School to Work policy, explained the discretion used to adapt the policy to local need. For some schools, it meant
29 of 43implementing School to Work components as described within the language of the legislation. Other schools participated for different reasons. One school stated that it used School to Work support as a means for exposing its staff to new ideas and environments with the expectation that participation would effec t change for the benefit of children. There is evidence that this happened, both in that school and others. High school teachers involved in School t o Work activities described changes in curriculum scope and sequence or in instructional s trategies. Teachers described how classrooms and individual lessons had been affected positively because of their involvement in staff development programs funded by School to Work. In many cases, they became more aware of the resources (e.g., less ons being used by local teachers) around them. They discovered how the application of learning had a positive effect on the students. These teachers had previously believe d that the application of knowledge did not have to take place in the classroom for stu dents to learn and develop skills. Teachers also discovered that there were many oppor tunities for students to test their knowledge in surroundings not previously considered learning environments, such as factories, farms, stores, and the community.Competition for resources While this study did not set out to analy ze the issues of resources, there are some interesting findings are worth mentioning. The comp etition for scarce resources contributed to the variation of policy implementati on levels between school districts. Poor schools described two areas of concern: the le vel of resources necessary to become involved and the resources needed to sustain the ef fort. Poor schools with a small local employment base that perceived School to Work merel y as a career awareness strategy had low rates of participation. These educators did not see how they could become involved when there were so few community resources Other poor schools were concerned about sustaining the effort after funding was no longer available. Either they did not feel the school had the fiscal resources to continue the initiative, or they perceived School to Work as one more initiative tha t would simply disappear. Resource allocation also impacted the dec isions of more affluent schools. Some of these schools and their communities put high demand s on the students to perform academically. Their expectation of student preparat ion was for college and highly skilled careers. In some cases, this expectation resulted i n School to Work implementation at higher levels. It was a means of exposing students to the demands of the workplace and the high level of skills required to be successful in the positions these students sought. For other schools with the same high expectation of their students, the perception of School to Work as applicable only for the "career-b ound" was a detriment. Their perception was the student should focus solely on a cademic preparation leading to college.Summary of Backward Mapping Policy Analysis By using backward mapping in this study, attention was focused on the importance of the locality during the policy implem entation process. Backward mapping established a broad framework within which an analy sis of the actual implementation process was carried out. Within this process, the a nalyst is able to discover and explain what determines the level of success of the policy. By choosing any single Federal policy, one could determine the actual impact on th e local school district. Using this
30 of 43process, it can also be determined the role of the locality in relation to the policy. The policy exists to create change at the local level, but it is the locality which ultimately decides how the policy's intent will be adapted to meet its needs. The key elements of backward mapping in policy analysis are as follows: It is effective in describing actual versus perceiv ed implementation. It is effective in explaining actual versus perceiv ed participation. The effect of the locality is considered to help to explain variation in implementation. Perceptions of importance and policy effect are mea sured. A more complete picture of policy implementation is presented. The thoroughness with which policy implementation i nformation is provided is at a high level. Backward Mapping, Forward Mapping, and School to Wo rk In the case of School to Work, a form of forward mapping analysis had already been completed by the Mohawk Regional Consortium, t he New York State Education Department (NYSED), and the Federal government. Eac h level of governance had used previously set criterion established within the ori ginal Federal legislation, approved regional implementation plans, and state level impl ementation plans to determine the success of the policy. These studies found that the schools had the required representation in place and that they were engaged in the required activities. Thus, the state and the region appeared determined to have su ccessfully implemented School to Work policy. Objectives of the policy had been met to varying degrees in each school, and continuation of the project was contingent on t he objectives set for the following year of implementation. According to Mohawk Valley Consortium members, there had been little feedback given to the region from these studies upon which to base continuation on a commitment of resources to areas in need of improvement. The data collected during this process th at can be labeled as an example of forward mapping was for the most part unidimensiona l. There was snapshot of activities on a given day. But the discussion of the data coll ected assumed that all the activities were the result of School to Work participation and use of funds from the program. None of the data was in a format enabling the deter mination of association or any such relationship linking policy to actual implementatio n. Much of the data was compiled into anecdotal stories that spoke of program object ives being met in various schools. There were 44 school districts with high schools that agreed to be partners in the Mohawk Valley Workforce Preparation Regional Consor tium to implement the Federal School to Work Opportunities Act of 1994 (STWOA). C onsistent with other non-academic studies involving this region, this st udy found that schools participated in varying degrees. Therefore, the agreement to partic ipate in a consortium arrangement designed to facilitate the implementation of Federa l policy did not result in identical implementation levels being achieved by individual consortium members. By using backward mapping, it was found t hat the level of participation is best explained by the perceptions of the high school adm inistrator. In addition, the perceptions of the high school administrator are pr edicted by the characteristics of the school. The high school administrators' perceptions of School to Work implementation vary based on the performance of the students, cond itions in the community, and level of technology implemented in the school. The better th e students are performing, the more resourceful the external environment. The more tech nology present in the school, the
31 of 43higher the level of policy implementation. Therefor e, the level of school participation in the policy's initiatives varied according to percep tions and characteristics of the locality. Local Characteristics Characteristics of the local school distr icts in the Mohawk Valley Consortium were measured to explain the variation of high scho ols' involvement in School to Work policy initiatives. The school's characteristics an d stakeholders' perceptions were measured to determine their impact on high school p articipation. By grouping characteristics into three broad areas called profi les (performance, environment, and technology implementation), the combination of prof iles were found to explain the variations in success of School to Work implementat ion. Performance Profile The performance profile was used as a gro uping that characterized student performance as a means for measuring the school's e ffectiveness. When the performance profile was combined with both the environment char acteristic profile and the technology implementation levels profile, the varia tion of implementation between schools was explained to a high degree. Therefore, the combination of previous student performance, the conditions of the community, and t he level of participation in other reform efforts combined predict the level of implem entation of the School to Work policy. The performance profile of the high schoo l was found to have a few specific items that were significantly associated with the level o f School to Work implementation. The greater the percentage of the total number of stude nts receiving NYSED Regents Diplomas, the higher the level of School to Work im plementation. This becomes an important statistic considering the perception of t he teachers interviewed. Teachers attributed low implementation levels to the lack of flexibility of students working towards a Regents diploma. The teachers' cited spec ific problem areas, such as preparing students specifically for items on the Regents exam and the lack of time for "extracurricular" and exploratory activities that prevent ed Regents student participation. The teachers' statements are contradictory to the findi ng. Therefore, a question is raised as to what was happening in the classroom where teachers were asked to participate in School to Work but did not because of Regents requirement on students. Were the teachers' perceptions inaccurate? Were there a greater number of non-Regents students actively participating in School to Work and more School to Work activities in place in schools where higher percentages of students receive a Rege nts diploma? If so, could this be for the purpose of providing more opportunities in thos e schools for those nonRegents students if they were assumed to be workplace bound ? These are questions to be addressed in further studies of the policy. Schools within the Consortium with high p ercentages of Regents students and a high level of School to Work implementation had per ceived School to Work to be something more than career exploration. In these sc hools, there was evidence of career awareness activities such as tours, speakers, and j ob shadowing all of which were typically already taking place, and which took less commitment by the teacher and the school to integrate. But there was also evidence of higher levels of integration taking place within the curriculum, instruction, and even the assessment. Schools graduating 85% or more students with Regents Diplomas had crea ted interdisciplinary projects in
32 of 43areas such as Math, Science and Technology. They al so changed methodologies in the classroom, incorporating applied learning technique s and utilizing the community and the workplace as learning environments. Schools had even begun to assess student knowledge through hands-on application and problem solving using actual workplace situations. These schools did not perceive the Rege nts diploma as a barrier; they used School to Work initiatives as a means for creating new ways to meet its rigors. According to this study's findings, these occurrenc es can be attributed to two factors: the leadership provided by the high school administrato r and the support of multiple reform efforts by the superintendent.Environment Profile The environment profile was a grouping wh ich characterized the conditions of the environment in which the local high school operated Schools located in communities of high and low unemployment and wealth were able to i mplement School to Work policy initiatives. Measures of association were applied t o determine the extent to which the level of School to Work implementation was explaine d by specific community characteristics. It was found that the types of bus iness in the community and the percentage of youth at risk were highly correlated to levels of implementation. The higher the percentage of people in the community em ployed by agriculturally-based businesses, the higher the level of implementation of School to Work. Even though agriculture was the lowest percentage of the total businesses, no such significant correlation could be found for manufacturing, retai l, or service-based businesses. The analysis also determined that the higher the percen tage of youth who are "at-risk", the higher the level of School to Work implementation. Several factors were discovered during the quantitative follow-up interview process that could explain this fact. Prior to the implementation of the STWOA of 1994, several St ate and Federally funded programs were addressing the needs of the youth-atrisk population through School to Worktype activities. The Governor's School And Bu siness Alliance (SABA), Private Industry Councils (PICs), and Workforce Preparation Pilot projects were all based heavily on the needs of high school children from l ow wealth families. Twenty-seven of the 44 Mohawk Valley Workforce Preparation Consorti um school districts were located in a BOCES service area that also had a SABA. Fifte en of the 44 school districts were involved in the original New York State Workforce P reparation Pilots funded through the Governor's office. Twenty-four of 44 school dis tricts were located in a BOCES that was involved in the second round of Workforce Prepa ration grants funded by the Governor's office and managed by NYSED. All of the schools were supported by a regional PIC. Therefore, all of the schools had bee n participating in a combination of School to Work-type initiatives based on their at-r isk youth population prior to the implementation of STWOA of 1994.Technology Profile The third local characteristic groupings to be measured were those related to participation in local reform efforts. There were a ctually two measurements of commitment to other reform efforts. One was the lev el of implementation of educational technology within the school, which was determined from an existing data set. The second was stakeholder perceptions of the importanc e of involvement in other reform efforts that was collected from this study's survey The level of technology implemented
33 of 43in a school was not significantly associated with t he level of School to Work implementation. However, the more important the inv olvement in multiple reform efforts, the higher the level of technology integra tion into a school. There was found to be a strong association between technology implemen tation level and the level of importance of participation on multiple reform effo rts as indicated by the school superintendent. In addition, there was a strong and significant association between the level of School to Work implementation as perceived by teachers and the level of importance of participation in multiple reform effo rts as indicated by superintendents. The more important a superintendent perceived parti cipation in multiple reform efforts, the more School to Work activities and technology w ere implemented by the teachers. Therefore, it was not the actual participation in a nother reform effort that explained future participation in other reform efforts. It wa s the perception of the superintendent of the importance of involvement in other reform effor ts that was associated with implementation of other Federal policies. Stakeholders' perceptions of involvement in multiple reform efforts were measured as a local characteristic that was associa ted with the level of School to Work implementation. The perceptions of the superintende nt were not indicative of the level of School to Work implementation in the high school The analysis of local level data did find that there was a significant and very stro ng relationship between the number of years the superintendent has been employed in the d istrict under his/her current title, the total number of years he/she has held the title, an d the level of School to Work implementation as reported by the high school admin istrator. Therefore, the longer the person held the title of superintendent, the more l ikely School to Work was reported by the high school administrator to have been implemen ted. In comparing the perceptions of educators and employers involved in School to Work, it was found that neither superintendent's no r employers' perception of an effect on organizational operations was associated with in volvement. This can be attributed superintendents perception of School to Work as mea ns for creating change in the classroom and providing opportunities for students, rather than as a means for impacting the operations of the organization. But part of the impetus for the region to become involved in School to Work was to work with employe rs to improve the future workforce. However, the results showed teachers kne w very little of the partnerships that existed with business. There was ample documentatio n in the region of partnerships including directories of partners, teacher/student documented activities with business, and a computerized database of available employer p artners and the types of activities they would support. But the teachers and administra tors did not identify them as existing in their building. It was assumed, based on the available li terature, that businesses were involved, at least partly, because they were concerned for their future employees and how well they were being prepared. However, employers participate d in the activities for reasons other than organizational impact. During the follow-up in terview process employer-partners indicated there had not been enough time invested i n the implementation of School to Work to realize impact on their own business. But i t was expected by employer-partners interviewed that employers would be able to see an impact in the long term. Continuation of Implementation Most schools indicated that it would be i mportant to participate in School to Work activities during the 1997-98 school year. The leve l of previous implementation was not associated with the level of stated importance of p articipating the following year. A
34 of 43majority of the schools indicated a high level of a greement that the integration of School to Work into curriculum and instruction with the in volvement of community members and employers was very important. The business and industry members of the community described School to Work involvement during l995-l996 as being important. Al l of the respondents strongly agreed that School to Work should continue, but at the sam e time they did not perceive the policy as having any impact on the operations of th e business. A few employer partners did expand on this point. Their view was that Schoo l to Work is a means for helping students learn, which was their rationale for parti cipating. There would need to be a longer-term interaction before they could assume th at the policy would have an impact on the operations of the business. In the long-run they did assume that students would be better prepared for work with such programs. For th at level of impact to take place, they asserted that the School to Work implementation wou ld have to be an integral part of all schools.Local Characteristics Summary By using backward mapping this study foun d that the perceptions of stakeholders in district and building level educational leadersh ip positions were an important factor in determining the level of implementation of School t o Work policy in 9-12 high schools located in the Mohawk Valley Region. The high schoo l administrators' reported level of implementation varied according to local characteri stics and conditions. When superintendents found involvement in multiple refor m efforts to be important, teachers reported higher levels of School to Work implementa tion. Actual implementation was best predicted by the high school administrator. Ye t, only one BOCES in the region documented providing technical assistance and train ing specifically to high school administrators. Had the significance of the high sc hool administrator been known the region and its leadership may have placed greater e mphasis on staff development for high school administrators. It may have marked reso urces dedicated to improving the leadership and understanding of School to Work and prepared them to provide more effective support during the implementation of this policy. This should not be considered a weakness on the part of the regional l eadership, but a lack of information available to policymakers who required superintende nts to sign-off and teachers to implement School to Work. Backward mapping has prov ided the means for collecting this information that, if used, could impact the Sc hool to Work implementation policy for the last two of the five-year policy implementa tion phase in New York State and subsequent years in other states.Using Backward Mapping Process for Policy Analysis Policy analysis through a backward mappin g lens has proven to be an effective process for determining the predictors and identify ing items that explain the success of implementation. Previous analyses of School to Work implementation in the Mohawk Valley Region followed a forward mapping and top-do wn process. These attempts to analyze implementation of the Federal policy determ ined the level to which the region met goals and objectives and how much School to Wor k was implemented. It did not explain why the implementation took place, or who o r what contributed to the level of implementation. The results of the forward mapping process did not explain the variation of implementation levels between schools. The forward mapping process focused on the measurement of the volume of activit ies and the documentation of the
35 of 43involvement of stakeholder groups. It is the backwa rd mapping approach which addressed how their involvement impacted decision m aking and final implementation in the form of local adaptation. The backward map explained first, that sc hools participated at varying levels and secondly, why schools participated at varying level s. To reach these findings, the process required an initial analysis to determine that ther e was in fact a difference between the schools and the environments in which they operated A determination was then needed that each school agreed to and followed the same gu idelines, adapting the initiatives to meet it's students needs. Each individual school th en participated at a level which best suited it's stakeholders. The stakeholders' decisio ns and perceptions impacting implementation are attributed to a combination of p re-existing characteristics of the school's internal and external environment. Preex isting conditions in and around the school and the perceptions of stakeholders were mea sured by the backward mapping process, and resulted in an awareness that such act ions are valuable to policy analysis. If the policy analysis process had stopped at measurin g goals and objectives, there would have been no determination of the reasons some scho ols contributed to meeting those objectives. It would not have explained how the att ainment of some objectives was actually due to schools participating in other init iatives. It is the backward mapping of the policy that enables as analysis recognizing that the locality may interpret the intent of the policy differently from that of the policymaker. Forward mapping assumes the failure to meet objectives is due to the locality's inability to implement. Backward mapping asks the local actor as implementor how they interpreted the policy. It assumes a varia tion in interpretation results in a variation in adaptation and implementation. If Scho ol to Work was locally defined as a career exploration program, then implementation wou ld reflect activities to support student career exploration. In that case, school-ba sed and work-based learning activities would be less likely to take place. Therefore, a sc hool's lack of implementation is not related to its ability; it has to do with discretio n based on perceptions. Choices were made because of forces acting upon the school that impact local actors' decision making. The analysis was built from a local-level perspective to explain local actions to meet a Federal policy guideline. The survey instrum ent of the backward mapping process asked the district level, building level, c lassroom level, and employer partner to describe their perception of School to Work, what i t meant to them, how important it was, and to what extent had it really taken place. To take the analysis a step further, the process used this data to identify special cases. T hese cases either met or went against expectations and were slated for a follow-up proces s. This second step looked for actual documentation and reviewed initial findings to gath er a clearer understanding why certain perceptions were held and actions taken. Th e effort yielded a compilation of data that explained how it was the conditions within whi ch the locality operated and made decisions that impacted implementation, not the cla rity of goals or objectives. Therefore, future policy making should be based on more comple te information on the locality in which the target population resides and for which p olicy is intended, not clearer policy goals.Limitations of the Study Limitations of the study can mostly be at tributed to the use of the technology implementation profile. The technology implementati on profile used the total number, rather than the ratio of computers to students or t eachers in the school. This was problematic because a large school could have large r numbers of computers, yet have a
36 of 43lower student or teacher to computer ratio. A small er school with a lower total number of computers but a higher student to computer ratio would have a comparably better level of technology integration. This may have skew ed the rankings of this particular profile and may have impacted this profile's lack o f explanation of variance in the School to Work implementation profile. Another issue with the creation of the te chnology implementation profile was the time lag between the data collected for the Mann st udy and the data collected for this study. The time period was actually five months. Sc hools that were surveyed last had more time opportunity to install more computers. Th erefore, data could have been skewed in favor of school districts that were surve yed later in the study.Implications for Future Research and Policymaking Backward mapping is not specific to any o ne type of policy. The process followed in this study could be replicated for any policy. I n this study, backward mapping was successfully applied to a Federal policy implemente d at the local level. The actual items specifically measured related to School to Work pol icy could be replicated in other areas of New York State. There are over 50 other partners hips throughout the state for which this process could be useful in determining factors of local success and further testing the application of the backward mapping process. Th ere were seven other states that began implementation during the same time period, a nd several more that have begun implementation in the last year. All of these local partnerships could be analyzed using this approach. The specifics of replicating this pa rticular study could be challenging. Student performance items from other states would h ave to be substituted with that particular state's student performance indicators. Therefore, the use of backward mapping is universal in its application to other po licies. The exception would be in cases where local data was not available or the analysis does not warrant the consideration of local action. In cases where there is no means or c ause for the use of discretion by the local actor, backward mapping may not be the most a ppropriate method. The creation of future policy could be en hanced through the use of results from a backward mapping policy analysis. The findings cont ribute to an improved understanding of how the implementing organization and the environment in which it operates react to the policy. This seems to be most useful in cases of multi-year implementation policies. These are situations where the policy is implemented, the policy can be analyzed after the first year, and ad justed to improve the effectiveness of the policy in future years. Short-term policy imple mentations would most likely not yield information useful for the improvement of pol icy implementation in that specific case. However, it would be useful for replication o f the policy implementation in other states or localities and provide useful information for the improve of other future policies. It also would be effective in comparing i mplementation levels during funding years and continuation levels after funding has run out. Essentially, backward mapping provides for a more global understanding of how pol icy implementation works and what impacts its success.ReferencesBach, B.W. (1989). The Effect of Multiplex Relation ships Upon Innovation Adoption: A Reconsideration of Rogers' Model. Communication Monographs,56,2. 133-150. Benz, M.R., et al. (1997). School to Work Component s That Predict Postschool Success
37 of 43for Students with and Without Disabilities. Exceptional Children,63,2. 151-165. Berman, P. and McLaughlin, M.W. (1973). Implementing Innovations: Revisions for an Agenda for a Study of Change Agent Programs in Educ ation. Study of Change Agent Programs: A Working Note. RAND Corporation. Santa Monica. Coleman, J.S., and Dobeare, K.M. (Ed). (1975). Prob lems of Conceptualization and Measurement in Studying Policy Impacts. Public Policy Evaluation. Sage. California. Council of Chief State School Officers. (1996). 199 6 CCSSO Issues and Positions: Federal Education Policy and Funding. (On-line). ht tp://www.ccsso.org/96legpap.htm. Davis, O.L., Jr. (1994). Don't Adopt; Adapt; A Remi nder for Every Year. Journal Of Curriculum and Supervision, 9,4. 403-405. Decker, P.T. (1997). Education and the Economy: An Indicators Report. Mathematica Policy Research, April Dunn, W.N. (1994). Public Policy Analysis: An Introduction, 2nd Ed. Prentice Hall. New Jersey.Education Reform and School to Work Transition In W ashington State. (1997). (On-line). http://www.wa.gov/wtb/edreform.htmlElmore, R.F. (1978). Organizational Models of Socia l Program Implementation. Public Policy, 26,2. 185228. Elmore, R. F. (1979). Backward Mapping: Implementat ion Research and Policy Decisions. Political Science Quarterly, 94. 601-616. Elmore, R. F. (1987). Instruments and Strategy In P ublic Policy. Policy Studies Review, 7, 1 174-186. Elmore, R.F., and McLaughlin, M.W. (1988). Steady Work: Policy, Practice, and the Reform of American Education. The RAND Publication Series, February. RAND Corporation. Santa Monica.Elmore, R. F, and Williams, Walter, (Ed.). (1982). Studying Implementation. New Jersey: Chatham House.Ely, D.P. (1990). Conditions that Facilitate the Im plementation of Educational Technology Innovations. Journal of Research and Computing in Education, 23, 2. 298-305.George, C.A. (1993). Strategic Planning: Key Enviro nmental Indicators for Pennsylvania. Pennsylvania Educational Policy Studies, 17. Giesecke, J. (1993). Recognizing Multiple DecisionMaking Models: A Guide for Mangers. College and Research Libraries, 54,2. 103-114. Giesecke, J. (1994). Recognizing and Managing Multi ple Organizational Approaches. The Dynamic Library Organizations in a Changing Env ironment. 29-46.
38 of 43Grummon, P. T.H. (1994). Evaluating Systemic Change in School to Work Initia tives. Paper presented at Annual Meeting of the American E ducational research Association, New Orleans, April.Hall, G. and Galluzzo, G. (1991). Changing Policy I nto Practice: School-based Decisionmaking. Policy Issues, November. State Policy Program: Appalachia Education Laboratory.Hess, M.A. (1997). School to Work: Linking Learning to Livelihoods. Curriculum Update. Fall 1997.Ingram, H., and Schneider, A. (1990). Improving Imp lementation Through Smarter Statutes. Journal of Public Policy, 10,1, 69-70. Jennings, J. H. (Ed.). (1995). National Issues In Education: GOALS 2000 and School to Work. Washington, D.C.: Phi Delta Kappa International an d The Institute for Educational Leadership.Johnston, B.J. (1993). The Transformation of Work a nd Educational Reform Policy. American Educational Research Journal, 30,1. 39-65. Legislative Analyst's Office. (1995). Implementing New Federal Education Legislation. (On-line). http://www.lao.ca.gov/rp020195.html.Lindblom, C. E. and Woodhouse, E. J. (1993). The Policy Making Process. New Jersey. Simon and Schuster.Mazmanian, D.A., and Sabatier, P.A. (1981). Effective Policy Implementation. Lexington Books, Massachusetts.McDonnell, L.M., and Elmore, R.F. (1987). Getting t he Job Done: Alternative Policy Instruments. Education Evaluation and Policy Analysis, 9,2. 133-152. McLaughlin, M.W. (1987). Learning From Experience. Education Evaluation and Policy Analysis. 9,2, 171178. Mohawk Valley Workforce Preparation System Consorti um. (1995). Application for a Subgrant to Local Partnership under the New York St ate Education Department School to Work Opportunities Implementation Grant.Murnane, R.J. (1987). Improving Education Indicator s and Economic Indicators: the Same Problems?. Educational Evaluation and Policy Analysis, 9,2. 101-116. National Center on Education and the Economy. (1990 ). America's Choice: High Skills or Low Wages. National Center on Education and the Economy Commi ssion on the Skills of the American Workforce. Rochester.New York Association of Training and Employment Pro fessionals. (1996). Policy Frameworks for New York's Workforce Development Sys tem. New York State Education Department. (1995). Planning and Implementation of a
39 of 43School to Work Opportunities System: Guidelines and Application Materials. Office of Workforce Preparation and Continuing Education.New York State Education Department. (1997). New Yo rk State School to Work: The Evaluation Program. (Online). http:// www.nysed.g ov/workforce/stweval.html New York The State of Learning. (1995). Building a School to Work Opportunities System in New York State: New York's Application fo r an Implementation Grant Under the School to Work Opportunities Act.Oakes, J. (1986). Educational Indicators: A guide f or Policymakers. Center for Policy Research in Education, Occasional Paper Series, Oct 1986. Oakes, J. (1989). What Educational Indicators? The Case for Assessing the School Context. Educational Evaluation and Policy Analysis, 11,2. 181-199. Ormrod, R.K. (1990). Adaptation and Cultural Diffus ion. Journal of Geography, November/December. p. 258262. Orr, M.T. (1995). Evaluating School to Work Transit ion. Education Reform and School to Work Transition Series. Academy for Education Development, National Institu te for Work and Learning. Washington, D.C. Pressman, J. L., and Wildavsky, A. Implementation. Los Angeles, CA: University of California Press.Report to Congress. (1997). Background on the School to Work Opportunities Act. (On-line). http://www.stw.ed.gov/congress/CG_TOC.ht m. Rogers, E.M. and Valente, T.M. (1995). The Origins and Development of the Diffusion of Innovations Paradigm as an Example of Scientific Growth. Science Communication, 16,3. 242-273. Rogers, E. M., and Van de Ven, A. H. (1988). Innova tions and Organizations: Critical Perspectives. Communication Research 15,5, p.632-651. Sabatier, P. A. Top Down and Bottom-Up Approaches t o Implementation Research: A Critical Analysis and Suggested Synthesis. Journal of Public Policy. 6, 1, pp.3036. Scriven. M. (1993). Hard-Won Lessons In Program Eva luation. New Directions for Program Evaluation, American Evaluation Association 58, Summer. JosseyBass. San Francisco.Simon, H. A. (1993). Decision Making: Rational, Non rational, and Irrational. Education Administration Quarterly. 29,3, 392-411. School to Work in new Hampshire. (1997). About Scho ol to Work in New Hampshire. (On-line). http://webster.state.nh.us/doe/stwabout. htm#activ School To Work Opportunities Act of 1994, Public La w 103-239, 108 Stat 568, May 4, 1994, H.R. 2884, Section 3, Washington D.C..
40 of 43State of New York. (1995). In Assembly: 1995-96 Reg ular Sessions. June 9, 1995. 8106. U.S. Department of Education. (1983). A Nation at Risk: A Report to the Nation from the Secretary of Education. The National Commission on Excellence in Education Washington, D.C. U.S. Department of Education. (1996). Progress of E ducation in the United States of America990 through 1994: Reform at the Local Leve l. (On-line). http://www.ed.gov/pubs/Prog95/pt3local.htmlU.S. Department of Education. (1996). School to Wor k Initiatives October 1996: Executive Summary. (On-line). http://www.ed.gov/pub s/SER/SchoolWork/study3.html. U.S. Department of Education. (1996). School to Wor k Initiatives October 1996: Study Aims and Questions. (Online). http://www.ed.gov/p ubs/SER/SchoolWork/study3.html. U.S. General Accounting Office. (1993). Systemwide Education Reform: Federal Leadership Could Facilitate District-level Efforts. Report to Congressional Requestors. GAO/HRD-93-97.Weiss, C.H. (1977). Evaluating Action Programs: Readings in Social Acti on and Education. Allyn and Bacon. Boston. Weiss, C.H., and Bucuvalas, M.J. (1980). Social Science Research and Decision Making. Columbia University Press. New York. Wiles, David K. (1996). Networking High Performance In New York's Secondary Education: The Regents Curriculum Story. University Press of America. New York. Windham, D.M. (1988). Effectiveness Indicators in t he Economic Analysis of Educational Activities. International Journal of Education Research, 12,6. 575-665. Windham, D.M. (1990). Improving the Efficiency of Educational Systems: In dicators of Educational Effectiveness and Efficiency. United States Agency for International Development, Office of Education.Note1. The Hamilton-Fulton-Montgomery BOCES was not a p art of the original Dale Mann study because it is not part of the Mohawk Regional Information Center. The same questions and procedures were followed to replicate the technology survey study portion that provided pertinent information as to the level of technology implemented for this study.About the AuthorArthur M. RecessoAssistant ProfessorDepartment of Secondary Education, Curriculum, and Instructional Technology 1500 N Patterson Street
41 of 43 Valdosta State UniversityValdosta, Ga 31698Phone: 912-259-5099 Fax: 912-333-7167 Email: email@example.com Dr. Recesso serves as Assistant Professor of Instru ctional Technology at Valdosta State University. He received his Ed.D. from the Universi ty at Albany, State University of New York. His primary research interests are in the fields of education policy analysis and instructional/educational technology. Dr. Reces so's recent activities may be found at http://education.valdosta.edu/dept/sedit/ and http: //innovativeknowledge.comCopyright 1999 by the Education Policy Analysis ArchivesThe World Wide Web address for the Education Policy Analysis Archives is http://epaa.asu.edu General questions about appropriateness of topics o r particular articles may be addressed to the Editor, Gene V Glass, firstname.lastname@example.org or reach him at College of Education, Arizona State University, Tempe, AZ 85287-0211. (602-965-96 44). The Book Review Editor is Walter E. Shepherd: email@example.com The Commentary Editor is Casey D. Cobb: firstname.lastname@example.org .EPAA Editorial Board Michael W. Apple University of Wisconsin Greg Camilli Rutgers University John Covaleskie Northern Michigan University Andrew Coulson email@example.com Alan Davis University of Colorado, Denver Sherman Dorn University of South Florida Mark E. Fetler California Commission on Teacher Credentialing Richard Garlikov firstname.lastname@example.org Thomas F. Green Syracuse University Alison I. Griffith York University Arlen Gullickson Western Michigan University Ernest R. House University of Colorado Aimee Howley Ohio University Craig B. Howley Appalachia Educational Laboratory William Hunter University of Calgary Richard M. Jaeger University of North Carolina--Greensboro Daniel Kalls Ume University Benjamin Levin University of Manitoba Thomas MauhsPugh Green Mountain College Dewayne Matthews Western Interstate Commission for HigherEducation
42 of 43 William McInerney Purdue University Mary McKeown-Moak MGT of America (Austin, TX) Les McLean University of Toronto Susan Bobbitt Nolen University of Washington Anne L. Pemberton email@example.com Hugh G. Petrie SUNY Buffalo Richard C. Richardson Arizona State University Anthony G. Rud Jr. Purdue University Dennis Sayers Ann Leavenworth Centerfor Accelerated Learning Jay D. Scribner University of Texas at Austin Michael Scriven firstname.lastname@example.org Robert E. Stake University of Illinois--UC Robert Stonehill U.S. Department of Education Robert T. Stout Arizona State University David D. Williams Brigham Young University EPAA Spanish Language Editorial BoardAssociate Editor for Spanish Language Roberto Rodrguez Gmez Universidad Nacional Autnoma de Mxico email@example.com Adrin Acosta (Mxico) Universidad de Guadalajaraadrianacosta@compuserve.com J. Flix Angulo Rasco (Spain) Universidad de Cdizfelix.firstname.lastname@example.org Teresa Bracho (Mxico) Centro de Investigacin y DocenciaEconmica-CIDEbracho dis1.cide.mx Alejandro Canales (Mxico) Universidad Nacional Autnoma deMxicocanalesa@servidor.unam.mx Ursula Casanova (U.S.A.) Arizona State Universitycasanova@asu.edu Jos Contreras Domingo Universitat de Barcelona Jose.Contreras@doe.d5.ub.es Erwin Epstein (U.S.A.) Loyola University of ChicagoEepstein@luc.edu Josu Gonzlez (U.S.A.) Arizona State Universityjosue@asu.edu Rollin Kent (Mxico)Departamento de InvestigacinEducativaDIE/CINVESTAVrkent@gemtel.com.mx email@example.com Mara Beatriz Luce (Brazil)Universidad Federal de Rio Grande do SulUFRGSlucemb@orion.ufrgs.brJavier Mendoza Rojas (Mxico)Universidad Nacional Autnoma deMxicoMarcela Mollis (Argentina)Universidad de Buenos Airesmmollis@filo.uba.ar
43 of 43 firstname.lastname@example.org Humberto Muoz Garca (Mxico) Universidad Nacional Autnoma deMxicohumberto@servidor.unam.mxAngel Ignacio Prez Gmez (Spain)Universidad de Mlagaaiperez@uma.es Daniel Schugurensky (Argentina-Canad)OISE/UT, Canadadschugurensky@oise.utoronto.ca Simon Schwartzman (Brazil)Fundao Instituto Brasileiro e Geografiae Estatstica email@example.com Jurjo Torres Santom (Spain)Universidad de A Coruajurjo@udc.es Carlos Alberto Torres (U.S.A.)University of California, Los Angelestorres@gseisucla.edu