USF Libraries
USF Digital Collections

Educational policy analysis archives

MISSING IMAGE

Material Information

Title:
Educational policy analysis archives
Physical Description:
Serial
Language:
English
Creator:
Arizona State University
University of South Florida
Publisher:
Arizona State University
University of South Florida.
Place of Publication:
Tempe, Ariz
Tampa, Fla
Publication Date:

Subjects

Subjects / Keywords:
Education -- Research -- Periodicals   ( lcsh )
Genre:
non-fiction   ( marcgt )
serial   ( sobekcm )

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
usfldc doi - E11-00302
usfldc handle - e11.302
System ID:
SFS0024511:00302


This item is only available as the following downloads:


Full Text
xml version 1.0 encoding UTF-8 standalone no
mods:mods xmlns:mods http:www.loc.govmodsv3 xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govmodsv3mods-3-1.xsd
mods:relatedItem type host
mods:identifier issn 1068-2341mods:part
mods:detail volume mods:number 11issue 4series Year mods:caption 20032003Month January1Day 2929mods:originInfo mods:dateIssued iso8601 2003-01-29



PAGE 1

1 of 40 Education Policy Analysis Archives Volume 11 Number 4January 29, 2003ISSN 1068-2341 A peer-reviewed scholarly journal Editor: Gene V Glass College of Education Arizona State University Copyright 2003, the EDUCATION POLICY ANALYSIS ARCHIVES .Permission is hereby granted to copy any article if EPAA is credited and copies are not sold. EPAA is a project of the Education Policy Studies Laboratory. Articles appearing in EPAA are abstracted in the Current Index to Journals in Education by the ERIC Clearinghouse on Assessment and Evaluation and are permanently archived in Resources in Education .The Varieties of Knowledge and Skill-Based Pay Desi gn: A Comparison of Seven New Pay Systems for K-12 Teac hers Anthony Milanowski University of Wisconsin-MadisonCitation: Milanowski, A. (2003, January 29). The va rieties of knowledge and skill-based pay design: A comparison of seven new pay systems for K -12 teachers, Education Policy Analysis Archives 11 (4). Retrieved [date] from http://epaa.asu.edu/epaa /v11n4/.AbstractThis article describes the design of knowledge and skill-based pay systems for K-12 teachers in six U.S. school distri cts and one charter school. Based on a theory of action that relates kn owledge and skillbased pay systems to improvements in instruction, a nd the expectancy theory of motivation, seven dimensions for comparis on are identified and the systems are compared based on these dimensions. While there were a variety of reasons for designing new pay systems, s imilarities included that teachers were involved in the design processes and that the knowledge and skills rewarded are more closely rela ted to instruction

PAGE 2

2 of 40than in the traditional salary schedule (though non e of the systems placed heavy emphasis on content-specific pedagogy). Most systems made use of existing standards or definitions of good teachi ng, such as the Framework for Teaching (Danielson, 1996). While mos t of the systems involved performance-based assessments of teacher s kills, in no case were seniority and graduate degrees eliminated as a basis for pay progression. Few of the programs had developed a co ordinated professional development program specifically linke d to the knowledge and skills rewarded by the new pay system. Implicat ions for policy makers and system designers are drawn. IntroductionA number of lines of research (e.g. National Commis sion on Teaching and America's Future, 1996; Slavin and Fashola, 1998; Wright, Hor n, and Sanders, 1997; Bembry et al, 1998; Ferguson and Ladd, 1996) have identified teac her instructional capacity as a key variable in the success of educational reforms in i mproving student achievement. For the past two years, the CPRE Teacher Compensation Proje ct has been studying a new form of teacher compensation that may have the potential to support improvements in the capacity of teachers to deliver instruction that wo uld enable all children to achieve to high academic standards, as well as to respond to t he growing public concern that there be some link between teacher salaries and teacher p erformance. This innovation, knowledge and skill-based pay, rewards teachers wit h base pay increases and/or bonuses for acquiring and demonstrating specific knowledge and skills needed to meet educational goals, such as improving student achiev ement. The application of this pay concept to K-12 education has been suggested by Con ley and Odden (1995), Mohrman, Morhman and Odden (1996), and Odden and Kelley (199 7). This article reports on a study of seven knowledge and skill-based pay system s for teachers that have been developed by U.S. schools or districts.Knowledge and skill-based pay can be better underst ood by contrasting it with two other teacher pay systems. Unlike the traditional single salary schedule, on which teachers progress through the salary schedule based on the n umber of years of service and the additional degrees or college credits they acquire, knowledge and skill-based systems provide pay increases when teachers demonstrate, us ually though some form of performance assessment, that they have acquired and can apply classroom-relevant knowledge and skills. Ideally, pay progression is b ased on mastering a sequence or of knowledge and skills that represent higher levels o f expertise or higher levels of teaching practice. The intent of knowledge and skillbased pay is to supplement or replace the traditional schedule with a pay system that motivat es teachers to acquire and demonstrate the application of knowledge and skills that more directly contribute to better school performance and student achievement. The importance of seniority as a basis for pay is reduced or even eliminated. The other contrast is with merit pay programs. Meri t pay typically involves providing individual teachers with base pay increases by allo tting a fixed fund of money based on administrators' subjective judgments of teacher per formance during the prior year. While knowledge and skill-based pay programs also reward individual teachers, the reward is based on demonstrating knowledge and skills with re spect to public, relatively detailed standards or descriptions of practice. These standa rds both guide assessor judgments and

PAGE 3

3 of 40make known to teachers 'up front' what they need to do to demonstrate the knowledge and skills. Since any teacher who demonstrates the skills receives the reward, teachers do not compete for a share of a fixed fund or merit pay pool. These features of knowledge and skill-based pay may make it more effe ctive in motivating more teachers than merit pay.Because knowledge and skill-based pay programs are new and quite rare in the K-12 sector, it is not yet possible to obtain definitive evidence about the success of these programs in influencing instructional capacity or i n improving student achievement. This article therefore concentrates on describing a nd comparing seven pioneer knowledge and skill-based pay programs. To do so, a set of dimensions were derived from an explicit theory of action which links knowl edge and skill-based pay to improvements in instructional capacity and student achievement, and from the literature on knowledge and skillbased pay in the private se ctor.The Theory of Action for Knowledge and Skill-Based PayKnowledge and skill-based pay systems have the pote ntial to positively impact instructional capacity, and in turn student achieve ment, in three ways. First, they provide incentives for teachers to develop specific knowled ge and skills needed to increase instructional capacity. More highly skilled teacher s, in turn, have the capacity to deliver higher quality instruction, which, when combined wi th motivation to improve instruction and a context conducive to applying the skills, should lead to improved instruction. Second, by allocating higher pay to te achers who have these skills, these programs should help attract and retain high capaci ty teachers, and by denying higher pay to teachers without the skills, discourage lowe r capacity teachers from staying. Over time, the average skill level of a faculty should i ncrease, improving the average quality of instruction. Third, a well-developed knowledge a nd skill-based pay systems rests on a model of competence that can also be used in teache r evaluation, professional development, and even recruitment and selection. To the extent this model informs these human resource management functions, the organizati on communicates and reinforces a normative vision of quality instruction. This model can also be used by teachers as a guide to professional development activities, a fra mework for self-reflection and self-evaluation, and a vocabulary for the discussio n of teaching practice. Over time a shared conception of quality instruction should dev elop that supports teacher skill seeking and efforts to improve practice. This in tu rn contributes to improved student achievement. Figure 1 below summarizes this "theory of action" for knowledge and skill-based pay.

PAGE 4

4 of 40Figure 1. Theory of Action for Knowledge and SkillBased Pay The most important process by which knowledge and s kill-based pay is expected to function to improve instructional capacity is by pr oviding a pay incentive for knowledge and skill acquisition. However, simply offering tea chers a pay increase or bonus will not necessarily motivate them to acquire the needed ski lls. We have used a modified version of Expectancy Theory (Vroom, 1964) to develop a mod el to identify what a knowledge and skill based pay program needs to do in order to motivate skill acquisition (see Figure 2). Figure 2. Motivational Model for Knowledge and Skil l-Based Pay Based on Expectancy Theory This model suggests that in order for knowledge and skill-based pay to motivate effort toward skill acquisition, teachers must first belie ve that it is likely that if they put forth the effort, they can actually acquire the specified knowledge and skills. This is called the expectancy perception, and is symbolized by the arr ow running from effort to knowledge and skill acquisition in Figure 2. This perception is influenced by several factors, including the teacher's sense of self-efficacy for acquiring the skills and conditions the organization can more easily influence, including t he degree to which the teacher understands what knowledge and skills are required and how they are to be demonstrated, the perceived degree of peer and admi nistrator support for developing the skills, and the perceived availability of opportuni ties to develop the skills (such as high-quality professional development). To the exte nt that the teacher understands the skill requirements, believes that peers and adminis trators support their acquisition, and believes there are the required opportunities to de velop the skills available, s/he will be more likely to believe that if s/he tries, s/he wil l be able to acquire the skills. Teachers must also believe that there is a strong c onnection between acquiring the skills and positive consequences such as receiving the pay increase. This link is called the instrumentality perception, and it reflects the com mon-sense idea that if teachers do not believe that the reward is contingent on acquiring the skills, then the promised reward won't motivate skill-seeking. This link is represen ted by the arrow from knowledge and skill acquisition to consequences in Figure 2. In o rder for this perception to be strong, teachers must believe that the promised pay increas es will be provided when the skills are demonstrated, and will not be provided when the y are not. One set of conditions likely to support this belief include a reliable so urce of funding for the pay increases and the past performance of the organization in keeping promises to teachers. Another condition is that the methods used to assess knowle dge and skill acquisition be fair, valid, and reliable. If teachers believe that favor itism or measurement error determines how well one does on the assessment, rather than th eir true skill level, they will be less

PAGE 5

5 of 40likely to expend effort to acquire the skills. If s kill acquisition cannot be validly measured, pay increases will be less contingent on skill acquisition, and when teachers realize this, they will be less motivated to acquir e the skills. Acquiring and demonstrating the skills must also ha ve consequences teachers value. While it is safe to assume teachers value pay incre ases, these rewards also must be large enough to be perceived as worth the effort expended to acquire the specified skills. These rewards will also be more motivating if the k nowledge and skill model on which the program is based is accepted by teachers as con sistent with their conceptions of quality instruction and a highly-skilled teacher. P resumably, most teachers want to consider themselves good at what they do and are in terested in developing their skills toward their ideal of a highly-skilled teacher. The y may find this process of development intrinsically rewarding. If the knowledge and skill model is contrary to this ideal, teachers are presented with a choice: develop diffe rent skills and get more pay, or develop skills consistent with the ideal and forgo the extra pay. The extrinsic and intrinsic rewards work against each other. It is li kely that the extrinsic pay reward will have a more motivating effect if it is consistent w ith the intrinsic reward. This means that the knowledge and skill model needs to be cons istent with teachers' beliefs about what constitutes of a highly-skilled teacher. Final ly, teachers may also value avoiding certain negative consequences, such as not being re cognized as highlyskilled or expert. Avoiding these may also be motivating, especially i f the definition of "expert" is shared by school-level peers.Comparison DimensionsBased on the theory of action, the motivational mod el, and the research and practitioner literature on private sector knowledge and skill-ba sed pay systems, seven dimensions were developed to structure the analysis and compar ison of the seven cases of knowledge and skillbased pay we studied.1. Impetus Or Motivation For Developing The Knowled ge And SkillBased Pay ProgramThe theory of action assumes that policy makers cho ose to initiate these programs in order to improve instruction and in turn to improve student achievement. Alternatively, adoption of new forms of teacher compensation by pi oneer organizations may be motivated by the desire to appear innovative or by the desire of influential decision makers to implement strongly-held ideas about which teachers should be paid more. In addition, a pay system change can present an opport unity to further other agendas, such as providing additional pay for all teachers or ass uring the public that teacher pay is related to teacher performance. The motivation for moving to knowledge and skill-based pay is important, because it is likely to be relate d to design features such as the knowledge and skills in the model and the extent to which the new pay structure departs from the traditional salary schedule. One might exp ect that where the primary motivation is to improve student achievement, the k nowledge and skill model will focus on instruction, and pay increases for developing in structional skills will be greater. 2. The Design Process

PAGE 6

6 of 40The motivation model suggests that teachers' views of the fairness of various aspects of the program and their acceptance of the model of go od teaching implied by the knowledge and skills rewarded will influence their motivation to acquire the knowledge and skills. One way to promote the perceived fairne ss and acceptability of the system is to have teachers participate in its design. The pri vate sector prescriptive literature on compensation program design (e.g. Lawler, 2000; Wil son and Phalon, 1996; Ledford, 1989) has advocated such employee participation. So did Odden and Kelley (1997) for education. Participation is thought to increase the level of information employees have about the program's rationale and operation. Employ ees also have valuable information to share about what they value, how the program is likely to work in practice, and how they are likely to react to it. Participation is al so thought to increase "buyin". Because a high level of participation is likely to result in greater acceptability and perceived fairness, an important facet of this dimension is t he degree to which teachers participated in the design of the program.One form of teacher participation is through collec tive bargaining. However, it may be difficult to design a knowledge and skill-based pay program through the standard adversarial collective bargaining process of propos al and counter-proposal, with each side seeking maximum advantage. Knowledge and skill -based pay programs require a coherent design based on some agreed-upon conceptio n of good teaching. Many technical details, such as how knowledge and skills will be assessed, need to be addressed. So it is expected that these programs wo uld be designed either outside the formal contract negotiation process or though an in terest-based process (Fisher and Ury, 1981) that focuses the parties' attention on mutual goals. Another important aspect of the design process is h ow program designers decide what knowledge and skills to reward. Designers in the pr ivate sector appear to have used inductive, deductive, or adaptive approaches. The i nductive approach involves using job analysis or relying on research to identify those k nowledge and skills likely to contribute to employee performance. One version of this method is to study known good and average performers to find out what knowledge and s kills differ between these groups (Spencer and Spencer, 1993, American Compensation A ssociation, 1996). The deductive approach involves starting from the organ ization's strategy, then trying to identify the knowledge and skill employees need to carry it out (Heneman and Thomas, 1997, American Compensation Association, 1996). The adaptive approach involves starting with a knowledge and skill model developed elsewhere, then changing it to fit local goals and conditions. Though the use of the a daptive method in the private sector has been criticized because it does not provide a u nique source of competitive advantage (Ledford and Heneman, 2000; Zingheim, Ledford, and Schuster, 1996), it avoids 'reinventing the wheel', especially for those core knowledge and skills likely to be common across organizations.In the K-12 sector the core technology of instructi on is similar across schools. Since there is currently little competition among schools there is little incentive for very different specifications of knowledge and skills to be identified. There are also economies of effort to be realized by adapting work already done by recognized bodies of experts, such as the standards proposed by the I nterstate New Teacher Assessment and Support Consortium (1992), state teacher licens ing standards, the National Board for Professional Teaching Standards (1999)(NBPTS) s tandards, or the Danielson's Framework for Teaching (Danielson, 1996). Thus we m ight expect that many of these

PAGE 7

7 of 40pioneer knowledge and skill-based pay programs woul d have adapted external standards, perhaps adding locally-important skills or modifyin g language to fit local conditions, rather than attempting to develop an organization-s pecific model. This approach also allows program designers to appeal to the authority of these external experts when seeking support from teachers and the community.3. Types and Structure of Knowledge And Skills Rewa rded. At the heart of a knowledge and skill-based pay pro gram is the specification of the knowledge and skills teachers will be rewarded for developing. The theory of action assumes that the knowledge and skills specified wil l be those teachers need to deliver instruction that contributes directly to student ac hievement. Thus an important facet of this dimension is the degree to which the knowledge and skills rewarded are related to instruction.Another important facet is the extent to which the knowledge and skills rewarded are organized into an integrated model with a defined c ontinuum of skills or expertise. Knowledge and skill-based pay programs in the priva te sector often structure the knowledge and skills rewarded into a set of career levels (Jones, 1995; Daniels, 1997), levels defined by rating scales (Heneman and Thomas 1997; Gorsline, 1996; American Compensation Association, 1996) or sequences of ski lls to be mastered (Gupta et al, 1986; Jenkins et al, 1992). Odden (2000) and Odden and Kelley (1997) sketched a number of different structures of knowledge and ski lls representing a progression from entry level to accomplished teaching. Such a struct ure could provide a roadmap for teachers seeking to develop their knowledge and ski lls as well as convenient attachment points for pay increases. It could also be used to align other parts of the human resource management system, especially professional developm ent programs, and as a guide for teachers working to develop mastery of quality inst ruction. 4. How Knowledge And Skill Acquisition Is Assessed.The motivational model suggests that knowledge and skills should be assessed in a way that teachers see as fair and valid, and the theory of action implies that the assessment method must ensure teachers can apply the skills in practice. The traditional degrees and credits seem to be viewed as fair by teachers, but they may not have high validity as indicators of whether skills can be applied in the classroom. Properly constructed and administered, performancebased assessments, which function as samples of teachers' instruction, have the potential to ensure that the skills can be applied and to be perceived as valid and fair, due to their close connection wi th practice. Thus one facet of this dimension is the extent to which performance-based assessments are used, rather than degrees and credits, to provide evidence of knowled ge and skill acquisition. While private sector knowledge and skill-based pay programs typically appear to depend on relatively simple, locally-developed assessments (Heneman and Ledford, 1998), program designers in the K-12 sector have the optio n of using externally-developed assessments, such as PRAXIS III (Dwyer, 1998), the Framework for Teaching (Daniealson, 1996), or the NBPTS assessments. Thirt y-one states and more than 200 districts provide some salary incentive for certifi cation. (National Board, 2001). This avoids the expense and effort of developing local a ssessments for core teacher skills that are likely to be common across districts or schools (Milanowski, Odden, and Youngs,

PAGE 8

8 of 401998, Heneman and Ledford, 1998). External assessme nts may also have the potential for greater validity and fairness than assessment d eveloped locally, due to the greater expertise and resources of their developers. Extern al assessments could be used in combination with local assessments to maintain the rigor of the system. The teacher performance evaluation literature (e.g. Wise et al, 1984) suggests that local assessors such as principals face many incentives to be less than rigorous. If almost all teachers are judged to have the skills, due to leniency of local assessors, the contingency between skill acquisition and receiving the reward the moti vational model postulates as necessary is reduced. (The reward won't motivate effort towar d skill acquisition if the assessors certify teachers without the skills as eligible for the reward.) Odden (2000) outlined a model knowledge and skill-based pay structure that combined the use of external and local assessments. So a second facet of this dimens ion is the extent to which external and locally-developed assessments are used to provi de evidence of knowledge and skill acquisition.5. The Size and Structure of the Knowledge and Skil l Incentives. The theory of action proposes that the extra pay of fered will motivate teachers to acquire the knowledge and skills needed to improve instruct ion. To motivate, the pay incentive provided must be valued. Experience with the tradit ional salary schedule suggests that teachers value pay rewards enough to collect years of seniority, credits, and degrees. But to motivate the acquisition of the new, possibly ha rd-tomaster skills needed to improve instruction, the incentives must be of sufficient s ize to attract teachers' attention and to be perceived as commensurate with the effort needed to acquire the skills. It is reasonable to expect that the greater the size of t he incentive, the more motivational effect, all else equal. So an important facet of th is dimension is the size of the incentive offered.To the extent that knowledge and skill rewards repl ace the traditional pay increases for seniority and educational attainment, we might expe ct teachers to be more motivated to attain the skills, since the traditional opportunit ies for pay increases have been reduced. A more radical change in the pay structure, de-emph asizing seniority and educational attainment unrelated to classroom instruction, send s a stronger signal that new knowledge and skills are needed. Knowledge and skil l pay programs might be located on a continuum ranging from those that supplement the traditional salary schedule by simply adding additional pay opportunities based on knowledge and skill acquisition, to complete replacement of the traditional schedule's seniority steps and educational attainment lanes with a set of pay levels based onl y on knowledge and skill attainment. One might expect that the greater the perceived nee d to improve instruction, the more the traditional salary schedule would be modified a nd the larger the incentives for knowledge and skill acquisition would be.A knowledge and skill-based pay system carries risk s for teachers accustomed to automatic pay increases based on seniority. It may be particularly unattractive to more senior teachers because it places less emphasis on seniority as a criterion for pay differentiation, and can require developing new ski lls, which may not be as good an investment of effort for them. To get a knowledge a nd skill-based pay program accepted may require some provision that compensates teacher s in some way for the increased risk or reduces the threat that the emphasis on new skills can represent to senior teachers. Thus another feature of interest is wheth er the programs include provisions

PAGE 9

9 of 40intended to make the new system acceptable to poten tial opponents such as veteran teachers.6. Alignment of Other Human Resource Programs in Su pport of the Knowledge and Skill ModelEnsuring that the professional development programs available to teachers are aligned with the knowledge and skill model is likely to be a determinant of program success, because according to the motivational model, teache rs need to perceive that opportunities to acquire the rewarded skills are av ailable in order to believe that their efforts are likely to be successful. Private sector employers appear to take on the responsibility for providing and communicating oppo rtunities to acquire skills, to ensure availability and to show employees their efforts to acquire skills are being supported (American Compensation Association, 1996; Jenkins e t al, 1992). Thus an important of aspect of alignment is whether organizations provid e professional development opportunities linked to the knowledge and skills th eir pay systems reward. A knowledge and skill model can also provide a foun dation for other human resource management programs such as performance evaluation, recruitment, and selection (Spencer and Spencer, 1993, Schippman et al, 2000). The theory of action postulates that a human resource management program aligned with th e model will contribute to the development of a shared conception of good instruct ion consistent with the model. If the model is shared with job candidates during recruitm ent, those who do not believe that they can develop the skills or are not in agreement with the underlying philosophy of instruction may "self-select" out of the hiring pro cess. Selecting new teachers based on the knowledge and skill model helps to ensure those who are hired have the skills, or the potential to develop them. If teachers select the d istrict or school and the district or school selects teachers based on the model, converg ence on the conception of instruction it embodies should increase over time. With respect to current staff, if teacher evaluation is made consistent with the knowledge and skill mod el, this will avoid confusing teachers about what it values as good teaching, and teachers will not be faced with two unrelated assessments on which they must spend time and energy. Teachers should be more likely to use the model to guide their own pro fessional development efforts, and to absorb the model as the appropriate way to think ab out teaching, again reinforcing a share conception of instruction. Thus a second aspe ct of alignment is the extent to which the knowledge and skill model is integrated with ot her human resource management programs besides pay and professional development.7. Costs and Funding While knowledge and skill-based pay offers substant ial benefits, it is also likely to require additional investments, including the costs of increased professional development and additional administrative overhead (e.g. assessment and record-keeping) as well as of higher salaries. Priv ate sector experience with knowledge and skill-based pay programs suggests that administ rative costs increase and per-employee salary costs increase. (Gupta et al, 1 986; Jenkins et al, 1992 ). Individual pay increases are thought to be offset by increases in productivity and greater flexibility in staff utilization due to cross-training. However these offsets are less likely to appear in the K-12 sector because increased productivity, in the form of higher student achievement, typically does not allow reductions in staff nor savings in materials or

PAGE 10

10 of 40equipment. (Nor is it immediately marketable for in creased revenue.) The knowledge and skills are not those that allow teachers to do more different jobs, therefore allowing elimination of support staff. Therefore in the long run we would expect higher costs, which need to be funded by new money or reallocatio n of existing resources.Case Selection, Data, and MethodThe cases compared here include of six school distr icts and one charter school all of which had adopted some form of knowledge and skillbased pay. The cases were selected based on project researchers' knowledge of districts or schools designing and implementing these pay programs, and a survey of st ate department of education and teacher association staffs which asked them to iden tify districts with innovative pay systems. These "early adopters" are not representat ive of U.S. schools or districts, merely illustrative of the variety of knowledge and skillbased pay programs that are being developed and of the process of design and im plementation. Description and comparison is based on the programs' operation or d esign as of the 1999-200 school year. In each case, project staff visited the distr ict or school, in some cases multiple times, during the 1998-2000 period. Administrators, union officials, and in some instances, teachers were interviewed. A semi-struct ured interview protocol guided most of the interviews. Documents describing the program were also collected, and in some cases internal research done by the districts to ev aluate the programs was obtained. The researcher who visited the site wrote a case descri ption from which the information relevant to the comparison dimensions was abstracte d. The Appendix provides a brief description of each case site. Extended case descri ptions are available at www.wcer.wisc.edu/cpre. In a few cases, additional contacts were made by the author to clarify information in the case descriptions. The a uthor then summarized the features of each case along the comparison dimensions, then att empted to identify patterns and important differences, and to draw conclusions abou t the implications of the experiences of these early adopters for research and program de sign.Program ComparisonsThe similarities and differences among the seven pr ograms are presented below, structured according to the seven comparison dimens ions. Motivation For Developing The Knowledge And Skill-B ased Pay Program The varied motivations for pay system change we fou nd suggest that knowledge and skill-based pay was not simply seen as a way to imp rove student achievement through improving the skill level of current staff, as assu med by our theory of action. Though supporting improved instruction was a common goal, there were other important reasons for initiating change. It does not appear that most of these early adopters were primarily focused on using the programs increase teachers' in structional capacity in order to improve student performance. Decision-makers at mos t of the sites did not appear to have based their programs on a theory of action lik e the one described above. Table 1 summarizes the key factors in each case.Table 1 Initiation of Knowledge and Skill-Based Pay Design Process

PAGE 11

11 of 40 SitePrimary Motivation for Developing KSBP Supporting State Policy Initiatives Champion(s)Labor-Management Relations CincinnatiStrategic planning process identified KSBP as one districtstrategy for improving student achievement. State proficiency testsChange in State licensingstandards Union Bargaining Chair, Associate Superintendent,outside consultant from university, Variable; but underlain by trust relationship between key union andmanagement staff CoventryDesire to differentiate pay according to performance andsupport new model of instruction. Secondarily, concern about pay inequitiesbetween junior and senior teachers; desire to keep goodteachers in the classroom rather than moving to administration toreceive more money None directly, though new state funding formula may have providedsome of the additional funds needed. Superintendent, union president Cooperative, after period of conflict in the 70's Douglas County Part of broader compensation redesign aimed atreassuring public concerned about accountability for use of public funds State teacher licensing reforms Assistant Superintendent for HR; union president Cooperative LimonDesire to link pay with teacher performance coupledwith teacher dissatisfaction with prior individual pay for performancesystem. Desire to add group reward component linked to state assessmentresults. State financial incentives for local districts to adopt payfor performance systems; state assessment system. Superintendent initially, then teachers Basically cooperative, due to new superintendent andrelative weakness of union in "right to work" state. ManitowocDesire to provide incentives for teachers to developtheir skills in order to implement more constructivist instruction; improveteacher retention; support union-initiated professionaldevelopment program State licensing reforms Superintendent initially; joined by regional unionrepresentative Cooperative; new leaders changed previous moreadversarial relationship

PAGE 12

12 of 40 RobbinsdaleConcerns with recruiting and retaining qualityteachers in competition with other local districts; concern about payinequities between junior and senior teachers. None directly.Union presidentCooperative, based on stable district and unionleadership VaughnPart of broader compensation redesign aimed atimproving recruitment and retention of good teachers, desire to add teacheraccountability to external accountability provided by charter,and to address perceived pay inequities between junior and seniorteachers. Charter status and performance pressure from explicit chartergoals Principal, initially, then junior teacher and several moresenior staff No union; relatively high level of trust betweenteachers and administrators based on commitment to charter,participatory governance structure, and charisma of principalIn Cincinnati and Vaughn, cases where external acco untability pressures emphasized the need to improve student achievement, the primary ra tionale for the programs was not expressed in terms of remedying a knowledge or skil l deficit among current staff. Rather, the programs seemed to be intended to motiv ate staff to change practice and to reward more accomplished teaching. In Cincinnati, t his was supplemented by dissatisfaction with the current teacher evaluation system and changes in the state licensing system. At Vaughn, the recruitment and re tention of highly-skilled teachers was an important additional aim. In Robbinsdale, re cognizing and rewarding accomplished teachers, and recruitment of skilled t eachers, appear to have been the major goals. In Coventry, program designers wanted to differentiate teacher pay according to performance and to keep good teachers in the classroom as well as to support a particular vision of quality instruction. In Manitowoc, the superintendent's vision of quality instruction and his desire to pro vide incentives for teacher learning were joined by the union leadership's interest in i mproving pay while staying under state-imposed expenditure limits and supporting a p rofessional development initiative developed by the state teachers' association. Both the superintendent and the association leadership wanted to improve retention, and begin a dapting the pay system to state licensing changes. In Douglas County, knowledge and skill-based pay came about as part of a pay system redesign primarily intended to respond to public pressure to link teacher pay and teacher performance, in order to im prove accountability for the use of public funds. Limon, the current program replaced o ne in which pay increases were based on individual teacher evaluations. Program de signers there found a way to respond to public interest in linking pay to performance an d teachers' concerns about unfairness of the old system by rewarding both individual prof essional development and meeting building and grade-level student achievement goals. Perceived inequities in the traditional salary schedule between younger, high p erforming teachers and more senior

PAGE 13

13 of 40 teachers contributed to motivating pay system chang e in Coventry, Robbinsdale and Vaughn. This is interesting given that one advantag e often cited for the traditional salary schedule is that teachers perceive it to be highly equitable (Odden and Kelley, 1997). State policy, while not a primary driver of pay sys tem change, was an important background condition in many of the cases. Teacher licensing policy provided a reinforcement for change in Cincinnati and Manitowo c. State incentives may have provided motivation for the Limon's initial experim entation with non-traditional pay systems, and for the district to continue to includ e teacher and student performance elements in its current plan. But while all of the sites were in states with some form of student testing and accountability program, only th e two sites with relatively low student achievement (Cincinnati and Vaughn) felt much press ure from these programs. In the others, student achievement was either relatively h igh or not a major issue in the other communities.While in all of the cases, a champion or set of cha mpions was important in keeping it going to a successful conclusion, in three the cham pion's own agenda was a key impetus to initiating change. In Coventry and Manitowoc, th e programs were initiated partly to pursue the superintendent's personal vision of good instruction, though in Coventry the union president actually got discussion going by pr oposing rewards for National Board certification. In Robbinsdale, the former union pre sident initiated discussions with the district based on his desire to ensure that new tea chers who fit his conception of a good teacher would be available when it became necessary to replace retirees. At Vaughn, the principal began to explore pay innovations to stren gthen teachers' sense of accountability for student performance by adding individual stakes to the overall external accountability provided in the charter. It is inter esting that in four of the seven cases, a union official was one of the champions, and in a f ifth, a Uniserve representative was a key catalyst of innovation. This suggests that teac hers' unions can be supportive of changing the traditional salary structure. Where un ion and management relations are good, and a hightrust relationship exits between union leaders and at least some management leaders, it appears that teacher compens ation innovation can be successfully initiated.Process Used To Design The ProgramTable 2 summarizes three key aspects of the design process at these sites: the type and level of teacher participation in design, the relat ionship to the collective bargaining process, and the methods used to identify the knowl edge and skills to be rewarded.Table 2 Characteristics of Knowledge and Skill-based Pay De sign ProcessSiteRelationship to the Collective Bargaining Process Type/Level of Teacher Participation Method of Knowledge & Skill Identification CincinnatiCommitment to develop plan agreed to in contract. Design process tookplace outside normal collective bargaining though system ofunion-management committees. Steering committee jointly chaired by 24 teachers from a variety of schools participated on thevarious committees. Adaptation of standards for teacher performance found in the Frameworkfor Teaching (Danielson, 1996)

PAGE 14

14 of 40 union and management representatives. Teachers voted toapprove contract including the results, and have opportunity to voteout system before 9/2002 implementation. Coventry Worked out as part of interest-based bargaining process Limited to members of bargaining team, though rank and file teachersparticipated in design of the teacher evaluationsystem incorporated into the plan. Adaptation of NBPTS standards for one pay incentive; inductiveprocess drew from best practice literature (including work of T.Sizer and F. Newman) & district action research for the other Douglas County Commitment to develop plan agreed to in contract. Design process tookplace outside normal bargaining via a 30 member performance paycommittee. Performance pay committee included 20 teachers from a crosssection of union members Deductive and inductive processes used to developskill blocks; adaptation of NBPTS and Colorado licensing standards forthe outstanding teacher award. LimonNo formal contract. Program concept developed by superintendent andteacher representatives as part of informal negotiations. School Boardpassed proposal & teachers' association agreed to try proposal. 3 teachers worked with Supt. to develop concept; detail designdone by a committee with 7 teachers and oneadministrator. Left to teacher and building administrator, based on district-providedguidelines. ManitowocWorked out as part of interest-based bargaining process and approved aspart of teacher contract. 8 association bargaining team members participated as part ofbargaining process Inductive, based on education research; adoption of NBPTSstandards. RobbinsdaleInitial concept and outline of skill areas and pay levels agreed to inbargaining. Detail design by set of unionmanagement committees. A few teachers participated as members of the bargaining team.More teachers participated as members of each of 8 committeesresponsible for defining skill levels and methods of assessment. Adoption of NBPTS standards; induction from research and experience. VaughnNo collective bargaining.Design by small grou p of teachers and administrators, thenextensive discussions via informational meetings in committees of governance structure. Primarily deductive from educational goals incharter, with some adaptation of the Framework for Teaching for the rubricsTeacher participation Large scale teacher participation in the design p rocess was present in the three largest organizations (Cincinn ati, Douglas County, and Robbinsdale). In these districts a formal committee process was used to involve a substantial number of teachers in some aspect of sy stem design. These cases suggest that broad teacher involvement can have a substantial in fluence on the content of the plan. For example, in Cincinnati, teachers on the committ ees probed for ambiguities in the

PAGE 15

15 of 40design proposals and contributed specific ideas for design, as well as pushed for provisions to reassure teachers about fairness. The input of National Board certified teachers was influential in persuading other teache rs on the committee to take the risk of trying a new system of evaluation and pay. Of the s maller organizations, Limon provided for relatively extensive participation giv en the size of the district, while in Coventry and Manitowoc teacher participation appear ed limited to the association bargaining team. A relatively small group developed the Vaughn plan, though it was adopted by a vote of the governance committee on wh ich teachers were heavily represented and after considerable formal and infor mal discussion among the faculty. However, involving a substantial number of teachers does not guarantee broad communication. Outside evaluations of both Douglas County (Hall and Caffellera, 1997) and Cincinnati (Milanowski and Kellor, 2000b) sugge st that many teachers who had not been active participants in the design process did not seem informed on some aspects of the systems. In Cincinnati, the large size of the d istrict and the complexity of the program seemed to require more intensive district o r association-sponsored communications efforts than were initially undertak en. Relationship to collective bargaining As expected, none of the programs were developed though traditional adversarial collective bargaining. It was also expected that knowledge and skill-based pay programs would be too complex and timeconsuming to work out in detail through the normal negotiation p rocess. However, in three of the cases, the details of the process were worked out w ithin the negotiation process. It appears that pay changes of substantial complexity can be developed in the bargaining process, as long as the parties have achieved a hig h level of trust and focus on a vision or desired goal shared by both sides. Where the progra ms' details were bargained, a clear sense emerges from the case studies that the shared vision or goal was an important influence in keeping the discussions from getting s idetracked by issues of who gains and who loses from particular details of the program.Knowledge and skill identification There was no one method of knowledge and skill identification that dominated in these cases. As ex pected, many programs made use of existing standards or definitions of good teaching. In five of the seven cases, an external set of teacher standards, either the NBPTS or the F ramework for Teaching (Danielson, 1996) were influential. Cincinnati is the clearest example of adaptation. Starting with the Framework for Teaching, the design committee examin ed each component and revised wording to fit the district context. Adapting the F ramework for teaching allowed the district to design a system in a relatively short t ime. In contrast, Robbinsdale, though beginning with NBPTS standards and making Board cer tification a major determinant of knowledge and skill-based pay increases, had not be en able to implement its system in the school year intended in part because of the dif ficulty in defining the key indicators of knowledge and skill in the parts of its system not related to the NBPTS standards. In Coventry, the Framework for Teaching is the basis f or the teacher evaluation system, but the pay incentives are based on separate standards. One provision is based on the NBPTS certification, and the other on locally-devel oped criteria with content that differs from both Framework and the Board's standards. The programs in Douglas County, Limon, Manitowoc, and Vaughn were not primarily bas ed on existing external standards, though Vaughn did adapt the format of th e Framework for Teaching's rubrics. The strengths and weaknesses of the deductive appro ach are illustrated by the Vaughn case. The knowledge and skills developed were close ly tied to school goals, so they had the potential to focus all teachers on key skills. However, the process of identifying the

PAGE 16

16 of 40 skills and the standards for measuring them was dif ficult and time consuming. As a consequence, in the first year the criteria for kno wledge and skill demonstration were not well specified, causing many of the initial partici pants to have concerns about fairness of application (Milanowski and Kellor, 1999). These ex periences suggest that adapting an existing model of teacher practice may be the most efficient way to get a knowledge and skill-based pay system up and running.Knowledge and Skills Rewarded and Their Organizatio n into a Structure Table 3 summarizes the knowledge and skills rewarde d in the cases, and how (if at all) the knowledge and skills were organized into some f orm of developmental sequence or set of performance levels. As expected all programs rewarded knowledge and skills relevant to instruction, especially pedagogical ski lls. There are differences in emphasis, however. Coventry, Douglas County's skillblocks, and Manitowoc appeared to be trying to promote constructivist or "authentic" ins truction, while Cincinnati and Vaughn were concerned with a more generic model of good te aching, though with some constructivist elements. Limon allowed the teacher and/or building administrator to determine what sort of instructional skills should be developed within broad district guidelines. Robbinsdale had not yet worked out its model in detail at the time of our study. None of the programs appeared to emphasize m astery of content-specific pedagogy (Shulman,1987, National Commission on Teac hing and America's Future, 1996), except as embodied in NBPTS certification, t hough some of Cincinnati's and Coventry's locally-developed rubrics referenced it and some of Vaughn's represented basic aspects of it.Table 3 Knowledge and Skills Rewarded—Content and StructureSiteKnowledge and Skill Domains IdentifiedDevelopme ntal Levels of Knowledge and Skills CincinnatiThree part system: 1) 4 core domains: pla nning and preparing for student learning, creating anenvironment for learning, teaching for learning,and professionalism; specific behavioralstandards in each domain with rubrics describing 4 levels of performance on each standard 2)content knowledge; 3) NBPTS standards(a). Core of system had five developmental or career levels; aggregation of rubricscores on standards define career level. CoventryTwo separate pay provisions. One used NBPTS standards(a); The other (RHODE program) covered authentic pedagogy (instruction andassessment), self-reflection, differentiatinginstruction, family and community involvement, and professional development. Two separate programs with limited overlap; they did not represent adevelopmental sequence, though RHODE could be useful in preparing forNBPTS certification. Douglas County Two part system: 1) 9 skill blocks coveringtechnology, authentic assessment, and diversity; 2) Outstanding Teacher award with options using:a) NBPTS standards; b) standards-basedinstruction; or c) assessment and instruction, content and pedagogy, and collaboration.Standards for Outstanding Teacher were a mixture of knowledge and skill descriptions and Two separate programs with limited overlap; some of the skill blocksrepresent developmental sequences.

PAGE 17

17 of 40 descriptions of behavior. LimonProgram did not specify knowledge and skills t o be sought, leaving this up to teacher &administrator based on general guidelines thatemphasized the need to focus on instruction and student learning. Unstructured; content and sequence open to development by teacher andadministrator ManitowocThree aspects of system: 1) district-devel oped courses covering authentic instruction, technology, writing instruction; 2) NBPTSstandards (a); 3) content knowledge representedby degrees and credits. Knowledge and skills primarily defined in terms of courses andcertifications Program components are independent ; no developmental levels identified attime of study, though district courses andProfessional Development Certificate could prepare teacher for Boardcertification. RobbinsdaleNBPTS standards (a), content knowledge, classroom teaching, program/curriculum design,district and school leadership, parental/studentsatisfaction. Knowledge and skills defined mostly in terms of indicators such as NBPTS certificationand documentable teacher accomplishments. Program had ten independent elements that are evaluated, and the evaluationsaggregated to produce a pay level. Thedifferent elements represented multiple ways to define good teaching rather thana developmental sequence. VaughnCore system based on 11 locally-defined domai ns of skill in lesson planning and classroommanagement, literacy, language development,technology, special education inclusion, mathematics, history and social science, andscience pedagogy, instruction in primary languagefor English learners, arts. Additional knowledge and skills rewarded defined by NBPTS standards(a), Masters' degree, state licensure level. Core of system defined 3 levels for additional pay: level 1 based onachieving an average rubric score of 2.5 in 6 of the "essential" domains; level 2required an average of 3 in those domains, then provides additional payfor rubric score of 3 in any of 5additional domains; level 3 based on achieving an average rubric score of 3.5in all domains.Note: (a) The content of the National Board standards varies by subject and level among the 30+ certifications offered; however, almost all standards include the domains of knowledge of students, knowledge of subject, knowle dge of pedagogy, creating a learning environment, use of a variety of assessmen t methods, reflection on practice, and collaboration with parents and colleagues.Most of the programs are eclectic in the way they s pecify what knowledge and skills rewarded. While most of the rewards in the Cincinna ti and Vaughn programs are based on developing knowledge and skills that are describ ed in terms of teaching behaviors or skilled performance, there is some reward provided for degrees or certifications analogous to the credits in the traditional schedul e. Another set of programs, Douglas County, Coventry, and Robbinsdale, mixed external c ertifications with more or less detailed descriptions of desired performance or beh avior. Limon provided relatively little guidance, leaving the teacher and administra tor wide leeway as to the knowledge and skills to be developed. The Manitowoc program s pecified its knowledge and skills in terms of courses and certifications, analogous t o degrees and credits, rather than describing behaviors or skills. In only a few of the programs, most notably those o f Cincinnati and Vaughn, did the knowledge and skills specified approach the ideal o f an integrated developmental sequence or structure of levels. The other programs had not organized the knowledge and skills into a core set of standards, nor provid ed a continuum of skill development that unified the knowledge and skill domains along a developmental path or career

PAGE 18

18 of 40 progression. Even the Cincinnati and Vaughn program s did not appear to integrate their local standards with the National Board standards, treating Board certification as an additional credential like a Masters' degree rather than as another developmental level. Though the programs in Coventry and Manitowoc were informed by a coherent vision of instruction on the part of their original champions these programs did not include a developmental progression linked with pay increases at the time we studied them. How Knowledge and Skill Acquisition Was AssessedTable 4 summarizes the assessment methods used in e ach of the seven programs, including the use of external assessments. All use some form of performance assessment, though the extent to which these assess ments are central to the program varies. The Cincinnati and Vaughn assessment system s were primarily based on demonstrating knowledge and skills via classroom pe rformance and are part of the regular teacher evaluation. Programs that use Natio nal Board certification as a criteria for pay increases (Manitowoc, Robbinsdale, and Cove ntry) incorporated the performance emphasis of the Board's assessments. Co ventry also used a performance-based approach in its local assessments The assessments at the end of Douglas County's skill blocks are performance-based though performance in training is not always the same as classroom performance. The g uidelines for the Limon process emphasized connecting the professional development documented in the portfolio to classroom practice and student learning, though it is up to the teacher and administrator to implement these guidelines. One of Robbinsdale's performance dimensions involved principal evaluation via classroom observation, and several others are based on real-world accomplishments rather than degrees or c redits. The observations were to be part of the regular teacher evaluation process, and the results one element in a teacher portfolio that documents knowledge and skill. Manit owoc's program mostly relied on indirect evidence like certifications and course at tendance, more analogous to the traditional degrees and credits. Performance assess ment was incorporated mostly though the incentive provided for National Board certifica tion.Table 4 Methods of Assessment SiteLocally-Developed Assessments UsedExternal Asse ssments Used CincinnatiInternal assessment of performance in the four primary knowledge and skill domains by site administrators and peer evaluators with subjectexpertise. Types of evidence: 6 classroomobservations; portfolio including artifacts such as lesson plans, student work, parent contact logs,professional development logs. NBPTS assessment, degree completion, and licensure will be used to determineeligibility for additional pay elements. CoventryInternal assessment based on a portfolio pr epared by teacher used for the RHODE program.Portfolio including evidence that/of: teachersknow students, have prepared for and practiceddifferentiated learning, ability to motivate and support all students, family and communitycontact, and professional development. Also self-analysis of teaching and assessment of student work. NBPTS assessment used for pay increment for NBPTS certification.

PAGE 19

19 of 40 RHODE: 9 element portfolio evaluated using rubrics. Each element scored for evidence ofpresence of 5 behaviors or outcomes each worth 1 point. Total score of 43 points qualifies for award Douglas County Performance-based assessment at end of each skill block done by course instructors. No specificrubrics or standards defined outstanding teacher;review of portfolio prepared for outstandingteacher award done by administrators. NA LimonPortfolio documenting activities toward fulfil ling professional growth goal evaluated by administrators. No specific rubrics or standards toevaluate skill acquisition NA ManitowocMixture of external (NBPTS, Professional Development Certificate, degrees) and internal(local teacher-taught courses) opportunities.Standards or rubrics used to evaluate skill acquisition depended on course or certification. Grades/degrees/certifications from higher education. NBPTS assessmentused for pay element rewarding NBPTScertification. RobbinsdaleDocumentation of achievements via portfo lio, classroom observations; student/parent surveys.Evidence evaluated by committee consisting of 3appointees of superintendent, 3 appointees ofunion president. Specific rubrics/guidelines remained to be developed for most domains. NBPTS assessment used for pay element rewarding NBPTS certification. VaughnClassroom observations, artifacts such as les son plans and student work evaluated by an administrator, grade-level peer, and self.Four-level rubrics used specific behavioral examples to define levels of performance in eachdomain. Grades/degrees/certifications from higher education used for pay elementsrewarding credentialing. NBPTS assessment used for pay increment forNBPTS certification.Use of external assessments Five of the seven programs included the NBPTS assessments. However, these assessments were not ty pically integrated with the local assessment system, nor used as measure of core teac hing skills. In most cases, the NBPTS assessment was included because of pay incent ives for NBPTS certification, which in turn was treated as an additional degree, rather than as an integral part of the knowledge and skill model. The exceptions are Coven try and Robbinsdale. In Coventry, while the domains measured by the local and Board a ssessments differ, the processes are similar, with the local process designed to help te achers prepare for the Board assessment. In Robbinsdale, the NBPTS assessment wa s the criterion for a sizable part of the knowledge and skill pay incentive, and seeme d to form the conceptual anchor for the program, but there were several other locally-a ssessed ways for teachers to demonstrate knowledge and skill in order to increas e their pay. Validity and reliability According to the theory of action, knowledge and skill-based pay programs require methods of assessment that are valid and reliable, and recognized as such by teachers. At the time of our studies, li ttle information was available about the validity or reliability of the assessments in any o f the programs. From the information available, it appears that the most common external assessments, those of the NBPTS, have at least as much reliability and validity as m any accepted human resource selection and evaluation techniques (Milanowski, Odden, and Y oungs, 1998; Jaeger, 1998). Only Cincinnati and Vaughn appeared to have confronted t hese issues with respect to their

PAGE 20

20 of 40locally-developed assessments. Both have looked at the inter-rater agreement of their internal assessment systems and have been relativel y satisfied that an acceptable level of agreement exists. At this point, no district has lo oked at the relationship between its assessments and independent measures of teacher per formance, such as student achievement, though at least two were planning to e xplore this connection. Information on teacher perceptions of the fairness of these assessments was available for three cases. In Cincinnati, teachers participating in the field test of the assessment system generally believed that the results of the p rocess were fair, but many had concerns about the fairness of the process, especia lly with respect to administrator qualifications and the consistency of rating across administrators (Milanowski and Kellor, 2000b). At Vaughn, in the initial year, man y of the teachers covered by the system had concerns about the consistency of the ev aluations and the difficulties evaluators had in making the requisite number of cl assroom observations (Milanowski and Kellor, 1999). After the rubrics were more full y developed and problems with scheduling of observations addressed in the second year, fairness perceptions improved substantially (Milanowski and Kellor, 2000a). In Do uglas County, while fairness perceptions were not explicitly assessed, Hall and Caffarella (1997) did interview and survey teachers about their reactions to the progra m. Teachers did not identify fairness concerns as a major issue, though several teachers did mention problems such as subjectivity in the evaluation of the teacher portf olios. It may be that the lack of expressed fairness concerns was due to the lower st akes of the assessments for individual teachers (due to the relatively small do llar amounts associated with the skill blocks and Outstanding Teacher award) the fact that individual teachers could choose to participate or not.Size and Structure of Knowledge and Skill Incentive s Table 5 describes the programs' knowledge and skill -based pay structures. The programs can be roughly categorized as falling into three gr oups: those that have essentially replaced the traditional schedule (Cincinnati, Limo n), those that have modified the schedule, typically by reducing the importance of d egrees and credits in exchange for more performance-oriented representations of knowle dge and skill (Vaughn, Robbinsdale), and those that have supplemented the traditional design by adding knowledge and skill-based elements (Coventry, Dougl as County, Manitowoc). We expected that organizations feeling more pressure t o improve student achievement would be more likely to modify or replace the tradi tional schedule, but the association is not that strong. Cincinnati and Vaughn fit the patt ern of organizations under pressure implementing major pay change, and Coventry, Dougla s County and Manitowoc fit the pattern of less pressure and more incremental chang e. Limon was an exception, in that it replaced the traditional schedule completely, but f or reasons other than providing a stronger incentive for knowledge and skill acquisit ion. Robbinsdale was another exception, with a relatively major pay system chang e but little pressure to improve student achievement. It should be noted that the po tential impact of the more radical changes represented by the Robbinsdale and Vaughn p ay systems were offset by limitations on who is covered by the new system. To allay apprehension on the part of senior teachers socialized to the traditional syste m, these two programs required that only new or less senior teachers participate, leavi ng others on the traditional schedule.Table 5

PAGE 21

21 of 40 Knowledge and Skill Pay Provisions and Relative Siz e of IncentiveSitePay ProvisionsKSBP Incentive as % of Beginning Base and Maximum Salary Provisions to Win TeacherAcceptance CincinnatiCore program: 5 career levels of teaching practice with salary ranges of $30,000,32,000-35,750, 38,750-49,250, 52,50055,000,and 60,000-62,500; movement between levels based on knowledge & skill assessment,movement within levels through a limited number of steps based on seniority; additionalbase pay add-ons of $4,600 for Masters degree,9,375 for Ph.D.; $1,250 for dual certification, $1,000 for NBPTS certification, up to $4,000(time-limited) for skill blocks. Base: 60.8% Maximum: 23.4% High seniority teachers (22 years and up) can remain on old salaryschedule or can volunteer to participate in newsystem. Coventry$6,500 add-on for life of NBPTS certificati on; $1,000 per year for four years based onachieving a cut-off score on a locally-assessedportfolio (RHODE program). Base: 22.8% (19.8% for NBPTS, 3% for RHODE) Maximum: 9.7% (8.4% for NBPTS, 1.3% for RHODE Improved district contribution to teacher retirement plan; earlyretirement option. Douglas County $300500 bonuses per skill block for 9 blocks; $1000 annual bonus for being designated anoutstanding teacher. Base: 17.4% Maximum: 6.9% 3% across the board pay increase; knowledge and skill part of planvoluntary LimonEntry pay based on a traditional seniority and credits schedule, but after entry progression based on an across the board increase, $1,000for a Master's and $3,000 for a Ph.D., plus up to$1,200 in performance based increases, $400 of which is based on meeting individualprofessional development goals. Base: 1.5% Maximum: 1.0% Cost of living adjustment added to pay system,rectification of base pay inequities between new hires and more seniorteachers. ManitowocExpanded traditional salary schedule to pr ovide more lanes and allow movement between lanesbased on locally-developed courses andclassroom-relevant university certification aligned to NBPTS, as well as traditional creditsand degrees; 13% salary add-ons for NBPTScertification and Ph.D. degree. Seniority movement within a lane capped at lower paylevels to encourage obtaining advanced degreesand other recognized professional development. Base: 13% NBPTS NA other parts Maximum: 13% NBPTS NA other parts Improved funding of retiree health insurancepremiums; new payelements not covered by cost controls RobbinsdaleTraditional salary schedule modified by reducing number of lanes from 11 to 4 and stepsfrom 13 to 7. Knowledge and skill-basedcomponent provided for additional pay of up to $15,000, with the actual amount based onpoints earned in 10 categories: NBPTS Base: 56.1% Maximum: 23.8% To be applied to newly-hired teachers orvolunteers

PAGE 22

22 of 40 certification, principal evaluation, individual accomplishments, district projects, contributionto teams, content knowledge, professionalleadership, and customer satisfaction. VaughnOne 11-step seniority-based lane, $1,000 add-on for California teaching credential,$2,000 add-on for Masters degree, $2,000add-on for qualifying as demonstrator for student teachers, $4,000 add-on for NBPTScertification. Three levels of competency-basedpay add-ons (up to $13,100) earned by achieving a minimum score or better on rubricsin ten areas: Lesson planning and classroommanagement, literacy, language development, technology, special education inclusion,mathematics, history and social science, andscience pedagogy, instruction in primary language for English learners, arts. Base: 48.7% Maximum: 22.7% Applied only to newly-hired teachers or volunteers from amongveteran teachersIn all of the cases, some sort of quid quo pro or p rovision was added to sell the program, especially to highly senior teachers. It is interes ting that in two of the cases, the consideration was relatively small. In Manitowoc, i t was an additional district contribution toward retiree health insurance premiu ms. In Cincinnati, it was the exemption of a relatively small number of very seni or teachers who would be likely to retire soon after the pay provisions took effect. I n four cases (Robbinsdale, Vaughn, Douglas County, and Coventry) participation in the knowledge and skill-based pay part of the system was voluntary for all or senior teach ers (though at Vaughn most of the senior teachers opted into the system in the second year). In Coventry, the district also increased its contribution to the teacher retiremen t plan and provided an early retirement option. In Limon, a small cost of living adjustment was added to pay system, and the school board corrected some base pay inequities tha t had emerged between new hires and more senior teachers. These experiences suggest that the potential opposition of senior teachers was an important issue to program d esigners. However, limiting the program to new teachers or volunteers may dilute th e impact of the program on motivating improvements in instructional capacity. At Vaughn, the hope of the program's designers was that experienced teachers w ould volunteer to participate, and many did in the second year. In Robbinsdale, this w as not a major concern because the primary impetus for designing the system was not to improve the skills of current teachers. Both base pay increases and bonuses were used to re ward knowledge and skill acquisition. Cincinnati, Coventry, Limon, Manitowoc and Robbinsdale relied primarily on base pay increases, while the Vaughn and Douglas County knowledge and skill-based pay programs used bonuses. In some of the cases, ho wever, some or all of the base pay increases were time limited "add-ons": the pay incr ease continued only for a fixed period, after which knowledge and skills had to be re-demonstrated. The programs that rewarded NBPTS certification provided the extra pay for the 10 year life of the certification. Increases based on locallyassessed knowledge and skills were provided for four to five years in three cases. At Vaughn, t he base/bonus distinction was blurred because the bonuses are pro-rated and the extra pay is included as an add-on to the monthly base. This provided continuity of income fo r teachers, though the extra money needs to be re-earned every year.

PAGE 23

23 of 40 Though the pay systems in these cases were diverse, one notable similarity across them was that six of seven retained a seniority-based el ement. (Limon eliminated seniority as a basis for progression after entry.) In this regar d, most of these programs differed from private sector implementations of the knowledge and skill pay concept, which typically eliminate seniority as a basis for pay (Jenkins et al, 1992). However, in five of the cases where seniority increases remained, the new pay sys tem decreased the emphasis on seniority by reducing the number of seniority steps or capping seniority-based pay progression at a lower level. Another similarity wa s that all of the systems continue to reward Master's degrees, suggesting that the K-12 s ector continues to value higher educational attainment, and that moving away from t his traditional valuation may be too radical a change to be accepted by teachers. It sho uld be noted, however, that Cincinnati planned to limit pay increases for Master's degrees to those relevant to the teaching assignment.The motivation model suggests that, all else equal, more substantial incentives will be more effective in motivating knowledge and skill ac quisition. One way to assess the size of the incentive is to compare it to the entry-leve l salary rate and to the maximum salary pay a teacher can earn in a school or district. Tab le 5 contains estimates of the magnitude of the knowledge and skill incentive in t he form of the percentage available for knowledge and skill-based elements (beyond thos e recognized in the traditional salary structure) as a percentage of the beginning base pay and as a percentage the highest pay rate available (including the knowledge and skill-based incentive, but exclusive of pay for additional activities like coa ching). Again, significant variation existed, but it is clear that in three cases, Cinci nnati, Robbinsdale, and Vaughn, the knowledge and skill incentive was substantial. The incentives provided by these three, as a percent of beginning pay, were on the order of th ose reported for private sector plans, which provide for 50 to 100% increases based on kno wledge and skills (Gupta et al, 1986, Jenkins et al, 1992; Tucker and Cofsky, 1994. ) The expectation that a larger incentive would be found where the motivation for i mplementation was to improve student achievement was partially fulfilled, in tha t some of the largest incentives were provided by Cincinnati and Vaughn. However, recruit ing good teachers was the primary motivation in Robbinsdale, the other organization w ith a large incentive. How the Acquisition of the Knowledge and Skills are Supported Table 6 summarizes the professional development ass ociated with the knowledge and skill-based pay programs, and the links between kno wledge and skill-based pay and other aspects of the human resource management syst em.Table 6 Integration with Professional Development and Other Human Resource Management ProgramsSiteDistrict Support for Acquiring the Knowledge & Skills Needed Relationship to Other HR Programs CincinnatiWhile the district had an extensive professional development program coveringmany of the skills relevant to the teaching The knowledge and skill assessment system is same as used for teacher performanceevaluation. At the time of our study there

PAGE 24

24 of 40 standards, there was no explicit linkage between the program and the standards thatwould allow teachers to determine which courses applied to each standard. District hadnew teacher mentoring and peer reviewprograms that were being converted to use the teacher standards. were no links to teacher recruitment and selection. CoventryMajor changes to the professional development program were made to supportimproved instruction, but this was done before the knowledge and skill-based pay programwas developed. Several courses have beendeveloped to address procedural aspects of the knowledge and skill-based pay system, butotherwise there does not appear to be muchexplicit linkage between professional development and the pay program. A modified version of the Framework for Teaching was used for teacher evaluation.This was also provided to job candidates aspart of the recruitment process, and some interview questions are based onFramework elements. Though the district regarded the Framework as consistent withthe NBPTS standards and the RHODE program, there was no formal link to theknowledge and skill-based pay programs. Douglas County Courses for skill blocks are provided by thedistrict. Although the district offers a substantial number of other professionaldevelopment classes, none are directly linked to the outstanding teacher program.Completing the portfolio itself was considereda form of professional development. Originally, a connection with the state's multiple level licensing system wasplanned, but delays and changes in the stateprogram prevented this development. Teacher evaluation was connected toregular pay progression, but the only explicit link to the knowledge andskill-based pay program was that teachers rated unsatisfactory cannot apply for theoutstanding teacher award. LimonThe overall professional development program was expanded at the same time the new payprogram was developed. The nature of theindividual professional growth goals left the choice of development activities to the teacherand supervisor, subject to general districtguidelines. Initial program tied pay to teacher evaluation system; current program nolonger has the direct tie. The overall pay forperformance system was explained to job candidates during recruitment;administrators felt this led to self-screening and higher retention. ManitowocThe knowledge and skill-based pay program was directly linked to a specified universityprofessional development program and tolocally-developed and provided courses. Existing local courses covered instruction,technology, and student writing. No specific links between this program and other HR programs had yet been developed. RobbinsdaleDue to delays in implementing the progra m, no specific skill-based pay program had beendeveloped at the time of our study. Due to delays in developing the system, connections with other HR systems were notbeen developed at the time of our study.Regular teacher evaluations were one element in teacher portfolio documentingknowledge and skill. Initially, the knowledge and skillbased pay programwas used as a recruitment tool, but this ceased with the delay in implementation. VaughnSome formal professional development provided around literacy and classroommanagement, provision of mentors for newteachers or those having difficulties meeting basic standards, and using coaching tied to theassessment process as part of skill building. The knowledge and skill-based pay assessment standards were also used forteacher evaluation. The pay system wasexplained to job candidates as part of job interview. Administrators felt this was arecruiting advantage since most newer teachers could earn more at Vaughn than inthe surrounding district.

PAGE 25

25 of 40At the time of our study, it did not appear that ma ny of the programs had a strong professional development component specifically des igned to provide teachers with the knowledge and skills rewarded. Those programs with larger incentives and more radical structures had not yet developed a corresponding co mprehensive professional development programs. Vaughn had only begun to deve lop a comprehensive professional development program linked to the spec ific skills in the model. Formal professional development was provided on some domai ns, including literacy and classroom management, and mentoring and teaming wer e beginning to be used to help develop skills in the program domains. Cincinnati, though it had a comprehensive knowledge and skill model, had not yet modified the ir fairly extensive professional development program to link up with it. Robbinsdale 's model was not yet fully fleshed out, but the diversity of the elements rewarded (ra nging from principal evaluation to professional leadership and parent satisfaction) ma y make it difficult to identify specific skills and develop a coherent professional developm ent program linked to the pay system. Those programs with tight links had smaller incentives and made less radical changes in the pay schedule. Manitowoc and Douglas County had the tightest links, in that parts of their pay program were directly tied to taking specific courses. However, both had not yet developed more than a relatively f ew courses covering a limited range of skills, and other professional development oppor tunities were not yet aligned with a comprehensive knowledge and skill model. The strate gy of these two districts appeared to be to start small, paying first for attaining a few important skills. Coventry did not appear to link the professional development program to the pay program, perhaps due to the limited scope of the latter. The Limon program, and Douglas County's outstanding teacher award, gave teachers considerable choice as to what skills would be rewarded and therefore did not provide the basis for a compr ehensive, linked professional development program.At the point at which we studied these programs, th e pay systems were not closely integrated with other human resource management act ivities. Two of the programs with the largest incentives, Cincinnati and Vaughn combi ned knowledge and skill assessment with teacher evaluation. Robbinsdale integrated the evaluation system by using it as one of eight elements in its assessment system. Vaughn used the knowledge and skill pay system in recruiting teachers, and Robbinsdale had planned to, but since the system was insufficiently developed, had not done so at the ti me of our study. None of the organizations appear to have used the knowledge and skill model in selecting teacher candidates at the time we studied them.Additional Costs of the Programs and Methods of Fun ding Table 7 shows the estimated additional costs, where available, of the knowledge and skill-based pay programs, and the method of funding these costs. It should be noted that the extra cost of salaries was hard to estimate, si nce there was little experience at most sites to tell how many teachers will move to the hi gher pay levels, and at what rate they will move. Therefore few solid costs estimates are shown. From the limited data provided, it appears that transition costs can be q uite low, as can the costs in the first years before many teachers have had a chance to dev elop the full range of knowledge and skills. However, it is also clear that some of the programs provided the potential of substantially higher salaries. Comparing the maximu m pay attainable under the former system with that attainable under the knowledge and skill-based pay system, a teacher in

PAGE 26

26 of 40 Cincinnati at the top of the schedule has the poten tial to achieve a 21% higher pay rate. In Manitowoc, such a teacher has the potential to e arn 38% more, and Vaughn, 22% more. The other programs provided a substantially s maller additional pay opportunity. Limon provided only about 1.5% more pay, Coventry, about 11%, and Douglas County, about 7.4%. Robbinsdale's proposed plan provided fo r the same maximum as in the old schedule.Table 7 Costs and FundingSiteCosts of Pay and AdministrationSource of Funds for Pay and Administration CincinnatiTransition cost to new pay schedule estim ated at 0.20.4% of payroll; ultimate extra cost ofpay changes not estimated. Cost ofadministration not known, but compensation for 8 full-time teachers to do classroomobservations could be about $500,000annually. Re-allocation of some of the dollars spent on degrees and credits in the current payschedule, reallocation of staff time andbudget resources to administer the system. Some new money raised via higher localtaxes. CoventryEstimate not available because program had just begun. Most funding appeared to have come from increases in state funding. Reallocation ofexisting time and funds used to coveradministration, most notably conversion of an administrator position to Director ofProfessional Development. Douglas County District estimate of cost of additional knowledge and skill pay elements was about0.5% of payroll. No estimate of administrativecosts is available. Additional funds raised from local tax base. LimonDistrict has not made an estimate, but if all teachers received the professional growth bonus, the cost would be about 1.4% ofpayroll. Reallocation of existing funds and additional funds raised from local tax base. ManitowocNo estimate of additional salary costs sol ely due to knowledge and skill elements wasavailable. Total package increase estimated at1.5% to 2% of operating budget, and 3.8% of payroll. No additional administrative costsexpected by district. Local Academy was expected to be self-financing. New money available fromtax base within legal limits used to finance pay costs. RobbinsdaleNo estimate available from district. Sin ce the program would be applied initially to new teachers, immediate additional costs wouldlikely be quite low. Plan was to reallocate existing funds to cover additional pay costs. VaughnTotal performance plan cost about 3.5% of payroll in 1999-2000, expected to rise to 6% in2000-2001. No estimate of administrativecosts available, but some of the time of three new administrative positions should beconsidered part of the administrative cost. Reallocation of savings from efficiencies inmanagement and in managing funds provided by formula from the state anddistrict; also, new money provided in the state funding formula was allocated to pay.In none of these cases had estimates of additional administrative costs been made. The assumption appeared to be that the time and staff n eeded to administer the programs

PAGE 27

27 of 40could be added to exiting workload or handled by re allocations of current staff. For several of the programs, this assumption will proba bly not prove problematic, because of limited scope or reliance on external assessments. For example, because it relied on discrete course grades and external certifications, the Manitowoc system requires little in the way of new administrative machinery, beyond rec ord-keeping. The Limon, Douglas County, and Coventry systems required additional ad ministrator time to review portfolios, but Douglas County, as described by Hal l and Caffarella (1997), had not found this a major strain, perhaps because a relati vely limited proportion of teachers participated in that part of the program. On the ot her hand, Cincinnati's experience, in a field test of the assessment system, suggested that most administrators did not have the time to do extensive evaluation (Milanowski and Kel lor, 2000). In response, the district decided to hire eight teachers to specialize as eva luators. Vaughn, too, had initial difficulties finding the time for peers and adminis trators to do the required number of classroom observations. The school responded by rea llocating positions to provide for more administrators, by increasing use of substitut es to free up the time of peer assessors, and by hiring two retired teachers as pa rt-time assessors. As a charter school, Vaughn had a considerable amount of budgetary flexi bility, and was able to tap grant funds to pay for part of these additional administr ative costs. These experiences suggest that it is likely that knowledge and skill-based pa y designs that use extensive internal assessment will require the allocation of additiona l resources to program administration.DiscussionThis article has attempted to summarize some of the main features of seven innovative teacher compensation programs that rewarded teacher s for developing their knowledge and skills. Based on a simple theory of action, a m odel of motivation, and descriptions of private sector experience, a set of dimensions w as developed to guide the analysis and comparison of the design of the programs. The major findings from the comparison are summarized below.Motivation for change There are a variety of reasons for designing know ledge and skill-based pay programs. Contrary to the assumptio ns underlying our theory of action, most programs were not primarily motivated by a des ire to improve the knowledge and skills of the existing teacher workforce in order t o improve student achievement, though some programs were motivated by a desire to recruit and retain more highly-skilled teachers and to support a particular vision of inst ruction. Other reasons illustrated in these cases were to respond to public pressure for a linkage of teacher pay to performance and to differentiate teacher pay based on teacher quality. State-level student assessment and accountability programs were an impo rtant factor in only those cases where student achievement was low. In most of the c ases, however, student achievement was not considered a problem. Districts with high o r acceptable student achievement appear just as likely to innovate, though the most comprehensive of the programs we studied were found where there was pressure to impr ove student achievement. Programs in Cincinnati and Coventry and at Vaughn seemed des igned to motivate teachers to practice in certain ways, rather than to motivating them to develop specific skills. These programs more closely resemble private sector compe tency-based pay programs, which often include a more general performance component, while the skill blocks in Douglas County and the Manitowoc program resemble the skill -based pay model, in which the development of specified skills is rewarded.

PAGE 28

28 of 40Design process In all cases, teachers participated in the design of the programs. The larger districts used formal committee structures o utside of the collective bargaining process to involve relatively large numbers of teac hers in developing the details. The smaller organizations were more likely to have desi gned their systems with less involvement. Contrary to expectations, some program s were designed within the negotiation process, though these tended to be the simpler ones. In all of our cases, a relatively high level of association-management coo peration, or trust between administrators and teachers, was present. Teacher c ompensation change is possible in a collective bargaining environment, and association or union leaders have been champions of the process.Knowledge and skills rewarded The knowledge and skills rewarded are generally t hose related to instruction, though none of the programs studied placed heavy emphasis on content-specific pedagogy. At the time we studied t hem, relatively few of the programs had defined an integrated model of the knowledge an d skills needed for quality instruction, nor a progression of levels of skill d evelopment providing a path to mastery, though some of the organizations may have been movi ng incrementally toward such a model. There was no dominant method of knowledge an d skill identification in these cases. Though most of the programs included the Nat ional Board standards as part of their model, the Board's standards were typically n ot highly integrated with the other knowledge and skills rewarded.Knowledge and skill assessment All of the programs use some form of performance assessment to assess the acquisition of at least so me of the knowledge and skills rewarded, rather than relying completely on degrees or credits as indicators of teacher knowledge and skill. Five of the seven programs inc luded external assessments, typically the NBPTS assessments. However, these ass essments were not typically integrated into the assessment system as a check on internal assessments, or used as an indicator of a higher level of core teaching skills Size and structure of knowledge and skill incentive s As expected, there was some tendency for programs that were motivated by the ne ed to improve student achievement to move farthest from the traditional schedule. The se programs were likely to send the strongest motivational signals to teachers. However in none of these cases were seniority and graduate degrees eliminated as a basi s for pay progression. In four of the seven cases, movement away from the traditional sal ary concepts was incremental. Support for knowledge and skill development Few of the programs we studied have developed a coordinated professional development pr ogram that is specifically linked to the knowledge and skill model. Lack of alignment of professional development programs with the knowledge and skill model may red uce the motivational force of the rewards if teachers do not perceive they have the o pportunities to acquire the knowledge and skills. That direct links to professional devel opment programs are not strong may be due to the fact that the programs were not intended to remedy knowledge and skill deficits on the part of current staff. None of the programs have fully aligned their human resource management programs with a developmentally -sequenced knowledge and skill model. This suggests that the promise of alignment in fostering a shared conception of good teaching has not yet been fulfilled.Costs and funding The cost of transition to a knowledge and skill-b ased pay system

PAGE 29

29 of 40appeared to be low in the short run, though costs a re likely to increase over time to the point that new money will be needed to fund them. L ittle information on administrative costs was available, and in most of the cases, the increased administrative costs were met using existing resources. For the more ambitiou s programs, administrative costs are likely to be significant, and may not have been ful ly realized at the time of our study.Implications for Research on Knowledge an Skill-Bas ed Pay in the K12 SectorUnlike the private sector, where skill-based pay an d competencybased pay systems appear to have become relatively codified, there ar e multiple models of knowledge and skill-based pay in the K-12 sector. These various m odels were designed to serve a variety of purposes, not simply to support improved instruction. This implies that in evaluating the success of knowledge and skill-based pay programs, it will be important to take into account other program goals and to dev elop measures of program impact in addition to measures of instructional capacity or s tudent achievement. For example, to the extent that recruitment and retention of highly -skilled teachers is an important goal, the quality of new hires and the degree to which mo re skilled teachers are retained and less skilled teachers leave will be an important ou tcome to measure. To the extent that programs are a response to community pressure for p ay for performance or accountability, it may be necessary to look at comm unity perceptions of the program. One rough indicator that Douglas County and Cincinn ati informants mentioned was increased willingness on the part of the community to pass referenda providing more tax money for education. To the extent that the goal is to support the diffusion of a particular vision of teaching, measuring teacher acceptance an d implementation of this vision will be important.Of course, the most important outcome to many organ izations considering developing and funding knowledge and skill-based pay programs is likely to be whether they are effective in motivating skill acquisition, changing instruction, and improving student achievement. But because knowledge and skill-based pay at this point encompasses such a variety of designs, it will be important to devel op some measures of the potential causal "strength" of the program. A set of benchmar ks could be developed, as was done by evaluators of the New American Schools implement ation in Memphis (Smith et al, 1998, Ross, 2000). This would entail using the theo ry of action and motivational model to specify dimensions and develop some rubrics for judging how close the design and implementation comes to the ideal specified in the theory, then relating these ratings to measures of effects. For example, the theory of act ion and motivational model suggest that a program providing few professional developme nt opportunities, little administrator and peer support for new skill acquis ition, and relatively small incentives, would have a limited effect on instruction and stud ent achievement. In such a case lack of evidence that knowledge and skill-based pay was associated with improved instruction or student achievement would not be sur prising, but also would not provide much information about whether a stronger knowledge and skill-based pay design can help improve instruction.It is interesting to speculate as to whether knowle dge and skillbased pay in the K-12 sector will evolve toward a plurality of systems re flecting strategic district or school goals, local history, and designers' preferences, o r toward a family of similar systems based on external, generic standards. In the privat e sector, the theoretical argument for

PAGE 30

30 of 40knowledge and skill-based pay is that it can provid e incentives to develop organization-specific skills that support a unique competitive strategy. The K-12 analog would be a set of schools operating under school ch oice or voucher systems, with little procedural regulation. Of our cases, the Vaughn cha rter school best fits this model and it did "tailor" its knowledge and skill model more clo sely to its mission, as set forth in its charter. But as argued above, in the K-12 sector th e core skills are likely to be similar across schools and districts, and there is little c ompetition across schools or districts. Thus over time we may see a tendency for convergenc e on external standards and assessments. The limiting factor appears to be a la ck of external assessments aimed at differentiating among mid-career teachers.Implications for Policy Makers and Program Designer sThe experiences of the seven organizations we studi ed suggest a number of fairly clear lessons for the design of knowledge and skillbase d pay programs. First, that even the most radical of the seven programs we studied retai ned seniority and degrees as pay criteria suggests that it may be unrealistic to exp ect completely performancebased pay systems to emerge. It may be necessary to retain so me aspects of the traditional structure in order to have a realistic chance of implementing a pay system that rewards the acquisition of instruction-relevant knowledge and s kills. As the Cincinnati and Vaughn cases illustrate, a program can be designed to prov ide significant incentives for knowledge and skill development while retaining som e rewards for seniority. Second, teachers' associations may be more open to changes in pay systems than administrators or school board members expect, but this openness is likely to be the product of high levels of trust developed through c ooperation on other issues, and design features aimed at encouraging acceptance by senior teachers may be needed. Third, it may be easier and faster to adapt a set o f pre-existing teacher standards rather than to develop a knowledge and skill model from sc ratch. Fourth, the transition costs to even a fairly exten sive knowledge and skill-based pay structure can be low. It is, however, likely that a dministrative costs will be higher where an extensive system of internal skill assessment is used, and that payroll costs may significantly increase in the long term. So it is a dvisable to plan for ways to cover these costs. It is also interesting to note that, for these pion eer organizations, state programs such as assessment and accountability systems or teacher li censing supported rather than drove teacher compensation change. Local issues, union-ma nagement relations, and the agendas of leaders were probably more important as initiators of change, and will likely be very important in sustaining and guiding a progr am until it has taken hold. For state-level policy makers who desire to use salary dollars more strategically to improve student achievement, one implication is that it may be useful to provide a comprehensive and coherent model of knowledge and skills that are directly related to improving instruction for local organizations to adapt and cu stomize. This would help focus teacher compensation change on strategically-important goal s. The model could also be linked to state standards for students. If one way to impr ove student achievement toward state content standards is to ensure that teachers can de velop and teach high quality standards-based curriculum units to all students (C ohen and Hill, 2000), then the model

PAGE 31

31 of 40should emphasize the knowledge and skills needed to do this. Integration of the model with a multilevel licensing system and providing funds to increase pay for teachers with higher-level licenses could be another way for state-level policy makers to focus the system on strategically important goals.Not only might state-level action help to encourage greater coherence and focus on improved instruction, but there may also be signifi cant efficiencies to be gained from developing a state-level model rather than having e ach district or school work on the problem alone. External assessments could be develo ped for common knowledge and skill elements, to lower the burden on local school s and districts. One state that has been working along these lines is Iowa. In that state, e ducation, business, and political leaders developed a comprehensive model for teacher perform ance evaluation, licensing, and compensation (Iowa Department of Education, 2000). Yet the ability of local organizations to customize a state model should be retained, in order to maximize the potential for local acceptance and to recognize tha t, at this early stage, no one knowledge and skill-based pay model has emerged as 'best prac tice'.AcknowledgmentThis article is primarily based on case studies don e by staff researchers with the Consortium for Policy Research in Education's Teach er Compensation project at the University of Wisconsin-Madison. The original autho rs of the cases studies are: Eric Conti: Limon, Colorado, Robbinsdale, Minnesota, and Coventry, Rhode Island; Eric Conti, with the assistance of Eileen Kellor: Manito woc, Wisconsin; Carolyn Kelley, Douglas County, Colorado; Eileen Kellor, Vaughn Nex t Century Learning Center, Los Angeles, California; Eileen Kellor and Allan Odden, Cincinnati, Ohio. I thank these researchers for allowing me to use their work, and to assure them and the reader that any errors in the descriptions, analysis, or conclusion s are my responsibility. The reader should be aware that staff of the Consor tium for Policy Research in Education at University of Wisconsin-Madison were i nvolved in the development of two of these programs. Alan Odden and Eileen Kellor pro vided consultation and technical assistance to the district committee that designed the Cincinnati program. Anthony Milanowski and Eileen Kellor have been involved in evaluations of the program paid for by the district. Allan Odden has provided technical assistance to staff at the Vaughn Next Century Learning Center, and Anthony Milanowsk i has provided occasional advice to Vaughn in the course of exploratory research on that school's program. Though the author does not believe that this involvement has b iased his description or implicit evaluation of the programs discussed here, the read er may want to remember this involvement when reading this article.NotesThe research reported here was supported by a grant from the U.S. Department of Education, Office of Educational Research and Impro vement, National Institute on Educational Governance, Finance, Policy-Making and Management, to the Consortium for Policy Research in Education (CPRE) and the Wis consin Center for Education Research, School of Education, University of Wiscon sin-Madison (Grant No. OERI-R3086A60003). The opinions expressed are those of the authors and do not necessarily reflect the view of the National Instit ute on Educational Governance,

PAGE 32

32 of 40Finance, Policy-Making and Management, Office of Ed ucational Research and Improvement, U.S. Department of Education, the inst itutional partners of CPRE, or the Wisconsin Center for Education Research.ReferencesAmerican Compensation Association (1996). Raising the bar: Using competencies to enhance employee performance Scottsdale, AZ: Author. Bembry, K. L., Jordan, H.R., Gomez, E., Anderson, M .C., & Mendro, R.L. (1998, April). Policy implications of longterm teacher effects o n student achievement. Paper presented at the 1998 Annual Meeting of the A merican Educational Research Association, San Diego, CA. Cohen, D. &, Hill, H. (2000). Instructional policy and classroom performance: The mathematics reform in California. Teachers College Record 102 (2), 294-343. Conley, S.C., & Odden, A. (1995). Linking teacher c ompensation to teacher career development. Educational Evaluation & Policy Analysis 17 (2), 219-237. Daniels, D.R. (1997). Competency-based compensation approaches. In P. T. Chingos (Ed.), Paying for performance: A guide to compensation man agement (pp. 72-95). New York: Wiley. Danielson, C. (1996). Enhancing professional practice: A framework for te aching Alexandria, VA: Association for Supervision and Cur riculum Development. Dwyer, C.A. (1998). Psychometrics of PRAXIS III: Cl assroom performance assessments. Journal of Personnel Evaluation in Education 12 (2), 163-187. Ferguson, R., & Ladd, H. (1996). How and why money matters: An analysis of Alabama schools. In H. Ladd, (Ed.), Holding schools accountable (pp. 265-298). Washington, DC: Brookings Institution. Fisher, R., & Ury, W. (1981). Getting to yes Boston: Houghton-Mifflin. Gorsline, K. (1996). A competency profile for human resources: No more shoemaker's children. Human Resource Management 35 (1), 53-66. Gupta, N., Jenkins, G.D. Jr., & Curington, W.P. (19 86). Paying for knowledge: Myths and realities. National Productivity Review 5 (2), 107-123. Hall, G. E., & Caffarella, E.P. (1997). Third year implementation assessment of the Douglas County, Colorado school district performanc e pay plan for teachers No city: Authors. Heneman R. L.. & Ledford, G.E. (1998). Competency p ay for professionals and managers in business: A review and Implications for teachers. Journal of Personnel Evaluation in Education 12 (2), 103-122. Heneman, R. L., & Thomas, A.L. (1997). Using strate gic performance management to drive brand leadership. Compensation and Benefits Review 27 (6), 33-40.

PAGE 33

33 of 40Interstate New Teacher Assessment and Support Conso rtium. (1992). Model standards for beginning teacher licensing and development: A resource for state dialog Washington, DC: Author. Iowa Department of Education (2000, November). A proposal for a new teacher compensation system based on teacher performance, c areer development, and the national labor market Des Moines, IA: Author. Jaeger, R.J. (1998). Evaluating the psychometric qu alities of the National Board for Professional Teaching Standards' Assessments: A met hodological accounting. Journal of Personnel Evaluation in Education 12 (2), 189-210. Jenkins, G.D., Jr., Ledford, G.E., Jr., Gupta, N., & Doty, D.H. (1992). Skill-based pay: Practices, payoffs, pitfalls and prescriptions Scottsdale, AZ: American Compensation Association. Jones, T.W. (1995). Performance management in a cha nging context: Monsanto pioneers a competency-based developmental approach. Human Resource Management 34 (3), 425-442. Lawler, E.E. III (2000). Rewarding excellence: Pay strategies for the new ec onomy San Francisco: Jossey-Bass. Ledford, G.E. Jr. (1991). The design of skill-based pay plans. In M. L. Rock & L.A. Berger (Eds.), The compensation handbook (3rd ed, pp. 199-217). New York: McGraw Hill. Ledford. G.E., & Heneman, R.L. ( 2000). Pay for ski lls, knowledge, and competencies. In L. A. Berger & M. L. Rock (Eds.), The compensation handbook (4th ed.). New York: McGraw-Hill. Mansfield, R.S. (1996). Building competency models: Approaches for HR professionals. Human Resource Management 35 (1), 7-18. Milanowski, A.T., & Kellor, E. (1999) Vaughn Next Century Learning Center performance pay survey: Preliminary report. Unpublished manuscript, University of WisconsinMadison. Milanowski, A.T., & Kellor, E. (2000a). Vaughn Next Century Learning Center performance pay survey: School report. Unpublished manuscript, University of Wisconsin-Madison. Milanowski, A.T., & Kellor, E. (2000b). Teacher and evaluator reactions to standards-based teacher evaluation in the Cincinnat i Public Schools: An evaluation of the 19992000 field test of the new evaluation system. Unpublished manuscript, University of Wisconsin-Madison. Milanowski, A., Odden, A. & Youngs, P. (1998). Teac her knowledge and skill assessment and teacher compensation: An overview of measurement and linkage issues. Journal of Personnel Evaluation in Education 12 (2), 83-101.

PAGE 34

34 of 40Mohrman, A., Mohrman, S.A., & Odden A. (1996). Alig ning teacher compensation with systemic school reform: Skill-based pay and group based performance rewards. Educational Evaluation and Policy Analysis 18 (1), 51-71. National Board for Professional Teaching Standards. (2001). State Incentives. Retrieved from http://www.nbpt.org/news/_center/facts/state_i ncentives.html. National Board for Professional Teaching Standards. (1999). What teachers should know and be able to do Arlington, VA: Author. National Commission on Teaching and America's Futur e. (1996). What matters most: Teaching for America's future. New York: Author. Odden, A. (2000). New and better forms of teacher c ompensation are possible. Phi Delta Kappan 81 (5), 361-366. Odden, A., & Kelley, C. (1997). Paying teachers for what they know and do: New and smarter compensation strategies to improve schools Thousand Oaks, CA: Corwin Press. Ross, S.M. (2000). How to evaluate comprehensive sc hool reform models. Getting Better by Design Series 8 1-14. Arlington, VA: New American Schools Development Corporation. Shippmann, J. S., Ash, R. A., Battista, M., Carr, L ., Eyde, L. D., Hesketh, B., Kehoe, J., Pearlman, K., Prien, E. P., & Sanchez, J. I. (2000) The practice of competency modeling. Personnel Psychology 53 (3), 703-740. Slavin, R., & Fashola, O. (1998). Show me the evidence! Proven and promising programs for America's schools Thousand Oaks, CA: Corwin Press. Shulman, L.S. (1987). Knowledge and teaching: Found ations of the new reform. Harvard Educational Review 57 (1), 1-22. Smith, L., Ross, S., McNelis, M., Squires, M., Wass on, R., Maxwell, S., Weddle, K., Nath, L., Grehan, A., & Buggey, T. (1998). The Memp his restructuring initiative: Analysis of activities and outcomes that affect imp lementation success. Education and Urban Society 30 (3), 296-325. Spencer, L.M., & Spencer, S.M. (1993). Competence at work: Models for superior performance New York: Wiley. Tucker, S.A., & Cofsky, K.M. (1994). Competencyba sed pay on a banding platform. ACA Journal 3 (1), 30-45. Vroom, V.H. (1964). Work and motivation New York: Wiley. Wise, A.E., Darling-Hammond, L., McLaughlin, M.W., & Bernstein, H.T. (1984). Teacher evaluation: A study of effective practices Santa Monica, CA: Rand Corporation.

PAGE 35

35 of 40Wilson, T.B., & Phalen, C.P. (1996). Rewarding group performance: An approach to designing and implementing incentive pay programs Scottsdale, AZ: American Compensation Association, Wright, S.P., Horn, S.P, & Sanders. W.L. (1997). Te acher and classroom context effects on student achievement: Implications for teacher ev aluation. Journal of Personnel Evaluation in Education 11 57-67. Zingheim, P.K., Ledford, G.E. Jr., & Schuster, J.R. (1996). Competencies and competency models: Does one size fit all? ACA Journal 5 (1), 56-65.About the AuthorAnthony MilanowskiAssociate ResearcherConsortium for Policy Research In EducationandIndustrial Relations Research InstituteUniversity of Wisconsin-MadisonMadison, WI 53706(608) 263-4260amilanow@facstaff.wisc.eduAnthony Milanowski is an Associate Researcher with Consortium for Poli cy Research in Education (CPRE) and a participating faculty mem ber of the Industrial Relations Research Institute at the University of Wisconsin-M adison. He received his Ph.D. in Industrial Relations from the University of Wiscons in-Madison in 1997. Prior to joining CPRE, he worked in human resource management for 16 years, primarily with the Wisconsin Department of Employment Relations. His current research interests include teacher performance evaluation, pay system innovati ons, and the teacher labor market.AppendixBrief Descriptions of the Seven Knowledge and Skill -Based Pay (KSBP) Case Sites CincinnatiThe Cincinnati Public School district is the State of Ohio's third largest, enrolling about 48000 students in 78 schools. The student populatio n is 71% African-American, the rest white or other. About 65% are eligible for free or reduced price lunch. A relatively large proportion of Cincinnati's school-age children atte nd private schools, which, given the state's school funding system and laws requiring re ferenda for increases in school spending, has faced the district with pressures to reduce costs and improve student achievement. Average perpupil spending was about $8,000 in 1998-99. The average teacher base salary was about $44,000. Fifty-one pe rcent of the district's 3,000 teachers have Master's degrees and the average level of teac hing experience is 15 years. The district has a seven-person elected school board on which the members serve staggered four year terms. Teachers are represented by a loca l American Federation of Teachers affiliate. CPS began developing its knowldge and sk ill-based pay program in 1996 with

PAGE 36

36 of 40a commitment to redesign the teacher evaluation sys tem. The new evaluation system, which will be the foundation for the KSBP program, was field-tested in the 1999-2000 school year and will be used in the 2001-2002 and 2 002-2003 school years before the pay component. The pay component has been fully des igned and was included in the collective bargaining agreement approved by the Boa rd and the local Federation of Teachers in the Spring of 2000. In 2003-2004, the p ay component will be added unless a supermajority of teachers vote to reject the progra m in May of 2003. In addition to the knowledge and skill-based pay structure, a group bo nus of $1,400 is to be paid to all teachers in schools that meet school-wide goals for improving student achievement. CoventryThe Coventry, Rhode Island district is one of the f astest growing suburban districts in the Northeast. It serves 5,600 students in 9 school s. The student population is 98% white and 2% minority. About 22 % are eligible for free o r reduced price lunch. Student population has been growing moderately. Average per -pupil spending was $7,400 in 199798, and the average teacher base salary was a bout $50,00. About 80% of the district's 351 teachers have Master's degrees and t he distribution of teaching experience is bimodal, with about 60% very long service and 40 % 5 or fewer years. Teachers are represented by a local American Federation of Teach ers affiliate. Coventry began developing its new pay system in 1995, when the ass ociation proposed recognition of National Board certification. The initial element o f the program, a bonus for NBPTS certification, was implemented in 1996. Additional pay for knowledge and skill elements were implemented for the 2000-2001 school year.Douglas CountyThe Douglas County, Colorado school district, locat ed in a fastgrowing area between Denver and Colorado Springs, enrolls more than 32,0 00 students in 49 schools. (Thirty-two schools were opened since 1989.) The st udent population is 91% white, 4% Hispanic, and 5% other. About 2% are eligible for f ree or reduced price lunch. Average perpupil spending was $7.817 in 1996-97, and the average teacher base salary about $39,680 in 1998. The average level of teaching expe rience is about 8 years. Teachers are represented by a local American Federation of Teach ers affiliate. The process of developing the new pay system began in the 1991-92 school year, but the major design activities took place from July 1993 to the beginni ng of the 94-95 school year. The current plan was first implemented during the 199495 school year and has continued with minor modification since. Besides knowledge an d skill elements, it also includes a modification of the traditional pay schedule that m akes seniority pay progression dependent on satisfactory performance evaluation, a school group bonus program, and added pay for additional school-or district-level r esponsibilities. LimonThe Limon, Colorado district serves 660 students in two schools. Located in a rural area, the district's students are 91% white, 5.6% Hispani c, and 3.4% other. About 34% are eligible for free or reduced price lunch. The size of the student population is now stable after an brief period of increase in the early 90's Average per-pupil spending was $5,643 in 1996-97, and the average teacher base salary was $27,900 in 1998. 20% of the 44 teachers have Master's degrees, and the average lev el of teaching experience is about 10 years. Teachers are represented by a local associat ion affiliated with the National

PAGE 37

37 of 40Education Association, but only a minority of teach ers pay state and local dues, and collective bargaining is essentially informal, with no formal contract negotiated. Pay innovation began in 1994-95, with the development o f a link between pay increases and teacher performance evaluations. The traditional st ep and lane schedule was eliminated in favor of merit pay, which was based on principal evaluation in accordance with the state evaluation standards. The current plan was in troduced for the 1998-99 school year. In addition to knowledge and skill-based pay, the p ay system also the potential for a $400 increase if building-level goals are met and a $400 increase for achieving unit or grade-level goals.ManitowocThe Manitowoc, Wisconsin school district is located in a community of 33,000 in the eastern part of the state. It serves almost 6,000 s tudents in 5 schools. The student population is 86% white and 14% minority. Student p opulation growth has leveled off and is expected to decline. About 1% are eligible f or free or reduced price lunch. Average per-pupil spending is about $7,692, and the average teacher base salary about $ 37,240. Relatively few of the district's 420 teache rs have Master's degrees, but the average level of teaching experience is relatively high. Teachers are represented by a local National Education Association affiliate. Man itowoc began developing the new pay system in early 1999, as part of negotiations f or the 1999-2001 teacher contract. The system went into effect for the 2000-2001 school ye ar. RobbinsdaleThe Robbinsdale, Minnesota school district is locat ed in a suburban area outside Minneapolis-Saint Paul. It serves approximately 14, 000 students, of who 20% are nonwhite and 22 are eligible for free or reduced pr ice lunch. (It should be noted, however, that within the district, the percentage e ligible for free or reduced price lunch varies widely by school, from 15 to 80%.) The stude nt population is slowly increasing. Average per-pupil spending is about $8,555, and the average teacher base salary about $ 44,950. 48% of the 900 teachers have Master's degre es and the average level of teaching experience is 14 years. Teachers are represented by a local American Federation of Teachers affiliate. The process of developing a new pay system began in 1994. Though the negotiation process, an outline of a plan was d eveloped as part of the 1995-97 collective bargaining agreement, but the tentative contract was rejected by the membership. A revised program structure was approve d as part of the 1997-2000 contract. The district is still working to develop the components, and the program has not yet gone into effect.VaughnVaughn Next Century Learning Center is a public cha rter school in San Fernando, California. Previously a public school in the Los A ngeles Unified School Distinct, the school converted to charter status in July of 1993, It currently serves about 1,200 students in pre-K through grade 5. The student popu lation is 94% Hispanic and 5.5% African American, and .5% other. Only 13% of the st udents are considered to be English proficient. About 98% are eligible for free or redu ced-price lunch. The average teacher base salary was $42,000 in 1999-2000. The average l evel of teaching experience is 7 years. Vaughn is governed by three staff/parent com mittees, with a special council existing to oversee and resolve disputes between th e three committees. Teachers as a

PAGE 38

38 of 40 group are not represented at this time, though some individual teachers are members of the American Federation of Teachers or National Edu cation Association. The school began developing its KSBP program during the 1997-9 8 school year. An initial implementation for new teachers and volunteers was done in the 1998-99 school year. During that year, 19 of the 50+ teachers participat ed in the program. An improved version was put into place, again for new teachers and volunteers, for the 1999-2000 school year, during which 37 of the classroom teach ers participated. The program was continued during the 2000-2001 school year. Vaughn' s knowledge and skill-based pay is part of a complete redesign of pay system that also included pay for additional duties and a group bonus of $1,500 for all teachers if the sch ool meets the student achievement goals in the charter. In addition, the school is el igible for a state program that provides bonuses to teachers in schools that meet state-set goals for improving student achievement.Copyright 2003 by the Education Policy Analysis ArchivesThe World Wide Web address for the Education Policy Analysis Archives is epaa.asu.edu General questions about appropriateness of topics o r particular articles may be addressed to the Editor, Gene V Glass, glass@asu.edu or reach him at College of Education, Arizona State University, Tempe, AZ 8 5287-2411. The Commentary Editor is Casey D. Cobb: casey.cobb@unh.edu .EPAA Editorial Board Michael W. Apple University of Wisconsin David C. Berliner Arizona State University Greg Camilli Rutgers University Linda Darling-Hammond Stanford University Sherman Dorn University of South Florida Mark E. Fetler California Commission on Teacher Credentialing Gustavo E. Fischman California State Univeristy–Los Angeles Richard Garlikov Birmingham, Alabama Thomas F. Green Syracuse University Aimee Howley Ohio University Craig B. Howley Appalachia Educational Laboratory William Hunter University of Ontario Institute ofTechnology Patricia Fey Jarvis Seattle, Washington Daniel Kalls Ume University Benjamin Levin University of Manitoba Thomas Mauhs-Pugh Green Mountain College Les McLean University of Toronto Heinrich Mintrop University of California, Los Angeles Michele Moses Arizona State University Gary Orfield Harvard University

PAGE 39

39 of 40 Anthony G. Rud Jr. Purdue University Jay Paredes Scribner University of Missouri Michael Scriven University of Auckland Lorrie A. Shepard University of Colorado, Boulder Robert E. Stake University of Illinois—UC Kevin Welner University of Colorado, Boulder Terrence G. Wiley Arizona State University John Willinsky University of British ColumbiaEPAA Spanish Language Editorial BoardAssociate Editor for Spanish Language Roberto Rodrguez Gmez Universidad Nacional Autnoma de Mxico roberto@servidor.unam.mx Adrin Acosta (Mxico) Universidad de Guadalajaraadrianacosta@compuserve.com J. Flix Angulo Rasco (Spain) Universidad de Cdizfelix.angulo@uca.es Teresa Bracho (Mxico) Centro de Investigacin y DocenciaEconmica-CIDEbracho dis1.cide.mx Alejandro Canales (Mxico) Universidad Nacional Autnoma deMxicocanalesa@servidor.unam.mx Ursula Casanova (U.S.A.) Arizona State Universitycasanova@asu.edu Jos Contreras Domingo Universitat de Barcelona Jose.Contreras@doe.d5.ub.es Erwin Epstein (U.S.A.) Loyola University of ChicagoEepstein@luc.edu Josu Gonzlez (U.S.A.) Arizona State Universityjosue@asu.edu Rollin Kent (Mxico)Departamento de InvestigacinEducativa-DIE/CINVESTAVrkent@gemtel.com.mx kentr@data.net.mx Mara Beatriz Luce (Brazil)Universidad Federal de Rio Grande do Sul-UFRGSlucemb@orion.ufrgs.brJavier Mendoza Rojas (Mxico)Universidad Nacional Autnoma deMxicojaviermr@servidor.unam.mxMarcela Mollis (Argentina)Universidad de Buenos Airesmmollis@filo.uba.ar Humberto Muoz Garca (Mxico) Universidad Nacional Autnoma deMxicohumberto@servidor.unam.mxAngel Ignacio Prez Gmez (Spain)Universidad de Mlagaaiperez@uma.es Daniel Schugurensky (Argentina-Canad)OISE/UT, Canadadschugurensky@oise.utoronto.ca Simon Schwartzman (Brazil)American Institutes for Resesarch–Brazil(AIRBrasil) simon@sman.com.br

PAGE 40

40 of 40 Jurjo Torres Santom (Spain)Universidad de A Coruajurjo@udc.es Carlos Alberto Torres (U.S.A.)University of California, Los Angelestorres@gseisucla.edu


xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam a22 u 4500
controlfield tag 008 c20039999azu 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E11-00302
0 245
Educational policy analysis archives.
n Vol. 11, no. 4 (January 29, 2003).
260
Tempe, Ariz. :
b Arizona State University ;
Tampa, Fla. :
University of South Florida.
c January 29, 2003
505
Varieties of knowledge and skill-based pay design :a comparison of seven new pay systems for K-12 teachers / Anthony Milanowski.
650
Education
x Research
v Periodicals.
2 710
Arizona State University.
University of South Florida.
1 773
t Education Policy Analysis Archives (EPAA)
4 856
u http://digital.lib.usf.edu/?e11.302