USF Libraries
USF Digital Collections

Educational policy analysis archives


Material Information

Educational policy analysis archives
Physical Description:
Arizona State University
University of South Florida
Arizona State University
University of South Florida.
Place of Publication:
Tempe, Ariz
Tampa, Fla
Publication Date:


Subjects / Keywords:
Education -- Research -- Periodicals   ( lcsh )
non-fiction   ( marcgt )
serial   ( sobekcm )

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
usfldc doi - E11-00432
usfldc handle - e11.432
System ID:

This item is only available as the following downloads:

Full Text
xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam a22 u 4500
controlfield tag 008 c20059999azu 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E11-00432
0 245
Educational policy analysis archives.
n Vol. 13, no. 10 (January 29, 2005).
Tempe, Ariz. :
b Arizona State University ;
Tampa, Fla. :
University of South Florida.
c January 29, 2005
Factors affecting the impact of professional development programs on teachers knowledge, practice, student outcomes & efficacy /Lawrence Ingvarson, Marion Meiers [and] Adrian Beavis.
x Research
v Periodicals.
2 710
Arizona State University.
University of South Florida.
1 773
t Education Policy Analysis Archives (EPAA)
4 856


E DUCATION P OLICY A NALYSIS A RCHIVES A peer reviewed scholarly journal Editor: Sherman Dorn College of Education University of South Florida Copyright is retained by the first or sole author, who grants right of first publication to the Education Policy Analysis Archives EPAA is published jointly by the Colleges of Education at Arizona State University and the University of South Florida. Articles are indexed in the Directory of Open Access Journals ( Volume 13 Number 10 January 2 9 2005 ISSN 1068 2341 Factors Affecting t he Impact o f Professional Development Programs On Teachers Knowledge, Practice, Student Outcomes & Efficacy Lawrence Ingvarson Marion Meiers Adrian Beavis Australian Council for Educational Research Citation: In gvarson, L., Meiers, M. & Beavis, A. (2005, January 29). Factors a ffecting t he i mpact o f p rofessional d evelopment p rograms o n t eachers k nowledge, p ractice, s tudent o utcomes & e fficacy Education Policy Analysis Archives, 13 (10). Retrieved [date] from Abstract This report examines effects of structural and process features of professional development programs on teachers knowledge, practice and efficacy. It is based on four recent (2002 2003) studies undertaken through the Australian Government Quality Teacher Programme, designed to enhance teacher quality. The total data set for the survey study includes 3 250 teachers who had participated in eighty individua l professional development 1 activities within these studies. Teachers were surveyed at least three months after participating in an activity, which provided them with the opportunity to gauge the impact of programs on their practice. To investigate factors affecting impact, a theoretical model was developed based on recent research into the characteristics of effective professional development and tested using blockwise regression analysis. The model included contextual factors (e g ., school support), structur al features of programs (e g length), process features (e.g. emphasis on content; active learning; examination of student work; feedback; follow up), a mediating variable (level of professional community generated), and four outcome measures (knowledge; pra ctice; student learning and efficacy). Consistent significant direct effects were found 1 In this article programme refers to the Australian Government Quality Teacher Programme (AGQTP). P rojects funded under AGQTP consist of either State and Territory professional learning projects or national strategic projects (see p. 3 for further description). State and Territory professional learning projects develop and deliver professional learning activities for teachers, either on a State or Territory wide ba sis or for individual education authorities within that State or Territory.


Education Policy Analysis Archives Vol. 13 No. 10 2 across the four studies for the impact of content focus, active learning, and follow up on knowledge and professional community. Feedback was rarely incorporated into p rogram design. Impact on efficacy was strongly related to the perceived impact of activities on teachers practice and student learning outcomes. Introduction Professional development for teachers is now recognised as a vital component of policies to enh ance the quality of teaching and learning in our schools. Consequently, there is increased interest in research that identifies features of effective professional learning. Considerable funds are allocated to a wide variety of professional development pr ograms from a variety of sources. As investment increases, policy makers are increasingly asking for evidence about its effects not only on classroom practice, but also on student learning outcomes. They are also looking for research that can guide them in designing programs that are more likely to lead to significant and sustained improvement in students opportunities to learn. There is a need, therefore, for more sophisticated methods for evaluating professional development programs, with the capacity to meet these information needs. In the not too distant past, when many professional development courses placed teachers in the role of an audience, questionnaires distributed at the door as teachers left sufficed. Strategies for professional developmen t have now become much more complex, long term and embedded in schools. Major funds may be allocated, for example, to training school based staff developers and providing them with time release, developing curriculum support materials, time release, and o n line learning. The kinds of questions that evaluators now need to answer are much more penetrating than questions such as What did you learn from the workshop? They are questions about program logic and the presumed links between professional learn ing strategies, and changes in teacher knowledge, classroom practices and student outcomes. These questions call for large scale studies with the capacity to tests these relationships across large numbers of different professional development programs. Purposes of the study This paper reports on the effects of structural and process features of professional development programs on teachers knowledge, practice and efficacy. It is based on four evaluation studies recently completed by ACER from 2001 to 2 003 under the Australian Government Quality Teacher Programme (AGQTP). The AGQTP is intended to update and improve teachers skills and understanding and enhance the status of teaching. It currently has two elements: State and Territory professional lear ning projects, which have been contracted with government and non government school education authorities to develop and deliver professional learning activities to school teachers; and Strategic national projects focussing on research, investigation and d evelopment of a range of issues relating to teacher quality and school leadership.


Ingvarson et al.: Impact o f Professional Development Programs 3 The AGQTP, (which was formerly titled the Quality Teacher Programme over 2000 2003), received total funding of $A76.8 million over 2000 2003, with a further $A82.4 million provided for 2003 2005. Of these amounts, $97.2 million has so far been allocated to professional learning projects. Most of the State and Territory professional learning projects conducted over 2000 2003, undertook an evaluation or review of activitie s. The four studies included in this paper are: Evaluations of the Australian Government Quality Teacher Program professional learning projects, as implemented by education authorities in three states (hereafter Projects A, B and C). (Note: Education au thority refers to an organisation which represents, or is the governing body, of a system of schools, either government, Catholic or Independent schools. For example, one of the education authorities included in this study is the Catholic Education Commi ssion in the state of Victoria). An AGQTP strategic national project involving a major research study of ten professional development activities, conducted by ACER and titled, Investigating the links between professional development and student learning o utcomes (hereafter Project D). While this paper utilises data and analyses from these studies, it should not be seen as providing a summary or overview of their findings. The findings from some of these studies, such as the AGQTP project, Investigating t he links between professional development and student learning outcomes have not yet been released. The central purpose of the evaluation was to examine the effectiveness and quality of the professional development initiatives funded under the NSW QTP, a nd the extent to which the intended outcomes of the NSW QTP had been achieved. Approach to E valuating P rofessional D evelopment A ctivities Cross program analysis In each of these evaluation studies, data was gathered from a number of professional devel opment programs. In evaluating one state program, for example, data was gathered from over 40 different professional development programs and 1731 teacher participants. Australian Government guidelines required that state programs target experienced prim ary and secondary teachers (more than ten years) and focus on literacy, mathematics, science, technology and vocational education. The professional development programs utilised a range of delivery modes, including: Workplace learning through action resea rch, coaching and mentoring; Institutional learning to facilitate understanding of research findings and best practice; Online learning; Participation in formal award programs; Conferences and seminars.


Education Policy Analysis Archives Vol. 13 No. 10 4 Individual professional development programs in each state used a combination of professional development strategies, such as workshops, accredited courses, online learning, mentoring, and action learning. Some projects were conducted over extended periods of time, others were conducted as single session w orkshops, and others used a combination of organisational structures, such as an initial workshop sessions prior to a school based project. Some projects were conducted in phased stages, and some included related sub projects. (Appendix C provides a profi le of one of the professional development programs (Program G), summarising the main features of its program logic.) States and Territories receiving Australian Government funding were also expected to commission independent evaluations of their program s that addressed the following issues: a) the nature and extent of the participation of teachers from specified target groups; b) participant satisfaction with the quality of the programs; c) the extent to which the program achieved its outcomes and objectives; d) t he contribution of the program to improving and expanding pedagogy; e) the contribution of the program to improving student learning outcomes in schools; f) the extent to which the program enhanced the status of teaching. While it was feasible to address some o f these issues in the time available for the evaluation studies, others, such as effects on student outcomes and the status of teaching, were more problematic. Our evaluations (Projects A. B, and C) relied mainly on teacher self reports. (Project D mentio ned above did include separate measures of student learning outcomes before and after the 10 professional development programs). While we have some confidence in teachers self reports of the effects of professional development programs on their knowledge and practice, we have less confidence in teachers self reports of the impact of these programs on their students learning outcomes. In all, data was gathered from a total of 3250 teachers who had participated in over eighty different professional progr ams. These evaluation studies provided a unique opportunity to conduct research looking at the differential impact of a wide range of professional development strategies. Participants in each of these programs were invited to complete a common survey i nstrument, which asked them to describe both the processes of learning that they had experienced and the impact of these programs on their knowledge, practice, sense of efficacy, and their students learning. The survey also asked participants about the i mpact of the programs on the nature and extent of collaborative work amongst colleagues in their schools. The extent to which programs strengthened, or integrated with professional community activity turned out to be a significant predictor of impact. A s might be expected, there were significant differences between programs in the mode of delivery and in the extent to which teachers reported that programs had influenced


Ingvarson et al.: Impact o f Professional Development Programs 5 their practice and benefited their students. These differences opened up the possibi lity for cross program analyses that might: a) increase understanding of those features of program design and delivery that might explain variation in impact; b) identify school level factors that influence or mediate the outcome of the programs. Another feat ure of these studies was that teachers were surveyed at least three months after participating in a program, which provided them with a better basis on which to gauge the impact that programs had had on their practice. However, this delay was at some cost to response rates to our mailed surveys, which averaged around 50%. Research based conceptual framework These analyses called for the development of a conceptual framework to guide the evaluation. The approach to evaluation in each of the four studies wa s based on the theoretical framework, shown in Figure 1. It presents a model of the main features of programs that might explain variation in their reported impact. The framework was based on a review of recent research into the characteristics of effect ive professional development programs (Kennedy, 1998; Wilson & Berne, 1998; Garet et al., 2001; Sykes, 2002; Cohen & Hill, 2000, Hawley & Valli, 1999; Guskey & Sparks, 2002; Loucks Horsley et al. 1998; Supovitz, 2001). This research has become increasingly sophisticated over recent years (Ingvarson, 2002) and provided a firm foundation on which to develop a model to account for the relative differences in the effectiveness of professional development programs. Figure 1 distinguishes four, linked, types o f impact resulting from PD programs. These include impact on teachers knowledge and practice, student learning and teacher efficacy. The model also includes background (control) variables, structural features, such as the duration of the program and opp ortunity to learn features, such as active learning, or follow up. (Details of how these variables were measured are provided below.) Measures Control variables Control variables included: teacher gender; experience, measured as number of years teac hing; school sector; and school support for professional development. School support was measured by asking teachers the extent to which they agreed or disagreed with the following statements: The leaders at my school actively support and encourage all s taff to take part in professional development. Insufficient time is available in my school to support teachers professional learning Follow up support for professional development is available within my school. Teachers at my school work collaboratively t o resolve teaching and learning issues.


Education Policy Analysis Archives Vol. 13 No. 10 6 Figure 1: Relationships between structure, learning processes and impact of professional development programs Gender Experience School sector School level School support School size Knowledge Contact hours Time span Sufficient time Collective participation Content focus Active learning Follow up Collaborative examination of student work Feedback on practice P rofessional community Practice Student learning Efficacy Background variables Structural features Opportunity to learn Mediating factors Impact


Structural features Duration includes two slightly different, but related, aspects: conta ct hours and time span To measure contact hours, teachers were asked to indicate the total number of hours they spent in activities related to the professional development program (1 = less than 10; 2 = 10 20 hours; 3 = 20+ hours; and 4 = 50+ hours). Ne arly 40 % of the programs in this sample were over 20 hours in length. To measure time span, teachers were asked to indicate the total time the professional development activity covered; from 1 = less than one week; 2 = one month or less; 3 = six months o r less; and 4 = more than six months. Roughly 25% of programs were one week or less, while nearly 35% of teachers reported that their programs lasted over six months. Collective participation was measured by whether more than one teacher from a school was required to participate in the program. (This measure was not included in all four studies and has been deleted from this analysis. In studies where it was included, it did not have a significant impact). Opportunity to learn An important feature of the model in Figure 1 is that it makes a distinction between design and implementation features of professional development programs. One of our first steps in each of the four evaluation studies reported here was to conduct analyses of the logic and theory o f action underpinning professional development programs. We worked in close collaboration with program designers to identify the essential and critical features of the professional development models they were using. This included identifying the assump tions about teacher learning on which their models were based, and teasing out the theory of action underlying their programs (how the features of the proposed model linked to each other and how they would lead to change). The purpose was to identify the key design and process features of a professional development program and how it was meant to work. This task was not always as straightforward as it might seem, as some program designers had not been asked to articulate the theory of action underpinnin g their programs before. Designers of professional development activities select from a wide range of strategies to promote professional learning. They often describe the strategies they have chosen in ways that are not necessarily helpful as measures for research purposes. They may use terms such as, hands on, action research, workshops, training sessions, and case methods. What these terms actually mean in terms of teacher learning processes is not always clear. To make the research task eve n more complex, designers often say they use a large number of these strategies in the same activity. So we found it difficult to gain useful measures of actual teacher learning processes by asking program designers about the strategies that characterised their activities. In working with program designers, ACER staff drew extensively from recent research on the critical features of effective professional development programs (Hawley & Valli, 1999). Use was made of other researchers (e.g. Loucks Horsl ey et al., 1998; Sykes, 2002) who provide useful guides to the major types of strategies used to promote professional learning. Heller et al. (2003) and Killion (2003) provide approaches that help to identify the logic underlying programs and how the piec es fit together to promote effective teacher learning. The evaluation team used this research to create an instrument for measuring the extent to which teachers actually experienced certain types of learning opportunities during the professional develop ment program. In developing this instrument ( The Quality of Professional


Education Policy Analysis Archives Vol. 13 No. 10 8 Learning Index ) we used our review of the research literature to identify a number of characteristics of effective professional development. These included: Content focus Follow up Active learning Feedback Collaborative examination of student work Each of these measures is described briefly below. Content focus Recent research (Kennedy, 1998) indicates the importance of what teachers have the opportunity to learn during a professi onal development activity this research suggests that the substance of what teachers learn is more important than the form or structure of the program (e.g. whether programs are school based or not, collaboratively planned or not, extended over time, etc .). In summary, this research indicates that professional learning is more likely to improve student learning outcomes if it increases teachers understanding of the content they teach, how students learn that content and how to represent and convey that content in meaningful ways (Cohen & Hill, 2000). To measure content focus teachers were asked about the emphasis given to four aspects of content: content or subject knowledge, knowledge of how students learn content, knowledge of methods of teaching co ntent and models to illustrate those methods of teaching of that content. 2 Average scores for content focus across the four Programs were about three on the four point scale (Cronbach Alpha = 0.79). Active learning Recent research confirms the importance of importance of teachers being actively engaged in their own learning, but it is the nature of this engagement that seems to matter as much, if not more, than the level. Effective professional development programs draw teachers into an analysis of their cu rrent practice in relation to professional standards for good practice. They also draw teachers into close comparison of what their students are learning in relation to what students of that age and circumstance are capable of learning. To measure activ e learning, teachers were asked about the extent to which a program engaged them actively in reflecting on their practice, in identifying specific areas of their practice that they needed to develop, and gave them opportunities to test new teaching practic es. Average scores for active learning across the Programs were slightly above three on the four point scale (Cronbach Alpha = 0.79). Feedback Feedback on practice has long been recognised as a vital requirement for professional development programs that a im to help teachers develop new skills and integrate them into their practice (Joyce & Showers, 1982). Effective integration of new skills requires programs to have a clear theoretical foundation supported by research, modelling in real settings, and oppo rtunities to practice the new skills and receive feedback from a coach or supporting teacher. Most of the programs we have evaluated recently aimed to help 2 To measure content focus an index was developed based on four items. Teachers responded to these items on a four point scale (1 = no emphasis, 2=minor emphasis, 3= moderate emp hasis, 4 = major emphasis). The scores of each of these items were averaged to give a measure of content focus. A similar process was used to construct all measures


Ingvarson et al.: Impact o f Professional Development Programs 9 teachers learn new skills. However, we found that few participants actually received assistance and feedback in their classrooms during the critical and difficult implementation phase when they were trying out new practices. To measure feedback, teachers were asked about the number of times they received feedback on their teaching from other teachers or people involved in the program; and the number of times their teaching was observed by others involved in the program (e.g. from a mentor, or in a team teaching situation). Average scores for feedback across the Programs were low about 1.5 on the fo ur point scale (Cronbach Alpha = 0.86). Collaborative examination of student work Effective professional development programs lead teachers to examine their students work in relation to external reference points or standards. Hawley and Vallis (1999) revi ew of research rates this feature as a critical component of effective professional learning programs. It has become clear over recent years that teachers gain a great deal of valuable learning from opportunities to examine student work in collaboration w ith colleagues especially their own students work, and in relation to standards for what students should know and be able to do. Collaborative analyses of student work opens up many avenues for teachers to de privatise their practice and learn from eac h other. It also leads to deeper understanding of student learning outcomes and greater discrimination about what counts as meeting those objectives. To measure collaborative examination of student work we developed an index based on the extent to whi ch teachers said they received opportunities to collaborate with colleagues in examining their own students work as well as that of other teachers. Average scores for collaborative examination of student work across the Programs were about 2.5 on the fou r point scale (Cronbach Alpha = 0.84). Follow up Follow up support to teachers during the implementation phase of change has long been identified as an important feature of more effective programs (Fullan, 1982). Perhaps the strongest criticism of many pro fessional development programs over the years has been the lack of built in provision for at the elbow support for teachers in their classrooms as they apply new ideas and skills (Huberman & Miles, 1984) To measure follow up w e developed an index based on the extent to which teachers reported that a program provided time for follow up and ongoing assistance in their school or classroom to help them implement changes advocated in the program and opportunities to practice their n ew learning. Average scores for follow up across the Programs were about 2.5 on the four point scale (Cronbach Alpha = 0.79). Factor analysis confirmed the scales used to measure the five opportunity to learn constructs described above. Details about the psychometric properties of these opportunity to learn variables can be provided on request. Appendix A presents means and confidence intervals for each of the opportunity to learn variables for 26 professional development programs within one of the sta te level programs. The findings show significant variation across the 26 programs in this state program. Readers will notice that some programs such as G are consistently rated highly across the process measures and others such as P are consistently ra ted low. Mediating variables Many professional development programs aim to strengthen professional community in schools in order to enhance the impact of their programs on classroom practice. Therefore, professional community was included in our model as a mediating variable. In


Education Policy Analysis Archives Vol. 13 No. 10 10 measuring, professional community, teachers were asked to respond to items such as: Teachers at my school discuss teaching and learning more with their colleagues Teachers have increased their collaboration in planning, teachin g and assessment activities I have passed ideas I learned from the program on to other teachers in my school. Once again, there was significant variation across programs in the extent to which they stimulated professional community activity in schools (Cr onbach Alpha 0.96). Measures of impact In order to conduct research across several state level programs, based on the conceptual model in Figure 1, we developed an approach to conceptualising and identifying outcomes of professional development programs based on standards for effective teaching (Ingvarson, 1998; 2002). We argued that the quality of impact of a program should not only be measured in terms of whether it meets the developers objectives, but also in terms of the extent to which the program m oves teachers practices towards those associated with research based standards for effective teaching (Ingvarson, 1998; 2002). (The objectives and standards may be much the same, but we found this is not necessarily the case) Four aspects of impact were selected: impact on teachers knowledge ; impact on teachers practice ; impact on student learning outcomes ; and, impact on teacher efficacy These variables are described in more detail below. Knowledg e Teachers were asked to indicate the extent to wh ich their participation in the professional development program had led to increased knowledge of: the content they teach, teaching and learning strategies appropriate to the content they teach, how students learn the content, individual differences amongs t students and how to cater for their needs, how to link assessment into the teaching and learning cycle, classroom organisation and management. Teachers reported their responses to the individual items on a four point scale from strongly agree to strongly disagree. The same scale applied to all four impact measures (Cronbach Alpha = 0.92). Practice Teachers were asked whether, as a result of their participation in the professional development program, they now: make clearer links between their teaching go als and classroom activities; manage classroom structures and activities more effectively; use more effective teaching and learning strategies appropriate to the content that they teach; use more effective teaching and learning strategies appropriate to th e classroom context; use teaching and learning strategies that are more challenging and engaging; are better able to meet the individual learning needs of their students; link assessment into the teaching and learning cycle more effectively; provide more e ffective feedback to their students to support their learning; engage students in higher order thinking;


Ingvarson et al.: Impact o f Professional Development Programs 11 access and use materials and resources more effectively. (Cronbach Alpha = 0.93) Student learning outcomes For example, teachers are asked whether, as a result of the PD program, their students now: have fewer difficulties in understanding what they are being taught; are learning more purposefully; are more actively engaged in learning activities; demonstrate enhanced learning outcomes; access and use ma terials and resources more effectively. (Cronbach Alpha = 0.95) (Precise wording of these items varied slightly from program to program to match specific goals) Teacher efficacy Teachers are asked about the extent to which they agree or disagree with the fo llowing statements as a result of the PD program: My ability to meet the learning needs of my students has expanded My confidence in teaching [subject] has increased. (Cronbach Alpha = 0.85) Factor analyses confirmed that the four impact measures had strong scale characteristics and were sensitive to differences across programs. Details about the psychometric properties of the impact variables can be provided on request. Comparisons of PD programs in terms of impact Appendix B presents means and confidence intervals for each of the impact measures across the same 26 professional development programs as in Appendix A. The findings show the extent of variation across the 26 programs. Readers will note once more that some programs (eg G) are consistently r ated high across the measures while others are consistently rated low (e.g. JJ). Appendix C provides a summary of the professional development strategy for Program G. It is clear from this summary how the designers have incorporated most of the featu res characteristic of programs consistent with research on effective professional learning. Especially noteworthy is the inclusion of feedback using videos to promote reflection and follow up visits by the facilitator. Analysis Separate analyses of the d ata were conducted for each of the four state level programs. This paper brings the findings of these four independent studies together. As a first step in each analysis, a blockwise regression analysis was conducted using the SPSS PC procedure regressio n This procedure is based upon a least squares algorithm to estimate the strength of the linear relationship between the dependent variable and a set of independent variables. The order in which these variables were entered into the equation was determin ed by the theory underlying the research (as summarised in Figure 1). There were six control, or background (exogenous), variables in this model, and three blocks of


Education Policy Analysis Archives Vol. 13 No. 10 12 intervening (endogenous) variables: structural features, learning processes and professio nal community. (An example of the results a full analysis of a state level program is shown in Appendix D). For the purposes of this paper, however, these full tables have been limited to summarising the regression coefficients for the structural and opp ortunity to learn variables. Table 1 shows the standardised regression coefficients and significance levels for each of the structural and opportunity to learn variables in the model, across the four state level programs. Significant relationships are indicated in bold type. The use of standardised co efficients permits easy comparison of the strength of associations within the model. For example, a standardised beta coefficient of 0.27 is three times as strong in its effect as one of 0.09. When exam ining these effects it is important to remember that they are net of the effects of other variables in the model. The regression analysis thus shows the unique contribution that each variable makes to changes in the dependent variable. Findings Table 1 provides an amalgamation of findings across the four evaluation studies. 3 These four studies, in effect, are replications of the same experiment. When findings are replicated across several studies, they can be reported with greater confidence. This a malgamation of findings has enabled a more rigorously testing of our conceptual model (of factors affecting the impact of professional development programs). Using this model, we have investigated relationships between process features of professional dev elopment programs, such as active learning and follow up, and outcome measures, such as teacher practice and efficacy, across the four studies. Table 1 shows the proportion of variance explained by the model (R 2 ) across the four studies. Although there is some variation in the explained variance, overall there is a degree of consistency across the studies For example, the full model accounted for around 48% of the variance in the dependent variable teacher efficacy in Program A, 51 % in Program B, 45% in Program C and 69% in Program D which meant that several features in our model were reasonably good predictors of whether teachers would rate a professional development program as effective. One of the main findings from this study is that, among all the variables in the model (Figure 1), the block of opportunity to learn or process variables had the largest effect on individual program outcomes. When we look at the reported impact of these variables across the four studies, as summarised in Table 1, a reasonably consistent pattern emerges. 3 Reports of each evaluation study can be found at ams/teaching.html


Ingvarson et al.: Impact o f Professional Development Programs 13 Table 1 Relationships between PD characteristics & impact measures across four Programs PROGRAM A Prof Cmty Knowledge Practice Student outcome E fficacy Total h rs for activity 0.01 0.68 0.04 0.30 0.03 0.44 0.03 0.53 0.00 0.96 Span of time 0.15 0.00 0.05 0.20 0.08 0.07 0.02 0.73 0.04 0.40 Content 0.15 0.00 0.19 0.00 0.20 0.00 0.16 0.00 0.15 0.00 Active 0.00 0.90 0.17 0.00 0.10 0.04 0.04 0.48 0.13 0.00 Follow up 0.25 0.00 0.09 0.03 0.06 0.15 0.04 0.43 0.19 0.00 Collaboration 0.21 0.00 0.08 0.05 0.00 0.95 0.00 0.93 0.15 0.00 Feedback 0.00 0.93 0.05 0.11 0.02 0.62 0.12 0.00 0.09 0.01 Prof community 0.10 0.01 0.27 0.00 0.00 0.96 0.11 0.01 Knowledge 0.17 0.00 0.2 9 0.00 0.12 0.00 Practice 0.23 0.00 0.10 0.02 Student outcomes 0.20 0.00 R squared (adjusted) 0.36 0.23 0.34 0.35 0.48 PROGRAM B Total h rs for activity 0.02 0.58 0.04 0.19 0.03 0.20 0.07 0.02 0.03 0.39 Span of time 0.01 0.66 0.04 0.22 0.06 0.03 0.03 0.37 0.06 0.06 Content 0.11 0.01 0.39 0.00 0.09 0.01 0.13 0.00 0.13 0.00 Active 0.00 0.91 0.11 0.01 0.12 0.00 0.03 0.48 0.20 0.00 Follow up 0.29 0.00 0.09 0.03 0.09 0.01 0.08 0.02 0.03 0.40 Collaborat ion 0.23 0.00 0.05 0.22 0.10 0.00 0.04 0.24 0.12 0.00 Feedback 0.16 0.00 0.03 0.26 0.02 0.44 0.06 0.05 0.09 0.00 Prof community 0.12 0.00 0.15 0.00 0.11 0.00 0.07 0.03 Knowledge 0.56 0.00 0.01 0.81 0.00 0.98 Practice 0. 56 0.00 0.18 0.00 Student outcomes 0.33 0.00 R squared (adjusted) 0.36 0.39 0.60 0.51 0.51 PROGRAM C Total h rs for activity 0.06 0.48 0.01 0.91 0.07 0.30 0.12 0.08 0.08 0.32 Span of time 0.12 0.13 0.01 0.90 0.09 0 .17 0.15 0.03 0.04 0.65 Content 0.01 0.87 0.21 0.00 0.00 0.97 0.01 0.88 0.09 0.24 Active 0.00 0.95 0.27 0.00 0.18 0.01 0.08 0.24 0.05 0.55 Follow up 0.27 0.00 0.15 0.04 0.14 0.06 0.08 0.29 0.01 0.88 Collaboration 0.33 0.00 0.08 0.28 0.07 0.29 0.12 0. 11 0.10 0.23 Feedback 0.15 0.03 0.06 0.32 0.08 0.17 0.09 0.15 0.06 0.42 Prof community 0.24 0.00 0.17 0.02 0.06 0.39 0.12 0.17 Knowledge 0.54 0.00 0.28 0.00 0.09 0.39 Practice 0.48 0.00 0.16 0.14 Student outcomes 0.34 0.00 R squared (adjusted) 0.36 0.51 0.59 0.58 0.45 PROGRAM D Total h rs for activity 0.08 0.15 0.04 0.52 0.06 0.14 0.06 0.25 0.03 0.50 Span of time for activity 0.12 0.01 0.09 0.07 0.00 0.93 0.00 0.92 0.02 0.7 0 Content 0.05 0.47 0.33 0.00 0.00 0.92 0.09 0.19 0.01 0.85 Active 0.08 0.20 0.08 0.17 0.15 0.00 0.02 0.72 0.16 0.00 Follow up 0.50 0.00 0.23 0.00 0.02 0.68 0.10 0.20 0.03 0.67 Collaboration 0.31 0.00 0.01 0.87 0.04 0.44 0.02 0.72 0.03 0.58 Feedba ck 0.08 0.17 0.08 0.14 0.01 0.76 0.11 0.05 0.03 0.49 Prof community 0.14 0.03 0.09 0.05 0.11 0.09 0.01 0.87 Knowledge 0.72 0.00 0.16 0.08 0.02 0.80 Practice 0.65 0.00 0.26 0.00 Student outcomes 0.52 0.00 R s q. (adjusted) 0.49 0.51 0.76 0.58 0.69


Education Policy Analysis Archives Vol. 13 No. 10 14 The most important influence on reported impact on knowledge, perhaps unsurprisingly, is the extent to which the program focused on content, as defined above. Across all four state level Programs, th e relationship between content focus and impact on knowledge is strong (Program A .19, Program B .39, Program C .21, Program D .33). The relationship between follow up and reported impact on knowledge is also significant across all four studies and the o pportunity for active learning is significant in three of the studies. The most important influence on reported impact on practice apart from knowledge is entered into the equation, is the extent to which individual programs provide many opportunities for active learning and reflection on practice Across all four state level Programs, the relationship between active learning and impact on practice is significant (Program A .10, Program B .12, Program C .18, Program D .15). When we look at reported impact on knowledge and practice together, the significance of professional community as a mediating variable becomes apparent. Table 1 indicates that relationships between professional community and the reported level of impact on knowledge and practice were significant in all four studies. The extent to which a professional development program influences knowledge and practice, as reported by teachers, is enhanced by the extent to which that program also strengthens the level of professional community i n the school; that is, the extent to which it increases opportunities for teachers to talk about the specifics of their teaching practice and student learning, share ideas and support each other as they attempt to implement ideas from the professional deve lopment program. The extent to which programs influenced the level of professional community activity was enhanced to the extent that their designers built in active learning processes, follow up and opportunities for collaborative examination of student work. Table 1 shows some interesting features when we look at reported impact on student learning outcomes and efficacy The level of content focus continues to be important in Programs A and B, the level of active learning is strongly related to teach er efficacy in three of the Programs and, for Programs A and D, feedback starts to have a significant effects over and above the strong effects of knowledge and practice. The fact that the influence of active learning carries through to influence teacher efficacy, net of the effects of other variables in the model, is particularly noteworthy. It suggests that this feature is having a pervasive and generative influence on factors that increase teachers confidence and ability to meet student needs than mak ing specific changes in practice alone. The opportunity to learn features in this model had a significant direct effect on teacher knowledge. As teacher knowledge was found, in turn, to be strongly related to impact on practice, these features also had s ignificant indirect effects on practice, student learning outcomes and teacher efficacy. As mentioned above, there were marked differences in the extent to which programs incorporated the opportunity to learn variables into their design. On a four point scale the average score was about 3 for content focus and active learning 2.5 for follow up and collaboration and 1.5 for feedback Not reported in Table 1 are the strong relationships between content focus, active learning, collaborative examination of student work and follow up across the four studies. These results suggest that programs with an emphasis on the subject matter that is being taught, how it is learned and how to teach it, tend to facilitate more active school based professional learning processes. As might be expected, this study found that programs that did build opportunities for follow up support into their design were more likely to provide opportunities for teachers to receive feedback as they tried out new skills.


Ingvarson et al.: Impact o f Professional Development Programs 15 Not included i n this paper are findings related to control variables and structural features of programs, as set out in the model in Figure 1. In summary, these showed that for most Programs the level of school support influenced the extent of active learning follow up and feedback and was related to the level of professional community activity resulting from programs. As these aspects of opportunity to learn are significantly related to the reported level of impact on knowledge, this indicates that the level of sch ool support, as defined above, has substantial, though indirect effects on the extent to which program outcomes are achieved. Though not a design feature of most professional development programs in this study, the level of school support comes through the analysis as an important enabling condition with a significant shaping influence on the opportunities to learn that teachers experience. By and large the duration of programs (contact hours) did not have direct effects on the impact measures in these st udies. However, it was related significantly to the reported level of content focus, active learning, collaboration and feedback (0.15). As these opportunity to learn features were found to have significant direct effects on reported impact levels, durat ion emerges as an important structural feature of programs. This study indicates the importance of giving program designers the time and resources that will enable them to incorporate these learning opportunities into their programs. These studies indicat e that time span enabled programs to strengthen professional community activity, which, in turn, increased the likelihood that programs would have significant effects on teacher knowledge and practice. The time span of programs had a significant effect on the amount of time program participants reported they spent meeting informally with other participants in related activities, such as joint lesson planning and developing curriculum materials. These structural features of contact hours and time span both have substantial, though indirect, effects on program outcomes. The main message from these studies is that, among all the variables in the conceptual model (Figure 1), the block of opportunity to learn or process variables had the largest effect on progr am outcomes. Of this set of variables, the ones with the most consistent effects were content focus (especially those that focused on how students learn the content and on methods to teach the content), active learning, and follow up. The fact that the i nfluence of active learning carries through to influence teacher practices and teacher efficacy, net of the effects of other variables in the model, is particularly noteworthy. It suggests that this feature is having a pervasive and generative influence o n factors that increase teachers confidence and ability to meet student needs than making specific changes in practice alone. Discussion This study indicates that the most effective programs, in terms of reported impact, had profiles consistent with rese arch on effective professional development (Hawley & Valli, 1999). They were rated highly by teachers across all five opportunity to learn measures in the conceptual model (Figure 1). They provided opportunities for teachers to focus on what students wer e to learn and how to deal with the problems students may have in learning that subject matter. They focused on research based knowledge about student learning of content. They included opportunities for teachers to examine student work collaboratively and in relation to standards for what the students in question should know and be able to do. They led teachers to actively reflect on their practice and compare it with high standards for professional practice. They engaged them in identifying what the y needed to learn, and in planning the learning experiences that would help them meet those needs. They provided time for teachers to test new teaching methods and to receive follow up


Education Policy Analysis Archives Vol. 13 No. 10 16 support and coaching in their classrooms as they faced problems of imp lementing changes. They included activities that led teachers to deprivatise their practice and gain feedback about their teaching from colleagues. This study is similar in methodology to the study by Garet et al. (2001). There are some differences in our model, reflecting the different context for the professional development programs included in the study. In the AGQTP, programs had to target, for example, experienced primary and secondary school teachers (10+ years) and areas such as literacy, numera cy, science and technology. Individual professional development programs, within these Programs, generally had to go through an extensive period of development and quality control mechanisms and, consequently, were generally rated highly by teachers in te rms of the impact measures. Our analysis was restricted to programs for which we had responses from at least ten teachers and teachers were not surveyed until at least three months had elapsed since the program. Garets study draws on their evaluation of the Eisenhower Professional Development Program directed specifically at mathematics and science teachers. In selecting programs, they drew on a national probability sample of school districts and others who had received Eisenhower funds. Their surv ey is based on a nationally representative sample of about 1500 teachers who had attended these activities. They sub sampled two teachers from each activity. By and large the findings from the two studies are similar as well. Content focus and active le arning have a significant impact on knowledge in both studies. The ACER study includes separate measures for follow up, feedback and collaborative examination of student work, but the Garet study includes these in its overall index of active learning. We did not include a measure like Garets coherence as it did not seem applicable to the Australian context where programs would not receive funding unless they were consistent with Commonwealth objectives and state level curriculum standards. However, we did include a measure of the extent to which a professional development program facilitated the development of professional community at the school level as part of its strategy. This turned out to be a significant mediating variable in the ACER study. The strong relationship between content focus and reported impact on practice has been noted. This supports findings from Joyce and Showers (1982), Cohen and Hill (200), and reviews by Kennedy (1998) and Hawley and Valli (1999); that a strong knowledge ba se and a clear theoretical rationale grounded in research are necessary conditions for effective programs. An increased sense of teacher efficacy is, not surprisingly, dependent on the extent to which teachers think their practises have improved (increase d competence) and evidence that student learning outcomes have improved as a result. The strongest influence on teachers reported levels of impact on efficacy in all four studies was the extent to which teachers saw that a program had had an impact on the ir students learning outcomes. This finding echoes earlier research by Gusky (1985), which found that the more effective strategy is to ask teachers to try out new practices and see the effects on their students, rather than trying to change attitudes fi rst in the hope that this will lead to change in practice. Programs that model effective practice and invite teachers to try them out tend to be more successful than programs that devote resources primarily to changing attitudes first. The opportunity t o learn variables that appeared to have had the least influence in this study were feedback and collaborative examination of student work despite strong evidence for their importance in other research studies. However, as reported above, few program desi gners in this study built opportunities for feedback about practice into their strategies. Teachers


Ingvarson et al.: Impact o f Professional Development Programs 17 indicated that opportunities to receive feedback as they tried new practices in the classroom were rare. Consequently there was little variation across pr ograms in this variable and the study was not able to provide a fair test of the effect that feedback could have on teachers practice. Similarly, few programs provided opportunities for collaborative examination of student work. Although this study does not provide a convincing case, it would not be appropriate to assume these two features were not important, given other research (Hawley & Valli, 1999). This study does indicate that program designers may not be incorporating these features to the extent that is needed to support implementation and sustained change in practice. The relative success of programs also depended on the extent to which programs were extended in time, and planned so that they included activities that strengthened interaction an d collaboration in the school the level of professional community activities. They linked to other programs in the participants schools designed to improve learning. The fact that both span of time and contact hours show significant, though independent effects in this analysis indicates that both aspects of duration are important in the design of effective professional learning activities. This finding is consistent with the research of Garet et al. (2001) who claim that, Professional development is likely to be of higher quality if it is both sustained over time and involves a substantial number of hours (p. 933). Other clear messages come through this analysis across the Programs. The role of follow up is noteworthy. The level of follow up was found to increase significantly the extent to which teachers reported a sense of increased knowledge, perhaps reflecting the critical role that at the elbow coaching and support in classrooms plays in learning new skills and putting them into practice. This kind of support was built into the more effective programs in our study. The findings from this study are also consistent with long standing research findings about the importance of school context. The pre existing level of support for profession al development in a school has a significant indirect effect on the outcomes of programs. It follows from this research that is not enough to provide well designed professional development programs from outside the school. Policy makers and school admini strators need to give equal attention to building the conditions that will enable schools to provide fertile ground for professional learning on an ongoing basis and as a routine part of the job. This study indicates that a substantial level of profession al community is vital to significant change. The key ingredients here are time to think, analyse and talk about the specifics of what is going on in classrooms and what students are doing and learning. Effective school administrators in strong professional communities expect evidence of professional development and act in ways that demonstrate they value teacher learning. The findings from this cross program analysis reflect findings from other research on professional development and challenges that polic y makers have faced for many years. These studies often find a considerable gap between the conditions that research indicates are optimal for professional learning and those that are provided. The capacity of policy makers to provide funding sufficient to ensure their incorporation into most professional development programs can be constrained depending on the level of available funding for professional development. While the research has long indicated that there are no short cuts to significant and su stainable change at the classroom level, short cuts often have to be taken. However, money spread thinly, when it comes to professional development is unlikely to produce significant change. Where significant change is sought, it may be wiser to involve fewer teachers than produce less significant change among many.


Education Policy Analysis Archives Vol. 13 No. 10 18 Not only policy makers, but also professional development providers face a challenge in designing programs consistent with research in this field. This cross Program study reinforces the c entral importance of building follow up support into the design of professional development programs. However, this feature of professional development is labour intensive and time consuming and, consequently, expensive. The same applies to feedback Perhaps one of the most significant findings in this study across the eighty programs was how rarely designers built in opportunities for feedback and coaching in the workplace, despite research on their centrality to learning new and complex skills. Refl ection on practice and the development of understanding about the change is unlikely to be optimal without sources of timely and insightful feedback on what one is doing. Ensuring opportunities for every teacher to receive at the elbow support and coach ing during the difficult phase of implementing significant change in the classroom is a feature of effective programs. We suspect that one of the most important challenges emerging from this study, especially for people with responsibility for ensuring pr ofessional development budgets are spent wisely, is to develop funding guidelines that ensure that programs designers include greater opportunities for participants to benefit from rich and frequent feedback. Similarly, time to bring teachers together in the workplace to examine student work and to provide opportunities for feedback needs to be built into current conceptions of teachers work. These studies indicate that teachers work needs to be organised on the assumption that it is professional work; that is, work that requires an appropriate balance between up front practice and back room collegial analysis and reflection on practice in the light of standards for student learning and professional practice. Limitations As described above, our approac h to evaluation was based primarily on teacher self report data. Given the time frame and the level of resources usually allocated to evaluations of professional development programs, there is often little opportunity to gather first hand evidence about c hanges in teacher knowledge, practice, efficacy and students learning outcomes. We are more confident about the measures of impact on practice than impact on student learning outcomes. The graphs in Appendix C show greater variation for impact on practic e than impact on student outcomes. However, recent studies (e.g. Mayer, 2001) indicate that it is reasonable, in certain circumstances, to place a certain level of confidence in surveys that rely on teachers reports about their practice. Reliability of t hese self report data increases as outcome measures become more specific, as those used here in the ACER approach. Also, teachers are not reluctant to speak their minds frankly when it comes to assessing the value of professional development programs. Th ere is little reason to think that their responses might be biased one way or another, or any desire to please, especially when, in studies such as the above, they are contacted several months at least after the programs have finished. Note Paper presented to the Annual Meeting of the American Educational Research Association, San Diego, April 12 16, 2004 The projects reported here were funded by the Australian Government Department of Education, Science and Training, the Catho lic Education Commission of Victoria, the NSW Quality Teacher Programme, and the Northern Territory Department of Education


Ingvarson et al.: Impact o f Professional Development Programs 19 References Cohen, D., and Hill, H. (2000). Instructional policy and classroom performance: The mathematics reform in California. Teachers College Record, 102 (2), 294 343. Fullan, M. (2001). The new meaning of educational change. 3 rd Edition. New York: Teachers Colle ge Press. Garet, M., Porter, A., Desimone, L., Birman, B., & Yoon, K. (2001). What Makes Professional Development Effective? Results From a National Sample of Teachers. American Educational Research Journal, 38 (3), 915 945. Guskey, T.R. (1985). Staff de velopment and the process of teacher change. Educational Researcher 15 (5), 5 12. Guskey, T.R., & Sparks, D. (2002). Linking professional development to improvements in student learning. Paper presented at the annual meeting of the American Educational Re search Association, April 2002 Hawley, D., & Valli, L. (1999). The essentials of effective professional development: A new consensus. In L. Darling Hammond & G. Sykes (Eds.) Teaching as the Learning Profession. Handbook of Policy and Practice. (pp. 127 150) San Francisco: Jossey Bass Publishers. Heller, J.I., Daehler, K.R., & Shinohara, M. (2003). Connecting all the pieces Journal of Staff Development. 24 (4), 36 41. Huberman, M., & Miles, M. (1984). Innovation up close New York: Plenum Press. Ingva rson, L.C. (1998). Teaching standards: foundations for the reform of professional development. In A. Hargreaves, A. Lieberman, M. Fullan & D. Hopkins (Eds). International handbook of educational change. Dordrecht: Kluwer. Ingvarson, L.C. (2002). Building a Learning Profession Paper No 1, Commissioned Research Series, Australian College of Education. Canberra: Australian College of Education. ( Joyce, B., & Showers, B. (1995). Student achievement thr ough staff development: Fundamentals of school renewal Second Edition. White Plains, NY: Longman. Kennedy, M. (1998). Form and substance in in service teacher education (Research monograph No. 13). Arlington, VA: National Science Foundation. Killio n, J. (2003). Solid footwork makes evaluation of staff development a song. Journal of Staff Development 24 (4), 14 21. Loucks Horsley, S., Hewson, P., Love, N., and Stiles, K. (1998). Designing professional development for teachers of mathematics and s cience. Thousand Oaks, CA: Corwin Press. Mayer, D. (1999). Measuring instructional practice: Can policymakers trust survey data? Educational Evaluation and Policy Analysis 21 (1), 29 45.


Education Policy Analysis Archives Vol. 13 No. 10 20 Supovitz, J. (2001). Translating teaching practice into improv ed student achievement. In S. Fuhrman (Ed.). From the capitol to the classroom: Standards based reforms in the states. The one hundredth yearbook of the National Society for the Study of Education, Part Two (pp. 81 98). Chicago: University of Chicago P ress. Sykes, G. (2002). Professional development for teachers: principles, practices and contexts. Paper prepared for the Learning First Alliance (Draft). Wilson, S., & Berne, J. (1999). Teacher learning and the acquisition of professional knowledge: An examination of research on contemporary professional development. In A.Iran Nejad and P. D. Pearson (Eds.). Review of Research in Education, Vol. 24 (pp.173 209). Washington, DC: American Educational Research Association. About the Authors Lawrence Ingvarson Research Director, Adrian Beavis Principal Research Fellow Marion Meiers Senior Research Fellow Teaching and Learning Program Australian Council for Educational Research The Teaching and Learning research program at ACER includes pr ojects that examine educational policy as it relates to teacher quality, teaching practice, teacher working conditions and student achievement. This includes research on teacher preparation, induction and professional development. Current projects also inc lude the development of teaching standards and new methods for assessing teacher performance for registration and professional certification. ACER 19 Prospect Hill Road Camberwell Australia 3124 E mail: Website:


Ap pendix A Opportunity to Learn Means and confidence intervals for 26 professional development activities Content Focus Content focus 1.5 2 2.5 3 3.5 4 4.5 JJ MM P LL BB V DD F AA NN CC L X T B FF A II D N J I O GG K G Professional Development Program Collaborative examination of student work Collaborative examination of student work 1 1.5 2 2.5 3 3.5 4 JJ I P BB LL B L V NN X MM J DD II A AA O GG CC N T K F G FF D Professional Development Program Active learning Active learning 2 2.5 3 3.5 4 4.5 P V BB LL DD MM B F A JJ L NN AA X J CC I FF N K T II GG O D G Professional Development Program Opportunity to Learn Measures


Education Policy Analysis Archives Vol. 13 No. 10 22 Feedback Feedback 0.5 1 1.5 2 2.5 3 I P DD JJ B L BB A LL FF F NN N MM D X I GG J AA V CC T K O G Professional Development Program Follow up Follow up 1.5 2 2.5 3 3.5 4 P DD I MM JJ LL B A I CC V L BB J FF F AA N NN X O K T GG D G Professional Development Program


Appendix B Measures of Impact Means and confidence intervals for 26 professional development activities Impact on knowledge Knowledge 1.5 2 2.5 3 3.5 4 JJ CC J DD AA P LL V MM F B L A BB N D I II X GG NN T K G FF O Professional Development Program Impact on practice Practice 1.5 2 2.5 3 3.5 4 JJ CC DD AA V MM I BB X LL P J B T FF F NN GG L A II N O G K D Professional Development Program Impact on professional community Professional Community 1 1.5 2 2.5 3 3.5 4 JJ P DD MM II J I L CC LL B K BB V NN AA T X O A F G FF GG D N Professional Development Program Impact Measures


Education Policy Analysis Archives Vol. 13 No. 10 24 Impact on student outcomes Student Outcomes 1.5 2 2.5 3 3.5 4 JJ DD F B P MM AA II CC FF L I NN X LL V J BB T D A O N GG G K Professional Development Program Impact on teacher efficacy Teacher Efficacy 1.5 2 2.5 3 3.5 4 P JJ CC DD L F AA FF MM B V LL II X A D BB T GG N I J NN K O G Professional Development Program


Appendix C PROJECT G: PROFILE Teaching and Learning Literacy BACKGROUND Program G was developed in response to K 6 classroom teachers who were asking for support in their teaching of literacy. The sk ills they had identified they needed were similar to those provided to Reading Recovery teachers, but adapted for regular classroom teaching. A trial program was developed and evaluated by a group of 14 teachers over 12 months, before going to scale. The program advocated a functional approach to learning about literacy teaching and used a approach adapted from the Reading Recovery training model. STRATEGY The professional learning strategy for teachers in this project was the provision of 10 workshops over one year. Using an immersion in inquiry case study model, teachers examined student work to develop standards and continua in reading and writing. Teacher participants joined the course with at least one other teacher from their school to form sch ool partnerships in learning. This created a learning community of participants, built over the year and continuing through ongoing contact. It also was designed to contribute to professional development in the area of leadership, with teachers working i n schools as leaders; in addition, some teachers became facilitators of groups within each phase of the course. Opportunity was provided to the participants to negotiate the agenda. Teachers were required to use videos to capture their teaching for obs ervation, discussion and reflection. This entailed coaching and mentoring, with school visits by the facilitator and joint video watching to set goals, guide practice and evaluate practice. The facilitator communicated through the training workshops, tele conferences and through intranet discussion. Technology schools, in particular, were visited for review and inspiration regarding the diverse ways technology was used to support learning. Participants were expected to share their experience and learning from the course with their non participant colleagues in their schools. The course aimed to reduce the notion and practice of isolated learning and to develop school culture change. A number of key beliefs underpinned this project. These included the nee d to develop and provide professional development according to the principles of effective professional development for example, that both the participants and the school make the course a priority; that there is a balance between theory and practice; th at learning is both formative and reflective, and that teachers can work with and learn from their colleagues and their students in trialling new learning.


Education Policy Analysis Archives Vol. 13 No. 10 26 Appendix D Program B: Relationship between background variables, structural features, opportunity to learn, professional community in the school, and teacher knowledge, teacher practice, student learning and teacher efficacy Content Active Follow up Collaboratio n Feedbac k Professiona l Community Knowled ge Practice Student Outcom es Teacher Efficacy Gen der (F = 0 M=1) 0.01 0.02 0.08 0.00 0.03 0.01 0.01 0.01 0.02 0.04 School size 0.04 0.01 0.01 0.04 0.00 0.03 0.04 0.01 0.07 0.01 Primary = 1 else = 0 0.16 0.02 0.10 0.00 0.06 0.02 0.09 0.02 0.13 0.05 Volunteered =1 Directed=0 0.01 0.06 0 .01 0.06 0.01 0.00 0.04 0.05 0.06 0.02 School Sector 1=State 0=No 0.06 0.02 0.09 0.15 0.03 0.14 0.01 0.03 0.05 0.02 PD Support in School 0.14 0.05 0.12 0.03 0.06 0.13 0.05 0.01 0.05 0.03 Duration Hours 0.21 0.12 0.07 0.04 0.05 0.01 0.05 0.05 0.07 0.02 Span months 0.11 0.02 0.03 0.13 0.05 0.01 0.04 0.07 0.03 0.05 Sufficient Time 0.19 0.11 0.09 0.02 0.04 0.03 0.06 0.01 0.04 0.06 Content 0.55 0.08 0.28 0.07 0.10 0.35 0.11 0.14 0.11 Active 0.47 0.14 0.00 0.00 0.09 0.09 0.02 0.22 Follow u p 0.30 0.23 0.25 0.09 0.10 0.10 0.00 Collaboration 0.20 0.21 0.07 0.11 0.03 0.10 Feedback 0.16 0.04 0.02 0.06 0.09 Professional Community 0.12 0.15 0.11 0.08 Knowledge 0.57 0.04 0.02 Practice 0.55 0.17 Student Outco mes 0.31 R square Adjusted 0.20 0.42 0.43 0.49 0.15 0.39 0.38 0.59 0.51 0.50


Education Policy Analysis Archives Editor: Sherman Dorn University of South Florida Production Assistant: Chris Murrell, Arizona State University General questions about appropriateness of topics or particular articles may be addressed to the Editor, Sherman Dorn, epaa editor@shermando rn .com EPAA Editorial Board Michael W. Apple University of Wisconsin David C. Berliner Arizona State University Gre g Camilli Rutgers University Linda Darling Hammond Stanford University Mark E. Fetler California Commission on Teacher Crede ntialing Gustavo E. Fischman Arizona State Univeristy Richard Garlikov Birmingham, Alabama Gene V Glass Arizona State University Thomas F. Green Syracuse University Aimee Howley Ohio University Craig B. H owley Appalachia Educational Laboratory William Hunter University of Ontario Institute of Technology Patricia Fey Jarvis Seattle, Was hington Daniel Kalls Ume University Benjamin Levin University of Manitoba Th omas Mauhs Pugh Green Mountain College Les McLean University of Toronto Heinrich Mintrop University of California, Berkeley Michele Moses Arizona State University Gary Orfield Harvard University Anthony G. Ru d Jr. Purdue University Jay Paredes Scribner University of Missouri Michael Scriven Western Michigan University Lorrie A. Shepard University of Colorado, Boulder Robert E. Stake University of Illinois UC Kevin Welner University of Colorado, Boulder Terrence G. Wiley Arizona State University John Willinsky University of British Columb ia


Education Policy Analysis Archives Vol. 13 No. 10 28 Archivos Analticos de Polticas Educativas Associate Editors Gustavo E. Fischman & Pablo Gentili Arizona State University & Universidade do Estado do Rio de Janeiro Founding Associate Editor for Spanish Language (1998 2003) Roberto Rodrgue z Gmez Editorial Board Hugo Aboites Universidad Autnoma Metropolitana Xochimilco Adrin Acosta Universidad de Guadalajara Mxico Claudio Almonacid Avila Universidad Metropolitana de Ciencias de la Educacin, Chile Dalila Andrade de Oliveira Universidade Federal de Minas Gerais, Belo Horizonte, Brasil Alejandra Birgin Ministerio de Educacin, Argentina Teresa Bracho Centro de Inves tigacin y Docencia Econmica CIDE Alejandro Canales Universidad Nacional Autnoma de Mxico Ursula Casanova Arizona State Universit y, Tempe, Arizona Sigfredo Chiroque Instituto de Pedagoga Popular, Per Erwin Epstein Loyola University, Chicago, Illinois Mariano Fernndez Enguita Universidad de Salamanca. Espaa Gaudncio F rigotto Universidade Estadual do Rio de Janeiro, Brasil Rollin Kent Universidad Autnoma de Puebla. Puebla, Mxico Walter Kohan Universidade Estadual do Rio de Janeiro, Brasil Roberto Leher Uni versidade Estadual do Rio de Janeiro, Brasil Daniel C. Levy University at Albany, SUNY, Albany, New York Nilma Limo Gomes Universidade Federal de Minas Gerais, Belo Horizonte Pia Lindquist Wong California State University, Sacramento, California Mara L oreto Egaa Programa Interdisciplinario de Investigacin en Educacin Mariano Narodowski Universidad Torcuato Di Tella, Argentina Iolanda de Oliveira Universidade Federal Fluminense, Brasil Grover Pango Foro Latinoamericano de Polticas Educativas, Per Vanilda Paiva Universidade Estadual do Rio de Janeiro, Brasil Miguel Pereira Catedratico Universidad de Granada, Espaa Angel Ignacio Prez Gmez Universidad de Mlaga Mnica Pini Universidad Naciona l de San Martin, Argentina Romualdo Portella do Oliveira Universidade de So Paulo Diana Rhoten Social Science Research Council, New York, New York Jos Gimeno Sacristn Universidad de Valencia, Espaa Daniel Schugurensky Ontario Institute for Studies in Education, Canada Susan Street Centro de Investigaciones y Estudios Superiores en Antropologia Social Occidente, Guadalajara, Mxico Nelly P. Stromquist University of Southern California, Los Angeles, California Daniel Suarez Laboratorio de Politicas Publicas Universidad de Buenos Aires, Argentina Antonio Teodoro Universidade Lusfona Lisboa, Carlos A. Torres UCLA Jurjo Torres Santom Universidad de la Corua, Espaa

xml version 1.0 encoding UTF-8 standalone no
mods:mods xmlns:mods http:www.loc.govmodsv3 xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govmodsv3mods-3-1.xsd
mods:relatedItem type host
mods:identifier issn 1068-2341mods:part
mods:detail volume mods:number 13issue 10series Year mods:caption 20052005Month January1Day 2929mods:originInfo mods:dateIssued iso8601 2005-01-29