Educational policy analysis archives

Educational policy analysis archives

Material Information

Educational policy analysis archives
Arizona State University
University of South Florida
Place of Publication:
Tempe, Ariz
Tampa, Fla
Arizona State University
University of South Florida.
Publication Date:


Subjects / Keywords:
Education -- Research -- Periodicals ( lcsh )
non-fiction ( marcgt )
serial ( sobekcm )

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
E11-00257 ( USFLDC DOI )
e11.257 ( USFLDC Handle )

Postcard Information



This item has the following downloads:

Full Text
xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam a22 u 4500
controlfield tag 008 c20029999azu 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E11-00257
0 245
Educational policy analysis archives.
n Vol. 10, no. 9 (January 28, 2002).
Tempe, Ariz. :
b Arizona State University ;
Tampa, Fla. :
University of South Florida.
c January 28, 2002
Confusing the messenger with the message : a response to Bolon / Victor L. Willson [and] Thomas Kellow.
x Research
v Periodicals.
2 710
Arizona State University.
University of South Florida.
1 773
t Education Policy Analysis Archives (EPAA)
4 856

xml version 1.0 encoding UTF-8 standalone no
mods:mods xmlns:mods http:www.loc.govmodsv3 xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govmodsv3mods-3-1.xsd
mods:relatedItem type host
mods:identifier issn 1068-2341mods:part
mods:detail volume mods:number 10issue 9series Year mods:caption 20022002Month January1Day 2828mods:originInfo mods:dateIssued iso8601 2002-01-28


1 of 4 Education Policy Analysis Archives Volume 10 Number 9January 28, 2002ISSN 1068-2341 A peer-reviewed scholarly journal Editor: Gene V Glass College of Education Arizona State University Copyright 2002, the EDUCATION POLICY ANALYSIS ARCHIVES Permission is hereby granted to copy any article if EPAA is credited and copies are not sold. Articles appearing in EPAA are abstracted in the Current Index to Journals in Education by the ERIC Clearinghouse on Assessment and Evaluation and are permanently archived in Resources in Education .Confusing the Messenger with the Message: A Response to Bolon Victor L. Willson Texas A&M University Thomas Kellow University of Houston Citation: Willson, V.L. & Kellow, T. (2002, January 28). Confusing the messenger with the message: A response to Bolon. Education Policy Analysis Archives 10 (9). Retrieved [date] from The conclusions by Bolon (2001) based on the relati onship between per capita income and school mean grade 10 mathematics scores in Massachusetts and on instability in year-to-year me an school scores are criticized by us. Our concerns focus on the uninter pretable covariation of economic condition with test performance and the li mitations in interpreting cross-time variability. We agree with Bolon's conclusions but consider the methodology employed inadequate to support them. We suggest alternative requirements and discuss our ow n previous efforts in this area.


2 of 4 In an analysis of the Massachusetts graduation exam ination, Bolon (2001) examined the aggregate grade 10 mathematics test scores for 47 h igh schools and the demographic characteristics of the communities in which they we re situated. From several data analyses, Bolon determined that since the best sing le predictor of mean high school score was community per capita income, "The state is treating scores and ratings as though they were precise educational measures of high significance. A review of thenth-grade mathematics test scores from academic high schools in metropolitan Boston showed that statistically they are not." Further, when removing the variability due to per c apita income, "Large uncertainties in residuals of school-average d scores, after subtracting predictions based on community income, tend to make the scores ineffective for rating performance of schools. Large uncertaint ies in year-to-year score changes tend to make the score changes ineffective for measureing performance trends." While we agree with Bolon's concerns, on the whole, we find little support in the evidence he presents to support them. Our discussio n below details our concerns.Predicting aggregate test scoresOne of the problems with regression analysis is tha t without reasonable theoretical support, all sorts of predictors can be found that produce high correlation. In examining aggregate scores, such as high school test means, i t is no secret that for many decades, as Bolon himself pointed out (Bolon, 2000), achievemen t has been associated with socioeconomic conditions in communities. In earlier eras, when school spending was much more unequal, these differences were more indi cative of opportunity to learn for students. In a judicial climate that has tended to minimize, although not eliminate such disparities, it is much less persuasive, although i t remains an important area for study. The difficulty with using a community aggregate mea sure as a predictor is that it is a surrogate for many other indicators, some of which are absurd at face value but interpretable. Variables such as driver's-license p assing rate or per capita champagne consumption may predict student achievement as well as community per capita income. We can construct meaningful arguments why they migh t. For none is the test invalidated using accepted standards (American Educational Rese arch Association, American Psychological Association, and National Council on Measurement in Education, 1999). In other areas of research such aggregation has pro duced fundamentally misleading conclusions. For example, the literature on intelli gence and income is directly parallel to the discussion here. White (1982) demonstrated the difference between using an aggregate measure of SES (school or community) and individual measure in relating SES to intellectual functioning. Since Bolon used s chool as his unit of analysis, he eliminated proximate measures more appropriate to h is analysis. The school-level variables Bolon eliminated are more appropriate tha n community per capita income on


3 of 4 Copyright 2002 by the Education Policy Analysis ArchivesThe World Wide Web address for the Education Policy Analysis Archives is General questions about appropriateness of topics o r particular articles may be addressed to the Editor, Gene V Glass, or reach him at College of Education, Arizona State University, Tempe, AZ 8 5287-2411. The Commentary Editor is Casey D. Cobb: .EPAA Editorial Board Michael W. Apple University of Wisconsin Greg Camilli Rutgers University John Covaleskie Northern Michigan University Alan Davis University of Colorado, Denver Sherman Dorn University of South Florida Mark E. Fetler California Commission on Teacher Credentialing Richard Garlikov Thomas F. Green Syracuse University Alison I. Griffith York University Arlen Gullickson Western Michigan University Ernest R. House University of Colorado Aimee Howley Ohio University Craig B. Howley Appalachia Educational Laboratory William Hunter University of Calgary Daniel Kalls Ume University Benjamin Levin University of Manitoba Thomas Mauhs-Pugh Green Mountain College Dewayne Matthews Education Commission of the States William McInerney Purdue University Mary McKeown-Moak MGT of America (Austin, TX) Les McLean University of Toronto Susan Bobbitt Nolen University of Washington Anne L. Pemberton Hugh G. Petrie SUNY Buffalo Richard C. Richardson New York University Anthony G. Rud Jr. Purdue University Dennis Sayers California State University—Stanislaus Jay D. Scribner University of Texas at Austin Michael Scriven Robert E. Stake University of Illinois—UC Robert Stonehill U.S. Department of Education David D. Williams Brigham Young University


4 of 4 EPAA Spanish Language Editorial BoardAssociate Editor for Spanish Language Roberto Rodrguez Gmez Universidad Nacional Autnoma de Mxico Adrin Acosta (Mxico) Universidad de J. Flix Angulo Rasco (Spain) Universidad de Teresa Bracho (Mxico) Centro de Investigacin y DocenciaEconmica-CIDEbracho Alejandro Canales (Mxico) Universidad Nacional Autnoma Ursula Casanova (U.S.A.) Arizona State Jos Contreras Domingo Universitat de Barcelona Erwin Epstein (U.S.A.) Loyola University of Josu Gonzlez (U.S.A.) Arizona State Rollin Kent (Mxico)Departamento de InvestigacinEducativa-DIE/ Mara Beatriz Luce (Brazil)Universidad Federal de Rio Grande do Sul-UFRGSlucemb@orion.ufrgs.brJavier Mendoza Rojas (Mxico)Universidad Nacional Autnoma deMxicojaviermr@servidor.unam.mxMarcela Mollis (Argentina)Universidad de Buenos Humberto Muoz Garca (Mxico) Universidad Nacional Autnoma deMxicohumberto@servidor.unam.mxAngel Ignacio Prez Gmez (Spain)Universidad de Daniel Schugurensky (Argentina-Canad)OISE/UT, Simon Schwartzman (Brazil)Fundao Instituto Brasileiro e Geografiae Estatstica Jurjo Torres Santom (Spain)Universidad de A Carlos Alberto Torres (U.S.A.)University of California, Los


Download Options

Choose Size
Choose file type
Cite this item close


Cras ut cursus ante, a fringilla nunc. Mauris lorem nunc, cursus sit amet enim ac, vehicula vestibulum mi. Mauris viverra nisl vel enim faucibus porta. Praesent sit amet ornare diam, non finibus nulla.


Cras efficitur magna et sapien varius, luctus ullamcorper dolor convallis. Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Fusce sit amet justo ut erat laoreet congue sed a ante.


Phasellus ornare in augue eu imperdiet. Donec malesuada sapien ante, at vehicula orci tempor molestie. Proin vitae urna elit. Pellentesque vitae nisi et diam euismod malesuada aliquet non erat.


Nunc fringilla dolor ut dictum placerat. Proin ac neque rutrum, consectetur ligula id, laoreet ligula. Nulla lorem massa, consectetur vitae consequat in, lobortis at dolor. Nunc sed leo odio.