Educational policy analysis archives

Educational policy analysis archives

Material Information

Educational policy analysis archives
Arizona State University
University of South Florida
Place of Publication:
Tempe, Ariz
Tampa, Fla
Arizona State University
University of South Florida.
Publication Date:


Subjects / Keywords:
Education -- Research -- Periodicals ( lcsh )
non-fiction ( marcgt )
serial ( sobekcm )

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
E11-00009 ( USFLDC DOI )
e11.9 ( USFLDC Handle )

Postcard Information



This item has the following downloads:

Full Text
xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam a22 u 4500
controlfield tag 008 c19939999azu 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E11-00009
0 245
Educational policy analysis archives.
n Vol. 1, no. 9 (July 16, 1993).
Tempe, Ariz. :
b Arizona State University ;
Tampa, Fla. :
University of South Florida.
c July 16, 1993
Evidence, ethics and social policy dilemmas / Steven I. Miller [and] L. Arthur Safer.
x Research
v Periodicals.
2 710
Arizona State University.
University of South Florida.
1 773
t Education Policy Analysis Archives (EPAA)
4 856

xml version 1.0 encoding UTF-8 standalone no
mods:mods xmlns:mods http:www.loc.govmodsv3 xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govmodsv3mods-3-1.xsd
mods:relatedItem type host
mods:identifier issn 1068-2341mods:part
mods:detail volume mods:number 1issue 9series Year mods:caption 19931993Month July7Day 1616mods:originInfo mods:dateIssued iso8601 1993-07-16


1 of 15 Education Policy Analysis Archives Volume 1 Number 9July 16, 1993ISSN 1068-2341A peer-reviewed scholarly electronic journal. Editor: Gene V Glass, Glass@ASU.EDU. College of Edu cation, Arizona State University,Tempe AZ 85287-2411 Copyright 1993, the EDUCATION POLICY ANALYSIS ARCHIVES.Permission is hereby granted to copy any a rticle provided that EDUCATION POLICY ANALYSIS ARCHIVES is credited and copies are not sold.Evidence, Ethics and Social Policy DilemmasSteven I. Miller L. Arthur Safer Loyola University Chicago Abstract: Within the philosophy of the social sciences, the r elationship between evidence, ethics, and social policy is in need of further analysis. The p resent paper is an attempt to argue that while important social policies can, and perhaps ought to be, grounded in ethical theory, they are seldom articulated in this fashion due to the ambig uity surrounding the "evidence condition." Using a consequentialist-utilitarian framework, and a case study of a policy dilemma, the authors analyze the difficulties associated with resolving policy-based dilemmas which must appeal to evidential support as a justification for an ethica l stand. Implication for the relevance of ethics to social policy formulation are discussed in detail. This paper attempts to examine what we will call th e "evidence-ethics-policy triad." Our initial claim will be that while the "ethical" comp onent of the triad has increasingly become an important consideration in the conduct of social po licy inquiry, its actual influence on the formulation of social policy is minimal, at best, a nd, at worse, irrelevant. Likewise, the purported utility of addressing ethical concerns within the c ontext of social science practice directed towards social policy is grounded in a variety of s tances broadly spanning, but not isometric with, the traditional divisions of ethics within philosop hy, namely, "consequentialism-utilitarianism" and "deontology" (Strike and Soltis, 1985). However a certain type of asymmetry characterizes the consideration and application of ethical thinki ng as a possibly relevant guide to social policy considerations. What we are suggesting, specifically, is that an in fluential and significant portion of the social science community is, implicitly or explicit ly, committed to some version of ethical relativism. Now, such a commitment is "asymmetrical in the sense that it neglects the possible


2 of 15importance of "deontological" ethical positions in explaining human actions, individually or collectively. In other words, relativism is more cl osely aligned with the "consequentialist-utilitarian" position in traditio nal ethical theory with its emphasis on the "outcomes" or "consequences" of human action, and t he tacit recognition that such outcomes are socially constructed and changeable overtime and wi thin and across cultural contexts (Barnes and Bloor, 1982, Winch, 1964). Although the contemporar y emphasis within the social sciences (both empirical and hermeneutic) on "relativism" mi ght seem removed from the classic utilitarian positions of Mill and Bentham (see Rawls, 1971, Ch. 1, sec. 5), it shares an affinity with these positions by virtue of its emphasis on the possibil ity of objectively assessing the factors (economic and cultural) which will produce the "gre atest good for the greatest number." While the interpretation of such a "maximization" princip le might vary cross-culturally, for instance, it is the presumed methodological underpinning of the social sciences to a utility or maximization principle that is the salient "ethical" component o f these disciplines. To put this somewhat differently, the incorporation by the social sciences of "post-modern", "anti-positivistic" or "deconstructi onist" values as a result of Kuhn's (1970) initial insights (especially on "incommensurability") has s hifted the "ethical" balance away from deontological views to consequentialist views. (How ever, some critical theorists, such as Habermas (1979), argue that values such as freedom and justice have an "objective justification." In this sense, if Habermas is to be viewed as a soc ial scientist, such ideas could be construed as broadly deontological.) Thus, deontological positio ns, which favor the interpretation of values under general principles rooted in a "universal" hu man nature, relevant cross-culturally, have been undermined by the strong emphasis on relativis m in the social sciences. Perhaps the only notable exception to this relativistic interpretati on of values has been the work of Kohlberg (1976) and, perhaps, Piaget (1965), both arguing, a lthough in different ways, for the universality of "stages" of moral development. While Kohlberg's work has come under criticism (Simpson, 1974, Sullivan, 1977, Wilson, 1976) in terms of gen der biases, lack of sufficient evidence for the "higher" stages of moral development, and a variety of methodological concerns about validity and reliability, his work stands as probably the cl osest empirical-deontologically based argument against strict relativism. None-the-less, there sti ll appears to be a strong commitment among the social science community to the relativist-conseque ntialist point of view. In our view, this has been brought about by viewing the Kuhnian and post Kuhnian interpretations of scientific incommensurability as overriding the efficacy of de ontological interpretations.II Perhaps a further rejection of the deontological fr amework lies in the closer (presumed) affinity between consequentialistutilitarian assu mptions and the requirement of evidence. That is, the historical commitment of the social science s to justify their knowledge claims empirically (i.e., through reliable and valid methodological te chniques) is more compatible with consequentialism with its emphasis on "utility," th e "greatest good for the greatest number," and the possibility of measuring "consequences" themsel ves. Thus, for example, if we can show that welfare-policy X will provide the "most benefits" f or a given group, the evidence for such a claim will carry more "weight" than if we argue tha t it "ought" to be implemented because it falls under a "binding" deontological principle. (See, fo r instance, Carol H. Weiss, "Ideology, Interests, and Information" in Ethics, The Social Sciences and Policy Analysis, D. Callahan and B. Jennings (eds.). NY: Plenum, 1983, pp. 213-245. Weiss discusses the role of social science information and how it may be "weighted" by policym akers for particular ends.) The latter may, indeed, be persuasive; however, if viewed from a so cial science perspective, the former has the appearance of being more scientific, or at least mo re allegedly scientific, although possibly "relativistic" in terms of specific applications. N evertheless, the reliance on evidence as the


3 of 15arbitrator in social policy debates, whose justific ation is situated in a consequentialist-utilitarian ethical framework, is a double-edged sword. Its pos itive aspect lies in the ability to generate evidential support for "maximizing" a given positio n, while its negative aspect lies in the systematic ambiguity surrounding the term "evidence itself. That is, the very notion of "evidence" within the social sciences is often too quickly perceived to be non-problematic, while in fact, we will argue, it is at the very heart of the problem of trying to justify a relativisticconsequentialist view of social policy formulation. (For a good discussion of the assumptions about empirical evidence that policy analysts rely on see Ruth S. Hanft, "Use of Social Science Data for Policy Analysis and Policymaking," in Ethics, The Social Sciences and Policy Analysis, D. Callahan and B. Jennings (eds.). NY: Plenum, 198 3, pp. 249-270, especially, pp. 258-261.) The following are a few examples of the ambiguity s urrounding the concept of evidence. "He has adequate evidence for this belief in X." 1. "The evidence certainly seems to support her positi on." 2. "There is certainly conclusive evidence of his guil t." 3. "There is some evidence of his guilt." 4. "The evidence confirms/supports the hypothesis." 5. "Her evidence for X seems better than his." 6. "His evidence was relevant at T1, but not now at T2 ." 7. "The evidence for his theory is very (highly) proba ble." 8. Such a list could be further extended, but the abov e examples give a representative "flavor" of how the term is often used. We will not compare and contrast the various possible meanings of the term "evidence" in these examples but only note two features: first, it is often taken as a "given" or "primitive" term (i.e., left undefined), and second, the term is "relational" in the sense that its "meaning" is supposedly clarified or modif ied "in relation to" another term (e.g., "total" evidence). These two features are important because they pinpoint, or at least suggest, where the ambiguity lies (Miller and Fredericks, 1992). This "linguistic" dimension of the term "evidence" has, furthermore, often been overlooked because of the persuasive influence of the branch of epistemology called Confirmation Theory. Stemming f rom its origins in Local Positivism, Confirmation Theory has become a highly technical a nd esoteric sub-specialty of epistemology, and specifically in terms of its applications to va rious problems (e.g., "confirming" theories) within the philosophy of science. Basically, confir mation theory is concerned with explicating the "rules" that exist between a claim (i.e., a hyp othesis) and "evidence" for this claim. These "rules" are of two types: (1) logical and (2) induc tive. The first refers to traditional and "modern" (e.g., "sentential calculus" and "predicate calculu s") extensions of the rules of inference. The second refers to the rules of inductive logic and s pecifically to the "probability calculus", i.e., th e rules one follows in establishing the "probability" of events. Briefly, then, what confirmation theorists attempt to do is to "workout" the logic (i.e., the rules) that must pertain between a statement and ev idence for that statement, so that one can say the statement has been "confirmed." Now, while ther e is a vast literature on confirmation theory (e.g., Achinstein, 1983), and a host of thorny prob lems associated with it, we have mentioned it here to illustrate three points: (1) it has become the dominant form of analysis of thinking of how evidence is used to give "warranted support" to sta tements of belief, (2) it has gained this "power" by reliance on applications of the probabil ity calculus, and (3) it has assumed the term "evidence" to be non-problematic because "evidence" simply becomes any and all statements translatable into probability statements. The last point (#3) is significant because it has o bscured the complexity of the term "evidence" by subsuming it under probability theory In effect, something is evidence if it can be given a probability estimate. While there is nothin g inherently wrong in such a conception of


4 of 15evidence, it does however limit the range of what c an count as evidence. Specifically, such a construal rules out, in terms of being acceptable e vidence, many "qualitative" claims in such areas as history, law, psychoanalysis, literary cri ticism, and those branches of the social sciences which opt (for philosophical reasons) for evidence which is not quantifiable. To rule out these other sources of evidence, or relegate them to a le sser status, because they do not conform to a probabilistic definition, is to drastically restric t what can count as evidence. However, to admit "qualitative" evidence is to raise additional quest ions, most notably, how does one assess such evidence as "support" for a given claim? (A very go od defense of using qualitative data for policy analysis is given by Bruce Jennings, "Interpretive Social Science and Policy Analysis," in Ethics, The Social Sciences and Policy Analysis, D. Callahan and B. Jennings (eds.). NY: Plenum, 19 83, pp. 3-35.) There is, then, a parallel problem of de termining the "rules" for this type of evidence. If such rules cannot be formulated, one is "stuck" with the problem of having potentially "good" evidence for a claim but not being able to say how it counts for the claim (see Miller and Fredericks, 1992, and Miller, 1991 to see how such a case for qualitative evidence can be made). Within these considerations for what constitutes ev idence, some other distinctions need to be briefly mentioned. One is what we will call the "temporal status" of the evidence. That is, does one "admit" evidence for a claim that is specifical ly generated for that claim, or evidence that is "related" to the claim; i.e., obtained from existin g evidence about the claim? Some would argue that only the former constitutes "legitimate" evide nce since it directly addresses the issue at hand. Others would contend that "related" (or "after the fact") evidence is equally legitimate and admissible if it concerns the claim in question. Ho wever, in this instance, one must also address the "closeness-of-fit" issue. That is, if "related" evidence is admitted how "close", methodologically and substantively, must it come to the claim? For instance, will study A be admitted as evidence for the claim if, and only if, it replicates a study that would have ordinarily been conducted on the claim? In other words, how "b road" or "narrow" must related evidence be before it is admitted as "legitimate" evidence for the claim? Do we need "rules here?" We would say "yes", and furthermore argue that one of the fa ilures of traditional epistemology has been its inability to formulate appropriate rules for such c ases. Lastly, there is the vexing problem of what we will call the "mixed evidence" case. In this situation we can have a combination of direct and r elated evidence bearing on a claim but the evidence comes from a variety of methodological app roaches. There may also be variance as to the "closeness-of-fit" criterion, with, however, al l the diverse evidence still bearing on the claim. These considerations suggest that the term "evidenc e" is a highly complex one involving not only different forms or types but also the possibility o f different "rules" which translate data into evidence. The point of these comments, so far, has been to su ggest the fundamental importance of the "evidence condition" for the ways evidence is r elated to a variety of ethical concerns within the specific context of policy-related dilemmas.III In a policy-related dilemma, one is confronted with two (or more) seemingly contradictory alternatives. The problem is that only one can be c hosen, but choosing one produces negative consequences (for someone) in terms of the other. F or ethical theory, the issue becomes one of making a given choice and justifying it by way of a n ethical position. This assumes, of course, that policymakers do reflect on the ethical implica tions of their decision(s). This, probably, is an unwarranted assumption: decisions are often based o n such considerations as power, expediency, persuasion and fatigue. Although we believe importa nt policy decisions are seldom made in regard to ethical considerations, we want to outlin e some of the considerations a policymaker would encounter if he/she were in the position of j ustifying decisions based on ethical concerns.


5 of 15 We are also assuming that the policymaker is reflec ting his/her "proper" role as a policymaker (i.e., he/she has the "authority" to do so, leaving aside the source(s) of such authority), and that the level or type of decision to be made, in itself has no bearing on the type of ethical justification to be utilized, i.e., a more "serious decision requires (it may be argued) a deontological justification. An initial way of describing the type of analysis w e wish to pursue may be illustrated as follows: Need for "Evidence" TypeYesNoConsequentialismabDeontologycd Figure 1 In this simple "crossbreak," we are roughly classif ying the two major types of ethical frameworks as: (1) consequentialist (utilitarian) r eferring, again very broadly, to the type of ethical reasoning which argues that the "goodness", "rightness" or "justness" of an action is determined as a function of the "consequences" it p roduces, either for the "many" (i.e., the "majority") or for the relevant group under conside ration, and (2) deontological, referring to the evaluation of an action by way of "rules" which in some sense are considered to be "moral", "fundamental", "basic", "a priori", or "universal." Likewise, we will not deal with such variations as "rule utilitarianism" or the origins of the rule s within the deontological framework, e.g., if they are "truly" universal or culturally specific. These are important issues, but ones that would lead us into other directions. Also, because of its ambiguity, we have put off try ing to define the term "evidence"; however, a tentative (and rather broad) definition follows below: Evidence consists of statements of belief concernin g the "objective truth" of some given state of affairs. Such statements can assume many f orms, especially if they are of a "synthetic-a posteriori" variety, and are (in this sense) relate d to the methodological approaches used to generate them. Statements of evidential belief beco me evidence for a claim to the extent they can be justified by "rules" which themselves are taken to be "self-evident," "reasonable", or "warranted." Referring to Figure 1, the question is whether some type of evidence is necessary to address ethical claims? Again, generally, it is ass umed that "consequentialist" views require some justification by way of "empirical" evidence. That is, if either "positive" or "negative" consequences are projected for some course of actio n, there ought to be "evidence" for supporting such claims. On the other hand, for deon tological approaches, the appeal to "empirical" evidence is not necessary since the pri nciple itself (i.e., the "rule[s]") are either take n to be self-evident or somehow grounded in "human na ture" or human reason. Where "evidence" is appealed to in such instances, it is in the form of an appeal to non-empirical "reasons." (Note: "empirical" is put in quotation marks to indicate t hat such sources of evidence are fundamentally related to some notion of "sense perceptions" or "s ense data," but (also) that they are not limited to only quantitative methodological approaches). As an aside here, we would like to mention that the issue of the relationship of evidence to certain types of ethical reasoning is situated in a broader (and difficult) epistemological context, namely the "empirical justification" of beliefs (se e Moser, 1985 for an excellent overview of this subject). Generally, empirical justification, or mo re broadly "epistemic" justification, is


6 of 15 concerned with formulating and defending a variety of logical "rules" for saying that X has some notion of "justified true belief." There are severa l different (and often competing) views--e.g., "contextualism," "foundationalism,"--of how one com es to justify a particular claim about the world. Now, while Confirmation Theory, previously m entioned, is an important sub-set of theories of epistemic justification and related to the issue of how evidence relates to certain types of ethical reasoning, the broader issues of epistem ic justification are only indirectly relevant (if known at all) to the reasoning utilized by most pol icymakers in trying to "resolve" dilemmas. (Moore (pp. 281-285), for instance, argues that eve n if policymakers are aware of the technical aspects of what constitutes adequate evidence in th e social sciences, such considerations must be balanced against the practical demands of the polic y being formulated. This "tension" may, of course, result in "ethical" problems for the policy maker, i.e., what evidence ought to be chosen and how justified? See Mark H. Moore, "Social Scien ce and Policy Analysis," in D. Callahan and B. Jennings (eds.), Ethics, The Social Sciences, and Policy Analysis NY: Plenum, 1983, pp. 271291.) Thus, based on our own observations and readings, w e do not believe these larger (and technical) concerns play an important part in guidi ng concrete policymaking decisions. For example, a policymaker is probably not overly conce rned (even if she or he is aware of it) of the "infinite regress" problem associated with using em pirical evidence for a particular decision (BonJour, 1985). This issue would have the policyma ker worrying about the epistemological problem of justifying a given "piece" of empirical evidence by having to justify it by some other piece of evidence (i.e., a belief about it), which, in turn, must be justified, etc. We are not suggesting that policymakers ought not to be intere sted in these broader epistemological concerns, but only that they probably seldom are. S imilarly, we do not believe policymakers (again, even if known) are primarily concerned with the different varieties of their consequentialist or deontological ethical positions in justifying a particular course of action. For the policymaking process, this suggests that cr ucial decisions are most likely made by reference to nonepistemological (and non-ethicaltheoretical) considerations such as power, persuasion, time, decision-immediacy and so forth. Nevertheless, continuing with the issue of how some conception(s) of evidence may relate to resolving policy dilemmas by way of ethical posi tions, we list some of the possibilities in Figure 2. Policy-Related Dilemmas Alternatives A B Appeals 1. Consequentialistvs.Deontological2. Deontologicalvs.Consequentialist3. Consequentialistvs.Consequentialist4. Deontologicalvs.Deontological Figure 2 Figure 2 simply outlines the possibilities for init ially addressing a policy-related dilemma in terms of either a consequentialist or deontologi cal stance. There is, of course, no mention yet of the "evidence" issue; this will be forthcoming.


7 of 15 Thus, we are assuming that the policymaker must cho ose one of the alternatives, and that there is a twofold "justification process": first, in terms of the ethical position, and, secondly, by appeal(s) to "evidence" to support the position. Ev en here, however, a complexity arises; namely, the policymaker may choose one alternative because of the availability of evidence and, then, after the fact, justify it by an appeal to an ethic al position. This possibility suggests how a complete understand ing of the problem is also one that confronts us with complicated factors relating to t he psychological state(s) of the policymaker. Very briefly, some of these might include: A presumption that the policymaker knows how either ethical position may/may not relate to each of the policy alternatives. 1. A presumption that the policymaker can (then) decid e on one ethical position for one alternative. 2. A presumption that the policymaker can (then) disco unt either ethical position for the other alternative, while justifying his/her ethical choic e for the chosen alternative by "weighting" it more heavily. 3. In most cases, it is probably unrealistic to assume such omniscience for policymakers, but such considerations do figure into a complete under standing of such a process. Thus, in Figure 2, the first (1) possibility suggests that the policym aker has (on whatever grounds) decided that for the dilemma under consideration, Alternative A is m ost susceptible to a consequentialist interpretation, while B is best handled by a deonto logical position. Now, there is the related difficult problem, again possibly psychological in nature, of knowing the policymaker's a prior preference for either ethical stance, and whether s uch preference has any bearing on choosing either A or B. Further considerations would include the degree of personal commitment to an ethical position, whether this can be changed in th e light of evidence, and the degree of persuasiveness, pro or con, that is provided by oth ers involved in the policymaking process. We will not pursue these issues but will simply ass ume that the policymaker is faced with a dilemma, that he or she chooses one alternative, and that he/she does so because of a belief in a certain ethical position that can be "informed" by appeals to "evidence." This relationship is illustrated in Figure 3. Evidence -----------------Ethical Position Alternative Policy Decision Figure 3 Now we believe that many policymakers tend to choos e some form of consequentialism in decision-making because of some notion that "eviden ce" is important in making difficult decisions. Therefore, in the following section disc ussing evidence, we will take the example of a consequentialist vs. consequentialist position, Fig ure 2 (3). We do this simply as a convenient way of illustrating the complexities of the "eviden ce condition." Of course, in another sense, a deontological position could be adopted, with "evid ence" being the kinds of reasons one puts forth.IV As a "case study" of how evidence enters the policy making process in terms of a


8 of 15consequentialist framework, we will use the recent decision by a major urban university to close its School of Dentistry. While this is an actual ev ent, we have only limited personal knowledge of how (and by whom) the final decision was made. T hus, our "case study" is partially hypothetical, and in the analysis which follows, we will only be concerned with putting further some plausible reasons that a policymaker might uti lize for choosing the "closing" option. We shall also try to give reasons for the "not closing position. Let us assume the policymaker is confronted with th e following dilemma: Either close the Dental School or do not close the Dental School. If the school is closed it will impact negatively on those associated with it, if the School is not closed it will impact negatively on the University. What should be done? We will also assume that a decision has not yet bee n made, that the principal policymaker broadly identifies him or herself as a "Consequenti alist," and, as such, wants to examine "evidence" on both sides of the issue. Based on the evidence, the policymaker will then make a decision in which an attempt is made to "maximize" the "best" benefits for the greatest number of people. Furthermore, let us assume that based on hi s or her own thinking, and the advice of others, the policymaker sets forth some tentative h ypothesis for both sides of the issue. We are not concerned here with the origin of such hypothes es, although we do believe this is an important topic as some socialpsychological studi es have suggested (Hewstone, 1983, Kelley, 1972). A list of plausible hypotheses are given in the following figure. Dilemma-Related Hypotheses If we Close the Dental School If We Do Not Close the Dental School It will put tenured faculty out of work. 1. It will not permit students to complete their training. 2. It will cause economic hardships not only for faculty but for others working there as well as those providing support services (e.g., food services). 3. It will cause psychological stress among faculty. 4. It will affect the availability of dental care, generally, and for poor people, specifically. 5. It will increase the University's deficit which will have an impact on other parts of the University--faculty, students, programs, etc. Figure 4 Now, the assessment of these "hypotheses" in terms of "evidence" must be made, and here a variety of the problems emerge. "Direct" vs. "Supporting" Evidence By "direct" evidence, we mean a specific inquiry is carried out to obtain evidence for a claim. For alternative B, this would mean that the administration undertakes a study that would (presumably) support one or both aspects of the hyp othesis. The first "aspect" would simply be the presentation of data showing the University's p resent financial status, and a "justifiable extrapolation" of what it will become if the Dental School is not closed. A "justifiable extrapolation" might consist of some type of econom etric model which predicts the impact (i.e., "how much") on the existing deficit (if there is on e) and how much it will increase over some


9 of 15time period. The second "aspect" is to show how suc h an increasing deficit will impact on other sectors of the University, e.g., new faculty additi ons will have to be "frozen." The evidence is "direct" because it is the result of a "study" carr ied out specifically on the hypothesis. On the other hand, "supporting" evidence may consis t of an appeal to studies previously done on one or both "aspects" of the above. Now, of course, the supporting evidence idea begs other types of questions: (1) the "closeness of fit criterion. By this we mean a determination of how "close" the supporting evidence has to be, i.e. should we only "admit" evidence from other studies if such evidence is based on studies "exact ly" (i.e., methodologically) like the original?; (2) the "weight of the evidence" criterion. This cr iterion refers to either the number of studies chosen (as sufficient) for supporting a particular position, or some assessment of the importance of one or more studies within the array chosen as s upporting evidence. Here, the total evidence is considered as relevant to the present hypothesis, b ut some evidence (perhaps on methodological grounds) is deemed to be "better" or "stronger" tha n the rest, i.e., some type of "ranking" system is appeal to; (3) finally, the evidence believed re levant for a decision, especially supporting evidence, may be "methodologically mixed." By this we mean the policymaker uses evidence produced by a variety of methodological approaches (e.g., quantitative, qualitative, historical) as a means of supporting the particular policy decisio n. How such choices among evidence are or ought to be made is both a logical and psychologica l question that cannot be addressed here. Even these simple examples, however, may begin to i llustrate the complex relations that came into play when considering the ethicsevidence-pol icy triad. Scope of the Hypothesis and Evidence The number and types of hypotheses in Figure 4 are merely illustrations of many which could be formulated on both sides of the issue. Suc h hypotheses may be either narrow or broad in scope, and, again, their selection by the policymak er may be a function of psychological, logical, ethical and political factors. Also, as the number of hypotheses for or against a position increases, evidence (of whatever kind) will generally be more difficult to obtain. If such evidence can be attained, and if it is in some sense positively rel evant for the chosen position, then it can either b e "summed" or "totaled" to strengthen the case for so me policy alternative. Alternatively, it may be "ranked" or "prioritized" in such a way that the po licymaker selects what she/he considers the "best" evident for the position. Again, such "ranki ng" could be made on methodological, "political" or other grounds. In terms of evidence, an interesting case occurs wh en the scope of a hypothesis is "increased" or "extended" to make a "conjunctive" c laim. A conjunctive claim is one in which a desirable or undesirable consequence is "added on" to an original claim. The extended claim may also, then, be given a "causal" interpretation. For example, in Figure 4, under A(2), the original hypothesis is that closing the Dental School will n ot permit students to finish their training. If, however, another claim is "added on" to this one, w e may have something like the following: "Closing the Dental School will not permit students to finish their training and this will result in dramatic increases in stress." Thus, not permitting students to finish their program is (allegedly) sufficient in predicting that some further undesira ble (in this case) consequences will follow. Whether or not such an extended claim is "true" wil l be a function of what type(s) of evidence can be brought forth to assess it. The poi nt, however, is that extending the original hypothesis involves the policymaker in a more compl ex evidencegathering task. Now, not only must she/he determine the viability of the original hypothesis, but also its (presumed) connection with "stress" must be assessed. Methodologically, t his involves more decisions concerning sampling, instrumentation, testing and interpretati on. Likewise, from the consequentialist position we are adopting, there is a shift in focus : we are now assessing the (presumed) negative consequences not by students' inability to finish t heir programs, but by the "stress" levels they


10 of 15will (presumably) exhibit. Thus, a "structural" con sideration is claimed to lead to a "psychological" one, and it is this latter consider ation that is used to evaluate "consequences." There are, additionally, a multitude of other quest ions here; we will list only a few of them: If the "extended" hypothesis is "supported," will t his be sufficient grounds for the policymaker? Will other "related" evidence need to be considered (i.e., "supporting" evidence) to bolster the case? Is such evidence ava ilable? If so, can it be critically appraised for relevance, etc.? 1. By way of "methodological strategies," should the e xtended hypothesis be studied quantitatively (empirically) or qualitatively (ethn ographically)? What criteria should be used to make this decision? 2. If this extended hypothesis is supported, should ot her hypotheses (in A) be tested? How should this decision be made? 3. If the extended hypothesis is "strongly" supported, is this sufficient? For example, let's say "high" stress levels are found on the average for t he dental students, but the levels are not evenly distributed across the students. That is, su ppose minority dental students experience very high stress levels, but their numerical repres entation is small compared to the entire sample. Should this "finding" be given additional weight?" More generally, how many (and on what grounds) other "mediating" or "interve ning" factors ought to be taken into consideration? 4. What are the limits of possibly relevant "casual cl aims" that must be considered? That is, even if high levels of stress are documented is thi s sufficient evidence, or do we need to postulate that the stress, in turn, will lead to so me other undesirable consequence, e.g., mental and/or emotional problems? 5. Whatever positive support is found for the extended hypothesis (or for that matter the original one) how should it be compared to the hypo thesis in B? 6. Finally, if different methodological approaches are used between A and B, which "counts" more decisively? And why? 7.Summary All of the "presentations" of the evidence-ethics-p olicy triad we have mentioned so far could be extended in many other ways. What we have been trying to point out, however, are the complex issues that arise not only in the triadic r elationship above, when viewed as a totality, but especially in terms of the seemingly simple concept of "evidence." The concept of evidence is complex on many levels: (1) the ambiguities associa ted with trying to define it, (2) the different forms, types or varieties of evidence which are gen erated by different methodological approaches; (3) the "rules" (or lack of them) relat ing evidence to a claim; and, (4) the overarching epistemological (and many times controversial) issu es concerning "truth," "evidential adequacy," "empirical justification," and so on. In addition to these difficult matters, there are o ther "levels" that "overlap" in equally complex ways. Thus, for the policymaker confronted with a policy-related dilemma and who chooses to resort to some notion of "empirical evid ence," there are seldom any clear ("justified") criteria that she/he can appeal to. If, for instanc e, the policymaker is philosophically trained and/or knowledgeable about the epistemological issu es that bear on the role of evidence to


11 of 15"epistemic justification," such considerations ev en though philosophically "correct" are usually irrelevant to the policy decision that must be made. Their "irrelevance" lies in the fact that they are too far removed from the issue at han d and/or do not provide for pragmatic applications by way of clear-cut criteria for decis ionmaking choices. There are also complex interplays between the "psyc hology" of the policymaker and his/her understanding of the nature of evidence. So me policymakers, mostly because of their training in social science research methods, view t he concept of evidence as that of "findings" produced by empirical studies; the findings, of cou rse, should be "statistically significant!" Usually, these policymakers do not understand the e pistemological issues surrounding the "evidence condition", nor the more technical statis tical-mathematical assumptions involved in empirical research. Based on our own experiences an d readings, policymakers usually have some vague notions about what constitutes "good" or "ade quate" evidence. Other "psychological" factors in the policymaking process concern "select ive attention" to only certain types of evidence, the choice of believability of "experts" usually utilized by policymakers, accommodations or allegiances to other vested inter ests, self-serving motives, etc. Thus, while a policymaker may have some personal belief in what, for instance, constitutes a "truth-and-evidence" relationship, such considerati ons are often over-ridden by the "political" reality of the policymaking process. Now, the ambiguities associated with the concept of evidence and the "psychological" factors associated with policymaking intertwine (at least hypothetically) with the ethical dimension. Although we doubt most policymakers are aware of ethical theories, let alone how they may figure into approaching and "resolving" po licy related dilemmas, if such considerations are taken into account they raise a host of further overlapping and complex issues. Since most policymakers are "pragmaticrealists", they could be classified in a very rough way as belonging in the Consequentialist "camp." That is, their prim ary concern is determining how a policy choice will result in producing the "best" outcomes for th e "most" people. And being committed (at least implicitly) to this position, they are in need of s upporting "evidence" to substantiate their particular choice/decision. (Parenthetically, we ha ve not found any examples where a policymaker appeals to a deontological position in justifying a policy choice.) Given these considerations, the central question be comes one of choosing the "best" evidence for a position within the constraints of o ne or more hypotheses. The policymaker needs to make at least a minimal case that the evidence c hosen is "positively relevant" to the issue at hand. However, this is no simple matter if the evid ence comes from different methodological approaches, if some of it is contradictory, or if i t is based on "supporting" studies which are only marginally relevant to the issue. There is also the issue of only selecting evidence which supports a favored point-of-view. This last strategy may, ho wever, be justified by the policymaker because the evidence chosen may be relevant, there may be more" of it than the contradictory evidence, and the contradictory evidence is "judged" to be fl awed in some way. For example, the policymaker may choose the findings of an empirical (quantitative) study over the findings of an ethnographic (qualitative) one because the latter i s judged not to be (methodologically) "scientific." Thus, the "moral" of this story, if there is one, i s that the evidence-ethics-policy triad is not only quite complex but somewhat paradoxical; "parad oxical" in the sense that: It is commonly believed that "evidence" is a necess ary condition to make rational and informed decisions concerning important policy matt ers; however, once we probe more deeply into what "evidence" is the more difficult i t becomes to apply evidence to policy choices. 1. If policy decisions are believed to be grounded (ul timately?) in ethical positions, and if the ethical position chosen is some form of Consequenti alism, and if "good" or "desirable" 2.


12 of 15consequences are believed to accrue if some policy is chosen, then there is a need to present evidence that particular consequences will most likely (or do) follow. But here, again, it may be very difficult to justify the deci sions about evidence that are eventually made.A similar situation holds if evidence is believed t o be the most "rational" way to arbitrate between policydilemmas. The "evidence" chosen for supporting one "side" of the dilemma is often difficult to completely justify. 3. Epistemological theories concerning the knowledgeevidence-justification triad are often logically persuasive but seldom practically relevan t to the policymaker. (One problem here is that the epistemologist sees "evidence" as a str ictly logical relationship or condition that must be met in saying that propositional (i.e., kno wledge) claims can be justifiably held; however, the many semantic issues (and some logical ones e.g., those obtaining between/among different ["mixed"] evidence-instance s) are often not adequately addressed). 4. If, in a policy-related dilemma, a "neutral" policy maker is given "good" empirical evidence in favor of one "side" of the dilemma, and "good" d eontological "evidence" (i.e., some "good" rule-based reason(s) for the other "side" of the dilemma, is there some "rational" way to choose between the two? 5. Was, then, the decision to close the Dental School a "good" or "rational" one? Since we cannot reconstruct the events that went into making the decision, we cannot tell. Our guess would be that some form of Consequentialism (at lea st implicitly) was used as a "rationale" for the decision. It would, of course, be very interest ing to know what (if any) "evidence" was used to make the decision; how such evidence was gathere d; how evaluated; and how "justified?" Our final guess would be that the decision used only a "minimal evidence" base and that it was "positively relevant" but "selective" evidence. Of course, these speculations only reinforce our be lief that the "triadic" relationship is highly complex and not fully understood. Its lack o f understanding underscores the need for in-depth empirical studies, which, if undertaken, m ay, however, (and ironically) only bring to fore the very problems mentioned throughout. Nevert heless, we presently simply do not have a clear view of how, if, and to what extent, ethical judgements enter into the policymaking process. Our own inclination is to believe, borrowing an ana logy from the language of Confirmation Theory and the philosophy of science, that ethical thinking among policymakers serves only as "weak background evidence." That is, policymakers m ay view their own ethical beliefs as relevant to the decision(s) they make, but only in a very general, implicit or diffuse way. A part of this attitude may be due to the lack of i n-depth training in ethical theory among policymakers, but a portion can also be attributed to such factors as the immediacy of the situation, cross-pressures brought to bear by other "invested" parties, and the need to arrive at some type of "pragmatic" compromise. In a parallel fashion, policymakers who appeal to some type of evidence do so primarily as a means of supp orting a favored position. The methodological adequacy of this evidence is simply assumed, although "negative" evidence may (then) be criticized on its methodological "inadequ acy." In any event, however, policymakers are not overly concerned with either the technical aspe cts of differing methodological approaches nor with the broader (and philosophically technical) ep istemological issues of justification and evidence. Again, as with ethical theories, many pol icymakers do not have adequate training in these areas, but the over-riding concern is with th e formation of the specific policy: these other concerns are viewed as academically important but p ractically remote. Of course, policymakers are seldom pressed, individ ually or professionally, to "justify"


13 of 15their decisions in terms of articulating in-depth e thical and/or epistemological positions related to the policy. Depending on the policymaker, such deli berations, if they are made at all, are left to a later time, when one leaves "office" and has time f or "reflection." Thus, we are left with the interesting and ironic conclusion that what "ought" to be the most central in policymaking, ethical and evidential reasoning, becomes the most peripheral.ReferencesAchinstein, Peter (Ed.). The Concept of Evidence Oxford: Oxford University Press, 1983. Barnes, B. and Bloor, D. "Relativism, Rationalism a nd the Sociology of Knowledge." In M. Hollis and S. Lukes (Eds.), Rationality and Relativism Oxford: Blackwell, 1982, 21-47. Bloor, D. "The Strengths of the Strong Programme," Philosophy of the Social Sciences (11), 1981: 199-213.BonJour, L. The Structure of Empirical Knowledge Harvard: Harvard University Press, 1985. Callahan, D. and Jennings, B. (eds.). Ethics, The Social Sciences and Policy Analysis. NY: Plenum, 1983.Habermas, J. Communication and the Evolution of Society, Tran. Thomas McCarthy. Boston: Beacon Press, 1979.Hempel, Carol. Aspects of Scientific Explanation New York: Free Press, 1965. Hewstone, M. Attribution Theory: Social and Functional Extension s. Oxford: Basil Blackwell, 1983.Kelly, H.H. Causal schemata in the attribution process. New York: General Learning Press, 1972.Kohlberg, L. "Moral Stages and Moralization," in T. Lackona (Ed.), Moral Development and Behavior New York: Rinehart and Winston, 1976, 31-53. Kuhn, T.S. The Structure of Scientific Revolutions 2nd ed. Chicago: University of Chicago Press, 1970.Marcuse, H. One-dimensional man Boston: Beacon Press, 1964. Miller, Steven I. "Some Notes on the Nature of Meth odological Indeterminacy," Synthese (88) No. 3, 1991: 359-378.Miller, Steven I. "Some Comments on the Projectibil ity of Anthropological Hypotheses: Samoa Briefly Revisited," Erkenntnis (30) 1989: 279-299. Miller, Steven I. and Fredericks, Janet. "Clarifyin g the adequate evidence condition in educational issues and research," Educational Theory (42) 1992: 461-472. Moser, Paul K. Knowledge and Evidence Cambridge: Cambridge University Press, 1989. Moser, Paul K. Empirical Justification Boston: D. Reidel Publishing Co., 1985.


14 of 15 Piaget, J. The Moral Judgement of the Child New York: Random House, 1965. Rawls, J. A Theory of Justice Harvard: Harvard University Press, 1971. Simpson, E. "Moral Development Research: A Case of Scientific Cultural Bias," Human Development (17) 1974: 81-106. Strike, K.A. and J.F. Soltis. The Ethics of Teaching New York: Teachers College Press, 1985. Sullivan, E. A Study of Kohlberg's Structural Theor y of Moral Development: A Critique of Liberal Social Science Ideology. Toronto: Ontario I nstitute for Studies in Education, 1977. Wilson, R.W. "Some Comments on Stage Theories of Mo ral Development," Journal of Moral Education 1976: 241-248. Winch, P. "Understanding a Primitive Society." American Philosophical Quarterly 1, 1964: 307-24.About the AuthorsSteven I. Miller L. Arthur SaferSchool of Education Loyola University Chicago 820 North Michigan Avenue Chicago, Illinois 60611Copyright 1993 by the Education Policy Analysis ArchivesEPAA can be accessed either by visiting one of its seve ral archived forms or by subscribing to the LISTSERV known as EPAA at (To sub scribe, send an email letter to whose sole contents are SUB EPAA y our-name.) As articles are published by the Archives they are sent immediately to the EPAA subscribers and simultaneously archived in three forms. Articles are archived on EPAA as individual files under the name of the author a nd the Volume and article number. For example, the article by Stephen Kemmis in Volume 1, Number 1 of the Archives can be retrieved by sending an e-mail letter to LISTSERV@a and making the single line in the letter rea d GET KEMMIS V1N1 F=MAIL. For a table of contents of the entire ARCHIVES, send the following e-mail message to INDEX EPAA F=MAIL, tha t is, send an e-mail letter and make its single line read INDEX EPAA F=MAIL.The World Wide Web address for the Education Policy Analysis Archives is Education Policy Analysis Archives are "gophered" at To receive a publication guide for submitting artic les, see the EPAA World Wide Web site or send an e-mail letter to and include the single l ine GET EPAA PUBGUIDE F=MAIL. It will be sent to you by return e-mail. General questions about ap propriateness of topics or particular articles may be addressed to the Editor, Gene V Glass, Glass@asu.ed u or reach him at College of Education, Arizona Sta te University, Tempe, AZ 85287-2411. (602-965-2692)


15 of 15Editorial Board John CovaleskieSyracuse UniversityAndrew Coulson Alan Davis University of Colorado--DenverMark E. Thomas F. GreenSyracuse Alison I. Arlen Gullickson Ernest R. Aimee Craig B. Howley u56e3@wvnvm.bitnet William Richard M. Jaeger Benjamin Thomas Dewayne Mary P. Les Susan Bobbitt Anne L. Hugh G. Richard C. Anthony G. Rud Dennis Jay Robert Robert T.


Download Options

Choose Size
Choose file type
Cite this item close


Cras ut cursus ante, a fringilla nunc. Mauris lorem nunc, cursus sit amet enim ac, vehicula vestibulum mi. Mauris viverra nisl vel enim faucibus porta. Praesent sit amet ornare diam, non finibus nulla.


Cras efficitur magna et sapien varius, luctus ullamcorper dolor convallis. Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Fusce sit amet justo ut erat laoreet congue sed a ante.


Phasellus ornare in augue eu imperdiet. Donec malesuada sapien ante, at vehicula orci tempor molestie. Proin vitae urna elit. Pellentesque vitae nisi et diam euismod malesuada aliquet non erat.


Nunc fringilla dolor ut dictum placerat. Proin ac neque rutrum, consectetur ligula id, laoreet ligula. Nulla lorem massa, consectetur vitae consequat in, lobortis at dolor. Nunc sed leo odio.