xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam a22 u 4500
controlfield tag 008 c20049999azu 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E11-00404
Educational policy analysis archives.
n Vol. 12, no. 55 (October 15, 2004).
Tempe, Ariz. :
b Arizona State University ;
Tampa, Fla. :
University of South Florida.
c October 15, 2004
High school graduation rates : alternative methods and implications / Jing Miao [and] Walt Haney.
Arizona State University.
University of South Florida.
t Education Policy Analysis Archives (EPAA)
xml version 1.0 encoding UTF-8 standalone no
mods:mods xmlns:mods http:www.loc.govmodsv3 xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govmodsv3mods-3-1.xsd
mods:relatedItem type host
mods:identifier issn 1068-2341mods:part
mods:detail volume mods:number 12issue 55series Year mods:caption 20042004Month October10Day 1515mods:originInfo mods:dateIssued iso8601 2004-10-15
EDUCATION POLICY ANALYSIS ARCHIVES A peer-reviewed scholarly journal Editor: Gene V Glass College of Education Arizona State University Copyright is retained by the first or sole aut hor, who grants right of first publication to the Education Policy Analysis Archives EPAA is a project of the Education Policy Studies Laboratory. Articles are indexed in the Directory of Open Access Journals (www.doaj.org). Volume 12 Number 55 Octobe r 15, 2004 ISSN 1068-2341 High School Graduation Rates: Alternative Methods and Implications Jing Miao Walt Haney Boston College Citation: Miao, J. & Haney, W. (2004, Oct ober 15). High school graduation rates: Alternative methods and implications. Education Policy Analysis Archives, 12 (55). Retrieved [date] from http ://epaa.asu.edu/epaa/v12n55/. Abstract The No Child Left Behind Act has brought great attention to the high school graduation rate as one of the ma ndatory accountability measures for public school systems. However, there is no consensus on how to calculate the high school graduation rate given the lack of longitudinal databases that track individual students. This study reviews literature on and practices in reporting high school graduation rates, compares graduation rate estimates yielded from alternative methods, and estimates discrepancies between alternative results at national, state, and state ethnic group levels. Despite the graduation rate method used, results indicate that high school graduation rates in the U.S. have been declining in recent years and that graduation rates for black and Hispanic students lag substantially behind those of white students. As to gradua tion rate method preferred, this study found no evidence that the conceptually more complex methods yield more accurate or valid graduation rate estimates than the simpler methods.
Alternative graduation rates 2 Introduction The high school graduation rate has a histor y of being used as a measure of school effectiveness in the United States. The No Ch ild Left Behind Act once again brought great attention to the high school graduation rate as one of the mandatory accountability measures for public school systems. However, calculating the high school graduation rate is no easy task due to the lack of longitudinal databases that track individual students. This study reviews literature on and practices in reporting high school graduation rates, compares graduation rate estimates yielded from alternative methods, and estimates discrepancies between alternative results. The goal of this study is to evaluate the relative strengths and weaknesses of alternative reporting strategies and to contribute to a discussion of the policy implications, rather than to recommend a single best method. Expansion of High School Education and the Value of A High School Diploma Enrollment in United States high schools expanded rapidly in the first half of the twentieth century (Dorn, 1993, 1996, 2003; Goldin 1998). According to the 2002 Digest of Education Statistics (Snyder & Hoffman, 20 03), only 10% of 14to 17-year-olds were enrolled in either public or private high sc hools in 1899-1900 school year. By the fall of 1963, high school enrollment had increased to 90% for the same age group, and the high school enrollment rate for fall 2000 was projected to be 93.4%. The expansion in high school enrollment has prompted an increase in the proportion of youths graduating from high school. At the turn of the twentieth century 6.4% of 17-year-olds graduated from high school, including both public and private schools. By the 1962-63 school year, the percentage exceeded 70% and stabilized at this level until the present. As a result, it is fair to conclude that high school attendance and graduation have become normative expectations for teenagers in the United States (Dorn, 1996). The value of the high school diploma experienced ups and downs during the period of high school expansion. Early on, the rari ty of a high school diploma assured high school graduates better opportunities in the job market (Dorn, 1996). Â“Until about the 1970s, a high school diploma was generally viewed as a credential that would ensure a reasonably secure and well-paying jobÂ” (Swanson & Chaplin, 20 03, p.1). With the expansion of higher education, the value of a high school diploma in the labor market diminished significantly. A recent survey by Public Agenda (Immerwahr, 2000) finds that 87% of Americans believe that Â“a college education has become as im portant as a high school diploma used to beÂ” (p.1). Despite its diminishing value in the job market, a high school diploma still serves as a gateway to post-secondary education as well as opportunities in the military. High school graduates fare better than dropouts in terms of employment opportunities and earnings. For the civilian noninstitutional1 population ages 25 years and over, the average monthly unemployment rate was 5.35% for high school gr aduates for the period of September 2002 through September 2003. This compares to 8. 78% for those without a high school diploma 1 Â“Civilian noninstitutional populationÂ” refers to persons 16 years of age and older residing in the 50 states and the District of Columbi a who are not inmates of institutions (for example, penal and mental facilities, homes fo r the aged), and who are not on active duty in the Armed Forces (Retrieved 10/16/2003 from http://www.bls.gov/bls/glossary.htm#C).
Education Policy Analysis Archives Vol. 12 No. 55 3 for the same period (Bureau of Labor Statistics n.d.). There is also plenty of evidence that earnings of high school graduates are consis tently higher than those without high school credentials (Day & Newburger, 2002; Sum & Harrington, 2003). For individuals 25 years old and over, the median income for high school graduates (including GED diploma holders) was $24,656 compared to $18,445 for non-gradua tes, as of March 2002. Such differences were observed across all races, as well as within each major racial group (Bureau of Labor Statistics, 2002a). Distinguish between a Regular High School Diploma and GED Certificate The labor statistics, as a convention, of ten group GED equivalency diploma holders together with persons who graduate with a regular high school diploma; however, the two credentials are not equivalent. Currently three ty pes of high school credentials are often seen in literature and practice. A regular diploma is awarded to students who complete a standard number of years (12 for most) and meet the st ate or local requirements for graduation. An alternative (or other) diploma usually refers to the certificate given to students who complete state approved alternative programs (e.g. sp ecial education programs in some states). Students may also be awarded an equivalency certificate by passing a test. The most common secondary certification test is the General Educational Development Test (the GED test), developed and distributed by The General Educational Development Testing Service of the American Council on Education (ACE). The GED tests were first developed for testing World War II veterans to determine their competence for higher education. After the war, the GED was also administered to civilians, and those who passed were granted high school credentials by the states. By 1959, civilian test takers outnumbered veterans and military members for the first time (Boesel, Alsalam, & Smith, 1998). Nowadays, ACE claims that Â“[a]bout one in seven high school diplomas issued in the United States each year is based on pa ssing the GED TestsÂ” (American Council on Education, 2003). The increase in the number of GED certif icate holders blurred the line between high school graduates and dropouts by creating a third category, namely the GED certificate holders. Studies show that the performance of GED holders in the job market and postsecondary institutions is not equivalent to that of regular diploma recipients although GED holders perform better than dropouts (B oesel, Alsalam, & Smith, 1998; Cameron & Heckman, 1993; Chaplin, 1999, 2002; National Research Council, 2001; Tyler, 2003). Although equivalency credentials, such as the General Educational Development (GED) equivalency diploma, are accepted for both co llege admission and military recruitment, a regular high school diploma is the preferred credential2 (Boesel, Alasam, & Smith, 1998; 2 The military services distinguish between thr ee tiers of educational attainment. Tier I includes traditional high school graduates, alternative/continuation high school graduates, or college/post-secondary students. Tier II includes holders of alternative credential such as GED, certificate of attendance, corresponden ce school diploma, and occupational program certificate. Tier III are non-high school graduates. The military services accept very few Tier III category personnel. When they do make a ra re exception, the applicant must usually score significantly higher on the ASVAB than Tier I and Tier II candidates. The services also limit the number of Tier II candidates it will a llow to enlist each year. In the Air Force, the
Alternative graduation rates 4 Military Enlistment Standards). Studies show that the completion rates in the postsecondary schooling and training programs are much higher for the regular high school graduates than for equivalency diploma recipients (Boesel, Alasam, & Smith, 1998; Cameron & Heckman, 1993). Graduation Rate as a Measur e for School Effectiveness Because of the social and economic value a ssociated with school credentials, the proportion of students graduating from a school system is often used as an indicator to evaluate the system. As early as 1907, Thornd ike examined records of more than two-dozen cities, and found that only 10 percent of white students graduated from high school (Thorndike, 1907). Two years later, Leonard Ayres conducted another study, which included 55 city school systems. Ayres found th at, on the average, one-sixth of the students were repeating grades, one third of all public school students were older than they should be for the grades they were in, and these students were more likely to drop out (Ayres, 1909). Both authors criticized the slow progression of students as inefficient and wasteful of resources. Conducted in the early 1900Â’s, the studies of Thorndike and Ayres were mostly concerned with school children at the elementary level. With the expansion of high school education in the first half of the twentieth century, the importance of the high school graduation rate has been repeatedly emphasized in federal legislation and practices. Â“The ontime graduation rate was routinely reported in the 1970s and 1980s by the U.S. department of Education and was a central part of Secretary BellÂ’s Â‘Wall ChartÂ’Â” (Kaufman, 2001, p.20). Â“The Hawkins-Stafford School Improvem ent Amendments of 1988Â… required the Commissioner [of Education Statistics] to report to Congress each year on the second Tuesday after Labor Day about the rate of sc hool dropouts and completions in the nation (under current legislation, this report is no longer mandatory)Â” (Young & Hoffman, 2002, p.59). Six years later in 1994 the Clinton administration passed the Goals 2000: Educate America Act which set out eight national goals for improving public education. The second national goal was that Â“by the year 2000 the high school graduation rate will increase to at least 90 percentÂ”(Goals 2000, 1, Sec. 102, (1), (A)). High School Graduation Rate in the No Child Left Behind Act Although the lofty goal of a 90% graduati on rate was not achieved by 2000, it did not stop the Bush administration from mandatin g states to report high school graduation rates. The most recent reauthorization of th e Elementary and Secondary Education Act (ESEA), also known as the No Child Left Behind Act (NCLB), requires states to report graduation rates for public secondary schools as one of the indicators for measuring whether school systems are making Adequate Yearly Progress (AYP) towards state performance goals. This legislation mandates states to report graduation rates for the total state student population, as well as for subgroups of students, including economically disadvantaged students, students from major racial and ethnic groups, students with disabilities, and students with limited English proficiency. limit is less than one percent each year (http://usmilitary.about.com/library/weekly /aa082701c.htm accessed on 02/17/2004).
Education Policy Analysis Archives Vol. 12 No. 55 5 The No Child Left Behind Act requirement is different from previous federal requirements in a number of ways. First, NCLB is more specific in defining the high school graduation rate as The percentage of students, measured from the beginning of high school, who graduate from high school with a regular diploma (not including an alternative degree that is not fully aligned with the StateÂ’s academic standards, such as a certificate or a GED) in the standard number of years (34C.F.R. Â§200). This definition of high school graduates explicitly excludes GED recipients, who were counted as high school completers in connection with the Goals 2000 legislation. The final regulation (34C.F.R. Â§200) further pointed out that states must avoid counting a dropout as a transfer in defining graduation ra tes, which is speculated to be one of the reasons for the underreporting of dropouts by local agencies (Haney, 2000; Kaufman, 2001; Swanson & Chaplin, 2003). In addition, the NCLB definition also requires youth to complete high school in Â“a standard number of yearsÂ”, which mandates a more specific time frame for school systems to achieve the objective of graduating students from high school. By shifting the focus from the 18-to 24-year old population (as in Goals 2000 ) to the population enrolled in high school and excluding alternative credentials, NCLB brings attention to regular day programs in the public school system where the majority of U.S. youths receive their secondary education and where most educational resources are devoted. NCLB also differs from Goals 2000 by allowing states to set their own goals instead of setting a single national goal for the grad uation rate. Thus, the focus is on each state making progress over time rather than lining up all states for a horse race by requiring the same national goal for each. Despite the specific definition for graduation rate, states are also allowed flexibility in choosing alternative definitions as long as the Secretary of Education approves (34C.F.R. Â§200). Thus, the NCLB definition leaves room for states to determine specific data collection and calculation strategies. One of the cornerstones of the NCLB is its strong emphasis on accountability for results. Failure to make Adequate Yearly Prog ress in the required time frame may lead to increasingly severe consequences ranging fr om public identification of low performing schools, withholding of federal funds, to loss of students to other schools and/or change in school personnel (Swanson & Chaplin, 2003). Issues have been raised about unintended consequences of attaching high stakes to results such as increased retention of low performing students at certain grade levels (Edley & Wald, 2002; Haney, 2000; National Research Council, 2001). Studies show that grade retention often does not help students make improvement academically as is intended; moreover, retained students are more likely to drop out of high school (Hauser, 2000; Jimerson, 2001; Jimerson, Anderson & Whipple, 2002; National Research Council, 2001; Shep ard & Smith, 1989). When increased retention rates and dropout rates occur, improvement in test scores may be the result of testing a smaller number of relatively better achieving students rather than an accurate assessment of the whole student population. Therefore, inclusion of the graduation rate as an accountability indicator is especially important in the current standards-based education reform and is likely to counter the potential pressure to Â“push outÂ” low achieving st udents so as to inflate test results, and shift the attention to helping all students meet the standards (Swanson & Chaplin, 2003).
Alternative graduation rates 6 Challenge for Measurement The high-stakes use of test results required by NCLB poses extraordinarily high demands on the validity and reliability of educational measurement. Experts in educational measurement have described the technical cha llenges for current standardized assessments to serve such high-stakes accountability pur poses (Linn & Baker, 2002). The dramatic increase in the demand for large-scale assessment is pushing the limits of the testing industry, which is dominated by a small number of testing companies. The increased volume of testing mandated by NCLB is likely to trigger more testing errors as the industry is being pushed to its limit. Already researchers have iden tified almost as many testing errors in 2002 alone as the total number of errors repor ted between 1976 and 1996 (Rhoades & Madaus, 2003). Compared to standardized testing, calculating high graduation rates may appear much more straightforward. The graduation rate is calculated simply by dividing one number into another. However, to calculate the high sc hool graduation rate, at least three things need to be specified. First, at what point should the rate be calculated? Or, at what time points are the numbers counted for both the numerator and the denominator? Although the standard length of high school is four years in most sc hool systems in the United States, it is likely that some students will take more than four years to graduate from high school for various reasons. The NCLB definition suggests following a high school cohort from the beginning of high school; however, it leaves it open for th e states to interpret Â“the standard number of yearsÂ” for completing high school. Second, questions arise about who counts as a graduate in the numerator? The NCLB legislation limits high school graduates only to regular diploma recipients. However, students may have completed high school in diffe rent number of years, fo ur or five in most cases, and NCLB is not clear on whether states need to distinguish between late graduates and on-time graduates when reporting the high school graduation rate. Third, one might also ask who is included in the denominator as the base population? The NCLB definition requires counting the base population from the beginning of high school; however, it does not specify how the base population should be adjusted for fluctuations, such as cases of transfer, grade retention and dropout, over four years of high school. The seemingly simple procedure of calcul ating high school graduation rates proves to be no easy task (Dorn, 1996; Kaufman, 2001; Swanson & Chaplin, 2003). Depending on the method and the source of data used, publis hed U.S. national high school graduation or completion rates for the class of 2000 range from 66.6% to 86.0%; the variation between alternative graduation rates at state level is of comparable magnitude (Greene, 2002b; Haney, 2003; Kaufman, Alt, & Chapman, 2001; Swanson, 2003; Warren, 2003). Even when limiting high school graduates to the NCLB definition, no consensus has been reached among researchers on how to calculate high school graduation rates. One major challenge in reporting graduation rates lies in the lack of comprehensive state data collection systems that track individual students through their schooling experiences (Swanson & Chaplin, 2003). Due to limited resources, it is very difficult for schools to account for students who have left the school before graduation. The status of many transfer students has never been verifi ed by schools, and some of these unverified transfers are virtually dropouts (Archer, 2003; Haney, 2001; Kaufman, 2001). Moreover, in a country like the United St ates where education decision-making is largely based at the local level, school districts around the nation do not follow a
Education Policy Analysis Archives Vol. 12 No. 55 7 standardized data collection and reporting pro cedure. For example, the National Center for Educational Statistics (NCES) has a definiti on for high school compl etionÂ—a concept close but not equivalent to high school graduation defined by NCLBÂ—for the Common Core of Data survey; however, not all states conform to this approach (Kaufman, Alt & Chapman, 2001; Winglee, Marker, Henderson, Young & Hoffman, 2000; Young & Hoffman, 2002). The No Child Left Behind Act undoubtedly revives public attention to the high school graduation rate, but it may not bring the country much closer to a standardized reporting approach because of its substantial regulatory flexibility. In a recent study, Swanson (2003) reviewed the NCLB implementation plans that all 50 states and the District of Columbia submitted to the U.S. Department of Education. He found that 45 states and the District of Columbia proposed one of four general approa ches, and the remaining six states proposed idiosyncratic approaches. Not surprisingly, such variation in the reporting strategies across states makes an accurate national evaluation of high school graduation rates very difficult. As an effort to clarify the confusion surro unding the calculation of the high school graduation rate, this study reviews recent literat ure on and current practices in reporting high school graduation rates, compares results and tr ends in state-level graduation rates from alternative methods over 10 high school cohor ts, and estimates discrepancies between results from different calculation methods. The goal of this study is to evaluate the relative strengths and weaknesses of alternative reporting strateg ies and to contribute to a discussion of the policy implications, rather than to recommend a single best method. Review of Alternative Methods Before we move on to discuss alternative procedures to compute the high school graduation rates, we will first introduce the two major data sources for the estimation: the Current Population Survey (CPS) sponsored jointly by the U.S. Census Bureau and the Bureau of Labor Statistics (BLS), and the Common Core of Data (CCD) collected by the National Center for Education Statistics (NCES) via annual surveys of public elementary and secondary schools. The Current Population Survey (CPS) The Current Population Survey (CPS) is a m onthly survey conducted in a state-based probability sample of 50,000-60,000 households One adult in each sample household (the reference person) responds to questions regard ing all eligible household members. To be eligible to participate in the CPS, individuals must be 15 years of age or over and not in the Armed Forces, nor in institutions, such as prisons, long-term care hospitals, and nursing homes. Therefore the target population of CP S is often referred to as the civilian noninstitutional population. The CPS instrument has a series of questions on school enrollment, college attendance, and high school graduation. Questions are asked about all people in the household 3-year-old or above regarding their school attendance in the year of the survey and the previous year. Questions are also aske d on educational attainment for those who are not enrolled in schools at the time of the survey. From 1972 to 1991, the CPS survey identified high school graduates based on atten dance and completion of grade 12. Starting in 1992, CPS distinguishes completion of 12th grade from high school graduation; however,
Alternative graduation rates 8 this classification does not distinguish GE D holders from regular high school diploma recipients (Hauser, 1997; Kaufman, Alt & Chapman, 2001). The CPS collects information on both the age and school enrollment or educational attainment for the sample. A cross-tabulation of age and enrollment data allows the examination of age-grade pr ogression and school completi on rates among various age groups. This forms the basis for the multiple indicators of school progression and completion rate, such as the high school completion rate adopted by the National Goals Panel, the status dropout rate and the ev ent dropout rate reported in NCESÂ’s Condition of Education The great advantage of CPS data is that it has been collected in a reasonably uniform manner every year for nearly four decades, and is considered by some researchers as the only source of long-term trends in dropout and completion rates (Kaufman, 2001). Meanwhile, researchers also noted a number of limitations of the CPS data for estimating high school graduation rates and dropout rates. Kaufman (2001) identified two broad sources of error in the CPS dataÂ—sampling and non-sampling error. The sampling errors for national estimates in the CPS are generally within accepted range for large surveys; however, the CPS was not designed to provide estimates of small subpopulations and the sampli ng errors for subgroups can become rather large (Greene, 2002b; Kaufman, 2001). Hauser (1997) noted there was Â“substantial statistical unreliability from year to year in the CPS mea sure of attainment for minority populationsÂ” (p.160). Reliability can be improved by aggr egating across years and reporting three-year average (Hauser, 1997; Kaufman, 2001; Ka ufman, Alt & Chapman, 2001; Annie E. Casey Foundation, 2002); however, the standard errors on the state estimates are still too large to allow meaningful state-to-state comparisons. Non-sampling errors come from a variety of sources and affect all types of surveys, universe as well as sample surveys. Non-sampling error can occur when members of the target population are excluded from the samplin g frame or when sampled members of the population fail to respond. It is estimated that the CPS survey has a coverage ratio of 93 percent (Bureau of Labor Statistics, 2002b); however, for some subgroups this ratio is much lower. Historically, black and Hispanic males have had lower coverage ratios. In 1996 the coverage ratio for black males aged 20 to 29 was only about 66 percent (Kaufman, 2001). CPS used weights to adjust for the undercounting of various subpopulations; however, such weighting will introduce bias into the estimates of graduation rates if those persons missed by CPS drop out of high school at higher ra tes than those covered in the survey. A couple of other issues with the use of CPS for calculating graduation rates are internal to the design of the survey. The ta rget population for CPS interviews is the Â“civilian noninstitutional populationÂ”, therefore the grad uation rate estimates derived from such a population does not include the military personnel, and prison inmates in the base population. Two potential biases exist in such results as an estimate of a Â“trueÂ” national high school graduation rate. On the one hand, excl usion of the military personnel is likely to underestimate the graduation rate since the m ilitary services accept very few personnel without a traditional secondary school credenti al. However, such effect is barely noticeable due to the small number of military pers onnel compared to the national population (Kominski, 1987). On the other hand, the exclusion of prison inmates is likely to overestimate the graduation rate since school dr opouts are found to have a larger risk of incarceration (Pettit & Western, 2002), an d dropouts are disproportionately represented among people in prison (Greene, 2002; Harlow 2003). However, the bias introduced by exclusion of prison inmates on the high school graduation rate is yet to be evaluated.
Education Policy Analysis Archives Vol. 12 No. 55 9 Other criticisms of CPS noted that change in the CPS instrument and data collection process over the years may threaten the tren d lines derived from the data (Hauser, 1997; Kaufman, 2001; Sum & Harrington, 2003). For example, changes in the instrumentation in 1992, though intended for improvement in the measurement, make it difficult to disentangle actual change in the construct from changes related to alteration of the instrument or operation process, especially for the first f ew data collection cycles immediately after the change is made. It is possible, though, to a ssess the effect of change in instrumentation by examining long-term time series (Kaufman, Alt, & Chapman, 2001). In light of the No Child Left Behind de finition for high school graduation rate, the CPS data have three additional limitations. First, the CPS categorization of education attainment considers holders of any second ary credentials as high school completers, without distinguishing between GED certifica tes and regular high school diplomas. Therefore, the CPS indicators are the more ge nerous measures of high school completion rate, and are overestimates for the high school graduation rate defined by NCLB. Second, the CPS completion rate indicators include high school completers from both public and private schools, while the NCLB requirement only concerns public secondary schools; hence, indicators based on CPS data are somewhat off the target for the NCLB purpose. Third, the CPS target population includes adul t residents of a state, who may not have attended schools in that state. Education attainment information derived from such a sample may provide accurate information for the labor market in a state, for which the CPS was originally designed; however, such information may not be an accurate reflection of the K-12 school system in that state. The Common Core of Data (CCD) The Common Core of Data (CCD) has been a program of the US Department of EducationÂ’s National Center for Education Stat istics (NCES) since 1986. The CCD program conducts annual census surveys of all pu blic elementary and secondary schools (approximately 95,000) and school districts (appr oximately 17,000) in the country. The CCD collects a wide range of information via a s et of five surveys sent to state education departments. Most of the CCD data are obtained from administrative records maintained by the state education agencies (SEAs). The SEAs compile CCD requested data into prescribed formats and transmit the information to NCES. The CCD data are different from CPS data in several ways. First, the two programs are designed to serve different purposes although they cover some common ground. The Current Population Survey was originally esta blished to provide direct measurement of monthly unemployment, while the Common Core of Data was designed specifically to Â“provide basic information and descriptive statistics on public elementary and secondary schools and schoolingÂ” (Thurgood, Walter, Carter, Henn, Huang & Notter, et al, 2003, p. 19). This contrast in the orientation determi nes the different focuses and strategies of the two programs. The CPS data are collected fr om a state-based sample of households on a monthly basis, and standard errors are reported along with the statistics to indicate the magnitude of sampling error. In contrast, the CCD survey is a census of public elementary and secondary schools in the United States, and therefore by definition has no sampling error associated with the observations. While the CPS data collection relies on self-report, the CCD is based on administrative records collected for each school year by local education agencies. Although sampling error is not an issue in the CCD data, the accuracy of CCD
Alternative graduation rates 10 data relies heavily on the quality of record k eeping in local school districts nation-wide (Young & Hoffman, 2002). Measures of the NCLB Graduation Rate Based on CCD Data Researchers have devised multiple measur es of high school graduation rates based on the Common Core of Data. This study foc uses only on measures of the graduation rate as is suggested in the NCLB, in particular, the simple grade 9 to graduation rate, the simple grade 8 (or 10) to graduation rate, the Greene rate, the CPI rate and the Warren rate. Simple on-time graduation rate The simple on-time graduation rate is a re asonable, though simplistic, interpretation of the NCLB definition of graduation rate, which is computed by taking high school graduates at the end of senior year as a pr oportion of the ninth graders three school years earlier: 9 ) ( ) 4 ( ) 4 ( G i Year Graduates i Year i yearN N GR For example, the simple on-time graduation rate for the class of 2000 is computed by dividing the 9th grade enrollment in the fall of 1996 into the number of high school graduates in the spring of 2000. This approach answers the three basic que stions regarding the specification of the numerator, the denominator, and the time span for the high school graduation rate. Criticisms of this approach focus upon three major deficiencies. First, these comparisons ignore possible effects on the graduation rate caus ed by inor out-migration of students between the ninth and twelfth grades (Ginsb urg, Noell & Plisko, 1988; Greene, 2002; Warren, 2003). Second, special education students, reported in ungraded classes (and in some states vocational students), are not coun ted in the ninth grade enrollments, although they are counted when they graduate (Ginsb urg, Noell & Plisko, 1988; Greene, 2002b; Warren, 2003). Third, since 9th grade is a common grade for students to be retained, the denominator may be artificially larger, leadin g to an underestimate of the graduation rate (Haney, 2000; Haney, Madaus, Abrams, Wheeloc k, Miao & Gruia, 2004; Greene, 2002b; Warren, 2003). In practice, the observed grade 9 enrollment is likely to differ from the Â“trueÂ” cohort size because students move in and out of a c ohort during the four years of high school for various reasons: transfer, grade retention, diseas e or death, etc. Because of grade retention, the denominator includes students who are repeating grade 9 in addition to first time 9th graders. A number of recent studies pointed to the 9th grade Â“bulgeÂ” in public school enrollment, suggesting relatively large numbers of students repeating grade 9 instead of being promoted to 10th grade on time (Carnoy, Loeb, & Smith, 2001; Greene, 2002b; Haney, 2003; Haney, et al., 2004). Analysis of enrollment da ta over the past three decades (Haney, et al., 2004) found that nationwide the grade 9 bulge tripled, and the bulging up trend is observed in many states as well. Meanwhile, due to the practice of grade retention, the numerator of this simple rate includes not only on-time graduates but also late graduates, who have been retained at certain grade levels and hence graduate with a later cohort than the one they started 9th grade with. As a result, the simple graduation rate does not accurately identify graduates who
Education Policy Analysis Archives Vol. 12 No. 55 11 complete high school on time in four years. It is also possible for students to graduate earlier than their peers; however, since the numb er is likely to be small, early graduates will not be further discussed in this study. In sum, three possible biases exist in using the simple on-time graduation rate approach. First, the grade 9 bulge makes the denominator artificially large, which leads to an underestimate of graduation rates. Second, inclusion of retained students in the numerator tends to overestimate on-time graduation. Third, other changes in the denominator, such as transfers, bring uncertain effects into the es timate. Unaccounted net immigration makes the denominator artificially small, leading to overestimate of the graduation rate; whereas unaccounted net emigration makes the denomin ator artificially large, leading to underestimate of the graduation rate. In the absence of an ideal data source, whic h tracks individual students through their school career, a number of alternative reporti ng strategies based on CCD data are suggested in recent studies to overcome the three issues affecting the simple on-time graduation rate. To facilitate the discussion, we will use the notat ion system shown in Table 1 when referring to a high school cohort. Table 1 Notation System in Proceeding Sections Grade School year Data Co llected Notation Example 7th grade Two years before high school (e.g. school year 1994-1995) Grade 7 enrollment 7 ) 2 ( G i yearN 7 1994 GN 8th grade One year before high school (e.g. school year 1995-1996) Grade 8 enrollment 8 ) 1 ( G i yearN 8 1995 GN 9th grade Freshman year of high school (e.g. school year 1996-1997) Grade 9 enrollment 9 ) ( G i yearN 9 1996 GN 10th grade Sophomore year of high school (e.g. school year 1997-1998) Grade 10 enrollment 10 ) 1 ( G i yearN 10 1997 GN 11th grade Junior year of high school (e.g. school year 1998-1999) Grade 11 enrollment 11 ) 2 ( G i yearN 11 1998 GN 12th grade Senior year of high school (e.g. school year 1999-2000) Grade 12 enrollment12 ) 3 ( G i yearN 12 1999 GN Graduation Senior year of high school (e.g. school year 1999-2000) Number of graduates Grads i yearN) 4 ( GradsN2000 Typically students start high school at grade 9 in September, and graduate at the end of grade 12 in June. Accordingly, the CCD enrollment data are collected during the fall in October, and the number of graduates reported is based on the count of students who Â“received a diploma during the previous school year or subsequent summer schoolÂ” (NCES, 2003, Appendix C) We will refer to 9th grade year as year i of high school, 10th grade year (i+1), 11th grade year (i+2) and 12th grade year (i+3) of high school. For example, by year 1999, we mean the fall to spring school year (i.e. school year 1999-2000) rather than the January to December calendar year 1999. By class 2000, we mean the cohort that started high school in the fall of 1996 and graduated on time in the spring of 2000. However, the number of graduates reported for spring 2000 is likely to include a small number of students who started high school before or after 1996 (i.e. late graduates and early graduates in addition to on-time graduates).
Alternative graduation rates 12 Substitute grade 9 enrollment with grade 8 or grade 10 enrollment As mentioned earlier, the widespread pr actice of grade 9 retention makes the 9th grade enrollment artificially large and theref ore graduation rates based on grade 9 enrollment tend to be an underestimate of the Â“trueÂ” grad uation rate. Figures 1 presents the U.S. total grade enrollment in public schools from Kindergarten through 12th grade for the 1999-2000 school year. The figure clearly illustrates the grade 9 bulge, with the grade 9 enrollment substantially larger than the two adjacent grades, whereas enrollment in grades 8 and 10 are much closer to each other. Fi g ure 1. National Public School Enrollment Trend (1999-2000 School Year) 0 500 1000 1500 2000 2500 3000 3500 4000 4500 K1st2nd3rd4th5th6th7th8th9th10th11th12thEnrollment (in thousands) Figure 2 presents the national enrollment trend from the 1969-1970 school year to the 1999-2000 school year, which allows an examination of the enrollment change in grades 8, 9 and 10 over three decades. The vertical axis represents the percent change in enrollment between two adjacent grade levels from one year compared to the previous year. For example, the national 8th grade enrollment in fall 1997 was 3,415,000 and 9th grade enrollment in fall 1998 was 3,856,000. The percent change between grade 8 and 9 for this cohort equals the difference between 3,856,00 0 and 3,415,000 divide d by 3,415,000, which yields 11.44%. A positive change rate indicates increase in grade enrollment compared to the previous grade/year, while a negative rate indicates decrease.
Education Policy Analysis Archives Vol. 12 No. 55 13 Figure 2. Percent Enrollment Change between Grades 8, 9 & 10 (Source: Digest of Educat ion Statistics, 1970 through 2002)-15.00 -10.00 -5.00 0.00 5.00 10.00 15.0069-70 71-72 73-74 75-76 77-78 79-80 81-82 83-84 85-86 87-88 89-90 91-92 93-94 95-96 97-98 99-00Academic year% of enrollment change relative to previous grade/year (G9-G8)/G8 (G10-G9)/G9 The gray line on the top represents the percent enrollment change between grades 8 and 9, and the black line at the bottom represents the percent enrollment change between grades 9 and 10 over three decades. The pattern shown in Figure 2 warrants two observations. First, the position of the two lines in regard to the vertical axis indicates the national grade 9 enrollment is larger than the enrollment in grades 8 and 10 for each and every year during the three decades, thus eviden cing a grade 9 bulge. Second, the divergence between the two lines indicates th e increasing size of the grade 9 bulge since the early 1980Â’s. The above two figures illustrate enrollment trends at the national level. At the state level, the same pattern holds although with variation across states regarding the magnitude of the grade 9 bulge. As a result of the increa sing grade 9 bulge, using grade 9 enrollment as the denominator in calculating graduation rate is likely to underestimate the graduation rate at national and state levels. One alternative is to substitute the grade 9 enrollment with either grade 8 or grade 10 enrollment as the denominator, which is likely to result in higher graduation rate estimates. It is noted that the grade 8 to graduation proxy is applicable at district and state level but not at the school level because 8th grade and 9th grade are often assigned to different schools, e.g. middle school and high school; however, this is not an issue with the grade 10 to graduation proxy. The major advantage of the simple longitud inal approach lies in its straightforward calculation, which makes it easier to communi cate to the public and the policy makers. Also, this indicator is less demanding on data collection since schools and local districts usually keep records of enrollment consistently. Smoothing out the grade 9 bulge Another strategy is to smooth out the grade 9 bulge by averaging enrollment in several grade levels. Haney (2001) calculated the cohort graduation ra te for the nationÂ’s 100 largest school district in school year 1997-98, and the denominator he used is the average
Alternative graduation rates 14 district enrollment for grades 7 to 9 in the freshman year (9th grade year) for a given cohort. The numerator is the number of high school diploma recipients (GED excluded) in senior year. For systems where grade 9 retenti on is common, cohort graduation rates thus calculated are likely to be larger, and supposed ly more accurate, than the estimates using grade 9 enrollment as the base population. HaneyÂ’s cohort graduation rate is conceptually similar to the simple on-time graduation ratio (or the simple grade 9 rate), yet more accurate when 9th grade retention is a serious issue. Since the average is based on en rollment at three grade levels in the same school year (i.e. average of 7 ) ( G i yearN, 8 ) ( G i yearN and 9 ) ( G i yearN) rather than in three different school years (i.e. average of 7 ) 2 ( G i yearN, 8 ) 1 ( G i yearN and 9 ) ( G i yearN), the underlying assumption for this method is that grade enrollment is similar fr om cohort to cohort. The advantage of this method is that it only requires data from two school years, and therefore is not much more complicated than the simple grade 9 rate. Alternatively, Greene (2002b) used the averag e enrollment of grades 8, 9 and 10 to get the Â“smoothedÂ” estimate of the cohortÂ’s first time 9th grade enrollment, for a given cohort. Unlike HaneyÂ’s method, Greene averaged grades 8, 9, and 10 enrollment from three different school years to come up with a smoot hed estimate of grade 9 enrollment. That is, Haney smoothed out the bulge in a cro ss-sectional approach, while Greene uses a longitudinal approach. The latter is conceptually more appealing, yet the calculation is more complicated since the Â“smoothingÂ” requires th ree years of data. In practice, the crosssectional Â“smoothingÂ” strategy can be a good choice for school systems with relatively stable enrollment from cohort to cohort. Stabilizing the cohort enrollment As is discussed earlier, enrollment for a si ngle high school cohort may fluctuate due to various factors; therefore, it may be in accurate to assume that the same number of students will move along from grade 9 to gradua tion. Adjustment is desirable to deal with the natural increase or decrease in cohort enrollment through high school years. The Intercultural Development Research Association (IDRA), a non-profit minority advocacy organization, conducted the first comp rehensive dropout analysis in the state of Texas (Cardenas, Robledo & Supik, 1986). The attrition rate developed and used by IDRA included a size change ratio to adjust for the enrollment change. The size change ratio was calculated by dividing the total district high school enrollment for the senior year by the total district high school enrollment for the freshman year for a cohort. While a change ratio of 1 indicates zero net change in the cohort enrollment, a change ratio larger than 1 indicates net increase, and a chan ge ratio smaller than 1 indicates net decrease in the cohort enrollment. The graduation rate index devised by Greene (2002a, 2002b) incorporated the same idea as the IDRA change ratio to make adjustment to the smoothed estimate of the grade 9 enrollment. The numerator in the Greene formula is the number of regular diploma recipients in senior year. Therefore, th e Greene graduation rate for a given cohort can be computed by dividing the adjusted grade 9 enrollment in to the number of graduates as follows: 9 ) ( ) 4 ( ) 4 (Âˆ .G i Year Grads i Year Greene i YearN Adj N GR
Education Policy Analysis Archives Vol. 12 No. 55 15 The Greene estimate for cohort graduation rate has conceptual advantages for systems with large fluctuations in high school enrollment. Empirically, Warren (2003) found through simulation that GreeneÂ’s estimates are biased under various conditions. Hence the complexity of GreeneÂ’s method is not justifie d by its conceptual advantage and lack of empirical accuracy. The cumulative promotion index (CPI) Swanson and Chaplin (2003) developed the Cumulative Promotion Index (CPI) to estimate the high school graduation rate. It conceives of high school completion as Â“a stepwise process composed of three grade-to-g rade promotion transitions in addition to the ultimate high school completion eventÂ” (p. 19), and estimates high school graduation rate as the probability that a student entering the 9th grade will complete high school on time with a regular diploma. For example, for the high school class of 1999-2000 (i.e. the class graduating in spring 2000), the CPI graduation rate for a given jurisdiction is calculated as follows: 12 1999 2000 11 1999 12 2000 10 1999 11 2000 9 1999 10 2000 2000 G Grad G G G G G GN N N N N N N N CPI The CPI uses a so-called Â“synthetic cohortÂ” and focuses on two school years for the estimation. The formula would look as fo llows if a longitudinal cohort were used: 12 2002 2003 11 2001 12 2002 10 2000 11 2001 9 1999 10 2000 2000*G Grad G G G G G GN N N N N N N N CPI *2000CPI is equivalent to the simple grade 9 rate (number of graduates divided by the freshman year enrollment) once enrollments in grades 10 to 12 cancel out each other. The authors claimed that CPIÂ’s Â“shortened window of observationÂ” (Swanson, 2004, p.8) has several potential advantages, which are not solidly grounded. First, Swanson claimed that large changes in student demogr aphics and school practices are less likely to occur in a shorter time period. This may hold for slow and gradual changes; however, not all changes are in administrative control and can be phased in gradually. Second, Swanson claimed that using a Â“syntheticÂ” cohort requires data from only two school years, so the CPI indicator can be estimated Â“very quickly after two waves of data collection over a one-year periodÂ” (Swa nson, 2004, p.8). This claim is completely misleading. The CPI method uses data from the junior and senior years of a focal cohort; however, data for the freshman and sophomore yea rs are already collected by the time data are available for the senior year. Therefore, usi ng CPIÂ’s Â“syntheticÂ” cohort is not any faster than using a four-year longitudinal cohort. As a matter of fact, in operation the CPI method still needs four years of data to determine inclusion of districts. Districts have to be in operation for at least four years and have not experienced boundary changes in order to be included in the SwansonÂ’s analysis. Third, Swanson and Chaplin claimed that the CPI weighs heavily the contemporary conditions (rather than the past conditions), and therefore provides Â“a more legitimate basis for estimating the current level of educati onal system performance and also for imposing sanctions that are experienced in the presentÂ” (Swanson & Chaplin, 2003, p. 21). However, the CPI rate is a significant departure from th e NCLB definition of high school graduation rate. In essence, the CPI rate indicates how well a system (school, district, state or country) promotes students from one grade level to another in a given pair of school years. The CPI
Alternative graduation rates 16 indicator can be a valuable measure for evaluati ng school systems, but not for the purpose of the No Child Left Behind legislation: it do es not really tell how well a system graduates students from high school in a standard number of years; at least, it is not a good way of providing information NCLB requires and the advantages claimed by the authors do not hold under close examination. Two additional issues are worth mentioning in the calculation of the CPI rate. First, the reported CPI rates are computed at school district level; the district rates are then weighted and aggregated to state and national le vel. Estimates based on aggregated district rates are likely to be inaccurate since the author sÂ’ operation rules excluded one-quarter of the districts from the analysis. Although such exclusion rules are justified for district level analysis, it brings unknown bias to the aggrega ted CPI rates at state and national level. Since CCD collects census data for all public schools an d report at state level, it is obviously a better procedure, in terms of efficiency and accuracy, to estimate state rates directly from CCD state data, and to estimate the national rate based on national data aggregated from state level. The CPI rate is a product of four progression rates (grades 9 to 10, 10 to 11, 11 to 12, and 12 to graduation), which have a theor etical range of 0 to 1. However, sometimes the progression rate can exceed 1 due to unique distri ct context. Progression rates that are larger than 1 yet smaller than 1.1 are Â“trimmedÂ” to 1; while even larger values are censored and assigned a missing value code. Such strategies are justified for obtaining meaningful district rates, however they also lead to exclusion of more districts, which increases the bias in aggregated state and national rates. In addition, the CPI estimates are subject to the same issues affecting the simple ontime graduation rate, namely, grade retention, transfer students and ungraded special education students. All these may lead to biases in the graduation rate estimates. Supplementing CCD with CPS Data The reader may come to a reasonable observation that the CCD enrollment data have limitations for estimating high school gr aduation rate. Through simulation of various conditions, Warren (2003) was able to illustra te systematic biases in the graduation rate estimates using the simple on-time graduati on rate method, the Greene method, and the Cumulative Promotion Index (CPI): In the case of positive net migration (i.e. in crease in cohort enrollment due to transfer), all three methods overestimate the graduation rate. In the case of negative net migration (i.e. d ecrease in cohort enrollment due to transfer), all three methods underestimate the graduation rate. In the case of grade 9 retention, all thr ee methods underestimate the graduation rate. When multiple factors are at work such as cohort size increase (i.e. increase in the size of entering class from year to year), negative net migration and grade 9 retention, the three methods underestimate the graduation rate with varying magnitude. Warren (2003)3 proposed a measure using CCD enrollment data supplemented with CPS data to adjust for grade retention and migration. This new measure conceptually 3 Warren proposed a revised measure for the high school graduation rate in 2004. For detail, see http://www.soc.umn.edu/%7Ewarre n/Warren%20---%20July%202004.pdf.
Education Policy Analysis Archives Vol. 12 No. 55 17 represents Â“the percentage of incoming public school 9th graders in a particular state and in a particular year who obtain a regular high school diploma within four or five years of first starting 9th gradeÂ” (p. 12). In essence, WarrenÂ’s measure is computed by making adjustments to both the numerator and denominat or of the simple on-time graduation rate. The denominator is adjusted by multiplying the observed CCD number of enrolled public school 9th graders in a particular state and a partic ular year by (1) the proportion of first time 9th graders in that state and year (denoted as P), and by (2) one plus the net migration rate (denoted as MR) for members of that particular cohort in that state. As for the numerator, the observed CCD number of graduates for a given year includes both on-time graduates and late gr aduates who were retained from previous cohort(s). On the other hand, some students of the reference cohort are retained and will not graduate until a year later than their peer s. For example, the number of graduates reported by CCD for school year 1999-2000 in a given state includes on-time graduates (those who started high school in fall 1996) and late graduates (those who started high school in the fall of 1995 or earlier). Meanwhile not all students started high school in the fall of 1996 graduate in 2000, some might have been retained or otherwise delayed, and graduate in 2001 or later. Therefore, in order to calc ulate the graduation rate for the class graduating in 2000, the adjusted numerator equals the observed CCD number of graduates minus the number of graduates who started high school in 1995 (denoted as N1), and plus the number of retained students who started high school in 1996 and graduate in 2001 (denoted as N2). Therefore, WarrenÂ’s measure for gradua tion rate can be expressed as follows: ) 1 ( *9 ) ( 2 1 ) 4 ( ) 4 (MR P N N N N GRG i Year Grad i year warren i Year In this formula, the number of graduates (Grads i YearN) 4 () and grade 9 enrollment (9 ) (G i YearN) are available from CCD files, while the other parameters (N1, N2, P and MR) are estimated based on CPS data (Warren, 2003). Through simu lation, Warren was able to illustrate that, theoretically, this new measure is not affected by migration, grade retention, and cohort size change. However, in practice, the mea sure is also subject to error because N1, N2, P and MR are estimated from CPS data, which are influenced by various sources of error. Therefore, the empirical accuracy of WarrenÂ’s measure of graduation rate remains to be evaluated. Using the measure thus defined, Warren (2003) estimated graduation rates for nine high school classes graduating from 1992 to 2000 at both state and national levels, and compared the results to the simple on-time graduation rate estimates. It appears that the two approaches yield very similar trends at nati onal level, yet WarrenÂ’s estimates are slightly higher (by 0.3% to 1.9%) than the simple ontime rate estimates. At state level, the two approaches yield similar state rankings for the class of 2000 except for the District of Columbia and Nevada, both of which have mu ch higher migration rates (-15% and 30% respectively) than the other states (ranging from -5% to 10%). In sum, WarrenÂ’s adjusted graduation rate measure is conceptually more accurate after accounting for migration, retention and cohort size change. Operationally, the computation for this measure requires bot h CCD and CPS data and is much more complicated. The empirical accuracy of the measur e is yet to be determined. For states with relatively low migration rates, WarrenÂ’s estima tes correlate highly with the simple on-time graduation rate.
Alternative graduation rates 18 High school graduates compared with population 17 years of age Another indicator based on both the CCD and CPS data is Â“graduates as a percent of 17-year-old populationÂ” (Snyder & Hoffman, 20 03, p.127). This indicator is reported by the National Center for Education Statistics in the Digest of Education Statistics back to 18691870 (DES, 2002). The denominator of this indicator is derived from Current Population Reports, which is based on the CPS survey an d reflects the October 17-year-old civilian noninstitutional population in a given year (e.g. October 1999). The numerator includes graduates of regular day school programs from both public and private schools in the spring of the next year (e.g. spring of 2000). Although a potentially accurate indicator for the ontime school progression behavior of the 17-year-old population, the DES 17-year-old rate is a significant departure from the graduation rate defined in NCLB. In addition, this measure is not designed to report for subpopulations due to the limitations of the CPS sampling design. In sum, this section introduced two major data sources for estimating the high school graduation rate and reviewed alternati ve measures based on the Common Core of Data (see Table 2 for a summary of these methods). A number of recent studies compared alternative graduation rate estimates for part icular years, at state level and/or for major ethnic groups. However, little research effo rt has been devoted to a comprehensive examination of the alternative graduation rate es timates over a longer period of time. The purpose of this study is to compare results and trends in national, state and state ethnic group level graduation rate estimates using alternative methods over time, and to evaluate the discrepancies between these results.
Alternative High School Graduation Rates 19 Table 2 Overview of Alternative Methods Method Formula Class 2000 Grad. Rate Source Simple Grade 8 Rate 8 ) 1 ( ) 4 ( 8 ) 4 (G i Year Grad i Year G i YearN N GR 75.9% Computed Simple Grade 9 Rate 9 ) ( ) 4 ( 9 ) 4 (G i Year Grad i Year G i YearN N GR 67.0% Computed Simple Grade 10 Rate 10 ) 1 ( ) 4 ( 10 ) 4 (G i Year Graduates i Year G i YearN N GR 75.4% Computed The Greene Rate ( ) ( ) ( * 312 ) 3 ( 11 ) 3 ( 10 ) 3 ( 9 ) 3 ( 10 ) 1 ( 9 ) ( 8 ) 1 ( 12 ) ( 11 ) ( 10 ) ( 9 ) ( ) 4 ( ) 4 ( G i Year G i Year G i Year G i Year G i Year G i Year G i Year G i Year G i Year G i Year G i Year Grad i Year Greene i YearN N N N N N N N N N N N GR 69.0% Greene, 2002 The CPI Rate 12 ) 3 ( ) 4 ( 11 ) 3 ( 12 ) 4 ( 10 ) 3 ( 11 ) 4 ( 9 ) 3 ( 10 ) 4 ( ) 4 ( G i year Grads i year G i year G i year G i year G i year G i year G i year i yearN N N N N N N N CPI 66.6% Swanson & Chaplin, 2003 The Warren Rate (2003) 9 ) ( ) 4 ( ) 4 (. / .G i year Grad i year Warren i yearN Adj N Adj GR 67.5% Warren, 2003 Methods and Results This study uses the grade enrollment and numbers of graduates4 at national and state levels to estimate U.S. high graduation rates The data are from two primary sources: the Common Core of Data (CCD) and the Digest of Education Statistics (DES), both published by the National Center for Educational Statistics (NCES). The state data are available from the CCD state nonfiscal files since 1986-87 school year5. The U.S. national6 enrollment and graduation data are obtained from the DES since the CCD program does not have the national aggregates in a readily available format. Moreover, the DES has national data available back to the 1968-1969 school year, which provides a much longer period for examining the national trends in high school graduation rates. Beginning in the 1992-1993 school year, enrollment and graduation data are broken down by the five major ethnic groups in most states, which allows for the comparison of the high school graduation rate across races. The race level analysis in this study will focus on three groups, white, black and Hispanic. 4 NCES changed the reporting categories fo r high school graduates since CCD 1997. See Appendix I for details. 5 State enrollment and graduation data from both CCD and DES were compared at state level for the 1986-87 to 2001-02 school years. Th ree large discrepancies were identified from these comparisons (see Appendix II). Based on the data from adjacent years, the current study resolves the discrepancies by adopting the DES reported data for all three cases. 6 The U.S. national data includes the 50 states and the District of Columbia.
Education Policy Analysis Archives Vol. 12 No. 55 20 The data allows for multiple levels of analyses to study the effect of alternative methods on the high school graduation rate esti mates. This study reports results of the analyses at three levels, namely, graduation ra tes at the national level, graduation rates for individual states, and graduation rates for major ethnic groups within states. National Level Analysis Alternative methods are applied to the national enrollment and graduation data complied from the DES to estimate high school graduation rates. These methods include (1) the simple grade 8 to graduation rate, (2) the simple grade 9 to graduation rate, (3) the simple grade 10 to graduation rate, (4) the Greene rate, and (5) the CPI rate. It is noted that the state and national CPI rate is comp uted here in a slightly different way from the published CPI rates (Swanson and Chaplin, 2003; Swanson, 2004). Rather than first computing district level CPI rates and then a ggregating to state and national level, the current study computes the national and state CPI rate directly from national and state level enrollment and graduation data. Despite this procedural difference, the national CPI rate computed in this study is conceptually the same as the measure proposed by Swanson and Chaplin. However, the results computed in the current study are likely to differ from Swanson and ChaplinÂ’s results. Such differences are probably attributed to the fact that Swanson and Chaplin excluded some districts in their analysis. The national CPI rate computed in this study is based on national data collected in a census approach, and therefore is likely to be more accurate. High school graduation rate estimates ar e not computed using WarrenÂ’s method, which requires the CPS data and more complex procedures. However, reported Warren rates (Warren, 2003) for classes 1992 to 2000 are compared to estimates yielded from other methods for respective years. The alternative graduation rate estimates are tabulated and graphed to reveal the trend of the national high school graduation rate over the past three decades. The national level analysis addresses the following research questions: What are the national graduation rates in the past three decades based on different methods? What are the trends in the national high school graduation rates based on different methods? Do these methods yield si milar or different patterns? Are the graduation rates yielded from different methods related? What are the directions of these relationships? How strong are these relationships? When different methods yield different hi gh school graduation rate estimates, what are the magnitudes of the differences? Alternative national rate estimates and trends Table 3 lists the graduation rate estimates for the past three decades. For example, the following are the six graduate rate estimates for the class of 2000: The simple grade 8 rate is 75.9%, which is the number of high school graduates in the spring of 2000 divided by the grade 8 enrollment in the fall of 1995; The simple grade 9 rate is 67.0%, which is the number of high school graduates in the spring of 2000 divided by the grade 9 enrollment in the fall of 1996; The simple grade 10 rate is 75.4%, which is the number of high school graduates in the spring of 2000 divided by the grade 10 enrollment in the fall of 1997;
Alternative High School Graduation Rates 21 The Greene rate is 69.6%, which is the number of high school graduates in the spring of 2000 divided by adjusted grade 9 enrollment in the fall of 1996; The CPI national rate7 is 67.5%, which is the product of the progression rates from 1999-2000 to 2000-2001 between every two adjacent high school grades; The Warren rate is 67.5%, which is the adjusted number of high school graduates in the spring of 2000 divided by adjusted grade 9 enrollment in the fall of 1996. Table 3 Alternative Estimates for National High School Graduation Rates Year of Graduation Grade 8 Rate1 Grade 9 Rate1 Grade 10 Rate1 Greene Rate1 CPI National Rate1 Warren Rate2 1973 79.8% 76.5% 79.0% 73.9% 75.1% n/a 1974 78.6% 75.6% 77.3% 73.3% 73.6% n/a 1975 78.4% 74.7% 77.4% 74.9% 76.7% n/a 1976 78.0% 75.0% 77.7% 74.7% 74.5% n/a 1977 77.7% 74.6% 77.2% 75.1% 73.7% n/a 1978 76.8% 73.7% 75.9% 74.8% 71.5% n/a 1979 76.0% 72.6% 75.4% 76.1% 71.1% n/a 1980 75.6% 71.8% 74.6% 77.8% 72.2% n/a 1981 76.2% 72.1% 75.5% 80.4% 72.9% n/a 1982 76.5% 72.6% 76.6% 82.6% 73.8% n/a 1983 77.4% 73.7% 77.1% 83.5% 74.6% n/a 1984 78.7% 73.9% 77.5% 82.4% 72.6% n/a 1985 78.2% 73.5% 77.0% 79.0% 71.9% n/a 1986 77.9% 73.4% 76.8% 76.0% 72.5% n/a 1987 77.8% 72.9% 77.2% 75.6% 71.8% n/a 1988 77.6% 72.7% 77.4% 77.3% 70.6% n/a 1989 77.2% 71.5% 76.5% 79.5% 70.0% n/a 1990 77.8% 71.3% 76.8% 81.4% 71.3% n/a 1991 77.9% 71.1% 77.2% 80.2% 72.2% n/a 1992 78.4% 71.7% 77.6% 76.7% 71.2% 73.3% 1993 78.3% 71.1% 77.1% 73.0% 69.3% 73.0% 1994 77.8% 70.1% 76.2% 70.4% 67.0% 71.9% 1995 76.3% 68.6% 75.1% 68.9% 67.1% 70.4% 1996 75.3% 67.8% 74.5% 67.7% 67.9% 69.2% 1997 75.4% 67.6% 75.3% 67.2% 66.1% 70.0% 1998 75.1% 67.7% 75.3% 67.6% 66.2% 68.5% 1999 75.3% 67.1% 74.8% 68.2% 66.8% 67.8% 2000 75.9% 67.0% 75.4% 69.6% 67.5% 67.5% 2001 75.5% 67.2% 75.9% 70.0% n/a n/a 1. Graduation rate estimates computed using national data from the Digest of Education 2. Graduation rate estimates report ed in Warren, 2003, Table 7. Figure 3 is a graphic presentation of the information in Table 3. The horizontal axis indicates the year in which students graduate. For example, the graduating class of 1973 refers to the cohort of students who started 9th grade in the fall of 1969 and graduated on 7 The computed CPI rate at national level is labeled as Â“CPI national rateÂ” rather than Â“Swanson rateÂ” in order to distinguish from the national rate reported by the Urban Institute. As mentioned earlier, this study co mputes the rate directly from national level data, while the Urban Institute researchers computed the national rate based on district aggregate.
Education Policy Analysis Archives Vol. 12 No. 55 22 time in the spring of 1973. The vertical axis indicates the graduation rate estimates. The lines with different markers represent graduate rate es timates yielded from the different estimation methods. Figure 3. Alternative U.S. Graduation Rate Estimates0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0%1973 1975 1977 1979 1981 1983 1985 1987 1989 1991 1993 1995 1997 1999 2001Graduating ClassRate Grad/G8 Grad/G9 Grad/G10 Greene rate CPI national Warren rate The national graduation rate estimates presented in Table 3 and Figure 3 suggest several observations. First, the national graduation rate slightly decreases from 1973 to 2001 regardless of the method used for the estimation. In 1973, the graduation rate estimates range between 73.9% (the Greene rate) and 79.8% (simple grade 8 rate), whereas the estimates for 2001 range between 67.2% (sim ple grade 9 rate) and 75.9% (simple grade 10 rate). One way to verify the observed decline in the six trend lines is by fitting an Ordinary Least Square (OLS) regression model on these observations and testing the significance of the slopes. Table 4 lists the OLS slopes and their significance level, which indicates that the decline in each of the six trend lines is statistically significant. This means the negative slopes of the OLS regression lines are significantly diffe rent from zero, hence indicating a declining trend in the national graduation rate. It is noted, however, that the OLS models are only a rough approximation to the data in this case since the national high school graduation rates are probably not independent from year to year. Also, in the case of the Greene rate, the relationship between the two variables is not a linear pattern, and therefore the OLS model is not a good fit. Table 4 Ordinary Least Square (OLS) Slopes of the Trend Line Unstandardized Coefficients Standardized Coefficients t Sig. Model B Std. Error Beta Simple Grade 8 Rate1 -.001 .000 -.546 -3.389 .002 Simple Grade 9 Rate1 -.003 .000 -.932 -13.406 .000 Simple Grade 10 Rate1 -.001 .000 -.505 -3.038 .005 Greene Rate1 -.003 .001 -.515 -3.120 .004
Alternative High School Graduation Rates 23 National CPI Rate1 -.003 .000 -.886 -9.739 .000 Warren Rate2 -.008 .001 -.972 -10.871 .000 1. Based on 28 years of observations (1973 to 2001). 2. Based on nine years of observations (1992 to 2000). Figure 4 is a re-scaled version of Figure 3, which allows a closer look at the trend lines of alternative graduation rate estimates. Five of the six trend lines appear to be quite stable from year to year, while the line repres enting the Greene estimates wildly fluctuates and departs from other methods during late 1970Â’s through the 1980Â’s. This raises questions about the reliability and validity of the Green e method. The line representing the CPI national rates appears less smooth than the three simple rates and the Warren rates, yet much more stable compared to the Greene rates. Figure 4. Alternative U.S. Graduation Rate Estimates (Rescaled)60.0% 65.0% 70.0% 75.0% 80.0% 85.0% 90.0%1973 1975 1977 1979 1981 1983 1985 1987 1989 1991 1993 1995 1997 1999 2001Graduating ClassRate Grad/G8 Grad/G9 Grad/G10 Greene rate CPI national Warren rate Third, if we put the Greene rate aside, the remaining five lines fall into two groups: (1) the simple grade 8 rates and simple grade 10 rates are close to each other, and (2) the simple grade 9 rates, the CPI national rates, and the Warren rates are close to each other. In addition, group 1 methods yield consistently high er estimates than group 2 methods, and the differences have been increasing over the year s. Such differences between the two groups are supported by the standardized OSL estimates presented in Table 4, with the simple grade 8 rate and the simple grade 10 rate having standardized slopes of over -.50, while the other three lines having standardized slopes around -.90. This pattern is not surprising given the increasing size of the grade 9 bulge over time at the national level, which was shown earlier in Figure 2. Simple arithmetic operation8 is sufficient to show that the larger the grade 9 bulge, the larger the difference between the simp le grade 8 rates and the simple grade 9 rates. Similarly, the higher the grade 9 to 10 attrit ion rate, the larger the difference between the simple grade 9 rates and the simple grade 10 rates. 8 See Appendix III for an illustration of the effect of grade enrollment change rate on the difference between alternative simple graduation rate estimates.
Education Policy Analysis Archives Vol. 12 No. 55 24 Correlation Analysis The correlation coefficient is a useful tool for examining the relationship between the alternative sets of graduation rate estima tes. Since the alternative methods are intended to estimate the same Â“trueÂ” graduation rate, the estimates should be highly correlated even though the observed values are somewhat different. We would expect the correlations to be fairly high; however, low correlations between alternative graduation rate estimates suggest the existence of other factors that add Â“disturbanceÂ” to the measurement. Table 5 lists the correlation coefficients between these alternative estimates for the national graduation rates. Table 5 Correlations Between Alternativ e National Graduation Rates Grade 8 Rate Grade 9 Rate Grade 10 Rate Greene Rate National CPI Grade 9 Rate .780**(N=29) Grade 10 Rate .925**(N=29) .745**(N=29) Greene Rate .454*(N=29) .609**(N=29) .463*(N=29) National CPI .635**(N=28) .907**(N=28) .669**(N=28) .684**(N=28) Warren .905**(N=9) .968**(N=9) .850**(N=9) .781*(N=9) .688*(N=9) ** Correlation is significant at the 0.01 level (2-tailed). Correlation is significant at the 0.05 level (2-tailed). Table 6 summarizes these correlation coefficients in the order of their magnitude. The 15 correlation coefficients fall into three groups9: high correlations (r > .8), moderate correlations (.6 < r < .8), and low correlations (r < .5). Table 6 Correlations between Alternative National Rates (Sorted by Value) Group Methods Correlation Coefficients Number of Years Compared High Grade 9 vs. Warren .968** 9 Grade 8 vs. Grade 10 .925** 29 Grade 9 vs. CPI national .907** 28 Grade 8 vs. Warren .905** 9 Grade 10 vs. Warren .850** 9 Moderate Greene vs. Warren .781* 9 Grade 8 vs. Grade 9 .780** 29 Grade 9 vs. Grade 10 .745** 30 CPI national vs. Warren .688* 9 CPI national vs. Greene .684** 28 Grade 10 vs. CPI national .669* 28 Grade 8 vs. CPI national .635** 28 Grade 9 vs. Greene .609** 29 9 The four groups are arbitrarily decided and are more stringent than the interpretation rule of thumb that correlation coefficients of larger than .9 are very high, .7 to .9 high, .5 to .7 moderate, .3 to .5 low, and coefficients lower than .3 are considered little or no relationship (Hinkle, Wiersma, & Jurs, 1998).
Alternative High School Graduation Rates 25 Low Grade 10 vs. Greene .463* 29 Grade 8 vs. Greene .454* 29 ** Correlation is significant at the 0.01 level (2-tailed). Correlation is significant at the 0.05 level (2-tailed). The correlation structure among the alternative estimates is in agreement with the previous observations of the trend lines in Figures 3 and 4. The simple grade 8 and grade 10 rates are highly correlated (r = .925). The Warren rate is highly correlated with th e three simple gradua tion rates (r = .968 with the simple grade 9 rate, r = .905 with the simple grad e 8 rate, and r = .850 with the simple grade 10 rate). The simple grade 9 rate is moderately correla ted with the other two simple rates (r = .780 with the simple grade 8 rate, and r = .745 with the simple grade 10 rate). The correlation between the CPI national rate and the simple grade 9 rate is high (r = .907), and the correlations between the CPI national rate and the simple grade 8 rate (r = .635) and the simple grad e 10 rate (r = .669) are moderate. The correlation between the Greene rate and the simple grade 9 rate is moderate (r = .609), and the correlations between the Greene rate and the other two simple rates are low (r = .454 with the simple grade 8 ra te, and r = .463 with the simple grade 10 rate). It is noted that since the graduation rate estimates are available for different numbers of years, the correlation coefficients are based on different numbers of cases. Correlation coefficients yielded from larger numbers of ca ses are likely to be more stable than those based on smaller numbers of observations. Effect Sizes Even though correlated, the estimates based on alternative methods are different in value. Effect sizes are computed in order to assess the magnitude of such differences. Mathematically, graduation rates are the sa me as proportions, ranging between 0 and 1. Therefore the effect size of the differen ce between two graduation rate estimates (r1 and r2) can be computed as follows: 2 / )] 1 ( ) 1 ( [2 2 1 1 2 1r r r r r r ES By using effect sizes, the difference b etween two groups is Â“represented as a proportion of the standard deviation of a reference group, and thus standardizes the differenceÂ” (Pedulla, Abrams, Madaus, Russell, Ramos, & Miao, 2003, p. 21). The interpretation of the magnitude of effect si zes depends on the discipline and situation. According to the criterion of Cohen (1965), an effect size of .25 is considered small, .50 medium, and 1.0 large (as cited in Hinkle, Wier sma, & Jurs, 1998, p.339). Alternatively, Feldt (1977) considers a standardized effect size of .2 0 as small, .50 medium, and .80 large (as cited in Hinkle, Wiersma, & Jurs 1998, p.339). In pra ctice, an effect size of over half a standard deviation is rare (Mosteller, 1995, p.120). The effect sizes in the current study should be interpreted differently from results of traditional experimental (or quasi-experimental) studies. In experimental studies, the objective is for a treatment to create a signif icant difference, and the research hypothesis
Education Policy Analysis Archives Vol. 12 No. 55 26 usually predicts a significant effect size. In th e current study, the focus is to examine the difference between two estimates of the same co nstruct (the high school graduation rate for a given cohort). Since the alternative methods are expected to approximate the same Â“trueÂ” graduation rate, the effect sizes are expected to be small in the current study. Table 7 lists the effect sizes of the differe nces between selected sets of estimates. For example, for the class of 2000, the standa rdized difference between simple grade 8 rate and simple grade 9 rate is .198, meaning that the simple grade 8 rate is higher than the simple grade 9 rate by almost one fifth of a stan dard deviation. For the class of 2001, the standardized difference between simple grad e 8 rate and simple grade 10 rate is -.011, meaning that the simple grade 8 rate is lower th an the simple grade 10 rate by slightly over one percent of a standard deviation. Just by ex amining the values in Table 7, we can see that the difference between the grade 8 rate and gr ade 9 rate (G8R vs. G9R) has more than doubled over the past three decades. In addi tion, the differences between grade 8 rates and grade 9 rates are much larger compared to those between grade 8 rate and grade 10 rate (G8R vs. G10R). The differences between the grade 10 rate and grade 9 rate (G10R vs. G9R) experienced even larger increases over the same time period. Table 7 Standardized Difference between Alternative Rates Class G8R vs. G10R G8R vs. G9R G10R vs. G9R Greene vs. G9R CPI vs. G9R Warren vs. G9R1 1973 0.021 0.080 0.059 -0 .062 -0.034 n/a 1974 0.030 0.070 0.040 -0 .052 -0.046 n/a 1975 0.024 0.088 0.064 0.004 0.048 n/a 1976 0.008 0.070 0.062 -0 .008 -0.012 n/a 1977 0.013 0.073 0.060 0.010 -0.021 n/a 1978 0.023 0.073 0.050 0.025 -0.050 n/a 1979 0.014 0.077 0.062 0.080 -0.034 n/a 1980 0.024 0.085 0.061 0.137 0.007 n/a 1981 0.016 0.093 0.077 0.196 0.017 n/a 1982 -0.001 0.091 0. 092 0.241 0.028 n/a 1983 0.007 0.087 0.080 0.240 0.020 n/a 1984 0.028 0.113 0.085 0.206 -0.029 n/a 1985 0.031 0.111 0.081 0.130 -0.035 n/a 1986 0.026 0.106 0.079 0.060 -0.019 n/a 1987 0.013 0.112 0.099 0.060 -0.026 n/a 1988 0.005 0.114 0.109 0.106 -0.047 n/a 1989 0.017 0.130 0.114 0.186 -0.033 n/a 1990 0.023 0.151 0.127 0.240 0.002 n/a 1991 0.016 0.156 0.139 0.213 0.025 n/a 1992 0.019 0.156 0.137 0.115 -0.010 0.037 1993 0.028 0.166 0.138 0.043 -0.039 0.043 1994 0.039 0.178 0.138 0.007 -0.066 0.040 1995 0.028 0.173 0.145 0.005 -0.034 0.038 1996 0.017 0.166 0.149 -0.002 0.002 0.030 1997 0.001 0.172 0.171 -0 .008 -0.033 0.051 1998 -0.006 0.164 0.171 -0.001 -0.031 0.018 1999 0.011 0.181 0.170 0.022 -0.006 0.015 2000 0.010 0.198 0.187 0.057 0.011 0.011 2001 -0.011 0.183 0. 194 0.060 n/a n/a 1. n/a: rates not available for comparison.
Alternative High School Graduation Rates 27 The data in Table 7 are graphed to facilitate understanding of these effect sizes. Figure 5 shows the standardized differences be tween simple grade 8 rate and simple grade 9 rate (G8R vs. G9R), and the standardized differences between the simple grade 10 rate and simple grade 9 rate (G10R vs. G9R). Figure 5. Standardized Differences between Simple Rates0.000 0.050 0.100 0.150 0.200 0.2501973 1975 1977 1979 1981 1983 1985 1987 1989 1991 1993 1995 1997 1999 2001Graduating ClassStandard Deviation Units G8R vs. G9R G10R vs. G9R Figure 5 warrants two observations abou t the difference between the grade 8 rate and the grade 9 rate. First, all differences are pos itive, meaning that the simple grade 8 rate is consistently higher than the simple grade 9 rate. Second, despite fluctuations, the difference between the grade 8 and grade 9 rates has increased by about one tenth of a standard deviation during the past three decades. In 1973, the standardized difference was 0.080, as compared to 0.183 in 2001. The same patter n holds for the differences between the simple grade 10 rate and the simple grade 9 rate (G10R vs G9R), and the increase in the effect size is even larger, from .059 in 1973 to 0.194 in 2001. These observations are consistent with the earlier observations concerning national grade enrollment (see Figure 2). That is, the increasing difference between the grade 8 rate and the grade 9 rate corresponds with the incr easing grade 9 bulge, and the increasing difference between the grade 10 rate and the grade 9 rate corresponds with the increasing grade 9 to 10 attrition rate. Figure 6 shows the standardized differen ces between simple grade 8 rate and simple grade 10 rate.
Education Policy Analysis Archives Vol. 12 No. 55 28 Figure 6. Standardized Differences between Simple Grade 8 Rates and Simple Grade 10 Rates-0.050 0.000 0.050 0.100 0.150 0.200 0.2501973 1975 1977 1979 1981 1983 1985 1987 1989 1991 1993 1995 1997 1999 2001Graduating ClassStandard Deviation Unit For most years during the observed time period (25 out of 29 years), the standardized difference is positive; this means th e simple grade 8 rate tends to be higher than the simple grade 10 rate. These positive values indicate that for most cohorts during the past three decades more students were enrolled in grade 10 nationwide than were enrolled in grade 8 two years earlier. However, the magnit ude of the standardized differences between the grade 8 rate and the grade 10 is less than .04 of one standard deviation, much smaller than the effect sizes observed in Figure 5, meaning the enrollment difference between grades 8 and 10 is much smaller compared to th e difference between grades 8 and 9 or the difference between grades 9 and 10. One compelling explanation of the enormous bulge in grade 9 enrollment is the common practice of holding students back at grade 9 (Greene, 2003; Haney, et al, 2004). An increasing number of students around the nation are repeating grade 9 instead of moving on to grade 10, which causes a Â“jamÂ” in the flow of students at grade 9. Figure 7 shows the standardized differences between the Greene rates and the simple grade 9 rates. For most years during the observed time period (23 out of 29 years), the positive values indicate that Greene rates are higher than the simple grade 9 rates. This is not surprising given that the Greene method smoothe s out the grade 9 bulge through adjustment and hence has a smaller denominator than that of the simple grade 9 rate. However, the magnitude of the standardized differences vary substantially from year to year, with a low of close to zero and a high of nearly a quarter of one standard deviation, which is substantial given that these measures are intended to estimate the same graduation rate.
Alternative High School Graduation Rates 29 Figure 7. Standardized Differences between Greene Rates and Simple Grade 9 Rates-0.100 -0.050 0.000 0.050 0.100 0.150 0.200 0.2501973 1975 1977 1979 1981 1983 1985 1987 1989 1991 1993 1995 1997 1999 2001Graduating ClassStandard Deviation Uni t Figure 8 shows the standardized differen ces between the CPI national rates and the simple grade 9 rates. The relationship between the two rates has been inconsistent over the past three decades. However, the magnitudes of the standardized differences are less than one tenth of a standard deviation in both directions, much smaller than the large magnitudes observed in Figures 5 and 7. Figure 8. Standardized Differences between CPI National Rates and Simple Grade 9 Rates-0.100 -0.050 0.000 0.050 0.100 0.150 0.200 0.2501973 1975 1977 1979 1981 1983 1985 1987 1989 1991 1993 1995 1997 1999Graduating ClassStandard Deviation Unit Figure 9 shows the standardized differe nces between the Warren rates and the simple grade 9 rates from 1992 to 2000. Consistent with the earlier observations from Figure 4, the Warren graduation rate estimates are slightly higher than the simple grade 9 rates, with standardized differences within one twentieth of a standard deviation, which are small relative to the large differences observed in Figures 5 and 7.
Education Policy Analysis Archives Vol. 12 No. 55 30 Figure 9. Standardized Differences between Warren Rates and Simple Grade 9 Rates0.000 0.050 0.100 0.150 0.200 0.2501992 1993 1994 1995 1996 1997 1998 1999 2000Graduating ClassStandard Deviation Unit Summary of National Level Analyses and Results Alternative national high school graduation rate estimates are computed for the past three decades using national enrollment and graduation data published in the Digest of Education Statistics Regardless of the method used, the national graduation rate shows a slightly declining trend from the early 1970Â’s to the end of the century, and the magnitude of the decrease depends on the estimation method used. Analysis of the trends suggests that th e three simple methods produce the most reliable estimates over the past three deca des. While the simple grade 8 and grade 10 methods yield very close estimates, the simple grade 9 rates are consistently lower. Moreover, the differences between the simple grade 9 rate and the other two simple rates have been increasing over the past three decades and are substantial in magnitude by the class of 2001 (over 8% for a given cohort). Two of the thr ee adjusted methods, the CPI national method and the Warren method, yield estimates and trends similar to those resulting from the simple grade 9 rate. Results from the Greene method ap pear to be unstable and depart from other methods during the late 1970Â’s through the 1980Â’s. The correlation structure among the alternative estimates is consistent with the observations based on trend lines. Correlations among the three simple rates, the CPI national rates, and the Warren rates are either high or moderate, while the correlations between the Greene estimates and the ot her estimates are moderate or low. Analysis of standardized differences yiel ds findings consistent with the trend pattern and the correlation structure. The simple grad e 9 rates are consistently lower than the other two simple rates, and the standardized diffe rences have more than doubled, which corresponds with the increasing grade 9 bulge and grade 9 to 10 attrition rate. The differences between the simple grade 8 rates and simple grade 10 rates are small and consistent, and so are the differences between the Warren rates and the simple grade 9 rates. The Greene estimates tend to be larger than the simple grade 9 rates, yet the standardized differences vary substantially over the past th ree decades and are substantial at times. The differences between the CPI rate and the simple grade 9 rate indicate no clear pattern, and the magnitude of the differences between CPI rate and grade 9 rate are small compared to
Alternative High School Graduation Rates 31 those between the Greene rate and the simple grad e 9 rate, or the simple grade 8 rate and the simple grade 9 rate. In summary, the above evidence leads to two conclusions at national level: (1) five of the six methods yield reliable results and similar trends, yet differences in the graduation rate due to estimation methods are substantial at times; and (2) the Greene method, despite its conceptual advantages, yields empirically unstable results at the national level. State Level Analysis The five alternative methods (the simple grade 8 rate, the simple grade 9 rate, the simple grade 10 rate, the Greene rates, and the CPI state rate10) are also applied to enrollment and graduation data for the 50 states to estimate high school graduation rates for each state. As CCD state level data are only available since the 1986-1987 school year, the state level analysis includes 10 high school cohorts graduating in the 1991-1992 through 2000-2001 school years; however, the reported Wa rren rates are only available for 9 classes graduating from 1992 to 2000. Before getting into the details of anal ysis, let us first examine several issues encountered at state level. First, state results are only available for 10 years, whereas national results are available for 29 years. Therefore, the state level observations are based on a much smaller sample size and not as robust as national level results. Second, state results are based on a much smaller student populations than the national results; therefore, state results are likely to be more volatile than national results. Also, enrollment size varies substantially from state to state; theref ore state graduation rate estimates are likely to be more stable in large states than smaller ones (Kane & Staiger, 2002). Figure 10 illustrates the effect of enrollment size on the change in graduation rate estimate for all 50 states. In this scatterplot, the horizontal axis represents the state high school enrollments (i.e. total enrollment for grades 9 to 12) in fall 2000 and the vertical axis represents the change in the simple grade 9 graduation rates between 1992 and 2000. 10 CPI state rates are computed based on state level enrollment and graduation data rather than aggregate from district level rates as the original Swanson CPI method.
Education Policy Analysis Archives Vol. 12 No. 55 32 Figure 10. State High School Enrollment Size vs. Change in Graduation RateSimple Grade 9 RateCA TX NY HI TN UT LA AZ FL 050000010000001500000total high school enrollment 2000 -0.12 -0.08 -0.04 0.00 0.04c h a n g e i n g 9 r a t e b e t w e e n 2 0 0 0 & 1 9 9 2 Figure 10 warrants two observations. First, more states experienced decreases in the simple grade 9 graduation rate than increases. This is consistent with the overall decrease observed in the national estimates. Second, states with smaller enrollments tend to have more variability in the change of simple grade 9 rates, while larger states are less likely to experience change in graduation rates. For exam ple, California and Texas, the two states with largest public high school enrollments, had almost no change in the simple grade 9 rate between 2000 and 1992; while states showing la rger changes during the same period tended to have small enrollments (e.g. Arizona, Hawa ii, Louisiana, Tennessee, and Utah). Similar patterns are observed when other graduation rate estimation methods are used instead of the simple grade 9 rate, although the magnitudes of changes vary from method to method. In sum, the size of state enrollment plays a role in the observed change in the graduation rate, and hence interpretation should be made with caution and with the local context in mind. We can now move on with the state analys is. The state analyses are intended to answer questions parallel to those posed at na tional level. Two additional questions are of interest at the state level: How do the state patterns compare to the national pattern? Are the national patterns also observed at state level? How are they similar or different? Is the relationship between alternative esti mates consistent across different states? Although addressing research questions similar to those examined in the national level analysis, we will proceed with the state anal yses differently for two reasons. First, it is not feasible to examine closely the correlations and effect sizes in each state since there are 50 states. Also, with results only available for 10 cohorts, correlation coefficients based on a sample size of 10 in each state are not robust enough to warrant close attention. State longitudinal trends
Alternative High School Graduation Rates 33 One feasible way to study the trend within and across the 50 states is to use graphic tools. Alternative state graduation rates are computed and graphed for each of the 50 states. Two raters examined the state graphs to classify the states using a simple rubric. The rater first determines if the six lines in each graph indicate a consistent trend for state high school graduation rate. If the answer is yes, the rater describes the state trend by choosing one category from six options: (1) rising; (2) falling; (3) first rising then falling; (4) first falling then rising; (5) fluctuating; or (6) stable. If the rater decides that the six lines in a graph indicate an inconsistent trend, the rater needs to describe the pattern. The following two examples will illustrate this process. Figure 11 shows the alternative graduation rate estimates for Massa chusetts from 1992 to 2001. The lines in Figure 11 show a consistent falling trend in high school graduation rates in the state of Massachusetts. Figure 11. Alternative Graduation Rate Estimates (Massachusetts)50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% 1992199319941995199619971998199920002001 CPI G10Rate G8Rate G9Rate Greene Warren Figure 12 presents the alternative estimates for Idaho. In contrast to the results for Massachusetts, the Idaho pattern is considered inconsistent with a fluctuating CPI line, a rising Greene line and the other four falling lines.
Education Policy Analysis Archives Vol. 12 No. 55 34 Figure 12. Alternative Gradua tion Rate Estimates (Idaho)50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% 1992199319941995199619971998199920002001 CPI G10Rate G8Rate G9Rate Greene Warren In essence, each rater examined 50 individual state trends and put each state in an appropriate category in Table 8. Based on the a bove figures, Massachusetts is classified as showing a consistent falling trend. In contrast, although Idaho shows an overall falling trend, the CPI estimates and the Greene estimates are inconsistent with the overall pattern as noted in the parentheses. Table 8 State Classification Table with Sample States Lines show consistent trend Lines show inconsistent trend1 Rising Falling MA ID (CPI & Greene) Rise/Fall Fall/Rise Stable Fluctuate 1. The method noted in the parentheses is showing trend inconsistent with trends yielded from other methods. The two raters examined the state trends independently and recorded the classifications. After the first rating, the two raters had complete agreement on 22 states and partial agreement on 15 states. Partial agreement means the two raters agreed on the overall state trend (e.g. falling or risi ng), but disagreed on the consistency of alternative rates. The two raters then reviewed the classification rubr ics together, and rated for a second round, independently, the 28 states they did not comple tely agree upon. After the second round of rating, the two raters reached complete agreemen t on five more states, and partial agreement on 15 states. In the third round, the two rater s discussed their ratings of the 23 states and reached a consensus. Based on this state classi fication, a descriptive of the trends in 50 states is arrived as is shown in Table 9.
Alternative High School Graduation Rates 35 Table 9 State Trend Classification Lines show consistent trend (32) Lines show inconsistent trend1 (18) Rising (2) LA, TX -Falling (34) AK, AL, CO, CT, FL, GA, HI, IL, IN, MA, MN, NH, NV, NY, PA, SC, SD, WA, WI DE (Greene), IA (Greene), ID (CPI & Greene), KS (CPI & Greene), MD (CPI), MS (Greene & CPI), MT (CPI & Greene), NC (CPI), ND (Greene), NE (Greene), OK (CPI & Greene), OR (CPI), RI (CPI & G10R), TN (CPI), WY (Greene) Rise/Fall (2) UT KY (CPI) Fall/Rise (6) AR, AZ, CA, ME, NJ, NM -Stable (3) VA MO (Greene), WV (Greene) Fluctuate (3) MI, OH, VT -1. The method noted in the parentheses is showing trend inconsistent with trends yielded from other methods. Consistent with the national pattern, the declining trend is also observed in over two thirds of the states. Of the 50 states, 34 show an overall pattern of declining graduation rates from 1992 to 2001. Only two states (Texas and Louisiana) show slight increases in graduation rates over the same period. The re maining states present a rise/fall pattern (two states), a fall/rise pattern (six states) pattern, a fluctuating pattern (three states), or a stable (three states) pattern in high school graduation rates. At national level, the CPI method and the Greene method yield results less reliable than the other four methods. This lack of reliability is also observed in about one third of the states. In 10 states, the Greene method yiel ded trends inconsistent with others, and in 11 states the CPI method yielded inconsistent trends. Meanwhile, in 32 states, alternative methods yielded similar graduation rate trends fr om 1992 to 2001, although the actual rates may differ. It is acknowledged that the classification of state trends based on graphic tools has its limitations. In order for this to work, the classification rubrics have to be simple enough. However, such simplification is at the cost of the discrimination of more complex patterns. For example, states with a declining trend differ from one another in the magnitude. Another observation from national rates and state average rates is the increasing difference between the simple grade 8 rate and the simple grade 9 rate, indicating that more students are enrolled in grade 9 than in gr ade 8 the previous year. Such difference s are observed in almost all states; however, the magnitude of such differences varies from state to state. This consistent pattern across states sugge sts that more students are enrolled in grade 9 than in grade 8 the previous year, probably due to grade retention. Correlations between alternat ive state rates and rankings Since state graduation rate estimates are only available for 10 cohorts, it is cumbersome to compute the correlation coeffici ents between alternative rates over time based on a sample size of only 10 in each of the 50 states. A more meaningful approach is to correlate alternative state rates for a given c ohort, with a sample size of 50 states.
Education Policy Analysis Archives Vol. 12 No. 55 36 The alternative methods intend to estima te the high school graduation rate for the same year in the same jurisdiction; therefore the results should be highly correlated if not exactly the same. Table 10 lists the correlati on coefficients between state graduation rate estimates for the class of 2000 derived from the six alternative methods. Given the very nature of the graduation rate, the data we curre ntly have are of a very restricted range: theoretically, graduation rates are within the ra nge of 0 to 1, and the observed range of the estimates for the class of 2000 is between .500 and .938. Given such limited range, the correlation coefficients presented in Table 10 are high, ranging between .871 and .947, suggesting that these six alternative methods yi eld similar state graduation rate estimates for the class of 2000. Table 10 Correlations (PearsonÂ’s r) between Alternat ive State Rates for Class of 2000 (N=50) Grade 8 rate Grade 9 rate Grade 10 rate Greene rate Warren rate Grade 9 rate .922** Grade 10 rate .899** .917** Greene rate .871** .899** .910** Warren rate .881** .931** .924** .923** CPI state rate .897** .934** .936** .913** .947** ** Correlation is significant at the 0.01 level (2-tailed). Another means by which to study the rela tionship of alternative estimates is to examine the state rankings derived from alterna tive graduation rate estimates. Since rankings are an ordinal measure, SpearmanÂ’s rho correlation coefficient is used instead of PearsonÂ’s r coefficient. By correlating the rankings, we examined the state graduation rate relative to each other in a norm-referenced approach, which provides a new perspective. The coefficients in Table 11 range between .873 and .948, which are comparable to those in Table 10. This suggests that these six alternative methods yield similar state rankings in terms of graduation rate for the class of 2000. Table 11 Correlations (SpearmanÂ’s rho) between Al ternative State Rankings for Class 2000 (N=50) Grade 8 rank Grade 9 rank Grade 10 rankGreene rank Warren rank Grade 9 rank .902** Grade 10 rank .909** .926** Greene rank .889** .906** .904** Warren rank .873** .945** .935** .915** CPI state rank .905** .948** .935** .918** .941** ** Correlation is significant at the 0.01 level (2-tailed). Magnitude of differences
Alternative High School Graduation Rates 37 At the national level, we standardized th e differences between alternative rates in order to compare across methods; however, it would be a cumbersome task to carry out at the state level given the number of states and ye ars included in the state level analysis. To get a sense of how big a difference the method makes, we can examine the descriptive statistics (e.g. range, median, and mean) of state rate estimates for a given year. Another approach is to examine the range of alternative esti mates for individual states in a single year. Distribution of alternative estimates for a given class By examining the descriptive statistics of state rate estimates, we can get a sense of how big a difference the method can make for a given class. Table 12 lists the descriptive statistics of alternative state rates for the cla ss of 2000. For example, the simple grade 9 rates for the class of 2000 range between 51.0% (Sou th Carolina) and 85.5% (New Jersey), with a mean of 69.7% and a median of 71.0%. Table 12 Descriptive Statistics for Alt ernative State Rates (Class 2000) Grade 9 Rate CPI State Rate Warren Rate Greene Rate Grade 8 Rate Grade 10 Rate Minimum 51.0% (SC)47.0% (TN)50.4% (SC)54.8% (FL) 60.9% (MS) 63.5% (FL) Mean 69.7% 70.0% 70.1% 71.9% 76.3% 76.7% Median 71.0% 71.6% 70.9% 73.0% 76.8% 77.9% Maximum 85.5% (NJ)98.0% (NJ)86.2% (NJ)87.5% (NJ)90.8% (NJ) 93.8% (NJ) # of states w/rates < 66.6% 17 15 14 15 7 6 # of states w/rates > 80.0% 6 7 9 10 16 17 The two rows in the bottom of Table 12 are very revealing. They indicate the number of states with graduation rate estimates in two arbitrary ranges Â– rates below 66.6% (two thirds) and rates over 80%. For the class of 2000, the simple grade 9 method yielded the most conservative estimates with 17 estimates (or one third of the states) falling below 66.6% and six estimates above 80%. In contrast, the simple grade 10 method yielded the most liberal state graduation rate estimates with only six estimates below 66.6% and 17 estimates above 80%. Based on earlier concep tual review and empirical results we have discussed so far, it is reasonable to specula te the Â“truthÂ” to be somewhere in between, although hard to pinpoint exactly. Figure 13 is a visual presentation of the information in Table 12. Each line segment in the figure represents the range of the results yielded from one method, with the dot in the middle representing the median estimates. The most striking observation from Figure 13 is the extraordinary length of the CPI line relative to others, indicating that the CPI state rates have more variability across states. For the class of 2000, New Jersey11 has the maximum CPI state rate of 98.0%, which appears unrealisti cally high compared to results yielded from other methods. The remaining five methods have ranges comparable to each other. 11 New Jersey has an alternative review proce ss for students who failed to pass the exam required for high school graduation. However, it is not clear as to how much this process affects the statewide high school graduation rate. For more information see http://www.nj.gov/njded/assessment/apa/.
Education Policy Analysis Archives Vol. 12 No. 55 38 Figure 13. Distribution of Alternative State Rates (Class 2000) 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0% Warren RateGrade 9 RateCPI State RateGreene RateGrade 8 RateGrade 10 Rate minimum median maximum Alternative estimates for indivi dual states in a given year Another approach is to examine the range of alternative estimates for individual states in a single year. Figure 14 presents the distribution of alternative state rates for the class of 2000. Each line segment in the figure represents the range of graduation rate estimates for the class of 2000 in one state, wi th the dot in the middle representing the mean of alternative estimates. The states are sor ted by the mean estimates. We can make two observations based on Figure 14. First, states vary substantially in terms of the mean graduation rate estimate for the class of 2000, with South Carolina having the lowest mean graduation rate estimate of 56.3%, and New Je rsey having the highest mean estimate of 90.3%. Second, the range of alternative estima tes also differs considerably from state to state. In North Dakota, the difference between the highest estimate (the simple grade 8 rate, 87.3%) and the lowest estimate (the CPI state rate, 83.0%) is only 4.33%. In contrast, in Nevada, the simple grade 8 rate (72.9%) is 19.3 % higher than the Warren rate (53.6%). Such a large discrepancy is probably attributable to the high net migration rate in the state of Nevada (Warren, 2003).
Alternative High School Graduation Rates 39 Figure 14. Distribution of Alternative State Rates (Class 2000)40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0%SC GA FL TN MS AZ LA AK NV NY NC AL NM DE TX OR HI KY CA IN WA MI CO RI NH OK AR OH MO MD IL WY WV ME KS VA MA PA ID VT SD MT CT WI MN UT IA NE ND NJState minimum mean maximum Next, Figure 15 shows the ranges of graduation rate estimates across methods in each individual state for the class of 2000. Of the 50 states, 27 have a range of less than 10%, 19 have a range between 10~15%, and the remainin g four states have a range of over 15%. Figure 15. Range of State Graduati on Rate Estimates across Methods (Class 2000)0.00% 5.00% 10.00% 15.00% 20.00%ND AR ID UT OR IA KS NE MT OK MA RI KY MN ME WY SD MO PA CO IN AZ VT NM WA NH OH CT DE WI MD AK CA VA WV NC MI NJ AL MS IL HI GA LA NY SC FL TX TN NV Figure 16 plots the range of state estimates against the mean state estimates for the class of 2000. An inverse relationship is appa rent in this graph, with higher mean state estimates associated with smaller range of al ternative state estimates. This suggests that alternative graduation rate methods make a larg er difference on the estimates for states with relatively poor graduation rates; whereas, the method effects are smaller for states with higher graduation rates. Only one state (New Je rsey) departs from the pattern, with a mean estimate of 90.3% and a range of .13. The CPI state rate (98.0%) is the highest estimate for New Jersey for the class of 2000. It is 7.7% hi gher than the mean estimate and 12.5% higher than the lowest estimate (85.5%, using the simple grade 9 method).
Education Policy Analysis Archives Vol. 12 No. 55 40 The correlation between the mean state estima tes and the range of state estimates is .636 (p<.01) for all 50 states, and -.709 (p <.01) when New Jersey is excluded. Such correlation coefficients are fairly high give n the limited range of mean state estimates. Figure 16. Range of State Estimate vs. Mean State EstimateClass 2000NJ NV TN ND TX FL 0.100.200.300.400.500.600.700.800.90mean state estimate for class 2000 0.00 0.05 0.10 0.15 0.20ra n g e o f s t a t e e s t i m a t e f o r c l a s s 2 0 0 0 The same negative relationships between th e range of state estimates and the mean state estimates is also observed for the class of 1992 as is shown in Figure 17. No state stands out from the general pattern, although Nevada is some distance away from the other states. For the class of 1992, the mean state graduation rate estimate for Nevada is 67.3%; however, the range between the lowest estimate (the reported Warren rate of 54.5%) and the highest estimate (the simple grade 8 rate of 74.0%) is almost 20%. A comparison of Figures 16 and 17 suggests that the difference between al ternative state estimates have increased as more states are located in the range of .15 and .20 in 2000 than in 1992. For the class of 1992, the correlation between the mean state estimates and the range of state estimates is -.590 (p < .01). This corre lation coefficient is smaller than the coefficient for the class of 2000, which is probably attrib utable to the fact that for the class of 1992, there is less variability among states in terms of the range of state graduation rate estimates.
Alternative High School Graduation Rates 41 Figure 17. Range of State Estimate vs. Mean State EstimateClass 1992NV 0.100200.300.400.500.600.700.800.90mean state estimate for class 1992 0.00 0.05 0.10 0.15 0.20ra n g e o f s t a t e e s t i m a t e f o r c l a s s 1 9 9 2 In summary, the alternative state graduati on rate estimates present similar long-term trends and are highly or moderately correlated with each other. Meanwhile, alternative estimation methods do have substantial influences on the graduation rate estimates for individual states. For states with relatively low graduation rates, the difference between the alternative estimates can be substantial. State Level Results by Major Ethnic Groups In an ethnically diverse society like the United States, racial gaps in various indicators of academic achievement have long been documented (The Education Trust, 2004). In regard to high school graduation and dropout rates, the Current Population Survey (CPS) results have long shown that black and Hispanic students are less likely to graduate from high school and currently more likely to drop out of high school than their white peers (Dorn, 1996; Hauser, 1997; Hauser, Simmons & Pager, 2000; Sum & Harrington, 2003). For example, the status dropout rates for the 16-to 24-year-old age group are consistently higher for the black and Hispanic student population than for the white in the last few decades (see Figure 18). Although the black and white gap has decreased since the 1960s, based on this measure, the difference in the status dropout rate between Hispanics and whites remains large.
Education Policy Analysis Archives Vol. 12 No. 55 42 Figure 18. Status Dropout Rate (16-to 24-years old)0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0 40.01960 1968 1970 1972 1974 1976 1978 1980 1982 1984 1986 1988 1990 1993 1995 1997 1999 2001YearPercent All Races White Black Hispanic Source: Snyder, T. D., & Hoffman, C. M. (Eds.) (2003). Digest of Education Statistics 2002, Table 108. Measures based on the CCD also revealed substantial racial gaps in high school graduation rate. Using the CPI method, Swanson (2004) found Â“tremendous racial gapsÂ” for the class of 2001: only 50% of students from historically disadvantaged minority groups (American Indian, Hispanic and black) finish high school with a diploma, while the high school graduation rates for whites is 75% and for Asians is 77% nationwide. Even larger racial gaps are observed within individual sta tes. For example, in Massachusetts, while 73.7% of white students graduated in 2001, only 36.1% of Hispanic students leave high school with a regular diploma. SwansonÂ’s graduation rate estimates are cl ose to what Greene (2003) reported at the national level. However, the two authorsÂ’ results differ substantially for some minority groups within individual states (see Table 13 ). For example, Swanson reported a graduation rate of 65% for blacks in Massachusetts, while GreeneÂ’s estimate is only 49.5%. Table 13 Graduation Rate for Class 2001 Source All Races American Indian AsianHispanicBlack White National Greene (2003) 68.0% 51.1% 76.8%53.2% 50.2% 74.9% Swanson (2004) 70.0% 54.0% 79.0%52.0% 51.0% 72.0% Massachusetts Greene (2003) 73.0% n/a1 76.0%49.0% 65.0% 78.0% Swanson (2004) 71.0% 25.4% 60.5%36.1% 49.4% 73.7% 1. Insufficient data to calculate graduation rate. In this section, we computed alternative graduation rates for major ethnic groups in selected states and compared the results to detect the effect of alternative methods on the magnitude of racial differences uncovered. As illustrated by the example in Table 13, the racial gap in graduation rates may differ substantially depending on the methods used. Not all 50 states are included in the race level analysis simply because some states (e.g. Idaho, New Hampshire, South Carolina, and Vermont) do not report enrollment and graduation information by student ethnicity, hence the information is not available in the CCD database. Other states are excluded for the concern of small sub-population sizes.
Alternative High School Graduation Rates 43 In the earlier discussion of state level analysis, we illustrated the potential influence of enrollment size on the change in the graduation rate. Graduation rate estimates for states with small enrollments are more likely to change than states with larger enrollments. This is a more salient issue when we break down st ate enrollment by student ethnicity. Due to historical and geographical reasons, student demo graphics differ tremendously from state to state in terms of ethnicity. While some states are overwhelmingly white (e.g. based on CCD data, Maine has only 3.8% non-white public school enrollment in 2001-2002), other states serve much more diverse student populations (e. g. over 60% of the public school enrollment in California, New Mexico, and Hawaii ar e non-white in 2001-2002). Therefore, subpopulation sizes differ drastically among states. For example, in the 2001-2002 school year, Montana had a total enrollment of only 962 black students (including students from prekindergarten through 12th grade and ungraded students), which means an average of fewer than 80 students in each grade in the whole state. In contrast, for the same school year, Florida enrolled 621,569 black students (pre-kindergarten through 12th grade), which is 646 times the black enrollment in Montana. Results based on such drastically different sample sizes are hardly comparable since any measur ement based on a small population size of less than 100 is likely to be unreliable and prone to fluctuation from year to year. In this section, we compute high school graduation rates for all three major racial groups in the states, for which the data are available. However, results based on an average cohort grade enrollment of fewer than 100 are excluded from the analysis. For the students graduating in 1997 and 2001, we computed graduation rates for the three major ethnic groups (black, Hispanic and white) using five methods: the simple grade 8 rate, the simple grade 9 rate, the simple grade 10 rate, the Greene rate, and the CPI race rate12. The Warren method is not included in this analysis since Warren did not compute graduation rates by race. Results from the five alternative methods are tabulated and graphed, and the summary descriptive statisti cs are examined to illustrate the difference across methods. Also, correlations are calculated across methods in order to examine the similarity between alternative results. The objective for this analysis is, again, to detect the effect of alternative methods on the magnit ude of racial differences uncovered. Table 14 lists the alternative graduation ra te estimates for the class of 2001 in three major ethnic groups within states. In Massach usetts, for example, the graduation rate for black students in 2001 ranges between 51.7% (C PI rate) and 80.9% (simple grade 8 rate), the graduation rate for Hispanic students ranges between a low of 34.2% (CPI rate) and a high of 65.9% (simple grade 10 rate), and the graduation rate for white students ranges between 76.3% (CPI rate) and 84.5% (simple grade 10 ra te). Alternative methods yielded relatively reliable estimates for the white student population, yet drastically different results for the black and Hispanic student population. Race results are not available for all states due to several reasons. First, there are insufficient data to compute the race rates for certain states since these states did not report enrollment and graduation data disaggregated by race. Second, race rates are suppressed, in some cases, when the cohort size is extremel y small, i.e. with fewer than 100 graduates statewide in a given group. For example, in the state of Montana, only 33 black students graduated in the year of 2001, therefore gr aduation rate estimates for the black population are not reported for Montana. Third, in a couple of cases, the CPI rate is Â“censoredÂ” due to abnormally high grade promotion rates. That is, when the promotion rate in any grade exceeds 110%, the CPI rate is not reported (s ee earlier discussions of the CPI method). 12 Again, this is based on race enrollment and graduation rates at the state level rather than aggregate rates from the district level as Swanson did in his report.
Education Policy Analysis Archives Vol. 12 No. 55 44 Table 14 Class of 2001 Graduation Rates for Major Ethnic Groups (Percent) Black Hispanic White STATE G8R G9R G10R GRN CPIG8RG9RG10RGRNCPIG8RG9R G10R GRNCPI A K 60.7 55.7 67.8 64.2 63.7 68.468.469.8 59.755.9 74.467.7 72.3 68.163.9 A L 59.4 50.1 64.1 59.3 53.5 65.255.270.4 44.363.0 68.062.8 73.5 70.365.3 AR 66.1 66.2 68.9 69.4 69. 288.380.076.2 52.3C 74.574.8 79.1 78.076.6 CA 67.7 56.7 62.9 58.1 57. 868.557.463.2 55.757.8 82.577.5 79.9 76.777.2 CO 62.7 53.0 63.2 56.0 53. 056.549.462.0 47.448.8 80.275.6 79.9 74.175.9 CT 68.0 54.2 66.5 55.6 56. 159.745.665.0 47.450.9 84.781.4 85.1 76.880.4 DE 63.5 50.7 64.9 57.6 52. 857.542.157.0 44.349.5 77.870.0 76.9 74.369.8 FL 56.7 44.2 55.3 47.2 40. 966.852.863.5 47.852.3 67.859.1 68.7 61.358.7 GA 51.6 40.3 55.3 46.4 43. 356.743.859.9 32.347.3 65.958.8 71.0 62.761.7 HI 37.2 44.1 57.1 50.5 60. 763.351.765.3 64.559.9 59.957.8 70.3 63.964.7 IA 64.4 56.1 63.3 57.8 61. 177.164.270.2 54.062.0 87.483.5 85.7 86.783.2 IL 54.4 45.1 57.8 52.7 44. 563.454.167.8 53.461.7 87.082.0 85.7 83.583.6 IN 54.0 43.8 58.0 53.3 45. 273.759.872.8 59.264.1 75.571.3 78.7 77.673.3 KS 62.2 I 64.1 I 61.260.6I 59.4 I 58.781.6I 82.0 I 82.6 KY I 50.0 61.5 I 45.4I 82.3 84.4 I 77.5I 67.0 76.9 I 66.3 LA 60.0 50.3 65.0 62.1 60. 280.764.976.4 74.486.5 71.765.2 77.1 75.768.1 MA 80.9 66.3 73.5 65.5 51. 763.850.465.9 49.034.2 82.679.4 84.2 77.676.3 MD 78.6 64.7 76.8 66.3 64. 092.673.777.2 61.976.5 83.679.5 85.4 78.279.5 ME S S S S S S S S S S 73.775.9 82.3 74.677.4 MI 61.7 46.7 64.0 55.6 48. 964.150.268.8 52.956.4 82.375.8 81.6 77.778.0 MN 60.8 51.8 55.9 42.7 56. 368.560.764.2 46.959.6 88.385.5 85.5 85.684.2 MO 66.9 54.0 65.0 58.0 53. 992.879.483.4 65.676.6 79.875.9 80.7 77.476.1 MS 57.6 52.8 64.9 61.1 53.3S S S S S 64.460.9 71.6 67.764.1 MT S S S S S 100.096.698.3 84. 391.183.580.2 84.0 84.781.6 NC 60.6 I 64.9 I 53.069.9I 71.5 I 60.072.5I 76.5 I 69.2 ND S S S S S S S S S S 89.188.4 90.3 93.286.0 NE 64.1 50.8 69.4 54.7 50. 674.963.876.7 56.960.5 88.183.7 87.6 88.786.3 NJ I I I I I I I I C I I I I 99.5 NM 88.8 63.1 70.9 72.7 62. 072.154.864.9 62.157.0 81.669.5 75.4 79.469.2 NV 61.8 56.4 59.9 50.0 40. 762.957.757.2 40.736.3 74.072.2 74.1 68.862.4 NY 56.9 36.8 47.1 46.7 39. 752.533.143.5 42.436.4 79.775.5 81.2 77.177.2 OH 56.1 42.5 61.8 52.3 46. 766.954.770.3 60.962.1 83.176.9 83.6 82.376.7 OK 68.7 60.0 70.7 66.1 60. 680.370.877.9 59.867.7 76.174.0 79.7 80.073.7 OR 59.9 51.6 55.0 50.1 60. 059.551.453.9 42.660.6 70.167.4 71.0 69.473.0 PA 69.7 49.9 66.9 57.6 54. 464.846.261.3 48.853.2 86.281.9 85.5 82.984.5 RI 76.8 64.5 78.0 62.5 85. 273.857.471.3 55.769.1 78.172.6 80.4 74.673.6 SD S S S S S S S S S S 85.783.1 86.3 88.287.0 TX 70.3 54.9 75.3 61.8 58. 066.651.473.9 56.957.1 79.372.1 82.8 76.774.9 UT 74.2 71.9 70.5 56.5 67. 773.970.668.7 53.869.9 85.184.5 84.3 90.484.5 V A 72.7 64.7 76.5 64.2 62.7 94.275.979.5 59.071.7 83.077.4 85.1 77.475.5 W A 61.1 52.8 58.9 52.5 53.9 61.954.758.7 48.253.4 73.267.2 70.7 68.667.8 W I 50.4 37.8 53.9 44.2 43.0 70.655.666.0 54.764.0 94.484.4 86.8 87.484.8 WV 66.5 65.6 67.0 70.0 63.3S S S S S 76.573.5 79.0 84.771.1 W Y S S S S S 55.058.564.3 64. 666.875.675.3 75.3 79.274.9 Note: Insufficient data to compute the rates for AZ, ID, NH, SC, TN, VT. S : Rate suppressed because of small c ohort size (less than 100 graduates) C : Value censored due to large progression rate.
Alternative High School Graduation Rates 45 Table 15 summarizes the race results for the class of 2001 by listing the maximum, median, and minimum estimates across states based on alternative methods. For example, the simple grade 8 method yielded valid graduation rate estimates for black students in 37 states, ranging from 37.2% to 88.8 %, with a median of 62.2%. Table 15 Descriptive Statistics for Alternativ e State Rates by Race (Class 2001) Group Method G8R G9R G10R GRN CPI Black Count 37 36 38 35 38 Maximum 88.80% 71.90% 78.00% 72.70% 85.20% Median 62.20% 52.80% 64.50% 57.60% 54.20% Minimum 37.20% 36.80% 47.10% 42.70% 39.70% Hispanic Count 37 36 38 35 37 Maximum 100.00% 96.60% 98.30% 84.30% 91.10% Median 66.90% 56.50% 68.30% 54.00% 60.00% Minimum 52.50% 33.10% 43.50% 32.30% 34.20% White Count 42 41 43 40 44 Maximum 94.40% 88.40% 90.30% 93.20% 99.50% Median 79.70% 75.50% 80.40% 77.50% 76.00% Minimum 59.90% 57.80% 68.70% 61.30% 58.70% The information in Table 15 is graphed in Figures 19 to 21, which allows a comprehensive view of the distribution of the alternative graduation rates by race. Several observations are obvious from the three figures. First, regardless of the estimation method, the median estimates for the white student population (ranging from 75.5% to 80.4%) are much higher than the median estimates for both the black student population (ranging from 54.2% to 64.5%) and the Hispanic student population (ranging from 54.0% to 68.3%). Second, differences between alternative estima tes are smaller for the white population than for the black and Hispanic student population. Third, the ranges of alternative rates are smaller for the white student population than for the black and Hispanic student population, indicating a larger disparity among states in terms of high school graduation rate for minority students.
Education Policy Analysis Archives Vol. 12 No. 55 46 Figure 19. Distribution of Alternative State Graduation Rate (black, Class 2001) 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0% G9RCPIGRNG8RG10R max med min Figure 20. Distribution of Alternative State Graduation Rates (Hispanic, Class 2001) 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0% GRNG9RCPIG8RG10R max med min Figure 21. Distrubution of Alternative State Graduation Rates (white, Class 2001) 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0% G9RCPIGRNG8RG10R max med min
Alternative High School Graduation Rates 47 Figure 22 provides yet another angle to examine the alternative graduation rate estimates within state by race. Earlier observa tions based on Figures 19 to 21 are also salient in Figure 22: (1) White rates are consistently higher than the black or Hispanic rates and (2) the difference between alternative state median rates is smaller for the white subpopulation than for the black or Hispanic subpopulation. In addition, Figure 22 shows that, in terms of the state median estimates by race, the simple grade 9 method reveals larger gaps between the white and the black/Hispanic student popula tion than the simple grade 8 method or the simple grade 10 method. This suggests that not only are race estimates affected by the method used, but also are the magnitudes of the racial gaps in high school graduation rates. The relatively larger racial gap revealed by the simple grade 9 rate is a reflection of larger grade 9 bulges for the black and Hispanic subpopulations than for the white. That is, black and Hispanic students are more likely to be retained in grade 9 than their white peers. Figure 22. State Median Estimates by Race (Class 2001) 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% G9RCPIGRNG8RG10R BLK HIS WHT Next, Table 16 summarizes the race results for individual states for the class of 2001 by listing the maximum, mean, and minimum estimates. The states are sorted by the mean estimates from low to high. For example, New York state has the lowest mean estimates for black and Hispanic students (45.4% and 41.6 %), while the mean estimate for the white population is much higher (77.2%).
Education Policy Analysis Archives Vol. 12 No. 55 48 Table 16 Descriptive Statistics for Individu al State Rates by Race (Class 2001) Black Hispanic White Min Mean Max Min MeanMax Min Mean Max NY 36.8% 45.4% 56.9% NY 33.1% 41.6%52.5% FL 58. 7% 61.3% 68.7% WI 37.8% 45.9% 53.9% GA 32.3% 48.0%59.9% GA 58.8% 62.7% 71.0% GA 40.3% 47.4% 55.3% DE 42.1% 50.1%57.5% HI 57.8% 63.9% 70.3% FL 40.9% 48.9% 56.7% NV 36.3% 51.0%62.9% MS 60.9% 64.4% 71.6% HI 37.2% 49.9% 60.7% MA 34.2% 52.7%65.9% KY 66.3% 67.0% 76.9% IN 43.8% 50.9% 58.0% CO 47.4% 52.8%62.0% AL 62. 8% 68.0% 73.5% IL 44.5% 50.9% 57.8% OR 42.6% 53.6%60.6% AK 63. 9% 68.1% 74.4% OH 42.5% 51.9% 61.8% CT 45.6% 53.7%65.0% WA 67.2% 68.6% 73.2% KY 45.4% 52.3% 61.5% PA 46.2% 54.9%64.8% OR 67.4% 70.1% 73.0% MN 42.7% 53.5% 60.8% WA 48.2% 55.4%61.9% LA 65.2% 71.7% 77.1% NV 40.7% 53.8% 61.8% FL 47.8% 56.6%66.8% NV 62.4% 72.2% 74.1% OR 50.1% 55.3% 60.0% MI 50.2% 58.5%68.8% NC 69.2% 72.5% 76.5% MI 46.7% 55.4% 64.0% KS 58.7% 59.6%60.6% DE 69. 8% 74.3% 77.8% WA 52.5% 55.8% 61.1% AL 44.3% 59.6%70.4% RI 72.6% 74.6% 80.4% AL 50.1% 57.3% 64.1% MN 46.9% 60.0%68.5% WY 74. 9% 75.3% 79.2% CO 53.0% 57.6% 63.2% IL 53.4% 60.1%67.8% NM 69.2% 75.4% 81.6% NE 50.6% 57.9% 69.4% CA 55.7% 60.5%68.5% IN 71. 3% 75.5% 78.7% DE 50.7% 57.9% 64.9% HI 51.7% 60.9%65.3% ME 73. 7% 75.9% 82.3% MS 52.8% 57.9% 64.9% TX 51.4% 61.2%73.9% CO 74.1% 75.9% 80.2% NC 53.0% 59.5% 64.9% WY 55.0% 61.8%66.8% OK 73. 7% 76.1% 80.0% LA 50.3% 59.5% 65.0% NM 54.8% 62.2%72.1% WV 71.1% 76.5% 84.7% MO 53.9% 59.6% 66.9% WI 54.7% 62.2%70.6% AR 74.5% 76.6% 79.1% PA 49.9% 59.7% 69.7% OH 54.7% 63.0%70.3% TX 72. 1% 76.7% 82.8% CT 54.2% 60.1% 68.0% AK 55.9% 64.4%69.8% NY 75. 5% 77.2% 81.2% IA 56.1% 60.6% 64.4% RI 55.7% 65.5%73.8% MO 75.9% 77.4% 80.7% CA 56.7% 60.6% 67.7% IA 54.0% 65.5%77.1% VA 75.5% 77.4% 85.1% AK 55.7% 62.4% 67.8% IN 59.2% 65.9%73.7% CA 76.7% 77.5% 82.5% KS 61.2% 62.5% 64.1% NE 56.9% 66.5%76.7% MI 75.8% 78.0% 82.3% TX 54.9% 64.1% 75.3% NC 60.0% 67.1%71.5% MA 76.3% 79.4% 84.2% OK 60.0% 65.2% 70.7% UT 53.8%67.4%73.9% MD 78.2% 79.5% 85.4% WV 63.3% 66.5% 70.0% OK 59.8% 71.3%80.3% CT 76. 8% 81.4% 85.1% MA 51.7% 67.6% 80.9% AR 52.3% 74.2%88.3% KS 81.6% 82.0% 82.6% AR 66.1% 67.9% 69.4% VA 59.0%76. 0%94.2% OH 76.7% 82.3% 83.6% UT 56.5% 68.1% 74.2% MD 61.9% 76.4%92.6% MT 80. 2% 83.5% 84.7% VA 62.7% 68.2% 76.5% LA 64.9%76. 6%86.5% IL 82.0% 83.6% 87.0% MD 64.0% 70.1% 78.6% MO 65.6% 79.5%92.8% UT 84. 3% 84.5% 90.4% NM 62.0% 71.5% 88.8% KY 77.5% 81.4%84.4% PA 81.9% 84.5% 86.2% RI 62.5% 73.4% 85.2% MT 84.3% 94.0%100.0%MN 84. 2% 85.5% 88.3% AZ n/a n/a n/a AZ n/a n/ a n/a IA 83.2% 85.7% 87.4% ID n/a n/a n/a ID n/a n/ a n/a SD 83.1% 86.3% 88.2% NH n/a n/a n/a ME n/a n/ a n/a WI 84.4% 86.8% 94.4% NJ n/a n/a n/a MS n/a n/ a n/a NE 83.7% 87.6% 88.7% SC n/a n/a n/a ND n/a n/ a n/a ND 86.0% 89.1% 93.2% TN n/a n/a n/a NH n/a n/ a n/a NJ 99.5% 99.5% 99.5% Figures 23 to 25 are graphic presentations of the information in Table 16. Each line segment in the figure represents the range of graduation rate estimates for class 2001 for a
Alternative High School Graduation Rates 49 designated group in a given state, with th e dot in the middle representing the mean estimates. The states are sorted by the mean estimates from low to high. Again, two observations are salient from these figures. Firs t, the average estimates for the white student population within states (ranging from 61.3% in Florida to 95.5% in New Jersey) are much higher than the mean estimates for both the black student population (ranging from 45.4% in New York to 73.4% in Rhode Island) and the Hispanic student population (ranging from 41.6% in New York to 94.0% in Montana). Second, the ranges of alternative race rates, indicated by the length of the bars, are re latively small for the white student population compared to the ranges for the black and Hi spanic student population. This suggests a larger method effect on the graduation rate estimates for the black and Hispanic student population than for the white within individual states. Figure 23. Distribution of Individual State Rates (black: Class 2001) 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0%NY WI GA FL HI IN IL OH KY MN NV OR MI WA AL CO NE DE MS NC LA MO PA CT IA CA AK KS TX OK WV MA AR UT VA MD NM RI AZ ID NH NJ SC TN VT ME MT ND SD WY min mean max Figure 24. Distribution of Individual State Rates (Hispanic: Class 2001) 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0%NY GA DE NV MA CO OR CT PA W FL MI KS AL MN IL CA HI TX W NM WI OH AK RI IA IN NE NC UT OK AR VA MD LA M KY MT AZ ID ME MS ND NH NJ SC SD TN VT W min mean max
Education Policy Analysis Archives Vol. 12 No. 55 50 Figure 25. Distribution of Individual State Rates (white: Class 2001) 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0%FL GA HI MS KY AL AK WA OR LA NV NC DE RI WY NM IN ME CO OK WV AR TX NY MO VA CA MI MA MD CT KS OH MT IL UT PA MN IA SD WI NE ND NJ AZ ID NH SC TN VT min mean max Correlations Up to this point, the race level results have focused on the differences between alternative estimates. Next, correlation coefficients are examined to determine the similarity of race results from alternative methods. Tabl e 17 lists the correlation coefficients between state graduation rate estimates (all races) for the class of 2001 derived from the five alternative methods. All correlation coefficien ts are high, except the correlation between CPI and G8R is moderate (.794). This suggests that th e five alternative methods yield similar state graduation rate estimates for the class of 2001. Table 17 Correlations between Alternative State Rates (All Races: Class 2001) ALL_G8R ALL_G9R ALL_G10R ALL_GRN ALL_G9R .925** (N=50) ALL_G10R .910** (N=50) .915** (N=50) ALL_GRN .830** (N=50) .883** (N=50).872** (N=50) ALL_CPI .794** (N=50) .864** (N=50) .850** (N=50) .841** (N=50) ** Correlation is significant at the 0.01 level (2-tailed). The next three tables list the correlation coefficients between graduation rate estimates for the three major ethnic groups within state. The correlation coefficients between alternative estimates for the white popula tion are high, ranging from .854 to .938. This suggests the five alternative methods yiel d fairly consistent graduation rate estimates for the white sub-population within states for the class of 2001.
Alternative High School Graduation Rates 51 Table 18 Correlations between Alternative State Rates (white: Class 2001) WHT_G8R WHT_G9R WHT_G10R WHT_GRN WHT_G9R .936** (N=40) WHT_G10R .896** (N=42).927** (N=41) WHT_GRN .854** (N=40).893** (N=40).867** (N=40) WHT_CPI .869** (N=42).938** (N=41).911** (N=43) .886** (N=40) ** Correlation is significant at the 0.01 level (2-tailed). The correlation coefficients between alternative estimates for the Hispanic population range from .664 to .904 (moderate to high). The correlation coefficients between alternative estimates for the black population are moderate (.60 to .80) except for the one low coefficient (.501 between the CPI rate and simple grade 8 rate) and one high coefficient (.814 between the Greene rate and the simple grad e 10 rate). This means that, for the class of 2001, the alternative graduation rate esti mates for the black and Hispanic sub-population are much less consistent than the altern ative estimates for white sub-population. Table 19 Correlations between Alternative St ate Rates (Hispanic: Class 2001) HIS_G8R HIS_G9R HIS_G10R HIS_GRN HIS_G9R .904** (N=35) HIS_G10R .864** (N=37).869** (N=36) HIS_GRN .664** (N=35).683** (N=35).767** (N=35) HIS_CPI .771** (N=36).796** (N=35).797** (N=37) .791** (N=34) ** Correlation is significant at the 0.01 level (2-tailed). Table 20 Correlations between Alternative State Rates (black: Class 2001) BLK_G8R BLK_G9R BLK_G10R BLK_GRN BLK_G9R .791** (N=35) BLK_G10R .766** (N=37) .797** (N=36) BLK_GRN .690** (N=35) .755** (N=35).814** (N=35) BLK_CPI .501** (N=37) .739** (N=36).680** (N=38) .636** (N=35) ** Correlation is significant at the 0.01 level (2-tailed). To verify the observations at the race level, the same procedures are applied to race results for the Class of 1997. The resulting patter n is consistent with the pattern for the Class of 2001. In summary, although alternative es timates are moderately to highly correlated at the state level both for the total student population and for the white subpopulation, the correlations are lower for alternative estimates for the black and the Hispanic subpopulation. Racial gaps are obvious in the graduation ra te between the white and black/Hispanic subpopulation regardless of the method used for estimation. However, th e graduation rate estimates by race are definitely a ffected by the method used, as well as the magnitudes of the racial gaps in high school graduation rates. For the class of 1997 and 2001, the simple grade 9 rate is the most conservative and reveals re latively larger racial gaps than the other methods.
Education Policy Analysis Archives Vol. 12 No. 55 52 Summary and Discussion The high school graduation rate is an important indicator of school effectiveness, and has appeared repeatedly in the federal legislations since the 1960Â’s. The No Child Left Behind Act of 2001 brought increased attention to the high school gradation rate by mandating that states report public school gr aduation rates for the statesÂ’ general student population, as well as for subpopulations disaggregated by major demographic characteristics. Currently, two major data sources are available for estimating the high school graduation rate. The Current Population Survey (CPS) targets the Â“civilian noninstitutional populationÂ” who are 15 years of age or older and collects information from a state-based probability sample of 50,000-60,000 house holds. Various measures of high school completion and dropout have been reported based on CPS data. However, there are a number of problems with these CPS based mea sures, and CPS based measures are not good options to report the public high school graduation rate as required by the No Child Left Behind Act The Common Core of Data (CCD) has been a program of the U.S. Department of EducationÂ’s National Center for Education Stat istics (NCES) since 1986. The CCD program conducts annual census surveys of all public elementary and secondary schools and school districts in the country. The CCD data are based on administrative records collected for each academic year by local education agencies. The accuracy of CCD data depends heavily on the quality of record keeping in local districts nationwide. NCES reports a four-year high school completion rate, which is the number of graduates divided by the sum of graduates and reported dropouts over four academic years. However, evidence has been gathering that graduation rates based on dropout statistics ar e often times inflated because of under reporting of dropouts. Alternative measures of the high school graduation rate based on the CCD grade enrollment and graduate counts have been devi sed and reported. These measures fall into two categories--simple graduation rates and adjusted graduation rates. The simple graduation rates are computed by dividing the number of graduates in a certain year by the cohort enrollment in an earlier grade. Conceptually straightforward and easy to compute, the simple graduation rates are often criticized for not accounting for various changes as a cohort progresses through high school, such as student migration and grade retention. A number of adjusted graduation rate measures have been devised, such as the Greene method, the CPI method and the Warren method, to addr ess these issues in various ways. In this study, these methods are applied to the CCD enrollment and graduation data at the national level, the state level and to major ethnic groups within states to compute alternative graduation rates. Statistical anal yses have been conducted to examine the relationship between results based on these alte rnative methods in order to examine these methods empirically with a focus on the association and differences between alternative estimates. Summary of findings The national graduation rate has been decr easing slightly from 1973 to 2001, regardless of the estimation method. Despite it s conceptual advantages, the Greene method is empirically unstable, especially during the late 70Â’s and early 90Â’s. In contrast, national
Alternative High School Graduation Rates 53 results from the other five methods are much more stable over the past three decades. The correlation coefficients between the alternative national estimates are all statistically significant and positive. The correlation stru cture matches the observation of the national graduation rate trend. While the correlation co efficients between the Greene estimates and the other estimates are only moderate to low, the correlations between other alternative estimates are high to moderate. Despite the positive correlation between alternative estimates, differences between these estimates are substantial at times. The simple grade 8 to graduation rates and the simple grade 10 to graduation rates are close to each other, yet consistently higher than the simple grade 9 to graduation rates, the CPI rates and the Warren rates. The difference between the simple grade 8 (or grade 10) rate and the simple grade 9 rate has been increasing since the 1970Â’s, and reaches almost one fifth of one standard deviation for the class of 2000, which is substantial. The state level analysis examined alternative graduation rate estimates for the 50 states for high school classes graduating during 1992 to 2001. Consistent with the national pattern, a decline in the graduation rate is obser ved in over two thirds of the states. Of the 50 states, 34 show an overall pattern of dec lining graduation rates fr om 1992 to 2001, while only two states (Texas and Louisiana) show slight increases in graduation rates over the same period. In 32 states, alternative methods yield similar graduation rate trends from 1992 to 2001, although the actual estimates may diffe r. Meanwhile, in 10 states, the Greene method yielded trends inconsistent with the trends from the other methods, and in 11 states the CPI method yielded inconsistent trends. There is a consistent pattern across states which shows that more students are enrolled in Grade 9 than in Grade 8 the previous year. Accordingly, the differences between simple grade 8 (and gr ade 10) to graduation rate and simple grade 9 rate are observed in almost all states; however, such differences vary in magnitude from state to state. Overall the six alternative methods yield consistent state level estimates for the class of 2000. For the class of 2000, the correlation coefficients (PearsonÂ’s r) between alternative state estimates are high, and so are the correlation coefficients (SpearmanÂ’s rho) between alternative state rankings. Despite the high co rrelations, the actual values of the alternative estimates differ substantially. For the class of 2000, the simple grade 9 method yielded the most conservative estimates with 17 estimates (or one third of the states) falling below 66.6% and 6 estimates above 80%. In contrast, th e simple grade 10 method yielded the most liberal state graduation rate estimates with only six estimates below 66.6% and 17 estimates above 80%. Alternative graduation rate methods make a la rger difference for states with relatively poor graduation rates; whereas, the method e ffects are smaller for states with higher graduation rates. For individual states, the range of alternative state estimates varies. For the class of 2000, 27 out of the 50 states have a range of less than 10%, 19 have a range between 10 and 15%, and the remaining four states have a range of over 15%. An inverse relationship is apparent between mean state estimates and th e range of alternative state estimates. Such an inverse relationship is confirmed by the state estimates for the class of 1992. Graduation rates were also computed for ma jor ethnic groups within state for the class of 1997 and the class of 2001. Racial gaps between the white and black/Hispanic subpopulation are obvious in the graduation rate regardless of the method used for estimation. The effect of method is inconsistent across ethnic groups, and the choice of method makes a larger difference in the graduation rate estimates for minority groups. Alternative estimates are moderately to highly correlated for the state total population and for the white subpopulation; however, the correlations are lower between alternative estimates for the black and the Hispanic su bpopulation. The graduation rate estimates by
Education Policy Analysis Archives Vol. 12 No. 55 54 race are definitely affected by the method used, as well as the magnitudes of the racial gaps in high school graduation rates. The simple grade 9 rate is the most conservative and reveals relatively larger racial gaps than other methods. These above findings of the current study suggest two major implications. Counting Graduates is No Easy Task It is obvious to the reader that counting the graduates is no easy task. The major obstacle is the lack of databases which track individual students throughout their entire school careers. Without such databases, it is not possible to conduct true cohort analyses by following the same students from school entry to graduation and to compute the true high school graduation rate. The alternative methods applied in the current study are quasicohort approaches, which are based on group counts on an annual basis. With such data, it is probable that the group we counted in year one is not the exact same group we count again in year two, despite the fact that the vast majority of the students included in both counts are the same. The National Center for Educational Accountability (NCEA) conducted a survey of statewide data-collection systems, by focusing on a list of essential elements (Dougherty, 2003; National Center for Educational Accountab ility, 2003). It found only 23 states use a statewide student identifier that assigns each student a unique statewide student number. The survey results, however, did not mention the history of state data-collection systems; hence it is unclear how far back these data syst ems are capable of tracking students in their school careers. For other states that plan to es tablish longitudinal student tracking systems, it will undoubtedly be years before they can fully benefit from their new statewide data collection systems. It is important to acknowledge that a we ll-designed data system does not guarantee accurate data collection. A recent Houston case on the undercount of dropouts exemplifies this perfectly. According to the NCEA survey, Texas has the best comprehensive data system to date, which comprises all the key elements identified by the organization (National Center for Educational Accountability, 2003). However, a recent investigation by the Texas Education Agency reviewed 5,458 student files at 16 Houston schools and found nearly 3,000 students left school in the 2000 -01 school year with incomplete files. These students should have been counted as dropouts but were not. For one thing, Texas uses the Â“leaver codesÂ” to explain why a student was no longer enrolled by identifying 30 different categories, 20 of which exempt students from being counted as dropouts. Such a large number of categories is cumbersome and confusing and requires a lot more school resources (i.e. personnel and time) to keep records straight than if a simpler system was used. Of the 16 schools, the investigator fo und all but one school had either assigned students the wrong leaver codes or had failed to back up their choices of codes with proper documents (Archer, 2003a; Galley, 2003a, 2003b). As the Houston case exemplifies, there is a trade-off between comprehensive coverage and feasibility, which is common to all social policies. If a policy is not comprehensive enough, certain individuals will fall through the cracks without being identified. On the other hand, when a policy exhausts all possible scenarios, it becomes restrictive and extremely difficult to implement. This certainly applies to the design of state data-collection systems. No data system is perfect and guarantees error-free results. Therefore the complexity in computing the hi gh school graduation rate is likely to remain, and we have to live with the less than perfect data.
Alternative High School Graduation Rates 55 Complexity Does Not Guarantee Validity Researchers have devised different approach es trying to reach a close estimation of the high school graduation rate based on differe nt assumptions. In the current study, we reviewed the conceptual advantages and disadvantages of six methods and also compared the empirical results from the six methods. Although we have no Â“trueÂ” values against which to compare the alternative estimates, we can examine the alternative estimates and compare them against each other to determine the relative strengths. Of the six methods, three simply compare the number of graduates in a given year to enrollment at an earlier grade level, two methods (Greene and Warren) incorporate adjustments to either the numerator or the denominator, while the CPI method conceptualizes the graduation rate as the pr oduct of promotion rates between every two adjacent high school grade levels and final grad uation. A critical review of the CPI method identified several conceptual and procedural flaws, and the advantages of the method claimed by its author do not hold up under sc rutiny. Empirically, the state trends based on CPI state rates are less consistent compared with the simple rates. The Greene method incorporates adjustme nts to the grade 9 enrollment as the denominator so as to address the issue of grade 9 retention and student population change over the four years of high school. Despite it s conceptual advantage, the empirical results from this study indicate that the Greene estimates are not reliable from year to year. Analysis of national graduation rate estimates from 1972 to 2001 indicates that the Greene estimates deviate substantially from estimates based on other methods during the 1980Â’s. In 12 states (see Table 9), the Greene rates presen t trends inconsistent with results from the other methods. In the state of Oregon, for ex ample, the Greene estimates for the class of 1992 is 71.6%, and shoots up by over 10% to 82.8% for the class of 1993, and then plummets even more to 65.2% for the class of 1994. Such dramatic changes in the state graduation rate in three adjacent years are hard to make sense of, and raise questions about the validity of the Greene method for producing accurate high school graduation rate estimates. WarrenÂ’s measure for the graduation rate incorporates adjustment to the CCD reported numbers of graduates and grade 9 enrollment based on CPS estimates. The national and state Warren rates for classes of 19 92 to 2001 are found reliable and consistent, and very close to the simple grade 9 estimates. However, the Warren method is operationally much more complicated than the simple grade 9 to graduation rate. Given the empirical findings from this study, there is no evidence that the conceptually more complex methods yield more accurate or valid graduation rate estimates than the simpler methods. Hence the recommendation is to use the simple graduation rate to measure the high school graduation rate required by the No Child Left Behind Act until better measures are devised. This study show s that the simple graduation rates are both conceptually and procedurally straightforward. These features are important for a measure used for accountability purposes, which needs to be simple enough for the various stakeholders to understand and for authorities and the general public to monitor. Moreover, the simple graduation method yields reliable tr ends over time, and the simple grade 9 rate yields similar results to the much more complicated Warren method, suggesting that the additional complexity is unnecessary.
Education Policy Analysis Archives Vol. 12 No. 55 56 Of the simple graduation rates, the simple grade 8 (or grade 10) to graduation rate is consistently higher than the grade 9 to grad uation rate. In addition, the difference between them has been increasing, indicating an increa sing grade 9 bulge across the nation. In this sense, the simple grade 8 (or grade 10) to graduation rate is likely to be more accurate than the simple grade 9 rate. In addition, the grade 8 rate is preferable to the grade 10 rate because the grade 10 enrollment is confounded by retention at grade 9 as well as attrition between grade 9 and 10. However, the simple grade 8 to graduation rate is not applicable at school level since high schools usually accept 9th graders from different middle schools. Although a more conservative measure, the simple grade 9 to graduation rate is conceptually the most straightforward and can be used at any administrative level. Moreover, the method also discloses larger gaps between the white and the black/Hispanic subpopulations. Given that one goal of the No Child Left Behind Act is to identify lowperforming schools and move every child forward to meet the standards, it is justifiable to err on the conservative side by magnifying problem areas than going the other direction. The conservative estimates of simple grade 9 to graduation rate bring to light the widespread practice of grade 9 retention, which is of questi onable educational value, yet costly to society and to individual students. By retaining low-performing students at grade 9, schools may delay these students from taking state manda ted tests and artificially inflate school test results. However, research shows that retenti on does not help low-achieving students to catch up with their peers (Jimerson, 2001; Na gaoka & Roderick, 2004). Moreover, retained students are at higher risk of dropping out of high school (Shepard & Smith, 1989; Jimerson, Anderson, & Whipple, 2002). The adoption of the simple grade 9 to graduation method is likely to counterbalance such practice. Requiring schools to report simple grade 9 to graduation rate in addition to test scores will encourage schools to promote as many students to graduation as possible instead of leaving low-achieving students behind.
Alternative High School Graduation Rates 57 References American Council on Education (2003, August 25). General educational development testing service: Introduction Retrieved October 21, 2003, from http://www.acenet.edu/calec/ged/intro-A.cfm. Annie E. Casey Foundation (2002). The kids count data book. Available on line at http://www.aecf.org/kidscount/databook/ Archer, J. (2003, September 24). Houston case offers lesson on dropouts Education Week Retrieved September 24, 2003 from: www.edweek.org/ew/ew_printstory.cfm?slug=04Houston.h23. Ayres, L. (1909). Laggards in our schools: A study of re tardation and elimination in city school systems. NY: Charities Publication Committee. Boesel, D., Alsalam, N. & Smith T. M. (1998). Research synthesis: Educational and Labor market performance of GED recipients. Washington, D.C: US Department of Education, Office of Educational Re search and Improvement. Retrieved 08/08/2003 from http://www.ed.g ov/PDFDOCS/GED/gedfrout.pdf. Bureau of Labor Statistics (2002a). Median income by educational attainment for people 25 years old and over. Retrieved 10/15/2003 from http://ferret.bls.census.gov/macro/032002/perinc/new03_001.htm. Bureau of Labor Statistics (2002b). Current population survey technical paper 63: Design and Methodology. Retrieved 07/09/2004 from http://www.census.gov/prod/2002pubs/tp63rv.pdf. Bureau of Labor Statistics (n.d.). Employment status of the civilian noninstitutional population 25 years and over by educ ational attainment, seasonally adjusted. Retrieved 10/09/2003 from ftp://ftp.b ls.gov/pub/suppl/empsit.cpseea5.txt. Cameron, S. V., & Heckman, J. J. (199 3). The nonequivalence of high school equivalents. Journal of Labor Economics, 11 (1), 1-47. Retrieved 06/30/2003 from http://links.jstor.org/sici?sici=0734306X%28199301%2911%3A1%3C1% 3ATNOHSE%3E2.0.CO%3B2-5. Cardenas, J. A., Robledo, M. R. and Supik, J. D. (1986). Texas school dropout survey project: A summary of findings. Intercultural Development Research Association. Carnoy, M., Loeb, S., and Smith, T.L. (2001). Do higher state test scores in Texas make for better high school outcomes? University of Pennsylvania: Consortium for Policy Research in Education. Retrieved 05/28/2003 from http://www.cpre.org/Publications/rr47.frompdf. Chaplin, D. (1999, Nov. 29). GEDs for teenagers: Are t here unintended consequences? The Urban Institute. Retrieved 09/25/2003 from
Education Policy Analysis Archives Vol. 12 No. 55 58 http://www.urban.org/UploadedPDF/GED.pdf. Chaplin, D. (2002). Tassels on the cheap. Education Next, 2(3), 24-29. Retrieved October 7, 2003 from http://www.educa tionnext.org/20023/pdf/24.pdf. Day, J. C. and Newburger, E.C. (2002, July). The big payoff: Educational attainment and Synthetic estimates of work-life earnings. Current Population Reports. U.S. Department of Commerce, Census Bureau. Retrieved October 15, 2003 from: http://landview.census.gov/prod/2002pubs/p23-210.pdf. Dorn, S. (1993). Origins of the Â“dropout problemÂ”. History of Education Quarterly 33(3), Retrieved 06/04/2003 from: http://links.jstor.org/sici?sici=00182680%28199323%2933%3A3%3C353%3AOOT%22P%3E2.0.CO%3B2-4 Dorn, S. (1996). Creating the dropout: An institution al and social history of school failure. Westport, Ct: Praeger Publishers. Dorn, S. (2003). High stakes testing and the history of graduation. Education Policy Analysis Archives 11(1), 33. Retrieved 05/28/2003 from http://epaa.asu.edu/epaa/v11n1/. Dougherty, C. (2003). Nine essential elements of statewide data-collection systems Austin, TX: The National Center for Ed ucational Accountability. Retrieved 04/21/2004 from: http://www.nc4ea.org/files/9%20elements%20Brochure.pdf. Edley, C., Jr., & Wald, J. (2002, December 16). The grade retention fallacy The Boston Globe Galley, M. (2003a, July 9). Houston faces questions on dropout data Education Week Retrieved 04/22/2004 from http://www.edweek.org/ew/ew_printstory.cfm?slug=42houston.h22. Galley, M. (2003b, September 3). Houston es capes lowered rating over dropout errors Education Week Retrieved 04/22/2004 from http://www.edweek.org/ew/ew_printstory.cfm?slug=01houston.h23. Ginsburg, A. L., Noell, J. & Plisko, V. W. (1988). Lessons from the Wall Chart. Educational Evaluation and Policy Analysis, 10 (1), 1-12. Goals 2000: Educate America Act, H.R.1804 (1994). Retrieved 07/09/2004 from http://www.ed.gov/legislation/ GOALS2000/TheAct/index.html. Goldin, C. (1998). AmericaÂ’s graduation from high school: The evolution and spread of secondary schooling in the twentieth century. The Journal of Economic History Vol.58 (2), 345-374. Greene, J. P. (2002a). High school graduation rates in the United States: Revised. Paper Prepared for the Black Alliance for Educational Options. New York, NY: Manhattan Institute for Policy Rese arch. Retrieved 05/28/2003 from
Alternative High School Graduation Rates 59 http://www.manhattan-institute.org/cr_baeo.pdf. Greene, J. P. (2002b). Public school graduation rates in the United States (Civic Report No.31). New York, NY: Manhattan Institute for Policy Research. Retrieved 05/28/2003 from http://www.manhattan-institute.org/cr_31.pdf. Greene, J. P. (2003). Public high school graduation and college readiness rates in the United States. New York, NY: Manhattan Institute for Policy Research. Retrieved 09/25/2003 from http://www.manhattan-institute.org/ewp_03.pdf. Haney, W. (2000). The myth of the Texas miracle in education. Education Policy Analysis Archives 8(41). Retrieved 09/27/2000 from http://epaa.asu.edu/epaa/v8n41. Haney, W. (2001, January). Revisiting the myth of the Texas miracle in education: Lessons about dropout resear ch and dropout prevention. Paper presented at the Dropouts in America: How severe is the problem? What do we know about intervention and prevention? Cambridge, MA. Retrieved 06/11/2003 from http://www.civilrightsproject.harvar d.edu/research/dropouts/haney.pdf. Haney, W. (2003, April). High school graduation rate s and high stakes testing. Paper Presented at the Annual Meeti ng of the American Education Research Association, Chicago, IL. Haney, W., Madaus, G., Abrams, L., Wheeloc k, A., Miao, J., & Gruia, I. (2004). The Education pipeline in the United States, 1970Â–2000 Chestnut Hill, MA: The National Board on Educational Testing and Public Policy. Retrieved February 17, 2004 from http://www.bc.edu/research/nbetpp/statements/nbr3.pdf. Harlow, C. W. (2003). Education and correctional populations (Special Report): Bureau of Justice Statistics. Retrieved February 17, 2004 from http://www.ojp.usdoj.gov/bjs/pub/pdf/ecp.pdf Hauser, R. M. (1997). Indicators of high school completion and dropout. In R. M. Hauser, B. V. Brown & W. R. Prosser (Eds.), Indicators of Children's WellBeing New York: Russell Sage Foundation. Hauser, R. M. (2000). Should we end social promotion? Truth and consequences. Paper Presented at the Conference of the Harvard Civil Rights Project on Civil Rights and High Stakes Testing, New York, NY. December 1998. Retrieved 11/18/2002 from http://www.edrs.com (ED445015). Hauser, R. M., Simmons, S. J. and Pager, D. I. (2000). High School Dropout, RaceEthnicity, and Social Background from the 1970s to the 1990s. Retrieved 06/23/2003 from http://www.edrs.com (ED449277). Hinkle, D. E., Wiersma, W., & Jurs, S. G. (1998). Applied statistics for the behavioral sciences (Fourth Edition). Boston, MA: Houghton Mifflin Company.
Education Policy Analysis Archives Vol. 12 No. 55 60 Immerwahr, J. (2000, May). Great expectations: How the public and parentsÂ—white, African American, and HispanicÂ—view higher education. National Center for Public Policy and Higher Education, Public Agenda, Consortium for Public Research in Education, National Center for Postsecondary Improvement. Retrieved 09/14/2003 from http://www.highereducation.org/reports/expectations/expectations.shtml. Jimerson, S. R. (2001). Meta-analysis of grade retention research: Implications for practice in the 21st Century. School Psychology Review, 30 (3), 420-437. Jimerson, S. R., Anderson, G. E., & Whipple, A. D. (2002). Winning the battle and losing The war: Examining the relation between grade retention and dropping out of high school. Psychology in the Schools, 39 (4), 441-457. Joftus, S., and Maddox-Dolan, B. (2003). Left out and left behind: NCLB and the American high school. Alliance for Excellent Educat ion. Retrieved 06/10/2003 from www.all4ed.org/policymakers/NCLB/index.html. Kane, T. J. & Staiger, D. O. (2002). Volatili ty in school test scores: Implications for testbased accountability systems. Brookings Papers on Education Policy, 2002, 235283. Washington, DC: Brookings Institut ion. Retrieved 09/28/2003 from http://www.dartmouth.edu/~dstaiger/Pa pers/KaneStaiger_brookings2002.pdf. Kaufman, P. (2001, January). The national dropout data co llection system: Assessing consistency. Paper presented at the Dropouts in America: How severe is the problem? What do we know about interven tion and prevention? Cambridge, MA: The Civil Rights Project at Harvard University. Retrieved 06/18/2003 from http://www.civilrightsproject.harvar d.edu/research/dropouts/kaufman.pdf. Kaufman, P., Alt, M. N. & Chapman, C. D. (2001) Dropout rates in the United States: 2000 (NCES 2002-114). Washington, DC: US Departme nt of Education, National Center for Education Statistics. Retrieved 06/23/2003 from http://nces.ed.gov/pubs2002/2002114.pdf. Kominski, R. (1987). School enrollment--Social and ec onomic characteristics: October 1983 (Current Population Reports Series P-20, No.413). Washington, D.C.: U.S. Bureau of the Census. Retrie ved October 27, 2003 from http://www.census.gov/population/www/socdemo/school/p20-413.html. Linn, R. & Baker, E. L. (2002). Accountability systems: Implications of requirements of the No Child Left Behind Act of 2001. University of California, Los Angeles, Center for the Study of Evaluation. Retrieved October 3, 2003 from http://www.cse.edu/CRESST/Reports/TR567.pdf Military Enlistment Standards. Retrieved February 17, 2004 from http://usmilitary.about.com/library/weekly/aa082701c.htm Mosteller, F. (1995). The Tennessee study of cl ass size in the early school grades. Future of Children, 5 (2), 113-127. R etrieved December 29, 2003 from
Alternative High School Graduation Rates 61 http://www.futureofchildren.org/usr_doc/vol5no2ART8.pdf. Nagaoka, J., & Roderick, M. (2004). Ending social promotion: the effects of retention (Charting Reform in Chicago Series). Chicago, IL: Consortium on Chicago School Research. Retrieved May 21, 2004 from http://www.consortium-chicago.org/publications/pdfs/p70.pdf. National Center for Education Statistics (n.d.). Documentation for the state nonfiscal survey of public elementary and secondary education: School year 1996-97 (Revised). Retrieved December 2, 2003 from http://www.nces.ed.gov/ccd/pdf/stNfis96genr.pdf. National Center for Education Statistics (n.d.). Digest of educational statistics. Retrieved March 1, 2004 from http://n ces.ed.gov/programs/digest/. National Center for Education Statistics (1976). Statistics of state school systems: 197374 U.S. Department of Health, Education, and Welfare. National Center for Education Statistics (1999, June). The condition of education 1999. Retrieved November 24, 2003 from http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=1999022. National Center for Education Statistics (2001, July). Documentation for the NCES common core of data local education ag ency universe survey: School year 19981999 (Revised). Retrieved December 2, 2003 from http://www.nces.ed.gov/ccd/pdf/pau98rgen.pdf. National Center for Education Statistics (2001, September). Documentation to the NCES common core of data state nonfiscal survey of publ ic elementary and secondar y education: School year 1998-99 (Revised September 2001). Retrieved December 2, 2003 from http://www.nces.ed.gov/ccd/pdf/stNfis98genr.pdf. National Center for Education Statistics (2002, June). The condition of education 2002. Retrieved November 24, 2003 from http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2002025. National Center for Education Statistics (April, 2003). Documentation for the state Nonfiscal survey of public elementary and secondary educat ion: School year 2001-02. Retrieved December 2, 2003 from http://www.nces.ed.gov/ccd/pdf/stNfis01gen.pdf. National Center for Educational Accountability (2003). Survey Results. Retrieved April 21, 2004, from http://www.nc4ea. org/index.cfm?pg=surveyresults. National Research Council, Committee on Educ ational Excellence and Testing Equity. (2001). Understanding dropouts: Statistics, strategies, and high-stakes testing Washington, D.C.: National Academy Press. Pedulla, J. J., Abrams, L. M., Madaus, G. F. Russell, M. K., Ramos, M. A., & Miao, J. (2003). Perceived effects of state-mandated testing programs on teaching and
Education Policy Analysis Archives Vol. 12 No. 55 62 learning: Findings from a national survey of teachers Chestnut Hill, MA: National Board on Educational Testing and Public Policy. Pettit, B. and Western, B. (2002). Inequality in Lifetime Risks of Imprisonment. Unpublished manuscript. Seattle, WA: University of Washington. Retrieved 06/28/2003 from http://www.jcpr.org/povsem/western_seminar.pdf. Rhoades, K. & Madaus, G. (2003). Errors in standardized tests: A systemic problem. Chestnut Hill, MA: Boston College, National Board on Educational Testing and Public Policy. Shepard, L. A. and Smith, M. L. (Eds.) (1989). Flunking grades: Research and policies on retention. New York, NY: The Falmer Press. Snyder, T. D., & Hoffman, C. M. (Eds.) (2003). Digest of education statistics 2002 Washington, D.C.: U.S. Department of Ed ucation, National Center for Education Statistics. Sum, A. and Harrington, P. (2003). The hidden crisis in the high school dropout problems of young adults in the U.S.: Re cent trends in overall school dropout rates and gender differen ces in dropout behavior. Boston, MA: Northeastern University, Center for Labor Market Stud ies. Retrieved on 06/28/2003 from http://www.brtable.org/pdf/914.pdf. Swanson, C. B. (2003). NCLB implementation report: Sta te approaches for calculating high school graduation rates. Washington, D.C.: The Urban Institute. Retrieved October 10, 2003 from http://www.urban.org/UploadedPDF/410848_NCLB_Implementation.pdf. Swanson, C.B. & Chaplin, D. (2003). Counting high school graduates when graduates count: Measuring graduation gates under the high stakes of NCLB. Washington, DC: The Urban Institute. Retrieved 05/23/2003 from http://www.urban.org/UploadedPDF/410641_NCLB.pdf. Swanson, C. B. (2004). Who graduates? Who doesn't? A st atistical portrait of public high school graduation, class of 2001 Washington, DC: The Urban Institute. Retrieved 02/25/2004 from http://www.urban.org/UploadedPDF/410934_WhoGraduates.pdf. The Education Trust, Inc. (2004). Educati on Watch: Achievement gap summary tables. Retrieved 05/12/2004 from http://www2.edtrust.org. Thorndike, E. (1907). The elimination of pupils from school. U.S. Bureau of Education Bulletin No.4, 1907, Whole No. 379. Wash ington, D.C.: Government Printing Office. Thurgood, L., Walter, E., Carter, G., Henn, S., Huang, G., Nooter, D., et al. (2003). NCES Handbook of Survey Methods National Center for Education Statistics. Retrieved November 30, 2003, from
Alternative High School Graduation Rates 63 http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2003603. Title I Â– Improving the Academic Achievem ent of the Disadvantaged; Final Rule. 34C.F.R. Â§200 (December 2, 2002). Retrieved 05/11/2003 from http://www.nasbe.org/Front_Page/ NCLB/NCLBfinaltitleIregs.pdf. Tyler, J. H. (2003). Economic benefits of the GED: Lessons from recent research. Review of Educational Research, 73 (3), 369-403. Warren, J. R. (2003, August). State-level high school grad uation rates in the 1990s: Concepts, measures, and trends Paper prepared for presentation at the annual meetings of the American Sociologic al Association, Atlanta, GA. Winglee, M., Marker, D., Henderson, A. Young, B., and Hoffman, L. (2000). A recommended approach to prov iding high school dropout and completion rates at the state level (NCES 2000-305). Washington, DC: US Department of Education, National Center for Education Stat istics. Retrieved 05/28/2003 from http://nces.ed.gov/pubs2000/2000305.pdf. Young, B. (2002). Public high school dropouts and completers from the Common Core of Data: School year 1998-99 and 1999-2000 (NCES 2002-382). Washington, DC: US. Department of Education, Nati onal Center for Education Statistics. Retrieved 05/28/2003 from http://nces.ed.gov/pubs2002/2002382.pdf. Young, B. A. & Hoffman, L. (2002). Public high school dropouts and completers from the Common Core of Data: School years 1991-92 through 1997-98 (NCES 2002-317). Washington, DC: US Department of Education, National Center for Education Statistics. Retrieved 06/23/2003 from http://nces.ed.gov/pubs2002/2002317.pdf. About the Authors Jing Miao is a PhD candidate in Educational Research, Measurement and Evaluation program in the Lynch School of Education at Boston College. She was the coauthor of several research reports at the National Board on Educational Testing and Public Policy (NBETPP) at Boston College. Her current res earch interests include grade retention, high school dropout and high-stakes testing. Walt Haney is Professor of Education in the Lynch School of Education and Senior Research Associate in the Center for the Study of Testing, Evaluation and Educational Policy, both at Boston College. He has been au thor or co-author of a number of previous articles in Education Policy Analysis Archives His most recent artic le in a paper journal is "Drawing on education: Using drawings to document schooling and support change." Harvard Educational Review Fall 2004 (74:3), pp. 241-272 (with M. Russell and D. Bebell).
Education Policy Analysis Archives Vol. 12 No. 55 64 Appendix I Reporting Categories for High School Graduates/Completers One clarification is necessary on the change of reporting categories for high school graduates/completers. CCD surveys are conducte d annually with reference to school years instead of the calendar year. In this study, the CCD survey for a school year (e.g., 1998-99) is often referred to as CCD followed by the sur vey year, which is the beginning of the school year (e.g., CCD for the 1998-99 school year is re ferred to as CCD 1998). In each CCD file, while the grade enrollment data are reported for the current school year being surveyed, the number of high school graduates and compl eters are reported for the previous school year. For example, in CCD 1998 (i.e., CCD for th e 1998-1999 school year), the grade enrollments are reported for 1998-1999 as of fall 1998, wh ile the graduate/completer counts are for the 1997-98 school year as of spring 1998. High school graduate/completer counts at st ate level are available in CCD files for the 1986-2001 survey years. There has been one change in the reporting strategy since CCD 1997. Up to CCD 1996, four categories of high school completers were reported in the CCD state non-fiscal data files:13 regular high school diploma completers (REGDIP), other diploma recipients (OTHDIP)14, high school equivalency recipients (EQUIV) and other high school completers (OTHCOM). According to the documentation for the State Nonfiscal Data Survey for the 1996-97 school year (NCES, n. d.), the definitions for the four categories are as follows: Regular Diploma Recipient: A graduate of regular day school who received a high school diploma during the previous school year or subsequent summer school. The diploma is based upon completion of high school requirements through traditional means. Other Diploma Recipient: A student w ho received a diploma by completing a program other than one in a regular school program during the previous school year or subsequent summer. Other High School Completer: A studen t who has received a certificate of attendance or other certificate of completion in lieu of a diploma during the previous school year and subsequent summer school. High School Equivalency Recipient: An in dividual, age 19 years or younger, who received a high school equivalency certifi cate during the previous school year or subsequent summer. Since CCD 1997, other diploma recipients are no longer reported separately but are combined with regular diploma recipients as one category15 (NCES, n.d.). Therefore, state non-fiscal files reported three categories of high school completers for the 1997-2001 survey years. 13 This is based on state Excel files downloaded from http://www.nces.ed.gov/ccd/stnfis.asp. The layo ut of the text files is slightly different. 14 For the 1986 survey year, the second category is OTHPRG, which refers to completers of other programs such as GED adult evening school etc. (layout file for 1986-87 st861b.dat retrieved from http://www.nces.ed. gov/ccd/data/txt/stNfis86lay.txt) 15 The state nonfiscal files continue to la bel this new category as REGDIP although TOTDPL (used in district files) seems to be more appropriate.
Alternative High School Graduation Rates 65 High school graduate/completer counts are not reported by CCD at the national level (namely, the 50 states plus the District of Columbia) but are available from the DES back to the 1968-69 school year. The DES counts include Â“graduates of regular day school programs, but exclude graduates of other pr ograms and persons receiving high school equivalency certificatesÂ” (Snyder & Hoffman, 20 03, p. 128). Since the comparison of DES and CCD data at state level identified very f ew discrepancies from 1986-87 to 2001-02, it is reasonable to speculate that the DES reporting on numbers of graduates experienced the same change as in the CCD State Nonfiscal Data Survey. That is to say, only regular diploma recipients are reported as graduates in DES before the class of 1997, while both regular diploma and other diploma recipients are coun ted as graduates in DES since the class of 1997. In order to compute alternative graduation rates, the current study uses the DES reported high school graduates at the national level. For state graduation rates, the REGDIP counts are used from CCD survey years 1986 to 2001. Table A1 CCD Reporting Categories for High School Graduates/Completers Year CCD Reported Current Study Uses National None None DES data State 1986-1996 4 categories: REGDIP, OTHDIP, EQUIV, OTHCOM REGDIP 1997-2001 3 categories: REGDIP1, EQUIV, OTHCOM REGDIP1 1. Sum of REGDIP and OTHDIP from previous years.
Education Policy Analysis Archives Vol. 12 No. 55 66 Appendix II State Level Data: DES vs. CCD Enrollment and graduation data from both CCD and DES were compared at state level for the 1986-87 to 2001-02 school years. Only nine discrepancies (less than .01% of the data entries) were found out of over 10,000 data entries, and the magnitudes of most discrepancies were within 5% of the observe d value reported in CCD files. Three large discrepancies (either with percent difference larg er than 5% or with absolute differences over 1000) were identified from these comparisons (see Table 3-2). Based on the data from adjacent years, the current study resolves th e discrepancies by adopting the DES reported data for all three cases. Table A2 Discrepancies between CCD & DES Data State Academic Year Variable Reported in CCD Reported in DES Used in Current Study CA 1996-97 Graduates311,8181 269,0712 269,071 TX 1992-93 Graduates162,2701 160,5463 160,546 VT 1992-93 Graduates5,6971 5,2153 5,215 1. CCD state non-fiscal data files retrieved from http://www.nces.ed.gov/ccd/stnfis.asp 2. Digest of Education Stat istics 2000, Table 102. 3. Digest of Education St atistics 1997, Table 99.
Alternative High School Graduation Rates 67 Appendix III The Effect of Enrollment Change on the Simple Graduation Rate The following example of a single cohort16 will help to illustrate the effect of grade enrollment change rate on graduation rate estimates. In general, students who were in grade 8 during 1996-1997 should graduate in the spring of 2001. Table A-1 shows that in the fa ll of 1996, a total of 3,403 thousand students enrolled students in grade 8 nationwide; one year later the enrollment in grade 9 is 3,819 thousand, 12.2% more than in grade 8 in 1996 In 1998, U.S. public schools enrolled 3,382 thousand students in grade 10, which is 11.4% less than in grade 9 the previous year. Therefore, for the class of 2001 in the U.S., th e simple grade 8 graduation rate is 75.5%, the simple grade 9 graduation rate is 67.2% and the simple grade 10 graduation rate is 75.9%. Although the grade enrollment change rates (i.e. the grade 9 bulge rate and the grade 10 attrition rate) are over 10%, the difference between the simple grade 9 graduation rate and simple grade 8 graduation rate (or simple grade 10 graduation rate) is substantial in this case, yet almost one third smaller than the grade enrollment change rate. Table A3 The Effect of Grade Enrollment Chang e on National Graduation Rate (Number in thousands) 1996-1997 1997-1998 1998-1999 1999-2000 2000-2001 G8 3,403 G9 3,819 G10 3,382 Grads 2,568 (G9-G8)/G8 12.2% (G10-G9)/G9 -11.4% Grads/G8 75.5% Grads/G9 67.2% Grads/G10 75.9% G8R-G9R 8.3% G10R-G9R 8.7% G10R-G8R 0.4% The following section illustrates the effect of grade 9 bulge and grade 10 attrition on the difference between three simple graduation rate estimates in more generalized terms. Suppose the grade 8 enrollment for a hypothetical cohort of students is N in year 1. Assume there are (100xRb)% more students in grade 9 in year 2 than in grade 8 the previous year, then the grade 9 enrollment for the reference cohort is (1+Rb)N. Moreover, assume there are (100xRa)% less students in grade 10 in year 3 than in grade 9 the previous year, then the grade 10 enrollment for the reference cohort equals 16 This is not a cohort analysis in the strictest sense since students may join or leave a cohort as time goes on, and it is not possible to tr ack individual students due to the nature of aggregate data.
Education Policy Analysis Archives Vol. 12 No. 55 68 (1-Ra)(1+Rb)N. Suppose the number of students graduate in year 5 is G, therefore the three simple graduation rates and the difference betwee n these rates can be expressed as in the lower part of Table A-2. Table A4 The Effect of Grade Enro llment Change on Graduation RateÂ—General Case Year 1 Year 2Year 3 Year 4Year 5 G8 N G9 (1+Rb)N G10 (1-Ra)(1+Rb)N Grads G (G9-G8)/G8 (100xRb)% (G10-G9)/G9 (100xRa)% Grads/G8 G/N Grads/G9 G/[(1+Rb)N] Grads/G10 G/[(1-Ra)(1+Rb)N] Grads/G8-Grads/G9 (G/N)[Rb/(1+Rb)] Grads/G10-Grads/G9 (G/N)[Ra/(1-Ra)(1+Rb)] Grads/G10Grads/G8 (G/N)[(Ra-Rb+RaRb)/ (1-Ra)(1+Rb)]
Alternative High School Graduation Rates 69 Education Policy Analysis Archives http:// epaa.asu.edu Editor: Gene V Glass, Arizona State University Production Assistant: Chris Mu rrell, Arizona State University General questions about appropriateness of topics or particular articles may be addressed to the Editor, Gene V Glass, firstname.lastname@example.org or reach him at College of Education, Arizona Stat e University, Tempe, AZ 85287-2411. The Commentary Editor is Casey D. Cobb: email@example.com. EPAA Editorial Board Michael W. Apple University of Wisconsin David C. Berliner Arizona State University Greg Camilli Rutgers University Linda Darling-Hammond Stanford University Sherman Dorn University of South Florida Mark E. Fetler California Commission on Teacher Credentialing Gustavo E. Fischman Arizona State Univeristy Richard Garlikov Birmingham, Alabama Thomas F. Green Syracuse University Aimee Howley Ohio University Craig B. Howley Appalachia Educational Laboratory William Hunter University of Ontario Institute of Technology Patricia Fey Jarvis Seattle, Washington Daniel Kalls Ume University Benjamin Levin University of Manitoba Thomas Mauhs-Pugh Green Mountain College Les McLean University of Toronto Heinrich Mintrop University of California, Berkeley Michele Moses Arizona State University Gary Orfield Harvard University Anthony G. Rud Jr. Purdue University Jay Paredes Scribner University of Missouri Michael Scriven University of Auckland Lorrie A. Shepard University of Colorado, Boulder Robert E. Stake University of IllinoisÂ—UC Kevin Welner University of Colorado, Boulder Terrence G. Wiley Arizona State University John Willinsky University of British Columbia
Education Policy Analysis Archives Vol. 12 No. 55 70 AAPE Editorial Board Associate Editors Gustavo E. Fischman & Pablo Gentili Arizona State University & Universida de do Estado do Rio de Janeiro Founding Associate Editor for Spanish Language (1998Â—2003) Roberto Rodrguez Gmez Hugo Aboites Universidad Autnoma Metropolitana-Xochimilco Adrin Acosta Universidad de Guadalajara Mxico Claudio Almonacid Avila Universidad Metropolitana de Ciencias de la Educacin, Chile Dalila Andrade de Oliveira Universidade Federal de Minas Gerais, Belo Horizonte, Brasil Alejandra Birgin Ministerio de Educacin, Argentina Teresa Bracho Centro de Investigacin y Docencia Econmica-CIDE Alejandro Canales Universidad Nacional Autnoma de Mxico Ursula Casanova Arizona State University, Tempe, Arizona Sigfredo Chiroque Instituto de Pedagoga Popular, Per Erwin Epstein Loyola University, Chicago, Illinois Mariano Fernndez Enguita Universidad de Salamanca. Espaa Gaudncio Frigotto Universidade Estadual do Rio de Janeiro, Brasil Rollin Kent Universidad Autnoma de Puebla. Puebla, Mxico Walter Kohan Universidade Estadual do Rio de Janeiro, Brasil Roberto Leher Universidade Estadual do Rio de Janeiro, Brasil Daniel C. Levy University at Albany, SUNY, Albany, New York Nilma Limo Gomes Universidade Federal de Minas Gerais, Belo Horizonte Pia Lindquist Wong California State University, Sacramento, California Mara Loreto Egaa Programa Interdisciplinario de Investigacin en Educacin, Chile Mariano Narodowski Universidad Torcuato Di Tella, Argentina Iolanda de Oliveira Universidade Federal Fluminense, Brasil Grover Pango Foro Latinoamericano de Polticas Educativas, Per Vanilda Paiva Universidade Estadual do Rio de Janeiro, Brasil Miguel Pereira Catedratico Universidad de Granada, Espaa Angel Ignacio Prez Gmez Universidad de Mlaga Mnica Pini Universidad Nacional de San Martin, Argentina Romualdo Portella do Oliveira Universidade de So Paulo Diana Rhoten Social Science Research Council, New York, New York Jos Gimeno Sacristn Universidad de Valencia, Espaa Daniel Schugurensky Ontario Institute for Studies in Education, Canada Susan Street Centro de Investigaciones y Estudios Superiores en Antropologia Social Occidente, Guadalajara, Mxico Nelly P. Stromquist University of Southern California, Los Angeles, California Daniel Suarez Laboratorio de Politicas Publicas-Universidad de Buenos Aires, Argentina Antonio Teodoro Universidade Lusfona Lisboa, Carlos A. Torres University of California, Los Angeles Jurjo Torres Santom Universidad de la Corua, Espaa Lilian do Valle Universidade Estadual do Rio de Janeiro, Brasil