|USFDC Home||| RSS|
This item is only available as the following downloads:
xml version 1.0 encoding UTF-8 standalone no
mods:mods xmlns:mods http:www.loc.govmodsv3 xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govmodsv3mods-3-1.xsd
mods:relatedItem type host
mods:identifier issn 1068-2341mods:part
mods:detail volume mods:number 12issue 5series Year mods:caption 20042004Month February2Day 44mods:originInfo mods:dateIssued iso8601 2004-02-04
xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam a22 u 4500
controlfield tag 008 c20049999azu 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E11-00354
Educational policy analysis archives.
n Vol. 12, no. 5 (February 04, 2004).
Tempe, Ariz. :
b Arizona State University ;
Tampa, Fla. :
University of South Florida.
c February 04, 2004
Comparison of academic development in Catholic versus non-Catholic private secondary schools / Mikyong Minsun Kim [and] Margaret Placier.
Arizona State University.
University of South Florida.
t Education Policy Analysis Archives (EPAA)
1 of 28 A peer-reviewed scholarly journal Editor: Gene V Glass College of Education Arizona State University Copyright is retained by the first or sole author, who grants right of first publication to the EDUCATION POLICY ANALYSIS ARCHIVES EPAA is a project of the Education Policy Studies Laboratory. Articles appearing in EPAA are abstracted in the Current Index to Journals in Education by the ERIC Clearinghouse on Assessment and Evaluation and are permanently archived in Resources in Education Volume 12 Number 5February 4, 2004ISSN 1068-2341Comparison of Academic Development in Catholic vers us Non-Catholic Private Secondary Schools Mikyong Minsun Kim University of Missouri-Columbia Margaret Placier University of Missouri-ColumbiaCitation: Kim, M., Placier, M., (2004, February 4). Comparison of Academic Development in Catholic versus Non-Catholic Private Secondary Scho ols. Education Policy Analysis Archives, 12 (5). Retrieved [Date] from http://epaa.asu.edu/epaa /v12n5/.AbstractUtilizing hierarchical linear models, this study of 144 private schools (72 Catholic and 72 non-Catholic schools) d rawn from the National Education Longitudinal Study of 1988 disco vered that Catholic school students scored lower in reading th an students at non-Catholic private schools. Analysis of internal school characteristics suggested that lower growth in read ing achievement might be related in part to lower stude nt morale in Catholic schools. However, we found no significant differences between Catholic and non-Catholic private secondary schools in the development of students' math, history/social s tudies, and science abilities from eighth to tenth grades. This study also
2 of 28 identified important studentand school-level vari ables such as Catholicism, gender, risk factor, parental involvem ent, and enrollment size that help to explain the outcomes. Comparison of academic achievement for Catholic ver sus public secondary schools has been an active field of research for ne arly 20 years, beginning with Coleman, Hoffer and Kilgore's (1982a, 1982b) analys is of 1980 High School and Beyond (HSB) data, which found a positive Cath olic school effect. This work has been grounded in social capital theory, wh ich explains the Catholic school advantage in terms of the value for young pe ople of being embedded in a network of relationships, in this case a network based on religious association (Coleman and Hoffer 1987). Subsequent studies have either lent support, albeit sometimes qualified, to their findings (Bryk, Lee, and Holland 1993; Gamoran 1992; Hoffer 2000; Hoffer, Greeley and Coleman 1985 ; Jencks 1985; Jensen 1986; Keith 1985; Marsh 1991; Marsh and Grayson 199 0; Riordan 1985; Sander 1996) or called them into question (Alexande r 1985; Gamoran 1996; Graetz 1990; LePore and Warren 1997; Noell 1982; Wi llms 1985). Coleman et al. (1982a) noted that findings of publi c-private school comparisons could have implications for policy decisions and pa rent choicesimplications that have become even more salient today. A decade later, however, Witte (1992) argued that in studies with proper controls, achievement differences between public and private schools were too small a nd uncertain to have policy import. Nevertheless, school choice advocates have relied heavily upon Coleman et al.'s findings (Chubb and Moe 1988); and the Catholic school effect contributes to legal arguments for inclusion of Cat holic schools in voucher plans in cities such as Cleveland, Ohio. Voucher systems are predicated on the argument that market competition among schools will produce higher achievement in all schools, without increasing cost s. Catholic schools may appear to have an advantage over other private scho ols in a market model because of their relative efficiency (though costs are rising, see Bryk, Lee, and Holland 1993; Harris 2000) and their effectiveness with disadvantaged urban students (Hallinan 2000). Urban school reformers al so advocate making the core curriculum and sense of community found in Cat holic schools part of public school restructuring efforts (Bushweller 1997; Hudo lin-Gabin 1994). Few studies, however, have compared Catholic school s with other private schools to examine whether any achievement effect i s associated with the schools' Catholic status or simply with their priva te status. Ornstein (1989) reports both similarities and differences between C atholic and other private schools. Private schools in general are smaller tha n public schools, but Catholic schools are larger on average than other privates. Catholic schools are also reported to be more urban, and their demographics i nclude more ethnic minority, immigrant, and low-income students. Priva te and Catholic schools both have more stringent academic requirements for gradu ation than public schools, but Catholic schools have the highest graduation ra tes despite their less elite student populations. Coleman et al. (1982b) found t hat both Catholic and other private schools exhibited higher student achievemen t than public schools, but private school students showed higher self-esteem a nd sense of fate control than either public or Catholic school students. How ever, the category other private schools in the HSB database was very hete rogeneous and
3 of 28 amorphous (p. 11), because the sampled schools var ied so widely in purpose, size, sustainability, and other characteristics (a limitation also noted by Noell 1982). Bryk, Lee and Holland (1993) argue that even if achievement differences are not supported, Catholic schools serve the commo n good by producing more than test scores. Catholic schools, these authors c ontend, are moral communities that emphasize equity and social justic e rather than individual self-interest. In contrast, they noted that other p rivate schools serve a greater variety of purposes and a narrower range of student s. Using data from the High School Effectiveness Suppl ement of the National Education Longitudinal Study of 1988 (NELS:88), Lee et al. (1998) compared math course-taking in public, Catholic, and indepen dent secondary schools and reported that in all private secondary schools stud ents on average take more advanced math courses. Catholic schools were especi ally notable for more math course-taking among a broader range of student s. However, this study had a limitation in that baseline math scores were unavailable for more than half of the sample.Gamoran (1996), in the analysis of urban high schoo ls in the NELS:88, found no advantages in achievement in mathematics, readin g, science, and social studies for either Catholic or secular private high schools compared with public magnet schools. Gamoran did not examine the differe nt (or similar) school characteristics that could influence these student outcomes. Because previous research did not settle the questi on of Catholic school effectiveness, and given the current salience of sc hool comparisons in policy-making, more research is needed.This study asks: Do students in Catholic secondary schools develop better academically than those in non-Catholic private sch ools? The primary purpose is to compare the effectiveness of Catholic schools with that of non-Catholic private schools in student academic development in reading, math, history/social studies, and sciencethe major subje ct areas in school curricula. The secondary purpose is to explore student-level a nd school-level factors influencing students' academic development. Finally if any significant school-level differences are found, this study is d esigned to develop explanations for such institutional effects.How does this study differ from previous ones? Whil e most previous Catholic school studies used public schools as a reference f or comparison, this study compares Catholic schools with non-Catholic private schools. Such a comparison makes sense because private schools have organizational structures and climates distinctly different from t hose of public schools. The institutional perspective focuses attention on pri vateness as an organizational characteristic, rather than social capital (e.g., n etwork, parental involvement) in the school community. According to Chubb and Moe (1 990), "All schools in the private sector have two institutional features in c ommon: society does not control them directly through democratic politics, and society does control them indirectly through the marketplace" (p. 475).Most previous studies used mathematics achievement as the dependent
4 of 28 variable, but this study examines four major subjec t areas, thereby more fully representing students' overall academic achievement in secondary school. Gamoran's (1996) study did include these four subje ct areas, but our study employed more extensive student-level and internal school-level variables. In addition, Gamoran's study included only urban schoo ls, while this study included Catholic and non-Catholic private schools from all geographic locations within the U.S. Moreover, previous studies did not generate explanations for differential effects among schools, even when Catho lic schools were found to be more effective than public schools. Our intentio n was to examine the reasons for any differences discovered, thus provid ing educators with important information for school reforms.Data and MethodsData and SampleWe used data from the National Education Longitudin al Study of 1988 (NELS:88) to create a two-level (student, school) h ierarchical linear model. We selected students in private schools (Catholic scho ols, non-Catholic religious schools, and independent schools) from the 1988 Bas e Year Study of eighth graders and from the 1990 Follow-Up Study of tenth graders. We created a database for student characteristics and for instit utional environment and characteristics by merging the student data file wi th the school component data file. This study included only students and schools that responded to both the 1988 (base year) and 1990 (the first follow-up) sur veys. The national scale of the survey data was extensive and the representatio n of schools by sector was justified by previous studies (Gamoran 1996; Rumber ger 1995). We obtained usable data for 1,789 students in 144 s chools: 841 students in 72 Catholic schools and 948 students in 72 non-Catholi c private schools. Among the non-Catholic private schools, there were 371 st udents from 31 non-Catholic religious schools and 577 students from 41 independ ent schools. In our preliminary analysis, we found that mean school cha racteristics and student characteristics of non-Catholic religious schools a nd independent schools were more alike than those of Catholic schools and non-C atholic religious schools. Thus, grouping non-Catholic religious schools and i ndependent schools together seems justified.VariablesDependent Variables. The outcome (dependent) variables for the HLM anal yses are students' achievement scores in reading, mathem atics, history/social studies, and science in the tenth grade. Achievemen t measures in the four subject areas are (1) reading comprehension, with 2 1 items consisting of five short passages followed by comprehension and interp retation questions; (2) mathematics, which consists of 40 items containing simple math, comprehension, and problem-solving items; (3) histo ry/social studies, consisting of 30 items that assessed students' knowledge of Am erican history, citizenship, and geography; and (4) science, consisting of 25 it ems from content areas of earth, life, and physical sciences. Using these com posite achievement test
5 of 28 scores makes academic achievement in reading, math, social studies, and science seem to be quite valid and reliable measure s. Independent Variables. Catholic school status was the key independent vari able. In addition, two kinds of independent variables wer e included in the analysis: student-level and school-level predictors. Studentlevel predictors are the base year achievement test scores in reading, math, hist ory/social studies, and science; student's initial GPA; minority status (Af rican American, Hispanic, or Asian-American); gender; family socioeconomic statu s (SES); risk factors; student's elective reading; student's religious aff iliation (Catholic vs. non-Catholic); student's perception of each subject 's usefulness (reading, math, history/social studies, and science); and the numbe r of hours spent on homework. These variables were included to statisti cally adjust for students' differences in initial academic preparation, religi ous affiliation, and family SES. Because students' religion data was not collected i n 1988 (8th grade survey), we used a student's religion (F1S81) in his/her 10t h grade as an alternative. It is based on the assumption that a student's religion w ould not change much during the period. Some of the variables for exam ple, the number of hours spent on homework, the amount of elective solitary reading, and student's perception of each subject's usefulness were not included in previous studies. We included them because our experience tells us th at these characteristics can affect students' academic achievement and invol vement in the subject areas, and thus would be worth exploring. The risk factors variable was included because some previous reports have noted that Catho lic schools help to develop disadvantaged students (Hallinan 2000). In the NELS:88 study, students received a risk factor score of 0-6 based on how many of the following risk factors are present in their lives: lowest soc ioeconomic quartile, single-parent family, older sibling dropped out of high school (asked in the tenth grade), changed schools two or more times from firs t through eighth grade, average grades of C or lower from sixth through eig hth grade, and repeated an earlier grade from first through eighth grade.School-level variables were divided into two catego ries: global and internal school characteristics. Global characteristics are defined in this study as geographical location, type of school, and school s tructural characteristics that are extremely difficult for school administrators t o change or manipulate. Internal school characteristics are defined as char acteristics that are relatively changeable and observable to students and faculty. Global school characteristics were included to adequately assess the effect of attending a Catholic school by controlling for other important global school characteristics. Variables in this category were Catholic school sta tus, enrollment size, average pre-test scores, average parental SES, percentage o f minority students, and institutional location (urban, suburban, or rural). Aggregate pre-test scores and mean SES were treated as global school characterist ics because these variables must be controlled to assess the effects of Catholic schools. Internal school characteristics were included to understand the reasons for any effects of Catholic schools. Examining internal characteris tics can also help us determine the kind of school policy or environment that can positively or negatively affect students' academic development. T he internal school variables were monitoring of academic progress, strictness of school rules, extent of school's encouragement for parental support and inv olvement, teachers'
6 of 28 morale, students' morale, and teacher-student ratio s. See Appendix A for the list of all the variables and their coding schemes.Analysis ProceduresWe began the analysis by generating descriptive sta tistics such as means, standard deviations, and correlations. Table 1 pres ents the means and standard deviations of variables included in HLM analysis, a s well as the correlation coefficients between the variables and Catholic sch ools. Except for Catholic school, mean pretests, student's perception of each subject's usefulness, and parental education, the listed variables had signif icant positive or negative effects on at least one of the outcomes when includ ed with other predictors in the HLM models.To test the null hypothesis that there is no signif icant difference in development of academic achievement in reading, mathematics, hi story/social studies, and science between Catholic and non-Catholic private s econdary schools, we used hierarchical linear modeling. HLM has two major adv antages over ordinary least-squares regression analysis (Bryk and Raudenb ush 1992; Kreft and de Leeuw 1998). First, it lets researchers investigate within a single analytic framework, hypotheses about the effects of both ind ividual(student) and institution(school) level predictors on the outco mes of interest. Second, in working with nested data (i.e., students nested wit hin schools), HLM takes into account dependencies among observations within clus ters (schools) when estimating parameters of interest such as the effec t of attending a Catholic school. If we ignore these dependencies, we may und erestimate standard errors (Burstein 1980; Bryk and Raudenbush 1992).There are four kinds of HLM models for each subject area: unconditional one-way Analysis of Variance (ANOVA) models, studen t models, global models, and full models. The unconditional model includes n o studentor school-level predictors. The student model consists only of indi vidual students' characteristics (e.g., gender, academic preparation ) or their family background (assessed at the eighth grade); it includes no scho ol-level predictors. The student models provided the foundation on which to build the individual-level models of the subsequent global models and full mod els. We created the global models to test the study's hypotheses. As the name implies, the model includes global school characteristics such as school enroll ment size, racial minority proportion, and Catholic school status. However, st udents' aggregate eighth-grade academic achievement scores and mean p arental SES were also considered for inclusion in the global model becaus e not only individual students' initial achievement and SES but also thei r aggregate scores can account for important initial student body characte ristics that are often beyond the school's control. The full model includes impor tant internal school characteristics related to academic development; th ese characteristics also help to explain the reasons for the differences between Catholic and non-Catholic private schools. The variables in the models were s elected in response to previous related studies and theories, researchers' intuitions and experiences, and statistical significance level (p = 0.10) in th e exploratory models. The alpha level for the hypothesis testing was set at 0.10.
7 of 28 For exploratory purposes, we attempted to determine whether there is any significant cross-level interaction effect. For exa mple, in the regression analysis of reading achievement we checked interaction effec ts between initial GPA (at the individual level) and Catholic school (at the i nstitution level). Finally, following similar analysis procedures, we created m odels that include almost the same variables across the four subject areas. "Almo st" indicates that we had to model somewhat differently because there were subje ct-specific variables. For example, "pre-test reading" and "reading useful" sh ould be included only in the models explaining reading achievement. The suppleme ntal analysis models consist of all the variables that were included at least once in the HLM models (throughout Tables 2-5). The supplemental analysis helped us to recheck the findings of the original models and to understand t he effects and patterns of independent variables across the models.Results and Interpretations: Examining the Effectiv eness of Catholic SchoolsMeans and standard deviations (Table 1) show that o verall, students at non-Catholic private schools had higher pre-test an d post-test means in all subjects than students in Catholic schools. Student s at non-Catholic private schools also came from wealthier families, and thei r parents had higher levels of educational attainment than parents of Catholic school students. Table 1. Means, Standard Deviations, and Correlatio n Coefficients of Variables Included in HLM Analyses Variable list Catholic schools Non-Catholic schools All schools Simple r with Catholic schools MeansSDMeansSDMeansSD Institution-level variables Enrollment1.330.531.640.701.490.64-0.24**Catholic school 0.500.50 Mean SES0.170.430.840.380.500.52-0.64**Mean pretest reading54.684.7758.834.4056.755.03-0.4 1** Mean pretest math53.005.2160.425.8956.716.68-0.56**Mean pretest social studies 53.855.0258.144.7255.995.31-0.41** Mean pretest science52.895.4257.955.5555.305.74-0.4 8** Teacher student ratios23.445.2513.945.4618.697.150. 67** Remedial reading6.617.983.425.915.017.180.22**Parental involvement4.280.743.990.884.130.830.18*Monitoring academic progress 4.700.474.120.924.750.54-0.02 Student morale4.110.594.270.594.170.68-0.24*Strict school rules2.980.412.920.392.950.400.08 Individual-level variables Pretest Reading55.259.3359.388.6957.399.25-0.22**
8 of 28 Math53.869.3661.059.1957.589.96-0.35** History54.988.8858.709.1256.889.27-0.20** Science53.469.1858.799.9556.919.97-0.26**Posttest Reading54.678.5559.217.5057.038.36-0.26** Math54.428.7460.317.6157.475.76-0.33** History54.318.6958.368.5056.398.84-0.22** Science53.538.9758.878.7356.289.27-0.28**Female1.550.501.510.501.530.500.04 Religion: Catholic0.820.390.150.360.460.500.67**Parental SES0.260.640.880.560.580.68-0.45**Initial GPA3.220.613.230.603.230.60-0.01 Risk factors0.330.590.210.460.270.530.11**Elective reading1.901.522.051.541.981.53-0.05*Homework hours4.471.345.101.674.791.55-0.19**Reading useful3.120.793.230.753.180.77-0.07**Math useful3.310.783.230.783.270.780.05*Social studies useful2.540.862.770.832.660.85-0.14* Science useful2.830.922.940.872.890.90-0.06*Parental Education3.521.194.561.204.061.31-0.39**Note: p <0.05; ** p <0.01 (two-tailed)We gathered preliminary information using unconditi onal one-way ANOVA models (not shown in tables). The grand means were similar: 56.64 for reading, 56.81 for math, 55.94 for history/social studies, a nd 55.65 for science. The 95% confidence interval of the means of these subjects falls between 54.71 and 57.73. The ANOVA model also let us calculate an int ra-class correlation coefficient, also called a cluster effect or the pr oportion of school-level variance. The intra-class correlation was 0.35 in reading, 0. 41 in math, 0.33 in history/social studies, and 0.39 in science. In oth er words, about 35% of the total variance in reading, 41% in math, 33% in hist ory/social studies, and 39% in science was located at the school level.Tables 2-5 present the summary results of the three other models student model (with level 1 predictors), global model (with the student-level predictors plus school-level predictors), and full model (with internal school-level predictors in addition to the global model variables). Except for the intercept, the random effects of student-level variables were fixed, beca use little variation was found across schools. All student-level variables were gr and mean centered; therefore, the intercepts are unadjusted means of t he outcomes. We will explain our main parsimonious HLM models first, then discus s additional findings from supplemental HLM models.Developing students' achievement in readingTable 2 presents the results of HLM analysis in rea ding achievement. Attending a Catholic school had a negative effect on developi ng reading skills between eighth and tenth grades compared with attending a n on-Catholic private school.
9 of 28 The student model consists of five student characte ristics variables: eighth-grade reading achievement score, eighth-grad e overall GPA, the number of risk factors, parental SES, and elective solitar y reading. Only risk factors were negatively associated with the dependent varia ble. The associations and directions of the variables are consistent with our expectation. The five student characteristics explain about 31% of the total stud ent-level variance. Table 2. Development of Students' Achievement in Re ading Student ModelGlobal ModelFull Model Independent Variablesbset-ratiobset-ratiobset-ratio Institution-level variables Intercept56.9940.194294.042***55.4890.61290.683***5 3.0171.35439.166*** Global characteristics Enrollment 0.7500.2852.636***0.7260.2812.589***Catholic school -0.7670.456-1.684*-0.6950.450-1.544 Mean SES 1.2920.5422.385**1.1670.5392.165** Internal characteristics Student morale 0.6080.2972.044** Individual-level variables Parental SES1.0460.2294.576***0.4030.2691.4990.4050 .2691.504 Initial GPA1.9780.2398.261***2.0630.2388.653***2.03 40.2388.530*** Pretest reading0.5430.01732.593***0.5330.01731.951* **0.5340.01732.023*** Risk factors-0.4410.242-1.818*-0.4290.241-1.781*-0. 4470.241-1.855* Elective reading0.3620.0874.180***0.3860.0864.480** *0.3870.0864.496***Note: *** p <= .01; ** p<=.05; p<=.10Global models were created to test the study's hypo theses. The global model consists of three school-level variables (enrollmen t size, mean parental SES, and Catholic schools) in addition to student charac teristics from the student model. The three school-level variables explained a bout 66% of the total school-level variance. Holding enrollment size, mea n parental SES, and the five student-level variables constant, we found that att ending a Catholic school was negatively associated with developing students' rea ding achievement scores. The negative effect of Catholic school attendance w as statistically significant (t=-1.684, p<0.1), and null hypothesis 1 was reject ed. In other words, if there are two students of comparable initial reading leve l, risk factors, and SES background, one attending a Catholic school and one attending a non-Catholic private school, and if the schools are similar in s ize and mean parental SES level, the student at the Catholic school is likely to have a slightly lower reading score than the student at the non-Catholic private school. Although there is no simple way to address the prac tical importance of statistical results, we present effect sizes to hel p readers understand some practical meanings of the expected mean differences of the four achievement outcomes between Catholic and non-Catholic schools. The global model of Table 2 shows that the expected difference in mean reading post-test scores between Catholic and non-Catholic schools is 0.767. We obtained a between-school standard deviation, 3.913, from the unconditional ANOVA
10 of 28 model (not shown in the table). Plugging two measur es into a commonly used effect size formula (to calculate standardized mean differences) (see Borg and Gall, 1989; Hopkins, Hopkins, and Glass, 1996; Kim, 1995; Kirk, 1996), we found a difference of 0.20 standard deviations (fro m 0.767/3.913) in students' reading scores between Catholic and non-Catholic pr ivate school sectors. In other words, non-Catholic private school students w ere estimated to score 0.20 standard deviations higher (or an 8 percentile diff erence) in their reading achievement test, on average, than Catholic school students. Differences of this magnitude have practical importance especially beca use students' reading ability is considered the foundation for most acade mic subjects at school. The full model includes one additional school-level variable, student morale. This variable raised the school-level variance 1%, and the total variance explained by the four school-level variables was 67 %. Student morale seemed lower in Catholic schools than in non-Catholic priv ate schools, indicated by its means and correlation (r = 0.24, p < 0.05). When st udent morale was held constant, the negative effect of Catholic schools b ecame insignificant (compare b coefficients and p levels of global and full mode ls). Lower student morale seems to partially explain the negative effect atte nding Catholic schools has on reading achievement, although the coefficient chang e is not impressive. These findings need much further exploration in future st udies.Developing students' achievement in mathematicsTable 3 presents the results of the HLM models in m athematics. Attending a Catholic school vs. a non-Catholic private school m ade no significant difference in developing mathematics scores between eighth and tenth grades. The student model consists of five student characterist ics: gender, students' Catholic religious affiliation, parental SES, stude nts' eighth-grade GPA, and students' eighth-grade math score. These variables explain about 44% of the total student-level variance. Consistent with previ ous studies of public schools, being female was negatively associated with tenth-g rade math scores. Again, eighth-grade math score, initial GPA, and pre-test math score were important predictors for a student's tenth-grade math score. Notably, however, Catholic religious affiliation was a positive predictor for tenth-grade math score, even when students' initial academic and family backgrou nds were statistically controlled. To our knowledge, the relationship betw een students' religious affiliation and their achievement scores has been a ddressed in only one study (Jeynes 1999). Table 3. Development of Students' Achievement in Ma thematics Student ModelGlobal ModelFull Model Independent Variablesbset-ratiobset-ratiobset-ratio Institution-level variables Intercept57.4410.148388.874***56.2420.490114.744*** 55.1060.80268.745*** Global characteristics Enrollment 0.4770.2232.143**0.4870.2212.206**Catholic school -0.5720.406-1.408-0.7210.412-1.752* Mean SES 1.3270.4273.111***1.2300.4272.877***
11 of 28 Internal characteristics Parental involvement 0.3000.1681.783* Individual-level variables Female-0.6440.206-3.118***-0.6130.204-3.008***-0.62 30.204-3.059*** Religion: Catholic0.4330.2341.852*0.8580.2653.233** *0.8580.2653.236*** Parental SES0.6200.1793.470***0.1260.2060.614*0.126 0.2060.612 Initial GPA1.9620.19310.153***2.1120.19310.924***2. 1000.19310.68*** Pretest math0.6670.01350.645***0.6470.01447.814***0 .6480.01447.914***Note: *** p <= .01; ** p<=.05; p<=.10The global model consists of three global school ch aracteristics: enrollment size, mean parental SES, and Catholic school status These three variables explain about 79% of the total school-level varianc e. The sharp drop of the coefficient and significance of parental SES at the individual level occurred when mean parental SES was included at the institut ion level. Holding enrollment size and mean parental SES (as well as i ndividual-level predictors) constant, Catholic school status was an insignifica nt (negative) predictor for math achievement scores. Null hypothesis 2 was not rejected. Concerning the practical significance of the school sector effect, there is a difference of 0.11 (from 0.572/5.056) standard dev iations in students' math achievement scores. The between-school standard dev iation 5.056 was obtained from the unconditional ANOVA model (not sh own in the table; see the previous reading section). That is, non-Catholic pr ivate school students were estimated to score 0.11s standard deviation higher (or a 4 percentile difference) in their math achievement test, on average, than Ca tholic school students. This magnitude in math score does not seem to have great practical importance. The full model includes one more school-level varia ble: the school's efforts in promoting parental support/involvement. It is not s urprising that parental involvement positively affects children's academic development in mathematics, because this subject needs special attention and co ntinuous efforts at home and school. It is, however, notable that the negati ve effect of Catholic schools increased and became significant (p = 0.079) in the full model when school effort in promoting parental involvement was held c onstant. The correlation of Catholic school and parental involvement was positi ve and significant (r = 0.18, p < 0.05). However, future studies should further e xplore the association and causal effects between math achievement, parental i nvolvement, and Catholic school. The full model's four variables explain abo ut 80% of the total school-level variance in tenth-grade math.Developing students' achievement in history/social studiesTable 4 presents the three HLM models for history/s ocial studies. Attending a Catholic school or a non-Catholic private school di d not make a significant difference in developing history/social studies ach ievement between eighth and tenth grades. Again, we found some pattern of repet ition in the studentand school-level variables included. The student model includes six variables: gender, parental SES, overall eighth-grade GPA, eig hth-grade history/social
12 of 28 studies score, elective reading, and eighth-grade s tudents' perception of the usefulness of history/social studies subjects. Thes e six variables explain about 32% of the total student-level variance in tenth-gr ade history/social studies test scores. Students' perception that social studies an d history are useful could lead them to devote more time and energy in these a reas. We included elective reading as a variable because extensive reading bey ond school materials could expand the knowledge base of historical and societa l issues. The negative effect of being female on history/social studies ac hievement was unexpected and noteworthy.The global model includes three school-level variab les: enrollment size, mean parental SES, and Catholic school status. These thr ee variables explain about 63% of the total school-level variance. Holding enr ollment size and mean parental SES constant, we found that attending a Ca tholic school was negatively associated with developing students' his tory/social studies achievement scores. However, the effect of attendin g a Catholic school was insignificant, and null hypothesis 3 was not reject ed. As for the practical significance of the school sec tor effect, there is a difference of 0.09 (from 0.376/4.003) standard deviation in s tudents' history/social studies achievement scores. (The between-school standard de viation 4.003 was obtained from the unconditional ANOVA model.) In ot her words, non-Catholic private school students were estimated to score 0.0 9 standard deviations higher (or about a 4 percentile difference) in their histo ry/social studies achievement test, on average, than Catholic school students. Th is magnitude in history/social studies score does not seem to have great practical importance. Table 4. Development of Students' Achievement in Hi story/social studies Student ModelGlobal ModelFull Model Independent Variablesbset-ratiobset-ratiobset-ratio Institution-level Variables Intercept56.4300.201281.304***54.8700.66183.024***5 5.9820.99356.397*** Global characteristics Enrollment 0.6710.3072.183**0.6930.2982.294**Catholic school -0.3760.492-0.764-0.0520.511-0.101 Mean SES 1.2500.5822.149**0.9710.5991.622 Internal characteristics Teacher student ratios -0.0800.037-2.171**Remedial reading 0.0710.0292.463** Individual-level Variables Female-1.8970.287-6.620***-1.8600.284-6.547***-1.84 70.283-6.538*** Parental SES0.9500.2383.997***0.3900.2831.3750.3870 .2831.369 Initial GPA1.9050.2567.455***1.9900.2557.790***2.00 80.2547.893*** Pretest history0.5820.01832.557***0.5710.01831.898* **0.5720.01832.027*** Social studies useful0.5540.1623.413***0.5420.1623. 348***0.5200.1623.217*** Elective reading0.4210.0914.622***0.4430.0914.875** *0.4360.0914.808***Note: *** p <= .01; ** p<=.05; p<=.10
13 of 28 The full model has two additional school-level vari ables: student-teacher ratio and the status of the school's remedial reading pro gram. Obviously, students' scores in history/social studies are closely relate d to their reading skills. It appears that developing students' reading skills th rough remedial reading programs has multiple impacts on their academic dev elopment. Remedial programs seem to increase achievement in history/so cial studies. Catholic schools are more likely to have remedial programs t han non-Catholic schools (r = 0.22). However, Catholic schools have higher teac her-student ratios than their counterparts (mean of Catholic schools: 23.44, SD = 5.25; mean of non-Catholic schools: 13.94, SD = 5.46), which were found to negatively affect students' development in history/social studies. Wi th the inclusion of these two school characteristics (remedial reading program an d student-faculty ratio), one positively and one negatively related to the outcom e variable, the Catholic school effect became miniscule. The five school-lev el variables explain about 66% of the total school-level variance.Developing students' achievement in scienceTable 5 shows the three HLM models for science. The type of private school attended made no difference in developing students' knowledge in science between eighth and tenth grades. The student model includes five individual student characteristics: female, eighth-grade GPA, eighth-grade science test score, parental SES, and hours spent on homework ea ch week. Being female was the only negative predictor in the model and se ems related to similar findings for mathematics. The positive effect of h ours spent on homework seems to suggest, not surprisingly, that students w ho spend considerable time doing science homework or projects may learn more. Combined, the five variables explain 29% of the total student-level va riance. The global model includes only two school-level var iables, mean eighth-grade science score and Catholic school status. These two variables explain a surprising 66% of the total school-level variance. No other global school characteristic considered (e.g., mean SES, enrollme nt size) had significant predictivity for the dependent variable, controllin g for school mean science test score. The effect of attending a Catholic school wa s insignificant (p = 0.25), and null hypothesis 4 was not rejected.Concerning the practical significance of the school sector effect, there is a difference of 0.11 (from 0.561/5.112) standard dev iations in students' science achievement scores. (The between-school standard de viation 5.112 was obtained from the unconditional ANOVA model.) That is, non-Catholic private school students were estimated to score 0.11 standa rd deviations higher (or a 4 percentile difference) in their science achievement test, on average, than Catholic school students. This magnitude in science score does not seem to have great practical importance.However, students' science knowledge and test score s rise significantly when attending schools that have other students with hig h science scores. Judging by the correlation between mean science score and hour s spent on homework per week (r = 0.22, p < 0.01), students surrounded by p eers with high science
14 of 28 scores may spend more time on homework. No signific ant change occurred in the coefficient of individual eighth-grade science scores, even when the mean score was included at the school level. This sugges ts that the individual score and the school's mean score have independent proper ties or contributions. Table 5. Development of Students' Achievement in Sc ience Independent Variables Student ModelGlobal ModelFull Model bset-ratiobset-ratiobset-ratio Institution-level variables Intercept56.2370.225249.975***42.8542.79315.341***4 3.9063.69011.899*** Global characteristics Catholic school -0.5610.487-1.152-0.5710.474-1.204 Mean pretest science 0.2440.0485.078***0.2260.0 484.743*** Internal characteristics Monitoring academicprogress 0.6830.3711.843* Strict school rules -1.1070.530-2.086** Individual-level variables Female-2.3990.323-7.435***-2.3590.292-8.078***-2. 3570.291-8.100*** Homework hour0.2030.0962.114**0.1370.0931.4730.13 40.0931.450 Parental SES1.4080.2665.301***0.8610.2663.240***0 .8490.2653.207*** Initial GPA2.6470.3188.331***2.8020.25910.809***2 .8010.25910.832*** Pretest science0.5200.2026.518***0.4930.01728.286 ***0.4930.01728.307***Note: *** p <= .01; ** p<=.05; p<=.10The full model consists of two internal school char acteristics in addition to the variables of the global model. School's emphasis o n monitoring students' academic progress was a positive predictor, and s chools with strict rules was a negative predictor for the development of science scores. As shown in Table 1, these internal school characteristics do not dif fer between Catholic and non-Catholic private secondary schools. There was n o significant change in the coefficients of the other variables when these vari ables were added to the HLM model.Throughout the four subject areas, we attempted to observe whether there is any significant cross-level interaction effect, but we found none.Supplemental HLM AnalysesThe models for supplemental HLM analyses were prese nted in Appendices B-1 through B-4. With all the independent variables inc luded in the original HLM analyses, HLM models were created and compared with the original (parsimonious) models. In other words, the suppleme ntal models include all the independent variables chosen for any HLM model of f our subjects, regardless of the variables' unique contribution to a different s ubject matter. To keep all achievement models comparable, reading useful, math useful, social studies useful, and science useful (to capture the impact o f students' perception of utility) as well as mean pre-test reading, mean pre -test math, mean pre-test
15 of 28 social studies and mean pre-test science were added to the corresponding achievement models. Although the coefficients of th e variables that were originally in the HLM models were changed by includ ing both significant and insignificant variables, the statistical significan ce level and signs of the independent variables rarely changed, except for th e statistical significance level of Catholic school.Interestingly, in the supplemental global models, t he negative effect of attending Catholic schools became stronger. In reading achiev ement, this negative effect became stronger, and its t-ratio increased from -1. 684 (p = 0.09) to -1.956 (p = 0.05). This provides a cross-validation of our majo r finding: that Catholic schools tend to produce lower student reading achie vement scores than non-Catholic private schools. In the subject areas in which hypotheses were not rejected, the negative effect of Catholic schools o n science achievement was more visible and became significant (p = 0.098). Ev en in history, the negative effect was more visible and very close to the cutof f point, although we do not reject the null hypothesis in conservative terms (p = 0.104). In short, there were indications that except for mathematics, non-Cathol ic private schools might be more effective in students' academic development th an Catholic schools. Nevertheless, these results should be discussed cau tiously, because the supplemental models tended to be overloaded with bo th significant and insignificant variables.Discussion and ConclusionThis study, because of its unique modeling and the consideration of important studentand school-level variables not included in previous studies, generated new findings in terms of both differences in achiev ement between Catholic and non-Catholic schools and possible explanations for such differences. In this discussion, we will address the major findings of t he study and their potential implications.Reading achievement: A negative effect for Catholic schools compared with non-Catholic schools. A major finding of this study, not found in previou s research, is the negative impact of Catholic school s on growth in reading achievement scores. The differential effect is not only statistically significant but is also practically important because of the impact students' reading comprehension abilities have on other subject matte rs. This was despite the finding that Catholic schools were more likely to h ave remedial reading programs (see Table 1), which presumably would have invested more resources on growth in this content area. At the sa me time, the presence of more remedial reading programs could suggest that m ore students in Catholic schools need this service compared with non-Catholi c schools. The internal characteristics variable, student morale, may not p rovide a definite reason for the negative effect but is suggestive of an area fo r further study. Mathematics achievement: No significant difference between Catholic and non-Catholic schools. This study found that, when controlling for potenti ally confounding factors, Catholic schools do not have a n advantage over other private schools in mathematics. The effect was very small, suggesting little practical significance. Attending a Catholic school or a non-Catholic private
16 of 28 school did not make a significant difference in dev eloping mathematics achievement scores. This result seems to conflict w ith those of other studies that found higher mathematics achievement in Cathol ic schools. However, in many previous studies using mathematics achievement as a dependent variable, Catholic schools were compared with publi c schools, and these studies seldom adjust extensively for potential con founding variables. Our finding about students' mathematics achievement was consistent with Gamoran (1996), although his sample included only urban sch ools. On the student level, the positive effect of being Catholic on mathematics achievement was a surprising finding. This study ca nnot identify whether students affiliated with Catholicism tend to study mathematics more, or whether other characteristics of Catholic students and thei r families contribute to this finding. Using NELS:88 data, Jeynes (1999) studied the effects of religious commitment on Black and Hispanic students' achievem ent in reading, mathematics, social studies, and science. He found that even when SES was included, religiously devout students performed bet ter on all measures. However, attendance at a religious school did not e xplain the results. Further research is needed. In the full model, this finding proved to be partially contingent on a school's efforts toward parental su pport/involvement; therefore, school leaders should be aware of this factor and i ts implications for their practice.History/social studies achievement: No significant differences between Catholic and non-Catholic schools. Attending a Catholic school or a non-Catholic private school did not make a signific ant difference in developing history/social studies achievement scores. The effe ct was very small, indicating little practical importance. On the student level, a surprising finding was the negative effect of being female on achievement in t his subject. Females may be less interested in social studies because most majo r historical actors tend to be male, and social studies textbooks tend to emphasiz e masculine themes, such as wars and national politics. Explaining this find ing is beyond the scope of this study, but educators and researchers should investi gate further. In the full model, student-teacher ratios, school s ize, and remedial reading programs contributed to the model. It was not surpr ising that a lower student-teacher ratio might contribute to students' learning, particularly because between the eighth and tenth grades history/social studies content becomes more complex and conducive to projects entailing cl assroom activities and classroom discussion. However, it was surprising th at a larger enrollment was positively related to achievement in this subject a rea. Perhaps a larger school's capacity to provide more specialized teachers, more curriculum options, and additional research resources in this subject expla ins this difference. Catholic schools were somewhat more likely to have remedial reading programs than non-Catholic schools, which, given the reading-inte nsiveness of history/social studies, may have contributed to achievement in thi s subject area. Science achievement: No significant differences bet ween Catholic and non-Catholic schoolsAttending a Catholic school or a non-Catholic priva te school did not make any
17 of 28 difference in developing science achievement scores The effect was very small, suggesting little practical importance. Howe ver, it is important to note that in our supplemental analysis, the negative effect o f Catholic school was more visible and statistically significant.Being female was the only negative predictor in the student model, which seems related to similar findings for mathematics. This suggests that schools need to work on closing this enduring gender gap. T he positive effect of hours spent on homework indicates, not surprisingly, tha t students who spend considerable time doing science homework or project s learn more. As shown in Table 1, the initial number of hours spent on homew ork is higher among students in non-Catholic private schools, which fro m a social capital perspective would suggest greater support for achievement in th is area among non-Catholic private school parents.Monitoring students' academic progress was a positi ve predictor for growth in science achievement, while strict rules had a detri mental effect. These two variables seem to provide an educational implicatio n: it is important to monitor students' academic progress, yet strict school rule s could be detrimental in developing students' achievement in science. Perhap s, as constructivist theorists (Brooks and Brooks 1993) might claim, sci entific exploration requiring "hands-on" activity is less likely to flourish in a strict school environment.Other findings and implicationsExamining studentand school-level variables can p rovide educators and school administrators with additional insights. All the student models had three predictors in common: subject pre-test, overall eig hth-grade GPA, and parental SES. Not surprisingly, eighth-grade pre-test score was the strongest predictor, and initial GPA, representing overall academic leve l, was the next strongest, for all four outcomes. Even when the effects of initial pre-test score and overall academic achievement were held constant, parental S ES was still a very significant explanatory variable for all four outco mes at both the student and school levels. Students from higher SES backgrounds developed more, regardless of the type of school they attended. Sev eral decades ago, studies in sociology of education found and established the im pact of parental education on students' school success and the generational re production patterns of socio-economic status.Students' elective reading was a significant positi ve predictor of development in both reading and history/social studies. Reading be yond school requirements appears to enhance both reading skills and knowledg e in social studies. In addition, a student's perception of the usefulness of history/social studies subjects was positively associated with history/soc ial studies achievement. Although the utility variable was positively associ ated only with history/social studies, teachers may need to inform their students of the utility of school knowledge in their lives, especially given the chan ging global economy and increasingly competitive society. It is not surpris ing that having more risk factors would detrimentally affect students' academic achie vement. However, future studies could take a closer look at the differences among risk factors in this data set (Horn, Chen and Adelman 1998).
18 of 28 Interestingly, large school enrollment was positive ly associated with three outcomes: reading, math, and history/social studies Although private schools tend to be small, we nevertheless found considerabl e variation in school enrollment size in the data. The data suggest that a moderate size of student enrollment seems to be necessary for student develo pment. This finding would also support the benefits of smaller class sizes, a lthough recent studies (e.g., Hoxby 2000) have called into question class size re duction as a public school reform issue. Connecting the negative effect of a s chool's teacher-student ratio on history/social studies with the positive effect of a large enrollment, we can induce a potentially desirable situation: keep a lo w teacher-student ratio at a moderately large high school.LimitationsFirst, NELS data were not created particularly to c onduct this type of study or to answer the questions that we raised. We acknowledge the potential for omitted variable bias because the necessary variables are s imply unavailable in spite of our efforts to isolate all the possible confounding factors for the school effects. Second, we acknowledge the problems associated with students' non-random selection into schools--a common issue of quasi-exp erimental design. Although the non-random choice issue might not be as serious as in studies in which Catholic schools were compared with public sector s chools, school choices are not random and control variables would not simply a djust all the group differences. Nevertheless, our study attempted to a djust this non-random selection bias through the multi-level research des ign and analysis as well as by controlling for more extensive background character istics than any previous studies examining Catholic school effects had done. Third, some may consider that two years is not a su fficiently long period for examining the Catholic school effect. We considered using the 1992 survey (the second follow-up) for students' outcome variables, but we realized there is too much vagueness and complexity in the data due to st udents' transferring from one sector to another during the four years of seco ndary schooling. Moreover, NELS surveys do not have all the necessary informat ion to trace all of the transfers during the period. We conducted this stud y using the eighth-grade initial survey and the tenth-grade follow-up survey to reduce the vagueness of the findings as well as to maintain a relatively la rge sample size. Fourth, the data from the NELS:88 study can be cons idered somewhat dated, but it is the best available national database for this type of study. Although the organizational characteristics of educational organ izations tend to change slowly, student populations in these two school sec tors may be shifting. In the late 1990s it appeared that Catholic secondary scho ol costs had risen sharply (Harris 2000), and there were signs that the Cathol ic school population was becoming increasingly elite (Baker & Riordan 1998). Successful legal efforts to include religious schools in school choice plans se em to favor growth in urban Catholic schools with low-income student population s. However, choice plans also favor the opening of a wider variety of non-Ca tholic private schools. This might change the demographic profiles of the two sc hool sectors in future large
19 of 28 database studies.ConclusionOur study provides education policymakers and the p ublic with new insights to consider when making decisions about relative schoo l effectiveness and allocation of resources to the private sector. We d iscovered that Catholic school students scored significantly lower than non-Cathol ic private school students in reading. Non-Catholic private schools were more eff ective in developing students' reading achievement from eighth grade to tenth grade than Catholic private schools. This finding was consistent in the main (parsimonious) and supplemental models. On the other hand, using the m ain HLM models, we found that attending a Catholic school does not mak e a significantly different impact on academic development in math, history/soc ial studies, and science. The supplemental models, however, suggested that th e effectiveness of Catholic schools could be worse than neutral. There were indications that except for mathematics, non-Catholic private school s might be more effective and beneficial than Catholic schools in developing academic abilities in the subject areas investigated. Most previous studies f inding a positive Catholic school effect were based on comparisons with publi c schools and often focused on a single subject, mathematics. Our resul ts suggest, at the very least, that no claims should be made about the distinctive advantages of Catholic schools in academic achievement.Finally, we hope that future studies can make the d iscussion of Catholic school effectiveness more comprehensive by comparing publi c schools, Catholic schools, and non-Catholic private schools in the sa me multi-level research design. There is also a need for studies that compa re Catholic schools with other religious schools. Coleman et al. (1982a) war ned that research findings do not lead in any simple way to policy recommendat ions, and Witte (1992) issued a similar caution about basing school choice policy on comparisons of achievement across school categories. Comparison of school effectiveness will continue to be a volatile and important area of res earch not only because of its educational implications for student development, b ut also because of its policy implications.ReferencesAlexander, K. L. and Pallas, A. M. (1985). School s ector and cognitive performance: When is a little a little? Sociology of Education, 58 (2), 115-128. Baker, D. P., & Riordan, C. (1998). The "eliting" o f the common American Catholic school and the national education crisis. Phi Delta Kappan, 80 (1), 16-23. Brooks, J.G. & Brooks, M.G. (1993). In search of understanding: The case for constructi vist classrooms. Alexandria VA: Association for Supervision and Cur riculum Development. Borg, W. R. and Gall, M. D. (1989). Educational Res earch: An Introduction (5th edition), White Plains, NY: Longman. Bryk, A. S., Lee, V. E., & Holland, P. B. (1993). Catholic Schools and the Common Good. Cambridge MA: Harvard University Press. Bryk, A. S., and Raudenbush, S. W. (1992). Hierarch ical linear models: Applications and data
20 of 28 analysis methods. Newbury Park, CA: Sage. Burstein, L. (1980). The analysis of multi-level da ta in educational research and evaluation. Review of Research in Education 8 158-233. Bushweller, K. (1997). Working miracles: Catholic s chools' success formula. American School Board Journal, 184 (1), 14-19. Chubb, J. E. and Moe, T. M. (1988). Politics, marke ts and the organization of schools. American Political Science Review, 82, 1065-1087. Chubb, J. E., and Moe, T. M. (1990). Politics, Markets and America's Schools Washington DC: Brookings Institute. In Richard Arum and Irenee R. Beattie (2000, reprint), The Structure of Schooling 474-488. Mountain View: Mayfield Publishing Compa ny. Coleman, J. S., Hoffer, T., and Kilgore, S. (1982b) High school achievement: Public, Catholic and private schools compared. New York: Basic Books. Coleman, J. S., Hoffer, T., and Kilgore, S. (1982a) Cognitive outcomes in public and private schools. Sociology of Education, 55 (2-3), 65-76. Coleman, J. S., and Hoffer, T. (1987). Public and Private High Schools: The Impact of Comm unities New York: Basic Books. Gamoran, A. (1992). The variable effects of high sc hool tracking. American Sociological Review, 57 (6), 812-828. Gamoran, A. (1996). Student achievement in public m agnet, public comprehensive, and private city high schools. Educational Evaluation and Policy Analysis, 18 (1), 1-18. Graetz, B. (1990). Private schools and educational attainment: Cohort and generational effects. Australian Journal of Education, 34 (2), 174-191. Hallinan, M. T. (2000). Conclusion: Catholic educat ion at the crossroads. In J. Youniss & J. J. Convey (Eds.), Catholic Schools at the Crossroads (pp. 201-220). New York: Teachers College Press. Harris, J. C. (2000). The funding dilemma facing Ca tholic elementary and secondary school teachers. In J. Youniss & J. J. Convey (Eds.), Catholic Schools at the Crossroads (pp. 55-71). New York: Teachers College Press. Hoffer, T. B. (2000). Catholic school attendance an d student achievement: A review and extension of research. In J. Youniss & J. J. Convey (Eds.), Catholic Schools at the Crossroads (pp. 87-112). New York: Teachers College Press. Hoffer, T., Greeley, A. M., & Coleman, J. S. (1985) Achievement growth in public and Catholic schools. Sociology of Education, 58 (2), 74-97. Hopkins, K. D., Hopkins, B. R., and Glass, G. V. (1 996). Basic statistics for the behavioral sciences (3rd edition) Needham Height, MA: A Simon & Schuster Company. Horn, L. J., Chen, X., & Adelman, C. (1998). Toward resiliency: At-risk students who make it to college [Report]. Washington DC: U.S. Department of Educat ion, Office of Educational Research and Improvement. Hoxby, C. M. (2000). The effects of class size on s tudent achievement: New evidence from population variation. Quarterly Journal of Economics, 115 (4), 1239-1285. Hudolin-Gabin, J. (1994). Lessons from Catholic sch ools: Promoting quality in Chicago's public schools. Educational Forum, 58 (3), 282-288. Jencks, C. (1985). How much do high school students learn? Sociology of Education, 58 (2), 128-135. Jensen, G. F. (1986). Explaining differences in aca demic behavior between public school and Catholic school students. Sociology of Education, 59 (1), 32-41.
21 of 28 Jeynes, W. H. (1999). The effects of religious comm itment on the academic achievement of Black and Hispanic children. Urban Education, 34 (4), 458-479. Keith, T. Z. (1985). Do Catholic high schools impro ve minority school achievement? American Educational Research Journal, 22 (3), 337-349. Kim, M. (1995). Organizational effectiveness of women-only colleges : The impact of college environment on students' intellectual and ethical d evelopment. Unpublished doctoral dissertation, University of California, Los Angeles UMI number: 9610531. Kirk, R. E. (1996). Practical significance: A conce pt whose time has come. Educational and Psychological Measurement 56 (5), 746-759. Kreft, I., and de Leeuw, J. (1998). Introducing mul tilevel modeling. Thousand Oaks, CA: Sage. Lee, V. E., Chow-Hoy, T. K., Burkam, D., Geverdt, D ., & Smerdon, B. A. (1998). Sector differences in high school course taking: A private school or Cath olic school effect? Sociology of Education, 71 (4), 314-335. LePore, P. C., & Warren, J. R. (1997). A comparison of single-sex and coeducational Catholic secondary schooling: Evidence from the National Edu cational Longitudinal Study of 1988. American Educational Research Journal, 34 (3), 485-511. Marsh, H. W. (1991). Public, Catholic single-sex, a nd Catholic coeducational high schools: Their effects on achievement, affect and behaviors. American Journal of Education, 99 (3), 320-356. Marsh, H. W., & Grayson, D. (1990). Public/Catholic differences in the High School and Beyond data: A multigroup structural equation modeling app roach to testing mean differences. Journal of Educational Statistics, 15 (3), 199-235. Noell, J. (1982). Public and Catholic schools: A re analysis of "public and private schools." Sociology of Education, 55 (2-3), 123-132. Ornstein, A. C. (1989). Private and public school c omparisons. Education and Urban Society, 21 (2), 192-206. Raudenbush, S. W. and Bryk, A. S. (2002). Hierarchical Linear Models: Applications and Data Analysis Methods (Second Edition). Newbury Park, CA: Sage. Riordan, C. (1985). Public and Catholic schooling: The effects of gender context policy. American Journal of Education, 93 (4), 518-540. Rumberger, R. W. (1995). Dropping out of middle sch ool: A multilevel analysis of students and schools. American Educational Research Journal, 32(3), 583-622. Sander, W. (1996). Catholic grade schools and acade mic achievement. Journal of Human Resources, 31 (3), 540-548. Willms, J. D. (1985). Catholic school effects on ac ademic achievement: New evidence from the High School and Beyond Follow-up Study. Sociology of Education, 58 (2), 98-114. Witte, J. F. (1992). Private versus public school a chievement: Are there findings that should affect the educational choice debate? Economics of Education Review, 11 (4), 371-394.About the AuthorsMikyong Minsun KimDepartment of Educational Leadership and Policy Ana lysis Hill HallUniversity of Missouri-ColumbiaColumbia MO 65211
22 of 28 Mikyong Minsun Kim is an assistant professor in the Department of Educational Leadership and Policy Analysis at the U niversity of Missouri, Columbia. Kim's areas of specialization include edu cation policy, assessment and equity issues in education, college and school impact, organizational analysis, and quantitative research methods.Margaret PlacierDepartment of Educational Leadership and Policy Ana lysis Hill HallUniversity of Missouri-ColumbiaColumbia MO 65211Email: email@example.com Margaret Placier is an associate professor in the Department of Edu cational Leadership and Policy Analysis at the University of Missouri, Columbia. Placier's areas of specialization include education policy, s ociology of education, teacher education, and qualitative research methods. Appendix A. Variables and Coding SchemesVariablesNELS:88 source variables Coding scheme Institution-level variables EnrollmentG8ENROL1='1-49' students, 2='50-99,' 3='1 00-199,' 4='200-299,' 5='300-399,' 6='400+'. Catholic schoolG8CTRLRecoded, 1=Catholic school, 0= non-Catholic school. Mean SESBYSESAggregated, composite variable.Student moraleF1C93GAggregated, continuous scale.Promoting parentalsupport/involvement F1C91EAggregated, continuous scale Teacher-student ratiosBYRATIOContinuous scale Remedial readingF1C30BPercentage of students receiv ing remedial reading Mean pretest scienceBY2XSSTDAggregated, continuous scale. Monitoring academic progress F1C91HRange: 2-5; 3=minor emphasis, 5=major emphasi s Strict school rulesF1S7CRecoded. From 1=strongly di sagree to 4=strongly agree. (Variables excluded in the original institution-lev el models) School minority proportion G8MINOR0=none, 1=1-5%, 2=6-10%, 3=11-20%, 4=21-40%, 5=41-60%, 6=61-90%, 7=91-100% Mean pretest readingBY2XRSTDAggregated score, conti nuous scale Mean pretest mathBY2XMSTDAggregated score, continuo us scale Mean pretest history/social studies BY2XHSTDAggregated score, continuous scale Mean pretest scienceBY2XSSTDAggregated score, conti nuous scale Urban locationG8URBANRecoded,1=urban school, 0=nonurban school Suburban locationG8URBANRecoded, 1=suburban school, 0=non-suburban school. Rural schoolG8URBANRecoded, 1=rural school, 0=non-r ural school. Teacher moraleF1C93FAggregated, continuous scale.
23 of 28 Remedial mathF1C30CPercentage of students receiving remedial math, continuous scale. Individual-level variables PretestsReadingBY2XRSTDReading standardized score taken dur ing 8th grade, continuous scale. MathBY2XMSTDMath standardized score taken during 8t h grade, continuous scale. History/social studiesBY2XHSTDHistory/social studie s standardized score taken during 8th grade, continuous scale ScienceBY2XSSTDScience standardized score taken dur ing 8th grade, continuous scale. PosttestsReadingF12XRSTDReading standardized score taken dur ing 10th grade, continuous scale. MathF12XMSTDMath standardized score taken during 10 th grade, continuous scale. HistoryF12XHSTDHistory/social studies standardized score taken during 10 grade, continuous scale. ScienceF12XSSTDScience standardized score taken dur ing 10th grade, continuous scale. Initial GPABYGRADSGrades composite (averaged and we ighted self-reported grades, from A to D, across foursubjects--reading, math, history/social studies, an d science) Risk factorsBYRISKThe number of risk factors, range from 0 (no risk) to 6 (6 risk factors) Elective solitary readingBYS800=none, 1=1 hour or l ess per week 2=2 hours, 3=3 hours, 4=4-5 hours, 5=6 hours or more per week. FemaleSEX1=male,2=femaleReligion: CatholicF1S81Recoded, 1=Catholic, 0=non-C atholic Parental SESBYSESComposite scoreSocial studies are usefulBYS71CRecoded, 1=strongly disagree, 2=disagree, 3=agree, 4=strongly agree Homework hoursBYHOMEWKThe number of hours spent on homework per week. From 1=none to 8=21 and up hours (Variables excluded in the original individual-leve l models) BlackRACERecoded, 1=Non-black, 2=BlackHispanicRACERecoded, 1=non-Hispanic, 2=HispanicAsian-PacificRACERecoded, 1=non-Asian Pacific, 2=As ian Pacific. English is useful.BYS70CRecoded, 1=strongly disagre e, 2=disagree, 3=agree, 4=strongly agree Math is useful.BYS69CRecoded, 1=strongly disagree, 2=disagree, 3=agree, 4=strongly agree Science is useful. BYS72CRecoded, 1=strongly disagr ee, 2=disagree, 3=agree, 4=strongly agree Parental EducationBYPAREDFrom 1=didn't finish high school to 6=Ph.D., M.D.Appendix B-1 Development of Students' Achievement i n Reading
24 of 28 Independent Variables Global ModelFull Model bset-ratiobset-ratio Institution-level variablesIntercept51.4492.85218.040***50.9703.91513.020***Global characteristics Enrollment0.7710.2822.733***0.7480.2652.825*** Catholic school -1.1330.579-1.956**-1.2030.620-1.939* Mean SES0.7480.6481.154 0.5840.6600.885 Mean pretest reading0.0780.0511.538 0.0730.0521.4 05 Internal characteristics Teacher student ratio -0.0100.038-0.274 Remedial reading 0.0130.0300.443 Parental involvement 0.3610.2281.583 Monitoring academic progress -0.3940.291-1.355 Student morale 0.5980.2802.134** Strict school rule -0.3560.443-0.804 Individual-level variables Female-0.1380.288-0.480 -0.1580.289-0.548 Religion: Catholic0.5150.4101.258 0.5320.4071.306 Homework hour-0.0000.085-0.004 -0.0110.084-0.128 Parental SES0.4080.2661.534 0.4090.2661.538 Initial GPA2.0640.2558.078***2.0300.2577.908*** Pretest reading0.5250.02124.452***0.5360.02124.55 9*** Risk factors-0.4250.253-1.680*-0.4340.251-1.726* Reading useful0.0660.1560.425 0.0690.1550.448 Elective reading0.3910.0904.352***0.4020.0894.509 ***Note: *** p <= .01; ** p<=.05; p<=.10Appendix B-2. Development of Students' Achievement in MathematicsIndependent Variables Global ModelFull Model bset-ratiobset-ratio Institution-level variablesIntercept55.7382.10426.487***55.0563.04118.104***Global characteristicsEnrollment0.4900.1852.650***0.5110.1902.693***Catholic school -0.5550.415-1.336 -0.6120.452-1.356 Mean SES1.2640.5152.456**1.0830.5052.146**Mean pretest math0.0090.0390.230 0.0170.0420.406 Internal characteristicsTeacher student ratio -0.0250.026-0.979 Remedial reading 0.0340.0331.045 Parental involvement 0.3280.1761.871*Monitoring academic progress -0.1280.273-0.467 Student morale -0.0600.210-0.286 Strict school rule 0.0360.3900.093
25 of 28 Individual-level variablesFemale-0.5840.222-2.631***-0.5990.223-2.692***Religion: Catholic0.8580.2992.870***0.8550.3002.870 *** Homework hour0.0080.0640.124 -0.0010.064-0.022 Parental SES0.1140.2060.552 0.1110.2060.540 Initial GPA2.0830.2159.682***2.0820.2139.785***Pretest math0.6430.01738.699***0.6430.01738.723***Risk factors-0.2220.182-1.224 -0.2310.182-1.270 Math useful0.1590.1441.102 0.1560.1431.093 Elective reading0.0130.0550.232 0.0170.0560.306 Note: *** p <= .01; ** p<=.05; p<=.10Appendix B-3. Development of Students' Achievement in History/social studiesIndependent Variables Global ModelFull Modelbset-ratiobset-ratio Institution-level variablesIntercept56.0983.04518.420***58.3053.88615.003***Global characteristicsEnrollment0.6800.3451.969**0.6810.3342.039**Catholic school -0.9540.587-1.625 -0.5990.637-0.940 Mean SES1.3100.6961.883*1.0340.7081.459 Mean pretest social studies-0.0180.059-0.298 -0.015 0.056-0.264 Internal characteristicsTeacher student ratio -0.0820.036-2.295**Remedial reading 0.0700.0302.335**Parental involvement -0.0100.234-0.043 Monitoring academic progress -0.1390.349-0.399 Student morale 0.0090.2900.030 Strict school rule -0.1870.459-0.407 Individual-level variablesFemale-1.8360.296-6.197***-1.8220.295-6.185***Religion: Catholic0.8240.3732.210**0.8090.2742.164* Homework hour-0.0130.094-0.141 -0.0270.093-0.292 Parental SES0.3370.2771.213 0.3290.2761.191 Initial GPA1.9610.2886.819***1.9830.2866.934***Pretest social studies0.5710.02225.936***0.5710.022 25.958*** Risk factors-0.2610.285-0.915 -0.2930.284-1.029 Social studies useful0.5350.1623.302***0.5230.1613. 191*** Elective reading0.4480.1024.403***0.4440.1024.325** *Note: *** p <= .01; ** p<=.05; p<=.10Appendix B-4. Development of Students' Achievement in Science
26 of 28 Independent Variables Global ModelFull Modelbset-ratiobset-ratio Institution-level variablesIntercept43.4583.43112.667***46.3994.31510.752***Global characteristicsEnrollment0.4370.2891.510 0.3770.2811.340 Catholic school -0.9700.586-1.654*-0.7940.602-1.319 Mean SES0.1660.7900.210 -0.1070.777-0.138 Mean pretest science0.2230.0663.365***0.1870.0642.9 09*** Internal characteristicsTeacher student ratio -0.0540.041-1.308 Remedial reading -0.0130.034-0.393 Parental involvement 0.1680.2710.618 Monitoring academic progress 0.6850.3022.268**Student morale -0.0020.322-0.005 Strict school rule -1.2490.625-1.999**Individual-level variablesFemale-2.3500.316-7.433***-2.3410.314-7.464***Religion: Catholic0.7470.3302.264**0.7520.3272.300* Homework hour0.1150.0991.161 0.1040.1001.040 Parental SES0.7760.2942.641***0.7660.2932.611***Initial GPA2.7050.3078.806***2.7070.3048.902***Pretest science0.4820.02123.346***0.4820.02123.418* ** Risk factors-0.1770.281-0.632 -0.2190.281-0.777 Science useful0.1930.1471.315 0.2170.1511.441 Elective reading0.1940.0952.042**0.1990.0962.069 Note: *** p <= .01; ** p<=.05; p<=.10The World Wide Web address for the Education Policy Analysis Archives is epaa.asu.edu Editor: Gene V Glass, Arizona State UniversityProduction Assistant: Chris Murrell, Arizona State University General questions about appropriateness of topics o r particular articles may be addressed to the Editor, Gene V Glass, firstname.lastname@example.org or reach him at College of Education, Arizona State Un iversity, Tempe, AZ 85287-2411. The Commentary Editor is Casey D. Cobb: email@example.com .EPAA Editorial Board Michael W. Apple University of Wisconsin David C. Berliner Arizona State University
27 of 28 Greg Camilli Rutgers University Linda Darling-Hammond Stanford University Sherman Dorn University of South Florida Mark E. Fetler California Commission on TeacherCredentialing Gustavo E. Fischman Arizona State Univeristy Richard Garlikov Birmingham, Alabama Thomas F. Green Syracuse University Aimee Howley Ohio University Craig B. Howley Appalachia Educational Laboratory William Hunter University of Ontario Institute ofTechnology Patricia Fey Jarvis Seattle, Washington Daniel Kalls Ume University Benjamin Levin University of Manitoba Thomas Mauhs-Pugh Green Mountain College Les McLean University of Toronto Heinrich Mintrop University of California, Los Angeles Michele Moses Arizona State University Gary Orfield Harvard University Anthony G. Rud Jr. Purdue University Jay Paredes Scribner University of Missouri Michael Scriven University of Auckland Lorrie A. Shepard University of Colorado, Boulder Robert E. Stake University of IllinoisUC Kevin Welner University of Colorado, Boulder Terrence G. Wiley Arizona State University John Willinsky University of British ColumbiaEPAA Spanish and Portuguese Language Editorial BoardAssociate Editors for Spanish & Portuguese Gustavo E. Fischman Arizona State Universityfischman@asu.eduPablo Gentili Laboratrio de Polticas Pblicas Universidade do Estado do Rio de Janeiro firstname.lastname@example.orgFounding Associate Editor for Spanish Language (199 8-2003) Roberto Rodrguez Gmez
28 of 28 Universidad Nacional Autnoma de Mxico Adrin Acosta (Mxico) Universidad de Guadalajaraadrianacosta@compuserve.com J. Flix Angulo Rasco (Spain) Universidad de Cdizfelix.email@example.com Teresa Bracho (Mxico) Centro de Investigacin y DocenciaEconmica-CIDEbracho dis1.cide.mx Alejandro Canales (Mxico) Universidad Nacional Autnoma deMxicocanalesa@servidor.unam.mx Ursula Casanova (U.S.A.) Arizona State Universitycasanova@asu.edu Jos Contreras Domingo Universitat de Barcelona Jose.Contreras@doe.d5.ub.es Erwin Epstein (U.S.A.) Loyola University of ChicagoEepstein@luc.edu Josu Gonzlez (U.S.A.) Arizona State Universityjosue@asu.edu Rollin Kent (Mxico) Universidad Autnoma de Puebla firstname.lastname@example.org Mara Beatriz Luce (Brazil) Universidad Federal de Rio Grande do Sul-UFRGSlucemb@orion.ufrgs.br Javier Mendoza Rojas (Mxico)Universidad Nacional Autnoma deMxicojaviermr@servidor.unam.mx Marcela Mollis (Argentina)Universidad de Buenos Airesmmollis@filo.uba.ar Humberto Muoz Garca (Mxico) Universidad Nacional Autnoma deMxicohumberto@servidor.unam.mx Angel Ignacio Prez Gmez (Spain)Universidad de Mlagaaiperez@uma.es DanielSchugurensky (Argentina-Canad) OISE/UT, Canadadschugurensky@oise.utoronto.ca Simon Schwartzman (Brazil) American Institutes forResesarchBrazil (AIRBrasil) email@example.com Jurjo Torres Santom (Spain) Universidad de A Coruajurjo@udc.es Carlos Alberto Torres (U.S.A.) University of California, Los Angelestorres@gseisucla.edu EPAA is published by the Education Policy Studies Laboratory, Arizona State University