USF Libraries
USF Digital Collections

Detecting publication bias in random effects meta-analysis

MISSING IMAGE

Material Information

Title:
Detecting publication bias in random effects meta-analysis an empirical comparison of statistical methods
Physical Description:
Book
Language:
English
Creator:
Rendina-Gobioff, Gianna
Publisher:
University of South Florida
Place of Publication:
Tampa, Fla
Publication Date:

Subjects

Subjects / Keywords:
Begg
Egger
File drawer
Funnel plot
Research synthesis
Trim and fill
Dissertations, Academic -- Measurement and Evaluation -- Doctoral -- USF
Genre:
bibliography   ( marcgt )
theses   ( marcgt )
non-fiction   ( marcgt )

Notes

Abstract:
ABSTRACT: Publication bias is one threat to validity that researchers conducting meta-analysis studies confront. Two primary goals of this research were to examine the degree to which publication bias impacts the results of a random effects meta-analysis and to investigate the performance of five statistical methods for detecting publication bias in random effects meta-analysis. Specifically, the difference between the population effect size and the estimated meta-analysis effect size, as well as the difference between the population effect size variance and the meta-analysis effect size variance, provided an indication of the impact of publication bias. In addition, the performance of five statistical methods for detecting publication bias (Begg Rank Correlation with sample size, Begg Rank Correlation with variance, Egger Regression, Funnel Plot Regression, and Trim and Fill) were estimated with Type I error rates and statistical power. The overall findings indicate that publication bias notably impacts the meta-analysis effect size and variance estimates. Poor Type I error control was exhibited in many conditions by most of the statistical methods. Even when Type I error rates were adequate the power was small, even with larger samples and greater numbers of studies in the meta-analysis.
Thesis:
Dissertation (Ph.D.)--University of South Florida, 2006.
Bibliography:
Includes bibliographical references.
System Details:
System requirements: World Wide Web browser and PDF reader.
System Details:
Mode of access: World Wide Web.
Statement of Responsibility:
by Gianna Rendina Gobioff.
General Note:
Title from PDF of title page.
General Note:
Document formatted into pages; contains 238 pages.
General Note:
Includes vita.

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
aleph - 001790286
oclc - 144585497
usfldc doi - E14-SFE0001494
usfldc handle - e14.1494
System ID:
SFS0025813:00001


This item is only available as the following downloads:


Full Text
xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam Ka
controlfield tag 001 001790286
003 fts
005 20070615143900.0
006 m||||e|||d||||||||
007 cr mnu|||uuuuu
008 070615s2006 flu sbm 000 0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0001494
040
FHM
c FHM
035
(OCoLC)144585497
049
FHMM
090
LB3051 (ONLINE)
1 100
Rendina-Gobioff, Gianna.
0 245
Detecting publication bias in random effects meta-analysis :
b an empirical comparison of statistical methods
h [electronic resource] /
by Gianna Rendina Gobioff.
260
[Tampa, Fla] :
University of South Florida,
2006.
3 520
ABSTRACT: Publication bias is one threat to validity that researchers conducting meta-analysis studies confront. Two primary goals of this research were to examine the degree to which publication bias impacts the results of a random effects meta-analysis and to investigate the performance of five statistical methods for detecting publication bias in random effects meta-analysis. Specifically, the difference between the population effect size and the estimated meta-analysis effect size, as well as the difference between the population effect size variance and the meta-analysis effect size variance, provided an indication of the impact of publication bias. In addition, the performance of five statistical methods for detecting publication bias (Begg Rank Correlation with sample size, Begg Rank Correlation with variance, Egger Regression, Funnel Plot Regression, and Trim and Fill) were estimated with Type I error rates and statistical power. The overall findings indicate that publication bias notably impacts the meta-analysis effect size and variance estimates. Poor Type I error control was exhibited in many conditions by most of the statistical methods. Even when Type I error rates were adequate the power was small, even with larger samples and greater numbers of studies in the meta-analysis.
502
Dissertation (Ph.D.)--University of South Florida, 2006.
504
Includes bibliographical references.
516
Text (Electronic dissertation) in PDF format.
538
System requirements: World Wide Web browser and PDF reader.
Mode of access: World Wide Web.
500
Title from PDF of title page.
Document formatted into pages; contains 238 pages.
Includes vita.
590
Adviser: Jeffrey D. Kromrey, Ph.D.
653
Begg.
Egger.
File drawer.
Funnel plot.
Research synthesis.
Trim and fill.
690
Dissertations, Academic
z USF
x Measurement and Evaluation
Doctoral.
773
t USF Electronic Theses and Dissertations.
4 856
u http://digital.lib.usf.edu/?e14.1494



PAGE 1

Detecting Publication Bias in Random Effects Meta-Analysis: An Empirical Comparison of Statistical Methods by Gianna Rendina-Gobioff A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy Department of Educational Measurement and Evaluation College of Education University of South Florida Major Professor: Jeffrey D. Kromrey, Ph.D. Robert F. Dedrick, Ph.D. John M. Ferron, Ph.D. Vicky Phares, Ph.D. Date of Approval: March 20, 2006 Keywords: Begg, Egger, File Drawer, Funnel Plot, Research Synthesis, Trim and Fill Copyright 2006, Gianna Rendina-Gobioff

PAGE 2

DEDICATION This dissertation is dedicated to my fa mily for all the undocumented little things they did to make this possible. Most notable is my husband, Neil, who knew my desire to seek a doctorate since before the day we took our vows and still married me. Although I think he had an idea of the craziness that a ccompanies this process he certainly had to bear the good and bad. This dream came to fruition because Neil let me fly my kite by his side and helped keep the lines from tangli ng. For the last two year s of my program my son Isidore has kept me motivated and gr ounded. Izzy was my little secret during qualifying exams and a very obvious expectatio n during my proposal defense. Since his birth Izzy’s smile has given me the fuel I needed to keep on track. As a family I am reminded daily of the importance of our time t ogether and as I transition into a new phase of my academic life I keep this privilege at the forefront. All my parents (Mom, Jay, Dad, Lois, Sharon, and Bruce) have encouraged, supported, and loved me throughout my pursuit. As for my brothers and sister-in-laws, I’ve always been in awe of th eir artistic talents, brilliant minds, and witty humor. I don’t think I would be as motivated to continually improve myself if I wasn’t always trying to keep up with their talent and genius.

PAGE 3

ACKNOWLEDGEMENTS As I have progressed through the adventur e that we call docto ral preparation and dissertation completion I’ve been surr ounded by friends and colleagues who have enlightened, motivated, and mentored me along the way. It is an understatement when I say that my doctoral committee was phenomenal Jeffrey Kromrey, my mentor and major professor, has provided me with just th e right amount of guidance, keeping me challenged but not frustrated. John Ferron is a model of balancing academic life and family life, I only hope I can be as produc tive academically and successful with my family. Bob Dedrick is one of the most caring people that I have met and he extends this empathy in his interactions with his student s, reminding us to take care of ourselves. Vicky Phares was my first academic female role model; she showed me that you can be an accomplished, respected, and compassionate researcher. In addition to my doctoral committee the whole Department of Measurement and Evaluation has influenced the completion of this degree; from our office manager (Lisa Adkins) to the rest of the faculty memb ers (Lou Carey, Constance Hines, and Tony Onwuegbuzie). In addition, Cynthia Parshall a former faculty member, also provided invaluable career and academic guidance. Several students and graduates were and continue to be mentors, a dvisors, and friends; Christine Harmes, Melinda Hess, Kris Hogarty, Peggy Jones, Tom Lang, Ha Phan, Jeanine Ramano, Heather Scott, Freda Watson, and Hesborn Wao.

PAGE 4

i TABLE OF CONTENTS LIST OF TABLES.............................................................................................................iv LIST OF FIGURES...........................................................................................................vi ABSTRACT.....................................................................................................................vi ii CHAPTER ONE: INTRODUCTION..................................................................................1 Background..................................................................................................................... .1 Statement of Problem.......................................................................................................3 Purpose........................................................................................................................ .....4 Research Questions..........................................................................................................6 Hypotheses..................................................................................................................... ..8 Procedures..................................................................................................................... .12 Limitations.................................................................................................................... .13 Importance of Study.......................................................................................................14 Definitions.................................................................................................................... .15 CHAPTER TWO: LITERATURE REVIEW....................................................................18 Meta-Analysis................................................................................................................18 Procedures..................................................................................................................20 Analyses.....................................................................................................................21 Weights..................................................................................................................22 Combining Effect Sizes and Bu ilding Confidence Intervals.................................24 Homogeneity of Effect Sizes.................................................................................25 Meta-Analysis Models...............................................................................................27 Fixed Effects Model...............................................................................................27 Random Effects Model..........................................................................................28 Fixed Effects vs. Random Effects Models.............................................................31 Alternative Methods: Rosenthal and Rubin and Hunter and Schmidt.......................32 Threats to Validity.....................................................................................................34 Limitations.................................................................................................................35 Publication Bias.............................................................................................................36 Non Statistical Methods for Detecting Publication Bias.........................................40 Statistical Methods for Detecting Publication Bias...................................................42 Begg Rank Correlation Method.............................................................................44 Egger Regression Method......................................................................................46 Funnel Plot Regression Method.............................................................................48

PAGE 5

ii Trim and Fill Method.............................................................................................49 Statistical Methods for Detecting Public ation Bias: Prevalence and Empirical Evidence..............................................................................................................51 Prevalence..............................................................................................................51 Empirical Evidence................................................................................................53 Publication Bias Detected: Options for Researchers.................................................56 Decreasing Publication Bias : A Broad Perspective...................................................57 CHAPTER THREE: METHOD........................................................................................59 Purpose........................................................................................................................ ...59 Research Questions........................................................................................................60 Sample......................................................................................................................... ..62 Primary Studies..........................................................................................................62 Meta-Analyses...........................................................................................................63 Procedures..................................................................................................................... .63 Primary Study Generation..........................................................................................65 Primary Studies: Number.......................................................................................65 Primary Studies: Sample Sizes of the Two Groups...............................................65 Primary Studies: Group Variances.........................................................................66 Population Effect Size: Magnitude........................................................................66 Population Effect Size: Variance...........................................................................66 Selection of Primary Studies: Ma gnitude of Publication Bias...................................67 Publication Bias Tests Applied..................................................................................68 Programming..............................................................................................................70 Data Analysis.................................................................................................................7 0 Research Question 1: Evaluation of the Impact of Publication Bias.........................70 Research Question 2 and 3: Evaluation of Methods to Detect Publication Bias.....................................................................................................................71 CHAPTER FOUR: RESULTS..........................................................................................72 Research Questions........................................................................................................72 Impact of Publication Bias.............................................................................................75 Number of Primary Studies Moderator......................................................................82 Sample Size Moderator..............................................................................................83 Group Variance Moderator........................................................................................83 Magnitude of Population Effect Size Moderator.......................................................84 Variance of Population E ffect Size Moderator..........................................................84 Summary....................................................................................................................84 Type I Error Rates of Methods to Detect Publication Bias...........................................85 Number of Primar y Studies Impact...........................................................................89 Sample Size Impact....................................................................................................91 Group Variance Impact..............................................................................................93 Magnitude of Population Effect Size Impact.............................................................95 Variance of Population Effect Size Impact................................................................97 Summary..................................................................................................................100

PAGE 6

iii Power Estimates of Methods to Detect Publication Bias............................................101 Number of Primar y Studies Impact.........................................................................110 Sample Size Impact..................................................................................................111 Group Variance Impact............................................................................................112 Magnitude of Population Effect Size Impact...........................................................113 Variance of Population Effect Size Impact..............................................................114 Summary..................................................................................................................115 CHAPTER FIVE: CONCLUSIONS...............................................................................116 Summary of the Study.................................................................................................116 Research Questions......................................................................................................120 Summary of Study Results..........................................................................................122 Impact of Publication Bias.......................................................................................123 Type I Error Rates of Methods to Detect Publication Bias.....................................125 Power Estimates of Methods to Detect Publication Bias.........................................127 Discussion....................................................................................................................1 29 Limitations...................................................................................................................1 34 Implications.................................................................................................................13 5 Importance of Study.................................................................................................135 Researchers in General............................................................................................136 Researchers Conducting Meta-Analyses.................................................................137 Researchers of Statistical Methods for Detecting Publication Bias.........................137 Suggestions for Future Research.................................................................................138 REFERENCES................................................................................................................140 APPENDICES.................................................................................................................145 Appendix A: Code for Monte Carlo simu lation and calculating publication bias detection methods................................................................................................146 Appendix B: Mean effect size and e ffect size variance bias estimates.......................167 Appendix C: Type I error rate estimates......................................................................169 Appendix D: Power estimates for conditions with adequate Type I error rates..........194 ABOUT THE AUTHOR.......................................................................................End Page

PAGE 7

iv LIST OF TABLES Table 1. Simplified description of the impact of the relationship among the variance and effect size observed in a study on the likelihood of publication........................................................................................................38 Table 2. Overview of methods fo r detecting publication bias...........................................43 Table 3. Formula variables and their values for a hypothetical sample of 10 effect sizes..................................................................................................................44 Table 4. Frequency of artic les citing the four statis tical methods to detect publication bias by article type and journal type.............................................53 Table 5. Controlled primary study mean sample sizes for each group..............................66 Table 6. Proportion of studies available for meta-analyses by study conditions...............77 Table 7. Estimated bias in mean effect size by study conditions.......................................79 Table 8. Estimated bias in effect size variance by study conditions..................................81 Table 9. Average Type I error rates for methods to detect publication bias by study conditions...............................................................................................86 Table 10. Proportion of conditions with ad equate Type I error for methods to detect publication bias by study conditions.....................................................88 Table 11. Power estimates when primary study sample size is equal (50) and population effect size variance is 0.00...........................................................103 Table 12. Power estimates when primary study sample size is equal (50) and population effect size variance is 0.10...........................................................104 Table 13. Power estimates when primary study sample size is equal (50) and population effect size variance is 0.33...........................................................105 Table 14. Power estimates when primary study sample size is equal (50) and population effect size variance is 0.50...........................................................106

PAGE 8

v Table 15. Power estimates when primary study sample size is equal (50) and population effect size variance is 1.00...........................................................107 Table 16. Maximum power estimates for me thods to detect publication bias by study conditions.............................................................................................109 Table 17. Estimated bias in mean effect size by study condition and magnitude of publication bias..............................................................................................167 Table 18. Estimated bias in effect size variance by study condition and magnitude of publication bias..........................................................................................168 Table 19. Type I error rate estimates for conditions when the population effect size variance is 0.00.......................................................................................169 Table 20. Type I error rate estimates for conditions when the population effect size variance is 0.10.......................................................................................174 Table 21. Type I error rate estimates for conditions when the population effect size variance is 0.33.......................................................................................179 Table 22. Type I error rate estimates for conditions when the population effect size variance is 0.50.......................................................................................184 Table 23. Type I error rate estimates for conditions when the population effect size variance is 1.00.......................................................................................189 Table 24. Power estimates for conditions wh en the population effect size variance is 0.00.............................................................................................................194 Table 25. Power estimates for conditions when the population effect size variance is 0.10.............................................................................................................203 Table 26. Power estimates for conditions wh en the population effect size variance is 0.33.............................................................................................................212 Table 27. Power estimates for conditions wh en the population effect size variance is 0.50.............................................................................................................221 Table 28. Power estimates for conditions when the population effect size variance is 1.00.............................................................................................................230

PAGE 9

vi LIST OF FIGURES Figure 1. Example funnel plot with no public ation bias and a true effect size of zero...................................................................................................................41 Figure 2. Example funnel plot wi th publication bias and a true effect size of zero...........41 Figure 3. Begg Rank Correlation plot w ith standardized effect size by standardized variance.......................................................................................46 Figure 4. Egger regression plot with st andardized effect size by precision.......................48 Figure 5. Funnel Plot Regression method with effect size by sample size........................49 Figure 6. Flowchart depicting stages a nd steps in the Mont e Carlo design.......................64 Figure 7. Weight functions to establish proba bility of inclusion in a meta-analysis as a function of the p-value..............................................................................68 Figure 8. Average Type I error rates for each method to detect publication bias by number of studies.............................................................................................90 Figure 9. Proportion of conditions with ade quate Type I error for each method to detect publication bias by number of studies...................................................90 Figure 10. Average Type I error rates for each method to detect publication bias by sample size..................................................................................................92 Figure 11. Proportion of conditions with ade quate Type I error for each method to detect publication bias by sample size.............................................................92 Figure 12. Average Type I error rates for each method to detect publication bias by primary study group variance.....................................................................94 Figure 13. Proportion of conditions with ade quate Type I error for each method to detect publication bias by primary study group variance................................94 Figure 14. Average Type I error rates for each method to detect publication bias by population effect size magnitude................................................................96

PAGE 10

vii Figure 15. Proportion of conditions with ade quate Type I error for each method to detect publication bias by popul ation effect size magnitude...........................96 Figure 16. Average Type I error rates for each method to detect publication bias by population effect size variance....................................................................98 Figure 17.Proportion of conditions with ade quate Type I error for each method to detect publication bias by popul ation effect size variance...............................98 Figure 18. Maximum power estimates for me thods to detect publication bias by magnitude of publication bias........................................................................108 Figure 19. Maximum power estimates for me thods to detect publication bias by number of studies...........................................................................................110 Figure 20. Maximum power estimates for me thods to detect publication bias by primary study sample size..............................................................................111 Figure 21. Maximum power estimates for me thods to detect publication bias by primary study group variance........................................................................112 Figure 22. Maximum power estimates for me thods to detect publication bias by population effect size magnitude...................................................................113 Figure 23. Maximum power estimates for me thods to detect publication bias by population effect size variance.......................................................................114

PAGE 11

viii DETECTING PUBLICATION BIS IN RANDOM EFFECTS META-ANALYSIS: AN EMPIRICAL COMPARISON OF STATISTICAL METHODS Gianna Rendina-Gobioff ABSTRACT Publication bias is one threat to validity th at researchers conducting meta-analysis studies confront. Two primary goals of this resear ch were to examine the degree to which publication bias impacts the re sults of a random effects meta -analysis and to investigate the performance of five statistical methods for detecting publication bias in random effects meta-analysis. Specifi cally, the difference between the population effect size and the estimated meta-analysis e ffect size, as well as the difference between the population effect size variance and the meta-analysis eff ect size variance, provi ded an indication of the impact of publication bias. In addition, th e performance of five statistical methods for detecting publication bias (Begg Rank Co rrelation with sample size, Begg Rank Correlation with variance, Egger Regression, Funnel Plot Regression, and Trim and Fill) were estimated with Type I error rates and statistical power. The ove rall findings indicate that publication bias notably impacts the meta-analysis effect size and variance estimates. Poor Type I error control was exhibited in many conditions by most of the statistical methods. Even when Type I error rates were adequate the power wa s small, even with larger samples and greater numbers of studies in the meta-analysis.

PAGE 12

1 CHAPTER ONE: INTRODUCTION Background Publication bias (terms will be defined on Page 15) is one issue that researchers face when conducting a literat ure review, designing a new study, or conducting a metaanalysis Unfortunately, when a researcher gather s literature their fi ndings are not going to include all studies that have occurred re garding the specified content area searched. This phenomenon was discussed by Rosenthal (1979) as the “file drawer problem” or publication bias. Essentially, researchers may ha ve studies that are sitting in their filing cabinets because they decided not to publish or were rejected by journals. Reasons for researchers to not submit studies or for journa ls to reject studies typically revolve around whether the results indicated si gnificant findings, whic h are influenced by sample size, or large effects. In addition, published research can inadvert ently contribute to publication bias when researchers exclude non-significant fi ndings from results or report data poorly. Thus, there is a pattern in th e published literature of a gr eater number of studies with significant findings and large effects. Researchers conducting meta-analytic studies go to great lengths (or at least they should) to gather both published and unpublis hed studies on the cont ent of their metaanalysis. This step in the meta-analysis design is time consuming but critical. When meta-analysts do not include unpublished studies, the results of the meta-analysis may be biased. Specifically, the meta-a nalysis results may indicate an inflated effect because the

PAGE 13

2 published studies are more likely to have si gnificant results and large effects (Sharpe, 1997). Thus, publication bias is considered to be a threat to the valid ity of meta-analyses. One method for detecting pub lication bias is the visu al interpretation of a funnel plot (a scatterplot of effect size s and sample sizes). However, the visual examination of the funnel plot is limited because the interpretati on is subjective and the plot can be difficult to interpret when there are a small number of studies included in the meta-analysis (Greenhouse & Iyengar, 1994; Thornton & Lee, 2000). Consequently, some researchers have developed statistical met hods for detecting publication bi as that are not subjective. The following statistical methods for de tecting publication bias have been introduced and applied in th e literature: (1) Be gg Rank Correlation, (2) Egger Regression Method, (3) Funnel Plot Regression Method, and (4) Trim and Fill Method. Begg Rank Correlation method (Begg & Mazumdar, 1994) examines the relationship between the standardized treatment effect and the varian ce of the treatment effect using Kendall's Tau. The Egger Regression method (Egger, Smith, Schneider & Minder, 1997) treats the standardized treatment effect as the criterion and the precisi on of effect size estimation (the inverse of its standard error) as the predictor in a regression model (estimated by either OLS or WLS, with observations weight ed by the inverse of their variances). The Funnel Plot Regression method, suggested by M acaskill, Walter, and Irwig (2001), uses a WLS regression model with the criterion variab le being the treatment effect and study size being the predictor variable. The Tr im and Fill method, introduced by Duval and Tweedie (2000a, 2000b), is a nonparametric appr oach which is based on the funnel plot. Using symmetry assumptions, th e observed studies are ranked

PAGE 14

3 based on the absolute values of their deviations from the mean effect size; positive ranks for studies with effect sizes greater than th e mean effect size, negative ranks for studies with effect sizes less than the mean effect size. Statement of Problem Three issues drove the pursu it of this research : (1) publication bias is a problem which is underreported in meta -analyses, (2) both the impact of publication bias and the comparison of statistical methods for detec ting publication bias are lacking in the literature, and (3) publication bias issues have not been explored in random effects metaanalysis models. A wealth of literature doc uments the phenomenon of publication bias and the statistical bias that it presents to meta-analysts (Begg, 1994; Greenhouse & Iyengar, 1994; Rosenthal & Rubin, 1979; Sharpe, 1997; Smith, 1980; Sterling, 1959; Sutton, Abrams, Jones, Sheldon, & Song, 2000). This problem is compounded by the infrequent acknowledgement and application of methods for detecting (or adjusting for) publication bias. A review of 20 meta-analyses published in Psychological Bulletin from January 2003 to March 2005 identified 11 meta -analyses that addressed publication bias. The following methods were used to addre ss publication bias: publi cation status as a predictor, funnel plot examination, the Trim and Fill Method (Duval & Tweedie 2000a, 2000b), and the Fail Safe N (Rosenthal, 1979). A similar review of seven meta-analyses published in Review of Educational Research from Spring 2003 to Winter 2004 revealed no meta-analyses addressing publication bias These data indicate a need in the psychological and educational research disc iplines for increasing the awareness and implementation of publication bias detecti on techniques. The second issue may be the reason why publication bias de tection methods are inconsis tently being implemented,

PAGE 15

4 there are few empirical invest igations of the degree to wh ich publication bias impacts meta-analysis results or the comparison of sta tistical methods to dete ct publication bias (Begg & Mazumdar, 1994; Bradley & Gupta, 1997; Duval & Tweedie, 2000a; Duval & Tweedie, 2000b; Macaskill, Walter, & Irwi g, 2001; Rendina-Gobioff & Kromrey, 2004; Schwarzer, Antes, & Schumacher, 2002; Ster ne, Gavaghan, & Egger, 2000). Lastly, there is a need in the literature to examine the impact of publication bias and the performance of publication bias detection methods within the random effects meta-analysis model. The review of Psychological Bulletin and Review of Educational Research, detailed above, also documented the meta-analysis m odel implemented (fixed effect or random effect). The 20 articles in Psychological Bulletin indicated six fixed effects, seven random effects, and three implementing both (t he model could not be identified in four articles). The seven articles in Review of Educational Research indicated two fixed effects, one random effects, and one im plementing both (the model could not be identified in three articles). This review indicates that the random effects model is being implemented and that there is confusion surrounding which model to implement (as evidenced by the frequency of both models be ing implemented). Ther efore, investigation of the impact of publication bias and the pe rformance of statistical methods to detect publication bias within random e ffects models are of value. Purpose There were two primary goals overarching this research endeavor, (1) examine the degree to which publication bias impacts th e results of a random effects meta-analysis and (2) investigate the performance of five statistical methods for detecting publication bias in random effects meta-a nalysis. First, the impact on the meta-analysis results was

PAGE 16

5 estimated by examining the difference be tween the population effect size and the estimated meta-analysis effect size, as well as, the difference between the population effect size variance and the estimated meta -analysis effect size variance. Second, the performance of the five statistical me thods (Begg Rank Correlation (V), Begg Rank Correlation (N), Egger Regression, Funnel Pl ot Regression, and Trim and Fill) was evaluated with estimated Type I error rate s and statistical power. This research expands on the study c onducted by Rendina-Gobioff and Kromrey (2004) by examining the performance of the st atistical methods for detecting publication bias in a random effects model, rather than a fixed effects model Using similar methods to Macaskill, Walter, and Irwig (200 1), Rendina-Gobioff and Kromrey (2004) empirically compared Begg Rank Correlation, Egger Regression, Funnel Plot Regression, and Trim and Fill methods. This study simulated Hedges's g effect sizes with a fixed effects meta-analysis methodology. The results indicated that the Type I error rate performance was closest to nominal alpha level when Begg Rank Correlation method (with sample size) was used to detect publication bias. Tw o of the detection methods were found to have conservative Type I error rates, Funnel Plot Regression (with inverse variance weight) and Trim and Fill. All methods for detecting publication bias exhibited low power estimates. The Begg Rank Correla tion (with sample size) exhibited the greatest power, however it was still well be low the desired 0.80 target. The following research questions were of interest in this study:

PAGE 17

6 Research Questions 1. To what extent does publication bias imp act the estimated mean effect size and estimated variance in a random effects meta-analysis? a. To what extent does the number of primary studies included in the meta-analysis moderate the impact of pub lication bias on the estimated mean effect size and variance calculated for the meta-analysis? b. To what extent does the mean sample size of groups (including balanced and unbalanced) in the primary studies include d in the meta-analysis moderate the impact of publication bias on the es timated mean effect size and variance calculated for the meta-analysis? c. To what extent do the group variances ( homogeneous and heterogeneous) in the primary studies included in the meta-ana lysis moderate the impact of publication bias on the estimated mean effect size and variance calculated for the metaanalysis? d. To what extent does the magnitude of the population effect size moderate the impact of publication bias on the es timated mean effect size and variance calculated for the meta-analysis? e. To what extent does the variance of the population effect size moderate the impact of publication bias on the es timated mean effect size and variance calculated for the meta-analysis? 2. To what extent do Type I error rates vary across statistical methods for detecting publication bias in a random effects meta-analysis?

PAGE 18

7 a. To what extent does the number of prim ary studies included in the meta-analysis impact the extent that Type I error ra tes vary across sta tistical methods for detecting publication bias? b. To what extent does the mean sample size of groups (including balanced and unbalanced) in the primary studies included in the meta-analysis impact the extent that Type I error rates va ry across statistical methods for detecting publication bias? c. To what extent do the group variances ( homogeneous and heterogeneous) in the primary studies included in the meta-analysis impact the extent that Type I error rates vary across statistical met hods for detecting publication bias? d. To what extent does the magnitude of th e population effect size impact the extent that Type I error rates va ry across statistical methods for detecting publication bias? e. To what extent does the variance of the population effect size impact the extent that Type I error rates va ry across statistical methods for detecting publication bias? 3. To what extent do power estimates vary across statistical me thods for detecting publication bias in a random effects meta-analysis? a. To what extent does the number of prim ary studies included in the meta-analysis impact the extent that power estimates vary across statistical methods for detecting publication bias?

PAGE 19

8 b. To what extent does the mean sample size of groups (including balanced and unbalanced) in the primary studies included in the meta-analysis impact the extent that power estimates vary across statis tical methods for detecting publication bias? c. To what extent do the group variances ( homogeneous and heterogeneous) in the primary studies included in the meta-analysis impact the extent that power estimates vary across statistical me thods for detecting publication bias? d. To what extent does the magnitude of th e population effect size impact the extent that power estimates vary across statis tical methods for detecting publication bias? e. To what extent does the variance of the population effect size impact the extent that power estimates vary across statis tical methods for detecting publication bias? Hypotheses 1. Publication bias will impact the estimated mean effect size and estimated variance in a random effects meta-analysis; as the streng th of publication bias increases the mean effect size and estimated variance bias will increase. a. The number of primary studies included in the meta-analysis will not moderate the impact of publication bias on the es timated mean effect size and variance calculated for the meta-analysis; as the number of primary studies increases the mean effect size and estimated variance bias will be stable.

PAGE 20

9 b. The mean sample size of groups (incl uding balanced and unbalanced) in the primary studies included in the meta-a nalysis will moderate the impact of publication bias on the estimated mean eff ect size and variance calculated for the meta-analysis; as the mean sample size of groups increases the mean effect size and estimated variance bias will decrease. c. The group variances (homogeneous and he terogeneous) in the primary studies included in the meta-analysis will modera te the impact of publication bias on the estimated mean effect size and variance cal culated for the meta-analysis; as the degree of heterogeneity increases the mean effect size and estimated variance bias will increase. d. The magnitude of the population effect size will moderate the impact of publication bias on the estimated mean eff ect size and variance calculated for the meta-analysis; as the magnitude of the population effect size increases the mean effect size and estimated variance bias will decrease. e. The variance of the population effect size will moderate the impact of publication bias on the estimated mean effect size and variance calculated for the metaanalysis; as the variance of the population effect size increases the mean effect size and estimated variance bias will increase.

PAGE 21

10 2. The Type I error rates will va ry across statistical methods for detecting publication bias in a random effects meta-analysi s; the Begg Rank Correlation (N) method will have Type I error rates cl ose to the nominal 0.05 value, The Begg Rank Correlation (V) and Egger Regression will have Type I error rates greater than the nominal 0.05 value, and the Funnel Plot and Trim and Fill methods will have Type I error rates smaller than the nominal 0.05 value. a. The number of primary studies included in the meta-analysis will impact the extent that Type I error rates vary across statistical me thods for detecting publication bias; as the number of studies increases the Type I error rates will approach the 0.05 value. b. The mean sample size of groups (incl uding balanced and unbalanced) in the primary studies included in the meta-analy sis will impact the extent that Type I error rates vary across sta tistical methods for detecti ng publication bias; as the mean sample size of groups in the primar y studies increases the Type I error rates will approach the 0.05 value. c. The group variances (homogeneous and he terogeneous) in the primary studies included in the meta-analysis will impact th e extent that Type I error rates vary across statistical methods for detectin g publication bias; as the degree of heterogeneity increases the Type I error rates will deviate from the 0.05 value. d. The magnitude of the population effect si ze will impact the extent that Type I error rates vary across sta tistical methods for detecti ng publication bias; as the magnitude of the population effect size increases the Type I error rates will approach the 0.05 value.

PAGE 22

11 e. The variance of the population effect size will impact the extent that Type I error rates vary across statistical methods for de tecting publication bias ; as the variance of the population effect size increases the Type I error rates will deviate from the 0.05 value. 3. The power estimates will vary across statistical methods fo r detecting publication bias in a random effects meta-analysis overal l; the Begg Rank Correla tion (N) method will have better power estimates, compared to the other methods investigated. a. The number of primary studies included in the meta-analysis will impact the extent that power estimates vary acr oss statistical met hods for detecting publication bias; as the number of primar y studies increases power will increase. b. The mean sample size of groups (incl uding balanced and unbalanced) in the primary studies included in the meta-analysis will impact the extent that power estimates vary across statistical methods for detecting publication bias; as the mean sample size increases power will increase. c. The group variances (homogeneous and he terogeneous) in the primary studies included in the meta-analysis will impact the extent that power estimates vary across statistical methods for detectin g publication bias; as the degree of heterogeneity increases power will decrease. d. The magnitude of the population effect si ze will impact the extent that power estimates vary across statistical methods for detecting publication bias; as the magnitude of the population effect size increases power will increase.

PAGE 23

12 e. The variance of the population effect si ze will impact the extent that power estimates vary across statistical methods for detecting publication bias; as the variance of the population effect size increases power will decrease. Procedures This research simulated meta-analyses using a Monte Carlo design. The use of simulation methods allowed for the control a nd manipulation of research design factors and the incorporation of sampling error into the analyses. The first and second steps in the simulation were to generate observati ons in primary studies under known population conditions and to compute the effect size. The next step in the simulation was to impose the publication bias using the obtained p-valu es from the primary studies. The following two steps included computing the meta-analysis mean effect size and the statistical tests for publication bias. The final step in the research was to compute the analyses for determining the performance of the statistical tests for publication bias, Type I error rate and power estimates. In addition, the impact of imposing publication bias on the metaanalysis estimated mean effect size and variance was calculated. The simulation was modeled after that re ported by Macaskill, Walter and Irwig (2001) and Rendina-Gobioff and Kromrey (2004) but extends the conditions examined to random effects meta-analyses, rather th an fixed effects meta-analyses. For each primary study, Hedges's g effect size (Hedges & Olkin, 1985) was calculated based on the simulated data. The Monte Carlo study include d six factors in the design. These factors were (a) the number of primary studies in each meta-analysis (10, 20, 50, and 100), (b) the sample sizes of the two groups in each primary study (with mean total sample sizes ranging from 10 to 100 as well as balanced and unbalanced conditions), (c) group

PAGE 24

13 variances in the primary studies (variance ratios of 1:2, 1:4, and 1:8, as well as a homogeneous variance condition), (d) the ma gnitude of the population effect size (= 0.00, 0.20, 0.50, 0.80), (e) the variance of the population effect size (2 = 0, .10, .33, .50, and 1.00), and (f) the magnitude of the p ublication bias (no bias, moderate bias, and strong bias). Limitations There are several limitations to consider in relation to this research study. The simulation method implemented in this study pr ovides control of factors to investigate performance in specific situa tions. This benefit of simula tion studies also limits the generalizability of the study fi ndings. Thus, the controlled factors (number of studies, sample size, group variances, size of populati on effect size, and a random effects model) dictate the types of meta-analy ses the results can be generaliz ed to. Another restriction on generalizability is that only the Hedge’s g effect size is investigated. The impact of publication bias and the perfor mance of detection methods ma y vary across other effect size statistics. The final consideration of lim ited generalizability is the investigation of moderators. Although moderators are comm only explored in meta-analyses this simulation does not generalize to these analyses. Another limitation to cons ider relates to the methods used to impose publication bias. Although there are many f actors influencing the publicati on of studies (effect size, methodology, journal type, funding source, etc.), the function utilized to determine the selection of primary studies for the simulation of meta-analyses solely relies on the pvalues. The benefit of the func tion utilized is that it does not have a sharp cut off (p< .05)

PAGE 25

14 to determine the inclusion of primary studies. Therefore the function is in effect taking into consideration factors external to th e p-value. However, the accuracy of the function to represent the reality of external factors is unknown. This method is consistent with other studies that ha ve been done (Macaskill, Wa lter, & Irwig, 2001; RendinaGobioff & Kromrey, 2004). Importance of Study The detection of publication bias in th e context of meta-analysis is important because the validity of a meta-analysis is part ially determined by the selection of studies included in the synthesis. Examination of the performance of st atistical methods for detecting publication bias pr ovides information about the re search conditions for which they are appropriate to apply. In addition, an increased use of statistical methods for detecting publication bias may result, rather th an visually inspecting the funnel plot or not addressing publication bias at all. The reporti ng of publication bias detection will result in more accurate conclusions being drawn from meta-analysis results. Lastly, regular reporting of publication bias in meta-analysi s designs may result in a more favorable perception of meta-analysis methodology.

PAGE 26

15 Definitions Effect Size A point value that indicates th e strength and direction of the relationship of interest in th e research study. The e ffect size is what makes meta-analysis possible because it is a, “statistical standard ization of the study findings such that the resulting numerical values are interpretable in a consistent fashion across all the variables and measures.” (Lipsey & Wilson, 2001, p. 4). There are many different effect sizes, which are dependent on the statistical methods employed in the research. Fixed Effects Model Meta-analysis model that assu mes the population effect size being sampled by the studies within the meta-analysis is the same across studies (123...k ) (Shadish & Haddock, 1994).Thus, this model assumes that the estimated effect size will vary from the popul ation effect size by th e amount of sampling error due to the subjects associat ed with each study collected. Funnel Plot A graphical display (scatterplot) of the effect size and precision (sample size, standard error, or inverse standard error) of studies included in a metaanalysis, a method for detecting publica tion bias (Begg, 1994; Hedges & Vevea, 1996; Macaskill et al., 2001) Hedges's g Effect Size Effect size for standardized mean differences (Hedges, 1981), symbolized by g Meta-Analysis The summarization of empiri cal studies using quantitative methods; the effect size results from empi rical studies are collected and a summary statistics is calculated (Hedges & Vevea, 1998) Meta-Analytic Studies Studies that utilize meta-a nalysis research design.

PAGE 27

16 Power The probability of detecting a truly nu ll hypothesis (the accurate rejection of a null hypothesis), the probability of having accurate statistically significant findings (Cohen, 1992) Publication Bias Phenomenon introduced by Rosentha l (1979) as the “file drawer problem, which acknowledges that p ublished literature is more likely to have statistically significant findings producing a statistical bias (published li terature having a greater representation of statisti cal significance compared to non-published literature) Primary Studies The original studies that make up the sample for a meta-analysis Random Effects Model Meta-analysis model that assumes the population effect size distribution variance is greater than zero (20 ) (Shadish & Haddock, 1994). This model includes two sources of error associ ated with the estimated effect size; (1) sampling error and (2) random eff ects variance (Raudenbush, 1994). Random Effects Variance Component The unaccounted for and unidentifiable variation due to the studies sampled for the meta-analysis (Raudenbush, 1994). This variance is estimated and combined with sampling error in a random effects metaanalysis. Reporting Bias The exclusion of non-significant fi ndings in published research due to researchers’ incomplete repo rting (Begg, 1994; Sutton et al., 2000) Retrieval Bias The exclusion of studies due to the ability to locate and access published and unpublished resear ch (Sutton et al., 2000) Standardized Effect Size In contrast to a raw effect size, which indicates the strength and direction of a relationship in th e units of the scale that the relationship was measured; the standardized effect size is i ndependent of the scale used to measure the

PAGE 28

17 relationship. An example of a raw effect size is simply the difference between two means. In contrast, an example of a standardized ef fect size is the differe nce between two means divided by the standard deviat ion. Examples of other standardized effect sizes are correlation (r) and proporti on of explained variance (r2). Examples of raw effect sizes are covariance and raw score regressi on slopes (Abelson, R. P., 1995) Type I Error Rate The probability of inaccurate ly rejecting a null hypothesis (when the null hypothesis is true), the proba bility of having inaccu rate statistically significant findings (Cohen, 1992)

PAGE 29

18 CHAPTER TWO: LITERATURE REVIEW This literature review focuses on two main areas; meta-analysis and publication bias. The meta-analysis section details the pro cedures, analyses, models (fixed effects and random effects), alternative methods, threats to validity, and limitations. Provided within the publication bias section are: non-statist ical and statistical methods for detecting publication bias, prevalence and empirical ev idence for statistical methods to detect publication bias, options fo r researchers, and a broad perspective on decreasing publication bias. Meta-Analysis The development of methods for conducting research syntheses has a long history that is documented as early as 1904 with Pearson’s synthesis of correlations among typhoid inoculations and mortality (C ooper & Hedges, 1994a; Shadish & Haddock, 1994). Although research synthesis was introduced in the early 20th century, research synthesis was rare until the 1970’s with th e publication of the classic paper by Glass (1976) synthesizing psychothera py research. At this time Glass introduced the term “meta-analysis” to describe the methods for research synthesis. Gene Glass is possibly the most famous name associated with meta-a nalysis, yet there were other researchers at the time who introduced other methods for meta-analysis: Rosent hal and Rubin (1982), Schmidt and Hunter (1977), and Hedges and Olkin (1985).

PAGE 30

19 Meta-analysis is the summarization of empirical studies using quantitative methods. In short, the effect size results from empirical studies are collected and summary statistics are calculated (Hedge s & Vevea, 1998). Lipsey and Wilson (2001) provide a nice description of meta-analysis methods by comparing the process to survey research. Conceptually, meta-analysis is sim ilar to survey research because its methods involve surveying research repor ts, rather than people. During survey research a survey protocol is developed; similarly in meta-ana lysis the coding protocol is developed. In both methods, survey research and meta-analysis, a sample is gathered. The information or data are retrieved from the sample, people or research studies, with the survey or coding protocol. Lastly, the quantitative da ta from both survey research and metaanalysis are analyzed and summarized. Lipsey and Wilson (2001) outline specific guiding principles for the application of meta-analysis. All of the principles underl ying the application of meta-analysis focus on the sample, research studies being synt hesized, because not all research can be synthesized. For example, the first princi ple is that studies must be empirical investigations of the phenomenon of interest. In other words studies such as theoretical reports, research reviews, and policy proposal s are not appropriate for the meta-analysis sample. The second principle is that the research studies incl uded in a meta-analysis must include quantitative data. Thus, qualitative research studies are not appropriate for inclusion in a meta-analysis. The third prin ciple is that meta-analysis is applied to summary statistics provided by reports of previously conducted research. Accordingly meta-analysis is not conducted on complete data sets from previous research. Lastly, the research studies included in a meta-analysis must be similar in regards to: (1) the

PAGE 31

20 constructs and relationships be ing investigated, and (2) the research design and statistics. For example, research studies examining the relationship between self-efficacy and student achievement should not be combined with research studies examining the relationship between drug use and student achievement, two conceptually different constructs. Similarly, correlational resear ch studies should not be combined with experimental research. This last principle wi ll be discussed further in the threats to the validity of meta-analysis, “A pples and Oranges”, section. Procedures There are several aspects to collecting data for meta-analysis: (1) specifying the problem, (2) identifying sources, (3) defining inclusion/exclusion cr iteria, and (4) coding data. The first consideration for the meta-analy st is defining the problem of interest for the synthesis, which constructs and what t ype of relationship. In other words, which variables are being measured and what type of quantitative analys is is being conducted (central tendency, pre-post contrasts, group contrast, asso ciation) (Lipsey & Wilson, 2001). The following aspects of da ta collection are stipulated by this first consideration. Locating the primary studies to be included in the meta-analysis necessitates the specification of sources. Potential sources in clude electronic data bases, dissertations, conference proceedings/abstracts, hand search es, and contacting key researchers in the field (Lipsey & Wilson, 2001). The next step is defining which type s of studies will be included in the meta-analysis. A sample of crit eria that may be used is: country where the study was conducted, language of the study, pa rticipants included, type of publication (published vs. unpublished), study design, and date of study (Lipsey & Wilson, 2001). Lastly, the information within the studies (eff ect size, sample size, moderators, etc.) must

PAGE 32

21 be coded with specified methods and entered into a database. When multiple coders are utilized for extracting the data from studies, inter-rater reliability should be conducted to make sure the data are being coded consistently. Analyses Meta-analyses combine the research findings, effect sizes, gathered from previously conducted studies. An effect size is a point value that indicates the strength and direction of the relationship of interest in the research st udy. The effect size is what makes meta-analysis possible because it is a, “statistical standard ization of the study findings such that the resulting numerical values are interpretable in a consistent fashion across all the variable s and measures.” (Lipsey & Wils on, 2001, p. 4). There are many different effect sizes, which are dependent on the statistical methods employed in the research. For example, a research study investigating mean differences will have a different effect size calculat ion than a study investigating a correlation. However, the effect size from both types of research pr ovide standardized information about the strength and direction of the relationship. Th e effect sizes gathered from the research studies that are included in the meta-analysis must be the same type (i.e., all correlation effect sizes) and appropriate fo r the statistical analyses conducted in the research studies. An example effect size combined in me ta-analyses for the standardized mean difference is Hedges's g (Hedge s, 1981). This effect size is calculated with the following: 123 1 49i p X X g Ns Where N is the total sample size 1 X is the mean for group 1

PAGE 33

22 2 X is the mean for group 2 psis the pooled standard deviation = 22 1122 1211 11 nsns nn Where 1nis the sample size for group 1 2nis the sample size for group 2 2 1sis the standard deviation for group 1 2 2sis the standard deviation for group 2 Weights Study characteristics that vary among studi es to be included in a meta-analysis can be accounted for by weighting the study eff ect size estimates before combining them. According to Shadish and Haddock (1994), th ere are three assumptions underlying the use of weights. First, the accu racy of a study will be affect ed by a characteristic based on theory or evidence. Second, prior to combini ng, the nature and direction of the bias is predictable. Third, the methods for calcula ting the weights are avai lable and justified. Examples of characteristics for which wei ghts are applied in meta-analysis include: sample size, reliability, validity, restriction of range, and study quality (Lipsey & Wilson, 2001; Shadish & Haddock, 1994). Although the appropriateness of some of these characteristics for weighting prior to combi ng effect sizes can be argued, weighting for sample size is universal and fulfills all three assumptions outlined by Shadish and Haddock (1994). Since the sample sizes from research st udies included in a meta-analysis are bound to vary it is critical to weight the effect size accordingly. Essentially, an effect size

PAGE 34

23 estimated from a large sample size study will be more precise, have less sampling error, than a comparable study with a smaller sample size. In other words, effect sizes estimated from different sample size studies do not have equal precision. Thus, weighting the estimated effect sizes from studies with vary ing sample sizes contro ls the contribution of each effect size to the mean effect size for precision (Lipsey & Wilson, 2001). The precision of effect sizes is estimated with the standard error of the effect size. Thus, the effect size weight used to accommodate for precision is the inverse squared standard error (variance). Therefore, when conducting a meta-analysis the researcher needs to be able to calculate/gather the effect size from the studies and calculate the inverse variance. The latter can be challenging when the calcula tion for the standard error associated with the effect size is not known (Lipsey & Wilson, 2001). Continuing the example with Hedges's g effect size, a meta-analysis combining the effect sizes from standardized mean di fferences would weight the effect sizes to accommodate for precision differences due to varying sample sizes. The standard error for Hedges's g effect size is estimated with the following: 2 12 12122i ig nn SE nnnn Where ig is the estimated effect size for study i 1nis the sample size for group 1 2nis the sample size for group 2

PAGE 35

24 Therefore the weight for Hedges's g effect si ze is the inverse of the squared standard error, or: 1212 22 12122 2i innnn w nnnng Combining Effect Sizes and Bu ilding Confidence Intervals After gathering the effect sizes and calc ulating the weights for each effect size, the meta-analyst combines the effect sizes. The estimated mean effect size is a point estimate summary of the phenomenon of intere st. The Hedges's g mean effect size is estimated with the following: ii iwg g w Where is the sum over studies in the meta-analysis iwis the weight for study i igis the estimated effect size for study i This formula can be applied to any effect size by substituting ig with the effect size being summarized. In addition, the confidence interval around the estimated mean effect size is calculated by adding and subtracting 1.96 times the standard error of the mean 1g iSE w from the estimated mean effect size g With the confidence interval the meta-analyst can test for statistical sign ificance for the null hypothe sis that there is no

PAGE 36

25 effect or 0:0Hg(Shadish & Haddock, 1994). If th e confidence interval does not include zero then the null hypothe sis is rejected and the mean effect size is significantly greater than zero. Often meta-analysts are interested in investigating moderators, independent variables, which may influence the primary re lationship of interest. For example, a metaanalysis summarizing the relationship (effect size) between Graduate Record Exam (GRE) scores and graduate program success mi ght want to see if the relationship is different for males and females. Weighted least squares regression can be used to estimate mean effect sizes for moderators (Lipsey & Wilson, 2001). With this model the weights are the inverse varian ce (same as above), the eff ect sizes are the dependent variable, and the moderators ar e the independent variables. Homogeneity of Effect Sizes Although the heart of the meta -analysis results is the estimated mean effect size and the surrounding confidence interval, the me ta-analyst should de termine if all the variance has been accounted for by the observe d effect sizes (Shadish & Haddock, 1994). Thus, the homogeneity of the effect sizes is estimated and the null hypothesis that the effect sizes differ only by sampling error (2 0:0H or alternatively 0123:...kH ) is tested (Hedges & Vevea, 1998). The examination of homogeneity provides evidence for whether the variance observed from the studies included in the meta-analysis is greater th an what would be expected from sampling variance alone (Lipsey & Wilson, 2001). If heterogeneity is indi cated then the effect sizes

PAGE 37

26 are not estimating a common population mean In other words there may be study characteristics or moderators that are causing variability am ong the effect size. A commonly used statistic for homogene ity is the chi-square distributed Q statistic (Hedges & Olkin, 1985) Conceptually the Q statistic is a ratio of the between variance and within variance (Hedges & Vevea, 1998). Computationall y the value of Q is (Lipsey & Wilson, 2001): 2 iiQwgg Where iwis the weight for study i igis the effect size for study i gis the estimated mean effect size The estimated Qvalue is compared to the critical va lues for a chi-square with df=k-1 (k=number of studies). The null hypothesis of homogeneity is rejected when the Qvalue exceeds the critical value fo r chi-square, indicating th at the effect sizes are heterogeneous. The Q statistic has been docum ented to have low power in meta-analyses with small sample sizes and large nu mbers of studies (Harwell, 1997). Other characteristics of primary st udies included in a meta-analy sis investigated by Harwell (1997) which affect the power of the Q st atistic are non-normal score distributions, unequal sample sizes, and unequal variance s. The consequence of the low power associated with the Q statistic is that rese archers are more likely to conclude that the effect sizes are homogeneous when in fact they are not. Four options for the meta-analyst when heterogeneity is indicated are: (1) only interpret the resu lt descriptively, (2) include

PAGE 38

27 moderators in the analyses, (3 ) use regression tec hniques that account for the variance, or (4) incorporate the variance in to the model by using random effects methods (Shadish & Haddock, 1994). Meta-Analysis Models Fixed Effects Model When a researcher assumes that there is one population effect size that should be represented by the selection of studies collected for a meta-analysis, the fixed effects meta-analysis model is the appropriate model to implement. In other words, the fixed effects model assumes that the population eff ect size being sampled by the studies within the meta-analysis is the same across studies (123...k ) (Shadish & Haddock, 1994).Thus, this model assumes that the es timated effect size will vary from the population effect size by the amount of sampling error due to the subj ects associated with each study collected. Consequently, the varian ce of the effect sizes collected for the meta-analysis is only due to sampling error. The estimated study effect size in a fixed effects model can be represented with the following formula: iiT Where is the true population effect size iis the error associated with the estimate or sampling error The weights used in the analysis accommoda te for the sample sizes of the studies included in the meta-analysis, thus adju sting for variance due to sampling error. Specifically, the weights are the inverse va riance of the effect size statistic being summarized by the meta-analysis or 1i iw v The sample size of studies is inversely

PAGE 39

28 represented in the variances, as sample size increases the variance decreases (Shadish & Haddock, 1994). An example of a fixed eff ect weight for the standardized mean difference effect size (Hedge’s g) is: 1212 22 12122 2i innnn w nnnng Random Effects Model In contrast to the fixed effects model, in the random effects model the researcher assumes that the population effect size is a nor mal distribution of values, rather than one population effect size (Greenhouse & Iyengar, 1994). In other words, the random effects model assumes that the populati on effect size distribution vari ance is greater than zero (20 ) (Shadish & Haddock, 1994). Thus, this model includes two sources of error associated with the estimated effect size; (1) sampling error and (2) random effects variance (Raudenbush, 1994). Sampli ng error is the variation due to the subjects selected within each of the studies included in the me ta-analysis (also include d in the fixed effects model). The random effects variance is unacc ounted for and unidentifiable variation due to the studies sampled for the meta-analy sis (Raudenbush, 1994). Some examples of study characteristics that may contribute to the random effect variance are the year the study was conducted, geographic location, ch aracteristics of the person implementing treatment, cultural setting, and socioeconom ic status of the communities participating (Raudenbush, 1994). Another way to describe the random effects model is a two-stage sampling design (Raudenbush, 1994). The fi rst-stage variance component, random effects, is due to sampling studies from a population of studies. The second-stage variance component, sampling e rror, is due to sampling subjects from a population for a

PAGE 40

29 given study. Thus the variance of the effect sizes is due to variation in samples and variation in the studies. The variance associated with each study contributing to the estimated effect size can be represented with the following formula (Raudenbush, 1994): 2ii Where 2 is the variance associated with the random effects i is the variance associated with sampling error When the random effects variance is absent the value of 2 will be zero and then the model provides estimates that are the same as a fixed effect model. In other words the only difference among the fixed effects mode l and the random effects model is the inclusion of the random effects variance estimate (Raudenbush, 1994). The estimated study effect size in a random effects mode l can be represented with the following formula: *iiTv Where is the population mean effect size iv is the error associated with the es timate or the combination of random effects error and sampling error The weights used in the analysis are a combination of an accommodation for the sample sizes in the studies included in th e meta-analysis and an accommodation for the random variation among the studies. The estimate for the random effects variance component (REVC) (Shadish & Haddock, 1994) is: 2 21 /iiiQk www

PAGE 41

30 Where Q is the estimate of homogeneity detailed earlier kis the number of studies included in the meta-analysis iw is the inverse variance weight used in a fixed effects model The REVC is set to 0 when computationall y negative because conceptually it cannot be negative (Hedges & Vevea, 1998). The final wei ght used to calculate the random effect mean effect size is the invers e of the sum of the REVC (2 ) and the sampling error variance (iv ). Problems with the estimate of random e ffects variance revolve around the number of studies included in the meta-analysis. Essentially, the random effects variance component will have poor precision when the number of studies included is small (Raudenbush, 1994). As a result the inferen ces based on the random effects model with small numbers of studies will be inaccurate. This problem will still be significant even when the sample sizes within the studies ar e large. In other wo rds the accuracy is dependent on the number of studies included in the meta-analysis, not the sample sizes within those studies (Hedges & Vevea, 1998). There are advantages to us ing a random effects mode l (Raudenbush, 1994). First, the random effects model allows researchers to generalize beyond the studies included in the meta-analysis, which is often the intent of the researcher. Second, random effects models can provide results in a parsimonious way because the effects of study variation are incorporated. This has the greatest impact when there are lots of studies included in the meta-analysis. The third advantage is that the studies included in a meta-analysis are rarely replications that do not differ me thodologically or substantively (Shadish &

PAGE 42

31 Haddock, 1994). Since in reality the studies available to the meta-analyst have aspects that vary, the random effects model is more appropriate. Two threats to the validity of random e ffects meta-analysis include: (1) random effects are estimated from the studies includ ed, and (2) the random ef fects are assumed to be normally distributed (Raudenbush, 1994). Fixed Effects vs. Random Effects Models Shadish and Haddock (1994) provide two aspects to consider when deciding between the random effects and fixed effect s models: (1) statistic s and (2) concepts. Statistically one can choose the model based on the results of the test for heterogeneity among the effect sizes (Q-test). If the Qtest indicates homogeneity then one would choose the fixed effects model. In contrast, if heterogeneity is indicate d by the Q-test then the random effects model would be implem ented. Hedges and Vevea (1998) call this “Conditionally Random-Effects” (p.495). The othe r aspect to consider a priori when determining whether to implement random effect s versus fixed effects, is the type of conceptual conclusion the resear cher desires to make. When th e researcher is interested in generalizing only to the studies included in the meta-analysis, then the fixed effects model is appropriate. Hedges and Vevea ( 1998) call these “Conditional Inferences”. However, it is more typical that a researcher is interested in generalizing beyond the studies included in the meta-analysis, to which the random effects model is more appropriate. According to Hedges and Vevea (1998) these are “Unconditional Inferences”. Hedges and Vevea (1998) consider this second aspect, the conceptual, to be the most important influence for determining the model.

PAGE 43

32 The decision to use the fixed effects or random effects model may be dependent on the number of studies included in the meta-a nalysis. When there are a small number of studies included in a meta-analysis the random effects variance estimate will have poor precision (Raudenbush, 1994). This means that the generalizable statements from a random effects meta-analysis with a small nu mber of studies could be inaccurate. For example, if one did a meta-analysis with 2 st udies one would be skeptical of generalizing to a population of studies sin ce the variance could be erratic with a sample of 2. When fixed effects models are applied to random effects data (heterogeneous effect sizes) there are consequences to the accu racy of the confidence intervals. In other words if a researcher assumes that there is no random effects variance (applying the fixed effects model) when in fact there is varian ce, the total variance will be underestimated. As a result the confidence interval will be smaller than it should be (indicating a more precise estimate) (Hedges & Vevea, 1998). Thus, in this situation th e confidence interval may be 90% when the conclusions are being drawn thinking that th e confidence interval is 95%. The inaccuracy of the confidence inte rval estimate will increase with increasing heterogeneity among the effect sizes and decreasing numbers of studies (Hedges & Vevea, 1998). Although the confidence interval estimates will vary for the two models when applied to heterogeneous effect sizes, th e estimated mean effect size will be similar (Hedges & Vevea, 1998). Alternative Methods: Rosenthal and Rubin and Hunter and Schmidt The previously described meta-analysi s methods were developed by Hedges and Olkin (1985). However, there are other met hods that have been developed within the meta-analysis framework. Each of the methods was generated in a different field (Shulze,

PAGE 44

33 Holling, Grofsmann, Jutting, & Brocke 2003). The Rosenthal and Rubin method (Rosenthal & Rubin, 1982) or iginated in educational research, whereas the Hunter and Schmidt method (Hunter & Schmidt, 1990) or iginated in I/O psychology. The Hedges and Olkin method was developed focusing on th e statistical steps involved in conducting a meta-analysis. The three methods have similarities a nd differences. The Rosenthal and Rubin method is similar to the Hedges and Olkin method except for the determination of statistical significance (Field, 2001). Sin ce there is minimal difference among the Rosenthal and Rubin method and the Hedges an d Olkin method, the following discusses the differences between the Hunter and Schmidt method and the Hedges and Olkin method. Shulze et al. (2003) outline four di stinctions between th e Hedges and Olkin and Hunter and Schmidt methods: (1) transformation of effect sizes, (2) we ights, (3) standard error estimates, and (4) homogeneity testing. In addition to the f our differences that Schulze et al. (2003) outline, the Hunter a nd Schmidt method advocates using “artifact corrections” for the reliability of measures, va riable restriction of range, dichotomization of continuous variables, and construc t validity (Hunter & Schmidt, 1990). The final distinction of interest among a ll three methods is the consideration of fixed effects and random effects models. The Rosenthal and Rubin method did not specify random effects models (Field, 2001), thus these methods are only applicable to fixed effect models. As described earlie r, the Hedges and Olkin method distinctly specifies different methods for each of the m odels (fixed and random). The fixed effects and random effects application of the Schmid t and Hunter model is less clear (Field, 2001; Schulze, 2004; Schulze et al., 2003). Fi eld (2001) discusses the Hunter and

PAGE 45

34 Schmidt method as a random effects met hod. However, Field (2001) acknowledges a reviewer’s comment that the sample size weight utilized in the calculation of the mean effect size assumes a fixed effects model. Schulze et al. (2003) point out the discrepancies among the reporting by Hunter and Schmidt regarding whether their method is for fixed versus random effects models. Threats to Validity There are several potential threats to the va lidity of meta-analyses, which relate to the primary studies included or the meta-analy sis process. Several threats that relate to the primary studies included in the meta-analy sis are: (1) unreliability (2) restriction of range, (3) missing effect sizes (4) incompatible construc ts, and (5) poor study quality. The first two threats (poor reliability of m easures and measures with restricted range) attenuate the effect size in the primary study, thus rendering the combined effect sizes across studies attenuated (Matt & Cook, 1994) The third threat, missing effect sizes, occurs when researchers provide incomplete results which do not include statistically non-significant findings (Matt & Cook, 1994). The consequences of this threat are that the estimated mean effect size will be in flated. When primary studies vary in the constructs being measured or variables of inte rest they should not be combined in a metaanalysis, also called “Apples and Oranges” thr eat to validity (Sharp e, 1997). Lastly, the quality of studies included in a meta-analysis can introduce e rror, also called “Garbage In Garbage Out” (Lipsey & Wilson, 2001; Sharpe, 1997). Other sources for validity threats in meta-analysis are associated with the processes involved: (1) incomple te data collection, (2) inaccurate data collection, (3) poor methodology, and (4) inadequate power. Meta -analysts should st renuously attempt to

PAGE 46

35 gather all studies (Matt & Cook, 1994), however when researchers do not publish or make accessible the results of their findings th e effect sizes included in the meta-analysis are incomplete. This is also called public ation bias (Lipsey & Wilson, 1994; Matt & Cook, 2001; Sharpe, 1997). The second threat, in accurate data collection, can occur when the data from primary studies are inaccurate ly coded or when the effect sizes from primary studies are inaccurately transforme d or calculated (Matt & Cook, 1994). There are several ways that a meta-analyst can implement poor methodology: combining effect sizes that are not independent failing to weight effect sizes for precision, and inappropriately applying fixed or random effects models (Matt & Cook, 1994). Finally, when the meta-analyst is interested in performing many statistical tests (moderator analyses) the number of studies included in the meta-analysis should be considered to determine the available power. Limitations Among the appeals for conducting a meta-a nalysis are the reduction of sampling error and the ability to summarize across se ttings, researchers, and circumstances. However, there are limitations to consider when interpreting the results of a metaanalysis. Cooper and Hedges (1994b) deta il three limitations: (1) evidence is correlational, (2) post hoc analys es, and (3) primary research needs. The first limitation is due to the nature of the previously generated studies being summarized. Essentially, the meta-analyst is unable to randomly assign va riables (especially moderators) in a metaanalysis. This limits the conclusions drawn to be correlational, not causal. In rare cases where all the primary studies combined in a meta-analysis are experimental, causal interpretations may be appropriate. The second limitation is that meta-analysis is a post

PAGE 47

36 hoc procedure, the generation of hypotheses are dependent on the data used to test it. In other words, the research st udies included in a meta-ana lysis are what prompt the hypotheses that the researcher has developed (without the primary studies there would not be theories to support the synthesis). The last limitation is that there is a dependent relationship among primary research and meta -analysis. Both types of research are valuable and each should provide new directi ons for the other (a circular evolution). Often the meta-analyst is interested in generalizing the results of the synthesis to the general population and/or universe, however there are limits to these statements to consider. The ability to genera lize to certain settings and populations is dependent on the primary studies included in th e synthesis (Lipsey & Wilson, 2001). In other words, if all of the primary studies included in a meta-analysis include children age 5 to 13 years, then it is not appropriate to generalize the findings from the synthesis to adults. The same principles of generalization in primary st udies apply to meta-analysis, however the criteria determining the boundaries for generali zing are the characteristics of the primary studies. Publication Bias Publication bias is one issue that res earchers face when conducting a literature review, designing a new study, or conducting a meta-analysis. Unfortunately, when researchers gather literature their findings are not going to include all studies that have occurred regarding the specified content ar ea searched. This phenomenon was discussed by Rosenthal (1979) as the “file drawer problem” or publication bias. Essentially, researchers may have studies that are sitting in their filing cabinets because they decided not to publish or were rejected by journals Typically, researchers do not submit studies

PAGE 48

37 when the results lack significant findings (w hich are influenced by sample size) or large effects; similarly, journals often reject studie s that lack significant findings. This can also be called selective publicati on or objective publication bi as (Begg, 1994). There are several reasons that a researcher might choose not to publish studies, affecting the inclusion of results in a me ta-analysis: students who leav e academia, the researcher’s personal interests can be in conflict with re sults, researcher’s political beliefs, and interference by the funding sour ce (Sutton et al., 2000). Publication bias is assumed to encompass both retrieval and reporting bias (Greenhouse & Iyengar, 1994). Retrieval bias is associated with the ability to locate and access unpublished results. Studies presented at conferences that were not subsequently published can sometimes contribute to retrie val bias (Sutton et al., 2000). Even the researchers most dedicated to gathering unpublis hed research will still have some studies that are unattainable. Begg (1994) describes s ubjective publication bias as the practice of exaggerating, or only including, statistically significant results also called reporting bias. In addition, authors may publish the same resu lts in multiple journals; this can cause a bias due to the duplication of studies in a meta-analysis (Sutton et al., 2000). Regardless of the source contributing to publ ication bias, there is a patter n in the published literature of a greater number of studies with significant findings (Ste rling, 1959) and large effects (Smith, 1980). To aid in describing the publication bias phenomenon Table 1 presents the impact of variance and effect sizes (small and larg e) on the potential for publication. Thus, one can see that when a study has small variance there is likely to be a large sample and statistical significance (Sutton et al., 2000). In this situation the probability of publication

PAGE 49

38 is likely regardless of the e ffect size (small or large). In contrast, when a study has large variance there is likely to be a small sample and non significant fi ndings (Sutton et al., 2000). In this situation the pr obability of publication is dependent on the size of the effect, larger effects being more likely to be published (B egg, 1994; Sutton et al., 2000). Begg (1994) indicates that when examining the relationship among study design features (sample size, randomization, prospective/re trospective) and effect size to detect publication bias that sample size has the larg est influence on publica tion bias. Note that both the statistical significance of findi ngs and the size of the effect found are contributing to the phenomenon of publication bias. Table 1. Simplified description of the impact of the relationship among the variance and effect size observed in a st udy on the likelihood of publication. Effect Size Small Large Small (N=large) Published (Statistical Significance) Published (Statistical Significance) Large Variance (N=small) Not Published (No Statistical Significance) Published (Statistical Significance) The prevalence of publication bias has a long history of documentation. Sterling (1959) reviewed 294 articles published in psychology journals and found 286 (97%) to report statistically sign ificant results. The magnitude of effect sizes were found to be greater in published, compared to non-publis hed, studies in a small sample of metaanalyses (k=12) by Smith (1980). Lipsey a nd Wilson (1993) reviewed 92 psychological and educational meta-analyses that reported mean effects separately for published and unpublished studies. The results indicated that published studies reported larger (0.14 SDs higher) mean effect sizes than their comparable unpublished studies.

PAGE 50

39 Researchers conducting meta-analytic studies go to great lengths (or at least they should) to gather both published and unpublis hed studies on the cont ent of their metaanalysis. This step in the meta-analysis design is time consuming but critical. When meta-analysts do not include unpublished studies, the results of the meta-analysis may be biased, a limitation that should be discussed. Specifically, the meta -analysis results may be statistically biased because their results indicate an infl ated effect due to publication bias which argues that published studies are more likely to have significant results and large effects (Begg, 1994; Sharpe, 1997; Sutt on et al., 2000). Thus, publication bias is considered to be a threat to the validity of meta-analyses. Begg (1994) emphasized the impact of publication bias on meta-analysis by stating that, “Public ation bias presents possibly the greatest methodologic threat to va lidity of a meta-analysis.” (p. 407). The importance of detecting publicati on bias within the realm of me ta-analysis is not only that the results will be inflated but they will have additional credibility due to false precision, imposed by the synthesis of mu ltiple studies (Begg, 1994). According to Sutton et al. (2000), ther e has been little em pirical evidence regarding the impact of publica tion bias on the results and c onclusions of meta-analyses. One study addressing the impact of publication bias on the re sults of meta-analyses was conducted by Bradley and Gupta (1997). This study presents a formula for estimating the mean effect size resulting from controlled population mean effect sizes and controlled missing studies (based on the value of the e ffect size that would eliminate the study from being included in the meta-analysis). The fi rst conclusion of this study is that as the percentage of missing studies decreases, the estimated mean effect size approaches the population mean effect size. Second, this rese arch indicates that as the population mean

PAGE 51

40 effect size increases, the pr oportion of studies not incl uded in the meta-analysis decreases. Although these conclu sions are somewhat intuitive, the surprising aspect of the results is the magnitude of the difference between the population mean effect size and the estimated mean effect size. For example, when the population mean effect size was 0 and all studies with an effect size of 0 or less were not included in the meta-analysis, the estimated mean effect size was 0.79, a difference of 0.79. Non Statistical Methods for Detecting Publication Bias One method for detecting public ation bias, which is wide ly used, is the visual interpretation of a funnel plot (a scatterplot of effect sizes and sample sizes) (Begg, 1994; Hedges & Vevea, 1996; Macaskill et al., 2001). An example funnel plot with no publication bias (Figure 1) based on a simu lation of 100 studies and an example funnel plot with publication bias pr esent (Figure 2) based on the re moval of 10 studies from the 100 studies simulated for Figure 1 are provided. The plot should take the shape of a funnel with the peak at the population true eff ect size when no publication bias is present. In contrast, when the funnel plot is skewed or missing studies related to small samples and small effect sizes, the pl ot indicates publication bias. However, the visual examination of the funnel plot is limited because the interpretation is subjective and the plot can be difficult to interpret when there are a small number of studies included in the meta-a nalysis (Greenhouse & Iyengar, 1994; Tang, & Liu, 2000; Thornton & Lee, 2000). In other words there needs to be enough studies included in the funnel plot to have varying sa mple sizes represented (Sutton et al., 2000). Tang and Liu (2000) argue that the scale used to represent precision (inverse standard

PAGE 52

41 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 5791113151719 Sample SizeEffect Size Figure 1. Example funnel plot with no publica tion bias and a true effect size of zero. -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 5791113151719 Sample SizeEffect Size Figure 2. Example funnel plot with publica tion bias and a true effect size of zero.

PAGE 53

42 error vs. sample size) can alter the interp retation of the funnel plot. In addition, asymmetry in a funnel plot can be an indica tion of other study characteristic variance (methodological design, inadequa te analyses, choice of eff ect measure, fraud, and chance) (Sutton et al., 2000). Consequently, so me researchers have developed statistical methods for detecting publication bias that are not subjective. Statistical Methods for De tecting Publication Bias The methods for detecting publication bias to be investigated in this research all examine the relationship among the effect sizes and the precision of the effect sizes. The various methods use different approaches fo r standardizing the effect size (or not standardizing it) and different definitions of precision (sam ple size, variance, inverse variance). However, the methods are all examini ng the relationship that is displayed in a funnel plot and the assumption that the absen ce of studies with small effect sizes and minimal precision (small sample size or larg e variance) provide evidence of publication bias. Therefore, a strong relationship is an indication of publication bias. An overview of the methods for detecting publication bias is presented in Table 2 along with the variables and analyses that are uti lized with the method.

PAGE 54

43 Table 2. Overview of methods for detecting publication bias. Method for Detecting Publication Bias Variables from Primary Studies Examined Analysis Funnel Plot Effect Size Sample Size Visual Interpretation (non-statistical) Begg Rank Correlation (V) Standardized Effect Size Variance of Effect Size Rank Correlation Begg Rank Correlation (N) Standardized Effect Size Sample Size Rank Correlation Egger Regression Standardized Effect Size Precision OLS Regression Funnel Plot Regression Effect Size Sample Size WLS Regression Trim and Fill Deviation of Effect Size from Mean Effect Size Number of studies included in Meta-Analysis Nonparametric Rank Method Note. The standardized effect size incl uded in the Begg Rank Correlation and Egger Regression analyses are calculated differently. As an aid for the presentation of the fo llowing statistical methods for detecting publication bias, a hypothetical sample of 10 studies were simulated with an average sample size of five observations in each of two groups. For each study, the sample effect size was calculated. These data (see Table 3) will be used to illustrate each method.

PAGE 55

44 Table 3. Formula variables and their values for a hypothetical sample of 10 effect sizes. Begg Rank Correlation Method Begg Rank Correlation method (Begg & Mazumdar, 1994) examines the relationship between the standardized treatmen t effect and the variance of the treatment effect using Kendall's Tau. The standardiz ed treatment effects are estimated as: B i i i S igg g Where ig is the ith observed study effect size ig is the weighted average effect size = ii iwg w Where iw is the weight or inverse of the effect size variance (vi) s ivis the standardized variance of the treatment effect = 11ii vv The relationship between the standardized treatment effect and the variance is illustrated in Figure 3. Ranks are assigned for the obser ved standardized treatment effects and the Study n1 n2 ig i 1i i ig B ig s i E ig 1 3 4 0.9740 0.6511 1.5359 1.4959 1.3903 0.6065 1.2070 2 8 4 1.0686 0.4226 2.3664 2.5288 1.9151 0.3779 1.6438 3 8 7 0.7538 0.2868 3.4868 2.6283 1.7528 0.2422 1.4075 4 8 7 -1.7505 0.3700 2.7027 -4.7311 -2.8782 0.3254 -2.8778 5 9 3 -0.5828 0.4586 2.1806 -1.2708 -0.7367 0.4140 -0.8606 6 4 3 -0.4348 0.5968 1.6755 -0.7286 -0.4388 0.5522 -0.5629 7 3 4 0.3858 0.5940 1.6836 0.6496 0.6673 0.5493 0.5006 8 6 3 -0.8983 0.5448 1.8354 -1.6488 -1.1163 0.5002 -1.2170 9 6 7 -0.6207 0.3243 3.0832 -1.9136 -0.9679 0.2797 -1.0898 10 5 3 0.2982 0.5389 1.8557 0.5533 0.5788 0.4943 0.4062 Sum 22.4057 -2.4371 ig= -0.1088

PAGE 56

45 variances of those treatment e ffects (alternatively, the sample sizes may be ranked rather than the estimated variances). The correlati on between these ranked values (Kendall’s Tau) leads to a statistical test for the presence of publication bias. The standardized treatment effects (B ig) are provided in Table 3. In this sample of effect sizes the weighted mean effect size is -0.1088 and the sum of the reciprocals of the sampling variances is 22.4057. The standardi zed treatment effect for the first study is 1 1 10.9740(0.1088) 1.3903 1 0.6511 22.4057B i Sgg g Cliff and Charlin ( 1991) derived the estimated samp ling variance of Kendall’s tau as 2 2 222 12121212 124121 var() 123ihih ihihtnnttnnt t nnnn where tih12 = a concordance indicator (equal to 1 if observations i and h are ranked in the same order on variable s 1 and 2, equal to -1 if they have opposite ranks, and equal to zero if they are tied), and t12 = the sample estimate of tau.

PAGE 57

46 -4 -3 -2 -1 0 1 2 3 0.000.100.200.300.400.500.600.70 Standardized VarianceBegg's Standardized Effect Size Figure 3. Begg Rank Correlation plot with st andardized effect size by standardized variance. The Kendall's tau for the hypothetical data example is 0.111 a nd its variance is 0.0830, using the effect size varian ce in the calculation. The ratio 12 12var() t z t is compared to a critical value from a standard normal distribution to provide a test of the null hypothesis that Tau = 0. For these data, 0.1111 0.3853 0.0830 z p > .05, and the null hypothesis is not rejected. Thus, these data do not provide sufficient evidence that publication bias exists in this sample of studies. Egger Regression Method The Egger Regression method (Egger, Smith Schneider & Minder, 1997) treats the standardized treatment effect as the criterion and the prec ision of effect size estimation (the inverse of its standard error) as the predictor in a regression model

PAGE 58

47 (estimated by either OLS or WLS, with obs ervations weighted by the inverse of their variances). The standardized trea tment effects are estimated as: E i i ig g Where gi is the observed study effect size for study i vi is the variance of the effect size for study i For example, for study 1 in Table 1, 10.9740 1.2070 0.6511Eg For Egger Regression method, the precision of the effect size is estimated as: 11 v Examining the example data, an OLS regres sion with the standardized treatment effect as the criterion and the precision as the predictor re sulted in a slop e of -0.7222 and intercept of 0.9268 (see Figure 4). Although not conducted for these sample data, this regression model can also be estimated with WLS regression. The slope of this regression equation provides an estimate of the true effect and the intercept is expected to have the value of zero when no publication bias exists. Thus, a test of the nu ll hypothesis that the regression intercept equals zero provides a te st of publication bias. In the example, the null hypothesis would no t be rejected ( t = 0.3972, p > .05, df =8), thus these data do not suggest a relationship between standardized tr eatment effects and prec ision of estimation. Therefore, these data do not pr ovide sufficient evidence that publication bias exists in this sample of studies.

PAGE 59

48 y = 0.9268 + -0.7222X-3 -2 -1 0 1 2 3 1.001.201.401.601.802.00 PrecisionEgger's Standardized Effect Size Figure 4. Egger regression plot with standardized effect size by precision. Funnel Plot Regression Method The Funnel Plot Regression method, suggested by Macaskill, Walter, and Irwig (2001), uses a regression model with the criterion va riable being the treatment effect and study size being the predictor variable. Es timation by WLS is recommended for such a model, using as weights the invers e of the estimation variance (e.g., 1 i ). For example, for study 1 in Table 1, the weight is: 11 1 1.5359 0.6511 v A WLS regression with the treatment effect as the criterion and the study size as the predictor resulted in a slope of -0.0660 and intercept of 0.6374 for the example data (see Figure 5). In contrast to the Egger met hod, in this regression equation the slope will indicate no publication bias wh en it has the value of zero and the intercept in this

PAGE 60

49 regression equation will indicate the true effect. Thus, a test of the null h ypothesis that the regression slope equals zero provides a te st of publication bias. In the example, the null hypothesis would no t be rejected ( t = -0.9714, p > .05, df = 8). Therefore, these data do not suggest a relationship be tween the observed effect sizes and sample size, and these data do not provide sufficient evidence that publication bias exists in this sample of studies. y = 0.6374 + -0.0660X -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 0246810121416 Sample SizeEffect Size Figure 5. Funnel Plot Regression met hod with effect size by sample size. Trim and Fill Method. The Trim and Fill method, introduced by Duval and Tweedie (2000a, 2000b), is a nonparametric approach which is based on th e funnel plot. Using symmetry assumptions the observed studies are ranked based on the absolute values of their deviations from the

PAGE 61

50 mean effect size; positive ranks for studies with effect sizes greater than the mean effect size, negative ranks for studies with effect si zes less than the mean effect size. The ranks are estimated as: *ii irrankgg For example, for study 1 in Table 1, 10.97400.10881.08278 rrankrank A negative algebraic sign is assigned to ranks where the ig is less than the ig For example, the rank for study 4 would be assigned a negative algebraic sign: 41.75050.10881.641710 rrankrank Using these ranks the number of research studies missing from th e funnel plot due to publication bias is estimated by: 01011 R where R0 is the estimated number of studi es concealed due to publication bias, **10100hkr where k is the number of studies includ ed in the meta-analysis, and *hr is the largest negative rank Publication bias is evidenced when R0 > 3, with power greater than 0.80 and = .05 (Duval & Tweedie, 2000a). In this example, the0 R is not greater than 3 and thus would suggest that these data do not provide sufficient evidence that publication bias exists in this sample of studies.

PAGE 62

51 As indicated earlier, the met hods for detecting publicati on bias examined in this research are similar in their focus on the re lationship between the effect sizes and the precision of the effect sizes. The various methods differ in their approaches to standardizing the effect size (or not standa rdizing it) and estima ting precision (sample size, variance, inverse variance). In additi on, these methods utilize different statistical procedures for estimating the relationship be tween the effect sizes and their precision. The Begg Rank Correlation method employs Kenda ll's tau correlation, whereas the Egger Regression and Funnel Plot Regression methods employ OLS and WLS regression, respectively. In contrast, the Trim and Fill method is a nonparametric rank method, which satisfies assumptions for both fixed effect and random effect meta-analysis methods (Sutton et al., 2000). Statistical Methods for Dete cting Publication Bias: Prevalence and Empirical Evidence Prevalence As a means for investiga ting the prevalence of the application of the above mentioned statistical methods for detecting publication bias, a sear ch for studies citing the introductory article of each method (Begg & Mazumdar, 1994; Duval & Tweedie, 2000a; Duval & Tweedie, 2000b; Egger et al ., 1997; Macaskill et al., 2001) was conducted (Table 4). The search returned a total of 1,007 citations, the majority were from Egger Regression method (N=611). Each of the citations were coded for type of study: (a) applied, (b) methodol ogy, and (c) descriptive. A pplied studies included metaanalyses that cited the method to detect publication bias. Methodol ogy studies included studies that compared methods to detect publication bias. Descriptive studies included

PAGE 63

52 studies that provided explanat ions of the phenomenon of publication bias and methods to address publication bias. The other aspect of the citati ons that was coded was the discipline of the journal; (a) medical, (b) educ ation/psychology, and (c) other. The results indicate that the medical journals are publishing more applied, methodology, and descriptive studies citing th e statistical methods for de tecting publication bias. In contrast, the education/psychol ogy journals are publishing ve ry few applied articles and none within the methodology and descriptive categories. Finally, Egger Regression method was the most cited in the medical jour nals. In contrast, the Trim and Fill method was most cited in the education/psychology journals. These results imply that the education and psychology disc iplines are not addressing publication bias detection methods in their applicati on of meta-analyses or expl oring the methodology of these methods. Another source for investig ating the prevalence of a pplying publication bias detection methods was a re view of two journals, Psychological Bulletin and R eview of Educational Research A review of 20 meta -analyses published in Psychological Bulletin from January 2003 to March 2005 identified 11 meta-analyses that addressed publication bias. The following methods were used to a ddress publication bias: pub lication status as a predictor, funnel plot examination, the Trim and Fill Method (Duval & Tweedie 2000a, 2000b), and the Fail Safe N (Rosenthal, 1979). A similar review of seven meta-analyses published in Review of Educational Research from Spring 2003 to Winter 2004 revealed no meta-analyses addressing publication bias Thus, according to this search the application of publication bias detection methods is lacki ng in the psychological and educational research disciplines.

PAGE 64

53 Table 4. Frequency of articles citing the four statistical met hods to detect publication bias by article type and journal type. Medical Education/ PsychologyOther Begg Rank Correlation (N=231) Applied18409 Methodology1205 Descriptive1407 Egger's Regression (N=611) Applied527410 Methodology2004 Descriptive4105 Funnel Plot Re gression (N=23) Applied1531 Methodology202 Descriptive000 Trim and Fill (N=71) Applied3597 Methodology804 Descriptive404 Note: N represents the total number of times the introductory article was cited. Citation Journal Type Empirical Evidence Each of the statistical methods for dete cting publication bias described above has been empirically investigated in previous literature (Begg & Mazumdar, 1994; Duval & Tweedie, 2000a; Duval & Tweed ie, 2000b; Macaskill, Walter, & Irwig, 2001; RendinaGobioff & Kromrey, 2004; Schwarzer, Ante s, & Schumacher, 2002; Sterne, Gavaghan, & Egger, 2000). Schwarzer, Antes, and Schu macher (2002) simulated one fixed effects

PAGE 65

54 meta-analysis with 30 studies (n=34-236), imposed publication bi as by identifying all studies with effect sizes le ss than the population effect size, then applied Begg Rank Correlation method and Egger Regression method. In this specific meta-analysis the Begg Rank Correlation method failed to detect publication bias and the Egger Regression method accurately detected publication bias. Using similar methods, Duval and Tweedie (2000b) simulated one random effects meta-ana lysis of 35 studies w ith a population mean effect size of 0. Publication bias was imposed by removing 5 studies from the funnel plot. In this situation, the thre e detection methods applied (Begg Rank Correlation, Egger Regression, and the Trim & Fill) accurately detected publication bias. However, both of these studies have very limited generalization to other meta-analysis situations since in each case only one meta-analysis was simulated under one set of conditions. Begg Rank Correlation method (with va riance correlated) was empirically investigated to examine the power associ ated with the test by Begg and Mazumdar (1994). In a fixed effects meta-analysis simu lation study, power estimates were compared across several conditions; (a) small (k=25) and large (k=75) meta-analyses, (b) small and large effect size variances, (c) strong and moderate public ation bias, and (d) various effect sizes (range from 0 to 3.0). The overall results indicated that power estimates for small meta-analyses are moderate and that larger meta-analyses have better power estimates. Another simulation study investigat ed the power associated with Begg Rank Correlation method (with variance correlated) and Egger Regression method (Sterne, Gavaghan, & Egger, 2000). The factors controll ed in the meta-analysis of log odds ratios included the number of studies (k=5, 10, 20, 30), various samp le sizes, and two levels of publication bias. Overall the resu lts indicate that power estimat es increase with increasing

PAGE 66

55 numbers of studies (k) and that Egger Regr ession method exhibite d higher power than Begg Rank Correlation method (with p<.10). Ho wever, both of these studies did not provide the Type I error rates associated with the power estim ate. Thus, the Type I error rates may have been beyond the nominal values. There are two studies empiri cally comparing the statisti cal methods for detecting publication bias that are sim ilar to the research study bei ng proposed (Macaskill, Walter, & Irwig, 2001; Rendina-Gobioff & Kromrey, 2004). In a simulation study of log odds ratio e ffect sizes with a fixed effects metaanalysis methodology Macaskill, Walter, a nd Irwig (2001) compared the Begg Rank Correlation, Egger Regression, and Funnel Plot Regression methods. The results indicate that the Funnel Plot Regression method (w ith inverse variance weight) and the Begg Rank Correlation (with sample size correlated) ex hibited the best Type I error rates. In contrast, the Egger Regression (weighted and unweighted) and Begg Rank Correlation (with variance correlated) exhi bited the highest power, due to their corresponding inflated Type I error rates. Overall all of the methods exhibited low power when the number of studies included in the meta-analysis was low. Using similar methods to Macaskill, Wa lter, and Irwig (2001), Rendina-Gobioff and Kromrey (2004) empirically compared Begg Rank Correlation, Egger Regression, Funnel Plot Regression, and Trim and Fill methods. This study simulated Hedges's g effect sizes with a fixed effects meta-analy sis methodology. The results indicated that the Type I error rate performance was closest to nominal alpha level when Begg Rank Correlation method (with sample size) was used to detect publica tion bias. Two of the detection methods were found to have conser vative Type I error rates, Funnel Plot

PAGE 67

56 Regression (with inverse variance weight) and Trim and Fill. All methods for detecting publication bias exhibited low power estimat es. The Begg Rank Correlation (with sample size) exhibited the greatest pow er, however it was still well below the desired 0.80 target. Publication Bias Detected : Options for Researchers There are few options for a researcher when publication bias is detected. One option is to return to the search for studies, seeking additi onal unpublished studies. Another option, which is proba bly the most likely to occur, is for the researcher to proceed with the meta-analysis and use caution when interpreting the results. Begg (1994) advocates conservative in terpretations or the exclusio n of statistical significance testing. Lastly, there are stat istical analyses that provide results which are corrected for the effects of publication bias. The utilization of these statistical methods is controversial (Begg, 1994; Sutton et al., 2000). Sutton et al. (2000) indicate that the use of correcting methods in meta-analysis should only be used as a sensitivity anal ysis. The researcher can do a sensitivity analysis by conducting the meta-analysis with and without the corrections to see if the results are different. The statistical methods related to public ation bias in meta-analysis that focus on correcting for the bias can be categorized in three ways: (a) sampling frames, (b) identification of potential unpub lished studies, and (c) mode ling the selection process (Hedges, 1992). Sampling frames for correcti ng publication bias utilize sampling frames that are not affected by the study results to select studies for a meta-analysis (Begg, 1994). These methods are dependent on database s like trial registries lists of studies accepted by boards, and the researcher dilig ence in finding unpublished studies. Simes (1986) is an example of a st udy conducted with the sampling frame method. Methods that

PAGE 68

57 identify potential unpublished studies assume that the researcher only has access to published studies and corrects the results based on the internal consistency of the studies collected (Begg, 1994). Examples include th e fail-safe N (Rosenthal, 1979) and the capture recapture method (Bennett, Latham, St retton, & Anderson, 2004). The third type of method used to correct fo r publication bias attempts to model the selection process (publication bias) (Hedges, 1992) Several researchers have pr oposed models of this type (Cleary & Casella, 1997; Copas & Jackson, 2004; Copas & Shi, 2000; Dear & Begg, 1992; Hedges, 1992; Hedges & Vevea, 1996; Vevea & Hedges, 1995) Decreasing Publication Bi as: A Broad Perspective The perspectives and procedures within the research community need to be modified to decrease the publication bias phenomenon. One procedural change, which has been implemented in the medical field, is to register trials (studies) in computer databases (Begg, 1994; Sutton et al., 2000). Havi ng studies registered at the inception of the study helps the meta-analyst identify studi es that were not published but that fit the meta-analysis literature search parameters. In addition, registries can standardize the reporting of results. The standardization of results might decrease the amount of inaccurately reported data, which is another r eason for studies to be missing from a metaanalysis. Another procedural change that could reduce public ation bias relates to the publication process (Sutton et al., 2000). Changing the mindset of researchers and publishers to a model emphasizing research me thods rather than results would increase the number of published studies with non-signifi cant results. Sutton et al. (2000) argue that increases in the number of electronic journals with minimal space restrictions will increase the publication of studi es based on research methods, ra ther than results. Lastly,

PAGE 69

58 the perspectives of publishers and review ers, regarding the quality of unpublished literature, needs to change (Sutton et al., 2000). Some still argue that unpublished literature should not be include d in meta-analyses due to compromised methods in these studies. However, unpublished studies ar e not necessarily of lower quality. This literature review is pertinent for understanding the resear ch questions in the present study. The literatur e regarding meta-analysis me thods provides support for the importance of random effects models, which ha s not currently been examined in the area of publication bias. Although pub lication bias is well establ ished as a threat to the validity of meta-analysis, the impact that publication bias has on the results of metaanalysis is unknown. Furthermore, it is appare nt that the development and application of statistical methods to detect publication bias have been dominated by the medical field. Finally, the examination of the performance of these statistical methods under conditions similar to those in the field of education and psychology is needed.

PAGE 70

59 CHAPTER THREE: METHOD The methods employed in this study, in cluding the purpose, research questions, sample, procedures, and data analyses are detailed in this chapter. Purpose There were two primary goals overarching this research endeavor: (1) examine the degree to which publication bias impacts th e results of a random effects meta-analysis and (2) investigate the performance of five statistical methods for detecting publication bias in random effects meta-a nalysis. First, the amount of impact on the meta-analysis results was estimated by examining the diffe rence between the populat ion effect size and the estimated meta-analysis mean effect size. Similarly, the impact on the meta-analysis results was estimated by examining the di fference between the population effect size variance and the estimated meta -analysis effect size variance. Second, the performance of the five statistical methods was estimated w ith Type I error rates and statistical power. This research expands on the study conduc ted by Rendina-Gobioff and Kromrey (2004) by examining the performance of the statistical methods for de tecting publication bias in a random effects model, rather than a fi xed effects model. The following research questions were of interest:

PAGE 71

60 Research Questions 1. To what extent does publication bias imp act the estimated mean effect size and estimated variance in a random effects meta-analysis? a. To what extent does the number of prim ary studies included in the meta-analysis moderate the impact of pub lication bias on the estimated mean effect size and variance calculated for the meta-analysis? b. To what extent does the mean sample size of groups (including balanced and unbalanced) in the primary studies include d in the meta-analysis moderate the impact of publication bias on the es timated mean effect size and variance calculated for the meta-analysis? c. To what extent do the group variances ( homogeneous and heterogeneous) in the primary studies included in the meta-analy sis moderate the impact of publication bias on the estimated mean effect size and variance calculated for the metaanalysis? d. To what extent does the magnitude of the population effect size moderate the impact of publication bias on the es timated mean effect size and variance calculated for the meta-analysis? e. To what extent does the variance of the population effect size moderate the impact of publication bias on the es timated mean effect size and variance calculated for the meta-analysis? 2. To what extent do Type I e rror rates vary acro ss statistical methods for detecting publication bias in a random effects meta-analysis?

PAGE 72

61 a. To what extent does the number of prim ary studies included in the meta-analysis impact the extent that Type I error ra tes vary across sta tistical methods for detecting publication bias? b. To what extent does the mean sample size of groups (including balanced and unbalanced) in the primary studies included in the meta-analysis impact the extent that Type I error rates va ry across statistical methods for detecting publication bias? c. To what extent do the group variances ( homogeneous and heterogeneous) in the primary studies included in the meta-analysis impact the extent that Type I error rates vary across statistical met hods for detecting publication bias? d. To what extent does the magnitude of th e population effect size impact the extent that Type I error rates va ry across statistical methods for detecting publication bias? e. To what extent does the variance of the population effect size impact the extent that Type I error rates va ry across statistical methods for detecting publication bias? 3. To what extent do power estimates vary across statistical me thods for detecting publication bias in a random effects meta-analysis? a. To what extent does the number of prim ary studies included in the meta-analysis impact the extent that power estimates vary across statistical methods for detecting publication bias?

PAGE 73

62 b. To what extent does the mean sample size of groups (including balanced and unbalanced) in the primary studies included in the meta-analysis impact the extent that power estimates vary across statis tical methods for detecting publication bias? c. To what extent do the group variances ( homogeneous and heterogeneous) in the primary studies included in the meta-analysis impact the extent that power estimates vary across statistical me thods for detecting publication bias? d. To what extent does the magnitude of th e population effect size impact the extent that power estimates vary across statis tical methods for detecting publication bias? e. To what extent does the variance of the population effect size impact the extent that power estimates vary across statis tical methods for detecting publication bias? Sample Conceptually the sample has two levels : (1) primary study samples and (2) metaanalysis samples. However, the samples that directly contribute to the investigation of the statistical methods for dete cting publication bias are th e meta-analyses samples. Primary Studies The first level of the sample consiste d of generating primary study observations which produced primary study statistics (sampl e size, effect size, p-value). The number of primary studies simulated per meta-analysi s (k=10, 20, 50, and 100) was controlled as one of the simulation conditions The application of publication bias was administered to

PAGE 74

63 the primary studies before the meta-analysis stat istics were calculated. The final results of the first sample level were the primary st udy statistics and the meta-analysis summary statistics (mean effect size a nd variance of effect sizes). Meta-Analyses The data that resulted from the first le vel make up the second level of the sample, meta-analyses. Each of the simulation conditi ons for the first leve l were generated 10,000 times. Thus, for each condition there were 10,000 meta-analyses contributing to the evaluation of the publication bias statistical methods. Procedures The procedures followed a three stage fr amework. The first stage related to the primary studies and had three st eps: (1) generate observations for the primary studies, (2) compute effect sizes from primary studies and (3) impose publication bias. The six Monte Carlo study factors controlled within th is stage were: (a) the number of primary studies in each meta-analysis (10, 20, 50, a nd 100), (b) the sample sizes of the two groups in each primary study (with mean total sample sizes ranging from 10 to 100, as well as balanced and unbalanced conditions), (c) group variances in the prim ary studies (variance ratios of 1:2, 1:4, and 1:8, as well as a homogeneous varian ce condition), (d) the magnitude of the population effect size ( = 0.00, 0.20, 0.50, 0.80), (e) the variance of the population effect size (2 = 0, .10, .33, .50, and 1.00), and (f) the magnitude of the publication bias (no bias, moderate bias, and strong bias). Please see Appendix A for the code created for the simulation and calculation of detection methods. The results of the first stage (primary st udy data) were used in the second stage, which had two steps. The first step invol ved the computation of the meta-analysis

PAGE 75

64 summary statistics. This step was followed by the application of each of the publication bias detection methods to each of the meta-analyses. The final stage was to evaluate the imp act of publication bias and the performance of the publication bias detection methods. The stages and steps invol ved in the procedure are graphically presented in Figure 6. Step 1: Generate Observations for the Primary Studies (Number of Primary Studies, Sample Sizes of Two Groups, Group Variances, Magnitude of the Population Effect Size, Variance of Population Effect Size) Step 2: Calculate Primary Study Statistics (p-value, Hedges' g Effect Size) Step 3: Impose Publication Bias (Function Based on p-values from Primary Studies) Step 1: Calculate Meta-Analysis Statistics (Variance of Effect Sizes, Estimated Mean Effect Size) Step 2: Application of Publication Bias Detection Methods (Begg's Rank Correlation, Egger's Regression, Funnel Plot Regression, and Trim and Fill) Step 1: Evaluate the Simulation Results (Impact of Publication Bias and Publication Bias Detection Method Performance) Stage 1 : Primary Studies Stage 2: MetaAnalysis Stage 3 : Evaluation Repeated 10,000 Times (for each condition) Figure 6. Flowchart depicting stages and steps in the Monte Carlo design.

PAGE 76

65 Primary Study Generation Primary studies were simulated using a Monte Carlo design w ith several factors controlled. There were primar y study characteristics that were controlled, as well as, population characteristics. Primary Studies: Number The number of primary studies in each meta -analysis had four levels (10, 20, 50, and 100). These levels were chosen as re presenting those frequently observed in psychological and educational meta-analyses. Lipsey and Wilson (1993) investigated 317 meta-analyses published in the psychology and education fields. Interested in describing the number of studies included in the meta-analyses investigated by Lipsey and Wilson (1993) the researcher conducted an analysis of the number of studies reported by Lipsey and Wilson (1993). The psychology meta-analyse s, total of 136, ranged from 5 to 475 studies included, with a median of 37.5 (25th percentile = 20 and 75th percentile=64.5). The educational meta-analyses, total of 181, ranged from 6 to 302, with a median of 41.0 (25th percentile = 22 and 75th percentile=71). Based on the information gleaned from the Lipsey and Wilson (1993) data, the number of studies incl uded in this Monte Carlo simulation was appropriate. Primary Studies: Sample Sizes of the Two Groups The sample sizes of the two groups in each primary study had mean total sample sizes ranging from 10 to 100. The sample size standard deviation was half of the mean sample size. For example, when n1=5 and n2=5 the mean total sample size is 10 with a standard deviation of 5. Specifically, half of the value 10 (n1 + n2) would be multiplied by the random number and then added to 10. In addition there were balanced

PAGE 77

66 (homogeneous) and unbalanced (heterogeneous) conditions. A total of 9 levels for the sample sizes of the primary studies were controlled. Tabl e 5 details the mean sample sizes for each group. Table 5. Controlled primary study mean sample sizes for each group. Homogeneous Heterogeneous Group 1, Group 2 SD Group 1, Group 2 Group1, Group 2 5, 5 2.5 4, 6 6, 4 10, 10 5.0 8, 12 12, 8 50, 50 25.0 40, 60 60, 40 Primary Studies: Group Variances The group variances in the primary studies had the following va riance ratios: 1:2, 1:4, and 1:8, as well as a ho mogeneous variance condition. Population Effect Size: Magnitude The magnitude of the population effect size was controlled with 0.00, 0.20, 0.50, and 0.80 values. This range of population effect sizes is often described as none, small, medium, and large (Cohen, 1992). Population Effect Size: Variance The variance of the population effect size wa s controlled with 0, .10, .33, .50, and 1.00 values. These values control the degree of random effects vari ance contributing to the meta-analysis weights. When the variance of the population effect size is zero, the calculations for the mean effect size produce si milar weights as the fixed effects model. However, all values greater than zero will in crease the amount of variance contributing to the weights by contributing random effects variance to the weights. These values correspond to values others have us ed in simulations (Field, 2001).

PAGE 78

67 Following the generation of the primary studies, step 1 in stage 1 of the simulation, the effect sizes were computed, a nd publication bias was imposed (steps 2-3). The computation of the mean effect sizes for each primary study was Hedges's g, which is appropriate for standardized mean differe nces (Hedges, 1981). The third step was to select the primary studies for inclusion in the meta-analysis, to impose publication bias, which is one of the Monte Carl o factors being controlled. Selection of Primary Studies: Magnitude of Publication Bias The magnitude of the publication bias had three levels: no bias, moderate bias, and strong bias. The publication bias in the meta-analyses was manipulated by generating a weight function to establis h the probability of a simula ted study being included in the meta-analysis, with the probability of in clusion being inversely related to the p -value obtained in the simulated study (Begg & Mazu mdar, 1994). That is, samples that yield statistically significant results are more likely to be published than those that do not yield statistically significant results (Sutton et al., 2000). The weight functi ons are illustrated in Figure 7. In a similar study to the one bei ng proposed, Rendina-Gobioff and Kromrey (2004) implemented the same method for select ion of studies to be included in metaanalyses. The average proportion of studies se lected when the imposed publication bias was moderatel was .66 (min=.49 and max=.97) When the imposed publication bias was strong the average proportion reporte d was .54 (min=.31 and max=.95).

PAGE 79

68 00 01 02 03 04 05 06 07 08 09 10 0001020304050607080910 p-valueWeight Moderate Bias exp(-2p**15) Strong Bias exp(-4p**15) Figure 7. Weight functions to establish probability of inclusion in a meta-analysis as a function of the p-value. The following primary study data were the product of stage 1: total sample sizes, effect sizes, and p-values. The first step in the second stage of the simulation was to calculate the metaanalysis statistics. The variance of the effect sizes was calculated and the mean effect size was estimated using random effects weights (sampling error and be tween study variance). Publication Bias Tests Applied The next step in stage 2 of the simulati on was the application of the statistical methods to detect publication bias. In each si mulated meta-analysis, publication bias was assessed using the Begg Rank Correlation met hod (using both the estimated variance and the sample size), the Egger Regression me thod, the Funnel Plot Regression method

PAGE 80

69 suggested by Macaskill, Walter and Irwig ( 2001) and the Trim and Fill method suggested by Duval and Tweedie (2000a, 2000b). For the Egger Regression, OLS estimation was employed because preliminary analyses suggested that WLS estimates were substantially biased (Rendina-Gobioff & Kromrey, 2004). The following meta-analysis data were the product of stage 2: variance of effect sizes, mean effect size, and whether publica tion bias was detected (for each of the methods). The estimate for the variance of the effect sizes or random effects variance component (REVC) (Shadish & Ha ddock, 1994) was estimated with: 2 21 /iiiQk www Where Q is the estimate of homogeneity detailed earlier kis the number of studies included in the meta-analysis iw is the inverse variance weight used in a fixed effects model The REVC was set to 0 when computationall y negative because conceptually it cannot be negative (Hedges & Vevea, 1998).The mean effect size was estimated with a random effects weight that combines the REVC w ith sampling error variance. Specifically, the weight was the inverse of the sum of the REVC and sampling error variance. In this case, for Hedges's g effect size, the sampling erro r variance was estimated with the following: 2 12 12122i ig nn SE nnnn

PAGE 81

70 Where ig is the estimated effect size for study i 1n is the sample size for group 1 2n is the sample size for group 2 The final stage (3) was to evaluate the impact of publicati on bias on estimated mean effect sizes and effect size varian ces. In addition, the performance of the publication bias methods implemented was eval uated. These evaluations are discussed in detail in the data analysis section. Programming This research was conducted using SAS/ IML version 9.1. Conditions for the study were run under Windows platforms. Norma lly distributed random variables were generated using the RANNOR random number generator in SAS. A different seed value for the random number generator was used in each execution of the program. The program code was verified by hand-check ing results from benchmark datasets. For each condition examined in the Monte Carlo study, 10,000 meta-analyses were simulated. The use of 10,000 samples provides a maximum 95% confidence interval width around an observe d proportion that is 0.0098 (Robey & Barcikowski, 1992). Data Analysis Research Question 1: Evaluation of the Impact of Publication Bias Research Question 1, evaluation of the impact of publication bias on the metaanalysis results, was addressed by examini ng the mean difference between the population effect size and the meta-analysis estimated m ean effect size. In addition, the difference between the population effect si ze variance and the meta-analysis estimated effect size variance was investigated. These differences were averaged over all conditions and

PAGE 82

71 averaged separately for each condition (number of primary studies, mean sample size of groups, group variances, magnitude of popula tion effect size, a nd variance of the population). Research Question 2 and 3: Evaluation of Methods to Detect Publication Bias Research Questions 2 and 3, evaluati ng the performance of the methods for detecting publication bias, were addresse d by examining the Type I error rates and estimated power. The proportion of simulated meta-analyses for which publication bias was indicated by each detection method was calculated. For simulation conditions with no publication bias, these proportions repres ent estimates of Type I error rates. Conversely, for conditions that included pub lication bias, these proportions estimate the statistical power. Hence, for Research Ques tion 2 the Type I error rates were averaged over all conditions and averaged separately for each condition (number of primary studies, mean sample size of groups, group variances, magnitude of population effect size, and variance of the population effect si ze). In addition, the proportion of studies with adequate Type I error rates for each condition and across a ll conditions were calculated. Type I error rate s that ranged from 0.025 to 0.075 were considered to be adequate based on Bradley’s (1978 ) liberal robustn ess criteria. Maximum power estimates over all conditi ons and separately for each condition (number of primary studies, mean sample size of groups, group va riances, magnitude of population effect size, and va riance of the population effect size) were calculated for Research Question 3. The maximum power estimat es were reported, rather than averages, because some power estimates were not reporte d due to inadequate Type I error control.

PAGE 83

72 CHAPTER FOUR: RESULTS This chapter details the results of the study in relation to each of the research questions. The chapter is organized in the order of the research questions. However, before presenting results for each of the re search questions the proportion of studies included in each of the meta-analyses by study conditions is presented. Next the first research question is addressed by presenting th e degree that publication bias impacted the results of the meta-analyses. This secti on is followed by details regarding the second research question, the Type I error rates of each of the statistical methods to detect publication bias for each study condition. The fi nal section presents the power estimates for each of the statistical methods across st udy conditions, attending to the third research question. The following research ques tions are addressed by the results: Research Questions 1. To what extent will publi cation bias impact the estim ated mean effect size and estimated variance in a random effects meta-analysis? a. To what extent does the number of prim ary studies included in the meta-analysis moderate the impact of pub lication bias on the estimated mean effect size and variance calculated for the meta-analysis?

PAGE 84

73 b. To what extent does the mean sample size of groups (including balanced and unbalanced) in the primary studies include d in the meta-analysis moderate the impact of publication bias on the es timated mean effect size and variance calculated for the meta-analysis? c. To what extent do the group variances ( homogeneous and heterogeneous) in the primary studies included in the meta-analy sis moderate the impact of publication bias on the estimated mean effect size and variance calculated for the metaanalysis? d. To what extent does the magnitude of the population effect size moderate the impact of publication bias on the es timated mean effect size and variance calculated for the meta-analysis? e. To what extent does the variance of the population effect size moderate the impact of publication bias on the es timated mean effect size and variance calculated for the meta-analysis? 2. To what extent do Type I e rror rates vary acro ss statistical methods for detecting publication bias in a random effects meta-analysis? a. To what extent does the number of prim ary studies included in the meta-analysis impact the extent that Type I error ra tes vary across sta tistical methods for detecting publication bias? b. To what extent does the mean sample size of groups (including balanced and unbalanced) in the primary studies included in the meta-analysis impact the extent that Type I error rates va ry across statistical methods for detecting publication bias?

PAGE 85

74 c. To what extent do the group variances ( homogeneous and heterogeneous) in the primary studies included in the meta-analysis impact the extent that Type I error rates vary across statistical met hods for detecting publication bias? d. To what extent does the magnitude of th e population effect size impact the extent that Type I error rates va ry across statistical methods for detecting publication bias? e. To what extent does the variance of the population effect size impact the extent that Type I error rates va ry across statistical methods for detecting publication bias? 3. To what extent do power estimates vary across statistical me thods for detecting publication bias in a random effects meta-analysis? a. To what extent does the number of prim ary studies included in the meta-analysis impact the extent that power estimates vary across statistical methods for detecting publication bias? b. To what extent does the mean sample size of groups (including balanced and unbalanced) in the primary studies included in the meta-analysis impact the extent that power estimates vary across statis tical methods for detecting publication bias? c. To what extent do the group variances ( homogeneous and heterogeneous) in the primary studies included in the meta-analysis impact the extent that power estimates vary across statistical me thods for detecting publication bias?

PAGE 86

75 d. To what extent does the magnitude of th e population effect size impact the extent that power estimates vary across statis tical methods for detecting publication bias? e. To what extent does the variance of the population effect size impact the extent that power estimates vary across statis tical methods for detecting publication bias? Impact of Publication Bias The number of studies included in each meta-analysis is pertinent to the interpretation of results. To simulate the influence of pub lication bias the proportion of studies available for the meta-analysis wa s manipulated by imposing a weight function (see Figure 7). The average pr oportion of studies available fo r the meta-analysis due to publication bias for each condition is presented in Table 6. The proportions of studies that were available for the meta-analyses verify th e accuracy of the code used to simulate the study conditions. For example, if the func tion used to impose publication bias was written into the code accurately one would e xpect more studies to be available when the publication bias was moderate (73%), comp ared to strong publication bias (62%). Furthermore, when there was no publication bias imposed 100% of the studies were available for the meta-analyses. When moderate publication bias was imposed the minimum proportion available to the meta-ana lysis was 49%, compared to the maximum proportion available to the meta-analysis which was 97%. When the publication bias strength was greater (strong) the minimum proportion of studies available to the metaanalysis was 31% and the maximum was 95% Thus a researcher conducting a meta-

PAGE 87

76 analysis under conditions similar to the si mulated strong publication bias in the worst case scenario could be producing results ba sed on only 31% of th e studies conducted on the topic. All of the proportions of studies availabl e to the meta-analysis increase as the study condition values increase, except for the number of studies (k) condition. The average proportion of studies available for me ta-analyses was the same (67%) across the number of studies included (k). The minimum proportion available, regardless of the number of studies included was 31% and the maximum was 97%. The increasing proportion of studies available for the me ta-analysis as the other study conditions (primary study sample si ze, primary study group variance, population effect size magnitude, and populat ion effect size variance) va lues increase was expected because all of these conditions affect the power available to detect a significant result (pvalue) which is what the publication bias function is based on for determining study inclusion. Thus, as the primary study sample size increased the proportion of studies included in the meta-analysis increased. Th e average proportion of studies available across sample sizes ranged from 57% (n1=4, n2=6) to 79% (n1=60, n2=40). The smallest minimum proportion of studies available ac ross sample sizes was 31% (n1=40, n2=60) and the largest maximum proportion of studi es available was 97% (n1=50, n2=50). The average proportion of studies available for th e meta-analyses slightly increased as the primary study group variances in creased (1:1, 67% to 1:8, 68%). The smallest minimum proportion of studies available across gr oup variance was 31% (1 :8) and the largest maximum proportion of studies available was 97% (1:8). Bo th the population effect size

PAGE 88

77 Table 6. Proportion of studies availabl e for meta-analyses by study conditions. NMeanMinimumMaximum Number of Studies (k) 1014400.67040.31230.9707 2014400.67040.31200.9706 5014400.67040.31280.9702 10014400.67040.31240.9705 Primary Study Sample Size (n1, n2) 4,66400.56810.32580.7473 5,56400.58630.35470.7558 6,46400.60160.35490.7727 8,126400.62530.31670.8005 10,106400.64350.35540.8084 12,86400.65620.35500.8208 40,606400.77480.31200.9687 50,506400.78600.35510.9707 60,406400.79180.35400.9699 Primary Study Group Variance 1:114400.66680.35400.9676 1:214400.66820.33390.9684 1:414400.67150.32060.9695 1:814400.67510.31200.9707 Population Effect Size Magnitude 0.014400.60000.31200.8756 0.214400.62030.34350.8795 0.514400.69230.43020.8962 0.814400.76900.55740.9707 Population Effect Size Variance 0.0011520.60080.31200.9707 0.1011520.63150.36660.9377 0.3311520.67540.43700.9042 0.5011520.69880.47610.9007 1.0011520.74540.55360.9066 Magnitude of Publication Bias None28801.00001.00001.0000 Moderate28800.72570.48780.9707 Strong28800.61510.31200.9539 Total57600.67040.31200.9707 magnitude and variance produced increasing proportions of available studies as their values increased. The population effect si ze magnitude average proportion of studies available increased from 60% ( =0.0) to 77% ( =0.8). Similarly, the population effect size variance average propor tion of studies available increased from 60% (2 =0.0) to

PAGE 89

78 75% (2 =1.0). The smallest minimum proportion of studies available across population effect sizes and population eff ect size variance was 31% ( =0.0 and 2 =0.0) and the largest maximum proportion of studies available was 97% ( =0.8 and 2 =0.0). One indicator of the impact of publication bias on the results of a meta-analysis is the degree that the estimated mean effect si ze varies from the population effect size or effect size bias. Table 7 presents the averag e, minimum, and maximu m effect size bias by study conditions. The effect size bias distribution shapes were investigated for normality. The distributions were skewed however the ov erall conclusions pres ented here would not be different if the medians were reported, rath er than the means. The effect size bias was calculated by subtracting the population effect size from the estimated mean effect size. Thus the total average effect size bias valu e of 0.1071 indicates th at on average (across all study conditions when publication bias was present) the estimated mean effect size was 0.1071 greater than the population effect si ze. In other words, when publication bias is present the estimated mean effect size was inflated by 0.1071. The effect size bias when no publication bias was imposed can be used as a baseline for comparison to the other study c onditions. When there is no publication bias one would expect to have minimal effect size bias. The results reflect this assumption because when no publication bias was imposed the average effect size bias was -0.0120 with a minimum value of -0.1125 and maximum value of 0.0997. In contrast, the average effect size bias increased to 0.0792 when m oderate publication bias was imposed, with a minimum value of -0.0327 and maximum value of 0.3030. The average effect size bias increased even more when the imposed publication bias was strong, 0.1350

PAGE 90

79 (minimum= -0.0295 and maximum=0.4491). A ccording to these results when a researcher is conducting a meta-analysis w ith strong publication bias they could be producing an average effect size with as much as 0.45 error. Table 7. Estimated bias in mean effect size by study conditions. NMeanMinimumMaximum Number of Studies (k) 1014400.1088-0.02710.4491 2014400.1073-0.03030.4417 5014400.1064-0.03200.4414 10014400.1060-0.03270.4392 Primary Study Sample Size (n1, n2) 4,66400.1275-0.00590.3100 5,56400.1421-0.00880.3649 6,46400.1621-0.01110.4491 8,126400.1024-0.00480.2445 10,106400.1164-0.00810.2863 12,86400.1377-0.00980.3754 40,606400.0437-0.03270.1254 50,506400.0569-0.00580.1632 60,406400.0754-0.00700.2299 Primary Study Group Variance 1:114400.0965-0.00760.3100 1:214400.1001-0.02020.3400 1:414400.1097-0.03030.3905 1:814400.1222-0.03270.4491 Population Effect Size Magnitude 0.014400.0001-0.01110.0085 0.214400.09590.01950.2245 0.514400.1659-0.00090.4104 0.814400.1666-0.03270.4491 Population Effect Size Variance 0.0011520.1021-0.03270.4280 0.1011520.1124-0.00810.4419 0.3311520.1145-0.00740.4491 0.5011520.1106-0.00980.4449 1.0011520.0961-0.01110.4099 Magnitude of Publication Bias None2880-0.0120-0.11250.0997 Moderate28800.0792-0.03270.3030 Strong28800.1350-0.02950.4491 Total57600.1071-0.03270.4491 Average Effect Size Bias

PAGE 91

80 Another indicator of the influence of pub lication bias on the results of a metaanalysis is the degree that th e estimated effect size vari ance varies from the population effect size variance. The effect size varian ce bias by study conditions are presented in Table 8. The effect size variance bias distri bution shapes were investigated for normality. The distributions were skewed however the ov erall conclusions pres ented here would not be different if the medians were reported, ra ther than the means. The effect size variance bias was calculated by subtra cting the population effect size variance from the estimated effect size variance. Thus th e total average eff ect size variance bi as value of 0.1576 indicates that on average (across all study condi tions when publication bias is present) the estimated effect size variance was 0.1576 greater than the population effect size variance. In other words when publication bias was pr esent the estimated effect size variance was inflated by 0.1576. The effect size variance bias when no publication bias was imposed can be used as a baseline for comparison to the other study conditions. When there is no publication bias one would expect to have minimal effect size variance bias. The results reflect this assumption because when no publication bias was imposed the average effect size variance bias was -0.0343 with a minimu m value of -0.3806 and maximum value of 0.2756. In contrast, the average effect si ze variance bias increased to 0.1101 when moderate publication bias was imposed, w ith a minimum value of -0.1593 and maximum value of 0.7757. The average effect size vari ance bias increased even more when the imposed publication bias was strong, 0. 2052 (minimum=-0.1413 and maximum=1.1622).

PAGE 92

81 According to these results when researcher s are conducting a meta-a nalysis with strong publication bias they could be producing an average effect size variance estimate with as much as 1.16 error. Table 8. Estimated bias in eff ect size variance by study conditions. NMeanMinimumMaximum Number of Studies (k) 1014400.1788-0.13771.1622 2014400.1589-0.14791.0971 5014400.1482-0.15921.0661 10014400.1447-0.15931.0508 Primary Study Sample Size (n1, n2) 4,66400.1786-0.14780.5639 5,56400.2692-0.10000.7987 6,46400.3937-0.09511.1622 8,126400.0890-0.14320.3506 10,106400.1518-0.09180.5232 12,86400.2426-0.08880.8256 40,60640-0.0039-0.15930.1116 50,506400.0271-0.10820.1633 60,406400.0706-0.10550.3430 Primary Study Group Variance 1:114400.1003-0.10820.5274 1:214400.1194-0.14780.6769 1:414400.1709-0.15930.9142 1:814400.2399-0.15701.1622 Population Effect Size Magnitude 0.014400.2643-0.08801.1622 0.214400.2286-0.09571.1534 0.514400.1165-0.12080.9752 0.814400.0211-0.15930.7101 Population Effect Size Variance 0.0011520.13330.00020.9100 0.1011520.1413-0.07920.9615 0.3311520.1693-0.10621.0531 0.5011520.1810-0.09651.1092 1.0011520.1633-0.15931.1622 Magnitude of Publication Bias None2880-0.0343-0.38060.2756 Moderate28800.1101-0.15930.7757 Strong28800.2052-0.14131.1622 Total57600.1576-0.15931.1622 Average Variance Bias

PAGE 93

82 The following section presents results in relation to the influence of moderators on the impact of publication bias when estima ting the effect size magnitude and variance. The term moderator is used in the sense of describing the influence of a particular indicator on the results of a meta-analysis (e ffect size magnitude and effect size variance bias) when publication bias is present. Thus a study condition is considered a moderator if for different values of the study condition the results of a meta-analysis (effect size magnitude and effect size variance bias) ar e larger/smaller when publication bias is present (both moderate and strong). Altern atively, one could examine a moderator by determining whether the results of a meta-ana lysis (effect size magnitude and effect size variance bias) are different across values of the study condition and across each value of publication bias (none, moderate, and strong) For this type of moderator investigation one would calculate the mean for each study conditi on value and each publication bias value. Please see Appendix B for the means, minimu m, and maximum calculations associated with this method. Number of Primary Studies Moderator The degree to which the number of pr imary studies moderates the impact of publication bias on the estimation of the effect size magnitude and variance was minimal. The average effect size bias decreased sli ghtly ranging from 0.1088 with 10 studies to 0.1060 with 100 studies (see Table 9). The aver age effect size variance bias decreased slightly as the number of st udies included in the meta-a nalysis increased, from 0.1788 with 10 studies in the meta-analysis to 0.1447 with 100 studies in the meta-analysis. The minimal moderating effect of the number of primary studies on the impact of publication

PAGE 94

83 bias should be couched with the knowledge that the proportion of studies available (see Table 6) for the meta-analyses was similar across this condition (k=10, 20, 50, 100). Sample Size Moderator The average primary study sample size does moderate the impact of publication bias on the estimation of the effect size ma gnitude and variance. Under conditions where the average primary study sample size was 5 the average effect size bias ranged from 0.1275 (n1=4, n2=6) to 0.1621 (n1=6, n2=4). In contrast, conditions where the average primary study sample size was 50 the average effect size bias was smaller, ranging from 0.0437 (n1=40, n2=60) to 0.0754 (n1=60, n2=40). The average effect size variance bias exhibited a similar pattern to the average effect size magnitude bias. Under conditions where the average primary study sample size wa s 5 the average effect size variance bias ranged from 0.1786 (n1=4, n2=6) to 0.3937 (n1=6, n2=4). In contrast, conditions where the average primary study sample size was 50 the average effect size variance bias was smaller, ranging from -0.0039 (n1= 40, n2=60) to 0.0706 (n1=60, n2=40). Group Variance Moderator The moderating influence of the primar y study group variances on the impact of publication bias was that as the group variances increased, the bias of the effect size magnitude and variance increased (became less a ccurate). Specifically, the average effect size bias increased from 0.0965 (1:1) to 0. 1222 (1:8) as the group variance increased. Similarly, the average effect size variance bias increased from 0.1003 (1:1) to 0.2399 (1:8) as the group variance increased.

PAGE 95

84 Magnitude of Population Effect Size Moderator The magnitude of the population effect si ze influenced the two indicators of the impact of publication bias (average effect size bias and effect size variance) differently. As the value of the population e ffect size increased the average effect size bias increased. Specifically, the average effect si ze bias increased from 0.0001 ( =0.0) to 0.1666 (=0.8). In contrast, as the value of the population effect size in creased the average effect size variance bias decreased. Specifica lly, the average effect size variance bias decreased from 0.2643 (=0.0) to 0.0211 ( =0.8). Variance of Population E ffect Size Moderator The degree that the variance of the populat ion effect size moderates the impact of publication bias on the estimation of the effect size magnitude and variance was minimal. Specifically, the average effect size bias ranged from 0.0961 with population effect size variance of 1.00 to 0.1145 with population eff ect size variance of 0.33. The average effect size variance bias ra nged from 0.1333 with population effect size variance of 0.00 to 0.1810 with population variance of 0.50. Summary Two of the study conditions minimally m oderated indicators of the impact of publication bias, number of primary studies and population effect size variance. In addition, these two factors minimally impacted the percentage of studies available to the meta-analyst. The average primary study sample size increased the accu racy of the effect size magnitude and variance estimates by d ecreasing bias when publication bias was imposed. In contrast, the primary study gr oup variances decreased the accuracy of the effect size magnitude and variance estimates by increasing bias when publication bias

PAGE 96

85 was imposed. Finally, the population effect si ze magnitude decreased the accuracy of the effect size magnitude estimate yet increased the accuracy of the effect size variance estimate. Type I Error Rates of Methods to Detect Publication Bias One indicator of the performance of stat istical methods for detecting publication bias investigated was the Type I error rates. Two approaches to describing the Type I error rates are presented in this research. The first approach presented is the average Type I error rates for each study condition and acro ss all conditions (see Table 9). The average Type I error rate for Begg Rank Correlation (N) method exhibited the best performance with an average close to the nominal va lue of 0.05 (M=0.0556). The Funnel Plot Regression and Trim and Fill methods’ average Type I error rates were smaller than the nominal 0.05 value; M=0.0338 and M=0.0267. The two statistical methods with poor average Type I error rates, greater than the nominal 0.05 value, were Egger Regression (M=0.2199) and Begg Rank Correlation (V) (M=0.1841).

PAGE 97

86 Table 9. Average Type I error rates for methods to detect publication bias by study conditions. N Begg Rank Correlation (V) (Begg V) Begg Rank Correlation (N) (Begg N) Egger's Regression (Egger) Funnel Plot Regression (Funnel) Trim and Fill (Trim) Number of Studies (k) 107200.09690.06490.06960.01150.0000 207200.11920.05400.16330.03130.0029 507200.20630.05130.27720.04360.0341 1007200.31410.05210.36930.04890.0698 Primary Study Sample Size (n1, n2) 4,63200.24500.05640.31380.02540.0386 5,53200.25440.05600.32990.02800.0386 6,43200.27530.05780.35680.03220.0375 8,123200.15070.05530.18870.03050.0255 10,103200.16580.05370.20840.03130.0259 12,83200.17950.05490.22840.03340.0252 40,603200.11800.05750.10990.04300.0160 50,503200.13040.05380.11860.04010.0166 60,403200.13800.05460.12430.04040.0165 Primary Study Group Variance 1:17200.17200.05410.17490.03180.0204 1:27200.17670.05470.18940.03250.0225 1:47200.18800.05600.22990.03420.0281 1:87200.19980.05750.28540.03680.0357 Population Effect Size Magnitude 0.07200.05980.05410.10880.03250.0079 0.27200.08570.05430.13780.03270.0137 0.57200.21520.05580.25450.03400.0295 0.87200.37580.05810.37840.03610.0556 Population Effect Size Variance 0.005760.13760.05630.15530.02320.0385 0.105760.15710.05540.18420.03050.0319 0.335760.19020.05530.22380.03480.0244 0.505760.20590.05520.24510.03720.0215 1.005760.22980.05560.29090.04330.0171 Total28800.18410.05560.21990.03380.0267 Publication Bias Detection Method

PAGE 98

87 The second approach to describing the Type I error rates was the proportion of conditions with adequate Type I error cont rol. Specifically, Type I error control was based on Bradley’s (1978) liberal robustness criterion which states that Type I error rates at the nominal 0.05 level should have half of added to and subtracted from 0.05 thus a range of 0.025 to 0.075 is considered adequate. The proportions of conditions with adequate Type I error cont rol for the methods to dete ct publication bias by study conditions are presented in Ta ble 10. Consistent with the average Type I error rate performance, the Begg Rank Correlation (N ) method had the greatest proportion of conditions with adequate Type I error contro l with 99% meeting th e criterion. The Funnel Plot Regression method exhibited the second best proportion of adequate Type I error rates with 67% of the conditions performing well. Although the Trim and Fill method exhibited average Type I error rates below the 0.05 value, the Type I error rates are too small to be considered adequate, thus only 17% of the conditions met the criteria. In contrast, the Begg Rank Correlation (V) and Egger Regression me thods exhibited poor proportions of Type I error control with 38% and 24% meeting criteria. However, this is consistent with high average Type I error rates which were well above the 0.05 level. Since there may be researchers who woul d like to look up Type I error rates for specific conditions, the Type I error rate s for every condition are presented in Appendix C.

PAGE 99

88 Table 10. Proportion of conditions with adequate Type I er ror for methods to detect publication bias by study conditions. N Begg Rank Correlation (V) (Begg V) Begg Rank Correlation (N) (Begg N) Egger's Regression (Egger) Funnel Plot Regression (Funnel) Trim and Fill (Trim) Number of Studies (k) 107200.39030.99440.51390.04440.0000 207200.46250.99860.24440.73060.0000 507200.36390.99440.12080.94720.2944 1007200.29860.97640.08610.94030.3819 Primary Study Sample Size (n1, n2) 4,63200.18750.99380.13440.54690.1469 5,53200.19381.00000.11250.62190.1500 6,43200.19690.98130.09690.64690.1469 8,123200.43441.00000.28750.65000.2094 10,103200.40631.00000.24060.68440.1969 12,83200.37810.99690.22810.70310.2000 40,603200.57190.96560.40000.68750.1531 50,503200.53441.00000.34380.72190.1625 60,403200.50630.98130.32810.72810.1563 Primary Study Group Variance 1:17200.40971.00000.38060.66940.1625 1:27200.39860.99720.32640.66810.1792 1:47200.37220.98750.18610.66670.1792 1:87200.33470.97920.07220.65830.1556 Population Effect Size Magnitude 0.07200.91531.00000.34440.65830.0083 0.27200.45691.00000.30000.66810.2361 0.57200.10690.99310.19860.66940.2667 0.87200.03610.97080.12220.66670.1653 Population Effect Size Variance 0.005760.51220.97220.35590.43580.1979 0.105760.41670.98790.29690.64580.1736 0.335760.34900.99830.25350.73440.1563 0.505760.32471.00000.20660.75690.1545 1.005760.29170.99650.09380.75520.1632 Total28800.37880.99100.24130.66560.1691 Publication Bias Detection Method

PAGE 100

89 Number of Primary Studies Impact The number of primary studies included in the meta-analysis did not impact the Type I error rates of th e Begg Rank Correlation (N), which ranged from 0.0649 to 0.0513. The Funnel Plot Regression and Trim and Fill methods exhibited small increases in average Type I error rates. The Funnel Plot Regression averag e Type I error rates increased from 0.0115 to 0.0489 and the Trim and Fill average Type I error rates increased from 0.000 to 0.0698. The Begg Ra nk Correlation (V) and Egger Regression methods average Type I error rates also increased as the number of primary studies included in the meta-analysis increased howev er the increase is greater. The Begg Rank Correlation (V) method average Type I erro r rates increased from 0.0969 (k=10) to 0.3141 (k=100). Similarly, the Egger Regre ssion method average Type I error rates increased from 0.0696 (k=10) to 0.3693 (k=100) These Type I error rates are presented in Table 9 and Figure 8. When examining the proportion of conditi ons with adequate T ype I error control (see Table 10 and Figure 9), the Begg Rank Correlation method performed well across all of the values of the number of studies included in the me ta-analysis condition (97% to 100%). The Funnel Plot Regression performe d poorly when the number of studies included in the meta-analysis was small (k =10, 4%) but improved as the number of studies increased (k=20, 73%; k=50, 95%; and k=100, 94%). The other three methods (Begg Rank Correlation (V), Egger Regression, and Trim and Fill) exhibited proportions of adequate Type I error c ontrol around 50% or less across the values for the number of studies condition. The Trim and Fill method incr eased from 0% to 38% as the number of

PAGE 101

90 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 102050100Number of StudiesAverage Type I Error Rate Begg V Begg N Egger Funnel Trim Figure 8. Average Type I error rates for each method to detect publication bias by number of studies. 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00Begg VBegg NEggerFunnelTrim Publication Bias Detection MethodProportion with Adequate Type I Error 10 20 50 100 Figure 9. Proportion of conditions with adequate Type I erro r for each method to detect publication bias by number of studies.

PAGE 102

91 studies included in the meta-analysis increased. In contrast, the Begg Rank Correlation (V) and Egger Regression methods exhibit decl ining proportions of ad equate Type I error control as the number of studies included in the meta-analysis increases. Sample Size Impact The sample size of the primary studies included in the meta-analysis did not impact the Type I error rates of the Begg Rank Correlation (N) which ranged from 0.0537 (n1=10, n2=10) to 0.0578 (n1=6, n2=4). The average Type I error rates remained under the nominal 0.05 lever for both the Trim and Fill and the Funnel Plot Regression methods. The Trim and Fill method average Type I error rates decreased slightly as the sample size of the primary studies include d in the meta-analysis increased, M=0.0386 (n1=4, n2=6) to M=0.0165 (n1=60, n2=40). In contrast, the F unnel Plot Regression average Type I error rates increased slightly as the sample size of the primary studies included in the meta-analysis increased, M=0.0254 (n1=4, n2=6) to M=0.0404 (n1=60, n2=40). The Begg Rank Correlation (V) and E gger Regression method average Type I error rates were above the nominal 0.05 valu e. Furthermore, the average Type I error rates for the Begg Rank Correlation (V) and the Egger Regression method decreased as the average sample size increased. For ex ample, the Egger Regression method average Type I error rate decreased from M=0. 3138 with a small primary study sample size (n1=4, n2=6) to M=0.1243 with a larger primary study sample size (n1=60, n2=40). The Type I error rates for the methods to detect publication bias are pres ented in Table 9 and Figure 10 by primary study sample size. The proportion of conditions with adequate Type I error control was consistently high across values of primary study sample sizes for the Begg Rank Correlation (N)

PAGE 103

92 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 4,65,56,48,1210,1012,840,6050,5060,40Sample Size (n1, n2)Average Type I Error Rate Begg V Begg N Egger Funnel Trim Figure 10. Average Type I error rates for each method to detect publication bias by sample size. 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00Begg VBegg NEggerFunnelTrim Publication Bias Detection MethodProportion with Adequate Type I Error 4, 6 5, 5 6, 4 8, 12 10, 10 12, 8 40, 60 50, 50 60, 40 Figure 11. Proportion of conditions with adequa te Type I error for each method to detect publication bias by sample size.

PAGE 104

93 method (97% when n1=40 and n2=60 to 100% under multiple values of sample size). In contrast, the Trim and Fill method exhibited consistently low proportions of conditions with adequate Type I error rates across pr imary study sample sizes (14% when n1=4 and n2=6 to 21% when n1=8 and n2=12). The Begg Rank Correlation (V), Egger Regression, and Funnel Plot Regression methods had in creasing proportions of conditions with adequate Type I error control across values of sample size, however the proportions were low. Specifically, the Begg Rank Correlation (V ) proportions with adequate Type I error control ranged from 19% (n1=4 and n2= 6) to 57% (n1=40 and n2=60) and the proportions with adequate Type I error c ontrol for the Egger Regression method ranged from 10% (n1=6 and n2=4) to 40% (n1=40 and n2=60). The proportions of conditions with adequate Type I error control by prim ary study sample size are presented in Table 10 and Figure 11. Group Variance Impact All of the average Type I error rates for the methods to detect publication bias increased as the primary study group variances become more heterogeneous. Although the Begg Rank Correlation (N) average Type I er ror rates increased slightly as the group variance increased, the values stayed cl ose to 0.05 (ranging from 0.0541 with 1:1 variance to 0.0575 with 1:8 variance). The F unnel Plot Regression and the Trim and Fill average Type I error rates remained below the 0.05 nominal level across values of primary study group variance (0.0357 for 1:8 and 0.0204 for 1:1). In contrast, the Begg Rank Correlation (V) and Egger Regression meth ods average Type I error rates were well above the nominal 0.05 value (0.1720 to 0. 2854). The pattern of av erage Type I error

PAGE 105

94 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 1:1 1:2 1:4 18Group VarianceAverage Type I Error Rate Begg V Begg N Egger Funnel Trim Figure 12. Average Type I error rates for each method to detect publication bias by primary study group variance. 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00Begg V Begg N Egger Funnel Trim Publication Bias Detection MethodProportion with Adequate Type I Error 1:1 1:2 1:4 1:8 Figure 13. Proportion of conditions with adequa te Type I error for each method to detect publication bias by prim ary study group variance.

PAGE 106

95 rates for the publication bias detection methods across primary group variances are presented in Table 9 and Figure 12. All of the methods to detect publicat ion bias had declining proportions of conditions with adequate Type I error cont rol when the primary study group variances increased in heterogeneity, except the Trim and Fill and Funnel Plot Regression methods which were stable across valu es of heterogeneity. The met hod with the best performance in regards to proportions of conditions with adequate Type I error control was the Begg Rank Correlation (N) method which ranged from 98% (Variance 1:8) to 100% (Variance 1:1). The method with the next best Type I error contro l performance across primary study group variance values was the Funnel Pl ot Regression with proportions ranging from 66% (Variance 1:8) to 67% (1:1). The proportion of studies with adequate Type I error control across values of primary study group variance did not exceed 50% for the Begg Rank Correlation (V), Egger Regre ssion, and Trim and Fill methods. The proportions of conditions with adequate Type I error control acro ss values of primary study group variance are presente d in Table 10 and Figure 13. Magnitude of Populatio n Effect Size Impact All of the average Type I error rates for the methods to detect publication bias increased as the population effect size magnitude increased. Th e Type I error rate upward trend was very small for the Begg Rank Co rrelation (N) and Funnel Plot Regression methods. The Begg Rank Correlation (N) averag e Type I error rates increased from 0.0541 (=0.0) to 0.0581 (=0.8). Similarly, the Funnel Pl ot Regression average Type I error rates increas ed from 0.0325 ( =0.0) to 0.0361 ( =0.8). The Trim and Fill

PAGE 107

96 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.0 02 0.5 08Population Effect Size MagnitudeAverage Type I Error Rate Begg V Begg N Egger Funnel Trim Figure 14. Average Type I error rates for each method to detect publication bias by population effect size magnitude. 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00Begg V Begg N Egger Funnel Trim Publication Bias Detection MethodProportion with Adequate Type I Error 00 02 05 08 Figure 15. Proportion of conditions with adequa te Type I error for each method to detect publication bias by populati on effect size magnitude.

PAGE 108

97 method Type I error rate average was we ll below the nominal 0.05 value when the population effect size was 0.0 (M=0.0079) and in creased to the nominal 0.05 value with a large effect size (M=0.0556, =0.8). Both the Begg Rank Co rrelation (V) and the Egger Regression methods had sharp increases in av erage Type I error ra tes as the population effect size increased, with values as high as 0.3758 and 0.3784 when the effect size was large (=0.8). Table 9 and Figure 14 present th e average Type I error rates for the methods to detect publication bi as by population effect size. The proportion of conditions with adequate Type I error contro l decreased as the population effect size increased with the Begg Rank Correlation (V), Begg Rank Correlation (N), and Egger Regression me thods. However, the Begg Rank Correlation (N) method was still exhibiting a consisten tly high proportion of conditions with adequate Type I er ror control (97%, =0.8 to 100%, =0.0). The Trim and Fill method had varying proportions of conditions with adeq uate Type I error cont rol across values of population effect size, although they were all low (0%, =0.0 to 27% =0.5). Lastly, the Funnel Plot Regression method had cons istent proportions of conditions with adequate Type I error cont rol across values of the population effect size (66%, =0.0 to 67%, =0.2). Table 10 and Figure 15 present th e proportion of studies with adequate Type I error control for the methods to dete ct publication bias by population effect size. Variance of Population Effect Size Impact The average Type I error rates of th e Begg Rank Correlation (N) method, which were close to the nominal 0.05 value, were not impacted by the population effect size variance. Three of the methods had increas ing average Type I error rates as the population effect size variance increased: Begg Rank Correlation (V), Egger Regression,

PAGE 109

98 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0000.100.330.501.00Population Effect Size VarianceAverage Type I Error Rate Begg V Begg N Egger Funnel Trim Figure 16. Average Type I error rates for each method to detect publication bias by population effect size variance. 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00Begg V Begg N Egger Funnel Trim Publication Bias Detection MethodProportion with Adequate Type I Error 000 010 033 050 100 Figure 17.Proportion of conditions with adequate Type I error for each method to detect publication bias by populati on effect size variance.

PAGE 110

99 and Funnel Plot Regression. However, the av erage Type I error rates of the Begg Rank Correlation (V) and Egger Regression methods were well above 0.05 for all of the population effect size variance values (0. 1376 to 0.2909). In contrast the Funnel Plot Regression method average Type I error rates we re below the value of 0.05 for all values of the population effect si ze variance (0.0232 to 0.0433). The only method with a declining average Type I erro r rate as the population effect size variance increased was the Trim and Fill method. The average Type I error rates decreased from 0.0385 (2 = 0.00) to 0.0171 (2 = 1.00). The average Type I error ra te trends across population effect size variance values for each of the detec tion methods are presented in Table 9 and Figure 16. The two better performing methods, Be gg Rank Correlation (N) and Funnel Plot Regression, had increasing propor tions of adequate Type I error control as the population effect size variance increased. The proportion of conditions with adequate Type I error control for the Begg Rank Correlation (N) were all greater than or equal to 97%. The Funnel Plot Regression method ha d proportions of conditions w ith adequate Type I error control ranging from 44% (2 = 0.0) to 76% (2 = 0.50). There was a declining trend in the proportion of conditions with adequate Type I error rate s as the population effect size variance increased for the Begg Rank Correla tion (V) and the Egge r Regression methods. In addition, the proportion of c onditions meeting the Type I e rror rate criteria was equal to or less than 51% for the Begg Rank Correlation (V) and the Egger Regression methods. The Trim and Fill method proportions of adequate Type I error control decreased as the population effect size vari ance increased until the population effect size variance reached 1.00, where a slight incr ease occurred. Regardless of the population

PAGE 111

100 effect size the proportions for the Trim and Fill method stayed below 20% across population effect size variance. The proportions of adequate Type I error control across population effect size variance conditions are presented in Table 10 and Figure 17. Summary Across the study conditions, it is evid ent that the Begg Rank Correlation (N) method had the best performance in regards to Type I error control. When the average Type I error rates were investigated the Begg Rank Correlation (N) method was consistently around the 0.05 level, ranging fr om 0.0649 (k=10) to 0.0513 (k=50) and an overall total average of 0.0556. The proportion of conditions with adequate Type I error rates was very high across a ll the study conditions for th e Begg Rank Correlation (N) as well, ranging from 97% (n1=12 and n2=8) to 100% (multiple conditions) and an overall total of 99%. The number of studies included in the me ta-analysis, primary study sample size, and population effect size variance did not seem to have a consistent affect on the Type I error performance of the methods. In othe r words, some of the methods exhibited increases in average Type I er ror rates (or proportion with ad equate Type I error) as the condition value increased, while others exhibite d decreases in average Type I error rates (or proportion with adequate Type I error) as the condition value in creased. The primary study group variance appears to im pact the Type I error of the methods, as the primary group variance increased the average Type I error rate s increased (conversely the proportion of conditions with adequate Type I error rates decreased). Similarly, the population effect size magnitude ap pears to impact the Type I e rror of the methods, as the

PAGE 112

101 primary group variance increased the average Type I error ra tes increased (conversely the proportion of conditions with adequate Type I error rates decreased). Power Estimates of Methods to Detect Publication Bias The other gauge for performance of th e methods to detect publication bias investigated was the estimated power availa ble or the proportion of times the method accurately indicated that ther e was publication bias. Since power estimates for conditions with inadequate Type I error rates can be misleading, the power estimates presented in this research are for conditions with ade quate Type I error control according to the criterion for liberal robus tness presented by Bradley (1978). Since there may be researchers who would like to look up power estimates for specific conditions a complete reporting of all of the power estimates for each condition is presented in Appendix D. A sample has been excerpted here (see Tabl e 11 to Table 15) for discussing the power estimates where the primary study sample size was equal (n1=n2=50) and the group variance was 1:1 and 1:4. Each table represents a different value of the population effect size variance. As was discussed in the sect ion on Type I error rates there were few conditions where the Type I error rates were adequate for the Trim and Fill and Begg Rank Correlation (V) methods which is reflected in the lack of power estimates for these methods (see Table 11 to Table 15). There are several trends th at are apparent across the sample of power estimates presented in Table 11 to Table 15. First, th e power estimates increased as the number of studies included in the meta-analysis incr eased. For example, in Table 11 when the population effect size variance was 0.0, popul ation effect size was 0.50, and publication bias was strong the power estimates for th e Begg Rank Correlation (N) method increased

PAGE 113

102 with larger numbers of studies (k=10, 0. 0781 to k=100, 0.3289). Another pattern was that as the population effect size variance increased the power estimates decreased. For example, the power estimates for the Begg Rank Correlation (N) under homogeneous group variance, number of studies include d in the meta-analysis of 100, strong publication bias, and small population effect size conditions decr eased across population effect size variance values from 0.3447 (0.0 ) to 0.0573 (1.00) (see Table 11 to Table 15). Overall the Begg Rank Correlation (V) met hod had greater power than the Begg Rank Correlation (N) method. For example, when the population effect size variance was 0.50 (Table 14), the primary study group variance was homogeneous (1:4 ), and the population effect size was 0.0, across all the values of number of primary studies included and strength of publication bias the Begg Ra nk Correlation (V) method had greater power estimates compared to the Begg Rank Correlation (N) method.

PAGE 114

103 Table 11. Power estimates when primar y study sample size is equal (50) and population effect size variance is 0.00. Homogeneous Group Variances (1:1) k Pub Bias Begg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrim 10Mod0.06350.0652------------0.08110.0733------------0.08010.0703------------0.06910.0594-----------Str0.07310.0714------------0.11730.1060------------0.10030.0781------------0.07400.0620-----------20Mod0.05660.05670.0595--------0.08120.06490.0520--------0.10220.06580.0552--------0.07910.05210.0447-------Str0.06500.06260.0784--------0.14730.11200.0820--------0.1541010070.0837--------0.09370.05460.0491-------50Mod0.05850.05440.08070.0428----0.11670.08430.08950.0442----0.19410.09700.15600.05330.1231----0.04920.11360.03220.0826 Str0.06170.05600.11370.0431----0.27430.20240.15360.0728----0.34030.18380.26970.09200.2432----0.06160.17160.03800.1225 100Mod0.05450.05170.08200.0432----0.18050.12050.11970.06030.01000.35870.15920.30200.09060.1657----0.0618----0.04120.1825 Str0.06090.06060.11370.0479----0.47750.34470.23210.12110.00440.61570.32890.53130.17860.3660----0.0924----0.05360.3064 Heterogeneous Group Variances (1:4) k Pub Bias Begg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrim 10Mod0.06670.06710.0769--------0.08330.07480.0666--------0.09020.06740.0669--------0.07800.06150.0633-------Str0.07980.07320.1019--------0.11960.10120.0996--------0.11400.08550.0768--------0.08850.06500.0694-------20Mod0.06460.05920.1399--------0.09330.0761------------0.11640.0726------------0.09500.0574-----------Str0.06900.05960.1924--------0.15460.1213------------0.167501030------------0.12020.0620-----------50Mod0.05980.0576----0.0426----0.12780.0929----0.04760.00960.22230.1113----0.06870.1692----0.0580----0.0393---Str0.06490.0639----0.0554----0.28190.2062----0.08020.01180.36280.1903----0.10580.2880----0.0802----0.0490---100Mod0.05830.0555----0.04540.02530.20090.1278----0.06050.0240----01714----01135--------0.0827----0.0565---Str0.06120.0565----0.04900.02850.47640.3435----0.12880.0183----0.3458----0.2158--------0.1196----0.0756---Note. Cells with dashes (----) indicate that the condition did not have adequate Type I error control for reporting power. Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8 Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 115

104 Table 12. Power estimates when primar y study sample size is equal (50) and population effect size variance is 0.10. Homogeneous Group Variances (1:1) k Pub Bias Begg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrim 10Mod0.06270.0671------------0.07110.0638------------0.08740.0662------------0.08770.0593-----------Str0.06850.0691------------0.07900.0699------------0.10190.0726------------0.09390.0587-----------20Mod0.05730.05710.06090.0373----0.07310.05950.06000.0403----0.11210.06120.06620.0396--------0.05210.06990.0315---Str0.05850.05610.06910.0374----0.08930.06670.06740.0402----0.14830.07260.08760.0437--------0.05990.09110.0337---50Mod0.05320.05060.07800.0471----0.08880.05840.08460.0500--------0.07750.13470.0592--------0.0538----0.0468---Str0.05530.05390.09790.0508----0.13320.07840.11670.0587--------011380.21040.0758--------0.0866----0.0666---100Mod0.05200.04630.08460.0478----0.12360.06060.10120.0524--------0.0955----0.07160.0045----0.0779----0.06430.1257 Str0.05180.05080.10550.0534----0.22500.10470.14700.0694--------0.1796----0.10960.0088----0.1240----0.08970.2637 Heterogeneous Group Variances (1:4) k Pub Bias Begg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrim 10Mod0.06590.06510.0653--------0.07300.06320.0678--------0.08690.06390.0712------------0.06170.0780-------Str0.06780.06770.0720--------0.08450.07310.0788--------0.11250.07450.0830------------0.06210.0858-------20Mod0.05460.0537----0.0411----0.07180.0582----0.0395----0.12130.0658----0.0439--------0.0588----0.0381---Str0.06340.0592----0.0493----0.09520.0703----0.0515----0.15650.0760----0.0508--------0.0654----0.0429---50Mod0.05530.0499----0.0539----0.09160.0568----0.0527--------0.0771----0.06230.0212----0.0769----0.06520.1145 Str0.05530.0548----0.0588----0.14250.0823----0.0663--------01216----0.08680.0426----0.0946----0.07850.1990 100Mod0.05340.0490----0.0564----0.13560.0639----0.06060.0135----0.1063----0.08380.0251----0.0926----0.0793---Str0.05580.0512----0.0638----0.23600.1066----0.07880.0054----0.1870----0.13100.0237----0.1389----0.1168---Note. Cells with dashes (----) indicate that the condition did not have adequate Type I error control for reporting power. Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8 Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 116

105 Table 13. Power estimates when primar y study sample size is equal (50) and population effect size variance is 0.33. Homogeneous Group Variances (1:1) k Pub Bias Begg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrim 10Mod0.06310.0696------------0.06760.06480.0281--------0.09020.06550.0299------------0.06680.0386-------Str0.06470.0639------------0.07060.06550.0298--------0.10510.06940.0341------------0.06490.0435-------20Mod0.05390.05230.05820.0410----0.06840.05720.06160.0415--------0.05610.07350.0362--------0.05720.08590.0363---Str0.05960.06250.07030.0482----0.07770.05780.06990.0417--------0.06410.08440.0420--------0.06130.11340.0396---50Mod0.05250.05420.07490.0586----0.09410.05430.08840.0550--------0.0589----0.0523--------0.0567----0.0523---Str0.05060.04850.09670.0541----0.11530.06160.11530.0603--------0.0722----0.0616--------0.0748----0.0612---100Mod0.05690.05480.08730.0593--------0.05820.10510.0622--------0.0719----0.0634--------0.0670----0.05950.0060 Str0.05380.05140.10870.0599--------0.06810.13670.0637--------0.1033----0.0780--------0.0995----0.07760.0045 Heterogeneous Group Variances (1:4) k Pub Bias Begg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrim 10Mod0.06460.06590.0594--------0.07360.06370.0670------------0.06490.0687------------0.06330.0894-------Str0.06720.06530.0641--------0.08090.06920.0661------------0.06850.0802------------0.07020.1022-------20Mod0.05500.0548----0.0491----0.07050.0566----0.0470--------0.0553----0.0449--------0.0587----0.0430---Str0.06080.0556----0.0459----0.08190.0590----0.0518--------0.0649----0.0480--------0.0669----0.0495---50Mod0.05110.0491----0.0539----0.09950.0557----0.0605--------0.0616----0.0684--------0.0655----0.06440.0166 Str0.05370.0523----0.0626----0.11760.0563----0.0623--------0.0832----0.0772--------0.0832----0.07920.0256 100Mod0.05300.0523----0.0625--------0.0599----0.06960.0134----0.0752----0.07420.0118----0.0805----0.08390.0220 Str0.05030.0512----0.0673--------0.0661----0.07480.0107----0.1096----0.09830.0054----0.1091----0.10030.0165 Note. Cells with dashes (----) indicate that the condition did not have adequate Type I error control for reporting power. Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8 Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 117

106 Table 14. Power estimates when primar y study sample size is equal (50) and population effect size variance is 0.50. Homogeneous Group Variances (1:1) k Pub Bias Begg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrim 10Mod0.06200.06350.0328--------0.06460.06520.0326------------0.06250.0381------------0.06460.0444-------Str0.06660.06730.0300--------0.07050.06120.0318------------0.06590.0408------------0.06410.0525-------20Mod0.05350.04980.06190.0411----0.06740.05400.06560.0419--------0.05720.07920.0414--------0.0557----0.0396---Str0.05580.05640.07250.0495----0.07930.05860.07650.0472--------0.06160.09390.0466--------0.0576----0.0395---50Mod0.05250.04900.08300.0545----0.09480.05630.09470.0583--------0.0530----0.0537--------0.0538----0.0510---Str0.05250.05020.09990.0579----0.11150.05700.11510.0618--------0.0662----0.0613--------0.0699----0.0598---100Mod0.05120.04950.09200.0552--------0.05100.11160.0597--------0.0586----0.0607--------0.0632----0.05950.0042 Str0.05040.04850.10660.0588--------0.05360.13550.0643--------0.0790----0.0712--------0.0895----0.07480.0015 Heterogeneous Group Variances (1:4) k Pub Bias Begg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrim 10Mod0.06510.06300.0665--------0.07090.06150.0716------------0.06530.0750------------0.0653-----------Str0.06890.06570.0719--------0.07740.06840.0757------------0.06760.0780------------0.0717-----------20Mod0.05810.0545----0.0490----0.07370.0569----0.0483--------0.0543----0.0488--------0.0567----0.0479---Str0.05890.0555----0.0505----0.08450.0602----0.0506--------0.0564----0.0472--------0.0632----0.0489---50Mod0.05730.0541----0.0662----0.09870.0549----0.0655--------0.0628----0.0704--------0.0602----0.0632---Str0.05270.0513----0.0634----0.12030.0601----0.0741--------0.0715----0.0750--------0.0769----0.0776---100Mod0.04840.0480----0.0658--------0.0547----0.0708--------0.0647----0.07130.0113----0.0711----0.08070.0182 Str0.05560.0513----0.0726--------0.0590----0.0747--------0.0882----0.08780.0088----0.1024----0.09820.0085 Note. Cells with dashes (----) indicate that the condition did not have adequate Type I error control for reporting power. Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8 Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 118

107 Table 15. Power estimates when primar y study sample size is equal (50) and population effect size variance is 1.00. Homogeneous Group Variances (1:1) k Pub Bias Begg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrim 10Mod0.06770.06380.0458--------0.06870.06260.0441------------0.06300.0547------------0.06010.0709-------Str0.06720.06510.0472--------0.07340.06660.0471------------0.05940.0520------------0.06880.0727-------20Mod0.04980.05640.07790.0514----0.07450.05680.08210.0528--------0.0542----0.0447--------0.0541----0.0463---Str0.05770.05820.08020.0556----0.07830.05670.08920.0532--------0.0591----0.0468--------0.0592----0.0461---50Mod0.04860.05070.09510.0592--------0.0529----0.0634--------0.0561----0.0631--------0.0519----0.0572---Str0.04920.04900.10810.0630--------0.0524----0.0636--------0.0658----0.0685--------0.0626----0.0618---100Mod0.05410.0494----0.0638--------0.0518----0.0679--------0.0597----0.0692--------0.0580----0.0691---Str0.05860.0528----0.0735--------0.0573----0.0731--------0.0610----0.0692--------0.0720----0.0723---Heterogeneous Group Variances (1:4) k Pub Bias Begg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrimBegg VBegg NEggerFunnelTrim 10Mod0.06450.0636----0.0285----0.07040.06520.0741------------0.0674----------------0.0621----0.0269---Str0.06680.0684----0.0286----0.07170.06100.0836------------0.0630----------------0.0613----0.0267---20Mod0.05780.0541----0.0558----0.07220.0552----0.0556--------0.0597----0.0563--------0.0538----0.0535---Str0.05970.0576----0.0585----0.07710.0595----0.0611--------0.0550----0.0555--------0.0608----0.0590---50Mod0.05530.0521----0.0721--------0.0521----0.0699--------0.0505----0.0666--------0.0566----0.0733---Str0.05460.0550----0.0764--------0.0498----0.0672--------0.0582----0.0721--------0.0640----0.0764---100Mod0.04960.0492----0.0759--------0.0514----0.0780--------0.0577----0.07900.0147----0.0663----0.08120.0173 Str0.05000.0540----0.0781--------0.0536----0.0809--------0.0662----0.08760.0093----0.0740----0.09360.0112 Note. Cells with dashes (----) indicate that the condition did not have adequate Type I error control for reporting power. Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8 Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 119

108 Given that some power estimates are not reported for certain conditions because of inadequate Type I error control it is not appropriate to averag e across conditions. Thus, to examine the impact of each of the study conditions, the maximum power estimates for the methods to detect publication bias are presented for each of the study conditions in Table 16. The maximum power estimates are only reported for those conditions where adequate Type I error contro l was exhibited. The criter ion used to determine the adequacy of the Type I error was Bradley’ s (1978) liberal robustness for the nominal value of 0.05 (0.025 to 0.075 was considered adequate). The maximum power estimates of the methods for detecting publication bias increased as the degr ee of publication bias increased (see Figure 18). The Begg Rank Correlation (V) and Begg Rank Correlation (N) had the largest maximum power estimates for the two values of publication bias; followed by Egger Regression, Funnel Plot Regression, and Trim and Fill. 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 ModerateStrong Magnitude of Publication BiasMaximum Power Estimates Begg V Begg N Egger Funnel Trim Figure 18. Maximum power estimates for methods to detect publication bias by magnitude of publication bias.

PAGE 120

109 Table 16. Maximum power estimates for met hods to detect publication bias by study conditions. N Begg Rank Correlation (V) (Begg V) Begg Rank Correlation (N) (Begg N) Egger's Regression (Egger) Funnel Plot Regression (Funnel) Trim and Fill (Trim) Number of Studies (k) 1014400.16340.12760.18670.0422. 2014400.26260.18080.19240.0792. 5014400.36640.38490.32210.21330.3196 10014400.62560.66610.54140.42950.3894 Primary Study Sample Size (n1, n2) 4,66400.11640.45750.14220.11270.0314 5,56400.11560.20930.14000.07350.0306 6,46400.11870.16300.14600.06470.0287 8,126400.27090.66610.17920.37320.2736 10,106400.26920.43460.18440.23810.2651 12,86400.26640.33580.17580.14610.2376 40,606400.61960.50970.54140.42950.3865 50,506400.62560.37160.53130.24630.3763 60,406400.62370.31900.53150.17460.3894 Primary Study Group Variance 1:114400.62370.36560.54140.17860.3660 1:214400.62560.47350.53640.24460.3894 1:414400.60110.58920.22320.34290.2897 1:814400.47950.66610.18670.42950.3196 Population Effect Size Magnitude 0.014400.08810.08220.22320.08590.0285 0.214400.48550.50970.26670.21190.0366 0.514400.62560.55940.54140.42950.3854 0.814400.19200.66610.36660.37320.3894 Population Effect Size Variance 0.0011520.62560.66610.54140.42950.3894 0.1011520.23710.48340.38190.26960.3003 0.3311520.14150.27820.17210.16160.0312 0.5011520.12030.21000.13980.14490.0283 1.0011520.10450.13960.10810.09360.0322 Magnitude of Publication Bias Moderate28800.37320.35790.31460.26540.2554 Strong28800.62560.66610.54140.42950.3894 Publication Bias Detection Method Note. Cells with a period (.) indicate that the condition did not have adequate Type I error control for reporting power.

PAGE 121

110 Number of Primary Studies Impact All of the maximum power estimates for the methods to detect publication bias increased as the number of studies included in the meta-analysis increased (see Figure 19). The maximum power estimates increased the most when k increased from 20 to 50 and then even more when k increased from 50 to 100. For example, the Begg Rank Correlation (N) had a small maximum power incr ease from k=10 to k=20 (0.05 increase), then a larger increase from k=20 to k=50 (0.20 increase), then an even larger increase from k=50 to k=100 (0.28 increase). Anothe r example is the power estimates for the Egger Regression method, where there was little increase when k increased from 10 to 20 (0.1867 to 0.1924), then the increase in maxi mum power was greater when k increased from 20 to 50 (0.1924 to 0.3221), and even more increase in maximum power when k 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 102050100 Number of Studies (k)Maximum Power Estimates Begg V Begg N Egger Funnel Trim Figure 19. Maximum power estimates for met hods to detect publication bias by number of studies. Note. The Trim and Fill method did not have adequate Type I error control for the conditions when k=10 and when k=20.

PAGE 122

111 increased from 50 to 100 (0.3221 to 0.5414). This indicates that for meta-analyses with 10 to 20 studies the maximum power is not imp acted as much as when there are 50 to 100 studies in the meta-analysis. Sample Size Impact Overall as the primary study sample si ze increased the maximum power estimates of the methods to detect pub lication bias increased (see Fi gure 20). A good example is the Begg Rank Correlation (V) method with maximum power estimates which ranged from 0.1156 to 0.1187 when the average primary study sample size was 5, then they increased to range from 0.2664 to 0.2709 when the average primary study sample size was 10, and then increased again to range from 0.6256 to 0.6196 when the average primary study sample size was 50. The only met hod that did not have this pattern was the 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 4,65,56,48,1210,1012,840,6050,5060,40 Primary Study Sample Size (n1, n2)Maximum Power Estimates Begg V Begg N Egger Funnel Trim Figure 20. Maximum power estimates for met hods to detect publica tion bias by primary study sample size.

PAGE 123

112 Begg Rank Correlation (N) method which had th e highest maximum estimates when the average sample size was 10 (0.3358 to 0.6661), ra ther than when the average sample size was 50 (0.3190 to 0.5097). Group Variance Impact The impact of primary study group variance on maximum power estimates differed across the methods to detect public ation bias (see Figure 21). Two of the methods, Begg Rank Correlation (N) and F unnel Plot Regression, had increasing maximum power estimates as the primary study group variance became more heterogeneous. The Begg Rank Correlation (N) maximum power estimates increased from 0.3656 (1:1) to 0.6 661 (1:8) and the Funnel Plot Regression maximum power estimates increased from 0.1786 (1:1) to 0. 4295 (1:8). In contrast, two methods, Begg 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 1:11:21:41:8 Primary Study Group VarianceMaximum Power Estimates Begg V Begg N Egger Funnel Trim Figure 21. Maximum power estimates for met hods to detect publica tion bias by primary study group variance.

PAGE 124

113 Rank Correlation (V) and Egger Regression, had decreasing maximum power estimates as the primary study group variances became more heterogeneous. The Begg Rank Correlation (V) maximum power estimates d ecreased from 0.6237 (1:1) to 0.4795 (1:8) and the Egger Regression ma ximum power estimates decrea sed from 0.5414 (1:1) to 0.1867 (1:8). The Trim and Fill maximum po wer estimates were consistent across primary study group variances with a slight dip when the group variance was 1:4. (ranging from 0.2897 to 0.3894). Magnitude of Populatio n Effect Size Impact As the magnitude of the population eff ect size increased from 0.0 to 0.5 the maximum power estimates of all of the methods increased (see Figure 22). However, some of the maximum power estimates for the methods decreased as the population effect size increased from 0.5 to 0.8. Specifically, the Begg Rank Correlation (V), Egger 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 0.00.20.50.8 Population Effect Size MagnitudeMaximum Power Estimates Begg V Begg N Egger Funnel Trim Figure 22. Maximum power estimates for methods to detect publication bias by population effect size magnitude.

PAGE 125

114 Regression, and Funnel Plot Regression maximum power estimates decreased when the population effect size increased from 0.5 to 0.8. In contrast, the Begg Rank Correlation (N) and Trim and Fill maximum power estimates continued to increase when the population effect size increased from 0.5 to 0.8. The method with the overall highest maximum power estimates across values of the population effect size was the Begg Rank Correlation (N) with values ranging from 0.0822 ( =0.0) to 0.6661 (=0.8). Variance of Population Effect Size Impact The maximum power estimates for all of the methods to detect publication bias decreased as the population effect size va riance increased (see Figure 23). The Begg Rank Correlation (N) method had the larges t maximum power estimates ranging from 0.6661 (2 = 0.00) to 0.1396 (2 = 1.00). The Trim and Fill method had the smallest maximum power estimates ranging from 0.3894 (2 =0.00) to 0.0322 (2 =1.00). 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.000.00 0.10 0.33 0.50 100 Population Effect Size VarianceMaximum Power Estimates Begg V Begg N Egger Funnel Trim Figure 23. Maximum power estimates for methods to detect publication bias by population effect size variance

PAGE 126

115 Summary Several study conditions consistently im pacted the maximum power estimates for the methods to detect publication bias. Speci fically, the number of studies, primary study sample size, population effect size magnit ude, and the publication bias magnitude produced increasing maximum power estimate s as their values increased. One study condition, population effect size varian ce, produced decreasing maximum power estimates as its values incr eased. Lastly, the primary study group variance impacted the maximum power estimates differently depe nding on the method to detect publication bias. Regardless of the study conditions none of the statistical methods exhibited satisfactory power estimates.

PAGE 127

116 CHAPTER FIVE: CONCLUSIONS Summary of the Study The phenomenon discussed by Rosenthal ( 1979) as the “file drawer problem”, publication bias, occurs when researchers have studies that are si tting in their filing cabinets because they decided not to publish or were rejected by journals. Reasons for researchers to not submit studies or for journa ls to reject studies typically revolve around whether the results indicated si gnificant findings, whic h are influenced by sample size, or large effects. In addition, published research can inadvert ently contribute to publication bias when researchers exclude non-significant fi ndings from results or report data poorly. Thus, there is a pattern in th e published literature of a gr eater number of studies with significant findings and large effects. Researchers conducting meta-analytic studies go to great lengths (or at least they should) to gather both published and unpublis hed studies on the cont ent of their metaanalysis. This step in the meta-analysis design is time consuming but critical. When meta-analysts do not include unpublished studies, the results of the meta-analysis may be biased. Specifically, the meta-a nalysis results may indicate an inflated effect because the published studies are more likely to have si gnificant results and large effects (Sharpe, 1997). Thus, publication bias is c onsidered to be a threat to the validity of meta-analyses. One method for detecting public ation bias is the visual interpretation of a funnel plot (a scatterplot of effect sizes and sample sizes). However, the visual examination of

PAGE 128

117 the funnel plot is limited becau se the interpretation is s ubjective and the plot can be difficult to interpret when there are a sma ll number of studies included in the metaanalysis (Greenhouse & Iyenga r, 1994; Thornton & Lee, 2000). Consequently, some researchers have developed sta tistical methods for detecting publication bias that are not subjective. The following statistical methods for detecting publica tion bias have been introduced and applied in the literature: (1 ) Begg Rank Correlation, (2) Egger Regression, (3) Funnel Plot Regression, and (4) Trim and Fill. Three issues drove the pursu it of this research : (1) publication bias is a problem which is underreported in meta-analyses, (2) both the impact of publication bias and the comparison of statistical methods for detec ting publication bias are lacking in the literature, (3) publication bias issues have not been expl ored in random effects metaanalysis models. A wealth of literature documents the phenomenon of publication bias and the statistical bias that it presents to meta-analysts (Begg, 1994; Greenhouse & Iyengar, 1994; Rosenthal & Rubin, 1979; Sharpe, 1997; Smith, 1980; Sterling, 1959; Sutton, Abrams, Jones, Sheldon, & Song, 2000). This problem is compounded by the infrequent acknowledgement and application of methods for detecting (or adjusting for) publication bias. The second issue may be the reason why publication bias detection methods are inconsistently being implemented, there are few empirical investigations of the degree to which publication bias impacts meta-analysis results or the comparison of statistical methods to detect publicati on bias (Begg & Mazumdar, 1994; Bradley & Gupta, 1997; Duval & Tweedie, 2000a; Duval & Tweedie, 2000b; Macaskill, Walter, & Irwig, 2001; Rendina-Gobioff & Kromrey, 2004; Schwarzer, Antes, & Schumacher, 2002; Sterne, Gavaghan, & Egger, 2000). Lastly there is a need in the literature to

PAGE 129

118 examine the impact of publication bias and the performance of public ation bias detection methods within the random ef fects meta-analysis model. There were two primary goals overarching this research endeavor: (1) examine the degree to which publication bias impacts th e results of a random effects meta-analysis and (2) investigate the performance of five statistical methods for detecting publication bias in random effects meta-analysis. First, the impact on the meta -analysis results was estimated by examining the difference be tween the population effect size and the estimated meta-analysis effect size. Similarly, the impact on the meta-analysis results was estimated by examining the difference betw een the population effect size variance and the estimated meta-analysis e ffect size variance. Second, th e performance of the five statistical methods (Begg Rank Correlati on (V), Begg Rank Correlation (N), Egger Regression, Funnel Plot Regre ssion, and Trim and Fill) was estimated with Type I error rates and statistical power. This research simulated meta-analyses using a Monte Carlo design. The use of simulation methods allowed for the control a nd manipulation of research design factors and the incorporation of sampling error into the analyses. The first and second steps in the simulation were to generate observati ons in primary studies under known population conditions and to compute the effect size. The next step in the simulation was to impose the publication bias using the obtained p-valu es from the primary studies. The following two steps included computing the meta-analysis mean effect size and the statistical tests for publication bias. The final step in the research was to compute the analyses for determining the performance of the statistical tests for publication bias, Type I error rate

PAGE 130

119 and power estimates. In addition, the impact of imposing publication bias on the metaanalysis estimated mean effect size and variance was calculated. The simulation was modeled after that re ported by Macaskill, Walter and Irwig (2001) and Rendina-Gobioff and Kromrey (2004) but extends to random effects metaanalyses. For each primary study, the Hedges' s g effect size (Hedges & Olkin, 1985) was calculated based on the simulated data. The Mon te Carlo study include d six factors in the design. These factors were (a) the number of primary studies in each meta-analysis (10, 20, 50, and 100), (b) the sample sizes of the two groups in each primary study (with mean total sample sizes ranging from 10 to 100 as well as balanced and unbalanced conditions), (c) group variances in the primary studies (varia nce ratios of 1:2, 1:4, and 1:8, as well as a homogeneous variance c ondition), (d) the magnitude of the population effect size (= 0.00, 0.20, 0.50, 0.80), (e) the vari ance of the population effect size (2 = 0, .10, .33, .50, and 1.00), and (f) the magnit ude of the publication bias (no bias, moderate bias, and strong bias). The proportions of studies available fo r each meta-analysis are presented as confirmation of the methods used to impose publication bias. The impact that publication bias had on the outcomes of meta-analyses was evaluated by examining the estimated mean effect size and estimated effect size va riance compared to the population effect size magnitude and variance values. The performan ce of the statistical methods to detect publication bias was evaluated in regards to Type I error rates and estimated power. Type I error is reported for each study c ondition two ways, averag e rate and proportion with adequate rates. Lastly, maximum pow er estimates are reported for each study condition.

PAGE 131

120 Research Questions 1. To what extent will publi cation bias impact the estim ated mean effect size and estimated variance in a random effects meta-analysis? a. To what extent does the number of prim ary studies included in the meta-analysis moderate the impact of pub lication bias on the estimated mean effect size and variance calculated for the meta-analysis? b. To what extent does the mean sample size of groups (including balanced and unbalanced) in the primary studies include d in the meta-analysis moderate the impact of publication bias on the es timated mean effect size and variance calculated for the meta-analysis? c. To what extent do the group variances ( homogeneous and heterogeneous) in the primary studies included in the meta-analy sis moderate the impact of publication bias on the estimated mean effect size and variance calculated for the metaanalysis? d. To what extent does the magnitude of the population effect size moderate the impact of publication bias on the es timated mean effect size and variance calculated for the meta-analysis? e. To what extent does the variance of the population effect size moderate the impact of publication bias on the es timated mean effect size and variance calculated for the meta-analysis? 2. To what extent do Type I e rror rates vary acro ss statistical methods for detecting publication bias in a random effects meta-analysis?

PAGE 132

121 a. To what extent does the number of prim ary studies included in the meta-analysis impact the extent that Type I error ra tes vary across sta tistical methods for detecting publication bias? b. To what extent does the mean sample size of groups (including balanced and unbalanced) in the primary studies included in the meta-analysis impact the extent that Type I error rates va ry across statistical methods for detecting publication bias? c. To what extent do the group variances ( homogeneous and heterogeneous) in the primary studies included in the meta-analysis impact the extent that Type I error rates vary across statistical met hods for detecting publication bias? d. To what extent does the magnitude of th e population effect size impact the extent that Type I error rates va ry across statistical methods for detecting publication bias? e. To what extent does the variance of the population effect size impact the extent that Type I error rates va ry across statistical methods for detecting publication bias? 3. To what extent do power estimates vary across statistical me thods for detecting publication bias in a random effects meta-analysis? a. To what extent does the number of prim ary studies included in the meta-analysis impact the extent that power estimates vary across statistical methods for detecting publication bias?

PAGE 133

122 b. To what extent does the mean sample size of groups (including balanced and unbalanced) in the primary studies included in the meta-analysis impact the extent that power estimates vary across statis tical methods for detecting publication bias? c. To what extent do the group variances ( homogeneous and heterogeneous) in the primary studies included in the meta-analysis impact the extent that power estimates vary across statistical me thods for detecting publication bias? d. To what extent does the magnitude of th e population effect size impact the extent that power estimates vary across statis tical methods for detecting publication bias? e. To what extent does the variance of the population effect size impact the extent that power estimates vary across statis tical methods for detecting publication bias? Summary of Study Results The proportion of studies available for the meta analyses increased as the primary study sample size, primary study group vari ance, population effect size and population effect size variance increased. In contrast, the proportion of studies available for the meta-analyses decreased as the magnitude of publication bias increased. These proportions are consistent with publication bias theory rega rding the types of studies available to the meta-analyst. Specifically, studies with large sample sizes and large effect sizes are expected to be available to the meta-analyst (Begg, 1994; Lipsey & Wilson, 1993; Sharpe, 1997; Smith, 1980; Sterling, 1959; Sutton et al., 2000). The studies with large primary study group va riance and population e ffect size variance

PAGE 134

123 would be published under conditions where th e population effect size is large (Begg, 1994; Sutton et al., 2000). Begg (1994) indicated that sample size of the primary study has the greatest influence on publication bias. In this study the simulation of publication bias resulted in the largest range of proporti ons of studies available to the meta-analyst under the primary study sample size condition. Impact of Publication Bias The impact of publication bias was ev aluated by the average amount that the estimated mean effect size varied from the population effect size a nd the average amount that the estimated effect size variance varied from the popul ation effect size variance. A substantial amount of effect size bias wa s evidenced with both moderate and strong publication bias, compared to no publication bias. Similarly, e ffect size variance bias was evident for moderate and st rong publication bias, compared to no publication bias. The inflated effect size magnitude and inflated effect size varian ce are consistent with the literature regarding the influence of publica tion bias on the results of meta-analyses. Several researchers have alluded to or empi rically investigated the phenomenon that the results of a meta-analysis with publication bias will have inflated estimates of the mean effect size (Begg, 1994; Bradley & Gupta, 1997; Lipsey & Wilson, 1993; Sharpe, 1997; Smith, 1980; Sutton et al., 2000) Several of the study conditions impacted the effect size magnitude and variance estimates in the same way. One moderator, primary study sample size, was found to have a positive impact on effect size magnitude and variance bias, increasing accuracy. The increasing accuracy found with increasi ng sample size was expected because the distribution of studies approaches norma lity, as sample size increases, which would

PAGE 135

124 increase the accuracy of the mean and vari ance estimates. One moderator, primary study group variance, was found to have a negativ e impact on effect size magnitude and variance bias, decreasing accuracy. The pr imary study group variance has the opposite effect on the distribution of studies, when compared to the influence of primary study sample size, as group variance increases the distribution deviates from normality and thus decreases accuracy of the mean and variance es timates. Two moderators had a very slight impact on the effect size magnitude and va riance bias; number of primary studies and population effect size variance. The number of primary studi es would be expected to increase the accuracy of the mean and varian ce estimates as the value increased because the distribution of studies would approach normality. This effect was found to be very small. The population effect size variance would be expected to distor t the distribution of studies from normality and thus would be expected to decrease the accuracy of estimates. This pattern was found to be very small as well. These findings of increased bias in Hedges's g effect size under c onditions of smaller sample si zes, increased group variance and increased non-normality are consistent w ith results found by Hess, Kromrey, Ferron, Hogarty, and Hines (2005). One study condition impacted the estim ates of the population effect size magnitude and variance differently. Specifical ly, the population effect size magnitude had a negative impact on the effect size ma gnitude bias and a pos itive impact on the effect size variance bias. Thus as the populat ion effect size increased the effect size magnitude estimate became less accurate and the effect size variance estimate became more accurate. One explanation for this impact is that when the population effect size is zero publication bias theory would project that studies in the upper a nd lower tails of the

PAGE 136

125 distribution would be published. Under these conditions, the estimate for the effect size magnitude would be balanced by the public ations on both ends of the distribution resulting in an estimate close to the populat ion value. The effect size variance estimate under these conditions would be inflated by the publications in the tails of the distribution. In contrast public ation bias theory would expect that when the effect size is large the studies published would include all of the distribution except th ose studies in the smaller tail. Under these conditions the effect size magnitude estimate would be inflated because of the suppression of just the lower ta il, smaller effect size, studies. The effect size variance estimate would approach the population effect size variance because the range of effect sizes published is wider (c ompared to the small effect size condition explained earlier). Type I Error Rates of Methods to Detect Publication Bias The performance of five methods to de tect publication bias were evaluated by examining average Type I error rates and the proportion of meta-analyses with adequate Type I error control. The Begg Rank Correla tion (N) method exhibite d the best Type I error control in regards to both average a nd proportion with adequate control. Across study conditions, the Begg Rank Correlati on (V) and Egger Regression methods exhibited large average Type I error rates and small proportion s of studies with adequate Type I error control. The Funnel Plot Regr ession method exhibited small average Type I error rates and moderate pr oportions with adequate Type I error control across study conditions. Although the Trim and Fill method exhibited small average Type I error rates across study conditions, the error rates were too small to be considered adequate as evidenced by the small proportion of conditions with adequate Type I error control.

PAGE 137

126 In theory the Type I error rates shoul d be robust to the study conditions; number of studies, primary study sample size, a nd population effect size magnitude (Glass & Hopkins, 1996). The Begg Rank Correlation (N), Funnel Plot Regression, and Trim and Fill methods approached the nominal 0.05 valu e as the number of studies increased. In contrast, both the Begg Rank Correlation (V) and Egger Regression methods increased beyond the nominal 0.05 value as the number of studies increased. Consistent with robustness theory, as the primary study samp le size condition values increased Type I error rates approached 0.05 with the Begg Rank Correlation (V), Begg Rank Correlation (N), Egger Regression, and F unnel Plot Regression methods. In contrast, the Trim and Fill method Type I error rates decreased well below the nominal 0.05 value as the primary study sample size increased. The Begg Rank Correlation (N), Funnel Plot Regression, and Trim and Fill methods exhibi ted Type I error rates approaching 0.05 as the population effect size increased. In contrast, the Begg Rank Correlation (V) and Egger Regression method Type I error rates increased beyond the nom inal 0.05 value as the population effect size increased. To su mmarize, the Begg Rank Correlation (N), Funnel Plot Regression, and Trim and Fill methods are exhibiting Type I error rate performance that is consistent with theory regarding the r obustness of statistical methods (Glass & Hopkins, 1996). Type I error rates should deviate fr om the nominal 0.05 value when study conditions have greater heterogeneity and larg er population effect si ze variance (Glass & Hopkins, 1996). The Begg Rank Correlation (V), Begg Rank Correlation (N), and Egger Regression methods to detect publication bias exhibited incr easing Type I error rates as the primary study group variance increased, de viating from the nominal 0.05 value.

PAGE 138

127 However the Begg Rank Correlation (N) method was still close to the 0.05 value across all values of primary study group variance. The Funnel Plot Regression and Trim and Fill methods also had increasing Type I error rates as the primary study group variance increased but they approached the nominal 0.05 value. The Type I error rates for the Begg Rank Correlation (V), Egger Regression, and Trim and Fill methods deviated from 0.05 as the population effect size variance increased. In contrast, the Funnel Plot Regression method Type I error rates appro ached the nominal 0.05 value. The Begg Rank Correlation (N) stayed around 0.05 value as th e population effect size variance values increased. In summary, even under conditions that statistical methods should not be robust the Begg Rank Correlation (N) method e xhibited robust Type I error rates. Power Estimates of Methods to Detect Publication Bias The performance of the statistical met hods to detect publication bias was also evaluated by investigating the maximum power estimates across all study conditions and separately for each study condition. The power estimates were only reported for those conditions where Type I error control was adequate accordi ng to Bradley’s (1978) liberal criteria. The maximum power estimates for all methods were greater when the strength of publication bias was large compared to when publication bias was small. The larger power estimates for the greater degree of pub lication bias is consistent with the well documented factors that influence statisti cal power, variance of the responses (study effect sizes) and effect size magnitude (Glass & Hopkins, 1996). In theory when publication bias is larger the variance of th e study effect sizes would decrease and the effect sizes would be larger. When the publication bias imposed was moderate the maximum power estimates were small across all methods, with th e greatest estimate

PAGE 139

128 being 0.3732 (Begg Rank Correlation V). The maximum power increased when publication bias was strong but was still low, with the greatest estimate being 0.6661 (Begg Rank Correlation N). Three factors that in theory should incr ease statistical power are number of studies included, primary study sample size, and population effect size magnitude (Glass & Hopkins, 1996). The number of studies and th e primary study sample size conditions consistently increased power estimates as th eir values increased across all methods. With the highest number of studies (k=100) th e Begg Rank Correlation (V) and the Begg Rank Correlation (N) methods exhibi ted the highest maximum power estimates. However, when the number of studies in cluded in the meta-analysis was small (k=10) the maximum power estimates for all methods were atro cious. The Begg Rank Correlation (N) method exhibited the best, although still low, maximum power es timates when the average sample size was small and medium. When th e average sample size was large the Begg Rank Correlation (V) method had the highest maximum power estimates. All methods exhibited an increase in maximum power estimates as the population effect size magnitude increased to the moderate value of 0.50. However, some methods, Begg Rank Correlation (V), Egger Regression, and Funnel Plot Regression, had a small to large dip in the maximum power estimates when th e population effect size was large (=0.8).The highest maximum power estimate across me thods for the population effect size magnitude condition was the Begg Rank Correlation (N). Two study conditions that were expected to negatively impact power estimates were the primary study group variance and the population effect size variance. Both of these conditions would increase variation in the response vari able, effect sizes, and thus

PAGE 140

129 decrease power estimates. Across all me thods the population e ffect size variance condition did exhibit decreasing maximum pow er estimates as the variance increased. The highest maximum power estimates we re for the Begg Rank Correlation (V) and Begg Rank Correlation (N) methods when ther e was no population effect size variance. The maximum power estimates were very lo w when the population effect size variance was large (1.00) across all methods. The maximum power estimates were impacted differently across the methods by the primary study group variance. Consistent with the expected impact of primary study group variance on power estimates, the power estimates decreased as group variance in creased for the Begg Rank Correlation (V), Egger Regression, and Trim and Fill methods In contrast the Begg Rank Correlation (N) and Funnel Plot Regression me thods exhibited increasing maximum power estimates as group variance increased. Regardless of the study condition or the detection method employed the maximum power estimates were low. The hi ghest estimate was 0.6661 which was much lower than the desired 0.80 power estima te. In addition, for many conditions the maximum power estimates were below 0.30. Discussion The inflated effect size estimates revealed in this research are consistent with previous research regarding the impact of publication bias (Begg, 1994; Bradley & Gupta, 1997; Lipsey & Wilson, 1993; Shar pe, 1997; Smith, 1980; Sutton et al., 2000). Therefore, the results of a meta-analysis conduc ted that has publicati on bias are likely to falsely indicate that there is a larger effect than reality. This inflation is especially problematic because the results of a meta-analy sis often are more infl uential than primary

PAGE 141

130 studies because they are the synthesis of multiple studies with perceived additional precision (Begg, 1994). These fi ndings suggest that meta-ana lysts should be concerned about the effect of publication bias on their results. In addi tion, the findings validate the perception that publication bias is a threat to the validity of meta-analysis. According to Begg (1994), “Publication bias presents possi bly the greatest met hodological threat to validity of meta-analysis.” (p .407). Furthermore, this impact on the results emphasizes the need to have and implement me thods to detect publication bias. The impact of publication bias on the estim ation of effect size variance is less documented in the literature. This study f ound that there was an increased effect size variance bias when publication bias was imposed. Thus the es timated variance was larger than the population effect size variance. Th is inflation of the variance could lead to inaccurate investigations of moderators or random effects when the researcher should be looking into publication bias. As part of the meta-analy sis procedures researchers estimate the homogeneity of the primary st udy effect sizes. When heterogeneity is indicated the effect sizes are not estimating a common population mean. In other words there may be study characteristics or modera tors that are causing variability among the effect sizes. Four options for the meta-analyst when heterogeneity is indicated are: (1) only interpret the results descriptively, (2) include moderators in the analyses, (3) use regression techniques that account for the vari ance, or (4) incorpor ate the variance into the model by using random effects methods (Shadish & Haddock, 1994). These four options should be influenced by the investigation of publication bias. In other words, before the researcher chooses one of these options they should employ methods to detect

PAGE 142

131 publication bias. If detected th en another option would be to return to the search for studies to include in the meta-analysis. There are several conditions that alleviate the impact of publication bias on the effect size magnitude and variance estima tion that meta-analysts should consider. Researchers with large primary study sample si zes and a large number of primary studies will have greater accuracy with these estima tes. In contrast researchers with large heterogeneity in the primary study and larg e population effect size variance should be more concerned about the impact of publica tion bias on the estimates of effect size magnitude and variance. The population effect size magnitude influenced the accuracy of the effect size magnitude and variance es timates differently. Thus researchers should consider that with a large population effect size they will have more accuracy in the effect size variance estimate than the effect size magnitude estimate. Regarding the performance of the sta tistical methods for detecting publication bias, the Begg Rank Correlation (N) method exhibited the best Type I error performance (consistently around the nomina l 0.05 value) and in comparison to the other methods superior power estimates (greatest estim ate 0.6661). The Begg Rank Correlation (V) and Egger Regression methods had astronomica lly high Type I error rates. The power estimates for the Begg Rank Correlation (V) method were, at times, comparable or greater than the Begg Rank Correlation (N) method, however, this was with the cost of greater Type I error rates. The Egger Regression method pow er estimates were low and did not exceed 0.5414. The Funnel Plot Regres sion and Trim and Fill methods typically had small Type I error rates (smaller than th e 0.05 value) and low power estimates. These findings indicate that among the methods inves tigated the method with the best Type I

PAGE 143

132 error and power performance was the Begg Rank Correlation (N). These findings are consistent with the similarly designed resear ch investigating the performance of these methods to detect publication bias within a fixed effects meta-analysis context (Kromrey & Rendina-Gobioff, 2004; Macaskill, Walter, & Irwig, 2001). According to theory on robustness a nd factors influencing power several conditions were expected to influence Type I error rates and power. For most of the methods, Type I error rates approached th e nominal 0.05 value when the number of studies, primary study sample size, and popula tion effect size magnitude increased in value. In contrast, conditions that resulted in deviating Type I error rates were larger primary study group variance and larger populat ion effect size variance. The factors expected to and found to increase power we re number of studies, primary study sample size, and population effect size magnitude. De creases in power estimates were expected and found for conditions where the primar y study group variance and population effect size variance were greater. In general, acros s the five methods investigated, the Type I error rates and power estimates for most condi tions performed consis tent with theory on these statistical indicators of performance. Although the Begg Rank Corre lation (N) method has good Type I error control it does not have adequate power. Regardless of the study condition or the detection method employed the maximum power estimates were low. This is concerning to the metaanalyst who wants to detect publication bias statistically. The adequate Type I error control by the Begg Rank Correlation (N) indicates that the researcher is likely to accurately detect when publication bias is not present. However, the low power estimates for all methods indicates that a researcher is less likely to accurately detect when

PAGE 144

133 publication bias is present. Just examin ing the power estimates for the Begg Rank Correlation (N) method the best conditions fo r accurately detecting the presence of publication bias are w ith a large number of studies (k =100), medium unbalanced sample size (n1=8, n2=12), large primary study group va riance (1:8), large population effect size magnitude (=0.8), and small population effect size variance (2 =0.0). As evidenced in this study, although the methods are similar the Begg Rank Correlation (N) method out perf orms (better Type I error) the Begg Rank Correlation (V) method. These results are similar to those found by Kromrey and Rendina-Gobioff (2004) who comment on the difference in performance among these two methods. Essentially, the Begg Rank Correlation (V) meth od correlates the standa rdized effect size with the variance of th e effect sizes. The variance of He dges's g effect size is estimated with the following: 2 2 12 1212ˆ 2i g ig nn nnnn Where ig is the estimated effect size for study i 1n is the sample size for group 1 2n is the sample size for group 2 This formula is a function of both the effect size (ig ) and the sample size (in ). If we consider the situation where there is no publication bias, for the Begg Rank Correlation (V) method the correlation of th e effect size and the variance of the effect sizes would be expected to be non-zero because both the e ffect size and the sample size are included in the variance formula. Therefore, it is not surprising that the Begg Rank Correlation (V)

PAGE 145

134 method had difficulty detecting when there wa s no publication bias present. In contrast, the Begg Rank Correlation (N) me thod correlates the standardi zed effect size with the sample size. If we consider the situation where there is no publication bias, the Begg Rank Correlation (N) method the co rrelation of the effect size with the sample size would be expected to be zero. Limitations There are generalizabil ity limitations to consider in re lation to this research study. The simulation method implemented in this study provides control of factors to investigate performance in specific situations This benefit of simulation studies also limits the generalizability of the study findings. Thus, the co ntrolled factors (number of studies, sample size, group va riances, size of population effect size, and a random effects model) dictate the types of meta-analyses the results can be generalized to. Another restriction on generalizability is that only th e Hedge’s g effect size was investigated. The impact of publication bias and the perform ance of detection methods may vary across other effect size statistics. Another generalization limitation is that this study calculated meta-analysis summary statistics based on the methods developed by Hedges and Olkin (1985), and the results cannot be generalized to other meta-analysis methods such as Hunter and Schmidt (1990). The final consideration of limited generalizability is the investigation of moderators Although moderators are co mmonly explored in metaanalyses this simulation does not generalize to these analyses. Another limitation to cons ider relates to the methods used to impose publication bias. Although there are many f actors influencing the publicati on of studies (effect size, methodology, journal type, funding source, etc.), the function utilized to determine the

PAGE 146

135 selection of primary studies for the simulation of meta-analyses solely relies on the pvalues. The benefit of the f unction utilized is that it does not have a sharp cut off (p< 0.05) to determine the inclusi on of primary studies. Therefor e the function is in effect taking into consideration factors external to the p-value. However, the accuracy of the function to represent the reality of external factors is unknown. This method is consistent with other studies th at have been done. Implications Importance of Study The degree of publication bias impact on th e results of meta-analyses is extremely important. Publication bias theory states that the mean effect size estimates that result from a meta-analysis will be inflated. Th ere have been few studies documenting the extent that this inflation has on the results of a meta-analysis. In addition, the conclusions and decisions made from the results of meta -analyses hold additional weight because they are perceived to have additional precision. Ther efore, it is pertinent that meta-analysts are aware of the potential degree that publication bi as can impact their results and reflect that in their conclusions. The detection of publication bias in the context of meta-analysis is important because the validity of a meta-analysis is part ially determined by the selection of studies included in the synthesis. Examination of the performance of statistical methods for detecting publication bias pr ovides information about the re search conditions for which they are appropriate to apply. In addition, an increased use of statistical methods for detecting publication bias may re sult, rather than visually in specting the funnel plot or not addressing publication bias at all. The reporting of publication bias detection will result in

PAGE 147

136 more accurate conclusions being drawn from meta-analysis results. Lastly, regular reporting of publication bias in meta-analysi s designs may result in a more favorable perception of meta-analysis methodology. Researchers in General The large impact of publication bias on th e results of meta-ana lyses indicates that researchers need to consider the need to decrease publication bias Several methods to decrease publication bias could be consid ered. In the medical field many studies are registered before they are conducted. In the field of psychol ogy and education the implementation of a resource for registering studies would bring aw areness to the degree of publication bias and increase the ability of meta-analysts to track studies. The registration of studies would allow for rese archers to determine how many studies are conducted compared to how many studies are pu blished or made available to the metaanalyst. In addition to being able to calculat e proportions of studies available to the metaanalyst, registration of studi es would improve the study re trieval process of a metaanalysis. Another method to d ecrease publication bias is to standardize the reporting of studies. Often the resu lts of a study are unavailable to the meta-analyst because the results are reported in a variety of formats that sometimes do not provide the necessary information to be included in the meta-analy sis. The registration of studies could provide a standardization of study resu lts. The publication process coul d facilitate a decline in the degree that publication bias exists by enc ouraging editors and re viewers to consider submissions that do not have statistically significant or large effect results.

PAGE 148

137 Researchers Conducting Meta-Analyses The large impact of publication bias on th e results of meta-ana lyses indicates that meta-analysts need to investigate the potenti al of publication bias within their study and report the findings. Due to the subjectivity and difficulty interpreting, the visual inspection of a funnel plot is not the ideal method on its own (Greenhouse & Iyengar, 1994; Tang & Liu, 2000; Thornton & Lee, 2000). Although the st atistical methods investigated in this study have their lim itations, they provide additional information regarding the potential of pub lication bias. The review of Psychological Bulletin and Review of Educational Research conducted for this study rev ealed that very few metaanalysts are conducting and re porting publication bias detection methods. As more metaanalysts conduct and report publication bi as detection in their meta-analyses the awareness and implementation will increase. In addition, meta-analysts should utilize meta-analysis procedures that decrease the potential for publication bias. For example, the incorporation of unpublished research studies within a meta-analysis will decrease publication bias. Some researchers have th e perspective that unpublished studies are characterized by poorer quality. This perspective needs to be overcome. Researchers of Statistical Methods for Detecting Publication Bias The large impact of publication bias on the results of a meta-analysis is a reflection of the need to detect publicati on bias. The performance of the statistical methods to detect publicati on bias were found to be l acking. Although the Begg Rank Correlation (N) method had adequate Type I e rror control the available power was low. Therefore, there is a need for new methods to be developed to de tect publication bias.

PAGE 149

138 Suggestions for Future Research One suggestion for future studies is to conduct additional studies on the impact of publication bias. This study uses one way of simulating publication bias which may or may not reflect the reality of what is occurri ng in the literature. Ther efore, studies using a different algorithm for imposing publication bias would be beneficial. Similarly, studies investigating the accuracy of th e theory of publication bias and its influences would be beneficial. For example, one could survey researchers regarding the factors that contribute to their decisions to publish and present research as well as factors that influence their decision to recommend research for publication as a reviewer or editor. Another suggestion for future research is to analyze the result s for this study in a way that the fixed effects can be compared to random effects. In ot her words the impact and performance of the methods to detect publication bias can be compared when the population effect size variance is zero to when the population effect size variance is greater than zero. Alt hough the performance of the statistical methods is expected to be similar across fixed and random effects since th e results of this study (random effects) are similar to the results of Kromrey and Rendina -Gobioff (2004) (fixed effects). However, the impact of publication bias on the results of the meta-analysis (estimated effect size and variance) could vary. In addition, it would be inte resting to investigate the impact of publication bias when a fixed effect meta-analysis design is applied to population conditions that are random effects. The literature indicates that the variance estimates will be underestimated, implying greater precision, wh en fixed effects methods are applied to

PAGE 150

139 truly random effects data (H edgers & Vevea, 1998). Howe ver, this phenomenon has not been investigated when publication bias is evidenced. Lastly, this study calculated meta-analy sis summary statistics based on the methods developed by Hedges and Olkin (1985 ), however another commonly used metaanalysis method is Hunter and Schmidt (1990). It would be of interest to conduct research to investigate the impact of publication bi as when Hunter and Schmidt (1990) metaanalysis techniques are applied. In addition, the performan ce of the methods to detect publication bias would probably be different when Hunter and Schmidt (1990) techniques are applied for the meta-analysis simulati ons, compared to Hedges and Olkin (1985).

PAGE 151

140 REFERENCES Abelson, R. P. (1995). Statistics as principled argument Hillsdale, NJ: Lawrence Erlbaum Associates. Bennett, D. A., Latham, N. K., Stretton, C., & A nderson, C. S. (2004). Capture-recapture is a potentially useful method for assessing publication bias. Journal of Clinical Epidemiology, 57, 349-357. Begg, C. B. (1994). Publication bias. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 399-409). New York, NY: Russell Sage Foundation. Begg, C. B., & Mazumdar, M. (1994). Operating characteristics of a rank correlation test for publication bias. Biometrics, 50(4), 1088-1101. Bradley, J. V. (1978). Robustness? British Journal of Mathematic al and Statistical Psychology, 31 144152. Bradley, M. T., & Gupta, R. D. (1997). Estimating the effect of the file drawer problem in metaanalysis. Perceptual and Motor Skills, 85 719-722. Cleary, R. J., & Casella, G. (1997). An applica tion of Gibbs sampling to estimation in metaanalysis: Accounting for publication bias. Journal of Educational and Behavioral Statistics, 22 (2), 141-154. Cliff, N. & Charlin, V. (1991). Variances and covariances of Kendall’s tau and their estimation. Multivariate Behavioral Research 26 693-707. Cohen, J. (1992). A power primer. Psychological Bulletin, 112 (1), 155-159. Cooper, H., & Hedges, L. V. (1994a). Research sy nthesis as a scientific enterprise. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 3-14). New York, NY: Russell Sage Foundation.

PAGE 152

141 Cooper, H., & Hedges, L. V. (1994b). Potentials and limitations of research synthesis. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 521-529). New York, NY: Russell Sage Foundation. Copas, J., & Jackson, D. (2004). A bound fo r publication bias based on the fraction of unpublished studies. Biometrics, 60, 146-153. Copas, J., & Shi, J. Q. (2000). Meta-analy sis, funnel plots and sensitivity analysis. Biostatistics, 1 (3), 247-262. Dear, K. B. G., & Begg, C. B. (1992). An approach for assessing publication bias prior to performing a meta-analysis. Statistical Science, 7 (2), 237-245. Duval, S. & Tweedie, R. (2000a). Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56, 455-463. Duval, S. & Tweedie, R. (2000b). A nonparame tric “Trim and Fill” method of accounting for publication bias in meta-analysis. Journal of the American Statistical Association, 95, 89 98. Egger, M., Smith, G. D., Schnedier, M. & Minder, C. (1997). Bias in meta-analysis detected by a simple graphical test. British Medical Journal 315 629-634. Field, A.P. (2001). Meta-analysis of correlation coefficients: A Monte Carlo comparison of fixedand random-effects methods. Psychological Methods, 6 (2), 161-180. Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5 3-8. Glass, G.V., and Hopkins, K. D. (1996). Statistical methods in education and psychology. Boston, MA: Allyn and Bacon. Greenhouse, J. B., & Iyengar, S. (1994). Sensitivit y analysis and diagnostics. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 383-398). New York, NY: Russell Sage Foundation. Harwell, M. (1997). An empirical study of Hedges’s homogeneity test. Psychological Methods, 2 (2), 219-231.

PAGE 153

142 Hedges, L. V. (1981). Distribution theory for Glass’s estimator of effect size and related estimators. Journal of Educational Statistics,6 (2), 107-128. Hedges, L. V. (1992). Modeling publication selection effects in meta-analysis. Statistical Science, 7 (2), 246-255. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. San Diego, CA: Academic Press. Hedges, L. V., & Vevea, J. L. (1998). Fixedand random-effects models in meta-analysis. Psychological Methods 3 (4), 486-504. Hedges, L. V., & Vevea, J. L. (1996). Estimati ng effect size under publication bias: Small sample properties and robustness of a random effects selection model. Journal of Educational and Behaviaoral Statistics, 21 (4), 299-332. Hess, M. R., Kromrey, J. D., Ferron, J.M., H ogarty, K.Y., and Hines, C.V. (2005, April). Robust Inference in Meta-Analysis: An Empirical Co mparison of Point and Interval Estimates Using Standardized Mean Difference and Cliff’s Delta. Paper presented at the annual meeting of the American Educational Research Association, Montreal, Canada. Hunter, J.E., & Schmidt, F. L. (1990). Methods of meta-analysis: correcting error and bias in research findings. Newbury Park, CA: Sage Publications. Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage Publications. Lipsey, M. W., & Wilson, D. B. (1993). The efficacay of psychological, educational, and behavioral treatment confirmation from meta-analysis. American Psychologist,48 (12), 1181-1209. Macaskill, P., Walter, S., & Irwig, L. (2001). A comparison of methods to detect publication bias in meta-analysis. Statistics in Medicine, 20, 641-654. Matt, G. E., & Cook, T. D. (1994). Threats to the validity of research synthesis. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 503-520). New York, NY: Russell Sage Foundation.

PAGE 154

143 Raudenbush, S. W. (1994). Random effects models In H. Cooper & L. V. Hedges (Eds.), The The handbook of research synthesis (pp. 301-321). New York, NY: Russell Sage Foundation. Rendina-Gobioff, G., & Kromrey, J. D. (2004, April). On Knowing What We Don't Know: An Empirical Comparison of Methods to Detect Publication Bias in Meta-Analysis. Paper presented at the annual meeting of the Americ an Educational Research Association, San Diego, CA. Robey, R. R. & Barcikowski, R. S. (1992). Ty pe I error and the number of iterations in Monte Carlo studies of robustness. British Journal of Mathematical and Statistical Psychology, 45 283-288. Rosenthal, R. (1979). The "file-drawer problem" and tolerance for null results. Psychological Bulletin, 86, 638-641. Rosenthal, R., & Rubin, D. B. (1982). Comparing effect sizes of independent studies. Psychological Bulletin, 9 (2), 500-504. Schmidt, Fl L., & Hunter, J. E. (1977). Development of a general solution to the problem of validity generalization. Journal of Applied Psychology, 62 (5), 529-540. Schulze, R. (2004). Meta-Analysis: A Comparison of Approaches. Cambridge, MA: Hogrefe & Huber. Schulze, R., Holling, H., Grofsmann, H., Jutting, A., & Brocke, M. (2001). Differences in the results of two meta-analytical approaches. In R. Schulze, H. Holling, & D. Bohning (Eds.), Meta-analysis: New developments and applications in medical and social sciences (pp. 21-39). Cambridge, MA: Hogrefe & Huber Publishers. Schwarzer, G., Antes, G., & Schumacher, M. ( 2002). Inflation of Type I error rate in two statistical tests for the detection of publ ication bias in meta-analyses with binary outcomes. Statistics in Medicine, 21, 2465-2477. Shadish, W. R., & Haddock, C. K. (1994). Combing estimates of effect size. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 261-281). New York, NY: Russell Sage Foundation.

PAGE 155

144 Sharpe, D. (1997). Of apples and oranges, file dr awers and garbage: Why validity issues in metaanalysis will not go away. Clinical Psychology Review, 17 (8), 881-901. Simes, R. J. (1986). Confronting publication bias: A cohort design for meta-analysis. Statistics in Medicine, 6, 11-30. Smith, M. L. (1980). Publication bias and meta-analysis. Evaluation in Education, 4, 22-24. Sterling, T. D. (1959). Publication decisions and their possible effects on inferences drawn from tests of significance—Or vice versa. Journal of the American Statistical Association, 54(285), 30-34. Sterne, J. A. C., Gavaghan, D. & Egger, M. (200 0). Publication and related bias in meta-analysis: Power of statistical tests and prevalence in the literature. Journal of Clinical Epidemiology, 53, 1119-1129. Sutton, A. J., Abrams, K. R., Jones, D. R., Sheldon, T. A., & Song, F. (2000). Methods for metaanalysis in medical research. New York, NY: John Wiley & Sons, LTD. Tang, J-L, & Liu, J. L. Y. (2000). Misleading funnel plot for detection of bias in meta-analysis. Journal of Clinical Epidemiology, 53, 477-484. Thornton, A. & Lee, P. (2000). Publication bias in meta-analysis: Its causes and consequences. Journal of Clinical Epidemiology 53 207 – 216. Vevea, J. L., & Hedges, L. V. (1995). A genera l linear model for estimating effect size in the presence of publication bias. Psychometrika, 60 (3), 419-435.

PAGE 156

145 APPENDICES

PAGE 157

146 Appendix A: Code for Monte Carl o simulation and calculating public ation bias detection methods. +------------------------------------------------------+ PUB_BIAS.SAS Investigation of methods to detect publication bias Random Effects Meta-Analysis +------------------------------------------------------+; *proc printto print='C:\PUB_BIAS.LST'; options ls = 132 ; proc iml ; +--------------------------------------------------------------+ These are user specifications: +--------------------------------------------------------------+; replicat= 10000 ; N of meta-analyses to simulate ; ******CHANGE TO 10,000 ONCE CODE IS COMPLETE; tau = 0; *TAU0; *Population variance; tau = 0.10; *TAU1; tau = 0.33 ; *TAU2; tau = 0.50; *TAU3; tau = 1.00; *TAU4; KK= 10 ; *K1; N of studies in each meta-analysis; KK=20; *K2; KK=50; *K3; KK=100; *K4; DELTA = 0.0; *D1; Population effect size; DELTA = 0.2; *D2; DELTA = 0.5 ; *D3; DELTA = 0.8; *D4; Select_bias = 0; *SB0; Selection bias = none;

PAGE 158

147 Appendix A (continued): Code for Monte Carlo simulati on and calculating publication bias detection methods. Select_bias = 1 ; *SB1; Selection bias = mild; Select_bias = 2; *SB2; Selection bias = moderate; +---------------------------------------------+ Subroutine to generate a random sample. User specifies the population mean and standard deviation. For population shapes, Fleishman constants are used. Inputs to the subroutine are NN desired sample size mu population mean variance population variance bb,cc,dd Fleishman constants Outputs are Rawdata column vector of NN observations from the specified population +---------------------------------------------+; start gendata(NN,variance,bb,cc,dd,mu,rawdata); seed1=round( 1000000 *ranuni( 0 )); rawdata=rannor(repeat(seed1,nn, 1 )); rawdata = (1 *cc) + (bb*rawdata) + (cc*rawdata## 2 ) + (dd*rawdata## 3 ); rawdata = (rawdata SQRT(variance)) + mu; finish; +---------------------------------------------------------+ Subroutine to calculate RANDOM EFFECTS weighted mean effect size, standard error, and confidence interval for mean. Inputs to the subroutine are di_vec column vector of effect sizes (d) var_di column vector of estimation errors (FIXED EFFECTS variance) tau2 scalar estimate of RANDOM EFFECTS variance

PAGE 159

148 Appendix A (continued): Code for Monte Carlo simulation and calculating publication bias detection methods. Outputs are d_mean = weighted mean d value *****RANDOM EFFECTS Mean ES resum_wt = scalar, sum of the RANDOM EFFECTS weights vistar = column vector of RANDOM EFFECTS total variance for each study d_SE = standard error of d upper90, lower90 = endpoints of 90% CI upper95, lower95 = endpoints of 95% CI upper99, lower99 = endpoints of 99% CI +---------------------------------------------------------+; start mean_d(di_vec,var_di,tau2,d_mean,resum_wt,vi_star,d_SE,upper90,lower90,upper95,lower95,upper99,lower99); calculate RANDOM EFFECTS weighted mean effect size; k = nrow(di_vec); dmean = 0 ; resum_wt = 0 ; vistar = J(k, 1 0 ); do i = 1 to k; d_mean = d_mean + di_vec[i, 1 ]/(var_di[i, 1 ]+tau2); resum_wt = resum_wt + (var_di[i, 1 ]+tau2)##1 ; *******resum_wt is scalar of the sum of random effect weights; vi_star[i, 1 ] = var_di[i, 1 ]+tau2; ****vi_star is the vector of RANDOM EFFECTS total variance; end; dmean = dmean/resumwt; *****Random Effects Mean ES; d_SE = SQRT(resum_wt##1 ); upper90 = dmean + 1.65#dSE; lower90 = dmean 1.65#dSE; upper95 = d_mean + 1.96#d_SE; lower95 = dmean 1.96#dSE; upper99 = dmean + 2.576#dSE; lower99 = d_mean 2.576#d_SE; finish; +---------------------------------------------------------+

PAGE 160

149 Appendix A (continued): Code for Monte Carlo simulation and calculating publication bias detection methods. Subroutine to calculate the Q test of homogeneity. Inputs to the subroutine are divec column vector of effect sizes (d) n_vec matrix (k X 2) of sample sizes corresponding to each effect size Outputs are QQ = the obtained value of Q dplus = weighted mean d value (FIXED EFFECTS MEAN EFFECT SIZE) prob_qq1 = chi-square probability associated with QQ fesum_wt = scalar, sum of the FIXED EFFECTS weights vardi = column vector of variances of effect sizes (fixed effects variance) +---------------------------------------------------------+; start calcq(di_vec,n_vec,qq,d_plus,prob_qq1,fesum_wt,var_di); calculate FIXED EFFECTS variance for each effect size; k = nrow(divec); vardi=J(k, 1 0 ); do i = 1 to k; var_di[i, 1 ] = ((n_vec[i, 1 ]+n_vec[i, 2 ])/(n_vec[i, 1 ]#n_vec[i, 2 ])) + ((di_vec[i, 1 ]##2 )/(2 #(n_vec[i, 1 ]+n_vec[i, 2 ]))); end; calculate FIXED EFFECTS weighted mean effect size; d_plus = 0 ; fesumwt = 0 ; do i = 1 to k; d_plus = d_plus + di_vec[i, 1 ]/var_di[i, 1 ]; fesum_wt = fesum_wt + var_di[i, 1 ]##-1 ; ****sum of the FIXED EFFECTS weights (1/var); end; d_plus = d_plus/fesum_wt; ****FIXED EFFECTS Mean ES;

PAGE 161

150 Appendix A (continued): Code for Monte Carlo simulation and calculating publication bias detection methods. calculate Q; QQ = 0 ; do i = 1 to k; QQ = QQ + ((di_vec[i, 1 ] d_plus)## 2 /var_di[i, 1 ]); end; prob_qq1 = 1 PROBCHI(QQ,k1 ); print di_vec var_di; print d_plus qq prob_qq1; finish; *+-----------------------------------------------------------+ Subroutine to calculate OLS and WLS tests of models. Inputs to the subroutine are divec column vector of effect sizes (d) n_vec matrix (k X 2) of sample sizes corresponding to each effect size XMatrix Matrix of potential moderator variables For tests of publication bias, vi are predictors vi reciprocals of variances Outputs are B_wls regression weights for WLS SEB Standard errors of the WLS weights B_ols regression weights for OLS SE_B_ols Standard errors of the OLS weights +---------------------------------------------------------+; start calcreg(di_vec,n_vec,X_Matrix,vi,B_wls,SE_B,B_ols,SE_B_ols); *+-------------------------------------------+ Weighted least squares estimation using Vi as weights +-------------------------------------------+;

PAGE 162

151 Appendix A (continued): Code for Monte Carlo simulation and calculating publication bias detection methods. k = nrow(divec); X = J(k, 1 1 )||X_Matrix; B_wls = INV(X`*DIAG(vi)*X)*X`*DIAG(vi)*di_vec; covb = INV(X`*DIAG(vi)*X); SE_B = SQRT(vecdiag(cov_b)); RSS_wls = (di_vec X*B_wls)`*DIAG(vi)*(di_vec-X*B_wls); ResidMS = RSSwls/(SUM(vi)-2); SE_B = SE_B (1/SQRT(Resid_MS)); *+-------------------------------------------+ Ordinary least squares estimation +-------------------------------------------+; Bols =INV(X`*X)*X`*di_vec; cov_b = INV(X`*X); SE_B_ols = SQRT(vecdiag(cov_b)); finish; +-----------------------------------------------------------------+ Subroutine Kendall Computes the Kendall Tau for Norman Cliff ordinal level analyses. Arguements to the subroutines are: A B = vectors of observed data for the two variables (A will be the observed tx effects B will be the variance of the tx effects) N = sample size Returned are: T_AB = Kendall Tau Coefficient Y and X1 UNTIE_A = proportion of scores that are not tied on A UNTIE_B = proportion of scores that are not tied on B VARTAB = VARIANCE of Y AND X1 +-----------------------------------------------------------------+; START KENDALL(A,B,N,T_AB,untie_A,untie_B,VART_AB,Z_TEST); print A B N;

PAGE 163

152 Appendix A (continued): Code for Monte Carlo simulation and calculating publication bias detection methods. +-----------------------------------------------------------------+ A = Y the criterion variable the A matrix is the criterion matrix dihy ####For pub bias study the A=observed tx effects +-----------------------------------------------------------------+; DOM_MTXA = J(N,N, 0 ); print 'Beginning:' DOM_MTXA; tiesA = 0 ; countsA = 0 ; do i = 1 to N; do j = 1 to N; if A[i, 1 ] > A[j, 1 ] then do; DOM_MTXA[i,j] = 1 ; end; if A[i, 1 ] < A[j, 1 ] then do; DOM_MTXA[i,j] = 1 ; end; if A[i, 1 ] = A[j, 1 ] then do; ties_A = ties_A + 1 ; end; counts_A = counts_A + 1 ; end; end; untie_A = 1 (ties_A N)/(counts_A N); print 'End of Loop:' DOM_MTXA; +-----------------------------------------------------------------+ B = Predictor X1 the B matrix is the predictor matrix dih1 ####For pub bias study the B=variance of the tx effects

PAGE 164

153 Appendix A (continued): Code for Monte Carlo simulation and calculating publication bias detection methods. +-----------------------------------------------------------------+; DOM_MTXB = J(N,N, 0 ); print 'Beginning:' DOM_MTXB; tiesB = 0 ; counts_B = 0 ; do i = 1 to N; do j = 1 to N; if B[i, 1 ] > B[j, 1 ] then do; DOM_MTXB[i,j] = 1 ; end; if B[i, 1 ] < B[j, 1 ] then do; DOM_MTXB[i,j] = 1 ; end; if B[i, 1 ] = B[j, 1 ] then do; ties_B = ties_B + 1 ; end; counts_B = counts_B + 1 ; end; end; untie_B = 1 (ties_B N)/(counts_B N); print 'End of Loop:' DOM_MTXB; +-----------------------------------------------------------------+ D = (the Y criterion matrix) (the X1 predictor matrix) D = (dihy) (dih1) = tih1y Produces T_AB (tau for criterion Y and predictor X1) +-----------------------------------------------------------------+; DOM_MTXD = J(N,N, 0 ); print 'Beginning:' DOM_MTXD;

PAGE 165

154 Appendix A (continued): Code for Monte Carlo simulation and calculating publication bias detection methods. do i = 1 to N; do j = 1 to N; DOM_MTXD[i,j] = DOM_MTXA[i,j]#DOM_MTXB[i,j]; end; end; print 'End of Loop:' DOM_MTXD; MTXD_sum = DOM_MTXD[,+]; print 'MTXD Sums:' MTXD_sum; T_AB = MTXD_sum[+,] #( 1 /(n#(n1 ))); print T_AB untie_A untie_B; +--------------------------VART_AB------------------------------------+; MTXF = J(N, 1 0 ); do i = 1 to N; *do j = 1 to N; MTX_F[i, 1 ] = (MTXD_sum [i, 1 ]/(n-1 )); *end; end; print MTX_F; ****creates [1 .75 .50 .25 1]; MTXG = J(N, 1 0 ); do i = 1 to N; *do j = 1 to N; MTX_G[i, 1 ] = (MTX_F[i, 1 ] T_AB)## 2 ; *end; end; print MTX_G; ****creates [(1-.7)2 (.75-.7)2 etc.]; MTXG_sum = 1 /(n-1 )#(MTX_G[+,]);

PAGE 166

155 Appendix A (continued): Code for Monte Carlo simulation and calculating publication bias detection methods. print MTXG_sum; ****sums MTX_G and divides by n-1 S2ti1y=.1063; MTX_H = J(N,N, 0 ); do i = 1 to N; do j = 1 to N; MTX_H[i,j] = DOM_MTXD[i,j]#DOM_MTXD[i,j]; end; end; print MTXH; ****squares MTXD; MTXH_sum = MTX_H[+,+]; print MTXH_sum; ****Sums MTX_H=18; NUMER_AB = MTXH_sum ((n) # (n1 ) # (T_AB#T_AB)); DENOM_AB = n#(n1 ) -1 ; VARTAB = NUMERAB/DENOM_AB; print VART_AB; ****Final Equation B6=.4316; vart_ab = (( 4 #(n-2 )#(MTXG_sum))+( 2 #VART_AB)) / (n#(n1 )); print vart_ab; ******Final equation B3=.1069; IF vart_ab > 0 THEN DO ; ZTEST = (T_AB /SQRT(vart_ab)); end; if vart_ab = 0 then do;

PAGE 167

156 Appendix A (continued): Code for Monte Carlo simulation and calculating publication bias detection methods. Z_TEST = 5.00; END; print Z_TEST; FINISH; +--------------------------------------+ Main program Generates samples, calls subroutines, computes rejection rates. +--------------------------------------+; ********************************************************************************************************* ********************************************************************************************************; do samplesize = 1 to 9 ; *CHANGE TO 1 TO 9 ONCE CODE COMPLETE; if samplesize = 1 then njs={ 5 5 }; if sample_size = 2 then njs={ 10, 10}; if samplesize = 3 then njs={ 50, 50}; if samplesize = 4 then njs={ 4 6 }; if sample_size = 5 then njs={ 8 12}; if samplesize = 6 then njs={ 40, 60}; if samplesize = 7 then njs={ 6 4 }; if sample_size = 8 then njs={ 12, 8 }; if sample_size = 9 then njs={ 60, 40}; do set_variances = 1 to 4 ; *CHANGE TO 1 TO 4 ONCE CODE COMPLETE; if setvariances = 1 then sds={ 1.0, 1.0}; if setvariances = 2 then sds={ 1.0, 2.0}; if set_variances = 3 then sds={ 1.0, 4.0}; if set_variances = 4 then sds={ 1.0, 8.0}; POOLED_VAR=( 0.5)#SUM(sds); POOLED_SD=SQRT(POOLED_VAR); print pooled_sd;

PAGE 168

157 Appendix A (continued): Code for Monte Carlo simulati on and calculating publication bias detection methods. do set_shape = 1 to 1 ; +--------------------------------+ Fleishman Transformations to nonnormality +--------------------------------+; if set_shape = 1 then do; The following give sk= 0, kr= 0; b= 1 ; c= 0 ; d= 0 ; end; if set_shape = 2 then do; The following give sk= 1.00, kr= 3.00; b= .83221632289426 ; c= .12839670935047 ; d= .04803205907079 ; end; if set_shape = 3 then do; The following give sk= 2.00, kr= 6.00; b= 0.82632385761082 ; c= 0.31374908500462 ; d= 0.02270660525731 ; end; +--------------------+ Initialize counters +--------------------+; REJ_EGGWLS = J( 3 1 0 );

PAGE 169

158 Appendix A (continued): Code for Monte Carlo simulation and calculating publication bias detection methods. REJEGGOLS = J( 3 1 0 ); REJBEGV = J( 3 1 0 ); REJ_BEGN = J( 3 1 0 ); REJFUNWLS = J( 3 1 0 ); REJFUNOLS = J( 3 1 0 ); REJ_TRIM = J( 3 1 0 ); AVEesBia = 0 ; AVEvaBia = 0 ; n_generated = 0 ; nsamples= 0 ; esbias = 0 ; varbias = 0 ; SE = J( 1 1 0 ); RMSE = J( 1 1 0 ); INBAND = J( 1 3 0 ); WIDEBAND = J( 1 3 0 ); do rep= 1 to replicat; This starts the big do loop, number of MA = 10000; k=kk; *number of studies, condition that varies (see beginning of code); n_vec=J(K, 2 0 ); study = 0 ; do until (study = k); Inner loop for primary studies; MEANDELTA=DELTA#POOLED_SD; *ALPHA_K IN OLD PUB BIAS CODE; VARDELTA=TAU#POOLED_VAR; mu1={ 0.0, 0.0}; Pop means for group 1; randomly generate a population mean for each study consistent with the random effects model;

PAGE 170

159 Appendix A (continued): Code for Monte Carlo simulation and calculating publication bias detection methods. ALPHA_K=RANNOR( 0 )#SQRT(VARDELTA)+MEANDELTA; Mean_exp=mu1[ 1 1 ]+alpha_k; ***MU2; randomly generate a sample size for each study; n1=( 0.5#njs[1 1 ]#rannor( 0 ))+ njs[ 1 1 ]; *IN TAU2WEIGHTS.SAS CODE THE VALUE IS 0.2 INSTEAD OF 0.5; n1=round(n1); if n1< 3 then n1= 3 ; n2=( 0.5#njs[2 1 ]#rannor( 0 ))+njs[ 2 1 ]; *IN TAU2WEIGHTS.SAS CODE THE VALUE IS 0.2 INSTEAD OF 0.5; n2=round(n2); if n2< 3 then n2= 3 ; *print meandelta vardelta alphaK Meanexp n1 n2; run gendata(n1,sds[ 1 1 ],b,c,d,mu1[ 1 1 ],z1); run gendata(n2,sds[ 2 1 ],b,c,d,mean_exp,z2); calculate sample means, SS, di and cd for primary studies; xbar1 = (J( 1 ,n1,1 )*z1)/n1; xbar2 = (J( 1 ,n2,1 )*z2)/n2; ss1 = (J( 1 ,n1,1 )*(z1## 2 )) ((J( 1 ,n1,1 )*z1)## 2 /n1); ss2 = (J( 1 ,n2,1 )*(z2## 2 )) ((J( 1 ,n2,1 )*z2)## 2 /n2); njstemp = n1//n2; di = ((xbar2 xbar1)/sqrt((ss1 + ss2)/(J( 1 2 1 )*njstemp 2 )))#( 1 (3 /(4 #(J(1 2 1 )*njstemp)9 ))); sw = SQRT((ss1 + ss2)/(n1 + n2 2 )); tvalue = (xbar1-xbar2)/(sw#(sqrt( 1 /n1 + 1 /n2))); df = n1 + n2 2 ; t_pvalue = 2 #(1 -probt(abs(t_value),df)); print xbar1 xbar2 ss1 ss2 n1 n2 njstemp sw t_value df t_pvalue;

PAGE 171

160 Appendix A (continued): Code for Monte Carlo simulation and calculating publication bias detection methods. Use Selectbias to determine if study is included in meta-analysis; if Selectbias = 1 then do; beta = 2 ; alpha = 1.5; end; if Selectbias = 2 then do; beta = 4 ; alpha = 1.5; end; if Select_bias ^= 0 then do; probselect = exp(1 *beta*tpvalue**alpha); if ranuni( 0 ) < prob_select then keepme = 1 ; else keepme = 0 ; end; if Selectbias = 0 then do; keepme = 1 ; prob_select = 1 ; end; n_generated = n_generated + 1 ; if keepme = 1 then do; Study is included in meta-analysis; study = study + 1 ; increment the 'study' variable; if study = 1 then di_vec = di; if study > 1 then di_vec = di_vec//di; nvec[study, 1 ]=n1; n_vec[study, 2 ]=n2; end; *print Select_bias t_pvalue prob_select di study; end; End inner loop for primary studies; print n_vec di_Vec; ********************************************************************************************************* ********************************************************************************************************;

PAGE 172

161 Appendix A (continued): Code for Monte Carlo simulation and calculating publication bias detection methods. calculate Q test of homogeneity; run calcq(di_vec,n_vec,qq,d_plus,prob_qq1,fesum_wt,var_di); Compute estimates of Tau-squared; Estimator #2: Q based REVC Estimate; CC = (J( 1 ,K,1 )*var_di##1 ) ((J( 1 ,K,1 )*var_di##2 ) / (J( 1 ,K,1 )*var_di##1 )); Tau2 = (QQ (K 1 )) / cc; *****REVC; if tau2 < 0 then tau2 = 0 ; compute mean d and CIs using estimator 2; run mean_d(di_vec,var_di,tau2,d_mean,resum_wt,vi_star,d_SE,upper90,lower90,upper95,lower95,upper99,lower99); esbias = d_mean delta; *supplement d_plus for FIXED EFFECTS; AVEesBia = AVEesbia + esbias; varbias = tau2 tau; *not applicable for FIXED EFFECTS; AVEvaBia = AVEvabia + varbias; SE[ 1 1 ] = SE[ 1 1 ] + d_SE; RMSE[ 1 1 ] = RMSE[ 1 1 ] + (dmean delta)## 2 ; *supplement dplus for FIXED EFFECTS; if (delta < upper99 & delta > lower99) then INBAND[ 1 1 ] = INBAND[ 1 1 ] + 1 ; if (delta < upper95 & delta > lower95) then INBAND[ 1 2 ] = INBAND[ 1 2 ] + 1 ; if (delta < upper90 & delta > lower90) then INBAND[ 1 3 ] = INBAND[ 1 3 ] + 1 ; WIDEBAND[ 1 1 ] = WIDEBAND[ 1 1 ] + (upper99 lower99); WIDEBAND[ 1 2 ] = WIDEBAND[ 1 2 ] + (upper95 lower95); WIDEBAND[ 1 3 ] = WIDEBAND[ 1 3 ] + (upper90 lower90); *free d_mean d_SE upper90 lower90 upper95 lower95 upper99 lower99;

PAGE 173

162 Appendix A (continued): Code for Monte Carlo simulation and calculating publication bias detection methods. print n_vec di_vec var_di fesum_wt d_plus qq cc vi_star resum_wt d_mean delta esbias AVEesBia tau2 tau varbias AVEvaBia; +---------------------------------------------------+ Standardize the treatment effects for bias analysis +---------------------------------------------------+; vi = J(k, 1 0 ); viinv = j(k, 1 0 ); mean = 0 ; weight = 0 ; tstar = J(k, 1 0 ); Vi_stand = J(k, 1 0 ); devdi = J(k, 1 0 ); RootVi=J(k, 1 0 ); Egger_z = J(k, 1 0 ); do i = 1 to k; vi[i, 1 ] = vi_star[i, 1 ]; ****supplement var_di for FIXED EFFECTS MODEL; vi_inv[i, 1 ] = vi[i, 1 ]##-1 ; **inverse of variance(vi); mean = d_mean; ****supplement d_plus for FIXED EFFECTS MODEL; weight = resum_wt; ****supplement fesum_wt for FIXED EFFECTS MODEL; devdi[i, 1 ] = ABS(divec[i, 1 ] mean); ***For Trim and Fill; Vi_stand[i, 1 ] = vi[i, 1 ] (weight##1 ); ***For Begg; tstar[i, 1 ] = (divec[i, 1 ] mean)/SQRT(Vistand[i, 1 ]); ***For Begg; Root_Vi[i, 1 ] = vi[i, 1 ]##-0.5; *reciprocal of the standard deviation of di (For Egger); Egger_z[i, 1 ] = di_vec[i, 1 ]#Root_Vi[i, 1 ];**For Egger; end; print vi vi_inv mean weight dev_di vi_stand t_star root_vi egger_z;

PAGE 174

163 Appendix A (continued): Code for Monte Carlo simulation and calculating publication bias detection methods. +---------------------------------------------------+ Egger Regression using precision as predictor +---------------------------------------------------+; run calcreg(Egger_z,n_vec,Root_Vi,vi_inv,B_wls,SE_B,B_ols,SE_B_ols); +--------------------------------+ WLS test of regression intercept +--------------------------------+; tWLS = Bwls[ 1 1 ]/SEB[ 1 1 ]; PROB_t= 2 #(1 -probt(abs(t_WLS),k2 )); if probt < .10 then REJEGGWLS[ 1 1 ] = REJEGGWLS[ 1 1 ] + 1 ; if probt < .05 then REJEGGWLS[ 2 1 ] = REJEGGWLS[ 2 1 ] + 1 ; if prob_t < .01 then REJ_EGGWLS[ 3 1 ] = REJ_EGGWLS[ 3 1 ] + 1 ; print b_wls se_b t_wls prob_t rej_EGGwls; +--------------------------------+ OLS test of regression intercept +--------------------------------+; tols = Bols[ 1 1 ]/SEBols[ 1 1 ]; probt = 2 #(1 -probt(abs(tols),k2 )); if prob_t < .10 then REJ_EGGOLS[ 1 1 ] = REJ_EGGOLS[ 1 1 ] + 1 ; if probt < .05 then REJEGGOLS[ 2 1 ] = REJEGGOLS[ 2 1 ] + 1 ; if prob_t < .01 then REJ_EGGOLS[ 3 1 ] = REJ_EGGOLS[ 3 1 ] + 1 ; print b_ols se_b_ols t_ols prob_t rej_EGGols; +---------------------------------------------------+ Kendall Tau using variance of d as predictor +---------------------------------------------------+; run KENDALL(tstar,vi,k,TX1Y,UTA,UT_B,VART_X1Y,Z_TEST); probz = 2 #(1 -probnorm(abs(Ztest))); if prob_z < .10 then REJ_BEGV[ 1 1 ] = REJ_BEGV[ 1 1 ] + 1 ;

PAGE 175

164 Appendix A (continued): Code for Monte Carlo simulation and calculating publication bias detection methods. if prob_z < .05 then REJ_BEGV[ 2 1 ] = REJ_BEGV[ 2 1 ] + 1 ; if prob_z < .01 then REJ_BEGV[ 3 1 ] = REJ_BEGV[ 3 1 ] + 1 ; *print T_X1Y UT_A UT_B VART_X1Y Z_TEST rej_BEGv; +---------------------------------------------------+ Kendall Tau using total n as predictor +---------------------------------------------------+; totaln = nvec J( 2 1 1 ); run KENDALL(t_star,total_n,k,T_X1Y,UT_A,UT_B,VART_X1Y,Z_TEST); probz = 2 #(1 -probnorm(abs(Ztest))); if probz < .10 then REJBEGN[ 1 1 ] = REJBEGN[ 1 1 ] + 1 ; if prob_z < .05 then REJ_BEGN[ 2 1 ] = REJ_BEGN[ 2 1 ] + 1 ; if prob_z < .01 then REJ_BEGN[ 3 1 ] = REJ_BEGN[ 3 1 ] + 1 ; *print T_X1Y UT_A UT_B VART_X1Y Z_TEST rej_BEGn; +---------------------------------------------------+ Funnel Plot +---------------------------------------------------+; run calcreg(di_vec,n_vec,total_n,vi_inv,B_wls,SE_B,B_ols,SE_B_ols); +--------------------------------+ WLS test of regression slope +--------------------------------+; tWLS = Bwls[ 2 1 ]/SEB[ 2 1 ]; PROBt= 2 #(1 -probt(abs(tWLS),k2 )); if prob_t < .10 then REJ_FUNWLS[ 1 1 ] = REJ_FUNWLS[ 1 1 ] + 1 ; if probt < .05 then REJFUNWLS[ 2 1 ] = REJFUNWLS[ 2 1 ] + 1 ; if prob_t < .01 then REJ_FUNWLS[ 3 1 ] = REJ_FUNWLS[ 3 1 ] + 1 ; *print B_wls SE_B t_wls prob_t rej_funwls; +--------------------------------+

PAGE 176

165 Appendix A (continued): Code for Monte Carlo simulation and calculating publication bias detection methods. OLS test of regression slope +--------------------------------+; tols = Bols[ 2 1 ]/SEBols[ 2 1 ]; prob_t = 2 #(1 -probt(abs(t_ols),k2 )); if probt < .10 then REJFUNOLS[ 1 1 ] = REJFUNOLS[ 1 1 ] + 1 ; if probt < .05 then REJFUNOLS[ 2 1 ] = REJFUNOLS[ 2 1 ] + 1 ; if prob_t < .01 then REJ_FUNOLS[ 3 1 ] = REJ_FUNOLS[ 3 1 ] + 1 ; *print B_ols SE_B_ols t_ols prob_t rej_funols; +--------------------------------+ Trim and Fill +--------------------------------+; devrank = rank(dev_di); do i = 1 to k; if di_vec[i, 1 ] < mean then dev_rank[i, 1 ] = -1 #dev_rank[i, 1 ]; end; +------------------+ Left tail check +------------------+; r=-1 #MIN(dev_rank); gamma=k-r; ro=gamma1 ; +------------------+ Right tail check +------------------+; r2=MAX(dev_rank); gamma2=k-r2; ro2=gamma21 ; if ro > 3 then REJ_TRIM[ 1 1 ] = REJ_TRIM[ 1 1 ] +1 ; if ro2 > 3 then REJ_TRIM[ 2 1 ] = REJ_TRIM[ 2 1 ] +1 ; +-----------------------------------------+ Either tail (note: alpha is .10 for this

PAGE 177

166 Appendix A (continued): Code for Monte Carlo simulation and calculating publication bias detection methods. +-----------------------------------------+; if (ro > 3 | ro2 > 3 ) then REJ_TRIM[ 3 1 ] = REJ_TRIM[ 3 1 ] +1 ; *print dev_di di_vec mean dev_rank r gamma ro rej_trim; nsamples=nsamples+ 1 ; end; *end the big loop; +-------------------------------------------------------+ Convert counts of rejected hypotheses into proportions +-------------------------------------------------------+; do row = 1 to 3 ; REJ_EGGWLS[row, 1 ] = REJ_EGGWLS[row, 1 ]/nsamples; REJEGGOLS[row, 1 ] = REJEGGOLS[row, 1 ]/nsamples; REJBEGV[row, 1 ] = REJBEGV[row, 1 ]/nsamples; REJ_BEGN[row, 1 ] = REJ_BEGN[row, 1 ]/nsamples; REJFUNOLS[row, 1 ] = REJFUNOLS[row, 1 ]/nsamples; REJFUNWLS[row, 1 ] = REJFUNWLS[row, 1 ]/nsamples; REJ_TRIM[row, 1 ] = REJ_TRIM[row, 1 ]/nsamples; end; AVEesBia = AVEesBia/nsamples; AVEvaBia = AVEvaBia/nsamples; Nindrawer = (ngenerated (kk#nsamples))/nsamples; print 'Tests for Publication Bias Impact and Detection of Publication Bias'; PRINT select_bias delta tau kk njs sds nsamples N_in_drawer AVEesBia AVEvaBia REJ_EGGWLS REJ_EGGOLS REJ_BEGV REJ_BEGN REJ_FUNWLS REJ_FUNOLS REJ_TRIM; end; end the shape loop; end; end the variances loop; end; end the sample size loop; quit;

PAGE 178

167 Appendix B: Mean effect size and effect size variance bias estimates Table 17. Estimated bias in mean effect size by study condition and magnitude of publication bias. N MeanMinimumMaximumMeanMinimumMaximumMeanMinimumMaximum N umber of Studies (k) 10720-0.0098-0.11190.09970.0810-0.02710.30300.1366-0.02390.4491 20720-0.0118-0.11030.09590.0794-0.03030.29240.1352-0.02620.4417 50720-0.0129-0.11250.09590.0784-0.03200.28740.1344-0.02920.4414 100720-0.0133-0.11130.09430.0781-0.03270.28950.1340-0.02950.4392 Primary Study Sample Size (n1, n2) 4,6320-0.0431-0.11250.00740.0848-0.00460.16230.1702-0.00590.3100 5,5320-0.0291-0.09430.00610.1000-0.00880.22080.1843-0.00800.3649 6,4320-0.0124-0.09390.04980.1195-0.00890.30300.2046-0.01110.4491 8,12320-0.0265-0.07210.00290.0723-0.00370.14150.1325-0.00480.2445 10,10320-0.0116-0.04850.00960.0868-0.00810.18710.1460-0.00580.2863 12,83200.0058-0.04920.08510.1075-0.00600.27230.1679-0.00980.3754 40,60320-0.0121-0.03780.00550.0324-0.03270.08310.0551-0.02950.1254 50,503200.0024-0.01160.02790.0457-0.00580.12080.0680-0.00310.1632 60,403200.0189-0.01130.09970.0639-0.00660.18820.0868-0.00700.2299 Primary Study Group Variance 1:1720-0.0204-0.09490.00490.0690-0.00600.16470.1239-0.00760.3100 1:2720-0.0175-0.11220.02260.0725-0.02020.19120.1277-0.01750.3400 1:4720-0.0098-0.11250.06110.0817-0.03030.24580.1377-0.02730.3905 1:8720-0.0002-0.10970.09970.0936-0.03270.30300.1508-0.02950.4491 Population Effect Size Magnitude 0.07200.0001-0.00800.01020.0002-0.00890.00850.0001-0.01110.0084 0.2720-0.0064-0.02950.02490.06960.01950.12990.12220.03620.2245 0.5720-0.0160-0.07010.06230.1227-0.00090.25820.20900.01210.4104 0.8720-0.0255-0.11250.09970.1243-0.03270.30300.2090-0.02950.4491 Population Effect Size Variance 0.00576-0.0094-0.09140.09180.0744-0.03270.28660.1297-0.02950.4280 0.10576-0.0109-0.09890.09480.0830-0.00810.30300.1418-0.00580.4419 0.33576-0.0122-0.10480.09540.0850-0.00630.29810.1441-0.00740.4491 0.50576-0.0128-0.10890.09800.0822-0.00890.29720.1389-0.00980.4449 1.00576-0.0144-0.11250.09970.0714-0.00880.27350.1207-0.01110.4099 ModerateStrong Magnitude of Publication Bias N one

PAGE 179

168 Appendix B: Mean effect size and effect size variance bias estimates Table 18. Estimated bias in effect size variance by study condition a nd magnitude of publication bias N MeanMinimumMaximumMeanMinimumMaximumMeanMinimumMaximum N umber of Studies (k) 10720-0.0147-0.32670.27560.1299-0.13770.77570.2277-0.11131.1622 20720-0.0328-0.36480.23680.1114-0.14790.69850.2063-0.13021.0971 50720-0.0432-0.37750.21670.1012-0.15920.67140.1951-0.13991.0661 100720-0.0467-0.38060.21540.0978-0.15930.66010.1915-0.14131.0508 Primary Study Sample Size (n1, n2) 4,6320-0.1233-0.38060.07610.0974-0.14780.26390.2598-0.08400.5639 5,5320-0.0575-0.32570.15360.1842-0.10000.45770.3543-0.05320.7987 6,43200.0291-0.32430.27560.3015-0.09510.77570.4858-0.05271.1622 8,12320-0.0867-0.28620.05100.0462-0.14320.17940.1319-0.10620.3506 10,10320-0.0336-0.23260.10070.1078-0.09180.31660.1958-0.08250.5232 12,83200.0362-0.23040.22150.1943-0.08880.58720.2909-0.07790.8256 40,60320-0.0573-0.21730.0211-0.0150-0.15930.06280.0071-0.14130.1116 50,50320-0.0276-0.16510.03530.0158-0.10820.10630.0383-0.09070.1633 60,403200.0117-0.16200.12760.0587-0.10550.24670.0825-0.08630.3430 Primary Study Group Variance 1:1720-0.0722-0.32680.07230.0576-0.10820.25960.1430-0.09070.5274 1:2720-0.0595-0.37520.11390.0752-0.14780.35930.1635-0.12620.6769 1:4720-0.0255-0.38060.17820.1223-0.15930.54460.2196-0.13990.9142 1:87200.0199-0.36370.27560.1853-0.15700.77570.2945-0.14131.1622 Population Effect Size Magnitude 0.0720-0.0380-0.37880.23990.1824-0.08800.77570.3461-0.02401.1622 0.2720-0.0373-0.38060.25120.1598-0.09570.76200.2975-0.03501.1534 0.5720-0.0341-0.37700.25300.0841-0.12080.65540.1490-0.07920.9752 0.8720-0.0280-0.36850.27560.0142-0.15930.53050.0280-0.14130.7101 Population Effect Size Variance 0.005760.04140.00120.27560.10390.00040.59480.16270.00020.9100 0.105760.0175-0.06750.27080.1075-0.06430.63220.1751-0.07920.9615 0.33576-0.0181-0.14820.25800.1238-0.08640.70760.2148-0.10621.0531 0.50576-0.0479-0.20110.25350.1270-0.09360.72720.2350-0.09651.1092 1.00576-0.1645-0.38060.19170.0883-0.15930.77570.2382-0.14131.1622 N oneModerateStrong Magnitude of Publication Bias

PAGE 180

169 Appendix C: Type I error rate estimates Table 19. Type I error rate estimates for conditions when the population effect size variance is 0.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:110460.0780.0640.0220.0030.0000.0810.0660.0250.0030 .0000.1100.0660.0370.0040.0000.1540.0670.0620.0050.000 1:110550.0760.0710.0220.0030.0000.0810.0640.0240.0040 .0000.1080.0650.0380.0030.0000.1550.0710.0590.0040.000 1:110640.0740.0670.0220.0040.0000.0790.0630.0240.0040 .0000.1050.0690.0400.0020.0000.1610.0710.0610.0050.000 1:1108120.0610.0620.0090.0040.0000.0630.0600.0100.0040 .0000.0710.0680.0140.0040.0000.0830.0640.0190.0040.000 1:11010100.0620.0610.0110.0030.0000.0650.0670.0100.0040 .0000.0770.0650.0150.0040.0000.0850.0670.0190.0040.000 1:1101280.0640.0650.0090.0040.0000.0680.0680.0110.0060 .0000.0750.0660.0140.0040.0000.0800.0660.0190.0040.000 1:11040600.0570.0580.0100.0050.0000.0580.0620.0120.0050 .0000.0640.0670.0120.0060.0000.0610.0550.0110.0050.000 1:11050500.0620.0600.0110.0060.0000.0600.0610.0110.0050 .0000.0580.0600.0120.0050.0000.0700.0610.0130.0060.000 1:11060400.0570.0610.0100.0060.0000.0630.0550.0110.0060 .0000.0600.0600.0100.0050.0000.0620.0600.0130.0050.000 1:120460.0580.0490.0550.0130.0010.0760.0490.0630.0140 .0020.1330.0520.1260.0160.0050.2510.0560.2340.0160.009 1:120550.0660.0590.0530.0150.0010.0730.0510.0630.0160 .0020.1410.0570.1200.0160.0050.2370.0540.2200.0140.012 1:120640.0610.0500.0530.0130.0010.0740.0550.0660.0140 .0020.1450.0540.1270.0140.0060.2570.0550.2360.0160.010 1:1208120.0590.0520.0290.0160.0020.0590.0530.0310.0160 .0030.0710.0480.0450.0160.0060.1030.0540.0740.0190.008 1:12010100.0510.0490.0270.0150.0020.0570.0520.0340.0170 .0040.0750.0540.0480.0200.0060.1050.0570.0750.0210.007 1:1201280.0530.0500.0280.0150.0020.0570.0480.0350.0180 .0030.0710.0530.0480.0180.0050.0990.0500.0740.0200.008 1:12040600.0520.0520.0270.0210.0030.0530.0500.0280.0190 .0070.0540.0510.0310.0220.0060.0570.0520.0300.0200.006 1:12050500.0510.0500.0270.0220.0050.0500.0520.0280.0220 .0050.0510.0460.0270.0180.0080.0620.0490.0350.0230.007 1:12060400.0470.0490.0260.0180.0040.0530.0520.0310.0230 .0050.0550.0510.0310.0210.0060.0560.0480.0310.0220.006 1:150460.0590.0480.0820.0260.0130.0880.0480.1130.0220 .0240.2690.0520.3110.0260.0520.5340.0550.5830.0310.108 1:150550.0570.0450.0740.0210.0120.0860.0490.1110.0250 .0230.2520.0520.2950.0250.0510.5110.0540.5580.0260.110 1:150640.0550.0470.0720.0230.0110.0910.0470.1190.0250 .0250.2720.0520.3180.0260.0550.5380.0580.5850.0310.108 1:1508120.0540.0490.0440.0290.0150.0570.0470.0530.0290 .0210.1010.0510.1000.0290.0360.1920.0520.1990.0290.062 1:15010100.0530.0490.0440.0280.0150.0640.0480.0560.0280 .0230.1130.0500.1090.0300.0360.1830.0480.1880.0340.064 1:1501280.0510.0470.0430.0280.0160.0570.0460.0490.0280 .0210.1010.0500.1020.0320.0360.1890.0500.2010.0340.059 1:15040600.0450.0480.0390.0350.0190.0470.0450.0410.0350 .0210.0550.0450.0440.0310.0240.0690.0480.0570.0350.032 1:15050500.0480.0440.0390.0310.0200.0470.0430.0390.0330 .0210.0580.0490.0490.0350.0300.0750.0470.0570.0320.034 1:15060400.0500.0480.0410.0340.0180.0460.0430.0390.0330 .0240.0510.0420.0450.0300.0280.0680.0490.0540.0360.033 1:1100460.0560.0490.0840.0260.0210.1260.0480.1680.0280 .0430.4840.0500.5530.0320.1190.8340.0610.8720.0390.250 1:1100550.0540.0470.0790.0250.0220.1210.0480.1600.0270 .0450.4560.0530.5180.0320.1250.8230.0620.8620.0390.246 1:1100640.0580.0480.0850.0250.0210.1220.0460.1670.0240 .0440.4860.0540.5540.0350.1170.8410.0570.8740.0350.248 1:11008120.0550.0470.0530.0340.0220.0700.0470.0710.0320 .0350.1650.0480.1830.0340.0700.3400.0490.3820.0400.123 1:110010100.0480.0450.0480.0290.0210.0680.0460.0680.0300 .0360.1620.0440.1760.0330.0780.3420.0460.3810.0370.126 1:11001280.0500.0480.0530.0290.0200.0680.0470.0710.0330 .0340.1660.0500.1800.0360.0700.3400.0470.3810.0360.120 1:110040600.0470.0480.0440.0360.0240.0460.0480.0470.0370 .0310.0630.0460.0540.0370.0370.0900.0450.0710.0410.045 Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 181

170 Appendix C (continued): Type I error rate estimates Table 19 (continued). Type I error rate estimates for condi tions when the population effect size variance is 0.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:110050500.0450.0420.0430.0340.0250.0500.0480.0480.0400 .0290.0650.0460.0580.0400.0360.0970.0510.0820.0420.044 1:110060400.0430.0440.0430.0370.0240.0440.0440.0420.0390 .0300.0700.0480.0600.0400.0390.0940.0460.0770.0400.043 1:210460.0810.0710.0240.0030.0000.0790.0640.0290.0020 .0000.1110.0680.0380.0020.0000.1580.0690.0610.0030.000 1:210550.0750.0670.0250.0030.0000.0860.0690.0290.0040 .0000.1090.0650.0400.0040.0000.1590.0690.0710.0050.000 1:210640.0730.0650.0270.0040.0000.0800.0680.0270.0040 .0000.1150.0680.0480.0040.0000.1740.0660.0770.0040.000 1:2108120.0650.0630.0110.0040.0000.0670.0660.0130.0040 .0000.0740.0650.0150.0050.0000.0900.0660.0210.0040.000 1:21010100.0650.0660.0140.0040.0000.0650.0620.0140.0050 .0000.0720.0620.0160.0040.0000.0860.0650.0230.0040.000 1:2101280.0650.0620.0160.0050.0000.0650.0630.0140.0050 .0000.0730.0600.0200.0050.0000.0960.0650.0280.0060.000 1:21040600.0570.0620.0140.0040.0000.0590.0610.0150.0050 .0000.0600.0630.0180.0060.0000.0640.0620.0180.0040.000 1:21050500.0640.0620.0200.0060.0000.0590.0610.0180.0050 .0000.0610.0620.0190.0060.0000.0630.0600.0180.0050.000 1:21060400.0600.0600.0250.0070.0000.0610.0580.0230.0050 .0000.0610.0580.0230.0060.0000.0640.0630.0220.0060.000 1:220460.0650.0540.0540.0120.0020.0760.0550.0650.0100 .0020.1350.0570.1260.0130.0050.2440.0570.2230.0140.010 1:220550.0680.0560.0610.0150.0010.0790.0550.0770.0170 .0030.1380.0510.1340.0140.0070.2480.0540.2440.0150.010 1:220640.0650.0520.0660.0170.0010.0770.0520.0820.0170 .0020.1550.0580.1640.0180.0040.2860.0560.2900.0180.009 1:2208120.0540.0540.0300.0160.0020.0570.0540.0320.0160 .0020.0730.0560.0460.0160.0040.0980.0520.0700.0150.007 1:22010100.0570.0530.0360.0200.0020.0610.0490.0410.0160 .0030.0700.0490.0520.0180.0050.1160.0580.0900.0190.009 1:2201280.0550.0470.0390.0200.0020.0630.0490.0440.0200 .0030.0840.0540.0650.0190.0050.1210.0540.1060.0210.011 1:22040600.0500.0510.0310.0180.0040.0480.0500.0320.0180 .0040.0500.0520.0340.0180.0070.0570.0540.0410.0210.007 1:22050500.0510.0500.0390.0220.0050.0510.0510.0420.0210 .0040.0550.0490.0420.0200.0070.0600.0480.0460.0200.008 1:22060400.0560.0480.0430.0220.0050.0520.0510.0440.0220 .0040.0540.0520.0500.0250.0060.0660.0530.0560.0260.008 1:250460.0620.0540.0810.0220.0110.0910.0500.1120.0190 .0220.2570.0540.2940.0210.0600.5120.0530.5610.0220.118 1:250550.0590.0460.0790.0240.0120.0890.0530.1210.0270 .0240.2770.0490.3320.0240.0630.5380.0550.5990.0270.128 1:250640.0610.0490.0970.0320.0130.1010.0510.1480.0310 .0230.3080.0550.3770.0330.0600.6080.0590.6830.0370.118 1:2508120.0470.0450.0440.0240.0130.0590.0500.0550.0240 .0240.1030.0510.1050.0260.0430.1820.0520.1930.0290.071 1:25010100.0510.0470.0520.0290.0140.0630.0500.0630.0300 .0220.1130.0530.1230.0320.0440.1990.0460.2290.0300.075 1:2501280.0520.0470.0560.0320.0140.0620.0470.0700.0300 .0210.1240.0510.1410.0330.0430.2280.0550.2690.0400.076 1:25040600.0430.0440.0420.0270.0180.0480.0510.0430.0300 .0240.0580.0520.0530.0320.0340.0720.0580.0700.0370.043 1:25050500.0490.0490.0520.0330.0180.0480.0480.0520.0330 .0250.0600.0500.0600.0350.0370.0780.0480.0790.0350.040 1:25060400.0520.0460.0610.0330.0210.0510.0450.0600.0360 .0250.0640.0510.0760.0420.0330.0910.0520.0940.0440.044 1:2100460.0600.0540.0850.0240.0200.1260.0490.1660.0240 .0460.4600.0510.5220.0230.1320.8150.0490.8560.0250.267 1:2100550.0550.0450.0920.0240.0220.1240.0480.1750.0290 .0470.4760.0520.5560.0340.1280.8450.0570.8860.0380.269 1:2100640.0580.0480.1090.0340.0220.1400.0480.2100.0330 .0440.5410.0560.6420.0400.1240.8930.0740.9280.0550.264 1:21008120.0500.0460.0480.0260.0190.0660.0510.0690.0290 .0390.1550.0460.1700.0300.0860.3180.0480.3650.0320.153 1:210010100.0500.0470.0540.0320.0190.0730.0510.0820.0360 .0400.1830.0490.2100.0350.0840.3670.0540.4310.0430.155 Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 182

171 Appendix C (continued): Type I error rate estimates Table 19 (continued). Type I error rate estimates for condi tions when the population effect size variance is 0.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:21001280.0530.0480.0650.0390.0250.0710.0450.0920.0340 .0360.2010.0520.2500.0410.0810.4130.0670.4950.0550.146 1:210040600.0440.0490.0440.0310.0280.0490.0500.0500.0390 .0330.0660.0550.0670.0360.0520.0950.0590.0970.0430.071 1:210050500.0480.0430.0560.0370.0280.0540.0480.0610.0400 .0340.0690.0430.0780.0370.0510.1190.0510.1270.0440.075 1:210060400.0500.0500.0660.0410.0220.0530.0460.0660.0410 .0330.0780.0550.0890.0480.0480.1290.0680.1320.0630.068 1:410460.0830.0730.0300.0030.0000.0870.0730.0340.0030 .0000.1150.0700.0510.0030.0000.1660.0690.0830.0030.000 1:410550.0840.0680.0380.0030.0000.0890.0700.0370.0050 .0000.1210.0700.0560.0040.0000.1770.0710.0950.0040.000 1:410640.0750.0630.0390.0060.0000.0850.0650.0430.0050 .0000.1280.0640.0780.0050.0000.2010.0700.1230.0070.000 1:4108120.0680.0690.0160.0040.0000.0670.0650.0180.0040 .0000.0780.0720.0250.0060.0000.0860.0690.0320.0050.000 1:41010100.0680.0630.0230.0040.0000.0670.0630.0210.0040 .0000.0750.0650.0260.0050.0000.0970.0640.0430.0050.000 1:4101280.0680.0600.0260.0060.0000.0700.0640.0280.0070 .0000.0880.0630.0370.0070.0000.1070.0680.0520.0050.000 1:41040600.0560.0630.0320.0040.0000.0630.0650.0350.0060 .0000.0630.0660.0350.0070.0000.0640.0700.0460.0060.000 1:41050500.0660.0660.0430.0070.0000.0610.0630.0410.0070 .0000.0670.0600.0450.0080.0000.0690.0620.0520.0070.000 1:41060400.0650.0580.0480.0060.0000.0630.0540.0500.0060 .0000.0720.0640.0560.0070.0000.0750.0610.0620.0100.000 1:420460.0710.0590.0630.0100.0010.0790.0530.0800.0110 .0010.1440.0580.1400.0110.0050.2600.0580.2680.0120.014 1:420550.0680.0530.0780.0160.0010.0820.0580.0990.0160 .0030.1560.0560.1780.0150.0050.2910.0550.3150.0140.016 1:420640.0590.0510.0940.0190.0010.0840.0530.1170.0190 .0020.1810.0560.2310.0220.0050.3460.0640.4010.0220.013 1:4208120.0570.0580.0420.0140.0020.0630.0570.0480.0150 .0030.0770.0550.0660.0150.0050.1060.0600.0990.0160.010 1:42010100.0560.0510.0510.0180.0020.0610.0560.0570.0210 .0030.0840.0510.0830.0180.0070.1310.0530.1420.0180.010 1:4201280.0560.0450.0590.0190.0020.0630.0490.0630.0210 .0020.0880.0520.1070.0250.0050.1460.0550.1700.0250.010 1:42040600.0470.0540.0550.0170.0050.0510.0560.0580.0200 .0050.0540.0610.0680.0220.0050.0670.0670.0860.0290.008 1:42050500.0520.0540.0690.0220.0030.0520.0510.0760.0220 .0040.0590.0520.0800.0200.0070.0700.0510.1030.0220.007 1:42060400.0540.0500.0850.0250.0030.0600.0470.0880.0230 .0030.0620.0500.0920.0240.0060.0820.0610.1170.0310.007 1:450460.0650.0570.0960.0190.0130.0950.0550.1350.0190 .0280.2650.0500.3340.0190.0690.5320.0500.6020.0180.160 1:450550.0590.0480.1080.0230.0120.1000.0500.1590.0260 .0300.3000.0480.3960.0250.0730.5940.0570.6980.0290.159 1:450640.0590.0480.1230.0300.0130.1120.0480.1980.0300 .0260.3680.0560.4980.0390.0710.7010.0620.8030.0430.156 1:4508120.0500.0520.0510.0230.0130.0600.0490.0660.0220 .0260.1100.0560.1310.0250.0550.1870.0540.2410.0230.103 1:45010100.0540.0490.0770.0290.0130.0660.0470.0890.0300 .0260.1320.0500.1760.0290.0580.2430.0520.3200.0340.115 1:4501280.0560.0490.0790.0310.0140.0730.0490.1050.0320 .0270.1570.0550.2240.0390.0540.3050.0640.4020.0480.105 1:45040600.0450.0490.0660.0260.0180.0550.0590.0760.0340 .0300.0600.0620.0920.0380.0470.0830.0790.1320.0530.079 1:45050500.0470.0500.0880.0350.0170.0560.0500.0900.0340 .0320.0730.0510.1220.0370.0540.1030.0510.1750.0370.085 1:45060400.0530.0460.1020.0320.0160.0580.0520.1110.0380 .0300.0800.0570.1340.0420.0480.1180.0750.1870.0510.086 1:4100460.0540.0510.1020.0190.0220.1290.0570.1880.0200 .0520.4580.0510.5610.0190.1490.8270.0520.8830.0220.317 1:4100550.0590.0520.1150.0300.0220.1330.0470.2230.0280 .0480.5120.0530.6380.0340.1510.8770.0560.9270.0390.318 1:4100640.0590.0470.1400.0330.0210.1640.0490.2860.0360 .0470.6300.0640.7570.0510.1480.9420.0870.9720.0730.303 Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 183

172 Appendix C (continued): Type I error rate estimates Table 19 (continued). Type I error rate estimates for condi tions when the population effect size variance is 0.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:41008120.0500.0470.0600.0240.0240.069 0.0520.0840.0280.0430.1650.0510.219 0.0270.1160.3400.0580.4430.0290.221 1:410010100.0550.0490.0780.0370.0210.080 0.0480.1180.0340.0460.2140.0450.297 0.0320.1120.4390.0480.5710.0350.223 1:41001280.0580.0470.0930.0350.0180.089 0.0500.1400.0380.0430.2720.0560.367 0.0430.1030.5450.0740.6740.0600.214 1:410040600.0480.0510.0720.0300.0240.051 0.0540.0840.0350.0460.0720.0750.125 0.0490.0970.1160.1040.2050.0830.165 1:410050500.0500.0470.0990.0380.0280.058 0.0490.1100.0380.0490.0880.0490.153 0.0380.0990.1520.0480.2540.0370.172 1:410060400.0490.0420.1060.0340.0260.062 0.0530.1210.0420.0460.1130.0750.182 0.0570.0940.2000.1000.2760.0780.163 1:810460.0840.0720.0460.0020.0000.091 0.0710.0460.0030.0000.1210.0710.068 0.0030.0000.1720.0720.1040.0050.000 1:810550.0870.0690.0500.0050.0000.087 0.0680.0590.0040.0000.1240.0650.082 0.0040.0000.1970.0680.1450.0050.000 1:810640.0830.0650.0580.0070.0000.089 0.0640.0720.0070.0000.1400.0680.112 0.0060.0000.2250.0690.1850.0080.000 1:8108120.0640.0700.0260.0030.0000.071 0.0740.0310.0040.0000.0770.0700.039 0.0040.0000.0920.0670.0510.0040.000 1:81010100.0740.0670.0380.0050.0000.067 0.0600.0400.0040.0000.0870.0640.056 0.0060.0000.1060.0660.0750.0070.000 1:8101280.0720.0570.0450.0060.0000.073 0.0600.0510.0070.0000.0920.0630.067 0.0070.0000.1250.0640.0990.0080.000 1:81040600.0650.0720.0570.0070.0000.059 0.0650.0650.0070.0000.0670.0710.073 0.0100.0000.0690.0770.0830.0110.000 1:81050500.0620.0600.0730.0080.0000.062 0.0620.0700.0080.0000.0680.0610.087 0.0100.0000.0790.0620.1040.0110.000 1:81060400.0670.0610.0860.0090.0000.068 0.0580.0810.0080.0000.0760.0650.100 0.0110.0000.0760.0640.1150.0120.000 1:820460.0690.0600.0920.0110.0020.079 0.0560.1020.0100.0030.1420.0580.177 0.0100.0060.2630.0590.3080.0110.016 1:820550.0690.0580.1100.0160.0010.087 0.0550.1310.0170.0030.1650.0580.227 0.0170.0070.3150.0540.4020.0160.017 1:820640.0650.0480.1350.0190.0020.089 0.0530.1580.0220.0030.2100.0560.308 0.0250.0070.4000.0610.5070.0230.016 1:8208120.0600.0600.0580.0140.0020.058 0.0570.0630.0170.0030.0810.0570.095 0.0160.0060.1150.0570.1450.0130.013 1:82010100.0580.0530.0780.0190.0020.061 0.0510.0870.0190.0030.0940.0520.129 0.0190.0060.1480.0570.2010.0200.012 1:8201280.0600.0520.0980.0200.0010.068 0.0490.1060.0200.0020.1070.0520.166 0.0250.0040.1840.0600.2700.0270.011 1:82040600.0540.0600.0920.0210.0030.055 0.0600.0970.0230.0040.0600.0640.119 0.0280.0060.0720.0770.1490.0380.009 1:82050500.0560.0550.1190.0230.0020.056 0.0500.1270.0200.0040.0610.0490.142 0.0250.0060.0800.0570.1780.0270.007 1:82060400.0600.0510.1400.0270.0030.059 0.0530.1440.0270.0030.0700.0530.162 0.0300.0050.1010.0640.2100.0340.007 1:850460.0620.0500.1200.0160.0120.091 0.0540.1620.0170.0300.2690.0520.375 0.0170.0850.5410.0540.6620.0180.193 1:850550.0620.0470.1400.0220.0120.105 0.0490.2020.0210.0300.3210.0560.477 0.0270.0910.6460.0580.7820.0320.206 1:850640.0610.0440.1740.0310.0120.122 0.0500.2540.0310.0280.4260.0620.605 0.0400.0820.7820.0710.8850.0520.203 1:8508120.0540.0540.0770.0220.0150.063 0.0540.0890.0210.0260.1110.0540.177 0.0240.0700.2150.0590.3270.0260.141 1:85010100.0570.0480.1000.0300.0140.072 0.0510.1350.0290.0280.1460.0510.255 0.0350.0770.2970.0480.4530.0270.160 1:8501280.0570.0500.1240.0310.0110.085 0.0490.1610.0300.0300.1980.0530.332 0.0370.0750.3950.0640.5750.0430.151 1:85040600.0500.0530.1200.0270.0170.051 0.0600.1210.0300.0310.0640.0700.170 0.0480.0740.0950.1000.2430.0790.132 1:85050500.0530.0490.1440.0330.0160.056 0.0490.1590.0340.0310.0830.0520.213 0.0360.0770.1260.0510.3020.0370.146 1:85060400.0580.0490.1720.0360.0160.060 0.0480.1820.0360.0350.1020.0620.245 0.0450.0830.1600.0860.3270.0570.146 1:8100460.0600.0520.1250.0210.0200.127 0.0510.2210.0190.0550.4830.0510.625 0.0180.1770.8400.0540.9140.0190.375Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 184

173 Appendix C (continued): Type I error rate estimates Table 19 (continued). Type I error rate estimates for condi tions when the population effect size variance is 0.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:8100550.0570.0460.1470.0290.0230.146 0.0460.2820.0310.0560.5720.0540.735 0.0330.1790.9100.0540.9640.0370.378 1:8100640.0570.0470.1860.0320.0210.185 0.0480.3680.0370.0570.7060.0740.840 0.0550.1710.9680.1060.9900.0820.369 1:81008120.0510.0490.0800.0260.0220.074 0.0540.1220.0280.0540.1850.0620.292 0.0280.1390.3850.0640.5600.0320.292 1:810010100.0540.0480.1170.0330.0210.087 0.0500.1650.0340.0540.2550.0460.420 0.0320.1480.5240.0510.7120.0330.313 1:81001280.0580.0460.1360.0330.0210.103 0.0450.2080.0300.0480.3440.0650.524 0.0460.1380.6700.0800.8320.0640.290 1:810040600.0490.0540.1190.0330.0230.054 0.0600.1400.0410.0560.0840.0980.222 0.0750.1530.1360.1530.3560.1400.287 1:810050500.0530.0500.1520.0360.0260.060 0.0440.1820.0350.0570.1150.0540.287 0.0410.1610.2000.0540.4340.0440.297 1:810060400.0520.0450.1830.0330.0260.072 0.0550.2060.0390.0590.1490.0780.323 0.0510.1580.2920.1220.4890.0790.301 Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 185

174 Appendix C (continued): Type I error rate estimates Table 20. Type I error rate estimates for conditions when the population effect size variance is 0.10. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:110460.0740.0670.0280.0050.0000.085 0.0670.0340.0050.0000.1180.0720.053 0.0060.0000.1700.0660.0830.0040.000 1:110550.0780.0640.0270.0050.0000.090 0.0730.0330.0060.0000.1160.0690.047 0.0050.0000.1720.0690.0810.0060.000 1:110640.0750.0620.0320.0050.0000.092 0.0720.0340.0060.0000.1160.0700.051 0.0050.0000.1710.0670.0830.0060.000 1:1108120.0650.0670.0170.0070.0000.064 0.0570.0140.0060.0000.0730.0590.021 0.0070.0000.0920.0620.0270.0070.000 1:11010100.0600.0590.0130.0070.0000.064 0.0630.0160.0070.0000.0760.0610.020 0.0060.0000.0890.0600.0280.0070.000 1:1101280.0650.0660.0150.0070.0000.064 0.0630.0160.0070.0000.0740.0630.021 0.0070.0000.0910.0660.0280.0070.000 1:11040600.0580.0590.0180.0110.0000.060 0.0640.0170.0120.0000.0650.0630.018 0.0100.0000.0670.0610.0180.0090.000 1:11050500.0600.0600.0180.0110.0000.059 0.0580.0180.0100.0000.0650.0610.017 0.0110.0000.0710.0590.0210.0100.000 1:11060400.0560.0610.0180.0100.0000.058 0.0590.0160.0100.0000.0630.0580.019 0.0100.0000.0750.0630.0200.0100.000 1:120460.0640.0510.0730.0220.0020.075 0.0520.0900.0200.0020.1530.0560.166 0.0200.0050.2770.0580.2950.0190.009 1:120550.0620.0490.0730.0190.0010.085 0.0560.0930.0230.0020.1510.0580.167 0.0210.0030.2660.0590.2880.0230.011 1:120640.0700.0570.0790.0220.0010.081 0.0530.0910.0200.0020.1530.0560.168 0.0190.0050.2730.0620.2960.0230.009 1:1208120.0580.0510.0420.0260.0020.058 0.0500.0450.0250.0020.0780.0500.067 0.0260.0040.1240.0530.1150.0250.006 1:12010100.0550.0500.0430.0250.0020.060 0.0490.0520.0240.0030.0780.0480.067 0.0250.0040.1220.0510.1080.0260.007 1:1201280.0530.0490.0440.0270.0020.057 0.0550.0480.0270.0020.0790.0520.071 0.0250.0030.1230.0510.1120.0270.006 1:12040600.0480.0510.0350.0290.0020.055 0.0510.0370.0300.0020.0630.0490.039 0.0310.0030.0800.0540.0450.0310.003 1:12050500.0520.0540.0390.0320.0020.053 0.0540.0360.0320.0020.0640.0510.039 0.0300.0020.0830.0520.0450.0320.002 1:12060400.0500.0520.0370.0330.0010.053 0.0480.0330.0280.0030.0640.0550.039 0.0330.0030.0790.0480.0440.0290.002 1:150460.0570.0460.1000.0300.0110.101 0.0530.1570.0320.0240.2940.0510.386 0.0360.0510.5970.0570.6840.0410.101 1:150550.0580.0480.0990.0320.0110.091 0.0500.1440.0330.0230.2910.0530.370 0.0350.0500.5760.0510.6650.0370.108 1:150640.0590.0480.1050.0320.0130.096 0.0490.1540.0330.0220.2980.0450.394 0.0320.0480.5950.0520.6810.0360.108 1:1508120.0510.0450.0590.0380.0120.065 0.0490.0750.0400.0170.1290.0490.155 0.0400.0340.2430.0460.2920.0390.050 1:15010100.0530.0490.0600.0400.0110.071 0.0460.0790.0380.0170.1320.0430.157 0.0380.0340.2530.0500.2970.0410.054 1:1501280.0520.0490.0600.0390.0140.065 0.0470.0790.0380.0180.1260.0460.150 0.0390.0320.2470.0500.2940.0410.054 1:15040600.0490.0480.0460.0410.0110.051 0.0490.0460.0420.0170.0890.0480.062 0.0440.0150.1360.0490.0770.0450.019 1:15050500.0470.0460.0440.0430.0120.055 0.0470.0480.0420.0150.0840.0480.058 0.0430.0160.1440.0520.0770.0470.020 1:15060400.0540.0510.0460.0450.0120.054 0.0430.0480.0400.0130.0860.0510.056 0.0440.0140.1380.0470.0720.0430.022 1:1100460.0610.0470.1080.0360.0200.141 0.0530.2230.0390.0440.5160.0510.631 0.0400.1150.8780.0590.9260.0480.219 1:1100550.0550.0460.1080.0370.0200.128 0.0480.2100.0390.0440.5060.0500.621 0.0380.1090.8580.0550.9070.0470.227 1:1100640.0590.0500.1170.0380.0170.137 0.0470.2220.0380.0390.5240.0510.632 0.0390.1070.8770.0570.9240.0480.219 1:11008120.0520.0480.0640.0410.0220.078 0.0480.0980.0420.0350.2160.0520.262 0.0470.0660.4600.0480.5330.0450.117 1:110010100.0540.0500.0660.0430.0220.077 0.0470.0930.0440.0340.2230.0510.262 0.0470.0670.4600.0500.5290.0460.110 1:11001280.0500.0430.0670.0380.0210.080 0.0460.0980.0440.0340.2130.0540.259 0.0510.0650.4530.0540.5270.0510.115 1:110040600.0500.0490.0520.0450.0190.063 0.0510.0550.0470.0220.1310.0510.072 0.0510.0290.2320.0500.1120.0480.036Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 186

175 Appendix C (continued): Type I error rate estimates Table 20 (continued). Type I error rate estimates for condi tions when the population effect size variance is 0.10. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:110050500.0520.0490.0520.0490.0210.062 0.0510.0540.0510.0240.1290.0480.078 0.0490.0330.2430.0520.1070.0530.037 1:110060400.0490.0520.0490.0470.0200.059 0.0490.0530.0480.0220.1240.0480.072 0.0480.0300.2340.0480.1110.0500.037 1:210460.0800.0700.0340.0060.0000.084 0.0650.0320.0040.0000.1190.0750.053 0.0040.0000.1740.0660.0830.0040.000 1:210550.0810.0710.0340.0050.0000.080 0.0630.0340.0050.0000.1170.0670.053 0.0050.0000.1750.0710.0860.0060.000 1:210640.0760.0640.0310.0060.0000.080 0.0680.0370.0060.0000.1250.0660.063 0.0050.0000.1840.0690.1030.0060.000 1:2108120.0620.0640.0160.0060.0000.066 0.0650.0180.0070.0000.0730.0620.022 0.0070.0000.0890.0630.0280.0050.000 1:21010100.0620.0600.0160.0080.0000.062 0.0620.0190.0070.0000.0800.0630.024 0.0070.0000.0950.0650.0370.0080.000 1:2101280.0610.0580.0210.0070.0000.067 0.0640.0220.0080.0000.0790.0590.027 0.0070.0000.1020.0660.0380.0080.000 1:21040600.0560.0640.0240.0120.0000.055 0.0610.0230.0100.0000.0650.0600.028 0.0120.0000.0700.0600.0260.0100.000 1:21050500.0590.0640.0230.0130.0000.061 0.0600.0250.0100.0000.0640.0600.027 0.0120.0000.0780.0580.0300.0110.000 1:21060400.0610.0580.0280.0100.0000.063 0.0580.0270.0110.0000.0660.0580.027 0.0110.0000.0800.0610.0350.0110.000 1:220460.0630.0510.0760.0160.0010.082 0.0530.0910.0200.0020.1480.0550.161 0.0160.0040.2720.0520.2890.0180.010 1:220550.0650.0540.0780.0200.0010.075 0.0540.0920.0200.0020.1510.0560.177 0.0230.0060.2840.0560.3140.0220.011 1:220640.0700.0530.0860.0230.0010.087 0.0570.1060.0230.0020.1670.0520.200 0.0240.0050.3170.0550.3600.0250.011 1:2208120.0520.0520.0440.0260.0020.060 0.0540.0520.0270.0030.0820.0540.070 0.0250.0040.1140.0540.1140.0270.007 1:22010100.0550.0520.0480.0280.0020.061 0.0510.0560.0270.0020.0840.0520.079 0.0290.0030.1230.0540.1190.0280.007 1:2201280.0610.0520.0550.0280.0020.061 0.0480.0570.0280.0020.0880.0520.088 0.0280.0030.1420.0550.1410.0300.007 1:22040600.0500.0530.0450.0340.0010.054 0.0530.0420.0330.0010.0580.0530.050 0.0320.0020.0790.0520.0630.0400.003 1:22050500.0510.0520.0460.0310.0010.053 0.0520.0440.0310.0020.0650.0520.052 0.0330.0020.0910.0500.0650.0300.003 1:22060400.0530.0530.0480.0320.0010.055 0.0470.0520.0320.0020.0730.0530.058 0.0310.0020.0910.0530.0670.0330.002 1:250460.0600.0500.1050.0270.0120.093 0.0510.1550.0300.0230.2820.0560.370 0.0340.0590.5690.0510.6570.0300.123 1:250550.0590.0500.1100.0330.0110.097 0.0490.1610.0310.0240.3020.0470.403 0.0320.0570.5940.0560.6900.0370.117 1:250640.0610.0490.1200.0330.0100.103 0.0490.1750.0350.0220.3370.0590.451 0.0390.0520.6600.0570.7540.0430.115 1:2508120.0520.0480.0630.0360.0150.060 0.0500.0750.0390.0180.1210.0440.155 0.0340.0390.2370.0460.3040.0370.067 1:25010100.0490.0460.0620.0370.0110.068 0.0500.0830.0410.0160.1300.0460.170 0.0380.0380.2730.0480.3230.0410.060 1:2501280.0490.0470.0660.0380.0100.065 0.0450.0860.0380.0170.1530.0490.190 0.0440.0350.2920.0530.3570.0420.062 1:25040600.0510.0510.0530.0440.0120.053 0.0500.0580.0460.0150.0830.0520.073 0.0470.0210.1410.0540.1010.0440.027 1:25050500.0520.0500.0600.0450.0110.055 0.0480.0650.0440.0150.0930.0500.082 0.0450.0200.1520.0530.1070.0490.028 1:25060400.0510.0480.0650.0400.0130.062 0.0490.0660.0440.0130.0930.0490.084 0.0440.0180.1620.0560.1080.0480.025 1:2100460.0540.0500.1090.0360.0220.135 0.0490.2130.0340.0430.4940.0490.619 0.0340.1160.8610.0480.9200.0330.234 1:2100550.0510.0470.1120.0370.0200.137 0.0480.2340.0360.0460.5240.0520.647 0.0400.1230.8820.0570.9330.0460.244 1:2100640.0610.0490.1300.0400.0210.146 0.0480.2520.0420.0420.5780.0540.707 0.0470.1160.9230.0710.9530.0610.235 1:21008120.0480.0430.0680.0390.0220.081 0.0500.1030.0430.0370.2040.0470.272 0.0420.0750.4280.0450.5330.0420.135 1:210010100.0470.0450.0660.0390.0210.077 0.0430.1040.0390.0360.2340.0470.294 0.0440.0760.4890.0510.5750.0470.135Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 187

176 Appendix C (continued): Type I error rate estimates Table 20 (continued). Type I error rate estimates for condi tions when the population effect size variance is 0.10. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:21001280.0560.0490.0760.0410.0210.086 0.0470.1150.0390.0340.2630.0550.328 0.0460.0730.5360.0630.6300.0590.134 1:210040600.0510.0490.0600.0450.0200.060 0.0480.0670.0480.0260.1210.0490.093 0.0490.0360.2260.0590.1410.0540.058 1:210050500.0510.0490.0610.0480.0200.065 0.0480.0720.0460.0300.1390.0470.099 0.0490.0390.2660.0490.1610.0460.056 1:210060400.0470.0440.0670.0420.0220.068 0.0490.0750.0480.0250.1520.0540.102 0.0520.0370.2840.0580.1570.0550.051 1:410460.0830.0690.0410.0040.0000.082 0.0650.0420.0050.0000.1220.0730.059 0.0050.0000.1760.0710.1040.0040.000 1:410550.0810.0680.0440.0060.0000.086 0.0680.0450.0050.0000.1270.0700.077 0.0060.0000.1850.0680.1190.0050.000 1:410640.0810.0640.0520.0070.0000.088 0.0620.0520.0060.0000.1390.0690.087 0.0080.0000.2090.0660.1440.0070.000 1:4108120.0670.0720.0240.0070.0000.063 0.0640.0270.0070.0000.0710.0640.029 0.0070.0000.0990.0670.0490.0050.000 1:41010100.0690.0640.0290.0070.0000.065 0.0590.0300.0080.0000.0780.0610.038 0.0080.0000.1050.0670.0540.0070.000 1:4101280.0700.0660.0330.0090.0000.072 0.0630.0380.0080.0000.0870.0620.047 0.0070.0000.1180.0650.0670.0100.000 1:41040600.0590.0620.0410.0140.0000.062 0.0670.0450.0140.0000.0640.0650.045 0.0130.0000.0710.0650.0570.0150.000 1:41050500.0600.0610.0460.0140.0000.064 0.0620.0480.0150.0000.0670.0600.054 0.0130.0000.0850.0650.0630.0150.000 1:41060400.0590.0570.0530.0100.0000.062 0.0590.0500.0120.0000.0730.0620.054 0.0150.0000.0870.0610.0700.0140.000 1:420460.0750.0590.0980.0190.0020.080 0.0560.1100.0180.0020.1500.0540.188 0.0180.0050.2740.0540.3160.0150.012 1:420550.0640.0490.0970.0180.0020.082 0.0540.1170.0210.0020.1630.0570.222 0.0240.0060.3000.0520.3740.0200.011 1:420640.0650.0530.1140.0230.0010.089 0.0580.1390.0280.0020.1860.0540.261 0.0280.0040.3660.0590.4500.0270.011 1:4208120.0570.0540.0560.0240.0010.058 0.0540.0610.0260.0020.0880.0560.092 0.0270.0040.1300.0540.1380.0260.009 1:42010100.0610.0540.0690.0260.0010.062 0.0500.0700.0250.0020.0970.0490.110 0.0260.0050.1510.0510.1750.0250.007 1:4201280.0590.0560.0760.0270.0010.066 0.0490.0840.0250.0020.1050.0540.141 0.0280.0030.1730.0530.2070.0280.006 1:42040600.0530.0540.0750.0390.0010.055 0.0540.0740.0380.0020.0660.0560.085 0.0390.0020.0800.0550.1050.0390.003 1:42050500.0510.0500.0810.0370.0020.058 0.0510.0820.0330.0020.0740.0540.093 0.0360.0020.1020.0550.1200.0350.002 1:42060400.0550.0510.0850.0350.0020.064 0.0560.0920.0370.0010.0740.0520.098 0.0340.0020.1060.0540.1240.0370.003 1:450460.0590.0490.1230.0270.0120.099 0.0510.1710.0280.0280.2910.0520.408 0.0270.0670.5810.0540.6910.0250.145 1:450550.0600.0480.1340.0310.0130.104 0.0470.1990.0310.0260.3250.0540.462 0.0350.0670.6530.0520.7690.0330.148 1:450640.0570.0480.1480.0320.0110.110 0.0480.2220.0340.0240.3960.0540.552 0.0410.0630.7340.0620.8420.0460.141 1:4508120.0530.0510.0780.0360.0120.065 0.0500.0980.0360.0230.1290.0510.189 0.0370.0450.2430.0550.3460.0400.084 1:45010100.0520.0510.0860.0380.0130.072 0.0480.1110.0350.0230.1550.0520.225 0.0370.0500.3130.0530.4160.0390.089 1:4501280.0550.0500.0970.0380.0110.077 0.0470.1260.0350.0210.1930.0500.272 0.0380.0480.3780.0560.4900.0450.088 1:45040600.0490.0490.0910.0490.0130.060 0.0590.0950.0500.0150.0850.0540.119 0.0520.0260.1500.0620.1760.0600.043 1:45050500.0510.0500.0950.0470.0090.066 0.0530.1100.0500.0150.1050.0480.138 0.0450.0270.1710.0470.1840.0440.044 1:45060400.0570.0510.1100.0480.0100.069 0.0520.1170.0490.0170.1090.0520.146 0.0470.0280.2060.0590.1990.0560.042 1:4100460.0560.0510.1330.0340.0180.127 0.0450.2400.0280.0500.5010.0480.639 0.0290.1250.8670.0500.9280.0310.289 1:4100550.0580.0480.1430.0340.0180.142 0.0470.2650.0380.0480.5560.0480.713 0.0380.1340.9070.0570.9530.0440.287 1:4100640.0590.0470.1640.0390.0200.169 0.0540.3170.0430.0470.6640.0620.789 0.0520.1290.9580.0780.9820.0660.273Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 188

177 Appendix C (continued): Type I error rate estimates Table 20 (continued). Type I error rate estimates for condi tions when the population effect size variance is 0.10. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:41008120.0540.0520.0930.0430.0210.075 0.0510.1270.0430.0410.2170.0460.317 0.0410.0980.4500.0530.5970.0420.191 1:410010100.0530.0480.0970.0380.0200.089 0.0490.1460.0400.0440.2720.0480.383 0.0410.1020.5540.0520.6880.0430.192 1:41001280.0550.0470.1040.0380.0220.102 0.0480.1720.0380.0380.3300.0570.445 0.0470.0950.6500.0720.7630.0610.185 1:410040600.0490.0490.0980.0510.0210.065 0.0560.1060.0570.0320.1300.0610.164 0.0650.0610.2560.0760.2680.0810.111 1:410050500.0490.0500.1100.0430.0190.070 0.0460.1190.0480.0300.1620.0540.177 0.0520.0660.3100.0560.2790.0540.114 1:410060400.0510.0520.1150.0480.0190.077 0.0530.1270.0520.0330.1940.0600.191 0.0580.0650.3720.0690.2940.0660.114 1:810460.0850.0700.0550.0060.0000.095 0.0740.0610.0050.0000.1260.0720.083 0.0040.0000.1790.0690.1260.0040.000 1:810550.0830.0630.0620.0050.0000.090 0.0630.0720.0070.0000.1330.0690.103 0.0070.0000.2020.0660.1630.0050.000 1:810640.0870.0720.0790.0070.0000.093 0.0610.0840.0070.0000.1510.0670.126 0.0080.0000.2460.0720.2100.0090.000 1:8108120.0640.0650.0370.0070.0000.065 0.0680.0380.0070.0000.0790.0660.048 0.0060.0000.1010.0690.0690.0070.000 1:81010100.0630.0630.0450.0080.0000.068 0.0620.0500.0080.0000.0860.0660.067 0.0080.0000.1130.0630.0920.0080.000 1:8101280.0680.0600.0560.0110.0000.080 0.0630.0630.0110.0000.0970.0690.080 0.0110.0000.1360.0660.1140.0100.000 1:81040600.0560.0600.0680.0170.0000.058 0.0600.0680.0150.0000.0650.0610.074 0.0170.0000.0790.0690.0940.0210.000 1:81050500.0640.0660.0810.0180.0000.063 0.0590.0810.0150.0000.0700.0600.086 0.0160.0000.0840.0630.1020.0170.000 1:81060400.0630.0560.0860.0140.0000.066 0.0600.0840.0150.0000.0770.0600.101 0.0150.0000.0970.0650.1200.0200.000 1:820460.0760.0580.1150.0180.0020.078 0.0500.1300.0150.0020.1570.0590.223 0.0190.0060.2870.0590.3670.0180.015 1:820550.0690.0560.1400.0220.0010.083 0.0540.1590.0210.0030.1930.0560.282 0.0240.0060.3370.0590.4570.0250.014 1:820640.0720.0520.1610.0240.0010.095 0.0540.1930.0260.0020.2160.0580.349 0.0290.0040.4170.0610.5550.0300.013 1:8208120.0570.0550.0790.0230.0020.064 0.0550.0840.0250.0020.0870.0560.120 0.0240.0040.1360.0580.1940.0260.008 1:82010100.0610.0530.0950.0250.0010.068 0.0520.1110.0270.0030.1040.0510.167 0.0250.0040.1660.0580.2400.0260.006 1:8201280.0590.0470.1100.0250.0010.075 0.0520.1300.0280.0020.1210.0530.200 0.0280.0040.2140.0570.3150.0300.008 1:82040600.0520.0540.1140.0390.0010.051 0.0550.1190.0420.0010.0680.0600.132 0.0440.0020.0930.0670.1680.0510.003 1:82050500.0570.0550.1290.0410.0010.055 0.0510.1340.0360.0010.0800.0560.148 0.0380.0020.1100.0550.1920.0400.003 1:82060400.0570.0520.1420.0380.0010.059 0.0510.1450.0330.0010.0840.0540.166 0.0360.0010.1290.0570.2120.0390.003 1:850460.0560.0500.1480.0250.0130.099 0.0550.2130.0250.0280.2980.0550.456 0.0280.0800.5970.0550.7410.0260.177 1:850550.0620.0500.1720.0310.0110.109 0.0520.2430.0330.0270.3580.0480.547 0.0290.0790.6810.0500.8250.0350.185 1:850640.0650.0540.2060.0390.0120.123 0.0510.2970.0340.0270.4390.0570.643 0.0430.0780.8060.0710.9010.0550.178 1:8508120.0550.0570.1060.0370.0130.066 0.0500.1260.0380.0260.1390.0530.247 0.0370.0540.2690.0570.4270.0370.120 1:85010100.0520.0500.1230.0360.0120.075 0.0520.1520.0390.0270.1790.0500.312 0.0370.0600.3540.0480.5270.0360.127 1:8501280.0570.0520.1500.0360.0130.083 0.0510.1930.0390.0230.2220.0560.382 0.0400.0600.4640.0620.6400.0450.121 1:85040600.0480.0500.1510.0490.0120.059 0.0550.1610.0610.0180.0980.0630.209 0.0670.0400.1550.0770.2760.0810.062 1:85050500.0530.0490.1700.0470.0110.065 0.0520.1810.0530.0180.1190.0510.231 0.0520.0410.2100.0530.3090.0520.076 1:85060400.0500.0510.1720.0480.0090.068 0.0510.1970.0460.0200.1410.0520.251 0.0520.0430.2680.0630.3370.0550.075 1:8100460.0550.0480.1620.0290.0220.134 0.0510.2710.0290.0530.5260.0520.697 0.0320.1540.8840.0540.9470.0300.341Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 189

178 Appendix C (continued): Type I error rate estimates Table 20 (continued). Type I error rate estimates for condi tions when the population effect size variance is 0.10. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:8100550.0600.0500.1870.0340.0200.156 0.0470.3340.0370.0500.6020.0570.776 0.0370.1610.9330.0510.9730.0390.350 1:8100640.0600.0510.2280.0410.0210.193 0.0510.3990.0390.0520.7350.0680.862 0.0560.1520.9780.0900.9910.0790.333 1:81008120.0520.0470.1100.0380.0220.083 0.0490.1660.0390.0510.2410.0540.403 0.0400.1330.4810.0610.6800.0450.259 1:810010100.0560.0490.1350.0390.0200.100 0.0470.2090.0400.0500.3120.0480.488 0.0390.1290.6230.0510.7890.0410.263 1:81001280.0580.0480.1630.0390.0190.118 0.0480.2540.0370.0450.4130.0620.592 0.0500.1280.7600.0730.8790.0630.260 1:810040600.0500.0520.1600.0550.0200.068 0.0540.1850.0590.0430.1440.0700.266 0.0800.0920.2790.0980.4010.1150.193 1:810050500.0520.0530.1770.0540.0200.076 0.0510.2050.0530.0420.1890.0520.304 0.0590.1010.3870.0520.4580.0580.206 1:810060400.0530.0480.1980.0520.0190.082 0.0490.2210.0500.0380.2360.0580.324 0.0570.1100.4700.0760.4830.0700.214 Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 190

179 Appendix C (continued): Type I error rate estimates Table 21. Type I error rate estimates for conditi ons when the population effect size variance is 0.33. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:110460.0820.0660.0470.0080.0000.089 0.0680.0530.0070.0000.1300.0660.074 0.0070.0000.1960.0680.1140.0100.000 1:110550.0800.0680.0480.0070.0000.087 0.0670.0520.0080.0000.1220.0670.073 0.0080.0000.1860.0650.1180.0070.000 1:110640.0850.0680.0480.0080.0000.086 0.0650.0520.0080.0000.1320.0670.075 0.0080.0000.1990.0660.1180.0070.000 1:1108120.0620.0660.0240.0090.0000.067 0.0660.0230.0090.0000.0790.0620.031 0.0080.0000.1160.0700.0460.0090.000 1:11010100.0620.0620.0230.0100.0000.069 0.0660.0260.0100.0000.0830.0620.034 0.0080.0000.1030.0610.0440.0100.000 1:1101280.0620.0610.0250.0090.0000.067 0.0660.0260.0100.0000.0810.0640.033 0.0100.0000.1130.0640.0490.0090.000 1:11040600.0640.0650.0260.0150.0000.057 0.0590.0260.0140.0000.0770.0640.025 0.0140.0000.0880.0610.0340.0140.000 1:11050500.0570.0600.0240.0160.0000.059 0.0610.0270.0130.0000.0750.0640.029 0.0140.0000.0930.0600.0320.0130.000 1:11060400.0610.0690.0230.0160.0000.059 0.0610.0270.0140.0000.0710.0620.023 0.0130.0000.0930.0610.0300.0140.000 1:120460.0640.0510.1130.0270.0010.086 0.0530.1300.0280.0020.1700.0500.236 0.0260.0040.3310.0520.3990.0270.007 1:120550.0670.0500.1080.0260.0020.083 0.0560.1250.0290.0020.1710.0550.229 0.0270.0040.3210.0570.3920.0270.008 1:120640.0660.0540.1070.0280.0010.079 0.0530.1250.0240.0020.1700.0570.234 0.0290.0040.3270.0540.4030.0270.009 1:1208120.0590.0530.0620.0310.0020.059 0.0500.0630.0280.0020.0990.0460.103 0.0280.0030.1670.0510.1690.0310.004 1:12010100.0570.0560.0590.0300.0010.064 0.0520.0670.0300.0010.1010.0530.100 0.0330.0020.1700.0510.1660.0290.003 1:1201280.0570.0540.0610.0320.0010.063 0.0540.0700.0320.0020.0980.0510.102 0.0310.0030.1690.0560.1700.0340.004 1:12040600.0540.0540.0440.0360.0000.056 0.0530.0440.0350.0010.0830.0510.052 0.0350.0010.1220.0510.0640.0360.002 1:12050500.0540.0550.0470.0380.0000.054 0.0500.0440.0350.0010.0880.0540.054 0.0380.0010.1270.0560.0600.0350.001 1:12060400.0460.0500.0420.0350.0010.059 0.0530.0440.0370.0010.0820.0540.050 0.0370.0010.1240.0550.0660.0360.001 1:150460.0580.0490.1470.0370.0100.108 0.0510.2140.0400.0210.3460.0470.494 0.0380.0430.6730.0540.7850.0440.082 1:150550.0580.0470.1400.0370.0110.108 0.0510.2100.0370.0210.3340.0530.473 0.0390.0410.6480.0520.7620.0410.085 1:150640.0600.0500.1490.0370.0120.104 0.0490.2150.0360.0200.3440.0510.490 0.0400.0430.6780.0530.7860.0430.080 1:1508120.0520.0500.0800.0420.0110.072 0.0490.1020.0400.0130.1820.0500.225 0.0410.0240.3660.0500.4230.0440.041 1:15010100.0550.0470.0770.0410.0120.075 0.0530.1050.0450.0150.1770.0500.224 0.0430.0230.3760.0500.4290.0440.042 1:1501280.0560.0470.0810.0410.0100.070 0.0490.1040.0400.0150.1760.0500.229 0.0410.0240.3720.0480.4280.0400.040 1:15040600.0500.0510.0520.0500.0090.067 0.0470.0560.0450.0090.1370.0500.074 0.0460.0090.2620.0520.1020.0500.014 1:15050500.0520.0510.0550.0490.0110.065 0.0510.0580.0500.0100.1460.0490.076 0.0440.0130.2820.0480.1060.0440.016 1:15060400.0500.0500.0540.0470.0090.064 0.0510.0490.0480.0110.1470.0540.075 0.0500.0100.2570.0530.1020.0460.012 1:1100460.0540.0440.1570.0410.0210.155 0.0470.2930.0410.0390.5950.0480.731 0.0430.0960.9260.0540.9590.0520.177 1:1100550.0560.0510.1530.0440.0200.149 0.0500.2850.0430.0400.5770.0520.720 0.0460.0970.9200.0520.9580.0470.176 1:1100640.0550.0500.1650.0420.0200.146 0.0490.2870.0440.0370.5890.0520.727 0.0440.0940.9290.0550.9620.0520.179 1:11008120.0510.0470.0870.0470.0160.094 0.0480.1430.0460.0280.3280.0510.400 0.0490.0570.6310.0500.7000.0490.086 1:110010100.0530.0530.0870.0470.0180.101 0.0450.1420.0420.0310.3290.0480.384 0.0490.0500.6530.0510.7030.0470.085 1:11001280.0500.0470.0880.0430.0170.098 0.0490.1450.0460.0300.3220.0500.396 0.0440.0510.6410.0510.7070.0500.088 1:110040600.0530.0540.0590.0550.0170.080 0.0470.0610.0470.0190.2410.0530.099 0.0540.0230.4720.0520.1520.0500.027Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 191

180 Appendix C (continued): Type I error rate estimates Table 21 (continued). Type I error rate estimates for condi tions when the population effect size variance is 0.33. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:110050500.0530.0500.0550.0540.0170.082 0.0480.0620.0510.0180.2410.0480.092 0.0490.0220.5010.0460.1520.0480.027 1:110060400.0500.0510.0580.0510.0150.082 0.0480.0630.0490.0160.2370.0520.095 0.0520.0220.4860.0550.1550.0550.027 1:210460.0790.0680.0460.0060.0000.092 0.0670.0530.0060.0000.1360.0700.075 0.0070.0000.1950.0640.1210.0070.000 1:210550.0780.0680.0520.0070.0000.091 0.0660.0570.0080.0000.1280.0700.085 0.0090.0000.1970.0690.1290.0070.000 1:210640.0840.0660.0520.0090.0000.088 0.0690.0600.0090.0000.1390.0650.090 0.0070.0000.2170.0660.1470.0090.000 1:2108120.0640.0650.0260.0090.0000.062 0.0630.0280.0100.0000.0820.0660.041 0.0100.0000.1130.0670.0510.0090.000 1:21010100.0640.0620.0290.0110.0000.067 0.0630.0310.0090.0000.0850.0620.041 0.0080.0000.1120.0620.0520.0100.000 1:2101280.0610.0570.0300.0090.0000.070 0.0630.0310.0100.0000.0850.0600.044 0.0100.0000.1210.0650.0570.0110.000 1:21040600.0580.0640.0290.0150.0000.062 0.0660.0310.0150.0000.0720.0640.031 0.0170.0000.0880.0620.0410.0150.000 1:21050500.0570.0630.0300.0140.0000.062 0.0640.0330.0150.0000.0750.0610.037 0.0140.0000.0970.0590.0440.0130.000 1:21060400.0580.0580.0300.0140.0000.066 0.0630.0340.0150.0000.0750.0680.038 0.0170.0000.1020.0610.0420.0130.000 1:220460.0710.0560.1160.0270.0000.082 0.0560.1380.0280.0020.1760.0560.236 0.0260.0040.3140.0560.3930.0270.009 1:220550.0650.0540.1130.0260.0010.084 0.0540.1370.0260.0020.1780.0550.247 0.0290.0040.3290.0510.4080.0260.010 1:220640.0680.0530.1220.0260.0010.086 0.0550.1490.0260.0020.1980.0590.272 0.0310.0040.3650.0570.4540.0310.006 1:2208120.0610.0520.0650.0310.0010.060 0.0520.0730.0310.0010.1060.0530.110 0.0330.0030.1720.0540.1830.0320.004 1:22010100.0570.0530.0650.0290.0010.063 0.0520.0750.0300.0010.1070.0540.116 0.0330.0030.1670.0530.1860.0310.003 1:2201280.0560.0490.0710.0290.0010.064 0.0520.0770.0280.0010.1100.0520.122 0.0320.0020.1860.0540.2030.0310.004 1:22040600.0500.0520.0530.0370.0000.056 0.0580.0530.0360.0000.0790.0530.063 0.0360.0010.1260.0550.0790.0360.001 1:22050500.0480.0470.0500.0330.0000.055 0.0530.0560.0350.0010.0910.0550.072 0.0380.0010.1310.0550.0860.0350.001 1:22060400.0550.0540.0550.0380.0000.058 0.0510.0600.0340.0010.0890.0560.062 0.0370.0010.1370.0570.0820.0400.001 1:250460.0590.0510.1540.0360.0120.101 0.0520.2130.0380.0220.3380.0480.487 0.0380.0510.6470.0520.7740.0370.097 1:250550.0630.0500.1560.0410.0110.111 0.0530.2250.0400.0200.3410.0500.497 0.0390.0460.6650.0530.7820.0430.097 1:250640.0610.0500.1620.0400.0090.113 0.0480.2420.0370.0190.3860.0550.549 0.0440.0490.7230.0550.8300.0450.087 1:2508120.0500.0480.0870.0440.0100.074 0.0480.1090.0400.0170.1710.0510.227 0.0410.0300.3520.0510.4260.0410.047 1:25010100.0550.0510.0880.0440.0090.072 0.0460.1170.0420.0160.1890.0490.247 0.0420.0280.3820.0520.4580.0440.047 1:2501280.0530.0500.0890.0430.0100.082 0.0460.1280.0390.0160.2060.0530.272 0.0450.0280.4120.0560.4870.0490.047 1:25040600.0500.0460.0640.0470.0100.064 0.0510.0700.0510.0110.1290.0500.094 0.0490.0120.2500.0540.1260.0520.016 1:25050500.0530.0520.0670.0510.0080.067 0.0520.0710.0520.0110.1420.0480.095 0.0460.0130.2840.0530.1310.0470.015 1:25060400.0510.0500.0660.0510.0100.065 0.0480.0730.0480.0100.1510.0480.093 0.0480.0140.2980.0530.1320.0540.016 1:2100460.0580.0490.1630.0420.0190.144 0.0470.2850.0420.0420.5690.0480.720 0.0390.0970.9180.0480.9590.0400.207 1:2100550.0600.0490.1690.0410.0190.155 0.0520.3000.0440.0390.5940.0500.738 0.0420.0970.9250.0560.9640.0490.192 1:2100640.0590.0500.1850.0430.0220.170 0.0490.3320.0410.0390.6520.0520.777 0.0490.0970.9500.0680.9760.0610.187 1:21008120.0550.0490.0940.0470.0190.093 0.0540.1440.0480.0290.3040.0500.386 0.0480.0610.6140.0470.7000.0460.103 1:210010100.0570.0470.0990.0480.0190.099 0.0480.1520.0440.0320.3480.0490.420 0.0430.0560.6720.0500.7370.0500.096Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 192

181 Appendix C (continued): Type I error rate estimates Table 21 (continued). Type I error rate estimates for condi tions when the population effect size variance is 0.33. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:21001280.0550.0460.1080.0440.0180.102 0.0440.1630.0430.0310.3730.0480.456 0.0460.0630.7090.0590.7710.0580.101 1:210040600.0490.0480.0730.0490.0180.084 0.0580.0860.0580.0190.2310.0470.122 0.0500.0280.4670.0530.1950.0510.037 1:210050500.0500.0500.0730.0540.0190.086 0.0510.0830.0520.0200.2560.0510.121 0.0550.0280.5340.0480.2010.0500.035 1:210060400.0530.0500.0720.0530.0160.085 0.0470.0850.0550.0190.2790.0520.129 0.0530.0270.5420.0570.2060.0560.036 1:410460.0840.0690.0600.0080.0000.094 0.0680.0680.0050.0000.1300.0700.094 0.0060.0000.2000.0670.1390.0060.000 1:410550.0820.0670.0610.0090.0000.089 0.0640.0680.0080.0000.1330.0630.105 0.0080.0000.2130.0670.1630.0100.000 1:410640.0790.0620.0700.0080.0000.094 0.0720.0820.0090.0000.1480.0630.120 0.0090.0000.2390.0680.1930.0100.000 1:4108120.0660.0660.0380.0100.0000.069 0.0670.0380.0100.0000.0890.0660.049 0.0100.0000.1140.0670.0650.0120.000 1:41010100.0680.0640.0370.0110.0000.071 0.0650.0430.0110.0000.0880.0630.053 0.0090.0000.1190.0610.0750.0090.000 1:4101280.0670.0620.0480.0100.0000.071 0.0630.0490.0110.0000.0940.0610.066 0.0100.0000.1390.0640.0920.0100.000 1:41040600.0650.0650.0460.0220.0000.061 0.0620.0470.0190.0000.0720.0650.053 0.0200.0000.0950.0720.0670.0220.000 1:41050500.0600.0630.0540.0190.0000.064 0.0610.0530.0190.0000.0830.0670.059 0.0190.0000.1040.0650.0740.0210.000 1:41060400.0610.0590.0570.0160.0000.067 0.0610.0560.0180.0000.0860.0670.062 0.0190.0000.1080.0640.0780.0170.000 1:420460.0680.0570.1310.0250.0010.082 0.0580.1510.0240.0020.1630.0550.253 0.0250.0050.3210.0540.4180.0240.009 1:420550.0700.0540.1360.0270.0010.083 0.0540.1630.0250.0010.1900.0600.291 0.0300.0050.3540.0540.4650.0280.009 1:420640.0640.0490.1560.0280.0000.086 0.0530.1860.0300.0010.2180.0600.337 0.0290.0040.4170.0600.5450.0330.009 1:4208120.0530.0540.0780.0290.0020.063 0.0530.0900.0310.0020.1020.0540.132 0.0300.0030.1700.0580.2100.0310.005 1:42010100.0570.0490.0870.0290.0010.064 0.0540.1010.0320.0020.1210.0550.160 0.0310.0030.1940.0520.2470.0310.004 1:4201280.0630.0530.0970.0320.0010.072 0.0510.1150.0300.0010.1250.0560.175 0.0330.0030.2250.0530.2800.0350.005 1:42040600.0540.0540.0830.0430.0010.057 0.0520.0860.0400.0000.0790.0550.097 0.0430.0020.1270.0540.1230.0430.001 1:42050500.0540.0540.0930.0410.0000.059 0.0510.0920.0380.0000.0920.0540.109 0.0410.0010.1470.0530.1320.0410.001 1:42060400.0540.0510.0920.0380.0000.057 0.0500.0910.0380.0010.0970.0500.116 0.0370.0010.1630.0570.1420.0400.001 1:450460.0590.0520.1700.0360.0120.098 0.0500.2340.0360.0220.3260.0490.499 0.0350.0570.6470.0520.7830.0360.115 1:450550.0600.0480.1810.0370.0100.107 0.0500.2570.0350.0230.3720.0500.557 0.0350.0530.6950.0500.8230.0400.115 1:450640.0610.0530.2020.0410.0110.118 0.0530.2890.0430.0220.4320.0520.617 0.0430.0530.7790.0600.8800.0510.103 1:4508120.0500.0480.1080.0370.0110.075 0.0470.1330.0380.0200.1760.0520.265 0.0410.0330.3630.0500.4800.0420.064 1:45010100.0520.0500.1200.0400.0100.080 0.0510.1520.0390.0190.2150.0470.310 0.0390.0350.4230.0520.5430.0420.066 1:4501280.0550.0510.1260.0390.0100.085 0.0500.1680.0440.0170.2450.0490.361 0.0440.0310.4950.0530.6110.0460.059 1:45040600.0530.0510.1060.0530.0090.066 0.0550.1100.0560.0110.1380.0520.145 0.0550.0180.2540.0550.1980.0600.022 1:45050500.0500.0500.1120.0530.0070.067 0.0520.1170.0500.0110.1650.0500.151 0.0530.0170.3130.0550.2190.0580.025 1:45060400.0510.0470.1120.0490.0090.073 0.0500.1190.0510.0110.1780.0480.159 0.0520.0170.3580.0540.2210.0530.024 1:4100460.0570.0480.1820.0380.0210.148 0.0480.3210.0370.0470.5790.0510.745 0.0400.1170.9180.0530.9620.0390.239 1:4100550.0570.0480.1940.0410.0200.160 0.0490.3360.0380.0460.6130.0510.775 0.0430.1110.9410.0500.9700.0460.223 1:4100640.0590.0480.2220.0430.0190.180 0.0500.3840.0460.0440.7000.0570.834 0.0520.1100.9740.0720.9860.0670.227Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 193

182 Appendix C (continued): Type I error rate estimates Table 21 (continued). Type I error rate estimates for condi tions when the population effect size variance is 0.33. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:41008120.0530.0500.1180.0460.0180.092 0.0480.1760.0460.0350.3150.0480.439 0.0450.0720.6400.0550.7590.0470.144 1:410010100.0550.0480.1310.0450.0180.108 0.0490.1940.0460.0340.3780.0480.507 0.0430.0740.7120.0510.8040.0470.139 1:41001280.0580.0490.1450.0450.0190.113 0.0500.2230.0480.0320.4370.0580.563 0.0530.0710.7960.0610.8680.0580.136 1:410040600.0500.0500.1120.0600.0170.082 0.0490.1270.0570.0210.2340.0560.192 0.0650.0400.4710.0630.2970.0730.065 1:410050500.0510.0510.1160.0560.0150.089 0.0500.1380.0580.0250.2830.0530.210 0.0560.0400.5700.0490.3230.0560.069 1:410060400.0520.0460.1240.0510.0160.098 0.0500.1410.0560.0250.3230.0540.210 0.0570.0410.6270.0600.3310.0650.065 1:810460.0860.0670.0750.0060.0000.096 0.0710.0840.0080.0000.1320.0710.115 0.0090.0000.2000.0660.1690.0070.000 1:810550.0900.0680.0880.0090.0000.093 0.0660.0950.0080.0000.1440.0700.136 0.0080.0000.2280.0700.2050.0080.000 1:810640.0860.0700.1060.0120.0000.102 0.0660.1100.0100.0000.1660.0660.175 0.0090.0000.2760.0680.2610.0100.000 1:8108120.0670.0670.0520.0100.0000.064 0.0660.0540.0110.0000.0840.0660.068 0.0110.0000.1210.0670.0960.0090.000 1:81010100.0660.0630.0620.0110.0000.075 0.0620.0670.0120.0000.0960.0630.087 0.0110.0000.1370.0660.1220.0110.000 1:8101280.0720.0610.0740.0130.0000.079 0.0640.0790.0100.0000.1050.0680.102 0.0130.0000.1550.0630.1470.0110.000 1:81040600.0590.0630.0740.0190.0000.063 0.0660.0780.0250.0000.0840.0720.089 0.0280.0000.0980.0720.1010.0300.000 1:81050500.0660.0640.0860.0240.0000.067 0.0630.0870.0220.0000.0830.0660.099 0.0240.0000.1130.0640.1190.0250.000 1:81060400.0670.0610.0880.0210.0000.064 0.0610.0880.0200.0000.0860.0640.094 0.0220.0000.1230.0610.1250.0210.000 1:820460.0670.0510.1630.0220.0010.082 0.0560.1830.0230.0020.1730.0550.288 0.0250.0050.3310.0570.4620.0230.011 1:820550.0720.0580.1810.0300.0010.092 0.0540.2050.0280.0020.1960.0560.335 0.0280.0050.3690.0520.5300.0260.012 1:820640.0690.0550.2040.0300.0010.103 0.0560.2520.0300.0020.2370.0570.402 0.0330.0040.4550.0600.6200.0360.011 1:8208120.0580.0530.1120.0310.0010.069 0.0590.1190.0330.0020.1040.0540.172 0.0320.0040.1790.0540.2680.0340.004 1:82010100.0660.0550.1240.0320.0010.073 0.0550.1370.0310.0010.1260.0570.204 0.0330.0030.2170.0520.3110.0300.005 1:8201280.0610.0530.1440.0310.0010.074 0.0530.1650.0330.0010.1430.0520.250 0.0320.0020.2510.0560.3750.0340.003 1:82040600.0500.0540.1290.0470.0010.058 0.0550.1320.0470.0010.0890.0610.149 0.0530.0020.1320.0610.1860.0570.002 1:82050500.0550.0510.1360.0430.0010.063 0.0520.1420.0480.0010.1000.0550.172 0.0490.0010.1570.0510.2020.0460.001 1:82060400.0580.0550.1450.0440.0000.063 0.0510.1480.0440.0010.1150.0540.177 0.0440.0010.1910.0560.2270.0470.001 1:850460.0610.0510.2100.0330.0120.112 0.0550.2700.0350.0250.3350.0570.543 0.0370.0650.6700.0570.8140.0350.144 1:850550.0640.0540.2340.0410.0130.116 0.0480.3060.0380.0250.3840.0520.607 0.0370.0680.7390.0530.8710.0370.137 1:850640.0610.0480.2610.0400.0100.133 0.0520.3550.0450.0250.4650.0530.683 0.0450.0660.8320.0620.9190.0540.136 1:8508120.0490.0510.1370.0400.0080.078 0.0500.1790.0410.0180.1820.0520.325 0.0400.0450.3700.0530.5460.0420.079 1:85010100.0560.0530.1690.0400.0120.084 0.0510.2040.0400.0190.2400.0510.409 0.0410.0440.4750.0460.6320.0400.086 1:8501280.0580.0460.1850.0420.0100.096 0.0550.2370.0460.0150.2770.0510.454 0.0440.0460.5630.0620.7150.0530.091 1:85040600.0520.0510.1690.0600.0080.066 0.0530.1860.0610.0130.1430.0610.223 0.0720.0210.2850.0630.3140.0760.034 1:85050500.0500.0480.1690.0550.0070.072 0.0510.1900.0580.0140.1850.0510.250 0.0610.0210.3430.0560.3350.0680.035 1:85060400.0510.0520.1880.0570.0070.081 0.0490.1980.0590.0130.2070.0510.255 0.0580.0210.4070.0530.3480.0610.038 1:8100460.0600.0510.2190.0390.0200.150 0.0530.3580.0380.0480.5780.0490.766 0.0390.1280.9240.0500.9660.0350.272Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 194

183 Appendix C (continued): Type I error rate estimates Table 21 (continued). Type I error rate estimates for condi tions when the population effect size variance is 0.33. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:8100550.0580.0490.2470.0350.0210.166 0.0470.3990.0350.0470.6440.0510.812 0.0380.1290.9540.0490.9770.0410.265 1:8100640.0600.0490.2770.0440.0230.202 0.0500.4590.0420.0480.7570.0580.873 0.0530.1230.9830.0810.9900.0770.266 1:81008120.0530.0470.1540.0430.0200.097 0.0510.2230.0470.0390.3200.0560.498 0.0450.0930.6540.0580.8050.0470.190 1:810010100.0550.0500.1860.0440.0210.117 0.0460.2680.0430.0420.4010.0460.587 0.0440.0920.7640.0500.8740.0440.191 1:81001280.0540.0490.2090.0430.0180.131 0.0490.3130.0460.0370.5000.0550.677 0.0510.0950.8570.0660.9280.0630.199 1:810040600.0510.0550.1860.0700.0170.081 0.0510.2030.0690.0300.2520.0620.311 0.0820.0520.5070.0740.4580.1030.102 1:810050500.0520.0520.2000.0640.0180.094 0.0510.2240.0620.0320.3070.0520.324 0.0660.0650.6150.0510.4770.0690.111 1:810060400.0530.0510.2080.0600.0170.114 0.0510.2310.0600.0300.3800.0500.338 0.0570.0600.7030.0620.5040.0690.116 Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 195

184 Appendix C (continued): Type I error rate estimates Table 22. Type I error rate estimates for conditions when the population effect size variance is 0.50. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:110460.0840.0670.0570.0080.0000.091 0.0670.0630.0080.0000.1330.0660.094 0.0090.0000.2050.0660.1430.0090.000 1:110550.0830.0700.0540.0100.0000.093 0.0660.0610.0090.0000.1290.0660.086 0.0090.0000.2120.0670.1470.0090.000 1:110640.0830.0670.0600.0110.0000.090 0.0670.0650.0080.0000.1410.0660.092 0.0100.0000.2120.0700.1420.0090.000 1:1108120.0700.0610.0310.0110.0000.065 0.0600.0320.0120.0000.0890.0700.041 0.0110.0000.1180.0640.0550.0090.000 1:11010100.0650.0630.0300.0130.0000.066 0.0600.0290.0090.0000.0860.0600.039 0.0120.0000.1200.0670.0550.0120.000 1:1101280.0610.0650.0320.0110.0000.067 0.0610.0320.0100.0000.0900.0600.045 0.0120.0000.1270.0600.0580.0110.000 1:11040600.0580.0640.0290.0180.0000.061 0.0610.0310.0170.0000.0700.0600.033 0.0140.0000.1060.0630.0390.0180.000 1:11050500.0560.0600.0280.0160.0000.062 0.0650.0290.0150.0000.0770.0650.033 0.0150.0000.1060.0630.0380.0170.000 1:11060400.0570.0620.0290.0170.0000.061 0.0640.0270.0150.0000.0770.0650.035 0.0160.0000.1000.0590.0380.0150.000 1:120460.0720.0560.1380.0270.0010.086 0.0560.1570.0300.0030.1840.0500.271 0.0260.0030.3530.0570.4450.0310.006 1:120550.0650.0550.1280.0320.0010.089 0.0550.1500.0290.0020.1770.0570.263 0.0320.0040.3450.0490.4410.0280.006 1:120640.0650.0520.1340.0290.0010.085 0.0520.1500.0290.0010.1920.0600.277 0.0310.0040.3510.0520.4460.0310.006 1:1208120.0570.0510.0730.0290.0010.066 0.0530.0820.0320.0010.1120.0490.125 0.0310.0020.1910.0540.2030.0300.003 1:12010100.0590.0510.0670.0310.0010.066 0.0520.0830.0320.0010.1100.0540.123 0.0310.0010.2000.0540.2110.0330.003 1:1201280.0570.0550.0740.0330.0010.067 0.0540.0820.0340.0020.1090.0550.128 0.0360.0020.1930.0560.2070.0330.004 1:12040600.0480.0510.0480.0340.0000.059 0.0560.0540.0390.0010.0950.0540.062 0.0390.0010.1510.0520.0760.0350.000 1:12050500.0520.0500.0450.0360.0010.057 0.0540.0490.0370.0010.1000.0560.067 0.0390.0010.1580.0530.0770.0370.000 1:12060400.0530.0530.0490.0360.0010.061 0.0560.0540.0390.0010.0950.0500.064 0.0390.0000.1530.0550.0800.0400.000 1:150460.0590.0500.1770.0400.0130.110 0.0470.2480.0380.0220.3710.0510.531 0.0400.0400.7020.0520.8160.0430.072 1:150550.0620.0520.1730.0420.0120.109 0.0490.2500.0400.0190.3470.0490.512 0.0400.0400.6850.0530.7980.0460.071 1:150640.0620.0440.1800.0390.0120.113 0.0520.2420.0440.0180.3600.0490.531 0.0400.0400.7070.0510.8190.0460.074 1:1508120.0520.0500.0990.0470.0100.077 0.0520.1240.0420.0130.2120.0520.274 0.0450.0220.4190.0540.4890.0490.033 1:15010100.0520.0500.0990.0430.0080.081 0.0480.1250.0430.0140.2130.0510.271 0.0460.0220.4440.0480.5020.0410.035 1:1501280.0520.0530.0990.0440.0100.080 0.0490.1240.0410.0140.2060.0480.270 0.0420.0230.4290.0520.4990.0480.037 1:15040600.0460.0450.0560.0470.0070.069 0.0510.0630.0520.0080.1710.0540.086 0.0520.0110.3330.0510.1230.0520.011 1:15050500.0540.0510.0630.0520.0090.071 0.0510.0630.0540.0090.1770.0490.088 0.0480.0090.3500.0480.1200.0500.013 1:15060400.0530.0520.0580.0520.0090.072 0.0500.0670.0520.0100.1720.0550.085 0.0510.0100.3270.0520.1180.0490.013 1:1100460.0560.0460.1940.0440.0180.161 0.0490.3300.0460.0360.6170.0490.762 0.0440.0860.9410.0520.9660.0500.163 1:1100550.0560.0480.1820.0450.0180.154 0.0480.3200.0430.0370.6090.0480.752 0.0470.0790.9370.0560.9670.0540.159 1:1100640.0560.0490.1920.0440.0220.158 0.0520.3250.0470.0340.6260.0510.761 0.0480.0830.9420.0550.9680.0520.149 1:11008120.0520.0490.1070.0500.0190.104 0.0490.1700.0450.0270.3730.0480.453 0.0480.0470.7250.0520.7790.0510.074 1:110010100.0510.0460.1060.0450.0200.109 0.0470.1730.0470.0260.3820.0460.452 0.0460.0480.7340.0500.7790.0500.082 1:11001280.0540.0510.1070.0460.0200.106 0.0500.1760.0480.0280.3820.0530.458 0.0520.0450.7300.0520.7850.0510.079 1:110040600.0490.0480.0590.0540.0190.089 0.0510.0680.0540.0170.2940.0500.116 0.0560.0210.5960.0480.1860.0510.026Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 196

185 Appendix C (continued): Type I error rate estimates Table 22 (continued). Type I error rate estimates for condi tions when the population effect size variance is 0.50. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:110050500.0500.0510.0640.0540.0180.093 0.0510.0680.0550.0170.3190.0530.118 0.0580.0240.6230.0510.1850.0540.027 1:110060400.0480.0500.0630.0510.0180.091 0.0510.0680.0580.0180.3030.0480.108 0.0540.0200.6020.0480.1930.0530.026 1:210460.0860.0710.0590.0100.0000.089 0.0640.0690.0090.0000.1350.0660.094 0.0080.0000.2090.0700.1420.0100.000 1:210550.0840.0680.0610.0090.0000.096 0.0680.0690.0100.0000.1370.0690.099 0.0100.0000.2140.0680.1530.0080.000 1:210640.0820.0650.0660.0090.0000.094 0.0680.0740.0100.0000.1470.0690.106 0.0090.0000.2320.0650.1730.0120.000 1:2108120.0610.0600.0280.0100.0000.066 0.0630.0320.0110.0000.0870.0640.043 0.0110.0000.1210.0660.0640.0100.000 1:21010100.0650.0660.0360.0130.0000.066 0.0620.0350.0130.0000.0900.0610.051 0.0110.0000.1300.0650.0690.0100.000 1:2101280.0650.0600.0360.0120.0000.069 0.0650.0390.0120.0000.0920.0620.052 0.0090.0000.1400.0630.0710.0110.000 1:21040600.0600.0620.0340.0170.0000.061 0.0640.0350.0160.0000.0780.0630.042 0.0200.0000.1000.0590.0450.0170.000 1:21050500.0560.0640.0330.0150.0000.062 0.0660.0370.0160.0000.0800.0620.046 0.0180.0000.1040.0600.0490.0160.000 1:21060400.0630.0660.0350.0180.0000.060 0.0590.0390.0170.0000.0800.0610.045 0.0170.0000.1070.0620.0500.0160.000 1:220460.0660.0510.1370.0260.0010.086 0.0560.1600.0320.0020.1820.0540.268 0.0280.0040.3340.0530.4360.0280.006 1:220550.0670.0530.1340.0290.0010.085 0.0570.1640.0290.0010.1890.0560.289 0.0290.0040.3560.0530.4610.0300.006 1:220640.0660.0510.1490.0300.0010.093 0.0560.1770.0300.0020.1990.0540.305 0.0290.0040.3860.0610.4920.0360.007 1:2208120.0570.0540.0760.0320.0010.067 0.0530.0910.0340.0010.1140.0540.135 0.0330.0020.1860.0550.2130.0350.004 1:22010100.0580.0550.0810.0310.0010.069 0.0520.0850.0330.0010.1140.0470.133 0.0300.0020.2050.0500.2270.0300.003 1:2201280.0570.0510.0850.0310.0010.068 0.0500.0920.0290.0010.1240.0550.151 0.0340.0020.2110.0490.2500.0310.004 1:22040600.0480.0530.0590.0410.0000.062 0.0560.0550.0420.0000.0880.0540.072 0.0420.0010.1440.0580.0910.0420.001 1:22050500.0490.0520.0580.0400.0010.060 0.0550.0660.0400.0010.0970.0560.074 0.0420.0010.1630.0510.0970.0400.001 1:22060400.0550.0560.0590.0390.0010.058 0.0530.0640.0400.0000.1000.0530.073 0.0380.0010.1620.0530.1020.0400.001 1:250460.0620.0500.1780.0400.0110.110 0.0500.2500.0400.0190.3480.0470.521 0.0370.0420.6810.0500.8010.0380.079 1:250550.0560.0520.1800.0410.0110.111 0.0500.2560.0400.0220.3680.0520.539 0.0400.0430.6960.0510.8130.0430.077 1:250640.0610.0490.1990.0390.0100.116 0.0480.2710.0380.0200.4010.0480.576 0.0420.0440.7460.0560.8480.0480.077 1:2508120.0540.0510.0990.0430.0100.077 0.0490.1370.0430.0130.2000.0470.282 0.0410.0240.4080.0500.4900.0440.042 1:25010100.0580.0480.1110.0460.0090.088 0.0520.1430.0440.0170.2190.0490.293 0.0450.0250.4520.0510.5260.0450.042 1:2501280.0560.0500.1180.0430.0090.084 0.0500.1430.0440.0140.2390.0550.323 0.0490.0250.4690.0500.5630.0440.039 1:25040600.0520.0500.0710.0520.0080.070 0.0510.0780.0520.0090.1550.0520.099 0.0500.0130.3150.0500.1430.0520.014 1:25050500.0490.0490.0750.0530.0090.074 0.0510.0720.0510.0100.1810.0480.105 0.0490.0120.3530.0530.1550.0550.014 1:25060400.0500.0490.0740.0470.0090.071 0.0500.0760.0500.0080.1920.0500.103 0.0510.0120.3690.0530.1530.0530.013 1:2100460.0550.0470.1880.0410.0210.157 0.0500.3230.0430.0380.5960.0520.745 0.0440.0860.9340.0510.9640.0440.178 1:2100550.0570.0510.2000.0470.0220.157 0.0500.3370.0440.0410.6220.0520.763 0.0480.0850.9410.0540.9720.0510.166 1:2100640.0610.0500.2180.0460.0190.170 0.0490.3660.0470.0360.6650.0520.794 0.0490.0810.9630.0610.9770.0560.168 1:21008120.0560.0510.1160.0460.0200.106 0.0480.1760.0490.0240.3520.0490.452 0.0490.0530.6980.0510.7640.0470.092 1:210010100.0580.0480.1210.0470.0190.114 0.0500.1920.0500.0280.3960.0510.476 0.0480.0530.7460.0540.8020.0540.090Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 197

186 Appendix C (continued): Type I error rate estimates Table 22 (continued). Type I error rate estimates for condi tions when the population effect size variance is 0.50. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:21001280.0530.0480.1150.0480.0190.115 0.0510.1980.0490.0280.4210.0550.518 0.0520.0480.7770.0540.8290.0520.090 1:210040600.0540.0560.0840.0600.0160.089 0.0470.0900.0570.0200.2940.0540.135 0.0600.0240.5860.0590.2320.0620.031 1:210050500.0480.0490.0780.0540.0180.096 0.0510.0940.0580.0200.3260.0470.133 0.0560.0260.6360.0520.2370.0580.032 1:210060400.0490.0490.0790.0520.0190.092 0.0470.0820.0540.0180.3390.0510.141 0.0540.0260.6570.0490.2370.0550.028 1:410460.0810.0690.0700.0090.0000.091 0.0730.0800.0090.0000.1340.0690.111 0.0080.0000.2140.0680.1730.0080.000 1:410550.0860.0680.0790.0090.0000.095 0.0680.0880.0090.0000.1390.0670.122 0.0100.0000.2290.0690.1920.0110.000 1:410640.0850.0680.0820.0120.0000.102 0.0650.0960.0110.0000.1520.0660.144 0.0110.0000.2530.0650.2240.0110.000 1:4108120.0660.0660.0440.0110.0000.067 0.0680.0450.0120.0000.0930.0650.061 0.0120.0000.1230.0670.0840.0120.000 1:41010100.0660.0670.0540.0120.0000.068 0.0620.0520.0130.0000.1000.0660.072 0.0120.0000.1370.0620.0970.0120.000 1:4101280.0710.0600.0550.0120.0000.076 0.0650.0560.0120.0000.0980.0620.073 0.0130.0000.1480.0670.1050.0120.000 1:41040600.0620.0700.0530.0230.0000.062 0.0660.0540.0220.0000.0770.0640.058 0.0220.0000.1010.0620.0720.0220.000 1:41050500.0610.0640.0580.0200.0000.066 0.0630.0620.0220.0000.0860.0650.067 0.0210.0000.1160.0640.0800.0240.000 1:41060400.0570.0580.0610.0200.0000.064 0.0570.0610.0190.0000.0870.0620.065 0.0190.0000.1250.0610.0840.0190.000 1:420460.0690.0520.1550.0270.0010.086 0.0550.1840.0280.0020.1820.0560.295 0.0290.0040.3260.0530.4600.0240.009 1:420550.0720.0550.1690.0300.0010.089 0.0560.1910.0290.0020.1920.0560.320 0.0310.0040.3660.0540.5000.0280.009 1:420640.0740.0540.1850.0330.0010.093 0.0500.2100.0300.0020.2240.0560.367 0.0330.0040.4310.0600.5770.0350.007 1:4208120.0570.0570.0960.0340.0010.068 0.0550.1060.0330.0010.1120.0520.157 0.0300.0020.1920.0580.2520.0340.003 1:42010100.0620.0580.1080.0360.0010.072 0.0530.1150.0330.0010.1260.0530.178 0.0340.0010.2180.0540.2860.0340.003 1:4201280.0650.0540.1170.0310.0010.071 0.0510.1290.0310.0010.1390.0530.207 0.0350.0030.2440.0550.3270.0370.002 1:42040600.0550.0570.0900.0490.0010.063 0.0550.0910.0430.0010.0920.0570.108 0.0470.0010.1490.0570.1330.0490.001 1:42050500.0530.0520.0910.0430.0010.064 0.0520.0970.0460.0000.1010.0520.114 0.0420.0010.1740.0570.1470.0450.001 1:42060400.0540.0500.0940.0400.0000.062 0.0500.0980.0390.0000.1140.0490.122 0.0420.0010.1950.0550.1570.0400.001 1:450460.0660.0550.2050.0410.0130.105 0.0510.2650.0370.0200.3530.0490.546 0.0350.0520.6810.0480.8180.0380.096 1:450550.0650.0500.2190.0400.0120.112 0.0490.2860.0380.0200.3850.0530.584 0.0400.0500.7340.0490.8480.0410.103 1:450640.0620.0490.2400.0420.0120.127 0.0510.3220.0420.0210.4450.0550.645 0.0470.0480.7940.0580.8930.0550.092 1:4508120.0530.0490.1310.0450.0090.078 0.0490.1650.0440.0140.2030.0500.319 0.0470.0280.4100.0500.5410.0430.050 1:45010100.0550.0460.1420.0420.0110.087 0.0480.1800.0420.0160.2390.0490.363 0.0440.0290.4790.0470.6030.0450.052 1:4501280.0580.0510.1540.0480.0090.091 0.0470.2010.0410.0150.2710.0490.407 0.0450.0290.5550.0580.6700.0510.049 1:45040600.0530.0530.1140.0600.0100.072 0.0560.1280.0610.0090.1670.0520.164 0.0590.0140.3270.0550.2280.0660.019 1:45050500.0500.0510.1240.0570.0100.073 0.0520.1240.0540.0090.1950.0490.173 0.0550.0140.3950.0510.2330.0560.020 1:45060400.0520.0500.1190.0580.0070.077 0.0480.1300.0570.0100.2100.0500.169 0.0560.0140.4270.0520.2440.0560.020 1:4100460.0570.0490.2150.0420.0210.159 0.0500.3480.0420.0410.6030.0490.767 0.0400.0970.9350.0480.9670.0380.199 1:4100550.0580.0520.2290.0440.0210.162 0.0490.3740.0450.0420.6380.0470.800 0.0420.0940.9520.0520.9730.0480.203 1:4100640.0590.0500.2480.0440.0200.187 0.0510.4180.0480.0400.7080.0530.836 0.0550.0910.9780.0710.9850.0710.193Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 198

187 Appendix C (continued): Type I error rate estimates Table 22 (continued). Type I error rate estimates for condi tions when the population effect size variance is 0.50. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:41008120.0550.0470.1430.0470.0170.107 0.0520.2120.0490.0340.3610.0510.501 0.0470.0670.7000.0520.7930.0480.123 1:410010100.0520.0440.1570.0440.0180.116 0.0450.2330.0470.0330.4310.0490.554 0.0500.0690.7890.0490.8590.0450.124 1:41001280.0550.0470.1630.0470.0170.125 0.0490.2550.0480.0320.4770.0530.611 0.0540.0610.8430.0580.8970.0640.112 1:410040600.0470.0450.1290.0580.0160.092 0.0500.1410.0630.0220.2910.0510.214 0.0680.0350.5800.0580.3370.0710.049 1:410050500.0530.0510.1320.0620.0140.098 0.0460.1390.0580.0200.3440.0510.219 0.0600.0330.6730.0510.3520.0620.055 1:410060400.0540.0490.1290.0580.0150.110 0.0460.1430.0560.0240.3980.0510.233 0.0620.0360.7230.0540.3620.0620.057 1:810460.0850.0710.0940.0080.0000.097 0.0670.0990.0080.0000.1360.0660.128 0.0080.0000.2170.0680.1950.0080.000 1:810550.0890.0710.1050.0100.0000.099 0.0680.1170.0090.0000.1430.0670.155 0.0090.0000.2370.0690.2330.0100.000 1:810640.0890.0610.1200.0110.0000.112 0.0690.1410.0130.0000.1740.0690.196 0.0130.0000.2760.0650.2850.0120.000 1:8108120.0620.0630.0580.0130.0000.071 0.0650.0660.0130.0000.0880.0690.079 0.0120.0000.1250.0650.1050.0110.000 1:81010100.0670.0630.0700.0120.0000.075 0.0650.0810.0120.0000.1020.0680.103 0.0130.0000.1470.0670.1400.0150.000 1:8101280.0690.0610.0800.0130.0000.077 0.0600.0890.0120.0000.1130.0670.118 0.0130.0000.1700.0670.1610.0140.000 1:81040600.0620.0690.0780.0270.0000.062 0.0680.0800.0260.0000.0790.0680.092 0.0300.0000.1070.0670.1100.0300.000 1:81050500.0640.0620.0890.0260.0000.064 0.0640.0820.0240.0000.0830.0650.097 0.0250.0000.1220.0670.1220.0260.000 1:81060400.0670.0620.0870.0240.0000.066 0.0630.0950.0260.0000.0900.0630.104 0.0240.0000.1300.0610.1350.0250.000 1:820460.0700.0550.1890.0250.0010.081 0.0550.2060.0270.0010.1830.0580.323 0.0290.0040.3490.0600.5020.0280.008 1:820550.0740.0560.2040.0290.0010.096 0.0560.2340.0310.0010.2020.0540.374 0.0270.0050.3910.0540.5670.0300.007 1:820640.0740.0530.2400.0300.0010.096 0.0550.2720.0310.0020.2310.0550.431 0.0310.0040.4510.0640.6370.0400.008 1:8208120.0570.0540.1280.0330.0010.071 0.0580.1350.0360.0010.1120.0510.195 0.0330.0030.2000.0550.2970.0330.003 1:82010100.0620.0520.1480.0320.0000.070 0.0550.1580.0350.0010.1320.0560.241 0.0360.0020.2380.0540.3490.0320.003 1:8201280.0660.0540.1650.0340.0000.077 0.0520.1840.0360.0010.1530.0510.276 0.0350.0010.2780.0550.4140.0340.004 1:82040600.0550.0560.1350.0530.0010.063 0.0530.1390.0520.0010.0970.0540.161 0.0540.0010.1590.0600.1980.0570.001 1:82050500.0540.0570.1470.0500.0000.064 0.0540.1460.0510.0010.1110.0550.174 0.0490.0010.1840.0530.2180.0490.000 1:82060400.0590.0540.1490.0500.0000.063 0.0520.1560.0470.0000.1220.0530.184 0.0480.0010.2130.0580.2210.0480.001 1:850460.0590.0500.2300.0350.0120.108 0.0510.3050.0370.0250.3530.0520.570 0.0390.0570.6840.0500.8360.0350.117 1:850550.0640.0520.2650.0390.0100.115 0.0510.3290.0380.0240.3930.0500.635 0.0430.0560.7490.0540.8750.0430.120 1:850640.0620.0490.2930.0430.0100.133 0.0470.3780.0430.0240.4740.0520.701 0.0470.0590.8290.0630.9150.0570.116 1:8508120.0570.0520.1670.0450.0100.075 0.0550.2050.0470.0170.2080.0520.376 0.0440.0360.4270.0590.5950.0490.065 1:85010100.0560.0510.1950.0490.0110.083 0.0520.2370.0490.0160.2530.0480.438 0.0460.0320.5170.0540.6780.0460.069 1:8501280.0580.0490.2180.0440.0110.102 0.0500.2740.0430.0170.3030.0510.502 0.0500.0370.6060.0570.7480.0580.071 1:85040600.0550.0530.1780.0680.0070.068 0.0500.1890.0640.0120.1760.0560.244 0.0740.0180.3380.0650.3280.0870.028 1:85050500.0550.0550.1900.0680.0070.077 0.0510.2020.0640.0110.2050.0500.260 0.0620.0200.4120.0510.3390.0660.028 1:85060400.0510.0500.1860.0610.0080.081 0.0510.2040.0610.0100.2370.0500.265 0.0630.0190.4810.0540.3740.0660.031 1:8100460.0620.0500.2600.0400.0220.161 0.0520.3950.0370.0480.6000.0500.776 0.0370.1140.9400.0520.9710.0410.236Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 199

188 Appendix C (continued): Type I error rate estimates Table 22 (continued). Type I error rate estimates for condi tions when the population effect size variance is 0.50. TAUs2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 0.51:8100550.0580.0520.2750.0420.0200. 1660.0510.4190.0440.0440.6710.0490. 8280.0420.1170.9640.0510.9840.0450.238 0.51:8100640.0580.0480.3080.0480.0190. 2010.0510.4780.0470.0440.7500.0590. 8660.0590.1080.9870.0730.9880.0730.235 0.51:81008120.0540.0510.1870.0480.0180. 1020.0470.2570.0490.0350.3760.0540. 5560.0490.0830.7180.0570.8420.0500.156 0.51:810010100.0550.0510.2140.0480.0200. 1220.0490.3020.0480.0330.4560.0490. 6350.0490.0830.8230.0460.8990.0450.172 0.51:81001280.0560.0500.2330.0510.0190. 1350.0480.3410.0490.0340.5500.0540. 7050.0550.0800.8950.0620.9400.0650.160 0.51:810040600.0540.0530.1960.0710.0160. 0940.0530.2190.0730.0290.2990.0570. 3200.0780.0450.5960.0690.4640.0980.080 0.51:810050500.0510.0510.2140.0680.0160. 1080.0540.2260.0700.0230.3770.0520. 3380.0680.0510.7160.0480.5080.0740.091 0.51:810060400.0530.0500.2160.0640.0170. 1140.0490.2320.0660.0270.4440.0520. 3570.0690.0490.7840.0590.5260.0690.094 Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 200

189 Appendix C (continued): Type I error rate estimates Table 23. Type I error rate estimates for conditions when the population effect size variance is 1.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:110460.0890.0690.0900.0120.0000.096 0.0660.0930.0110.0000.1510.0670.139 0.0130.0000.2350.0660.1950.0110.000 1:110550.0890.0670.0870.0120.0000.095 0.0700.0980.0130.0000.1470.0730.136 0.0120.0000.2320.0720.2020.0130.000 1:110640.0860.0620.0850.0120.0000.096 0.0640.0910.0100.0000.1450.0650.138 0.0110.0000.2380.0660.2010.0110.000 1:1108120.0670.0660.0490.0140.0000.075 0.0680.0510.0150.0000.0980.0630.068 0.0150.0000.1470.0650.0920.0160.000 1:11010100.0650.0660.0490.0140.0000.074 0.0650.0510.0140.0000.0940.0630.067 0.0130.0000.1430.0630.0930.0140.000 1:1101280.0720.0650.0500.0150.0000.072 0.0680.0510.0130.0000.1050.0710.063 0.0150.0000.1480.0650.0940.0130.000 1:11040600.0570.0660.0420.0250.0000.062 0.0600.0420.0210.0000.0880.0630.048 0.0210.0000.1180.0610.0600.0210.000 1:11050500.0590.0630.0420.0200.0000.064 0.0620.0430.0220.0000.0830.0630.050 0.0230.0000.1220.0640.0590.0200.000 1:11060400.0560.0610.0420.0230.0000.059 0.0610.0400.0200.0000.0890.0620.052 0.0210.0000.1270.0680.0620.0240.000 1:120460.0710.0550.1850.0310.0010.094 0.0570.2210.0370.0010.1980.0570.341 0.0340.0030.3840.0550.5250.0330.004 1:120550.0710.0560.1910.0330.0010.092 0.0570.2140.0350.0020.1910.0570.328 0.0360.0020.3700.0550.5110.0340.006 1:120640.0730.0580.1930.0340.0010.091 0.0590.2210.0370.0010.1980.0540.344 0.0340.0020.3780.0590.5190.0340.005 1:1208120.0590.0510.1100.0370.0010.069 0.0520.1210.0370.0010.1310.0510.186 0.0340.0010.2420.0550.2960.0390.002 1:12010100.0540.0510.1030.0370.0010.067 0.0520.1200.0360.0010.1310.0530.184 0.0370.0010.2470.0530.2930.0330.002 1:1201280.0600.0540.1070.0380.0010.075 0.0520.1280.0390.0010.1340.0500.183 0.0350.0020.2370.0530.2940.0360.002 1:12040600.0530.0530.0650.0480.0000.059 0.0510.0690.0430.0000.1140.0500.080 0.0430.0010.1980.0590.1060.0460.001 1:12050500.0520.0510.0670.0460.0000.060 0.0530.0680.0490.0010.1110.0550.083 0.0470.0010.2080.0520.1170.0460.000 1:12060400.0530.0490.0630.0440.0010.065 0.0540.0690.0470.0000.1170.0520.090 0.0410.0000.1930.0500.1130.0440.001 1:150460.0600.0480.2450.0440.0110.113 0.0520.3100.0460.0160.3810.0500.583 0.0470.0320.7340.0550.8400.0510.057 1:150550.0610.0500.2380.0480.0140.109 0.0490.3050.0440.0180.3690.0510.573 0.0450.0310.7220.0520.8380.0500.058 1:150640.0600.0500.2380.0460.0120.111 0.0520.3110.0470.0180.3860.0500.587 0.0470.0320.7340.0500.8410.0460.056 1:1508120.0590.0510.1530.0520.0080.090 0.0510.1900.0480.0130.2650.0520.377 0.0530.0200.5280.0480.6180.0500.027 1:15010100.0570.0510.1530.0510.0090.090 0.0500.1900.0520.0120.2710.0500.377 0.0480.0170.5380.0500.6190.0520.026 1:1501280.0550.0500.1470.0510.0090.091 0.0490.1900.0510.0140.2590.0490.373 0.0470.0210.5340.0500.6240.0490.026 1:15040600.0540.0540.0760.0590.0080.079 0.0530.0850.0620.0070.2250.0490.112 0.0560.0080.4610.0460.1750.0530.011 1:15050500.0490.0500.0740.0600.0090.079 0.0520.0800.0580.0080.2390.0490.119 0.0590.0090.4640.0510.1720.0580.011 1:15060400.0490.0500.0800.0570.0080.082 0.0520.0850.0600.0110.2200.0510.118 0.0560.0110.4530.0500.1710.0570.011 1:1100460.0570.0490.2670.0480.0210.161 0.0480.3880.0490.0330.6430.0530.782 0.0520.0670.9560.0550.9700.0550.125 1:1100550.0600.0520.2530.0510.0190.161 0.0480.3890.0490.0340.6210.0530.776 0.0580.0660.9450.0540.9660.0550.121 1:1100640.0580.0500.2700.0480.0220.161 0.0500.3890.0510.0330.6460.0520.778 0.0510.0640.9580.0500.9700.0500.118 1:11008120.0560.0500.1640.0520.0160.121 0.0510.2440.0520.0280.4750.0500.574 0.0550.0400.8250.0530.8590.0590.060 1:110010100.0540.0500.1620.0520.0170.127 0.0510.2480.0550.0200.4800.0540.573 0.0590.0400.8430.0500.8640.0550.060 1:11001280.0560.0480.1640.0550.0190.125 0.0500.2430.0530.0230.4760.0500.582 0.0540.0390.8370.0490.8690.0520.057 1:110040600.0530.0510.0830.0620.0170.110 0.0500.0970.0610.0170.4170.0510.161 0.0590.0190.7650.0500.2720.0610.023Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 201

190 Appendix C (continued): Type I error rate estimates Table 23 (continued). Type I error rate estimates for condi tions when the population effect size variance is 1.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:110050500.0530.0500.0830.0660.0160.107 0.0470.0920.0600.0190.4440.0490.171 0.0630.0220.7890.0490.2850.0610.023 1:110060400.0560.0490.0810.0630.0190.116 0.0550.0980.0670.0170.4240.0510.167 0.0610.0180.7730.0480.2860.0590.021 1:210460.0900.0700.0950.0120.0000.097 0.0670.0960.0110.0000.1440.0660.141 0.0110.0000.2300.0770.1990.0110.000 1:210550.0930.0710.0950.0130.0000.097 0.0650.1000.0130.0000.1490.0700.141 0.0120.0000.2310.0670.2000.0110.000 1:210640.0900.0690.1000.0120.0000.097 0.0660.1090.0140.0000.1580.0650.154 0.0140.0000.2490.0680.2260.0140.000 1:2108120.0710.0650.0540.0140.0000.079 0.0680.0570.0150.0000.1020.0640.072 0.0150.0000.1420.0650.0940.0130.000 1:21010100.0690.0670.0520.0140.0000.071 0.0650.0580.0150.0000.1010.0650.074 0.0160.0000.1510.0660.1050.0140.000 1:2101280.0660.0620.0580.0150.0000.077 0.0650.0630.0150.0000.1100.0670.085 0.0150.0000.1540.0640.1150.0160.000 1:21040600.0530.0620.0490.0230.0000.061 0.0600.0490.0220.0000.0810.0620.056 0.0230.0000.1210.0660.0660.0230.000 1:21050500.0610.0660.0500.0230.0000.061 0.0630.0520.0230.0000.0870.0600.061 0.0230.0000.1230.0640.0710.0230.000 1:21060400.0570.0600.0500.0210.0000.061 0.0610.0500.0200.0000.0840.0600.065 0.0230.0000.1330.0550.0800.0180.000 1:220460.0680.0590.1910.0370.0010.089 0.0560.2220.0360.0010.1930.0580.333 0.0330.0020.3750.0550.5190.0310.006 1:220550.0740.0580.2070.0360.0010.090 0.0530.2240.0310.0020.1910.0540.337 0.0340.0020.3820.0560.5340.0330.008 1:220640.0690.0550.2070.0350.0010.094 0.0550.2360.0340.0020.2100.0570.361 0.0390.0020.4080.0530.5620.0340.005 1:2208120.0570.0550.1130.0400.0000.068 0.0500.1270.0360.0010.1330.0530.197 0.0370.0010.2340.0550.3000.0390.002 1:22010100.0630.0540.1190.0390.0000.072 0.0550.1320.0380.0010.1330.0510.198 0.0390.0020.2480.0530.3110.0350.002 1:2201280.0590.0550.1230.0400.0010.075 0.0550.1370.0390.0010.1450.0530.215 0.0390.0010.2660.0550.3380.0370.002 1:22040600.0510.0510.0780.0480.0000.061 0.0560.0770.0470.0010.1160.0560.099 0.0510.0010.1910.0560.1240.0500.000 1:22050500.0550.0520.0760.0510.0000.060 0.0520.0770.0480.0000.1210.0580.109 0.0500.0000.2130.0580.1300.0530.000 1:22060400.0510.0530.0760.0450.0000.062 0.0520.0820.0470.0000.1250.0510.101 0.0430.0010.2160.0560.1300.0480.000 1:250460.0630.0530.2570.0450.0120.117 0.0530.3200.0460.0180.3650.0520.578 0.0430.0360.7160.0520.8240.0440.061 1:250550.0640.0480.2560.0420.0110.112 0.0510.3240.0450.0180.3820.0500.594 0.0460.0340.7290.0530.8410.0470.062 1:250640.0590.0470.2620.0450.0100.118 0.0500.3360.0470.0180.4060.0540.620 0.0490.0340.7570.0500.8530.0470.058 1:2508120.0570.0510.1600.0490.0080.088 0.0530.1990.0510.0120.2450.0510.371 0.0510.0190.5120.0520.6150.0510.030 1:25010100.0580.0500.1650.0520.0100.091 0.0520.2000.0520.0120.2750.0460.393 0.0470.0200.5500.0480.6500.0490.029 1:2501280.0550.0500.1650.0510.0090.094 0.0540.2210.0540.0120.2770.0490.412 0.0520.0200.5720.0530.6710.0570.031 1:25040600.0470.0470.0890.0560.0070.081 0.0540.1040.0610.0090.2100.0530.129 0.0620.0090.4390.0520.2030.0610.012 1:25050500.0480.0510.0940.0610.0080.078 0.0530.0970.0620.0070.2390.0490.143 0.0590.0110.4950.0500.2120.0590.012 1:25060400.0540.0490.0940.0610.0080.078 0.0470.1010.0550.0110.2480.0480.145 0.0590.0090.4930.0490.2120.0580.011 1:2100460.0550.0530.2670.0480.0180.159 0.0510.3930.0480.0340.6140.0510.763 0.0470.0680.9500.0480.9640.0510.131 1:2100550.0550.0470.2610.0480.0190.162 0.0480.4020.0460.0370.6390.0490.785 0.0500.0680.9540.0520.9690.0540.122 1:2100640.0590.0480.2800.0520.0210.166 0.0520.4150.0520.0340.6770.0540.803 0.0520.0680.9670.0610.9710.0640.120 1:21008120.0560.0540.1770.0570.0190.124 0.0520.2640.0560.0260.4420.0500.565 0.0510.0450.8100.0490.8550.0530.069 1:210010100.0540.0490.1740.0530.0160.129 0.0520.2640.0560.0260.4870.0510.589 0.0570.0460.8440.0520.8710.0550.070Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 202

191 Appendix C (continued): Type I error rate estimates Table 23 (continued). Type I error rate estimates for condi tions when the population effect size variance is 1.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:21001280.0540.0500.1840.0560.0170.126 0.0490.2720.0550.0230.5060.0510.621 0.0570.0410.8630.0540.8900.0620.067 1:210040600.0520.0510.1030.0660.0180.107 0.0500.1170.0620.0190.3940.0550.187 0.0690.0220.7420.0490.3170.0640.028 1:210050500.0500.0450.0970.0600.0180.118 0.0490.1140.0610.0170.4420.0510.190 0.0630.0210.7990.0540.3210.0670.026 1:210060400.0540.0540.0990.0690.0140.118 0.0530.1200.0650.0170.4530.0490.201 0.0630.0240.8070.0520.3290.0680.023 1:410460.0920.0720.1100.0120.0000.102 0.0680.1130.0110.0000.1520.0710.162 0.0140.0000.2310.0690.2200.0090.000 1:410550.0940.0670.1200.0140.0000.097 0.0650.1250.0130.0000.1510.0670.166 0.0130.0000.2470.0690.2420.0120.000 1:410640.0950.0740.1360.0150.0000.107 0.0660.1450.0140.0000.1690.0660.196 0.0160.0000.2740.0750.2750.0170.000 1:4108120.0660.0580.0640.0150.0000.075 0.0690.0710.0180.0000.0980.0650.086 0.0150.0000.1460.0660.1180.0160.000 1:41010100.0700.0670.0740.0150.0000.074 0.0680.0800.0170.0000.1090.0630.098 0.0160.0000.1590.0650.1370.0160.000 1:4101280.0740.0660.0850.0180.0000.081 0.0650.0900.0170.0000.1130.0650.114 0.0160.0000.1740.0650.1560.0160.000 1:41040600.0570.0660.0630.0260.0000.067 0.0670.0710.0300.0000.0820.0570.077 0.0260.0000.1210.0650.0920.0280.000 1:41050500.0620.0630.0760.0250.0000.066 0.0620.0730.0250.0000.0890.0610.082 0.0240.0000.1350.0590.1040.0250.000 1:41060400.0610.0610.0730.0240.0000.065 0.0620.0750.0260.0000.0860.0590.085 0.0230.0000.1480.0610.1060.0250.000 1:420460.0740.0590.2150.0340.0010.093 0.0530.2430.0320.0020.1910.0510.359 0.0320.0040.3640.0550.5330.0320.006 1:420550.0730.0590.2290.0350.0010.093 0.0530.2530.0330.0010.2020.0580.384 0.0370.0020.3890.0520.5660.0340.006 1:420640.0750.0540.2450.0360.0010.092 0.0520.2750.0390.0010.2220.0520.416 0.0380.0030.4400.0580.6200.0430.006 1:4208120.0600.0580.1420.0410.0010.074 0.0570.1520.0410.0010.1290.0560.222 0.0390.0010.2340.0520.3230.0360.002 1:42010100.0620.0560.1560.0400.0010.071 0.0490.1680.0380.0000.1400.0530.242 0.0380.0010.2750.0520.3760.0370.003 1:4201280.0610.0520.1650.0380.0000.080 0.0560.1820.0410.0010.1560.0570.269 0.0430.0010.2990.0570.4180.0430.003 1:42040600.0530.0540.1090.0550.0000.061 0.0510.1090.0520.0000.1120.0570.135 0.0590.0000.1880.0550.1650.0520.001 1:42050500.0540.0540.1120.0530.0000.062 0.0560.1210.0530.0000.1170.0550.142 0.0540.0010.2220.0550.1820.0470.001 1:42060400.0530.0540.1150.0530.0010.064 0.0510.1190.0500.0010.1340.0560.155 0.0530.0010.2470.0550.1860.0480.000 1:450460.0630.0500.2730.0420.0110.109 0.0500.3340.0430.0190.3600.0490.588 0.0410.0420.7130.0510.8340.0420.076 1:450550.0610.0520.2880.0460.0110.119 0.0530.3570.0460.0180.3890.0490.623 0.0450.0400.7480.0530.8670.0480.072 1:450640.0640.0510.3090.0500.0090.123 0.0500.3840.0480.0200.4240.0540.647 0.0510.0380.7970.0560.8840.0570.073 1:4508120.0580.0510.1950.0500.0090.083 0.0480.2240.0480.0140.2520.0510.406 0.0490.0230.5050.0530.6330.0520.036 1:45010100.0580.0490.2000.0550.0090.093 0.0570.2530.0570.0130.2830.0550.446 0.0540.0220.5790.0530.6910.0530.038 1:4501280.0600.0510.2200.0560.0110.096 0.0480.2680.0490.0140.3130.0510.488 0.0560.0220.6300.0500.7420.0550.037 1:45040600.0510.0510.1440.0670.0070.076 0.0540.1510.0680.0100.2100.0520.201 0.0660.0110.4420.0530.2680.0670.013 1:45050500.0540.0530.1430.0690.0080.086 0.0480.1510.0640.0100.2370.0550.196 0.0700.0110.5090.0510.2880.0660.015 1:45060400.0500.0490.1370.0650.0080.084 0.0510.1520.0650.0090.2760.0500.211 0.0630.0130.5480.0540.3020.0680.013 1:4100460.0590.0530.2920.0470.0190.161 0.0500.4090.0440.0370.6190.0490.777 0.0460.0830.9440.0540.9600.0480.150 1:4100550.0580.0480.2980.0470.0170.160 0.0490.4260.0490.0350.6500.0520.800 0.0540.0780.9610.0480.9710.0500.150 1:4100640.0650.0460.3250.0520.0190.183 0.0490.4520.0550.0340.7140.0550.828 0.0610.0790.9770.0660.9800.0710.141Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 203

192 Appendix C (continued): Type I error rate estimates Table 23 (continued). Type I error rate estimates for condi tions when the population effect size variance is 1.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:41008120.0580.0510.2120.0530.0180.127 0.0500.2850.0540.0270.4400.0510.586 0.0560.0510.8130.0530.8680.0530.085 1:410010100.0560.0500.2280.0550.0170.131 0.0500.3140.0550.0270.5070.0510.644 0.0580.0520.8690.0470.8990.0540.089 1:41001280.0580.0460.2470.0580.0160.142 0.0490.3330.0570.0290.5560.0510.693 0.0580.0510.9060.0550.9290.0650.086 1:410040600.0490.0520.1640.0680.0140.106 0.0530.1690.0720.0220.3920.0540.267 0.0780.0300.7430.0530.4050.0750.038 1:410050500.0540.0510.1570.0680.0150.121 0.0510.1770.0730.0220.4530.0500.273 0.0650.0260.8130.0460.4230.0660.038 1:410060400.0510.0510.1530.0710.0130.127 0.0500.1780.0670.0200.4950.0530.278 0.0690.0290.8500.0500.4520.0700.040 1:810460.0900.0710.1370.0120.0000.107 0.0690.1390.0110.0000.1510.0670.184 0.0100.0000.2380.0690.2550.0110.000 1:810550.0980.0680.1520.0140.0000.106 0.0680.1620.0120.0000.1640.0630.213 0.0130.0000.2600.0660.2940.0120.000 1:810640.1000.0730.1760.0190.0000.108 0.0670.1870.0160.0000.1870.0720.245 0.0170.0000.2950.0660.3420.0180.000 1:8108120.0740.0720.0920.0190.0000.075 0.0670.0920.0170.0000.1030.0670.112 0.0170.0000.1480.0670.1490.0150.000 1:81010100.0760.0650.1010.0170.0000.078 0.0610.1060.0150.0000.1120.0650.137 0.0180.0000.1680.0680.1860.0170.000 1:8101280.0770.0660.1210.0180.0000.077 0.0640.1310.0190.0000.1240.0690.159 0.0170.0000.1940.0660.2130.0210.000 1:81040600.0630.0660.0990.0320.0000.060 0.0680.0960.0320.0000.0910.0670.109 0.0370.0000.1310.0720.1340.0380.000 1:81050500.0630.0710.1050.0330.0000.072 0.0660.1120.0370.0000.0990.0680.122 0.0350.0000.1310.0620.1410.0310.000 1:81060400.0630.0610.1060.0300.0000.071 0.0630.1120.0320.0000.1020.0630.128 0.0280.0000.1570.0630.1570.0270.000 1:820460.0760.0570.2610.0340.0010.098 0.0570.2820.0360.0020.1910.0560.387 0.0300.0030.3740.0600.5720.0330.006 1:820550.0720.0540.2600.0360.0010.096 0.0580.2960.0350.0010.2060.0540.432 0.0360.0040.4080.0590.6040.0360.008 1:820640.0730.0560.3080.0390.0010.102 0.0540.3330.0380.0010.2350.0560.484 0.0390.0040.4710.0570.6620.0420.006 1:8208120.0630.0530.1800.0400.0010.077 0.0600.1950.0450.0010.1310.0590.251 0.0400.0020.2370.0570.3730.0390.003 1:82010100.0620.0520.1950.0390.0010.078 0.0570.2160.0430.0010.1470.0580.296 0.0390.0020.2790.0530.4320.0420.003 1:8201280.0640.0490.2270.0400.0010.072 0.0530.2380.0430.0010.1670.0490.338 0.0400.0010.3200.0570.4870.0430.003 1:82040600.0530.0620.1590.0640.0000.064 0.0520.1650.0560.0010.1150.0610.189 0.0640.0000.1970.0590.2310.0670.001 1:82050500.0580.0550.1680.0620.0000.068 0.0490.1720.0580.0000.1240.0540.198 0.0610.0000.2400.0540.2510.0580.001 1:82060400.0560.0520.1720.0560.0000.073 0.0480.1830.0550.0000.1360.0510.211 0.0570.0010.2590.0550.2660.0600.000 1:850460.0610.0520.3090.0440.0120.113 0.0540.3740.0430.0210.3700.0530.619 0.0430.0470.7170.0530.8420.0430.088 1:850550.0620.0500.3290.0440.0100.116 0.0500.3970.0470.0210.4050.0490.652 0.0440.0430.7600.0520.8720.0460.091 1:850640.0640.0540.3480.0560.0100.130 0.0480.4330.0520.0220.4500.0530.695 0.0560.0480.8190.0580.8940.0630.086 1:8508120.0580.0540.2390.0540.0100.090 0.0570.2820.0560.0150.2540.0570.460 0.0550.0290.5180.0540.6870.0520.046 1:85010100.0580.0540.2730.0540.0090.091 0.0500.3070.0510.0150.2920.0500.506 0.0540.0290.6080.0470.7470.0470.045 1:8501280.0560.0510.2840.0540.0090.106 0.0500.3450.0560.0150.3460.0530.563 0.0570.0260.6780.0540.7950.0650.053 1:85040600.0500.0500.2130.0740.0070.082 0.0530.2300.0750.0080.2150.0520.274 0.0770.0140.4450.0610.3710.0860.018 1:85050500.0520.0520.2100.0750.0070.091 0.0540.2340.0750.0100.2670.0520.293 0.0710.0150.5300.0530.3930.0770.021 1:85060400.0530.0520.2160.0710.0050.094 0.0500.2350.0720.0100.2870.0550.298 0.0710.0140.5800.0510.4190.0720.019 1:8100460.0580.0480.3270.0440.0210.156 0.0490.4340.0420.0370.6080.0520.780 0.0430.0890.9400.0500.9620.0430.173Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 204

193 Appendix C (continued): Type I error rate estimates Table 23 (continued). Type I error rate estimates for condi tions when the population effect size variance is 1.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:8100550.0550.0490.3450.0460.0220.170 0.0530.4630.0520.0380.6570.0500.819 0.0500.0950.9640.0480.9720.0490.174 1:8100640.0580.0480.3740.0520.0200.187 0.0540.4960.0590.0370.7340.0600.841 0.0640.0870.9810.0680.9800.0780.168 1:81008120.0570.0510.2640.0540.0180.124 0.0520.3430.0560.0320.4490.0530.636 0.0530.0640.8120.0560.8790.0610.110 1:810010100.0600.0540.2900.0600.0180.136 0.0490.3790.0530.0280.5220.0510.691 0.0590.0640.8760.0520.9100.0580.113 1:81001280.0610.0490.3120.0590.0160.150 0.0470.4190.0580.0310.5870.0530.737 0.0630.0610.9270.0590.9420.0720.118 1:810040600.0540.0500.2410.0790.0150.109 0.0500.2690.0790.0210.4040.0580.371 0.0930.0340.7420.0690.5170.1060.053 1:810050500.0520.0520.2580.0820.0170.115 0.0490.2730.0730.0210.4770.0510.381 0.0790.0320.8370.0500.5550.0790.053 1:810060400.0540.0500.2410.0750.0160.136 0.0510.2710.0770.0210.5400.0490.405 0.0760.0410.8850.0510.5950.0750.063 Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 205

194 Appendix D: Power estimates for conditions with adequate Type I error rates Table 24. Power estimates for conditions when the population effect size variance is 0.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:11046Mod----0.066----------------0.0660.035------------0.0710.055------------0.0730.077-------1:11046Str----0.068----------------0.0720.044------------0.0720.050------------0.0810.076-------1:11055Mod----0.067----------------0.073----------------0.0690.049------------0.0740.074-------1:11055Str----0.069----------------0.066----------------0.0820.054------------0.0760.079-------1:11064Mod0.0850.071----------------0.066----------------0.0690.049------------0.0710.073-------1:11064Str0.0810.068----------------0.073----------------0.0770.050------------0.0790.080-------1:110812Mod0.0700.066------------0.0760.067------------0.1090.071----------------0.070-----------1:110812Str0.0750.071------------0.0850.071------------0.1530.086----------------0.088-----------1:1101010Mod0.0700.064------------0.0730.068----------------0.072----------------0.073-----------1:1101010Str0.0710.067------------0.0870.073----------------0.091----------------0.088-----------1:110128Mod0.0720.069------------0.0750.070----------------0.076----------------0.073-----------1:110128Str0.0730.070------------0.0860.072----------------0.088----------------0.080-----------1:1104060Mod0.0650.066------------0.0840.077------------0.0800.067------------0.0760.060-----------1:1104060Str0.0740.071------------0.1180.096------------0.1040.076------------0.0730.058-----------1:1105050Mod0.0640.065------------0.0810.073------------0.0800.070------------0.0690.059-----------1:1105050Str0.0730.071------------0.1170.106------------0.1000.078------------0.0740.062-----------1:1106040Mod0.0670.065------------0.0760.070------------0.0830.069------------0.0710.060-----------1:1106040Str0.0760.076------------0.1160.093------------0.1020.081------------0.0750.063-----------1:12046Mod0.0660.0540.091------------0.0570.115------------0.056----------------0.057-----------1:12046Str0.0660.0570.107------------0.0570.126------------0.066----------------0.075-----------1:12055Mod0.0650.0500.083--------0.0950.0570.104------------0.062----------------0.058-----------1:12055Str0.0650.0540.103--------0.1140.0610.123------------0.067----------------0.081-----------1:12064Mod0.0660.0590.091--------0.0930.0540.112------------0.057----------------0.063-----------1:12064Str0.0700.0610.108--------0.1190.0590.124------------0.067----------------0.075-----------1:120812Mod0.0620.0570.056--------0.0730.0540.058--------0.1520.0650.083------------0.0630.135-------1:120812Str0.0630.0550.067--------0.0970.0610.072--------0.2410.0980.111------------0.0960.179-------1:1201010Mod0.0590.0540.056--------0.0710.0540.056------------0.0680.087------------0.0680.130-------1:1201010Str0.0650.0610.071--------0.0980.0670.071------------0.1050.113------------0.1090.184-------1:120128Mod0.0590.0560.056--------0.0720.0550.057--------0.1490.0660.082------------0.0660.133-------1:120128Str0.0620.0580.065--------0.0990.0660.070--------0.2450.1000.116------------0.1000.176-------1:1204060Mod0.0580.0560.059--------0.0830.0720.055--------0.1030.0700.055--------0.0780.0520.048-------1:1204060Str0.0620.0620.082--------0.1440.1150.078--------0.1530.0950.081--------0.1020.0560.057-------1:1205050Mod0.0570.0570.060--------0.0810.0650.052--------0.1020.0660.055--------0.0790.0520.045-------1:1205050Str0.0650.0630.078--------0.1470.1120.082--------0.1540.1010.084--------0.0940.0550.049-------Population Effect Size 0.8 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5

PAGE 206

195 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 24 (continued). Power estimates for conditions when the population effect size variance is 0.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:1206040Mod0.0590.0530.061--------0.0830.0690.054--------0.1090.0660.059--------0.0790.0510.047-------1:1206040Str0.0610.0580.079--------0.1430.1090.078--------0.1590.0960.081--------0.0980.0580.059-------1:15046Mod0.0600.052----0.035--------0.053----------------0.052----0.0280.007----0.057----0.021---1:15046Str0.0610.051----0.036--------0.059----------------0.073----0.0310.002----0.098----0.029---1:15055Mod0.0630.0510.120------------0.047----0.031--------0.051----0.0300.007----0.059----0.023---1:15055Str0.0600.0520.140------------0.055----0.036--------0.083----0.0350.002----0.110----0.030---1:15064Mod0.0560.0480.115------------0.053----------------0.055----0.0280.006----0.057----0.026---1:15064Str0.0640.0580.146------------0.060----------------0.079----0.0320.002----0.105----0.029---1:150812Mod0.0560.0530.0710.039----0.0940.0550.0910.038--------0.074----0.0370.008----0.087----0.0340.112 1:150812Str0.0610.0550.0930.041----0.1570.0700.1150.042--------0.161----0.0530.021----0.186----0.0710.239 1:1501010Mod0.0560.0510.0740.036----0.0870.0540.0860.038--------0.077----0.0330.008----0.084----0.0350.123 1:1501010Str0.0560.0530.0900.040----0.1550.0700.1080.040--------0.180----0.0590.024----0.202----0.0740.258 1:150128Mod0.0530.0500.0700.036----0.0900.0540.0880.039--------0.070----0.0330.009----0.087----0.0360.113 1:150128Str0.0580.0560.0890.039----0.1480.0670.1090.039--------0.163----0.0540.021----0.182----0.0690.238 1:1504060Mod0.0570.0540.0820.043----0.1140.0770.0880.040----0.1950.0960.1520.051----0.1320.0530.1180.0320.087 1:1504060Str0.0610.0600.1060.046----0.2640.1810.1510.067----0.3450.1720.2670.087----0.1880.0690.1730.0370.131 1:1505050Mod0.0590.0540.0810.043----0.1170.0840.0900.044----0.1940.0970.1560.0530.123----0.0490.1140.0320.083 1:1505050Str0.0620.0560.1140.043----0.2740.2020.1540.073----0.3400.1840.2700.0920.243----0.0620.1720.0380.123 1:1506040Mod0.0550.0550.0780.041----0.1150.0840.0890.042----0.2000.1000.1550.0530.1220.1360.0520.1180.0310.084 1:1506040Str0.0660.0610.1080.049----0.2640.1800.1480.069----0.3460.1760.2730.0860.2520.1920.0680.1780.0380.133 1:110046Mod0.0570.050----0.036--------0.049----0.0350.015----0.051----0.033--------0.061----0.025---1:110046Str0.0610.055----0.043--------0.054----0.0400.005----0.097----0.039--------0.150----0.045---1:110055Mod0.0640.053----------------0.049----0.0360.015----0.052----0.032--------0.073----0.028---1:110055Str0.0610.053----------------0.058----0.0380.006----0.111----0.041--------0.182----0.053---1:110064Mod0.0550.046----0.037--------0.053--------0.014----0.054----0.034--------0.063----0.028---1:110064Str0.0580.051----0.040--------0.061--------0.004----0.092----0.037--------0.163----0.052---1:1100812Mod0.0540.0520.0800.041----0.1240.0520.1120.0380.006----0.094----0.0480.010----0.128----0.062---1:1100812Str0.0600.0540.0940.047----0.2530.0910.1440.0420.001----0.274----0.0960.004----0.329----0.139---1:11001010Mod0.0570.0510.0770.040----0.1310.0580.1230.0420.006----0.099----0.045--------0.144----0.061---1:11001010Str0.0610.0550.1020.043----0.2570.0950.1470.0430.002----0.322----0.109--------0.366----0.164---1:1100128Mod0.0560.0520.0810.041----0.1230.0500.1120.0360.007----0.095----0.0470.010----0.131----0.059---1:1100128Str0.0610.0550.0960.045----0.2500.0820.1400.0410.001----0.275----0.0930.004----0.336----0.146---1:11004060Mod0.0540.0530.0860.043----0.1780.1100.1240.0590.0090.3650.1460.3050.0850.164----0.0660.2350.0440.182 1:11004060Str0.0560.0560.1090.046----0.4760.3180.2320.1130.0050.6200.3190.5410.1700.358----0.0870.3670.0560.305 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 207

196 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 24 (continued). Power estimates for conditions when the population effect size variance is 0.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:11005050Mod0.0550.0520.0820.043----0.1810.1210.1200.0600.0100.3590.1590.3020.0910.166----0.062----0.0410.183 1:11005050Str0.0610.0610.1140.048----0.4780.3450.2320.1210.0040.6160.3290.5310.1790.366----0.092----0.0540.306 1:11006040Mod0.0510.0520.0780.044----0.1820.1080.1210.0560.0100.3650.1560.3060.0880.165----0.063----0.0430.191 1:11006040Str0.0560.0520.1100.047----0.4760.3190.2270.1160.0050.6240.3160.5320.1750.349----0.091----0.0510.314 1:21046Mod----0.075----------------0.0720.046------------0.0730.054------------0.0720.074-------1:21046Str----0.073----------------0.0750.046------------0.0840.064------------0.0890.083-------1:21055Mod----0.070----------------0.0690.041------------0.0740.057------------0.0710.080-------1:21055Str----0.067----------------0.0710.044------------0.0730.060------------0.0800.095-------1:21064Mod0.0800.0670.038------------0.0670.043------------0.0660.055------------0.067-----------1:21064Str0.0880.0700.046------------0.0700.047------------0.0700.067------------0.071-----------1:210812Mod0.0700.068------------0.0760.069------------0.1130.086----------------0.078-----------1:210812Str0.0790.075------------0.0940.079------------0.1610.106----------------0.099-----------1:2101010Mod0.0650.066------------0.0770.069------------0.1180.075----------------0.072-----------1:2101010Str0.0740.070------------0.0910.073------------0.1630.093----------------0.087-----------1:210128Mod0.0670.067------------0.0750.064------------0.1120.069----------------0.0610.042-------1:210128Str0.0740.067------------0.0940.071------------0.1600.076----------------0.0750.055-------1:2104060Mod0.0700.070------------0.0850.082------------0.0850.073------------0.0680.065-----------1:2104060Str0.0780.077------------0.1200.112------------0.1050.095------------0.0780.070-----------1:2105050Mod0.0680.069------------0.0800.070------------0.0830.067------------0.0740.063-----------1:2105050Str0.0730.066------------0.1130.092------------0.1020.082------------0.0810.064-----------1:2106040Mod0.0700.061------------0.0790.068------------0.0860.064------------0.0760.063-----------1:2106040Str0.0790.070------------0.1220.086------------0.1080.067------------0.0810.059-----------1:22046Mod0.0700.0570.097------------0.0570.115------------0.060----------------0.067-----------1:22046Str0.0710.0560.119------------0.0640.137------------0.081----------------0.093-----------1:22055Mod0.0650.0530.097------------0.055----------------0.054----------------0.065-----------1:22055Str0.0720.0560.117------------0.060----------------0.070----------------0.076-----------1:22064Mod0.0670.0550.096------------0.054----------------0.057----------------0.057-----------1:22064Str0.0740.0610.117------------0.059----------------0.054----------------0.065-----------1:220812Mod0.0610.0630.067--------0.0720.0610.063--------0.1500.0730.084------------0.0820.131-------1:220812Str0.0640.0580.076--------0.0960.0690.074--------0.2530.1200.122------------0.1250.175-------1:2201010Mod0.0650.0560.070--------0.0690.0530.061--------0.1530.0660.095------------0.067-----------1:2201010Str0.0630.0560.079--------0.1000.0640.083--------0.2630.1050.140------------0.107-----------1:220128Mod0.0610.0510.062--------0.0720.0540.066------------0.0570.102------------0.059-----------1:220128Str0.0620.0560.075--------0.1070.0590.082------------0.0880.153------------0.082-----------Population Effect Size 0.8 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5

PAGE 208

197 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 24 (continued). Power estimates for conditions when the population effect size variance is 0.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:2204060Mod0.0590.0570.077--------0.0810.0800.062--------0.1020.0840.068--------0.0790.0640.059-------1:2204060Str0.0680.0690.112--------0.1530.1320.100--------0.1480.1170.095--------0.0960.0710.070-------1:2205050Mod0.0600.0610.079--------0.0830.0700.073--------0.1090.0710.079--------0.0850.0520.067-------1:2205050Str0.0620.0590.110--------0.1470.1130.108--------0.1580.0950.116--------0.0970.0560.084-------1:2206040Mod0.0650.0580.091--------0.0810.0590.078--------0.1110.0580.0900.026----0.0950.0520.0830.022---1:2206040Str0.0610.0600.114--------0.1460.0930.112--------0.1720.0740.1280.027----0.1110.0480.1040.022---1:25046Mod0.0640.054----------------0.053----------------0.062--------0.010----0.078-----------1:25046Str0.0640.051----------------0.057----------------0.101--------0.003----0.149-----------1:25055Mod0.0560.045----------------0.053----0.034--------0.052--------0.009----0.059----0.024---1:25055Str0.0640.054----------------0.054----0.034--------0.076--------0.002----0.120----0.034---1:25064Mod0.0570.052----0.037--------0.053----0.033--------0.053----0.0320.008----0.050----0.027---1:25064Str0.0650.054----0.041--------0.054----0.040--------0.058----0.0320.002----0.068----0.029---1:250812Mod0.0540.0530.079--------0.0900.0570.096------------0.097----0.0370.012----0.129----0.0460.143 1:250812Str0.0610.0560.097--------0.1490.0800.125------------0.219----0.0650.028----0.254----0.0890.274 1:2501010Mod0.0600.0500.0860.038----0.0940.0570.1050.037--------0.083----0.0390.013----0.095----0.0430.136 1:2501010Str0.0570.0530.0950.038----0.1590.0770.1240.042--------0.186----0.0600.023----0.202----0.0830.265 1:250128Mod0.0570.0500.0880.038----0.0990.0480.1080.034--------0.061----0.0340.008----0.067----0.035---1:250128Str0.0610.0540.1030.042----0.1590.0620.1300.039--------0.127----0.0470.015----0.132----0.060---1:2504060Mod0.0540.0540.0920.041----0.1140.1020.1010.047----0.1950.1350.1640.0680.1440.1430.0860.1460.0500.112 1:2504060Str0.0580.0610.1330.048----0.2770.2320.1750.077----0.3340.2390.2790.1150.2540.1920.1070.2000.0570.157 1:2505050Mod0.0550.0510.1020.041----0.1170.0850.1120.0420.0090.1980.1000.1890.0570.140----0.055----0.0350.110 1:2505050Str0.0600.0570.1390.046----0.2710.2000.1880.0770.0090.3540.1860.3220.1020.273----0.074----0.0420.158 1:2506040Mod0.0530.0510.1070.039----0.1140.0690.1180.043----0.2200.069----0.0440.137----0.045----0.0340.114 1:2506040Str0.0630.0580.1490.050----0.2720.1470.1930.057----0.3660.114----0.0680.263----0.051----0.0360.165 1:210046Mod0.0600.055----------------0.049--------0.017----0.069----------------0.097----0.029---1:210046Str0.0660.056----------------0.064--------0.005----0.146----------------0.254----0.072---1:210055Mod0.0600.051----------------0.051----0.0380.017----0.059----0.033--------0.071----0.030---1:210055Str0.0630.055----------------0.057----0.0360.005----0.112----0.038--------0.183----0.061---1:210064Mod0.0620.049----0.039--------0.048----0.0350.015----0.051----0.038--------0.050----0.030---1:210064Str0.0610.054----0.044--------0.052----0.0390.006----0.067----0.035--------0.092----0.039---1:2100812Mod0.0540.0510.0880.039----0.1190.0610.1230.0410.007----0.141----0.054--------0.202----0.083---1:2100812Str0.0590.0540.1060.044----0.2470.1000.1600.0400.002----0.402----0.130--------0.474----0.209---1:21001010Mod0.0580.0550.0900.042----0.1390.059----0.0450.008----0.109----0.047--------0.143----0.070---1:21001010Str0.0640.0560.1110.045----0.2690.096----0.0450.002----0.325----0.115--------0.383----0.187---Population Effect Size 0.8 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5

PAGE 209

198 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 24 (continued). Power estimates for conditions when the population effect size variance is 0.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:2100128Mod0.0550.0490.0900.040----0.1360.051----0.0370.005----0.066----0.039--------0.083----0.047---1:2100128Str0.0590.0530.1080.044----0.2660.073----0.0420.002----0.192----0.075--------0.224----0.111---1:21004060Mod0.0540.0580.1010.0470.0270.1800.1560.1420.0720.0130.3570.2390.3150.1360.195----0.126----0.0740.255 1:21004060Str0.0590.0580.1420.0510.0280.4770.4070.2540.1540.0090.6170.4510.5360.2450.385----0.179----0.1020.387 1:21005050Mod0.0540.0560.1030.0460.0260.2030.1320.1570.0670.0120.3730.158----0.0980.187----0.072----0.047---1:21005050Str0.0600.0550.1440.0480.0270.4770.3510.2670.1240.0070.6260.341----0.1920.376----0.096----0.059---1:21006040Mod0.0590.0540.1160.046----0.1910.0870.1490.0520.011----0.094----0.0640.166----0.044----0.0410.245 1:21006040Str0.0610.0590.1530.052----0.4740.2390.2540.0880.006----0.208----0.1240.343----0.052----0.0410.389 1:41046Mod----0.0740.054------------0.0730.054------------0.0740.068------------0.082-----------1:41046Str----0.0770.067------------0.0750.066------------0.0910.082------------0.095-----------1:41055Mod----0.0670.055------------0.0720.055------------0.0700.076------------0.076-----------1:41055Str----0.0730.059------------0.0720.061------------0.0720.084------------0.083-----------1:41064Mod----0.0650.053------------0.0640.061------------0.067----------------0.072-----------1:41064Str----0.0650.054------------0.0680.061------------0.069----------------0.066-----------1:410812Mod0.0710.071------------0.0810.078----------------0.0820.034------------0.0870.048-------1:410812Str0.0790.072------------0.0970.085----------------0.1170.050------------0.1150.060-------1:4101010Mod0.0690.066------------0.0820.070----------------0.0700.042------------0.0730.061-------1:4101010Str0.0760.075------------0.0990.072----------------0.0920.060------------0.0880.085-------1:410128Mod0.0770.0670.040--------0.0870.0650.044------------0.0690.055------------0.0620.076-------1:410128Str0.0820.0680.049--------0.0960.0650.049------------0.0730.071------------0.0690.100-------1:4104060Mod0.0660.0700.068--------0.0840.0880.059--------0.0840.0880.051--------0.0730.0750.055-------1:4104060Str0.0810.0810.102--------0.1210.1170.086--------0.1020.1020.064--------0.0830.0820.057-------1:4105050Mod0.0670.0670.077--------0.0830.0750.067--------0.0900.0670.067--------0.0780.0620.063-------1:4105050Str0.0800.0730.102--------0.1200.1010.100--------0.1140.0860.077--------0.0890.0650.069-------1:4106040Mod0.0700.0650.081--------0.0870.0610.082--------0.0930.0560.074--------0.0910.0580.073-------1:4106040Str0.0770.0670.109--------0.1170.0790.108--------0.1190.0660.090--------0.0990.0600.082-------1:42046Mod0.0730.0580.115------------0.055----------------0.066----------------0.078-----------1:42046Str0.0750.0630.142------------0.063----------------0.092----------------0.113-----------1:42055Mod0.0700.059----------------0.059----------------0.059----------------0.065-----------1:42055Str0.0720.054----------------0.061----------------0.071----------------0.081-----------1:42064Mod0.0660.049----------------0.054----------------0.056----------------0.051-----------1:42064Str0.0730.058----------------0.056----------------0.059----------------0.058-----------1:420812Mod0.0610.0600.081--------0.0800.0660.087------------0.0890.116------------0.094-----------1:420812Str0.0710.0630.103--------0.1070.0710.112------------0.1540.168------------0.152-----------Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 210

199 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 24 (continued). Power estimates for conditions when the population effect size variance is 0.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:4201010Mod0.0600.0540.088--------0.0800.0600.092------------0.069----------------0.072-----------1:4201010Str0.0680.0600.102--------0.1150.0720.124------------0.103----------------0.112-----------1:420128Mod0.0660.0520.092--------0.0860.0570.101------------0.054----0.022--------0.054----0.021---1:420128Str0.0700.0580.103--------0.1120.0610.117------------0.074----0.025--------0.071----0.024---1:4204060Mod0.0610.0640.125--------0.0850.0880.113--------0.1030.1010.113--------0.0820.081----0.030---1:4204060Str0.0680.0670.177--------0.1540.1500.169--------0.1550.1410.153--------0.1040.097----0.039---1:4205050Mod0.0650.0590.140--------0.0930.076------------0.1160.073------------0.0950.057-----------1:4205050Str0.0690.0600.192--------0.1550.121------------0.1680.103------------0.1200.062-----------1:4206040Mod0.0660.057------------0.0940.059------------0.1240.058----------------0.050----0.023---1:4206040Str0.0700.059------------0.1470.079------------0.1840.066----------------0.049----0.024---1:45046Mod0.0580.052----------------0.052--------0.010----0.073--------0.016----0.104-----------1:45046Str0.0620.055----------------0.068--------0.003----0.133--------0.008----0.212-----------1:45055Mod0.0620.054----------------0.051----0.0320.008----0.057--------0.013----0.065----0.024---1:45055Str0.0630.055----------------0.056----0.0350.003----0.087--------0.006----0.119----0.040---1:45064Mod0.0610.054----0.038--------0.051----0.0380.009----0.049----0.0330.011----0.052----0.030---1:45064Str0.0650.053----0.037--------0.049----0.0360.004----0.057----0.0340.003----0.057----0.030---1:450812Mod0.0580.0550.106--------0.0960.0630.128----0.006----0.124--------0.024----0.164-----------1:450812Str0.0680.0630.137--------0.1570.0880.163----0.002----0.279--------0.039----0.328-----------1:4501010Mod0.0600.053----0.038----0.1060.060----0.0360.006----0.091----0.0420.020----0.104----0.053---1:4501010Str0.0650.058----0.043----0.1720.074----0.0390.001----0.197----0.0720.024----0.212----0.096---1:450128Mod0.0570.051----0.039----0.1050.049----0.0340.005----0.057----0.0330.014----0.058----0.035---1:450128Str0.0590.052----0.038----0.1780.060----0.0400.002----0.103----0.0460.014----0.101----0.053---1:4504060Mod0.0580.0610.1540.046----0.1220.123----0.0600.0130.2020.187----0.0990.165------------0.078---1:4504060Str0.0620.0590.2160.052----0.2790.274----0.1000.0180.3390.301----0.1610.290------------0.098---1:4505050Mod0.0600.058----0.043----0.1280.093----0.0480.0100.2220.111----0.0690.169----0.058----0.039---1:4505050Str0.0650.064----0.055----0.2820.206----0.0800.0120.3630.190----0.1060.288----0.080----0.049---1:4506040Mod0.0610.054----0.043----0.1340.063----0.0360.010----0.058----0.0370.169----0.052----0.038---1:4506040Str0.0620.055----0.049----0.2760.120----0.0500.007----0.096----0.0590.290----0.047----0.035---1:410046Mod0.0580.054----------------0.055--------0.021----0.087----------------0.162-----------1:410046Str0.0610.056----------------0.068--------0.009----0.206----------------0.369-----------1:410055Mod0.0560.051----0.034--------0.055----0.0370.021----0.058----0.030--------0.084----0.036---1:410055Str0.0600.053----0.041--------0.058----0.0380.008----0.114----0.038--------0.196----0.068---1:410064Mod0.0580.049----0.038--------0.049----0.0390.019----0.053----0.040----------------0.037---1:410064Str0.0540.052----0.044--------0.048----0.0400.009----0.057----0.034----------------0.038---Population Effect Size 0.8 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5

PAGE 211

200 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 24 (continued). Power estimates for conditions when the population effect size variance is 0.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:4100812Mod0.0580.0550.116--------0.1250.067----0.0410.016----0.199----0.070--------0.288----0.126---1:4100812Str0.0620.0560.141--------0.2660.130----0.0470.004----0.491----0.173--------0.589----0.288---1:41001010Mod0.0580.053----0.042--------0.057----0.0350.017----0.121----0.053--------0.163----0.084---1:41001010Str0.0630.055----0.043--------0.094----0.0420.004----0.344----0.119--------0.396----0.209---1:4100128Mod0.0580.054----0.042--------0.051----0.0400.015----0.061----0.034--------0.067----0.042---1:4100128Str0.0630.054----0.047--------0.067----0.0420.004----0.159----0.061--------0.170----0.091---1:41004060Mod0.0550.0550.1610.046----0.1850.189----0.0830.0250.360--------0.200-----------------------1:41004060Str0.0610.0600.2230.054----0.4860.477----0.1890.0210.601--------0.343-----------------------1:41005050Mod0.0580.056----0.0450.0250.2010.128----0.0610.024----0.171----0.114--------0.083----0.057---1:41005050Str0.0610.057----0.0490.0290.4760.344----0.1290.018----0.346----0.216--------0.120----0.076---1:41006040Mod0.0600.051----0.0430.0270.2110.079----0.0480.022------------0.052-----------------------1:41006040Str0.0580.054----0.0510.0250.4680.195----0.0700.012------------0.086-----------------------1:81046Mod----0.0710.073------------0.0730.074------------0.0800.094------------0.084-----------1:81046Str----0.0700.078------------0.0770.086------------0.0890.115------------0.107-----------1:81055Mod----0.0650.073------------0.0720.080------------0.072----------------0.072-----------1:81055Str----0.0720.082------------0.0750.090------------0.077----------------0.082-----------1:81064Mod----0.0680.080------------0.0690.090------------0.064----------------0.065-----------1:81064Str----0.0710.083------------0.0690.094------------0.066----------------0.066-----------1:810812Mod0.0740.0740.059--------0.0840.0790.059------------0.0870.062------------0.0960.081-------1:810812Str0.0880.0820.077--------0.1060.0890.071------------0.1240.085------------0.1230.103-------1:8101010Mod0.0730.0640.065--------0.0880.0730.068------------0.0740.077------------0.0830.104-------1:8101010Str0.0870.0700.081--------0.1090.0790.080------------0.0950.104------------0.0920.134-------1:810128Mod0.0760.0610.066--------0.0870.0650.067------------0.0660.097------------0.067-----------1:810128Str0.0820.0680.075--------0.1020.0640.073------------0.0770.116------------0.072-----------1:8104060Mod0.0720.0750.118--------0.0880.0900.107--------0.0930.0990.100--------0.074---------------1:8104060Str0.0870.0820.172--------0.1260.1280.148--------0.1120.1120.112--------0.079---------------1:8105050Mod0.0790.0720.136--------0.0910.0800.120--------0.0960.072----------------0.065-----------1:8105050Str0.0860.0740.187--------0.1270.1020.167--------0.1190.088----------------0.064-----------1:8106040Mod0.0800.066------------0.0890.064----------------0.060----------------0.061-----------1:8106040Str0.0870.070------------0.1280.078----------------0.061----------------0.060-----------1:82046Mod0.0740.061----------------0.064----------------0.074----------------0.089-----------1:82046Str0.0740.057----------------0.066----------------0.097----------------0.138-----------1:82055Mod0.0760.055----------------0.054----------------0.062----------------0.061-----------1:82055Str0.0710.058----------------0.057----------------0.072----------------0.079-----------Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 212

201 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 24 (continued). Power estimates for conditions when the population effect size variance is 0.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:82064Mod0.0730.057----------------0.055----------------0.054----0.022--------0.054-----------1:82064Str0.0710.055----------------0.055----------------0.053----0.025--------0.052-----------1:820812Mod0.0630.0600.119--------0.0770.0640.112------------0.095----------------0.108-----------1:820812Str0.0670.0640.139--------0.1130.0840.160------------0.170----------------0.181-----------1:8201010Mod0.0680.059------------0.0840.059----------------0.073----------------0.076-----------1:8201010Str0.0680.058------------0.1160.070----------------0.107----------------0.119-----------1:820128Mod0.0650.052------------0.0890.054----------------0.058----------------0.056----0.023---1:820128Str0.0720.054------------0.1210.059----------------0.065----------------0.067----0.027---1:8204060Mod0.0650.064------------0.0870.096------------0.1120.118----0.048----0.093--------0.046---1:8204060Str0.0750.071------------0.1590.164------------0.1580.166----0.069----0.111--------0.055---1:8205050Mod0.0690.062------------0.0940.080------------0.1280.076----------------0.059----0.030---1:8205050Str0.0690.065------------0.1450.113------------0.1720.105----------------0.063----0.030---1:8206040Mod0.0670.054----0.039----0.1030.064----0.034----0.1440.051----0.026--------0.056----0.030---1:8206040Str0.0710.059----0.048----0.1530.082----0.039----0.2080.065----0.033--------0.053----0.028---1:85046Mod0.0630.054----------------0.054--------0.012----0.085----------------0.137-----------1:85046Str0.0640.056----------------0.070--------0.008----0.157----------------0.256-----------1:85055Mod0.0650.051----------------0.056--------0.013----0.056----0.024--------0.075----0.028---1:85055Str0.0630.049----------------0.059--------0.006----0.088----0.031--------0.130----0.041---1:85064Mod0.0610.050----0.037--------0.052----0.0400.009----0.054----0.035--------0.055----0.034---1:85064Str0.0610.047----0.040--------0.050----0.0370.006----0.056----0.035--------0.053----0.027---1:850812Mod0.0640.061------------0.0970.068--------0.011----0.156--------0.034----0.203----0.079---1:850812Str0.0630.056------------0.1700.103--------0.003----0.326--------0.049----0.385----0.166---1:8501010Mod0.0590.055----0.038----0.1120.060----0.0360.009----0.091----0.040--------0.117----0.056---1:8501010Str0.0640.055----0.039----0.1820.080----0.0400.004----0.200----0.076--------0.229----0.113---1:850128Mod0.0620.053----0.041--------0.054----0.0390.008----0.054----0.0360.020----0.052----0.033---1:850128Str0.0630.055----0.044--------0.058----0.0410.004----0.099----0.0460.017----0.098----0.049---1:8504060Mod0.0630.062----0.053----0.1260.140----0.0620.0200.2030.212----0.1310.200-------------------1:8504060Str0.0650.064----0.064----0.2750.301----0.1190.0230.3410.349----0.2130.320-------------------1:8505050Mod0.0600.054----0.045----0.1340.094----0.0500.016----0.116----0.076--------0.070----0.047---1:8505050Str0.0690.063----0.063----0.2880.206----0.0870.019----0.198----0.120--------0.085----0.060---1:8506040Mod0.0610.055----0.048----0.1420.065----0.0420.013----0.056----0.036----------------0.042---1:8506040Str0.0620.055----0.058----0.2620.110----0.0570.010----0.086----0.051----------------0.035---1:810046Mod0.0660.058----------------0.059--------0.031----0.115----------------0.211-----------1:810046Str0.0610.053----------------0.081--------0.015----0.263----------------0.458-----------Population Effect Size 0.8 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5

PAGE 213

202 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 24 (continued). Power estimates for conditions when the population effect size variance is 0.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:810055Mod0.0590.051----0.038--------0.048----0.0300.029----0.067----0.031--------0.086----0.033---1:810055Str0.0600.051----0.035--------0.058----0.0340.014----0.126----0.038--------0.209----0.074---1:810064Mod0.0610.051----0.040--------0.048----0.0400.026----0.054----0.041-----------------------1:810064Str0.0590.052----0.042--------0.049----0.0410.014----0.054----0.037-----------------------1:8100812Mod0.0600.059----0.044----0.1380.081----0.0460.021----0.253----0.096--------0.358----0.172---1:8100812Str0.0620.056----0.047----0.2710.148----0.0490.010----0.559----0.222--------0.666----0.373---1:81001010Mod0.0580.053----0.039--------0.061----0.0360.022----0.136----0.058--------0.176----0.087---1:81001010Str0.0640.060----0.049--------0.103----0.0430.009----0.357----0.129--------0.435----0.238---1:8100128Mod0.0600.051----0.038--------0.052----0.0390.018----0.061----0.034----------------0.037---1:8100128Str0.0600.053----0.049--------0.063----0.0450.008----0.142----0.054----------------0.072---1:81004060Mod0.0590.061----0.051----0.1920.222----0.1060.037------------0.265-----------------------1:81004060Str0.0630.061----0.067----0.4770.510----0.2120.034------------0.430-----------------------1:81005050Mod0.0600.054----0.0470.0260.2220.139----0.0700.034----0.189----0.128--------0.094----0.069---1:81005050Str0.0600.059----0.0640.0270.4800.347----0.1330.030----0.372----0.246--------0.128----0.099---1:81006040Mod0.0600.052----0.0490.0230.2270.068----0.0440.030------------0.043-----------------------1:81006040Str0.0620.052----0.0570.0290.4670.167----0.0660.027------------0.084-----------------------Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 214

203 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 25. Power estimates for conditions when the population effect size variance is 0.10. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:11046Mod0.0810.0680.039------------0.0650.044------------0.0680.060------------0.069-----------1:11046Str0.0850.0690.044------------0.0710.052------------0.0740.064------------0.077-----------1:11055Mod----0.0700.043------------0.0630.045------------0.0710.063------------0.071-----------1:11055Str----0.0680.044------------0.0710.047------------0.0730.065------------0.077-----------1:11064Mod----0.0640.043------------0.0650.045------------0.0690.064------------0.068-----------1:11064Str----0.0660.044------------0.0710.049------------0.0740.067------------0.073-----------1:110812Mod0.0670.064------------0.0710.063------------0.1030.071----------------0.0670.036-------1:110812Str0.0700.070------------0.0830.068------------0.1400.079----------------0.0760.043-------1:1101010Mod0.0650.063------------0.0730.066----------------0.066----------------0.0670.040-------1:1101010Str0.0710.068------------0.0810.068----------------0.074----------------0.0780.050-------1:110128Mod0.0680.071------------0.0740.065------------0.1050.067----------------0.0670.037-------1:110128Str0.0730.071------------0.0840.071------------0.1380.077----------------0.0760.048-------1:1104060Mod0.0640.065------------0.0700.065------------0.0820.063------------0.0810.055-----------1:1104060Str0.0710.065------------0.0790.069------------0.0980.068------------0.1020.063-----------1:1105050Mod0.0630.067------------0.0710.064------------0.0870.066------------0.0880.059-----------1:1105050Str0.0690.069------------0.0790.070------------0.1020.073------------0.0940.059-----------1:1106040Mod0.0580.062------------0.0710.065------------0.0810.067----------------0.059-----------1:1106040Str0.0700.068------------0.0790.068------------0.1020.074----------------0.060-----------1:12046Mod0.0630.0500.105--------0.0950.057----------------0.055----------------0.056-----------1:12046Str0.0710.0580.114--------0.1160.055----------------0.063----------------0.067-----------1:12055Mod0.0660.0580.103------------0.054----------------0.057----------------0.058-----------1:12055Str0.0660.0560.109------------0.060----------------0.060----------------0.070-----------1:12064Mod0.0690.057----------------0.061----------------0.059----------------0.057-----------1:12064Str0.0730.059----------------0.055----------------0.061----------------0.070-----------1:120812Mod0.0610.0530.0610.031----0.0700.0530.0640.030--------0.0590.1060.026--------0.059-----------1:120812Str0.0580.0540.0690.032----0.0890.0580.0780.033--------0.0780.1320.031--------0.079-----------1:1201010Mod0.0560.0540.0600.032----0.0730.0590.070------------0.0550.103------------0.064----0.028---1:1201010Str0.0680.0610.0740.033----0.0840.0580.070------------0.0760.137------------0.081----0.034---1:120128Mod0.0570.0580.0600.033----0.0660.0530.0650.028--------0.0580.1040.027--------0.059----0.025---1:120128Str0.0630.0560.0720.032----0.0910.0580.0770.034--------0.0750.1280.030--------0.082----0.029---1:1204060Mod0.0560.0570.0570.037----0.0680.0580.0540.035----0.1090.0620.0660.037--------0.0580.0740.035---1:1204060Str0.0600.0570.0670.036----0.0810.0640.0650.038----0.1460.0730.0870.043--------0.0600.0950.035---1:1205050Mod0.0570.0570.0610.037----0.0730.0600.0600.040----0.1120.0610.0660.040--------0.0520.0700.032---1:1205050Str0.0590.0560.0690.037----0.0890.0670.0670.040----0.1480.0730.0880.044--------0.0600.0910.034---Population Effect Size 0.8 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5

PAGE 215

204 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 25 (continued). Power estimates for conditions when the population effect size variance is 0.10. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:1206040Mod0.0540.0520.0530.036----0.0700.0570.0560.032----0.1030.0590.0660.034--------0.0560.0640.035---1:1206040Str0.0590.0550.0680.040----0.0880.0640.0700.038----0.1450.0700.0870.043--------0.0590.0910.037---1:15046Mod0.0590.051----0.039--------0.050----0.038--------0.056----0.0380.008----0.057----0.029---1:15046Str0.0580.049----0.038--------0.054----0.037--------0.066----0.0370.001----0.087----0.037---1:15055Mod0.0580.054----0.038--------0.054----0.039--------0.051----0.0340.006----0.055----0.033---1:15055Str0.0590.053----0.040--------0.056----0.038--------0.068----0.0360.002----0.089----0.042---1:15064Mod0.0610.050----0.037--------0.051----0.037--------0.051----0.0320.006----0.053----0.030---1:15064Str0.0600.051----0.038--------0.056----0.042--------0.065----0.0370.001----0.082----0.036---1:150812Mod0.0570.0530.0880.044----0.0950.0510.1090.042--------0.056----0.0430.003----0.067----0.0440.049 1:150812Str0.0580.0590.0940.047----0.1460.0650.1280.045--------0.095----0.0490.003----0.130----0.0690.122 1:1501010Mod0.0530.0490.0780.041----0.0970.056----0.042--------0.060----0.0420.003----0.073----0.0490.056 1:1501010Str0.0600.0540.0980.043----0.1370.060----0.042--------0.112----0.0600.002----0.138----0.0820.132 1:150128Mod0.0490.0500.0790.041----0.0920.052----0.040--------0.060----0.0410.003----0.067----0.0440.045 1:150128Str0.0600.0520.1000.047----0.1440.061----0.045--------0.103----0.0540.003----0.129----0.0740.119 1:1504060Mod0.0510.0510.0760.046----0.0860.0600.0840.049--------0.0690.1300.053--------0.059----0.047---1:1504060Str0.0530.0530.0970.052----0.1300.0710.1140.055--------0.1020.2150.070--------0.078----0.060---1:1505050Mod0.0530.0510.0780.047----0.0890.0580.0850.050--------0.0780.1350.059--------0.054----0.047---1:1505050Str0.0550.0540.0980.051----0.1330.0780.1170.059--------0.1140.2100.076--------0.087----0.067---1:1506040Mod0.0560.0550.0820.052----0.0840.0570.0830.049--------0.0690.1310.051--------0.0610.1710.051---1:1506040Str0.0540.0540.1020.053----0.1330.0750.1160.058--------0.1050.2050.072--------0.0840.2540.063---1:110046Mod0.0580.050----0.038--------0.051----0.0390.015----0.054----0.041--------0.053----0.034---1:110046Str0.0570.048----0.040--------0.055----0.0410.008----0.077----0.042--------0.105----0.046---1:110055Mod0.0600.050----0.040--------0.051----0.0410.016----0.049----0.038--------0.059----0.037---1:110055Str0.0580.051----0.044--------0.054----0.0380.006----0.084----0.041--------0.116----0.052---1:110064Mod0.0600.054----0.044--------0.050----0.0420.015----0.057----0.037--------0.056----0.037---1:110064Str0.0600.052----0.042--------0.053----0.0410.006----0.076----0.042--------0.110----0.049---1:1100812Mod0.0570.0490.0930.044--------0.053----0.0410.006----0.068----0.0490.005----0.088----0.059---1:1100812Str0.0550.0530.1010.047--------0.065----0.0460.002----0.156----0.0740.001----0.214----0.126---1:11001010Mod0.0520.0500.0870.044--------0.055----0.0460.007----0.073----0.0510.006----0.098----0.068---1:11001010Str0.0580.0550.1030.049--------0.073----0.0460.003----0.173----0.0800.001----0.234----0.139---1:1100128Mod0.0510.0480.0890.041--------0.052----0.0440.007----0.067----0.0460.006----0.091----0.063---1:1100128Str0.0590.0530.1050.046--------0.066----0.0470.002----0.154----0.0730.001----0.208----0.124---1:11004060Mod0.0530.0520.0830.050----0.1180.0610.1030.055--------0.0930.2240.0710.006----0.076----0.0610.123 1:11004060Str0.0550.0540.1100.057----0.2130.0940.1410.063--------0.1620.3800.1040.007----0.115----0.0880.258 Population Effect Size 0.8 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5

PAGE 216

205 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 25 (continued). Power estimates for conditions when the population effect size variance is 0.10. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:11005050Mod0.0520.0460.0850.048----0.1240.0610.1010.052--------0.096----0.0720.005----0.078----0.0640.126 1:11005050Str0.0520.0510.1060.053----0.2250.1050.1470.069--------0.180----0.1100.009----0.124----0.0900.264 1:11006040Mod0.0600.0540.0920.055----0.1240.0690.1050.058--------0.0970.2280.0700.007----0.079----0.0620.118 1:11006040Str0.0540.0530.1120.057----0.2200.0980.1480.066--------0.1690.3820.1090.007----0.114----0.0870.254 1:21046Mod----0.0730.049------------0.0660.044----------------0.069------------0.066-----------1:21046Str----0.0730.051------------0.0690.052----------------0.071------------0.078-----------1:21055Mod----0.0700.044------------0.0680.048------------0.0660.067------------0.069-----------1:21055Str----0.0670.051------------0.0700.054------------0.0720.074------------0.076-----------1:21064Mod----0.0660.048------------0.0670.050------------0.0670.073------------0.067-----------1:21064Str----0.0690.044------------0.0720.054------------0.0730.079------------0.065-----------1:210812Mod0.0680.069------------0.0770.072------------0.1010.072----------------0.0690.043-------1:210812Str0.0780.073------------0.0880.071------------0.1380.084----------------0.0870.048-------1:2101010Mod0.0640.067------------0.0760.064----------------0.066----------------0.0670.047-------1:2101010Str0.0740.070------------0.0900.074----------------0.078----------------0.0810.060-------1:210128Mod0.0670.070------------0.0720.062----------------0.0660.034------------0.0660.055-------1:210128Str0.0760.066------------0.0880.069----------------0.0770.043------------0.0730.064-------1:2104060Mod0.0660.064------------0.0690.071------------0.0850.0690.036--------0.0850.0630.035-------1:2104060Str0.0710.072------------0.0830.076------------0.1020.0780.043--------0.0970.0660.040-------1:2105050Mod0.0640.063------------0.0710.0670.034--------0.0830.0610.037------------0.0590.036-------1:2105050Str0.0700.065------------0.0840.0730.040--------0.1030.0720.045------------0.0650.044-------1:2106040Mod0.0660.0660.035--------0.0700.0600.037--------0.0920.0660.041------------0.0580.042-------1:2106040Str0.0700.0690.042--------0.0820.0670.042--------0.1060.0680.050------------0.0630.044-------1:22046Mod0.0730.062----------------0.056----------------0.058----------------0.062-----------1:22046Str0.0770.056----------------0.056----------------0.064----------------0.075-----------1:22055Mod0.0690.055------------0.1000.051----------------0.054----------------0.061-----------1:22055Str0.0710.057------------0.1160.058----------------0.062----------------0.069-----------1:22064Mod0.0700.054----------------0.056----------------0.056----------------0.050-----------1:22064Str0.0740.054----------------0.055----------------0.060----------------0.059-----------1:220812Mod0.0580.0540.0670.032----0.0680.0540.0740.032--------0.0610.113------------0.068----0.028---1:220812Str0.0620.0580.0790.035----0.0930.0650.0860.033--------0.0890.146------------0.100----0.036---1:2201010Mod0.0580.0510.0690.031----0.0750.0550.0800.031--------0.057----0.027--------0.063----0.029---1:2201010Str0.0620.0550.0760.032----0.0970.0670.0910.035--------0.077----0.033--------0.089----0.034---1:220128Mod0.0570.0500.0700.030----0.0770.0570.0790.032--------0.053----0.025--------0.053----0.025---1:220128Str0.0650.0520.0860.032----0.0940.0550.0900.033--------0.062----0.028--------0.069----0.029---Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 217

206 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 25 (continued). Power estimates for conditions when the population effect size variance is 0.10. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:2204060Mod0.0580.0540.0680.041----0.0670.0640.0700.041----0.1050.0630.0810.039--------0.0620.0930.042---1:2204060Str0.0610.0610.0870.045----0.0870.0680.0870.043----0.1480.0820.1100.048--------0.0730.1170.045---1:2205050Mod0.0600.0590.0700.037----0.0670.0560.0760.037----0.1090.0590.0890.039--------0.0590.0970.036---1:2205050Str0.0620.0590.0960.044----0.0910.0700.0960.041----0.1510.0740.1190.044--------0.0580.1140.038---1:2206040Mod0.0560.0570.0750.038----0.0690.0570.0780.036----0.1160.0560.0900.036--------0.0500.1010.033---1:2206040Str0.0590.0570.0960.043----0.0920.0630.0990.037----0.1540.0640.1140.038--------0.0510.1320.030---1:25046Mod0.0630.052----0.038--------0.056----0.040--------0.058----0.0340.007----0.064----0.034---1:25046Str0.0620.056----0.040--------0.060----0.039--------0.075----0.0370.001----0.114----0.042---1:25055Mod0.0640.051----0.037--------0.050----0.035--------0.054----0.0340.007----0.058----0.033---1:25055Str0.0690.055----0.041--------0.053----0.040--------0.066----0.0400.002----0.086----0.038---1:25064Mod0.0570.051----0.040--------0.054----0.039--------0.055----0.0350.007----0.052----0.034---1:25064Str0.0600.052----0.042--------0.052----0.042--------0.061----0.0420.001----0.062----0.032---1:250812Mod0.0540.0510.0910.039----0.0960.0570.1170.044--------0.065----0.0410.003----0.090----0.0530.067 1:250812Str0.0610.0550.1050.047----0.1410.0640.1390.048--------0.127----0.0550.003----0.170----0.0900.135 1:2501010Mod0.0550.0480.0880.040----0.0910.049----0.038--------0.064----0.0390.004----0.066----0.0440.062 1:2501010Str0.0560.0530.1060.044----0.1370.063----0.042--------0.110----0.0550.004----0.139----0.0760.138 1:250128Mod0.0590.0540.0960.046----0.1050.052----0.041--------0.057----0.0400.004----0.056----0.0390.052 1:250128Str0.0570.0500.1040.040----0.1480.056----0.042--------0.081----0.0450.002----0.096----0.0590.113 1:2504060Mod0.0510.0510.0900.050----0.0890.0620.1010.053--------0.0900.1570.070--------0.083----0.0700.082 1:2504060Str0.0540.0550.1210.054----0.1290.0840.1410.064--------0.1360.2510.094--------0.113----0.0930.154 1:2505050Mod0.0520.0520.1020.052----0.0860.0550.1050.049--------0.071----0.056--------0.067----0.0550.083 1:2505050Str0.0570.0540.1300.054----0.1370.0810.1460.059--------0.116----0.078--------0.085----0.0670.151 1:2506040Mod0.0530.0510.1010.049----0.0890.0560.1100.050--------0.055----0.047--------0.050----0.0420.081 1:2506040Str0.0550.0500.1310.051----0.1390.0650.1540.051--------0.092----0.067--------0.060----0.0480.140 1:210046Mod0.0610.048----0.037--------0.050----0.0420.019----0.057----0.035--------0.072----0.033---1:210046Str0.0580.052----0.041--------0.057----0.0370.009----0.107----0.046--------0.175----0.063---1:210055Mod0.0610.050----0.039--------0.054----0.0420.016----0.054----0.040--------0.066----0.037---1:210055Str0.0590.051----0.042--------0.056----0.0390.006----0.092----0.044--------0.134----0.058---1:210064Mod0.0590.051----0.043--------0.052----0.0430.017----0.053----0.045--------0.052----0.041---1:210064Str0.0600.049----0.043--------0.049----0.0400.007----0.065----0.045--------0.068----0.040---1:2100812Mod0.0550.0550.0990.046--------0.052----0.0440.008----0.090----0.0560.010----0.131----0.080---1:2100812Str0.0600.0570.1210.052--------0.081----0.0510.002----0.211----0.0910.003----0.303----0.164---1:21001010Mod0.0530.0490.1000.045--------0.053----0.0420.007----0.072----0.047--------0.100----0.067---1:21001010Str0.0590.0530.1180.046--------0.072----0.0500.003----0.170----0.074--------0.238----0.138---Population Effect Size 0.8 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5

PAGE 218

207 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 25 (continued). Power estimates for conditions when the population effect size variance is 0.10. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:2100128Mod0.0580.050----0.044--------0.057----0.0480.007----0.058----0.0450.009----0.066----0.045---1:2100128Str0.0580.053----0.049--------0.061----0.0460.003----0.118----0.0570.001----0.142----0.085---1:21004060Mod0.0540.0540.1000.054----0.1250.0710.1250.0590.005----0.128----0.0910.010----0.121----0.0970.149 1:21004060Str0.0530.0510.1350.055----0.2190.1160.1800.0760.002----0.230----0.1490.010----0.183----0.1380.300 1:21005050Mod0.0540.0510.1120.053----0.1310.0640.1280.0540.005----0.098----0.0740.012----0.083----0.0680.153 1:21005050Str0.0560.0550.1380.058----0.2290.1020.1820.0710.002----0.188----0.1200.010----0.131----0.1040.287 1:21006040Mod0.0540.0510.1120.051----0.1310.056----0.0540.007----0.071----0.0550.009----0.058----0.0490.130 1:21006040Str0.0560.0520.1400.059----0.2370.091----0.0660.002----0.120----0.0750.009----0.073----0.0560.271 1:41046Mod----0.0660.058------------0.0710.066------------0.0730.086------------0.080-----------1:41046Str----0.0680.063------------0.0750.066------------0.0770.091------------0.086-----------1:41055Mod----0.0730.060------------0.0680.065------------0.067----------------0.071-----------1:41055Str----0.0690.063------------0.0710.070------------0.071----------------0.079-----------1:41064Mod----0.0650.060------------0.0670.067------------0.069----------------0.062-----------1:41064Str----0.0680.064------------0.0670.071------------0.068----------------0.065-----------1:410812Mod0.0730.072------------0.0780.0690.039--------0.1080.0760.047------------0.0760.059-------1:410812Str0.0690.067------------0.0890.0740.045--------0.1430.0900.051------------0.1000.077-------1:4101010Mod0.0720.0690.043--------0.0790.0690.044------------0.0660.048------------0.0700.075-------1:4101010Str0.0740.0700.051--------0.0910.0720.047------------0.0790.062------------0.0790.093-------1:410128Mod0.0720.0660.044--------0.0790.0620.044------------0.0610.058------------0.0610.087-------1:410128Str0.0760.0650.046--------0.0930.0700.051------------0.0690.072------------0.0650.112-------1:4104060Mod0.0620.0670.061--------0.0710.0680.061--------0.0840.0700.067--------0.0900.0710.069-------1:4104060Str0.0710.0730.069--------0.0800.0790.074--------0.1050.0820.075--------0.1050.0740.081-------1:4105050Mod0.0660.0650.065--------0.0730.0630.068--------0.0870.0640.071------------0.0620.078-------1:4105050Str0.0680.0680.072--------0.0850.0730.079--------0.1120.0740.083------------0.0620.086-------1:4106040Mod0.0710.0640.069--------0.0740.0640.068--------0.0990.0660.075------------0.0630.081-------1:4106040Str0.0770.0710.073--------0.0910.0690.083--------0.1170.0620.093------------0.0620.088-------1:42046Mod0.0670.056----------------0.057----------------0.063----------------0.067-----------1:42046Str0.0740.061----------------0.063----------------0.075----------------0.093-----------1:42055Mod0.0710.057----------------0.052----------------0.057----------------0.061-----------1:42055Str0.0740.060----------------0.056----------------0.068----------------0.071-----------1:42064Mod0.0690.052----------------0.056----0.031--------0.054----0.024--------0.051----0.024---1:42064Str0.0720.055----------------0.058----0.033--------0.060----0.027--------0.059----0.023---1:420812Mod0.0600.0560.090--------0.0730.0600.1000.032--------0.073----0.032--------0.080----0.034---1:420812Str0.0670.0600.110--------0.0990.0680.1170.034--------0.099----0.039--------0.126----0.045---Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 219

208 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 25 (continued). Power estimates for conditions when the population effect size variance is 0.10. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:4201010Mod0.0640.0590.1000.032----0.0750.0540.0990.030--------0.060----0.028--------0.072-----------1:4201010Str0.0650.0580.1090.035----0.1060.0600.1220.034--------0.080----0.031--------0.096-----------1:420128Mod0.0610.051----0.031----0.0830.054----------------0.052----0.027--------0.054----0.025---1:420128Str0.0600.055----0.034----0.1030.061----------------0.060----0.031--------0.064----0.033---1:4204060Mod0.0600.0620.1160.047----0.0710.0620.1150.044----0.1070.073----0.049--------0.071----0.052---1:4204060Str0.0580.0580.1460.046----0.0900.0730.1500.052----0.1570.104----0.065--------0.090----0.063---1:4205050Mod0.0550.054----0.041----0.0720.058----0.040----0.1210.066----0.044--------0.059----0.038---1:4205050Str0.0630.059----0.049----0.0950.070----0.052----0.1570.076----0.051--------0.065----0.043---1:4206040Mod0.0570.052----0.040----0.0790.056----0.038----0.1260.053----0.037--------0.052----0.032---1:4206040Str0.0650.053----0.040----0.0920.059----0.042----0.1740.062----0.038--------0.049----0.029---1:45046Mod0.0620.052----0.037--------0.053----0.0330.010----0.065----0.0320.011----0.083----0.035---1:45046Str0.0600.055----0.040--------0.061----0.0370.004----0.097----0.0390.004----0.155----0.053---1:45055Mod0.0640.057----0.037--------0.052----0.0350.009----0.057----0.0330.013----0.053----0.029---1:45055Str0.0640.055----0.037--------0.055----0.0380.005----0.073----0.0360.002----0.094----0.041---1:45064Mod0.0610.055----0.041--------0.048----0.038--------0.049----0.0340.011----0.050----0.034---1:45064Str0.0600.049----0.041--------0.051----0.043--------0.053----0.0360.003----0.058----0.036---1:450812Mod0.0600.057----0.041----0.0890.055----0.044--------0.089----0.0510.007----0.123----0.066---1:450812Str0.0590.058----0.049----0.1480.076----0.047--------0.162----0.0730.006----0.232----0.119---1:4501010Mod0.0560.052----0.040----0.1070.055----0.040--------0.069----0.0440.008----0.079----0.049---1:4501010Str0.0630.058----0.049----0.1520.063----0.043--------0.118----0.0590.004----0.158----0.084---1:450128Mod0.0600.048----0.042--------0.048----0.038--------0.050----0.0370.006----0.053----0.037---1:450128Str0.0620.056----0.047--------0.050----0.040--------0.075----0.0410.003----0.082----0.050---1:4504060Mod0.0550.057----0.056----0.0880.064----0.058--------0.105----0.0880.023----0.105----0.0930.116 1:4504060Str0.0590.056----0.065----0.1370.093----0.072--------0.167----0.1230.051----0.150----0.1300.199 1:4505050Mod0.0550.050----0.054----0.0920.057----0.053--------0.077----0.0620.021----0.077----0.0650.115 1:4505050Str0.0550.055----0.059----0.1430.082----0.066--------0.122----0.0870.043----0.095----0.0790.199 1:4506040Mod0.0530.051----0.052----0.0960.053----0.051--------0.058----0.0510.019----0.051----0.0490.114 1:4506040Str0.0560.051----0.057----0.1380.065----0.054--------0.071----0.0580.034----0.053----0.0500.187 1:410046Mod0.0590.053----0.038--------0.057----0.0390.021----0.069----0.034--------0.112----0.043---1:410046Str0.0630.056----0.042--------0.069----0.0410.009----0.145----0.048--------0.266----0.088---1:410055Mod0.0620.057----0.044--------0.050----0.0370.021----0.057----0.037--------0.064----0.038---1:410055Str0.0600.049----0.040--------0.061----0.0410.009----0.085----0.041--------0.136----0.059---1:410064Mod0.0580.048----0.037--------0.052----0.0450.022----0.056----0.046----------------0.043---1:410064Str0.0620.053----0.046--------0.050----0.0440.009----0.051----0.037----------------0.037---Population Effect Size 0.8 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5

PAGE 220

209 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 25 (continued). Power estimates for conditions when the population effect size variance is 0.10. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:4100812Mod0.0610.058----0.049--------0.060----0.0470.016----0.126----0.069--------0.192----0.107---1:4100812Str0.0620.055----0.050--------0.093----0.0520.005----0.279----0.110--------0.411----0.224---1:41001010Mod0.0560.053----0.043--------0.055----0.0450.015----0.081----0.050--------0.110----0.066---1:41001010Str0.0630.054----0.051--------0.078----0.0470.006----0.196----0.074--------0.263----0.138---1:4100128Mod0.0600.051----0.044--------0.047----0.0410.013----0.054----0.038--------0.058----0.041---1:4100128Str0.0630.053----0.049--------0.063----0.0480.008----0.098----0.049--------0.121----0.065---1:41004060Mod0.0530.055----0.061----0.1220.077----0.0660.010----0.166----0.1300.028-------------------1:41004060Str0.0550.053----0.065----0.2140.122----0.0860.005----0.287----0.1980.024-------------------1:41005050Mod0.0530.049----0.056----0.1360.064----0.0610.014----0.106----0.0840.025----0.093----0.079---1:41005050Str0.0560.051----0.064----0.2360.107----0.0790.005----0.187----0.1310.024----0.139----0.117---1:41006040Mod0.0520.048----0.055--------0.055----0.0570.012----0.059----0.0520.023----0.049----0.048---1:41006040Str0.0550.055----0.062--------0.082----0.0690.005----0.099----0.0680.020----0.057----0.048---1:81046Mod----0.0770.077------------0.0730.083------------0.077----------------0.082-----------1:81046Str----0.0730.089------------0.0800.092------------0.086----------------0.095-----------1:81055Mod----0.0690.083------------0.0700.096------------0.070----------------0.070-----------1:81055Str----0.0690.085------------0.0710.094------------0.074----------------0.078-----------1:81064Mod----0.069----------------0.067----------------0.068----------------0.065-----------1:81064Str----0.072----------------0.065----------------0.066----------------0.070-----------1:810812Mod0.0760.0690.062--------0.0800.0700.059------------0.0800.068------------0.0820.093-------1:810812Str0.0770.0740.069--------0.0970.0800.073------------0.0990.089------------0.1060.117-------1:8101010Mod0.0710.0650.062--------0.0820.0660.073------------0.0710.087------------0.076-----------1:8101010Str0.0810.0700.077--------0.1000.0710.079------------0.0790.106------------0.087-----------1:810128Mod0.0740.0620.069------------0.0630.074------------0.067----------------0.065-----------1:810128Str0.0810.0680.077------------0.0680.083------------0.069----------------0.069-----------1:8104060Mod0.0690.0690.105--------0.0740.0740.096--------0.0870.0760.108------------0.079-----------1:8104060Str0.0720.0730.122--------0.0850.0800.121--------0.1080.0870.123------------0.084-----------1:8105050Mod0.0680.061------------0.0760.066------------0.0990.072----------------0.067-----------1:8105050Str0.0730.071------------0.0910.077------------0.1190.076----------------0.069-----------1:8106040Mod0.0710.067------------0.0770.063----------------0.063----------------0.064-----------1:8106040Str0.0800.067------------0.0910.066----------------0.065----------------0.062-----------1:82046Mod----0.058----------------0.058----------------0.064----------------0.079-----------1:82046Str----0.059----------------0.062----------------0.085----------------0.101-----------1:82055Mod0.0700.054----------------0.056----------------0.058----------------0.058-----------1:82055Str0.0750.058----------------0.054----------------0.066----------------0.073-----------Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 221

210 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 25 (continued). Power estimates for conditions when the population effect size variance is 0.10. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:82064Mod0.0700.052----------------0.055----0.031--------0.055----0.028--------0.052----0.026---1:82064Str0.0730.056----------------0.055----0.033--------0.058----0.032--------0.053----0.023---1:820812Mod0.0650.060------------0.0760.058----------------0.075----------------0.091----0.034---1:820812Str0.0720.063------------0.1020.071----------------0.109----------------0.135----0.049---1:8201010Mod0.0630.056------------0.0870.055----0.030--------0.062----------------0.063----0.025---1:8201010Str0.0690.057------------0.1090.064----0.037--------0.080----------------0.100----0.039---1:820128Mod0.0640.052------------0.0940.056----0.031--------0.050----0.029--------0.053----0.028---1:820128Str0.0720.057------------0.1120.059----0.038--------0.064----0.029--------0.063----0.028---1:8204060Mod0.0570.059----0.053----0.0710.063----0.050----0.1140.081----0.061--------0.080----0.066---1:8204060Str0.0600.061----0.056----0.0860.074----0.062----0.1610.105----0.075--------0.097----0.079---1:8205050Mod0.0640.058----0.049----0.0780.061----0.050--------0.067----0.051--------0.063----0.047---1:8205050Str0.0610.057----0.053----0.0960.068----0.058--------0.084----0.063--------0.070----0.054---1:8206040Mod0.0620.057----0.046----0.0800.056----0.044--------0.052----0.038--------0.050----0.038---1:8206040Str0.0620.052----0.052----0.0980.059----0.049--------0.060----0.047--------0.054----0.040---1:85046Mod0.0630.052----0.034--------0.055----0.0330.012----0.073----0.033--------0.108----0.039---1:85046Str0.0630.054----0.033--------0.066----0.0390.005----0.117----0.041--------0.192----0.060---1:85055Mod0.0630.048----0.032--------0.052----0.0370.012----0.054----0.029--------0.058----0.029---1:85055Str0.0610.055----0.039--------0.056----0.0380.006----0.073----0.035--------0.100----0.041---1:85064Mod0.0610.051----0.041--------0.054----0.0420.012----0.050----0.039--------0.053----0.037---1:85064Str0.0660.053----0.043--------0.050----0.0430.005----0.051----0.038--------0.051----0.035---1:850812Mod0.0540.052----0.043----0.1000.059----0.0440.006----0.094----0.0520.015----0.144----0.074---1:850812Str0.0620.061----0.052----0.1550.081----0.0510.003----0.195----0.0810.011----0.275----0.140---1:8501010Mod0.0590.050----0.040--------0.055----0.0400.009----0.073----0.0440.016----0.088----0.053---1:8501010Str0.0610.055----0.046--------0.064----0.0450.004----0.134----0.0600.009----0.163----0.087---1:850128Mod0.0620.049----0.039--------0.050----0.040--------0.054----0.0420.012----0.054----0.035---1:850128Str0.0610.054----0.048--------0.059----0.047--------0.074----0.0420.005----0.076----0.043---1:8504060Mod0.0540.054----0.062----0.0990.077----0.078--------0.123----0.1110.030----------------0.153 1:8504060Str0.0580.054----0.073----0.1410.100----0.087--------0.191----0.1520.055----------------0.231 1:8505050Mod0.0570.053----0.060----0.1010.064----0.064--------0.083----0.0790.026----0.075----0.075---1:8505050Str0.0570.055----0.068----0.1470.087----0.080--------0.126----0.0990.048----0.101----0.097---1:8506040Mod0.0520.048----0.052----0.1080.051----0.058--------0.055----0.0540.025----0.050----0.047---1:8506040Str0.0580.051----0.066----0.1590.064----0.066--------0.071----0.0650.035----0.048----0.050---1:810046Mod0.0580.052----0.037--------0.057----0.0370.025----0.093----0.040--------0.155----0.055---1:810046Str0.0590.053----0.039--------0.070----0.0370.016----0.178----0.048--------0.337----0.113---Population Effect Size 0.8 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5

PAGE 222

211 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 25 (continued). Power estimates for conditions when the population effect size variance is 0.10. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:810055Mod0.0590.052----0.039--------0.053----0.0380.029----0.057----0.034--------0.069----0.036---1:810055Str0.0550.048----0.040--------0.060----0.0410.015----0.098----0.042--------0.142----0.059---1:810064Mod0.0570.049----0.043--------0.051----0.0410.025----0.054----0.043-----------------------1:810064Str0.0580.049----0.043--------0.050----0.0470.015----0.050----0.040-----------------------1:8100812Mod0.0570.053----0.046--------0.065----0.0460.018----0.147----0.074--------0.253----0.137---1:8100812Str0.0550.054----0.050--------0.109----0.0570.010----0.336----0.123--------0.483----0.270---1:81001010Mod0.0570.051----0.044--------0.056----0.0440.020----0.091----0.050--------0.123----0.068---1:81001010Str0.0580.054----0.051--------0.080----0.0500.012----0.214----0.084--------0.302----0.150---1:8100128Mod0.0550.051----0.046--------0.049----0.0420.018----0.055----0.041--------0.055----0.040---1:8100128Str0.0570.048----0.046--------0.063----0.0520.012----0.092----0.050--------0.101----0.054---1:81004060Mod0.0570.053----0.070----0.1230.083----0.0840.017----0.199-------------------------------1:81004060Str0.0540.051----0.072----0.2270.146----0.1140.015----0.338-------------------------------1:81005050Mod0.0550.054----0.067--------0.068----0.0730.021----0.118----0.106--------0.102----0.110---1:81005050Str0.0580.054----0.074--------0.108----0.0910.016----0.211----0.163--------0.154----0.150---1:81006040Mod0.0530.049----0.059--------0.057----0.0660.024----0.058----0.059----------------0.053---1:81006040Str0.0540.051----0.069--------0.074----0.0730.012----0.094----0.078----------------0.060---Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 223

212 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 26. Power estimates for conditions when the population effect size variance is 0.33. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:11046Mod----0.0680.054------------0.0700.061------------0.0710.088------------0.068-----------1:11046Str----0.0650.055------------0.0700.058------------0.0700.082------------0.069-----------1:11055Mod----0.0660.052------------0.0670.061------------0.0740.085------------0.068-----------1:11055Str----0.0670.053------------0.0680.064------------0.0720.082------------0.071-----------1:11064Mod----0.0670.052------------0.0680.059------------0.0680.084------------0.072-----------1:11064Str----0.0670.054------------0.0710.061------------0.0730.087------------0.070-----------1:110812Mod0.0690.066------------0.0710.067----------------0.0650.036------------0.0620.056-------1:110812Str0.0670.069------------0.0810.069----------------0.0700.041------------0.0730.063-------1:1101010Mod0.0670.064------------0.0730.0630.028------------0.0690.038------------0.0750.059-------1:1101010Str0.0710.069------------0.0840.0690.035------------0.0700.041------------0.0730.064-------1:110128Mod0.0650.0620.029--------0.0720.0660.029------------0.0660.038------------0.0660.056-------1:110128Str0.0710.0680.033--------0.0840.0680.029------------0.0710.043------------0.0730.061-------1:1104060Mod0.0600.0610.028--------0.0630.0610.028------------0.0640.032------------0.0630.039-------1:1104060Str0.0710.0720.032--------0.0770.0690.033------------0.0670.034------------0.0650.039-------1:1105050Mod0.0630.070------------0.0680.0650.028--------0.0900.0660.030------------0.0670.039-------1:1105050Str0.0650.064------------0.0710.0660.030--------0.1050.0690.034------------0.0650.044-------1:1106040Mod0.0590.062------------0.0670.0640.028--------0.0910.064----------------0.0620.038-------1:1106040Str0.0660.066------------0.0710.0640.030--------0.1030.069----------------0.0590.042-------1:12046Mod0.0690.055----0.031--------0.053----0.027--------0.056----0.032--------0.056----0.027---1:12046Str0.0690.059----0.033--------0.056----0.027--------0.059----0.029--------0.065----0.026---1:12055Mod0.0700.056----0.028--------0.055----0.029--------0.060----0.031--------0.058----0.028---1:12055Str0.0700.056----0.031--------0.059----0.030--------0.059----0.028--------0.059----0.026---1:12064Mod0.0680.058----0.029--------0.051----------------0.058----0.031--------0.056----0.024---1:12064Str0.0700.057----0.032--------0.058----------------0.056----0.029--------0.059----0.025---1:120812Mod0.0570.0540.0760.034----0.0750.0560.0850.034--------0.059----0.032--------0.056----0.031---1:120812Str0.0610.0580.0810.040----0.0870.0570.0910.035--------0.064----0.035--------0.066----0.032---1:1201010Mod0.0550.0530.0740.033----0.0760.0530.0800.030--------0.058----0.033--------0.054----0.027---1:1201010Str0.0610.0570.0840.037----0.0940.0610.0950.038--------0.062----0.036--------0.066----0.034---1:120128Mod0.0610.0570.0760.034----0.0780.0610.0830.038--------0.057----0.032--------0.058----0.031---1:120128Str0.0570.0530.0780.036----0.0860.0530.0910.036--------0.064----0.035--------0.066----0.031---1:1204060Mod0.0550.0590.0560.042----0.0690.0580.0590.040--------0.0550.0700.040--------0.0550.0860.034---1:1204060Str0.0530.0530.0640.043----0.0770.0600.0720.042--------0.0590.0820.041--------0.0590.1100.042---1:1205050Mod0.0540.0520.0580.041----0.0680.0570.0620.042--------0.0560.0740.036--------0.0570.0860.036---1:1205050Str0.0600.0630.0700.048----0.0780.0580.0700.042--------0.0640.0840.042--------0.0610.1130.040---Population Effect Size 0.8 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5

PAGE 224

213 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 26 (continued). Power estimates for conditions when the population effect size variance is 0.33. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:1206040Mod0.0540.0570.0580.042----0.0690.0580.0590.041--------0.0580.0690.041--------0.0530.0880.034---1:1206040Str0.0580.0540.0650.043----0.0760.0580.0690.043--------0.0610.0870.042--------0.0630.1080.041---1:15046Mod0.0600.051----0.040--------0.049----0.040--------0.052----0.0390.006----0.051----0.040---1:15046Str0.0640.050----0.044--------0.054----0.044--------0.058----0.0410.001----0.065----0.044---1:15055Mod0.0600.053----0.044--------0.050----0.039--------0.052----0.0390.005----0.053----0.042---1:15055Str0.0660.054----0.046--------0.058----0.046--------0.058----0.0390.001----0.066----0.042---1:15064Mod0.0640.050----0.041--------0.051----0.041--------0.051----0.0390.007----0.054----0.038---1:15064Str0.0620.051----0.042--------0.052----0.043--------0.057----0.0420.001----0.063----0.042---1:150812Mod0.0570.048----0.042----0.1000.052----0.046--------0.053----0.044--------0.060----0.0500.008 1:150812Str0.0560.053----0.050----0.1310.055----0.047--------0.075----0.050--------0.087----0.0590.015 1:1501010Mod0.0610.056----0.049----0.0990.051----0.042--------0.057----0.045--------0.057----0.0440.010 1:1501010Str0.0590.054----0.050----0.1360.058----0.049--------0.082----0.055--------0.093----0.0610.016 1:150128Mod0.0510.048----0.042----0.1020.053----0.049--------0.054----0.045--------0.055----0.0440.007 1:150128Str0.0570.051----0.047----0.1320.060----0.051--------0.068----0.049--------0.086----0.0550.015 1:1504060Mod0.0550.0520.0780.050----0.0920.0510.0860.050--------0.0580.1210.052--------0.057----0.051---1:1504060Str0.0570.0550.0980.057----0.1180.0610.1150.059--------0.0750.1720.061--------0.073----0.059---1:1505050Mod0.0530.0540.0750.059----0.0940.0540.0880.055--------0.059----0.052--------0.057----0.052---1:1505050Str0.0510.0490.0970.054----0.1150.0620.1150.060--------0.072----0.062--------0.075----0.061---1:1506040Mod0.0530.0530.0810.053----0.0980.0560.0860.054--------0.0560.1260.052--------0.053----0.051---1:1506040Str0.0500.0520.0930.060----0.1100.0570.1120.056--------0.0720.1590.059--------0.078----0.063---1:110046Mod0.0590.053----0.045--------0.053----0.0440.016----0.052----0.045--------0.050----0.041---1:110046Str0.0590.052----0.046--------0.048----0.0450.011----0.062----0.045--------0.075----0.045---1:110055Mod0.0600.051----0.044--------0.051----0.0470.017----0.053----0.045--------0.053----0.039---1:110055Str0.0580.046----0.043--------0.053----0.0470.010----0.066----0.048--------0.084----0.049---1:110064Mod0.0590.050----0.045--------0.051----0.0460.016----0.052----0.046--------0.049----0.042---1:110064Str0.0580.048----0.043--------0.056----0.0480.010----0.062----0.046--------0.077----0.049---1:1100812Mod0.0550.051----0.049--------0.049----0.0480.010----0.058----0.0480.006----0.064----0.049---1:1100812Str0.0540.048----0.046--------0.061----0.0510.005----0.097----0.0620.001----0.120----0.070---1:11001010Mod0.0570.051----0.050--------0.050----0.0470.011----0.062----0.0490.006----0.072----0.055---1:11001010Str0.0540.052----0.053--------0.056----0.0480.004----0.098----0.0580.001----0.131----0.081---1:1100128Mod0.0570.052----0.052--------0.051----0.0480.010----0.056----0.0480.007----0.067----0.050---1:1100128Str0.0580.053----0.054--------0.060----0.0550.005----0.093----0.0600.001----0.125----0.075---1:11004060Mod0.0540.0510.0830.059--------0.0530.1040.058--------0.064----0.059--------0.066----0.0580.005 1:11004060Str0.0520.0480.1120.056--------0.0630.1380.065--------0.095----0.075--------0.097----0.0740.004 Population Effect Size 0.8 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5

PAGE 225

214 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 26 (continued). Power estimates for conditions when the population effect size variance is 0.33. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:11005050Mod0.0570.0550.0870.059--------0.0580.1050.062--------0.072----0.063--------0.067----0.0600.006 1:11005050Str0.0540.0510.1090.060--------0.0680.1370.064--------0.103----0.078--------0.100----0.0780.005 1:11006040Mod0.0520.0520.0880.055--------0.0590.1080.057--------0.068----0.064--------0.068----0.0620.006 1:11006040Str0.0520.0510.1160.060--------0.0690.1340.065--------0.102----0.078--------0.096----0.0750.005 1:21046Mod----0.0680.057------------0.0670.064------------0.0690.088------------0.065-----------1:21046Str----0.0710.058------------0.0760.064------------0.0740.090------------0.079-----------1:21055Mod----0.0660.057------------0.0660.063------------0.069----------------0.072-----------1:21055Str----0.0690.062------------0.0650.060------------0.075----------------0.075-----------1:21064Mod----0.0690.066------------0.0710.070------------0.069----------------0.073-----------1:21064Str----0.0720.064------------0.0650.064------------0.067----------------0.069-----------1:210812Mod0.0670.0680.032--------0.0760.0720.034------------0.0670.042------------0.0710.058-------1:210812Str0.0730.0720.034--------0.0890.0700.036------------0.0720.042------------0.0810.067-------1:2101010Mod0.0700.0650.036--------0.0800.0660.036------------0.0630.043------------0.0650.065-------1:2101010Str0.0730.0730.040--------0.0860.0760.038------------0.0710.051------------0.0790.081-------1:210128Mod0.0660.0610.033--------0.0800.0650.038------------0.0640.045------------0.0630.070-------1:210128Str0.0760.0650.037--------0.0870.0680.038------------0.0650.051------------0.0710.076-------1:2104060Mod0.0660.0660.038--------0.0680.0660.030--------0.0950.0680.041------------0.0640.046-------1:2104060Str0.0660.0650.038--------0.0760.0700.038--------0.1060.0650.047------------0.0680.053-------1:2105050Mod0.0660.0660.035--------0.0690.0660.037--------0.0980.0660.046------------0.0640.049-------1:2105050Str0.0670.0710.037--------0.0780.0680.041--------0.1110.0690.049------------0.0650.056-------1:2106040Mod0.0660.0650.037--------0.0730.0650.040------------0.0610.041------------0.0620.056-------1:2106040Str0.0660.0670.039--------0.0750.0650.046------------0.0610.049------------0.0620.058-------1:22046Mod0.0660.057----0.030--------0.055----0.027--------0.059----0.030--------0.057----0.028---1:22046Str0.0720.059----0.035--------0.057----0.033--------0.062----0.029--------0.071----0.027---1:22055Mod0.0720.055----0.031--------0.054----0.033--------0.058----0.029--------0.054----0.027---1:22055Str0.0670.054----0.034--------0.059----0.031--------0.061----0.029--------0.062----0.027---1:22064Mod0.0690.057----0.032--------0.053----0.031--------0.057----0.030--------0.055----0.028---1:22064Str0.0660.058----0.035--------0.058----0.035--------0.055----0.031--------0.057----0.030---1:220812Mod0.0600.0540.0830.033----0.0770.0590.0930.037--------0.059----0.032--------0.061----0.034---1:220812Str0.0570.0560.0910.038----0.0860.0540.0950.036--------0.071----0.040--------0.078----0.039---1:2201010Mod0.0590.0520.0870.034----0.0750.055----0.034--------0.056----0.030--------0.060----0.033---1:2201010Str0.0650.0630.0940.040----0.0930.058----0.038--------0.064----0.034--------0.068----0.035---1:220128Mod0.0580.0510.0810.033----0.0760.055----0.032--------0.057----0.035--------0.056----0.031---1:220128Str0.0630.0550.0890.037----0.0910.054----0.037--------0.059----0.032--------0.062----0.035---Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 226

215 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 26 (continued). Power estimates for conditions when the population effect size variance is 0.33. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:2204060Mod0.0510.0530.0660.042----0.0640.0590.0680.044--------0.0630.0840.044--------0.061----0.044---1:2204060Str0.0560.0520.0820.044----0.0770.0580.0820.044--------0.0690.1000.047--------0.068----0.046---1:2205050Mod0.0580.0520.0700.041----0.0690.0560.0740.041--------0.0600.0890.041--------0.057----0.039---1:2205050Str0.0550.0520.0830.042----0.0830.0580.0850.045--------0.0620.1020.044--------0.056----0.038---1:2206040Mod0.0540.0540.0750.044----0.0690.0560.0850.041--------0.0530.0960.036--------0.055----0.037---1:2206040Str0.0600.0570.0870.044----0.0820.0600.0890.044--------0.0600.1160.040--------0.054----0.034---1:25046Mod0.0610.050----0.039--------0.050----0.040--------0.052----0.0380.007----0.057----0.039---1:25046Str0.0540.052----0.040--------0.056----0.043--------0.064----0.0420.002----0.076----0.044---1:25055Mod0.0680.056----0.046--------0.051----0.042--------0.053----0.0420.008----0.050----0.036---1:25055Str0.0590.054----0.043--------0.054----0.042--------0.058----0.0400.002----0.063----0.041---1:25064Mod0.0620.051----0.043--------0.049----0.042--------0.051----0.0390.007----0.048----0.038---1:25064Str0.0620.052----0.048--------0.053----0.046--------0.053----0.0430.002----0.054----0.036---1:250812Mod0.0570.054----0.048----0.0980.053----0.045--------0.061----0.0470.002----0.068----0.0490.009 1:250812Str0.0580.055----0.050----0.1330.058----0.048--------0.086----0.0550.001----0.108----0.0640.017 1:2501010Mod0.0540.047----0.045----0.1040.053----0.045--------0.054----0.0430.003----0.059----0.0460.011 1:2501010Str0.0560.050----0.047----0.1420.057----0.052--------0.077----0.0520.000----0.098----0.0600.015 1:250128Mod0.0610.056----0.049--------0.054----0.046--------0.051----0.0450.003----0.051----0.0400.009 1:250128Str0.0550.050----0.049--------0.055----0.049--------0.068----0.0490.000----0.070----0.0500.013 1:2504060Mod0.0490.0510.1010.054----0.0890.0540.1040.056--------0.061----0.058--------0.068----0.060---1:2504060Str0.0540.0540.1180.056----0.1150.0610.1270.059--------0.085----0.067--------0.086----0.071---1:2505050Mod0.0550.0560.1040.054----0.0930.0540.1120.054--------0.058----0.052--------0.062----0.056---1:2505050Str0.0540.0530.1180.055----0.1150.0590.1360.057--------0.076----0.067--------0.077----0.067---1:2506040Mod0.0510.0520.1020.051----0.0970.0540.1120.057--------0.056----0.050--------0.051----0.049---1:2506040Str0.0530.0490.1160.053----0.1180.0570.1330.059--------0.066----0.058--------0.062----0.057---1:210046Mod0.0620.052----0.044--------0.053----0.0430.018----0.057----0.044--------0.061----0.042---1:210046Str0.0610.049----0.044--------0.056----0.0460.010----0.075----0.046--------0.108----0.054---1:210055Mod0.0560.052----0.043--------0.051----0.0420.021----0.050----0.044--------0.052----0.043---1:210055Str0.0600.056----0.050--------0.052----0.0430.010----0.067----0.045--------0.083----0.048---1:210064Mod0.0550.049----0.044--------0.056----0.0480.018----0.049----0.043--------0.053----0.046---1:210064Str0.0610.052----0.050--------0.053----0.0480.011----0.055----0.045--------0.054----0.040---1:2100812Mod0.0600.051----0.050--------0.054----0.0480.010----0.065----0.0530.009----0.089----0.059---1:2100812Str0.0560.053----0.054--------0.066----0.0540.005----0.122----0.0690.001----0.176----0.094---1:21001010Mod0.0580.053----0.052--------0.051----0.0490.011----0.062----0.0500.008----0.069----0.052---1:21001010Str0.0540.051----0.051--------0.060----0.0540.006----0.108----0.0620.002----0.137----0.078---Population Effect Size 0.8 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5

PAGE 227

216 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 26 (continued). Power estimates for conditions when the population effect size variance is 0.33. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:2100128Mod0.0540.052----0.052--------0.052----0.0500.011----0.054----0.0460.008----0.053----0.047---1:2100128Str0.0560.049----0.047--------0.056----0.0540.006----0.079----0.0530.002----0.090----0.056---1:21004060Mod0.0560.0560.1070.059--------0.060----0.064--------0.079----0.0730.004----0.093----0.0780.009 1:21004060Str0.0560.0520.1300.063--------0.066----0.067--------0.125----0.0950.001----0.138----0.1100.006 1:21005050Mod0.0490.0490.1070.055--------0.057----0.060--------0.071----0.0650.005----0.070----0.0620.008 1:21005050Str0.0560.0500.1350.060--------0.066----0.067--------0.104----0.0810.002----0.108----0.0880.006 1:21006040Mod0.0540.0480.1040.057--------0.057----0.061--------0.058----0.0560.005----0.057----0.0570.009 1:21006040Str0.0530.0530.1290.063--------0.060----0.064--------0.080----0.0680.002----0.073----0.0660.005 1:41046Mod----0.0710.072------------0.0690.082------------0.070----------------0.069-----------1:41046Str----0.0690.072------------0.0680.079------------0.073----------------0.080-----------1:41055Mod----0.0670.074------------0.0650.084------------0.069----------------0.068-----------1:41055Str----0.0700.075------------0.0710.084------------0.068----------------0.068-----------1:41064Mod----0.0660.078------------0.066----------------0.065----------------0.070-----------1:41064Str----0.0720.078------------0.068----------------0.070----------------0.066-----------1:410812Mod0.0670.0640.047--------0.0780.0690.047------------0.0690.061------------0.0730.081-------1:410812Str0.0750.0730.054--------0.0840.0680.055------------0.0780.064------------0.0850.093-------1:4101010Mod0.0690.0660.053--------0.0770.0650.050------------0.0660.066------------0.066-----------1:4101010Str0.0720.0680.050--------0.0860.0680.055------------0.0750.077------------0.075-----------1:410128Mod0.0710.0650.055--------0.0830.0610.056------------0.0640.071------------0.064-----------1:410128Str0.0710.0650.052--------0.0940.0640.058------------0.0650.075------------0.067-----------1:4104060Mod0.0630.0630.059--------0.0670.0650.058--------0.0890.0660.066------------0.0650.083-------1:4104060Str0.0710.0720.069--------0.0750.0700.068--------0.1050.0760.072------------0.0740.092-------1:4105050Mod0.0650.0660.059--------0.0740.0640.067------------0.0650.069------------0.0630.089-------1:4105050Str0.0670.0650.064--------0.0810.0690.066------------0.0690.080------------0.0700.102-------1:4106040Mod0.0640.0650.066--------0.0740.0610.066------------0.0600.073------------0.063-----------1:4106040Str0.0640.0650.069--------0.0830.0690.070------------0.0640.081------------0.062-----------1:42046Mod0.0700.061----0.031--------0.058----------------0.060----------------0.062-----------1:42046Str0.0710.054----0.028--------0.066----------------0.065----------------0.070-----------1:42055Mod0.0690.056----0.032--------0.059----0.028--------0.057----0.028--------0.052----0.026---1:42055Str0.0690.053----0.031--------0.057----0.034--------0.059----0.028--------0.065----0.027---1:42064Mod0.0760.056----0.034--------0.059----0.036--------0.055----0.029--------0.054----0.029---1:42064Str0.0760.054----0.035--------0.058----0.037--------0.053----0.030--------0.056----0.026---1:420812Mod0.0590.057----0.036----0.0750.059----0.036--------0.062----0.035--------0.066----0.035---1:420812Str0.0610.057----0.037----0.0940.064----0.041--------0.074----0.041--------0.087----0.042---Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 228

217 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 26 (continued). Power estimates for conditions when the population effect size variance is 0.33. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:4201010Mod0.0620.053----0.035----0.0810.054----0.033--------0.057----0.034--------0.064----0.035---1:4201010Str0.0650.059----0.042----0.0980.061----0.036--------0.064----0.037--------0.073----0.036---1:420128Mod0.0590.051----0.036----0.0850.055----0.038--------0.051----0.031--------0.052----0.030---1:420128Str0.0620.057----0.039----0.0960.054----0.035--------0.057----0.037--------0.058----0.031---1:4204060Mod0.0580.060----0.049----0.0670.060----0.048--------0.064----0.050--------0.064----0.049---1:4204060Str0.0560.058----0.051----0.0780.057----0.051--------0.069----0.055--------0.077----0.057---1:4205050Mod0.0550.055----0.049----0.0710.057----0.047--------0.055----0.045--------0.059----0.043---1:4205050Str0.0610.056----0.046----0.0820.059----0.052--------0.065----0.048--------0.067----0.050---1:4206040Mod0.0560.054----0.044----0.0700.052----0.043--------0.051----0.043--------0.058----0.041---1:4206040Str0.0600.054----0.049----0.0840.055----0.049--------0.053----0.040--------0.051----0.037---1:45046Mod0.0620.051----0.038--------0.047----0.036--------0.056----0.0360.011----0.071----0.039---1:45046Str0.0610.056----0.043--------0.058----0.043--------0.079----0.0440.004----0.107----0.053---1:45055Mod0.0640.050----0.040--------0.050----0.038--------0.049----0.0360.010----0.048----0.032---1:45055Str0.0640.050----0.044--------0.052----0.042--------0.059----0.0400.003----0.073----0.041---1:45064Mod0.0640.049----0.044--------0.053----0.046--------0.051----0.0410.010----0.051----0.041---1:45064Str0.0610.053----0.048--------0.052----0.046--------0.053----0.0420.003----0.053----0.042---1:450812Mod0.0580.054----0.049----0.0950.055----0.046--------0.066----0.0490.004----0.087----0.0570.018 1:450812Str0.0580.055----0.052----0.1390.064----0.053--------0.103----0.0600.001----0.138----0.0830.023 1:4501010Mod0.0590.053----0.049--------0.050----0.045--------0.056----0.0430.005----0.061----0.0460.018 1:4501010Str0.0600.051----0.050--------0.062----0.052--------0.081----0.0530.002----0.097----0.0590.018 1:450128Mod0.0560.047----0.045--------0.049----0.045--------0.048----0.0430.006----0.050----0.0420.015 1:450128Str0.0590.049----0.049--------0.050----0.046--------0.058----0.0480.001----0.066----0.0500.014 1:4504060Mod0.0510.055----0.066----0.0910.057----0.065--------0.067----0.069--------0.086----0.086---1:4504060Str0.0520.054----0.064----0.1130.061----0.067--------0.102----0.088--------0.117----0.112---1:4505050Mod0.0510.049----0.054----0.1000.056----0.061--------0.062----0.068--------0.066----0.0640.017 1:4505050Str0.0540.052----0.063----0.1180.056----0.062--------0.083----0.077--------0.083----0.0790.026 1:4506040Mod0.0530.050----0.059----0.1010.053----0.058--------0.055----0.053--------0.051----0.056---1:4506040Str0.0560.052----0.062----0.1200.054----0.059--------0.058----0.058--------0.055----0.054---1:410046Mod0.0580.050----0.040--------0.054----0.0430.023----0.059----0.042--------0.079----0.045---1:410046Str0.0570.051----0.045--------0.058----0.0430.011----0.091----0.042--------0.153----0.062---1:410055Mod0.0560.051----0.044--------0.053----0.0440.021----0.054----0.040--------0.060----0.041---1:410055Str0.0580.049----0.044--------0.053----0.0440.015----0.069----0.046--------0.090----0.045---1:410064Mod0.0590.050----0.049--------0.046----0.0440.023----0.054----0.048--------0.058----0.051---1:410064Str0.0560.054----0.051--------0.048----0.0450.012----0.050----0.042--------0.047----0.037---Population Effect Size 0.8 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5

PAGE 229

218 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 26 (continued). Power estimates for conditions when the population effect size variance is 0.33. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:4100812Mod0.0580.055----0.051--------0.057----0.0530.014----0.079----0.0550.016----0.113----0.069---1:4100812Str0.0590.057----0.057--------0.070----0.0550.006----0.143----0.0730.004----0.229----0.117---1:41001010Mod0.0600.052----0.050--------0.055----0.0510.015----0.061----0.0500.019----0.075----0.050---1:41001010Str0.0590.051----0.053--------0.061----0.0560.009----0.112----0.0660.006----0.155----0.082---1:4100128Mod0.0550.051----0.053--------0.049----0.0480.016----0.051----0.0500.018----0.053----0.047---1:4100128Str0.0580.052----0.055--------0.055----0.0500.009----0.067----0.0500.006----0.082----0.057---1:41004060Mod0.0510.048----0.063--------0.059----0.070--------0.093----0.0870.010----0.121----0.1210.024 1:41004060Str0.0520.052----0.071--------0.070----0.075--------0.148----0.1230.006----0.180----0.1620.016 1:41005050Mod0.0530.052----0.063--------0.060----0.0700.013----0.075----0.0740.012----0.081----0.0840.022 1:41005050Str0.0500.051----0.067--------0.066----0.0750.011----0.110----0.0980.005----0.109----0.1000.017 1:41006040Mod0.0530.052----0.061--------0.050----0.0610.011----0.054----0.0610.012----0.051----0.0580.028 1:41006040Str0.0510.056----0.069--------0.058----0.0690.008----0.070----0.0680.005----0.060----0.0610.017 1:81046Mod----0.0720.098------------0.067----------------0.076----------------0.080-----------1:81046Str----0.0730.093------------0.076----------------0.075----------------0.079-----------1:81055Mod----0.070----------------0.068----------------0.072----------------0.072-----------1:81055Str----0.068----------------0.071----------------0.070----------------0.074-----------1:81064Mod----0.068----------------0.068----------------0.067----------------0.069-----------1:81064Str----0.068----------------0.070----------------0.065----------------0.062-----------1:810812Mod0.0740.0720.064--------0.0790.0680.067------------0.0740.085------------0.076-----------1:810812Str0.0780.0730.077--------0.0960.0790.082------------0.0820.097------------0.088-----------1:8101010Mod0.0740.0710.081--------0.0840.0680.082------------0.071----------------0.067-----------1:8101010Str0.0840.0710.085--------0.0950.0720.086------------0.073----------------0.080-----------1:810128Mod0.0750.0640.085------------0.066----------------0.068----------------0.067-----------1:810128Str0.0790.0640.087------------0.065----------------0.066----------------0.066-----------1:8104060Mod0.0650.0700.092--------0.0740.068----------------0.068----0.027--------0.073----0.030---1:8104060Str0.0700.0710.101--------0.0790.073----------------0.076----0.032--------0.074----0.035---1:8105050Mod0.0690.068------------0.0770.070----------------0.068----------------0.068----0.028---1:8105050Str0.0750.067------------0.0850.068----------------0.070----------------0.073----0.028---1:8106040Mod0.0690.067------------0.0770.063----------------0.065----------------0.062-----------1:8106040Str0.0730.068------------0.0820.067----------------0.066----------------0.066-----------1:82046Mod0.0680.058----------------0.058----------------0.062----------------0.068-----------1:82046Str0.0770.061----------------0.062----------------0.071----------------0.090-----------1:82055Mod0.0730.057----0.030--------0.054----0.030--------0.061----0.032--------0.060----0.026---1:82055Str0.0770.058----0.035--------0.056----0.034--------0.059----0.030--------0.064----0.030---Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 230

219 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 26 (continued). Power estimates for conditions when the population effect size variance is 0.33. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:82064Mod0.0740.054----0.035--------0.056----0.035--------0.059----0.033--------0.054----0.030---1:82064Str0.0750.053----0.036--------0.058----0.040--------0.054----0.031--------0.054----0.027---1:820812Mod0.0610.058----0.038----0.0770.059----0.035--------0.063----0.036--------0.079----0.041---1:820812Str0.0670.062----0.041----0.0950.066----0.042--------0.086----0.044--------0.093----0.049---1:8201010Mod0.0640.056----0.036----0.0780.052----0.034--------0.058----0.031--------0.063----0.036---1:8201010Str0.0680.057----0.041----0.1030.058----0.036--------0.068----0.039--------0.074----0.039---1:820128Mod0.0650.055----0.037----0.0930.056----0.038--------0.055----0.037--------0.053----0.032---1:820128Str0.0630.052----0.038----0.1060.055----0.040--------0.059----0.037--------0.057----0.035---1:8204060Mod0.0570.054----0.056----0.0700.061----0.060--------0.065----0.060--------0.076----0.071---1:8204060Str0.0600.062----0.063----0.0820.065----0.065--------0.083----0.075--------0.086----0.075---1:8205050Mod0.0620.058----0.053----0.0780.062----0.056--------0.058----0.056--------0.061----0.055---1:8205050Str0.0610.062----0.064----0.0870.062----0.058--------0.065----0.062--------0.069----0.059---1:8206040Mod0.0620.054----0.051----0.0800.051----0.050--------0.057----0.051--------0.052----0.046---1:8206040Str0.0590.055----0.055----0.0900.059----0.053--------0.061----0.055--------0.055----0.046---1:85046Mod0.0650.051----0.037--------0.053----0.037--------0.063----0.0380.013----0.076----0.039---1:85046Str0.0590.051----0.040--------0.057----0.042--------0.083----0.0410.006----0.124----0.052---1:85055Mod0.0560.050----0.040--------0.051----0.037--------0.052----0.0370.019----0.056----0.035---1:85055Str0.0670.052----0.043--------0.053----0.040--------0.065----0.0410.007----0.080----0.040---1:85064Mod0.0660.050----0.045--------0.051----0.045--------0.051----0.0420.019----0.058----0.048---1:85064Str0.0650.049----0.048--------0.050----0.044--------0.050----0.0420.008----0.057----0.043---1:850812Mod0.0520.050----0.045--------0.056----0.050--------0.076----0.0570.009----0.096----0.066---1:850812Str0.0570.054----0.053--------0.062----0.053--------0.119----0.0670.004----0.166----0.090---1:8501010Mod0.0580.055----0.048--------0.059----0.049--------0.059----0.0450.010----0.070----0.048---1:8501010Str0.0590.051----0.049--------0.061----0.052--------0.089----0.0590.004----0.108----0.066---1:850128Mod0.0560.050----0.049--------0.052----0.051--------0.052----0.0460.008----0.052----0.044---1:850128Str0.0600.054----0.052--------0.051----0.047--------0.056----0.0500.003----0.066----0.049---1:8504060Mod0.0560.051----0.068----0.0920.059----0.075--------0.084----0.096--------0.103--------0.020 1:8504060Str0.0600.056----0.078----0.1180.069----0.083--------0.112----0.106--------0.134--------0.028 1:8505050Mod0.0540.051----0.070----0.1030.056----0.067--------0.062----0.072--------0.069----0.0770.021 1:8505050Str0.0550.052----0.075----0.1260.061----0.076--------0.083----0.084--------0.086----0.0950.031 1:8506040Mod0.0570.054----0.068--------0.052----0.066--------0.055----0.067--------0.052----0.0610.023 1:8506040Str0.0530.052----0.070--------0.057----0.072--------0.060----0.065--------0.057----0.0640.026 1:810046Mod0.0620.053----0.040--------0.056----0.0400.026----0.073----0.045--------0.099----0.047---1:810046Str0.0570.052----0.042--------0.063----0.0430.017----0.114----0.046--------0.203----0.076---Population Effect Size 0.8 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5

PAGE 231

220 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 26 (continued). Power estimates for conditions when the population effect size variance is 0.33. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:810055Mod0.0600.056----0.046--------0.049----0.0430.031----0.051----0.041--------0.061----0.039---1:810055Str0.0600.053----0.045--------0.056----0.0470.019----0.069----0.041--------0.101----0.048---1:810064Mod0.0570.052----0.050--------0.051----0.0460.027----0.054----0.048-----------------------1:810064Str0.0580.052----0.051--------0.048----0.0490.020----0.048----0.046-----------------------1:8100812Mod0.0570.053----0.051--------0.060----0.0540.018----0.097----0.066--------0.145----0.089---1:8100812Str0.0610.054----0.058--------0.080----0.0640.012----0.173----0.085--------0.278----0.149---1:81001010Mod0.0510.048----0.046--------0.054----0.0480.023----0.071----0.058--------0.083----0.057---1:81001010Str0.0590.052----0.052--------0.065----0.0580.012----0.123----0.072--------0.171----0.092---1:8100128Mod0.0580.055----0.052--------0.053----0.0520.020----0.050----0.047--------0.050----0.046---1:8100128Str0.0550.048----0.052--------0.054----0.0530.013----0.066----0.054--------0.072----0.050---1:81004060Mod0.0540.049----0.072--------0.067----0.0840.015----0.112--------0.022----0.152-----------1:81004060Str0.0500.049----0.077--------0.075----0.0900.012----0.170--------0.014----0.215-----------1:81005050Mod0.0550.051----0.069--------0.058----0.0800.017----0.082----0.0930.029----0.087----0.105---1:81005050Str0.0530.051----0.079--------0.068----0.0840.012----0.121----0.1200.019----0.130----0.137---1:81006040Mod0.0530.050----0.068--------0.055----0.0700.019----0.055----0.0640.028----0.049----0.061---1:81006040Str0.0570.052----0.072--------0.060----0.0770.014----0.068----0.0770.017----0.061----0.072---Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 232

221 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 27. Power estimates for conditions when the population effect size variance is 0.50. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:11046Mod----0.0710.064------------0.0670.071------------0.070----------------0.066-----------1:11046Str----0.0720.063------------0.0690.070------------0.074----------------0.070-----------1:11055Mod----0.0740.064------------0.0700.072------------0.069----------------0.071-----------1:11055Str----0.0690.063------------0.0690.061------------0.069----------------0.070-----------1:11064Mod----0.0650.064------------0.0690.069------------0.067----------------0.065-----------1:11064Str----0.0670.061------------0.0680.068------------0.072----------------0.070-----------1:110812Mod0.0680.0670.034--------0.0740.0670.034------------0.0660.044------------0.0680.066-------1:110812Str0.0670.0630.035--------0.0830.0730.038------------0.0710.048------------0.0730.070-------1:1101010Mod0.0700.0690.032--------0.0740.0630.036------------0.0710.044------------0.0670.065-------1:1101010Str0.0740.0710.036--------0.0830.0660.033------------0.0680.046------------0.0690.072-------1:110128Mod0.0680.0650.036--------0.0730.0710.038------------0.0610.041------------0.0650.068-------1:110128Str0.0720.0710.039--------0.0860.0690.039------------0.0710.043------------0.0710.071-------1:1104060Mod0.0600.0620.032--------0.0690.0670.032--------0.0950.0640.039------------0.0660.046-------1:1104060Str0.0670.0710.034--------0.0750.0680.035--------0.1050.0650.043------------0.0710.053-------1:1105050Mod0.0620.0640.033--------0.0650.0650.033------------0.0630.038------------0.0650.044-------1:1105050Str0.0670.0670.030--------0.0710.0610.032------------0.0660.041------------0.0640.053-------1:1106040Mod0.0600.0630.032--------0.0680.0660.033------------0.0670.037------------0.0630.048-------1:1106040Str0.0590.0640.031--------0.0740.0630.035------------0.0650.042------------0.0650.054-------1:12046Mod0.0650.054----0.030--------0.055----0.032--------0.051----0.029--------0.054----0.029---1:12046Str0.0730.058----0.035--------0.057----0.032--------0.058----0.033--------0.059----0.030---1:12055Mod0.0680.056----0.032--------0.056----0.033--------0.055----0.029--------0.056----0.028---1:12055Str0.0740.058----0.037--------0.058----0.033--------0.061----0.033--------0.063----0.030---1:12064Mod0.0680.052----0.032--------0.055----0.035--------0.055----0.031--------0.058----0.031---1:12064Str0.0720.056----0.034--------0.049----0.030--------0.057----0.030--------0.058----0.031---1:120812Mod0.0570.0540.0850.036----0.0790.054----0.038--------0.060----0.035--------0.053----0.031---1:120812Str0.0620.0550.0910.037----0.0860.053----0.035--------0.059----0.038--------0.064----0.034---1:1201010Mod0.0600.0590.0840.040----0.0810.057----0.038--------0.055----0.036--------0.056----0.034---1:1201010Str0.0580.0560.0820.037----0.0870.055----0.036--------0.063----0.037--------0.067----0.035---1:120128Mod0.0570.0530.0820.035----0.0770.055----0.036--------0.053----0.032--------0.056----0.032---1:120128Str0.0610.0620.0910.040----0.0890.060----0.040--------0.061----0.036--------0.066----0.036---1:1204060Mod0.0580.0550.0610.044----0.0690.0550.0650.045--------0.0560.0760.041--------0.056----0.040---1:1204060Str0.0540.0540.0680.040----0.0800.0590.0740.043--------0.0630.0890.046--------0.061----0.043---1:1205050Mod0.0540.0500.0620.041----0.0670.0540.0660.042--------0.0570.0790.041--------0.056----0.040---1:1205050Str0.0560.0560.0730.050----0.0790.0590.0770.047--------0.0620.0940.047--------0.058----0.040---Population Effect Size 0.8 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5

PAGE 233

222 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 27 (continued). Power estimates for conditions when the population effect size variance is 0.50. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:1206040Mod0.0500.0530.0630.041----0.0690.0570.0670.043--------0.0570.0820.043--------0.052----0.036---1:1206040Str0.0590.0600.0710.046----0.0790.0550.0750.041--------0.0620.0890.044--------0.062----0.041---1:15046Mod0.0620.052----0.043--------0.049----0.041--------0.051----0.0430.009----0.056----0.0430.011 1:15046Str0.0620.049----0.044--------0.053----0.043--------0.057----0.0430.002----0.063----0.0440.002 1:15055Mod0.0570.048----0.043--------0.051----0.046--------0.055----0.0430.006----0.049----0.0400.013 1:15055Str0.0600.052----0.045--------0.050----0.044--------0.062----0.0470.002----0.065----0.0430.005 1:15064Mod0.0670.051----0.042--------0.051----0.043--------0.052----0.0440.008----0.054----0.0440.011 1:15064Str0.0630.056----0.050--------0.055----0.047--------0.057----0.0460.001----0.057----0.0420.003 1:150812Mod0.0560.052----0.047--------0.055----0.051--------0.054----0.047--------0.056----0.0460.004 1:150812Str0.0560.055----0.050--------0.055----0.054--------0.068----0.050--------0.074----0.0520.003 1:1501010Mod0.0540.050----0.048--------0.051----0.045--------0.052----0.047--------0.060----0.0470.004 1:1501010Str0.0530.051----0.048--------0.057----0.051--------0.067----0.054--------0.080----0.0530.003 1:150128Mod0.0530.054----0.050--------0.053----0.048--------0.054----0.046--------0.058----0.0470.004 1:150128Str0.0560.049----0.048--------0.055----0.048--------0.063----0.047--------0.068----0.0480.003 1:1504060Mod0.0520.0540.0850.055----0.0940.0510.0900.057--------0.055----0.055--------0.058----0.056---1:1504060Str0.0520.0490.1000.057----0.1070.0570.1100.058--------0.063----0.057--------0.066----0.057---1:1505050Mod0.0530.0490.0830.055----0.0950.0560.0950.058--------0.053----0.054--------0.054----0.051---1:1505050Str0.0530.0500.1000.058----0.1120.0570.1150.062--------0.066----0.061--------0.070----0.060---1:1506040Mod0.0530.0520.0880.055----0.0870.0530.0900.059--------0.055----0.056--------0.063----0.054---1:1506040Str0.0540.0490.1030.054----0.1080.0560.1130.058--------0.063----0.059--------0.068----0.059---1:110046Mod0.0580.049----0.047--------0.051----0.0450.021----0.049----0.047--------0.050----0.042---1:110046Str0.0550.053----0.051--------0.052----0.0490.011----0.057----0.047--------0.069----0.046---1:110055Mod0.0590.050----0.047--------0.050----0.0470.018----0.050----0.046--------0.049----0.042---1:110055Str0.0540.050----0.045--------0.052----0.0470.011----0.059----0.046--------0.073----0.050---1:110064Mod0.0550.048----0.047--------0.049----0.0460.018----0.050----0.047--------0.048----0.040---1:110064Str0.0560.051----0.051--------0.057----0.0530.012----0.060----0.051--------0.063----0.046---1:1100812Mod0.0540.049----0.051--------0.057----0.0510.012----0.053----0.0470.007----0.061----0.0530.011 1:1100812Str0.0550.052----0.051--------0.057----0.0540.007----0.076----0.0560.002----0.098----0.0640.003 1:11001010Mod0.0550.050----0.049--------0.051----0.0490.015----0.055----0.0480.007----0.063----0.054---1:11001010Str0.0610.053----0.056--------0.057----0.0550.006----0.089----0.0590.002----0.109----0.071---1:1100128Mod0.0570.050----0.051--------0.049----0.0520.015----0.057----0.0530.008----0.060----0.049---1:1100128Str0.0560.053----0.054--------0.061----0.0580.007----0.082----0.0610.002----0.100----0.063---1:11004060Mod0.0540.0500.0940.056--------0.0510.1090.056--------0.055----0.055--------0.059----0.0570.003 1:11004060Str0.0530.0530.1110.060--------0.0570.1370.064--------0.075----0.069--------0.083----0.0710.002 Population Effect Size 0.8 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5

PAGE 234

223 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 27 (continued). Power estimates for conditions when the population effect size variance is 0.50. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:11005050Mod0.0510.0500.0920.055--------0.0510.1120.060--------0.059----0.061--------0.063----0.0600.004 1:11005050Str0.0500.0490.1070.059--------0.0540.1360.064--------0.079----0.071--------0.090----0.0750.002 1:11006040Mod0.0500.0490.0910.056--------0.0530.1090.058--------0.060----0.062--------0.063----0.0600.004 1:11006040Str0.0470.0480.1060.060--------0.0630.1400.065--------0.082----0.071--------0.082----0.0690.001 1:21046Mod----0.0680.071------------0.0680.076------------0.072----------------0.070-----------1:21046Str----0.0690.068------------0.0690.072------------0.077----------------0.076-----------1:21055Mod----0.0700.071------------0.0710.074------------0.071----------------0.073-----------1:21055Str----0.0720.067------------0.0710.079------------0.070----------------0.074-----------1:21064Mod----0.0680.075------------0.0670.075------------0.066----------------0.066-----------1:21064Str----0.0630.073------------0.0700.075------------0.072----------------0.070-----------1:210812Mod0.0670.0680.033--------0.0770.0670.036------------0.0700.046------------0.0700.068-------1:210812Str0.0710.0720.040--------0.0830.0680.041------------0.0750.056------------0.0740.077-------1:2101010Mod0.0660.0650.039--------0.0770.0650.041------------0.0680.052------------0.0630.074-------1:2101010Str0.0740.0740.045--------0.0820.0670.040------------0.0670.054------------0.0700.085-------1:210128Mod0.0700.0660.041--------0.0730.0700.042------------0.0610.057------------0.0650.082-------1:210128Str0.0730.0680.042--------0.0880.0650.044------------0.0680.056------------0.0690.087-------1:2104060Mod0.0600.0630.036--------0.0680.0640.040------------0.0670.047------------0.0690.051-------1:2104060Str0.0700.0690.042--------0.0740.0680.045------------0.0700.049------------0.0680.061-------1:2105050Mod0.0640.0640.042--------0.0700.0630.042------------0.0640.045------------0.0660.059-------1:2105050Str0.0670.0620.042--------0.0710.0630.040------------0.0760.054------------0.0630.064-------1:2106040Mod0.0650.0700.043--------0.0710.0630.042------------0.0630.049------------0.0580.058-------1:2106040Str0.0680.0640.045--------0.0750.0650.043------------0.0660.056------------0.0650.066-------1:22046Mod0.0720.052----0.027--------0.056----0.032--------0.051----0.030--------0.054----0.027---1:22046Str0.0710.058----0.036--------0.055----0.031--------0.062----0.029--------0.065----0.028---1:22055Mod0.0680.053----0.030--------0.058----0.035--------0.057----0.029--------0.056----0.027---1:22055Str0.0740.059----0.034--------0.055----0.030--------0.056----0.031--------0.066----0.031---1:22064Mod0.0800.060----0.038--------0.058----0.035--------0.054----0.031--------0.060----0.031---1:22064Str0.0740.054----0.034--------0.055----0.036--------0.053----0.030--------0.056----0.029---1:220812Mod0.0560.055----0.038----0.0750.059----0.039--------0.055----0.034--------0.058----0.033---1:220812Str0.0620.054----0.039----0.0880.058----0.037--------0.064----0.038--------0.068----0.037---1:2201010Mod0.0600.053----0.034----0.0790.056----0.036--------0.058----0.032--------0.055----0.033---1:2201010Str0.0650.058----0.038----0.0910.055----0.037--------0.063----0.037--------0.064----0.037---1:220128Mod0.0580.049----0.032----0.0830.055----0.034--------0.056----0.036--------0.056----0.034---1:220128Str0.0620.057----0.038----0.0910.057----0.037--------0.057----0.033--------0.054----0.031---Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 235

224 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 27 (continued). Power estimates for conditions when the population effect size variance is 0.50. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:2204060Mod0.0550.0530.0760.043----0.0710.0590.0790.044--------0.0590.0940.044--------0.057----0.044---1:2204060Str0.0580.0560.0850.050----0.0790.0550.0870.045--------0.0620.1040.044--------0.062----0.043---1:2205050Mod0.0560.0550.0750.048----0.0690.0540.0790.044--------0.0570.0950.046--------0.059----0.043---1:2205050Str0.0550.0580.0870.051----0.0810.0580.0920.046--------0.0600.1120.045--------0.056----0.041---1:2206040Mod0.0560.0540.0790.042----0.0670.0520.0770.039--------0.0570.1010.042--------0.050----0.040---1:2206040Str0.0530.0530.0860.044----0.0780.0550.0930.043--------0.0600.1070.044--------0.055----0.042---1:25046Mod0.0630.054----0.047--------0.051----0.042--------0.051----0.0390.009----0.056----0.043---1:25046Str0.0630.054----0.047--------0.056----0.049--------0.062----0.0460.003----0.071----0.043---1:25055Mod0.0630.052----0.046--------0.050----0.045--------0.052----0.0410.009----0.051----0.037---1:25055Str0.0610.051----0.046--------0.051----0.044--------0.057----0.0440.002----0.065----0.041---1:25064Mod0.0630.052----0.045--------0.050----0.043--------0.052----0.0440.008----0.050----0.039---1:25064Str0.0600.056----0.050--------0.050----0.047--------0.056----0.0460.002----0.056----0.042---1:250812Mod0.0540.051----0.047--------0.051----0.046--------0.059----0.049--------0.063----0.0470.004 1:250812Str0.0560.052----0.050--------0.057----0.052--------0.073----0.051--------0.092----0.0620.005 1:2501010Mod0.0560.049----0.048--------0.050----0.049--------0.056----0.046--------0.058----0.0460.004 1:2501010Str0.0580.055----0.052--------0.055----0.050--------0.067----0.054--------0.080----0.0550.005 1:250128Mod0.0620.054----0.050--------0.049----0.044--------0.048----0.0460.003----0.053----0.0480.004 1:250128Str0.0540.050----0.050--------0.053----0.053--------0.062----0.0500.001----0.063----0.0480.003 1:2504060Mod0.0520.0490.0980.056----0.0900.055----0.059--------0.061----0.058--------0.066----0.061---1:2504060Str0.0550.0530.1240.058----0.1070.058----0.066--------0.072----0.066--------0.083----0.073---1:2505050Mod0.0500.052----0.056----0.0940.0520.1120.058--------0.059----0.056--------0.061----0.056---1:2505050Str0.0570.053----0.062----0.1120.0550.1360.059--------0.071----0.064--------0.073----0.067---1:2506040Mod0.0540.0530.0990.055----0.0920.050----0.056--------0.052----0.051--------0.055----0.053---1:2506040Str0.0520.0540.1210.059----0.1110.054----0.059--------0.059----0.059--------0.058----0.053---1:210046Mod0.0590.053----0.047--------0.049----0.0450.017----0.052----0.045--------0.060----0.047---1:210046Str0.0580.054----0.049--------0.056----0.0530.012----0.071----0.049--------0.091----0.049---1:210055Mod0.0570.053----0.046--------0.049----0.0430.019----0.051----0.043--------0.053----0.041---1:210055Str0.0600.048----0.048--------0.052----0.0470.013----0.066----0.047--------0.068----0.046---1:210064Mod0.0600.056----0.051--------0.053----0.0490.020----0.049----0.045--------0.049----0.048---1:210064Str0.0600.051----0.051--------0.053----0.0520.012----0.053----0.048--------0.058----0.049---1:2100812Mod0.0560.051----0.053--------0.054----0.051--------0.063----0.0530.008----0.076----0.058---1:2100812Str0.0580.053----0.056--------0.061----0.059--------0.096----0.0650.002----0.134----0.077---1:21001010Mod0.0550.052----0.053--------0.050----0.0470.014----0.054----0.0490.010----0.066----0.054---1:21001010Str0.0540.050----0.050--------0.056----0.0540.009----0.089----0.0640.003----0.106----0.064---Population Effect Size 0.8 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5

PAGE 236

225 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 27 (continued). Power estimates for conditions when the population effect size variance is 0.50. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:2100128Mod0.0580.050----0.052--------0.051----0.0520.014----0.053----0.0510.010----0.050----0.047---1:2100128Str0.0530.050----0.053--------0.055----0.0540.009----0.071----0.0580.002----0.077----0.054---1:21004060Mod0.0530.051----0.064--------0.058----0.067--------0.068----0.066--------0.080----0.0800.006 1:21004060Str0.0530.048----0.063--------0.057----0.067--------0.093----0.081--------0.116----0.1000.002 1:21005050Mod0.0540.051----0.063--------0.051----0.061--------0.065----0.0620.006----0.065----0.0660.007 1:21005050Str0.0520.051----0.065--------0.060----0.068--------0.083----0.0730.003----0.091----0.0790.003 1:21006040Mod0.0500.049----0.057--------0.054----0.061--------0.057----0.0590.007----0.053----0.0540.006 1:21006040Str0.0540.050----0.063--------0.057----0.065--------0.073----0.0680.003----0.071----0.0650.003 1:41046Mod----0.0720.081------------0.069----------------0.072----------------0.067-----------1:41046Str----0.0680.085------------0.070----------------0.075----------------0.076-----------1:41055Mod----0.070----------------0.067----------------0.068----------------0.068-----------1:41055Str----0.071----------------0.068----------------0.069----------------0.070-----------1:41064Mod----0.068----------------0.064----------------0.068----------------0.068-----------1:41064Str----0.071----------------0.069----------------0.069----------------0.064-----------1:410812Mod0.0700.0680.052--------0.0770.0720.051------------0.0750.068------------0.070-----------1:410812Str0.0740.0680.055--------0.0850.0670.056------------0.0720.072------------0.076-----------1:4101010Mod0.0690.0660.056--------0.0760.0640.058------------0.0650.074------------0.069-----------1:4101010Str0.0740.0690.058--------0.0910.0750.065------------0.0760.084------------0.074-----------1:410128Mod0.0720.0640.062------------0.0630.061------------0.0640.081------------0.065-----------1:410128Str0.0760.0680.061------------0.0650.064------------0.0610.090------------0.065-----------1:4104060Mod0.0670.0660.059--------0.0730.0720.062------------0.0660.068------------0.0690.083-------1:4104060Str0.0680.0670.063--------0.0720.0690.070------------0.0680.078------------0.0770.093-------1:4105050Mod0.0650.0630.067--------0.0710.0620.072------------0.0650.075------------0.065-----------1:4105050Str0.0690.0660.072--------0.0770.0680.076------------0.0680.078------------0.072-----------1:4106040Mod0.0690.0640.068--------0.0730.0640.070------------0.0620.081------------0.060-----------1:4106040Str0.0680.0640.066--------0.0780.0650.076------------0.0640.087------------0.061-----------1:42046Mod0.0710.057----0.031--------0.056----0.030--------0.063----0.033--------0.058-----------1:42046Str0.0760.059----0.032--------0.054----0.030--------0.065----0.034--------0.072-----------1:42055Mod0.0710.054----0.033--------0.058----0.034--------0.057----0.030--------0.059----0.027---1:42055Str0.0720.056----0.036--------0.056----0.034--------0.060----0.032--------0.060----0.028---1:42064Mod0.0690.053----0.031--------0.056----0.034--------0.058----0.033--------0.060----0.035---1:42064Str0.0700.053----0.035--------0.055----0.035--------0.054----0.032--------0.052----0.031---1:420812Mod0.0630.058----0.040----0.0790.057----0.040--------0.062----0.036--------0.069----0.040---1:420812Str0.0650.060----0.042----0.0960.059----0.043--------0.067----0.038--------0.077----0.040---Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 237

226 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 27 (continued). Power estimates for conditions when the population effect size variance is 0.50. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:4201010Mod0.0600.056----0.040----0.0780.052----0.035--------0.057----0.036--------0.056----0.034---1:4201010Str0.0630.055----0.042----0.0940.058----0.039--------0.064----0.040--------0.067----0.036---1:420128Mod0.0640.052----0.038----0.0840.053----0.036--------0.056----0.038--------0.052----0.034---1:420128Str0.0640.051----0.037----0.0970.060----0.044--------0.054----0.036--------0.056----0.032---1:4204060Mod0.0530.056----0.050----0.0710.057----0.051--------0.062----0.049--------0.068----0.054---1:4204060Str0.0610.055----0.053----0.0750.061----0.055--------0.068----0.056--------0.071----0.058---1:4205050Mod0.0580.055----0.049----0.0740.057----0.048--------0.054----0.049--------0.057----0.048---1:4205050Str0.0590.056----0.051----0.0850.060----0.051--------0.056----0.047--------0.063----0.049---1:4206040Mod0.0530.057----0.049----0.0740.052----0.047--------0.050----0.045--------0.053----0.043---1:4206040Str0.0550.055----0.048----0.0840.057----0.052--------0.056----0.047--------0.051----0.041---1:45046Mod0.0650.053----0.042--------0.050----0.043--------0.054----0.0410.010----0.060----0.041---1:45046Str0.0610.048----0.040--------0.056----0.045--------0.068----0.0420.003----0.091----0.049---1:45055Mod0.0600.050----0.043--------0.052----0.044--------0.053----0.0400.011----0.052----0.041---1:45055Str0.0650.056----0.046--------0.056----0.047--------0.059----0.0450.004----0.060----0.040---1:45064Mod0.0640.051----0.047--------0.054----0.045--------0.052----0.0470.010----0.054----0.046---1:45064Str0.0650.051----0.049--------0.051----0.050--------0.051----0.0480.004----0.051----0.044---1:450812Mod0.0580.054----0.051--------0.054----0.050--------0.062----0.0500.004----0.077----0.0560.008 1:450812Str0.0570.055----0.052--------0.061----0.057--------0.082----0.0570.002----0.110----0.0680.005 1:4501010Mod0.0560.054----0.049--------0.057----0.051--------0.055----0.0450.006----0.060----0.0480.010 1:4501010Str0.0590.054----0.052--------0.054----0.054--------0.068----0.0540.002----0.086----0.0570.005 1:450128Mod0.0580.051----0.048--------0.051----0.050--------0.050----0.0470.006----0.053----0.0470.008 1:450128Str0.0620.056----0.055--------0.054----0.052--------0.057----0.0520.002----0.062----0.0520.004 1:4504060Mod0.0550.056----0.068----0.0920.054----0.066--------0.068----0.075--------0.078----0.080---1:4504060Str0.0490.043----0.058----0.1090.059----0.072--------0.084----0.082--------0.097----0.098---1:4505050Mod0.0570.054----0.066----0.0990.055----0.066--------0.063----0.070--------0.060----0.063---1:4505050Str0.0530.051----0.063----0.1200.060----0.074--------0.072----0.075--------0.077----0.078---1:4506040Mod0.0490.049----0.058--------0.053----0.058--------0.052----0.058--------0.050----0.055---1:4506040Str0.0550.050----0.064--------0.055----0.066--------0.058----0.064--------0.057----0.060---1:410046Mod0.0530.048----0.041--------0.051----0.0440.022----0.058----0.042--------0.069----0.046---1:410046Str0.0580.052----0.048--------0.056----0.0450.015----0.084----0.050--------0.121----0.054---1:410055Mod0.0650.056----0.051--------0.052----0.0470.025----0.050----0.048--------0.051----0.042---1:410055Str0.0620.054----0.049--------0.053----0.0470.016----0.068----0.047--------0.080----0.051---1:410064Mod0.0530.049----0.048--------0.046----0.0430.021----0.051----0.050--------0.055----0.052---1:410064Str0.0600.052----0.052--------0.052----0.0520.015----0.055----0.050--------0.052----0.047---Population Effect Size 0.8 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5

PAGE 238

227 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 27 (continued). Power estimates for conditions when the population effect size variance is 0.50. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:4100812Mod0.0540.052----0.055--------0.052----0.0490.016----0.068----0.0540.016----0.098----0.065---1:4100812Str0.0550.053----0.057--------0.060----0.0580.009----0.118----0.0710.006----0.177----0.098---1:41001010Mod0.0590.051----0.052--------0.051----0.0510.016----0.059----0.0550.016----0.066----0.052---1:41001010Str0.0590.053----0.055--------0.063----0.0590.011----0.090----0.0630.006----0.121----0.076---1:4100128Mod0.0570.050----0.051--------0.052----0.0540.016----0.051----0.0510.017----0.047----0.047---1:4100128Str0.0570.048----0.056--------0.050----0.0540.011----0.067----0.0560.007----0.067----0.052---1:41004060Mod0.0550.052----0.071--------0.060----0.073--------0.084----0.0860.012----0.109----0.1140.018 1:41004060Str0.0540.052----0.070--------0.065----0.076--------0.116----0.1070.006----0.158----0.1450.009 1:41005050Mod0.0480.048----0.066--------0.055----0.071--------0.065----0.0710.011----0.071----0.0810.018 1:41005050Str0.0560.051----0.073--------0.059----0.075--------0.088----0.0880.009----0.102----0.0980.009 1:41006040Mod0.0540.054----0.067--------0.052----0.061--------0.054----0.0660.011----0.055----0.0630.020 1:41006040Str0.0510.048----0.067--------0.055----0.069--------0.063----0.0670.006----0.064----0.0700.011 1:81046Mod----0.071----------------0.071----------------0.072----------------0.072-----------1:81046Str----0.070----------------0.072----------------0.078----------------0.079-----------1:81055Mod----0.066----------------0.067----------------0.064----------------0.068-----------1:81055Str----0.068----------------0.072----------------0.069----------------0.073-----------1:81064Mod----0.065----------------0.066----------------0.069----------------0.065-----------1:81064Str----0.071----------------0.069----------------0.067----------------0.067-----------1:810812Mod0.0780.0730.074--------0.0780.0690.073------------0.071----------------0.076-----------1:810812Str0.0770.0670.079--------0.0870.0690.083------------0.081----------------0.082-----------1:8101010Mod0.0720.0700.083--------0.0830.067----------------0.074----------------0.068-----------1:8101010Str0.0750.0690.088--------0.0920.065----------------0.073----------------0.072-----------1:810128Mod0.0750.067----------------0.064----------------0.063----------------0.061-----------1:810128Str0.0810.068----------------0.063----------------0.064----------------0.067-----------1:8104060Mod0.0700.067----0.032----0.0690.066----0.030--------0.067----0.031--------0.071----0.033---1:8104060Str0.0680.072----0.036----0.0760.069----0.035--------0.070----0.032--------0.074----0.037---1:8105050Mod0.0670.063----0.030----0.0740.063----------------0.065----0.028--------0.066----0.030---1:8105050Str0.0670.065----0.032----0.0880.067----------------0.068----0.029--------0.072----0.029---1:8106040Mod0.0680.064------------0.0770.065----0.028--------0.063----------------0.062----0.026---1:8106040Str0.0680.063------------0.0810.064----0.031--------0.069----------------0.064----0.025---1:82046Mod0.0720.055----------------0.056----0.032--------0.055----0.029--------0.061----0.030---1:82046Str0.0720.058----------------0.062----0.033--------0.063----0.032--------0.074----0.031---1:82055Mod0.0730.055----0.032--------0.059----0.035--------0.056----0.028--------0.056----0.029---1:82055Str0.0770.056----0.034--------0.054----0.033--------0.058----0.030--------0.063----0.029---Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 239

228 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 27 (continued). Power estimates for conditions when the population effect size variance is 0.50. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:82064Mod0.0720.053----0.035--------0.056----0.038--------0.055----0.035--------0.053----0.030---1:82064Str0.0750.054----0.039--------0.054----0.038--------0.056----0.036--------0.054----0.033---1:820812Mod0.0620.058----0.040----0.0790.059----0.042--------0.062----0.043--------0.071----0.039---1:820812Str0.0710.061----0.045----0.0980.063----0.044--------0.076----0.044--------0.087----0.048---1:8201010Mod0.0610.056----0.036----0.0870.056----0.040--------0.057----0.035--------0.059----0.036---1:8201010Str0.0680.058----0.046----0.1010.061----0.045--------0.063----0.039--------0.071----0.038---1:820128Mod0.0670.056----0.041--------0.054----0.040--------0.054----0.040--------0.055----0.035---1:820128Str0.0700.056----0.045--------0.052----0.042--------0.054----0.041--------0.055----0.034---1:8204060Mod0.0590.057----0.058----0.0700.058----0.060--------0.065----0.068--------0.072----0.074---1:8204060Str0.0550.057----0.062----0.0800.063----0.067--------0.070----0.065--------0.075----0.074---1:8205050Mod0.0590.058----0.060----0.0750.059----0.060--------0.060----0.059--------0.063----0.058---1:8205050Str0.0600.057----0.060----0.0860.056----0.061--------0.063----0.061--------0.063----0.061---1:8206040Mod0.0580.050----0.053----0.0820.054----0.053--------0.054----0.053--------0.052----0.047---1:8206040Str0.0590.056----0.058----0.0860.055----0.056--------0.054----0.055--------0.051----0.048---1:85046Mod0.0560.046----0.038--------0.053----0.043--------0.059----0.0410.016----0.074----0.042---1:85046Str0.0660.052----0.043--------0.058----0.044--------0.075----0.0420.006----0.104----0.052---1:85055Mod0.0620.052----0.044--------0.050----0.040--------0.054----0.0400.017----0.053----0.040---1:85055Str0.0640.052----0.045--------0.051----0.043--------0.061----0.0430.008----0.070----0.046---1:85064Mod0.0610.057----0.052--------0.051----0.047--------0.050----0.0470.015----0.055----0.050---1:85064Str0.0600.049----0.047--------0.049----0.048--------0.050----0.0440.006----0.052----0.043---1:850812Mod0.0550.053----0.053--------0.051----0.049--------0.068----0.0560.008----0.083----0.0610.017 1:850812Str0.0600.059----0.058--------0.058----0.058--------0.096----0.0620.002----0.134----0.0800.010 1:8501010Mod0.0580.051----0.048--------0.057----0.054--------0.056----0.0520.008----0.065----0.0520.018 1:8501010Str0.0630.055----0.057--------0.056----0.057--------0.074----0.0580.003----0.091----0.0580.011 1:850128Mod0.0590.050----0.049--------0.050----0.052--------0.047----0.0480.009----0.053----0.0470.019 1:850128Str0.0580.050----0.055--------0.050----0.052--------0.054----0.0490.004----0.060----0.0500.010 1:8504060Mod0.0530.055----0.075----0.0980.060----0.082--------0.068----0.086--------0.086--------0.009 1:8504060Str0.0490.050----0.074----0.1160.063----0.089--------0.092----0.104--------0.110--------0.007 1:8505050Mod0.0560.052----0.072--------0.053----0.074--------0.063----0.078--------0.068----0.0810.012 1:8505050Str0.0520.049----0.074--------0.056----0.076--------0.079----0.087--------0.080----0.0950.008 1:8506040Mod0.0580.054----0.073--------0.049----0.067--------0.052----0.069--------0.047----0.0620.011 1:8506040Str0.0560.052----0.071--------0.048----0.069--------0.062----0.071--------0.056----0.0660.008 1:810046Mod0.0610.051----0.044--------0.054----0.0460.028----0.065----0.046--------0.090----0.050---1:810046Str0.0600.052----0.048--------0.058----0.0480.021----0.098----0.054--------0.151----0.061---Population Effect Size 0.8 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5

PAGE 240

229 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 27 (continued). Power estimates for conditions when the population effect size variance is 0.50. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:810055Mod0.0580.048----0.042--------0.054----0.0510.028----0.057----0.048--------0.053----0.040---1:810055Str0.0590.050----0.049--------0.048----0.0450.019----0.067----0.048--------0.082----0.049---1:810064Mod0.0560.052----0.053--------0.056----0.0560.027----0.052----0.052--------0.056----0.055---1:810064Str0.0580.050----0.052--------0.052----0.0530.020----0.049----0.048--------0.050----0.048---1:8100812Mod0.0560.052----0.056--------0.055----0.0560.020----0.083----0.063--------0.123----0.084---1:8100812Str0.0590.056----0.062--------0.069----0.0630.016----0.133----0.078--------0.210----0.115---1:81001010Mod0.0560.052----0.053--------0.052----0.0540.021----0.065----0.055--------0.076----0.061---1:81001010Str0.0610.053----0.059--------0.059----0.0560.016----0.098----0.067--------0.142----0.090---1:8100128Mod0.0540.049----0.054--------0.047----0.0520.021----0.054----0.055--------0.055----0.053---1:8100128Str0.0600.052----0.060--------0.056----0.0570.016----0.064----0.058--------0.069----0.057---1:81004060Mod0.0580.055----0.079--------0.058----0.0840.017----0.095--------0.019----0.129-----------1:81004060Str0.0540.052----0.082--------0.071----0.0960.012----0.130--------0.013----0.186-----------1:81005050Mod0.0530.053----0.076--------0.056----0.079--------0.069----0.0920.024----0.085----0.104---1:81005050Str0.0550.051----0.079--------0.067----0.087--------0.100----0.1070.015----0.117----0.132---1:81006040Mod0.0530.052----0.073--------0.055----0.0750.019----0.053----0.0750.028----0.055----0.071---1:81006040Str0.0520.050----0.076--------0.058----0.0820.013----0.061----0.0780.016----0.061----0.079---Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 241

230 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 28. Power estimates for conditions when the population effect size variance is 1.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:11046Mod----0.074----------------0.067----------------0.067----------------0.069-----------1:11046Str----0.067----------------0.068----------------0.069----------------0.069-----------1:11055Mod----0.068----------------0.069----------------0.069----------------0.072-----------1:11055Str----0.067----------------0.070----------------0.073----------------0.071-----------1:11064Mod----0.068----------------0.072----------------0.071----------------0.068-----------1:11064Str----0.067----------------0.070----------------0.068----------------0.069-----------1:110812Mod0.0680.0710.054------------0.0660.053------------0.0630.065------------0.063-----------1:110812Str0.0770.0700.053------------0.0620.049------------0.0710.069------------0.072-----------1:1101010Mod0.0700.0670.051--------0.0820.0670.055------------0.0660.069------------0.067-----------1:1101010Str0.0710.0670.051--------0.0850.0670.051------------0.0670.068------------0.070-----------1:110128Mod0.0710.0670.049--------0.0800.0670.055------------0.0680.066------------0.065-----------1:110128Str0.0770.0690.052--------0.0840.0650.053------------0.0740.070------------0.071-----------1:1104060Mod0.0630.0670.042--------0.0680.0600.044------------0.0670.051------------0.0590.067-------1:1104060Str0.0640.0670.044--------0.0720.0640.043------------0.0660.054------------0.0680.070-------1:1105050Mod0.0680.0640.046--------0.0690.0630.044------------0.0630.055------------0.0600.071-------1:1105050Str0.0670.0650.047--------0.0730.0670.047------------0.0590.052------------0.0690.073-------1:1106040Mod0.0640.0600.042--------0.0700.0650.042------------0.0670.052------------0.0610.068-------1:1106040Str0.0670.0620.043--------0.0700.0620.039------------0.0670.057------------0.0630.065-------1:12046Mod0.0720.055----0.038--------0.058----0.032--------0.059----0.036--------0.055----0.035---1:12046Str0.0710.057----0.038--------0.057----0.039--------0.056----0.038--------0.061----0.036---1:12055Mod0.0740.058----0.037--------0.052----0.034--------0.059----0.037--------0.054----0.032---1:12055Str0.0740.057----0.034--------0.056----0.037--------0.053----0.037--------0.057----0.035---1:12064Mod0.0680.055----0.037--------0.059----0.040--------0.057----0.036--------0.058----0.035---1:12064Str0.0700.055----0.038--------0.057----0.039--------0.060----0.039--------0.061----0.036---1:120812Mod0.0610.054----0.042----0.0830.058----0.041--------0.055----0.038--------0.054----0.042---1:120812Str0.0600.060----0.046----0.0870.058----0.040--------0.055----0.041--------0.060----0.041---1:1201010Mod0.0640.051----0.038----0.0830.062----0.044--------0.052----0.038--------0.059----0.038---1:1201010Str0.0570.053----0.042----0.0920.057----0.043--------0.058----0.042--------0.059----0.035---1:120128Mod0.0600.054----0.041----0.0790.054----0.042--------0.051----0.037--------0.061----0.042---1:120128Str0.0560.053----0.041----0.0900.054----0.043--------0.057----0.037--------0.059----0.040---1:1204060Mod0.0530.0530.0760.048----0.0720.0520.0830.049--------0.054----0.048--------0.059----0.046---1:1204060Str0.0570.0560.0870.048----0.0760.0550.0880.049--------0.060----0.050--------0.056----0.043---1:1205050Mod0.0500.0560.0780.051----0.0750.0570.0820.053--------0.054----0.045--------0.054----0.046---1:1205050Str0.0580.0580.0800.056----0.0780.0570.0890.053--------0.059----0.047--------0.059----0.046---Population Effect Size 0.8 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5

PAGE 242

231 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 28 (continued). Power estimates for conditions when the population effect size variance is 1.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:1206040Mod0.0600.0530.0800.048----0.0710.0560.0830.047--------0.056----0.049--------0.056----0.049---1:1206040Str0.0510.0560.0810.050----0.0780.0540.0880.051--------0.054----0.047--------0.054----0.045---1:15046Mod0.0610.050----0.048--------0.052----0.052--------0.051----0.0470.009----0.049----0.0450.009 1:15046Str0.0590.048----0.048--------0.053----0.051--------0.053----0.0500.005----0.054----0.0440.002 1:15055Mod0.0620.053----0.052--------0.056----0.054--------0.053----0.0490.009----0.054----0.0450.009 1:15055Str0.0630.054----0.053--------0.054----0.052--------0.057----0.0500.004----0.054----0.0450.003 1:15064Mod0.0630.051----0.046--------0.055----0.053--------0.053----0.0470.010----0.053----0.0480.010 1:15064Str0.0630.054----0.053--------0.049----0.048--------0.051----0.0490.004----0.059----0.0470.003 1:150812Mod0.0570.053----0.058--------0.047----0.050--------0.052----0.052--------0.055----0.0530.004 1:150812Str0.0560.051----0.059--------0.056----0.056--------0.057----0.053--------0.061----0.0530.001 1:1501010Mod0.0570.050----0.052--------0.058----0.062--------0.056----0.054--------0.055----0.0510.003 1:1501010Str0.0580.055----0.057--------0.054----0.054--------0.060----0.059--------0.066----0.0540.002 1:150128Mod0.0550.050----0.053--------0.047----0.049--------0.056----0.055--------0.050----0.0490.004 1:150128Str0.0580.051----0.055--------0.053----0.054--------0.058----0.056--------0.061----0.0530.002 1:1504060Mod0.0530.052----0.059--------0.053----0.060--------0.053----0.061--------0.049----0.056---1:1504060Str0.0520.052----0.066--------0.052----0.066--------0.054----0.062--------0.055----0.057---1:1505050Mod0.0490.0510.0950.059--------0.053----0.063--------0.056----0.063--------0.052----0.057---1:1505050Str0.0490.0490.1080.063--------0.052----0.064--------0.066----0.069--------0.063----0.062---1:1506040Mod0.0520.050----0.064--------0.050----0.063--------0.053----0.063--------0.053----0.057---1:1506040Str0.0500.053----0.065--------0.055----0.066--------0.052----0.060--------0.058----0.063---1:110046Mod0.0540.053----0.053--------0.051----0.0530.020----0.048----0.0500.023----0.050----0.050---1:110046Str0.0600.052----0.054--------0.054----0.0560.015----0.056----0.0530.010----0.055----0.050---1:110055Mod0.0630.050----0.054--------0.051----0.0510.022----0.054----0.0520.023----0.050----0.048---1:110055Str0.0610.053----0.055--------0.053----0.0530.014----0.056----0.0510.012----0.063----0.051---1:110064Mod0.0570.055----0.056--------0.049----0.0520.021----0.047----0.0510.021----0.050----0.048---1:110064Str0.0590.050----0.051--------0.053----0.0560.015----0.053----0.0520.012----0.059----0.049---1:1100812Mod0.0550.053----0.056--------0.056----0.0610.014----0.050----0.0540.012----0.053----0.0550.011 1:1100812Str0.0610.053----0.064--------0.051----0.0580.012----0.063----0.0620.004----0.076----0.0620.003 1:11001010Mod0.0540.051----0.058--------0.048----0.055--------0.056----0.0590.011----0.055----0.0540.011 1:11001010Str0.0610.052----0.060--------0.052----0.058--------0.062----0.0600.005----0.079----0.0620.003 1:1100128Mod0.0550.047----0.055--------0.053----0.059--------0.051----0.0530.010----0.056----0.0570.012 1:1100128Str0.0560.052----0.062--------0.051----0.057--------0.066----0.0610.004----0.071----0.0570.004 1:11004060Mod0.0550.053----0.067--------0.052----0.068--------0.053----0.065--------0.060----0.071---1:11004060Str0.0520.049----0.066--------0.057----0.071--------0.064----0.075--------0.071----0.069---Population Effect Size 0.8 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5

PAGE 243

232 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 28 (continued). Power estimates for conditions when the population effect size variance is 1.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:11005050Mod0.0540.049----0.064--------0.052----0.068--------0.060----0.069--------0.058----0.069---1:11005050Str0.0590.053----0.074--------0.057----0.073--------0.061----0.069--------0.072----0.072---1:11006040Mod0.0520.052----0.068--------0.051----0.065--------0.057----0.069--------0.060----0.069---1:11006040Str0.0550.049----0.065--------0.052----0.069--------0.060----0.070--------0.067----0.069---1:21046Mod----0.064----------------0.068----------------0.068-------------------------------1:21046Str----0.069----------------0.070----------------0.069-------------------------------1:21055Mod----0.066----------------0.074----------------0.066----------------0.070-----------1:21055Str----0.067----------------0.066----------------0.069----------------0.070-----------1:21064Mod----0.068----------------0.066----------------0.067----------------0.065-----------1:21064Str----0.068----------------0.069----------------0.068----------------0.072-----------1:210812Mod0.0700.0660.050------------0.0670.054------------0.0670.072------------0.068-----------1:210812Str0.0730.0680.055------------0.0720.054------------0.0700.072------------0.072-----------1:2101010Mod0.0700.0690.059--------0.0760.0630.055------------0.0690.078------------0.064-----------1:2101010Str0.0740.0690.054--------0.0870.0720.061------------0.0710.081------------0.068-----------1:210128Mod0.0750.0650.060------------0.0660.064------------0.062----------------0.065-----------1:210128Str0.0790.0650.058------------0.0620.062------------0.063----------------0.064-----------1:2104060Mod0.0630.0610.047--------0.0680.0590.049------------0.0660.062------------0.0700.072-------1:2104060Str0.0620.0640.049--------0.0730.0660.054------------0.0670.061------------0.0670.077-------1:2105050Mod0.0630.0700.055--------0.0710.0620.055------------0.0650.063------------0.0630.077-------1:2105050Str0.0640.0630.055--------0.0750.0670.053------------0.0640.063------------0.0630.084-------1:2106040Mod0.0590.0630.055--------0.0740.0630.057------------0.0630.064------------0.063-----------1:2106040Str0.0660.0670.055--------0.0730.0630.054------------0.0680.070------------0.066-----------1:22046Mod0.0680.057----0.034--------0.056----0.034--------0.057----0.039--------0.055----0.031---1:22046Str0.0770.058----0.038--------0.051----0.032--------0.054----0.035--------0.065----0.034---1:22055Mod0.0730.055----0.037--------0.058----0.038--------0.057----0.039--------0.056----0.033---1:22055Str0.0750.059----0.041--------0.056----0.037--------0.054----0.036--------0.056----0.033---1:22064Mod0.0680.058----0.039--------0.052----0.040--------0.059----0.040--------0.057----0.036---1:22064Str0.0790.053----0.039--------0.055----0.038--------0.057----0.039--------0.055----0.035---1:220812Mod0.0620.056----0.040----0.0770.056----0.041--------0.056----0.041--------0.062----0.041---1:220812Str0.0630.056----0.043----0.0900.056----0.043--------0.058----0.041--------0.064----0.040---1:2201010Mod0.0630.055----0.042----0.0840.052----0.040--------0.055----0.037--------0.057----0.039---1:2201010Str0.0650.058----0.046----0.0920.060----0.046--------0.057----0.038--------0.058----0.039---1:220128Mod0.0640.053----0.044----0.0850.049----0.040--------0.053----0.042--------0.053----0.038---1:220128Str0.0660.055----0.044----0.0920.058----0.044--------0.059----0.044--------0.055----0.038---Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 244

233 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 28 (continued). Power estimates for conditions when the population effect size variance is 1.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:2204060Mod0.0550.060----0.053----0.0670.053----0.051--------0.053----0.050--------0.059----0.049---1:2204060Str0.0580.059----0.054----0.0700.054----0.049--------0.062----0.055--------0.065----0.051---1:2205050Mod0.0530.054----0.054----0.0750.055----0.053--------0.051----0.050--------0.056----0.048---1:2205050Str0.0580.054----0.050----0.0720.057----0.053--------0.053----0.051--------0.054----0.044---1:2206040Mod0.0550.055----0.048----0.0700.053----0.045--------0.053----0.045--------0.055----0.049---1:2206040Str0.0570.061----0.056----0.0780.055----0.053--------0.058----0.050--------0.054----0.047---1:25046Mod0.0660.051----0.047--------0.053----0.050--------0.050----0.0460.010----0.053----0.0450.013 1:25046Str0.0600.053----0.051--------0.053----0.051--------0.055----0.0470.004----0.064----0.0460.004 1:25055Mod0.0610.054----0.049--------0.051----0.047--------0.054----0.0490.011----0.052----0.0470.014 1:25055Str0.0600.054----0.051--------0.052----0.047--------0.051----0.0460.005----0.057----0.0490.003 1:25064Mod0.0680.056----0.055--------0.052----0.051--------0.052----0.0480.011----0.056----0.0510.012 1:25064Str0.0620.053----0.052--------0.052----0.051--------0.050----0.0470.004----0.051----0.0490.005 1:250812Mod0.0520.053----0.054--------0.050----0.054--------0.051----0.054--------0.057----0.0530.004 1:250812Str0.0570.051----0.058--------0.059----0.059--------0.061----0.055--------0.065----0.0560.002 1:2501010Mod0.0600.055----0.055--------0.048----0.052--------0.053----0.051--------0.054----0.0490.005 1:2501010Str0.0580.053----0.054--------0.054----0.059--------0.059----0.056--------0.066----0.0560.001 1:250128Mod0.0560.052----0.052--------0.054----0.056--------0.050----0.051--------0.050----0.0500.004 1:250128Str0.0550.055----0.059--------0.053----0.056--------0.055----0.054--------0.057----0.0540.002 1:2504060Mod0.0520.051----0.064--------0.051----0.065--------0.057----0.065--------0.062----0.066---1:2504060Str0.0580.052----0.068--------0.055----0.070--------0.058----0.065--------0.065----0.072---1:2505050Mod0.0490.047----0.064--------0.053----0.066--------0.052----0.062--------0.057----0.067---1:2505050Str0.0550.054----0.068--------0.050----0.065--------0.055----0.066--------0.062----0.070---1:2506040Mod0.0530.054----0.063--------0.047----0.061--------0.052----0.064--------0.053----0.063---1:2506040Str0.0510.051----0.066--------0.051----0.063--------0.054----0.063--------0.053----0.059---1:210046Mod0.0560.052----0.051--------0.056----0.0520.020----0.050----0.0500.025----0.054----0.049---1:210046Str0.0600.048----0.050--------0.052----0.0530.018----0.055----0.0500.015----0.066----0.048---1:210055Mod0.0610.049----0.051--------0.048----0.0520.020----0.048----0.0500.027----0.052----0.053---1:210055Str0.0540.049----0.054--------0.051----0.0530.016----0.062----0.0560.013----0.062----0.053---1:210064Mod0.0590.050----0.054--------0.050----0.0520.021----0.049----0.0530.027----0.052----0.055---1:210064Str0.0580.048----0.055--------0.052----0.0560.015----0.053----0.0550.014----0.055----0.053---1:2100812Mod0.0540.053----0.058--------0.051----0.0570.016----0.058----0.0610.012----0.062----0.0550.012 1:2100812Str0.0560.052----0.061--------0.056----0.0620.012----0.069----0.0610.005----0.095----0.0680.005 1:21001010Mod0.0530.047----0.052--------0.051----0.0550.018----0.053----0.0580.014----0.059----0.0570.015 1:21001010Str0.0560.050----0.061--------0.053----0.0580.013----0.066----0.0610.007----0.081----0.0640.005 Population Effect Size 0.8 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5

PAGE 245

234 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 28 (continued). Power estimates for conditions when the population effect size variance is 1.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:2100128Mod0.0510.045----0.054--------0.048----0.056--------0.052----0.0550.014----0.049----0.0540.015 1:2100128Str0.0530.053----0.062--------0.050----0.060--------0.053----0.0560.006----0.062----0.0550.005 1:21004060Mod0.0490.049----0.066--------0.054----0.073--------0.061----0.074--------0.067----0.0780.008 1:21004060Str0.0550.052----0.075--------0.057----0.073--------0.073----0.080--------0.080----0.0830.004 1:21005050Mod0.0540.052----0.074--------0.052----0.068--------0.052----0.067--------0.063----0.0720.008 1:21005050Str0.0510.049----0.068--------0.053----0.069--------0.064----0.077--------0.073----0.0770.005 1:21006040Mod0.0510.052----0.071--------0.056----0.072--------0.052----0.064--------0.054----0.066---1:21006040Str0.0540.053----0.069--------0.051----0.069--------0.058----0.069--------0.057----0.067---1:41046Mod----0.068----------------0.071----------------0.074----------------0.072-----------1:41046Str----0.070----------------0.071----------------0.069----------------0.070-----------1:41055Mod----0.073----------------0.063----------------0.067----------------0.072-----------1:41055Str----0.064----------------0.071----------------0.071----------------0.071-----------1:41064Mod----0.069----------------0.071----------------0.064-------------------------------1:41064Str----0.066----------------0.070----------------0.065-------------------------------1:410812Mod0.0710.0680.071--------0.0810.0710.076------------0.074----------------0.072-----------1:410812Str0.0770.0690.069--------0.0840.0710.074------------0.071----------------0.072-----------1:4101010Mod0.0760.0690.078--------0.0840.065----------------0.071----------------0.065-----------1:4101010Str0.0760.0650.076--------0.0880.070----------------0.070----------------0.066-----------1:410128Mod0.0760.068----------------0.069----------------0.066----------------0.059-----------1:410128Str0.0820.066----------------0.067----------------0.066----------------0.068-----------1:4104060Mod0.0670.0650.0750.032----0.0700.0690.0780.034--------0.068----0.031--------0.071----0.030---1:4104060Str0.0700.0710.0740.033----0.0810.0730.0770.033--------0.068----0.030--------0.070----0.030---1:4105050Mod0.0650.064----0.029----0.0700.0650.074------------0.067----------------0.062----0.027---1:4105050Str0.0670.068----0.029----0.0720.0610.084------------0.063----------------0.061----0.027---1:4106040Mod0.0640.0680.076--------0.0710.066----0.028--------0.062----------------0.063----0.025---1:4106040Str0.0710.0680.082--------0.0750.063----0.028--------0.063----------------0.066----0.028---1:42046Mod0.0730.059----0.037--------0.057----0.037--------0.060----0.036--------0.058----0.034---1:42046Str0.0810.056----0.038--------0.060----0.038--------0.061----0.036--------0.064----0.035---1:42055Mod0.0730.056----0.037--------0.055----0.038--------0.054----0.034--------0.060----0.037---1:42055Str0.0730.053----0.040--------0.057----0.041--------0.057----0.037--------0.055----0.032---1:42064Mod0.0730.062----0.040--------0.057----0.042--------0.057----0.040--------0.058----0.040---1:42064Str0.0770.054----0.043--------0.056----0.041--------0.055----0.039--------0.057----0.037---1:420812Mod0.0630.056----0.044----0.0830.057----0.042--------0.058----0.039--------0.062----0.039---1:420812Str0.0640.059----0.043----0.0880.059----0.044--------0.061----0.043--------0.069----0.044---Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 246

235 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 28 (continued). Power estimates for conditions when the population effect size variance is 1.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:4201010Mod0.0620.054----0.045----0.0860.055----0.042--------0.055----0.041--------0.057----0.040---1:4201010Str0.0650.057----0.047----0.0900.055----0.046--------0.060----0.044--------0.062----0.045---1:420128Mod0.0610.055----0.044--------0.053----0.041--------0.058----0.043--------0.054----0.042---1:420128Str0.0690.056----0.045--------0.054----0.045--------0.054----0.046--------0.054----0.042---1:4204060Mod0.0520.056----0.061----0.0720.060----0.060--------0.062----0.061--------0.061----0.058---1:4204060Str0.0560.055----0.063----0.0720.059----0.064--------0.061----0.061--------0.065----0.060---1:4205050Mod0.0580.054----0.056----0.0720.055----0.056--------0.060----0.056--------0.054----0.054---1:4205050Str0.0600.058----0.059----0.0770.060----0.061--------0.055----0.056--------0.061----0.059---1:4206040Mod0.0590.054----0.051----0.0800.054----0.051--------0.056----0.052--------0.055----0.051---1:4206040Str0.0610.053----0.057----0.0800.055----0.053--------0.054----0.055--------0.052----0.048---1:45046Mod0.0590.048----0.041--------0.053----0.047--------0.054----0.0470.013----0.059----0.047---1:45046Str0.0620.053----0.050--------0.052----0.049--------0.058----0.0490.006----0.073----0.051---1:45055Mod0.0620.052----0.050--------0.049----0.048--------0.053----0.0460.012----0.055----0.0480.017 1:45055Str0.0630.049----0.048--------0.052----0.051--------0.053----0.0480.006----0.056----0.0460.007 1:45064Mod0.0610.051----0.053--------0.055----0.054--------0.052----0.0510.014----0.050----0.0480.017 1:45064Str0.0590.052----0.054--------0.050----0.054--------0.050----0.0490.006----0.054----0.0520.009 1:450812Mod0.0590.055----0.058--------0.050----0.056--------0.057----0.053--------0.067----0.0590.008 1:450812Str0.0630.053----0.059--------0.059----0.061--------0.067----0.063--------0.086----0.0640.003 1:4501010Mod0.0570.054----0.054--------0.055----0.061--------0.054----0.056--------0.054----0.0510.009 1:4501010Str0.0600.050----0.057--------0.051----0.056--------0.060----0.060--------0.067----0.0550.002 1:450128Mod0.0590.051----0.058--------0.048----0.052--------0.052----0.056--------0.048----0.0500.009 1:450128Str0.0570.051----0.059--------0.051----0.057--------0.052----0.056--------0.054----0.0520.003 1:4504060Mod0.0550.052----0.073--------0.055----0.071--------0.059----0.078--------0.067----0.081---1:4504060Str0.0510.049----0.071--------0.062----0.079--------0.063----0.080--------0.074----0.088---1:4505050Mod0.0550.052----0.072--------0.052----0.070--------0.051----0.067--------0.057----0.073---1:4505050Str0.0550.055----0.076--------0.050----0.067--------0.058----0.072--------0.064----0.076---1:4506040Mod0.0510.050----0.068--------0.049----0.067--------0.051----0.068--------0.047----0.062---1:4506040Str0.0550.050----0.072--------0.051----0.069--------0.054----0.070--------0.056----0.069---1:410046Mod0.0590.050----0.049--------0.053----0.0530.024----0.056----0.052--------0.061----0.048---1:410046Str0.0600.054----0.052--------0.056----0.0520.018----0.071----0.055--------0.087----0.056---1:410055Mod0.0600.050----0.054--------0.051----0.0510.024----0.051----0.050--------0.053----0.050---1:410055Str0.0550.050----0.050--------0.052----0.0550.019----0.055----0.052--------0.065----0.053---1:410064Mod0.0560.050----0.054--------0.051----0.0590.025----0.054----0.058--------0.054----0.058---1:410064Str0.0580.051----0.056--------0.054----0.0610.019----0.051----0.055--------0.052----0.052---Population Effect Size 0.8 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5

PAGE 247

236 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 28 (continued). Power estimates for conditions when the population effect size variance is 1.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:4100812Mod0.0560.054----0.061--------0.050----0.0580.018----0.064----0.0610.017----0.077----0.066---1:4100812Str0.0560.053----0.062--------0.054----0.0590.009----0.082----0.0690.007----0.110----0.078---1:41001010Mod0.0550.051----0.059--------0.058----0.0650.018----0.052----0.0580.020----0.065----0.060---1:41001010Str0.0600.054----0.065--------0.056----0.0630.013----0.068----0.0630.009----0.086----0.067---1:4100128Mod0.0550.054----0.064--------0.046----0.0560.019----0.049----0.0580.021----0.049----0.056---1:4100128Str0.0550.049----0.059--------0.049----0.0620.015----0.055----0.0600.011----0.063----0.061---1:41004060Mod0.0520.052----0.077--------0.052----0.078--------0.064--------0.013----0.080--------0.015 1:41004060Str0.0540.050----0.076--------0.058----0.079--------0.077--------0.007----0.100--------0.007 1:41005050Mod0.0500.049----0.076--------0.051----0.078--------0.058----0.0790.015----0.066----0.0810.017 1:41005050Str0.0500.054----0.078--------0.054----0.081--------0.066----0.0880.009----0.074----0.0940.011 1:41006040Mod0.0530.050----0.071--------0.052----0.075--------0.052----0.0680.015----0.053----0.0680.016 1:41006040Str0.0550.055----0.078--------0.052----0.080--------0.058----0.0790.012----0.058----0.0720.012 1:81046Mod----0.071----------------0.068----------------0.070----------------0.072-----------1:81046Str----0.072----------------0.067----------------0.071----------------0.080-----------1:81055Mod----0.068----------------0.069----------------0.068----------------0.065-----------1:81055Str----0.070----------------0.075----------------0.069----------------0.071-----------1:81064Mod----0.065----------------0.065----------------0.071----------------0.062-----------1:81064Str----0.068----------------0.063----------------0.064----------------0.068-----------1:810812Mod0.0730.071------------0.0850.072----------------0.070----------------0.072-----------1:810812Str0.0820.074------------0.0860.065----------------0.073----------------0.076-----------1:8101010Mod----0.063----------------0.071----------------0.070----------------0.067-----------1:8101010Str----0.071----------------0.068----------------0.068----------------0.069-----------1:810128Mod----0.066----------------0.066----------------0.065----------------0.066-----------1:810128Str----0.068----------------0.068----------------0.062----------------0.065-----------1:8104060Mod0.0690.069----0.038----0.0680.068----0.034--------0.072----0.038--------0.073----0.042---1:8104060Str0.0680.069----0.038----0.0780.065----0.038--------0.073----0.039--------0.071----0.041---1:8105050Mod0.0670.063----0.033----0.0690.060----0.034--------0.067----0.035--------0.066----0.037---1:8105050Str0.0700.062----0.036----0.0840.070----0.039--------0.066----0.037--------0.070----0.035---1:8106040Mod0.0700.067----0.036----0.0800.063----0.034--------0.069----0.033--------0.060----0.028---1:8106040Str0.0690.064----0.032----0.0820.065----0.035--------0.057----0.032--------0.062----0.031---1:82046Mod----0.057----0.037--------0.061----0.039--------0.058----0.034--------0.060----0.033---1:82046Str----0.061----0.042--------0.056----0.036--------0.060----0.035--------0.066----0.035---1:82055Mod0.0760.055----0.039--------0.051----0.033--------0.052----0.036--------0.055----0.036---1:82055Str0.0740.056----0.038--------0.053----0.036--------0.059----0.039--------0.059----0.038---Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 248

237 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 28 (continued). Power estimates for conditions when the population effect size variance is 1.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:82064Mod0.0800.053----0.042--------0.051----0.040--------0.057----0.041--------0.061----0.043---1:82064Str0.0800.054----0.042--------0.052----0.043--------0.056----0.042--------0.057----0.042---1:820812Mod0.0630.057----0.045--------0.055----0.046--------0.061----0.047--------0.065----0.045---1:820812Str0.0650.062----0.052--------0.056----0.048--------0.066----0.048--------0.070----0.047---1:8201010Mod0.0660.054----0.046--------0.056----0.047--------0.059----0.048--------0.058----0.042---1:8201010Str0.0650.055----0.047--------0.057----0.049--------0.060----0.048--------0.062----0.048---1:820128Mod0.0660.051----0.046----0.0900.054----0.047--------0.051----0.040--------0.057----0.047---1:820128Str0.0690.054----0.050----0.1050.057----0.049--------0.056----0.050--------0.058----0.044---1:8204060Mod0.0590.057----0.065----0.0750.058----0.069--------0.062----0.072--------0.065----0.074---1:8204060Str0.0600.057----0.069----0.0760.057----0.068--------0.061----0.074--------0.068----0.076---1:8205050Mod0.0580.055----0.064----0.0780.060----0.066--------0.054----0.065--------0.060----0.064---1:8205050Str0.0610.057----0.070----0.0830.054----0.064--------0.059----0.066--------0.058----0.070---1:8206040Mod0.0560.057----0.063----0.0800.049----0.058--------0.055----0.061--------0.052----0.057---1:8206040Str0.0590.052----0.063----0.0860.058----0.066--------0.055----0.064--------0.053----0.057---1:85046Mod0.0630.055----0.049--------0.053----0.047--------0.058----0.0460.015----0.065----0.047---1:85046Str0.0630.052----0.049--------0.050----0.048--------0.062----0.0490.008----0.074----0.051---1:85055Mod0.0620.058----0.055--------0.050----0.050--------0.052----0.0460.018----0.055----0.045---1:85055Str0.0670.052----0.049--------0.054----0.052--------0.055----0.0500.012----0.059----0.047---1:85064Mod0.0650.047----0.053--------0.053----0.058--------0.052----0.0550.022----0.052----0.056---1:85064Str0.0650.055----0.058--------0.050----0.055--------0.049----0.0560.011----0.053----0.051---1:850812Mod0.0570.051----0.055--------0.054----0.058--------0.059----0.0570.010----0.070----0.0600.011 1:850812Str0.0600.052----0.063--------0.060----0.064--------0.071----0.0640.004----0.088----0.0660.004 1:8501010Mod0.0650.056----0.060--------0.053----0.060--------0.052----0.0550.010----0.056----0.0540.014 1:8501010Str0.0580.053----0.062--------0.053----0.060--------0.064----0.0630.006----0.067----0.0580.007 1:850128Mod0.0610.054----0.060--------0.048----0.057--------0.053----0.0580.010----0.050----0.0560.017 1:850128Str0.0590.049----0.060--------0.051----0.065--------0.053----0.0570.007----0.053----0.0550.009 1:8504060Mod0.0550.050----0.085--------0.053----0.087--------0.062----------------0.074-----------1:8504060Str0.0540.055----0.086--------0.053----0.084--------0.074----------------0.084-----------1:8505050Mod0.0510.054----0.082--------0.052----0.081--------0.050----0.075--------0.059-----------1:8505050Str0.0550.053----0.083--------0.049----0.081--------0.064----0.091--------0.066-----------1:8506040Mod0.0540.051----0.078--------0.051----0.073--------0.046----0.073--------0.051----0.073---1:8506040Str0.0540.053----0.081--------0.052----0.079--------0.054----0.076--------0.051----0.073---1:810046Mod0.0580.050----0.049--------0.054----0.0500.025----0.060----0.053--------0.074----0.054---1:810046Str0.0580.054----0.055--------0.054----0.0510.019----0.069----0.055--------0.102----0.058---Population Effect Size 0.8 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5

PAGE 249

238 Appendix D (continued): Power estimates for c onditions with adequate Type I error rates Table 28 (continued). Power estimates for conditions when the population effect size variance is 1.00. s2kn1n2Begg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrimBegg V Begg N EggerFunnelTrim 1:810055Mod0.0610.053----0.053--------0.050----0.0540.028----0.055----0.052--------0.049----0.048---1:810055Str0.0570.052----0.052--------0.051----0.0540.022----0.055----0.049--------0.067----0.054---1:810064Mod0.0560.050----0.061--------0.050----0.0620.029----0.055----0.065--------0.060-----------1:810064Str0.0590.048----0.060--------0.050----0.0590.023----0.048----0.056--------0.048-----------1:8100812Mod0.0560.051----0.062--------0.051----0.0590.020----0.068----0.0680.027----0.090----0.075---1:8100812Str0.0580.054----0.065--------0.056----0.0610.017----0.092----0.0770.015----0.140----0.094---1:81001010Mod0.0540.049----0.061--------0.053----0.0610.022----0.057----0.0590.028----0.064----0.061---1:81001010Str0.0510.052----0.064--------0.057----0.0670.015----0.073----0.0730.017----0.091----0.075---1:8100128Mod0.0570.047----0.063--------0.049----0.0610.020----0.051----0.0610.029----0.047----0.057---1:8100128Str0.0600.055----0.070--------0.052----0.0650.019----0.051----0.0620.021----0.057----0.066---1:81004060Mod0.0530.051----------------0.055----------------0.072--------0.022----0.096--------0.025 1:81004060Str0.0530.051----------------0.057----------------0.090--------0.016----0.120--------0.016 1:81005050Mod0.0500.046----------------0.057----0.089--------0.059--------0.025----0.069--------0.032 1:81005050Str0.0530.054----------------0.059----0.093--------0.072--------0.019----0.087--------0.022 1:81006040Mod0.0530.050----------------0.052----------------0.052--------0.024----0.051--------0.032 1:81006040Str0.0520.049----------------0.052----------------0.054--------0.016----0.056--------0.024 Pub Bias Population Effect Size 0.0 Population Effect Size 0.2 Population Effect Size 0.5 Population Effect Size 0.8

PAGE 250

ABOUT THE AUTHOR Gianna Rendina-Gobioff received a bachel or’s degree (BA) in Psychology from the University of South Florida in 1997 and a master’s degree (M.E d.) in Curriculum and Instruction with an emphasis in Measurement and Evaluation from the University of South Florida in 2001. She worked in the Research and Accountability Department of the Pinellas County School District for five years. In addition, she has worked as a measurement consultant on a St ate level project and a rese arch assistant on several evaluation projects contracted with the Cent er for Research, Evaluation, Assessment, and Measurement in the College of Education at the University of South Florida. In 2004 she was honored with the privilege of participating in the summ er internship program at Educational Testing Service. Her teaching experience ha s included undergraduate and masters level measurement course development and delivery (traditional and online).