SimReg: a software including some new developments in multiple comparison and simultaneous confidence bands for linear regression models (original) (raw)
Related papers
Journal of Statistical Software, 2005
The problem of simultaneous inference and multiple comparison for comparing means of k(≥ 3) populations has been long studied in the statistics literature and is widely available in statistics literature. However to-date, the problem of multiple comparison of regression models has not found its way to the software. It is only recently that the computational aspects of this problem have been resolved in a general setting. SimReg employs this new methodology and provides users with software for multiple regression of several regression models. The comparisons can be among any set of pairs, and moreover any number of predictors can be included in the model. More importantly predictors can be constrained to their natural boundaries, if known. Computational methods for the problem of simultaneous confidence bands when predictors are constrained to intervals has also recently been addressed. SimReg utilizes this recent development to offer simultaneous confidence bands for regression models with any number of predictor variables. Again, the predictors can be constrained to their natural boundaries which results in narrower bands, as compared to the case where no restriction is imposed. A by-product of these confidence bands is a new method for comparing two regression surfaces, that is more informative than the usual partial F test.
Some new methods for the comparison of two linear regression models
Journal of Statistical Planning and Inference, 2007
The frequently used approach to the comparison of two linear regression models is to use the partial F test. It is pointed out in this paper that the partial F test has in fact a naturally associated two-sided simultaneous confidence band, which is much more informative than the test itself. But this confidence band is over the entire range of all the covariates. As regression models are true or of interest often only over a restricted region of the covariates, the part of this confidence band outside this region is therefore useless and to ensure 1 − simultaneous coverage probability is therefore wasteful of resources. It is proposed that a narrower and hence more efficient confidence band over a restricted region of the covariates should be used. The critical constant required in the construction of this confidence band can be calculated by Monte Carlo simulation. While this two-sided confidence band is suitable for two-sided comparisons of two linear regression models, a more efficient one-sided confidence band can be constructed in a similar way if one is only interested in assessing whether the mean response of one regression model is higher (or lower) than that of the other in the region. The methodologies are illustrated with two examples.
A study of partial F tests for multiple linear regression models
Computational Statistics & Data Analysis, 2007
This paper studies the partial F tests from the view point of simultaneous confidence bands. It first shows that there is a simultaneous confidence band associated naturally with a partial F test. This confidence band provides more information than the partial F test and the partial F test can be regarded as a side product of the confidence band. This view point of confidence bands also leads to insights of the major weakness of the partial F tests, that is, a partial F test requires implicitly the linear regression model holds over the entire range of the covariates in concern. Improved tests are proposed and they are induced by simultaneous confidence bands over restricted regions of the covariates.
The multcomp package for the R statistical environment allows for multiple comparisons of parameters whose estimates are generally correlated, including comparisons of k groups in general linear models.
Some General Theory of Multiple Comparison Procedures
2008
Chapter 2 was devoted to the theory of multiple comparison procedures (MCPs) for fixed-effects linear models with independent homoscedastic normal errors, which was the framework for Part I. Much of that theory applies with minor modifications to many of the problems considered in Part 11. However, in other cases the theory of Chapter 2 needs to be supplemented and extended, which is the purpose of the present appendix. We assume that the reader is familiar with Chapter 2. Many references to that chapter are made in the sequel. As in Chapter 2, throughout this appendix we restrict to the nonsequential (fixed-sample) setting. The following is a summary of this appendix. Section 1 discusses the theory of simultaneous test procedures in arbitrary models. This discussion is based mostly on Gabriel (1969). When a simultaneous test procedure (and more generally a single-step test procedure) addresses hypotheses concerning parametric functions, it can be inverted to obtain a simultaneous confidence procedure for those parametric functions. Conversely, from a given simultaneous confidence procedure one can obtain the associated simultaneous test procedure by applying the confidenceregion test method. The relation between simultaneous confidence estimation and simultaneous testing is the topic of Section 2. Finally Section 3 discusses some theory of step-down test procedures, including the topics of error rate control, optimal choice of nominal significance levels, and directional decisions. Here no general theory for deriving the associated simultaneous confidence estimates is as yet available; some preliminary work in this direction by Kim, Stefhsson, and Hsu (1987) is discussed in Section 4.2.4 of Chapter 2.
Educational and Psychological Measurement, 2001
In this article, the authors introduce a computer package written for Mathematica, the purpose of which is to perform a number of difficult iterative functions with respect to the squared multiple correlation coefficient under the fixed and random models. These functions include, among others, computation of confidence interval upper and lower bounds, power calculation, calculation of sample size required for a specified power level, and providing estimates of shrinkage in cross validating the squared multiple correlation under both the random and fixed models. Attention is given to some of the technical issues regarding the selection of, and working with, these two types of models as well as to issues concerning the construction of confidence intervals. Much emphasis has been placed lately on the importance of using confidence intervals in data analysis in addition to or as opposed to performing null hypothesis testing. Some authors have gone so far as to suggest the replacement or banishment of hypothesis testing (e.g., Schmidt, 1996). In the replacement of hypothesis testing, it is argued that parameter estimates be accompanied by their margin of error-confidence intervals. Clearly, this recommendation is especially valid when sample size is either very large or very small. Large sample sizes can produce statistical tests that are overly sensitive and, thus, lead to the finding of statistically significant differences where only minuscule differences between parameters actually exist. On the other hand, small samples produce tests that are not very sensitive and lead to the detection of only large differences between the parameters. There are other reasons why these authors have argued against tests of hypotheses, but we
Multivariate Behavioral Research
Whenever multiple regression is applied to a multiply imputed data set, several methods for combining significance tests for R 2 and the change in R 2 across imputed data sets may be used: the combination rules by Rubin, the Fisher z-test for R 2 by Harel, and F-tests for the change in R 2 by Chaurasia and Harel. For pooling R 2 itself, Harel proposed a method based on a Fisher z transformation. In the current article, it is argued that the pooled R 2 based on the Fisher z transformation, the Fisher z-test for R 2 , and the F-test for the change in R 2 have some theoretical flaws. An argument is made for using Rubin's method for pooling significance tests for R 2 instead, and alternative procedures for pooling R 2 are proposed: simple averaging and a pooled R 2 constructed from the pooled significance test by Rubin. Simulations show that the Fisher z-test and Chaurasia and Harel's F-tests generally give inflated type-I error rates, whereas the type-I error rates of Rubin's method are correct. Of the methods for pooling the point estimates of R 2 no method clearly performs best, but it is argued that the average of R 2 's across imputed data set is preferred.
Computer Methods and Programs in Biomedicine, 2010
The objective of the method described in this paper is to develop a spreadsheet template for the purpose of comparing multiple sample means. An initial analysis of variance (ANOVA) test on the data returns F-the test statistic. If F is larger than the critical F value drawn from the F distribution at the appropriate degrees of freedom, convention dictates rejection of the null hypothesis and allows subsequent multiple comparison testing to determine where the inequalities between the sample means lie. A variety of multiple comparison methods are described that return the 95% confidence intervals for differences between means using an inclusive pairwise comparison of the sample means.
Confidence Intervals in Generalized Regression Models by Esa Uusipaikka
International Statistical Review, 2009
Table of contents 1. The nature of research 10. Between-subjects factorial experiments: factors 2. Principals of experimental design with more than two levels 3. The standard normal distribution: an amazing 11. Between-subjects factorial experiments: further approximation considerations 4. Tests for means from random samples 12. Within-subjects factors: one-way and 2 k factorial 5. Homogeneity and normality assumptions designs 6. The analysis of variance: one between-subjects 13. Within-subjects factors: general designs factor 14. Contrasts on binomial data: between-subject 7. Pairwise comparisons designs 8. Orthogonal, planned and unplanned comparisons 15. Debriefing 9. The 2 k between-subjects factorial experiment Appendix A. The method of least squares Appendix B. Statistical tables
Version 2.0-2 Date 2010/07/30 Title Companion to Applied Regression
2010
Calculates type-II or type-III analysis-of-variance tables for model objects produced by lm, glm, multinom (in the nnet package), polr (in the MASS package), coxph (in the survival package), coxme (in the coxme pckage), svyglm (in the survey package), rlm (in the MASS package), lmer in the lme4 package, lme in the nlme package, and (by the default method) for most models with a linear predictor and asymptotically normal coefficients (see details below). For linear models, F-tests are calculated; for generalized linear models, likelihood-ratio chisquare, Wald chisquare, or F-tests are calculated; for multinomial logit and proportional-odds logit models, likelihood-ratio tests are calculated. Various test statistics are provided for multivariate linear models produced by lm or manova. Partial-likelihood-ratio tests or Wald tests are provided for Cox models. Wald chi-square tests are provided for fixed effects in linear and generalized linear mixed-effects models. Wald chi-square or F tests are provided in the default case.