H. Keselman - Academia.edu (original) (raw)
Papers by H. Keselman
Seven test statistics known to be robust to the combined effects of nonnormality and variance het... more Seven test statistics known to be robust to the combined effects of nonnormality and variance heterogeneity were compared for their sensitivity to detect treatment effects in a one-way completely randomized design containing four groups. The six Welch-James-type heteroscedastic tests adopted either symmetric or asymmetric trimmed means, were transformed for skewness, and used a bootstrap method to assess statistical significance. The remaining test, due to Wilcox and Keselman , used a modification of the well-known one-step M-estimator of central tendency rather than trimmed means. The Welch-James-type test is recommended because for nonnormal data likely to be encountered in applied research settings it should be more powerful than the test presented by Wilcox and Keselman. However, the reverse is true for data that are extremely nonnormal.
British Journal of Mathematical and Statistical Psychology, 2001
Repeated measures ANOVA can refer to many different types of analysis. Speci®cally, this vague te... more Repeated measures ANOVA can refer to many different types of analysis. Speci®cally, this vague term can refer to conventiona l tests of signi®cance, one of three univariate solutions with adjusted degrees of freedom, two different types of multivariate statistic, or approache s that combine univariate and multivariate tests. Accordingly, it is argued that, by only reporting probability values and referring to statistical analyses as repeated measures ANOVA, authors convey neither the type of analysis that was used nor the validity of the reported probability value, since each of these approache s has its own strengths and weaknesses. The various approache s are presented with a discussion of their strengths and weaknesses, and recommendations are made regarding the`best' choice of analysis. Additional topics discussed include analyses for missing data and tests of linear contrasts.
Review of Educational Research, 1998
Psychometrika, 1998
Three approaches to the analysis of main and interaction effect hypotheses in nonorthogonal desig... more Three approaches to the analysis of main and interaction effect hypotheses in nonorthogonal designs were compared in a 2 2 design for data that was neither normal ‚ in form nor equal in variance. The approaches involved either least squares or robust estimators of central tendency and variability and/or a test statistic that either pools or does not pool sources of variance. Specifically, we compared the ANOVA F test which used trimmed means and Winsorized variances, the Welch-James test with the usual least squares estimators for central tendency and variability and the Welch-James test using trimmed means and Winsorized variances. As hypothesized, we found that the latter approach provided excellent Type I error control, whereas the former two did not.
Psychological Methods, 1998
... validity of these procedures when derivational as-sumptions are not satisfied (eg, see Hochbe... more ... validity of these procedures when derivational as-sumptions are not satisfied (eg, see Hochberg & Tamhane, 1987; Shaffer, 1994; Toothaker ... HJ Keselman, Lisa M. Lix, and Rhonda K. Kowalchuk, Department of Psychology, University of Manitoba, Win-nipeg, Manitoba, Canada ...
Psychological Methods, 2001
One approach to the analysis of repeated measures data allows researchers to model the covariance... more One approach to the analysis of repeated measures data allows researchers to model the covariance structure of the data rather than presume a certain structure, as is the case with conventional univariate and multivariate test statistics. This mixed-model approach was evaluated for testing all possible pairwise differences among repeated measures marginal means in a Between-Subjects x Within-Subjects design. Specifically, the authors investigated Type I error and power rates for a number of simultaneous and stepwise multiple comparison procedures using SAS (1999) PROC MIXED in unbalanced designs when normality and covariance homogeneity assumptions did not hold. J. P. Shaffer's (1986) sequentially rejective step-down and Y. Hochberg's (1988) sequentially acceptive step-up Bonferroni procedures, based on an unstructured covariance structure, had superior Type I error control and power to detect true pairwise differences across the investigated conditions.
Multivariate Behavioral Research, 2002
Educational and Psychological Measurement, 2004
... Correspondence concerning this article should be ad-dressed to Rhonda K. Kowalchuk, Departmen... more ... Correspondence concerning this article should be ad-dressed to Rhonda K. Kowalchuk, Department of Educational Psychology, University of Wis-consin-Milwaukee, PO Box 413 ... 64 No. 2, April 2004 224-242 DOI: 10.1177/0013164403260196 © 2004 Sage Publications 224 ...
Communications in Statistics-theory and Methods, 1999
Mixed-model analysis is the newest approach to the analysis of repeated measurements. The approac... more Mixed-model analysis is the newest approach to the analysis of repeated measurements. The approach is supposed to be advantageous (i.e., efficient and powerful) because it allows users to model the covariance structure of their data prior to assessing treatment effects. The statistics for this method are based on an F-distribution with degrees of freedom often just approximated by the residual
Communications in Statistics-simulation and Computation, 2000
Tests for mean equality proposed by Weerahandi (1995) and Chen and Chen (1998), tests that do not... more Tests for mean equality proposed by Weerahandi (1995) and Chen and Chen (1998), tests that do not require equality of population variances, were examined when data were not only heterogeneous but, as well, nonnormal in unbalanced completely randomized designs. Furthermore, these tests were compared to a test examined by Lix and Keselman (1998), a test that uses a heteroscedastic statistic
Communications in Statistics - Simulation and Computation, 1998
The mixed model approach to the analysis of repeated measurements allows users to model the covar... more The mixed model approach to the analysis of repeated measurements allows users to model the covariance structure of their data. That is, rather than using a univariate or a multivariate test statistic for analyzing effects, tests that assume a particular form for the covariance structure, the mixed model approach allows the data to determine the appropriate structure. Using the appropriate covariance structure should result in more powerful tests of the repeated measures effects according to advocates of the mixed model approach. SAS' (1996) mixed model program, PROC MIXED, provides users with two information criteria for selecting the `best' covariance structure, and Schwarz (1978). Our study compared these log likelihood tests to see how effective they would be for detecting various population covariance structures. In particular, the criteria were compared in unbalanced (across groups) nonspherical repeated measures designs having equal/unequal group sizes and covariance matrices when data were both normally and nonnormally distributed. The results indicate that neither criterion was effective in finding the correct structure. On average, for the 26 investigated distributions, the Akaike criterion only resulted in the correct structure being selected 47 percent of the time while the Schwarz criterion resulted in the correct structure being selected just 35 percent of the time. Not surprisingly, PROC MIXED default F-tests based on either of these selection criteria performed poorly according to results reported by the authors elsewhere.
British Journal of Mathematical and Statistical Psychology, 1999
ABSTRACT Looney & Stanley's (1989) recommendations regarding analysis strategies ... more ABSTRACT Looney & Stanley's (1989) recommendations regarding analysis strategies for repeated measures designs containing between-subjects grouping variables and within-subjects repeated measures variables were re-examined and compared to recent analysis strategies. That is, corrected degrees of freedom univariate tests, multivariate tests, mixed model tests, and tests due to Keselman, Carriere & Lix (1993) and to Algina (1994), Huynh (1978) and Lecoutre (1991) were compared for rates of Type I error in unbalanced non-spherical repeated measures designs having varied covariance structures and no missing data on the within-subjects variable. Heterogeneous within-subjects and heterogeneous within- and between-subjects structures were investigated along with multivariate non-normality. Results indicated that the tests due to Keselman et al. and Algina, Huynh and Lecoutre provided effective Type I error control whereas the default mixed model approach computed with PROC MIXED (SAS Institute, 1995) generally did not. Based on power differences, we recommend that applied researchers adopt the Welch-James type test described by Keselman et al.
British Journal of Mathematical and Statistical Psychology, 2000
In 1987, Jennings enumerated data analysis procedures that authors must follow for analyzing effe... more In 1987, Jennings enumerated data analysis procedures that authors must follow for analyzing effects in repeated measures designs when submitting papers to Psychophysiology. These prescriptions were intended to counteract the effects of nonspherical data, a condition know to produce biased tests of significance. Since this editorial policy was established, additional refinements to the analysis of these designs have appeared in print in a number of sources that are not likely to be routinely read by psychophysiological researchers. Accordingly, this paper includes additional procedures not previously enumerated in the editorial policy that can be used to analyze repeated measurements. Furthermore, I indicate how numerical solutions can easily be obtained.
British Journal of Mathematical and Statistical Psychology, 2001
Repeated measures ANOVA can refer to many different types of analysis. Speci®cally, this vague te... more Repeated measures ANOVA can refer to many different types of analysis. Speci®cally, this vague term can refer to conventiona l tests of signi®cance, one of three univariate solutions with adjusted degrees of freedom, two different types of multivariate statistic, or approache s that combine univariate and multivariate tests. Accordingly, it is argued that, by only reporting probability values and referring to statistical analyses as repeated measures ANOVA, authors convey neither the type of analysis that was used nor the validity of the reported probability value, since each of these approache s has its own strengths and weaknesses. The various approache s are presented with a discussion of their strengths and weaknesses, and recommendations are made regarding the`best' choice of analysis. Additional topics discussed include analyses for missing data and tests of linear contrasts.
Biometrical Journal, 2002
We compared three tests for mean equality: the Welch (1938) heteroscedastic statistic, the Zhou e... more We compared three tests for mean equality: the Welch (1938) heteroscedastic statistic, the Zhou et al. (1997) test, derived to be used with skewed lognormal data, and Yuen's (1974) procedure which uses robust estimators of central tendency and variability with the Welch test in order to combat the combined effects of nonnormality and variance heterogeneity. Over the 162 conditions of nonnormality and variance heterogeneity we investigated, only the Yuen procedure reliably controlled its rate of Type I error.
American Statistician, 2000
British Journal of Mathematical & Statistical Psychology, 2000
Boik (1997) presented an empirical ,Bayes (EB) approach ,to the ,analysis of repeated measurement... more Boik (1997) presented an empirical ,Bayes (EB) approach ,to the ,analysis of repeated measurements. The EB approach,is a blend,of the conventional univariate and multivariate approaches. Specifically, in the EB approach, the underlying covariance matrix is estimated by a ,weighted sum of the univariate and multivariate estimators. In addition to demonstrating, that his approach ,controls test size and frequently is
Seven test statistics known to be robust to the combined effects of nonnormality and variance het... more Seven test statistics known to be robust to the combined effects of nonnormality and variance heterogeneity were compared for their sensitivity to detect treatment effects in a one-way completely randomized design containing four groups. The six Welch-James-type heteroscedastic tests adopted either symmetric or asymmetric trimmed means, were transformed for skewness, and used a bootstrap method to assess statistical significance. The remaining test, due to Wilcox and Keselman , used a modification of the well-known one-step M-estimator of central tendency rather than trimmed means. The Welch-James-type test is recommended because for nonnormal data likely to be encountered in applied research settings it should be more powerful than the test presented by Wilcox and Keselman. However, the reverse is true for data that are extremely nonnormal.
British Journal of Mathematical and Statistical Psychology, 2001
Repeated measures ANOVA can refer to many different types of analysis. Speci®cally, this vague te... more Repeated measures ANOVA can refer to many different types of analysis. Speci®cally, this vague term can refer to conventiona l tests of signi®cance, one of three univariate solutions with adjusted degrees of freedom, two different types of multivariate statistic, or approache s that combine univariate and multivariate tests. Accordingly, it is argued that, by only reporting probability values and referring to statistical analyses as repeated measures ANOVA, authors convey neither the type of analysis that was used nor the validity of the reported probability value, since each of these approache s has its own strengths and weaknesses. The various approache s are presented with a discussion of their strengths and weaknesses, and recommendations are made regarding the`best' choice of analysis. Additional topics discussed include analyses for missing data and tests of linear contrasts.
Review of Educational Research, 1998
Psychometrika, 1998
Three approaches to the analysis of main and interaction effect hypotheses in nonorthogonal desig... more Three approaches to the analysis of main and interaction effect hypotheses in nonorthogonal designs were compared in a 2 2 design for data that was neither normal ‚ in form nor equal in variance. The approaches involved either least squares or robust estimators of central tendency and variability and/or a test statistic that either pools or does not pool sources of variance. Specifically, we compared the ANOVA F test which used trimmed means and Winsorized variances, the Welch-James test with the usual least squares estimators for central tendency and variability and the Welch-James test using trimmed means and Winsorized variances. As hypothesized, we found that the latter approach provided excellent Type I error control, whereas the former two did not.
Psychological Methods, 1998
... validity of these procedures when derivational as-sumptions are not satisfied (eg, see Hochbe... more ... validity of these procedures when derivational as-sumptions are not satisfied (eg, see Hochberg & Tamhane, 1987; Shaffer, 1994; Toothaker ... HJ Keselman, Lisa M. Lix, and Rhonda K. Kowalchuk, Department of Psychology, University of Manitoba, Win-nipeg, Manitoba, Canada ...
Psychological Methods, 2001
One approach to the analysis of repeated measures data allows researchers to model the covariance... more One approach to the analysis of repeated measures data allows researchers to model the covariance structure of the data rather than presume a certain structure, as is the case with conventional univariate and multivariate test statistics. This mixed-model approach was evaluated for testing all possible pairwise differences among repeated measures marginal means in a Between-Subjects x Within-Subjects design. Specifically, the authors investigated Type I error and power rates for a number of simultaneous and stepwise multiple comparison procedures using SAS (1999) PROC MIXED in unbalanced designs when normality and covariance homogeneity assumptions did not hold. J. P. Shaffer's (1986) sequentially rejective step-down and Y. Hochberg's (1988) sequentially acceptive step-up Bonferroni procedures, based on an unstructured covariance structure, had superior Type I error control and power to detect true pairwise differences across the investigated conditions.
Multivariate Behavioral Research, 2002
Educational and Psychological Measurement, 2004
... Correspondence concerning this article should be ad-dressed to Rhonda K. Kowalchuk, Departmen... more ... Correspondence concerning this article should be ad-dressed to Rhonda K. Kowalchuk, Department of Educational Psychology, University of Wis-consin-Milwaukee, PO Box 413 ... 64 No. 2, April 2004 224-242 DOI: 10.1177/0013164403260196 © 2004 Sage Publications 224 ...
Communications in Statistics-theory and Methods, 1999
Mixed-model analysis is the newest approach to the analysis of repeated measurements. The approac... more Mixed-model analysis is the newest approach to the analysis of repeated measurements. The approach is supposed to be advantageous (i.e., efficient and powerful) because it allows users to model the covariance structure of their data prior to assessing treatment effects. The statistics for this method are based on an F-distribution with degrees of freedom often just approximated by the residual
Communications in Statistics-simulation and Computation, 2000
Tests for mean equality proposed by Weerahandi (1995) and Chen and Chen (1998), tests that do not... more Tests for mean equality proposed by Weerahandi (1995) and Chen and Chen (1998), tests that do not require equality of population variances, were examined when data were not only heterogeneous but, as well, nonnormal in unbalanced completely randomized designs. Furthermore, these tests were compared to a test examined by Lix and Keselman (1998), a test that uses a heteroscedastic statistic
Communications in Statistics - Simulation and Computation, 1998
The mixed model approach to the analysis of repeated measurements allows users to model the covar... more The mixed model approach to the analysis of repeated measurements allows users to model the covariance structure of their data. That is, rather than using a univariate or a multivariate test statistic for analyzing effects, tests that assume a particular form for the covariance structure, the mixed model approach allows the data to determine the appropriate structure. Using the appropriate covariance structure should result in more powerful tests of the repeated measures effects according to advocates of the mixed model approach. SAS' (1996) mixed model program, PROC MIXED, provides users with two information criteria for selecting the `best' covariance structure, and Schwarz (1978). Our study compared these log likelihood tests to see how effective they would be for detecting various population covariance structures. In particular, the criteria were compared in unbalanced (across groups) nonspherical repeated measures designs having equal/unequal group sizes and covariance matrices when data were both normally and nonnormally distributed. The results indicate that neither criterion was effective in finding the correct structure. On average, for the 26 investigated distributions, the Akaike criterion only resulted in the correct structure being selected 47 percent of the time while the Schwarz criterion resulted in the correct structure being selected just 35 percent of the time. Not surprisingly, PROC MIXED default F-tests based on either of these selection criteria performed poorly according to results reported by the authors elsewhere.
British Journal of Mathematical and Statistical Psychology, 1999
ABSTRACT Looney & Stanley's (1989) recommendations regarding analysis strategies ... more ABSTRACT Looney & Stanley's (1989) recommendations regarding analysis strategies for repeated measures designs containing between-subjects grouping variables and within-subjects repeated measures variables were re-examined and compared to recent analysis strategies. That is, corrected degrees of freedom univariate tests, multivariate tests, mixed model tests, and tests due to Keselman, Carriere & Lix (1993) and to Algina (1994), Huynh (1978) and Lecoutre (1991) were compared for rates of Type I error in unbalanced non-spherical repeated measures designs having varied covariance structures and no missing data on the within-subjects variable. Heterogeneous within-subjects and heterogeneous within- and between-subjects structures were investigated along with multivariate non-normality. Results indicated that the tests due to Keselman et al. and Algina, Huynh and Lecoutre provided effective Type I error control whereas the default mixed model approach computed with PROC MIXED (SAS Institute, 1995) generally did not. Based on power differences, we recommend that applied researchers adopt the Welch-James type test described by Keselman et al.
British Journal of Mathematical and Statistical Psychology, 2000
In 1987, Jennings enumerated data analysis procedures that authors must follow for analyzing effe... more In 1987, Jennings enumerated data analysis procedures that authors must follow for analyzing effects in repeated measures designs when submitting papers to Psychophysiology. These prescriptions were intended to counteract the effects of nonspherical data, a condition know to produce biased tests of significance. Since this editorial policy was established, additional refinements to the analysis of these designs have appeared in print in a number of sources that are not likely to be routinely read by psychophysiological researchers. Accordingly, this paper includes additional procedures not previously enumerated in the editorial policy that can be used to analyze repeated measurements. Furthermore, I indicate how numerical solutions can easily be obtained.
British Journal of Mathematical and Statistical Psychology, 2001
Repeated measures ANOVA can refer to many different types of analysis. Speci®cally, this vague te... more Repeated measures ANOVA can refer to many different types of analysis. Speci®cally, this vague term can refer to conventiona l tests of signi®cance, one of three univariate solutions with adjusted degrees of freedom, two different types of multivariate statistic, or approache s that combine univariate and multivariate tests. Accordingly, it is argued that, by only reporting probability values and referring to statistical analyses as repeated measures ANOVA, authors convey neither the type of analysis that was used nor the validity of the reported probability value, since each of these approache s has its own strengths and weaknesses. The various approache s are presented with a discussion of their strengths and weaknesses, and recommendations are made regarding the`best' choice of analysis. Additional topics discussed include analyses for missing data and tests of linear contrasts.
Biometrical Journal, 2002
We compared three tests for mean equality: the Welch (1938) heteroscedastic statistic, the Zhou e... more We compared three tests for mean equality: the Welch (1938) heteroscedastic statistic, the Zhou et al. (1997) test, derived to be used with skewed lognormal data, and Yuen's (1974) procedure which uses robust estimators of central tendency and variability with the Welch test in order to combat the combined effects of nonnormality and variance heterogeneity. Over the 162 conditions of nonnormality and variance heterogeneity we investigated, only the Yuen procedure reliably controlled its rate of Type I error.
American Statistician, 2000
British Journal of Mathematical & Statistical Psychology, 2000
Boik (1997) presented an empirical ,Bayes (EB) approach ,to the ,analysis of repeated measurement... more Boik (1997) presented an empirical ,Bayes (EB) approach ,to the ,analysis of repeated measurements. The EB approach,is a blend,of the conventional univariate and multivariate approaches. Specifically, in the EB approach, the underlying covariance matrix is estimated by a ,weighted sum of the univariate and multivariate estimators. In addition to demonstrating, that his approach ,controls test size and frequently is