Effects of Comparison Question Type and Between Test Stimulation on the Validity of Comparison Question Test (original) (raw)
Tukey's Honestly Significant Difference (HSD) procedure (J. Tukey, 1953) is probably the most recommended and used procedure for controlling Type I error rate when making multiple pairwise comparisons as follow-ups to a significant omnibus F test. This study compared observed Type I errors with nominal alphas of 0.01, 0.05, and 0.10 compared for various sample sizes and numbers of groups. Monte Carlo methods were used to generate replications expected to provide 0.95 confidence intervals of +/-0.001 around the nominal alphas of 0.10, 0.05, and 0.01 for 42 combinations of n (5, 10, 15, 20, 30, 60, and 100) and numbers of groups (3, 4, 5, 6, 8, and 10) . Means and standard deviations of observed Type I error rates and percentages of observed Type I errors falling below, within, and above the 0.95 confidence intervals were determined for total number of Type I errors. The results indicate that HSD is conservative relative to experimentwise Type I error control across all alpha leve...