The aligned rank transform for nonparametric factorial analyses using only anova procedures | Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (original) (raw)

Published: 07 May 2011 Publication History

Abstract

Nonparametric data from multi-factor experiments arise often in human-computer interaction (HCI). Examples may include error counts, Likert responses, and preference tallies. But because multiple factors are involved, common nonparametric tests (e.g., Friedman) are inadequate, as they are unable to examine interaction effects. While some statistical techniques exist to handle such data, these techniques are not widely available and are complex. To address these concerns, we present the Aligned Rank Transform (ART) for nonparametric factorial data analysis in HCI. The ART relies on a preprocessing step that "aligns" data before applying averaged ranks, after which point common ANOVA procedures can be used, making the ART accessible to anyone familiar with the F-test. Unlike most articles on the ART, which only address two factors, we generalize the ART to N factors. We also provide ARTool and ARTweb, desktop and Web-based programs for aligning and ranking data. Our re-examination of some published HCI results exhibits advantages of the ART.

References

[1]

Conover, W. J. and Iman, R. L. (1981). Rank transformations as a bridge between parametric and nonparametric statistics. The American Statistician 35 (3), 124--129.

[2]

Fawcett, R. F. and Salter, K. C. (1984). A Monte Carlo study of the F test and three tests based on ranks of treatment effects in randomized block designs. Communications in Statistics: Simulation and Computation 13 (2), 213--225.

[3]

Findlater, L., Moffatt, K., McGrenere, J. and Dawson, J. (2009). Ephemeral adaptation: The use of gradual onset to improve menu selection performance. Proc. CHI '09. New York: ACM Press, 1655--1664.

[4]

Higgins, J. J., Blair, R. C. and Tashtoush, S. (1990). The aligned rank transform procedure. Proc. Conf. on Applied Statistics in Agriculture. Kansas State, 185--195.

[5]

Higgins, J. J. and Tashtoush, S. (1994). An aligned rank transform test for interaction. Nonlinear World 1 (2), 201--211.

[6]

Hodges, J. L. and Lehmann, E. L. (1962). Rank methods for combination of independent experiments in the analysis of variance. Annals of Mathematical Statistics 33 (2), 482--497.

[7]

Kaptein, M., Nass, C. and Markopoulos, P. (2010). Powerful and consistent analysis of Likert-type rating scales. Proc. CHI '10. New York: ACM Press, 2391--2394.

[8]

Littell, R. C., Henry, P. R. and Ammerman, C. B. (1998). Statistical analysis of repeated measures data using SAS procedures. J. Animal Science 76 (4), 1216--1231.

[9]

MacKenzie, I. S. and Zhang, S. X. (1999). The design and evaluation of a high-performance soft keyboard. Proc. CHI '99. New York: ACM Press, 25--31.

[10]

Salter, K. C. and Fawcett, R. F. (1985). A robust and powerful rank test of treatment effects in balanced incomplete block designs. Communications in Statistics: Simulation and Computation 14 (4), 807--828.

[11]

Salter, K. C. and Fawcett, R. F. (1993). The art test of interaction: A robust and powerful rank test of interaction in factorial models. Communications in Statistics: Simulation and Computation 22 (1), 137--153.

[12]

Sawilowsky, S. S. (1990). Nonparametric tests of interaction in experimental design. Review of Educational Research 60 (1), 91--126.

[13]

Vermunt, J. K. (1997). Log-linear Models for Event Histories. Thousand Oaks, CA: Sage Publications.

[14]

Wobbrock, J. O., Wilson, A. D. and Li, Y. (2007). Gestures without libraries, toolkits or training: A $1 recognizer for user interface prototypes. Proc. UIST '07. New York: ACM Press, 159--168.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '11: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

May 2011

3530 pages

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 May 2011

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. analysis of variance
  2. anova
  3. f-test
  4. factorial analysis
  5. nonparametric data
  6. statistics

Qualifiers

Conference

Acceptance Rates

CHI '11 Paper Acceptance Rate 410 of 1,532 submissions, 27%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

Reflects downloads up to 16 Nov 2024

Other Metrics

Citations

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Sign in

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Affiliations

Jacob O. Wobbrock

University of Washington, Seattle, Washington, USA

Leah Findlater

University of Washington, Seattle, Washington, USA

Darren Gergle

Northwestern University, Chicago, Illinois, USA

James J. Higgins

Kansas State University, Manhattan, Kansas, USA