Applying the permutation test to factorial designs

D. J.K. Mewhort, Brendan T. Johns, Matthew Kelly

Research output: Contribution to journalArticlepeer-review

12 Scopus citations


The permutation test follows directly from the procedure in a comparative experiment, does not depend on a known distribution for error, and is sometimes more sensitive to real effects than are the corresponding parametric tests. Despite its advantages, the permutation test is seldom (if ever) applied to factorial designs because of the computational load that they impose. We propose two methods to limit the computation load. We show, first, that orthogonal contrasts limit the computational load and, second, that when combined with Gill's (2007) algorithm, the factorial permutation test is both practical and efficient. For within-subjects designs, the factorial permutation test is equivalent to an ANOVA when the latter's assumptions have been met. For between-subjects designs, the factorial test is conservative. Code to execute the routines described in this article may be downloaded from

Original languageEnglish (US)
Pages (from-to)366-372
Number of pages7
JournalBehavior research methods
Issue number2
StatePublished - May 2010

All Science Journal Classification (ASJC) codes

  • Experimental and Cognitive Psychology
  • Developmental and Educational Psychology
  • Arts and Humanities (miscellaneous)
  • Psychology (miscellaneous)
  • General Psychology


Dive into the research topics of 'Applying the permutation test to factorial designs'. Together they form a unique fingerprint.

Cite this