Assessing and interpreting interaction effects: A reply to Vancouver, Carlson, Dhanani, and Colton (2021).

Van Iddekinge et al. (2018)’s meta-analysis revealed that ability and motivation have mostly an additive rather than an interactive effect on performance. One of the methods they used to assess the ability × motivation interaction was moderated multiple regression (MMR). Vancouver et al. (2021) presented conceptual arguments that ability and motivation should interact to predict performance, as well as analytical and empirical arguments against the use of MMR to assess interaction effects. We describe problems with these arguments and show conceptually and empirically that MMR (and the ΔR and ΔR2 it yields) is an appropriate and effective method for assessing both the statistical significance and magnitude of interaction effects. Nevertheless, we also applied the alternative approach Vancouver et al. recommended to test for interactions to primary data sets (k = 69) from Van Iddekinge et al. These new results showed that the ability × motivation interaction was not significant in 90% of the analyses, which corroborated Van Iddekinge et al.’s original conclusion that the interaction rarely increments the prediction of performance beyond the additive effects of ability and motivation. In short, Van Iddekinge et al.’s conclusions remain unchanged and, given the conceptual and empirical problems we identified, we cannot endorse Vancouver et al.’s recommendation to change how researchers test interactions. We conclude by offering suggestions for how to assess and interpret interactions in future research.