TY - JOUR
T1 - Metastudies for robust tests of theory
AU - Baribault, Beth
AU - Donkin, Chris
AU - Little, Daniel R.
AU - Trueblood, Jennifer S.
AU - Oravecz, Zita
AU - Van Ravenzwaaij, Don
AU - White, Corey N.
AU - De Boeck, Paul
AU - Vandekerckhove, Joachim
N1 - Publisher Copyright:
© 2018 National Academy of Sciences. All Rights Reserved.
PY - 2018/3/13
Y1 - 2018/3/13
N2 - We describe and demonstrate an empirical strategy useful for discovering and replicating empirical effects in psychological science. The method involves the design of a metastudy, in which many independent experimental variables—that May be moderators of an empirical effect—are indiscriminately randomized. Radical randomization yields rich datasets that can be used to test the robustness of an empirical claim to some of the vagaries and idiosyncrasies of experimental protocols and enhances the generalizability of these claims. The strategy is made feasible by advances in hierarchical Bayesian modeling that allow for the pooling of information across unlike experiments and designs and is proposed here as a gold standard for replication research and exploratory research. The practical feasibility of the strategy is demonstrated with a replication of a study on subliminal priming.
AB - We describe and demonstrate an empirical strategy useful for discovering and replicating empirical effects in psychological science. The method involves the design of a metastudy, in which many independent experimental variables—that May be moderators of an empirical effect—are indiscriminately randomized. Radical randomization yields rich datasets that can be used to test the robustness of an empirical claim to some of the vagaries and idiosyncrasies of experimental protocols and enhances the generalizability of these claims. The strategy is made feasible by advances in hierarchical Bayesian modeling that allow for the pooling of information across unlike experiments and designs and is proposed here as a gold standard for replication research and exploratory research. The practical feasibility of the strategy is demonstrated with a replication of a study on subliminal priming.
UR - http://www.scopus.com/inward/record.url?scp=85043762113&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85043762113&partnerID=8YFLogxK
U2 - 10.1073/pnas.1708285114
DO - 10.1073/pnas.1708285114
M3 - Article
C2 - 29531092
AN - SCOPUS:85043762113
SN - 0027-8424
VL - 115
SP - 2607
EP - 2612
JO - Proceedings of the National Academy of Sciences of the United States of America
JF - Proceedings of the National Academy of Sciences of the United States of America
IS - 11
ER -