Observe, hypothesize, test, repeat: Luttrell, Petty and Xu (2017) demonstrate good science

Many Labs 3 (Ebersole et al., 2016) failed to replicate a classic finding from the Elaboration Likelihood Model of persuasion (Cacioppo, Petty, & Morris, 1983; Study 1). Petty and Cacioppo (2016) noted possible limitations of the Many Labs 3 replication (Ebersole et al., 2016) based on the cumulative literature. Luttrell, Petty, and Xu (2017) subjected some of those possible limitations to empirical test. They observed that a revised protocol obtained evidence consistent with the original finding that the Many Labs 3 protocol did not. This observe-hypothesize-test sequence is a model for scientific inquiry and critique. To test whether these results advance replicability and knowledge transfer, we conducted direct replications of Luttrell et al. in nine locations (Total N = 1219). We successfully replicated the interaction of need for cognition and argument quality on persuasion using Luttrell et al.'s optimal design (albeit with a much smaller effect size; p < 0.001; f² = 0.025, 95%CI [0.006, 0.056]) but failed to replicate the interaction that indicated that Luttrell et al.’s optimal protocol performed better than the Many Labs 3 protocol (p = 0.135, pseudo R² = 0.002). Neither Luttrell et al.'s effect size estimate for the need for cognition by argument quality interaction nor their estimate for the interaction with replication protocol fell within our corresponding 95% confidence intervals. Nevertheless, pragmatically, we favor the Luttrell et al. protocol with large samples for future research using this paradigm.