Choosing the Best Method for Local Validity Estimation: Relative Accuracy of Meta-Analysis Versus a Local Study Versus Bayes-Analysis

This study assessed the relative accuracy of 3 techniques-local validity studies, meta-analysis, and Bayesian analysis-for estimating test validity, incremental validity, and adverse impact in the local selection context. Bayes-analysis involves combining a local study with nonlocal (meta-analytic) validity data. Using tests of cognitive ability and personality (conscientiousness) as predictors, an empirically driven selection scenario illustrates conditions in which each of the 3 estimation techniques performs best. General recommendations are offered for how to estimate local parameters, based on true population variability (σ_p²) and the number of studies in the meta-analytic prior (k). Benefits of empirical Bayesian analysis for personnel selection are demonstrated, and equations are derived to help guide the choice of a local validity technique (i.e., meta-analysis vs. local study vs. Bayes-analysis).