Maybe not so independent after all: The possibility, prevalence, and consequences of violating the independence assumptions in psychometric meta-analysis

Zhenyu Yuan, Frederick P. Morgeson, James M. LeBreton

Research output: Contribution to journalArticlepeer-review

9 Scopus citations

Abstract

Psychometric meta-analysis assumes that moderators are unrelated to study artifacts (e.g., criterion reliability), and that study artifacts are independent of true validities. Meeting these assumptions is important for researchers seeking to accurately partition the variance in effect sizes due to study artifacts from the variance due to meaningful moderators. Despite the critical role of these assumptions, we know very little about their tenability. To address this basic gap in the literature, we conducted three studies to determine if there are potential violations of the independence assumptions (Study 1), the prevalence of such violations (Study 2), and the consequences of violating the independence assumptions via a series of Monte Carlo simulations (Study 3). We found that violations of the independence assumptions are not only plausible but also routinely detected across a wide array of topics. Simulation results indicate that violating the independence assumptions can result in biases under certain circumstances, which are further accentuated due to the lack of stability in the estimators. We offer suggestions for the future use of psychometric meta-analysis and discuss the implications for research focused on refining psychometric meta-analysis.

Original languageEnglish (US)
Pages (from-to)491-516
Number of pages26
JournalPersonnel Psychology
Volume73
Issue number3
DOIs
StatePublished - Sep 1 2020

All Science Journal Classification (ASJC) codes

  • Applied Psychology
  • Organizational Behavior and Human Resource Management

Fingerprint

Dive into the research topics of 'Maybe not so independent after all: The possibility, prevalence, and consequences of violating the independence assumptions in psychometric meta-analysis'. Together they form a unique fingerprint.

Cite this