On Sampling and Measurement: A Case Study of the Association Between Loneliness and Cognitive Function in the Survey of Health Aging and Retirement in Europe

All research fields aspire to produce high-quality research, but there are disciplinary differences in how validity is conceptualized that create barriers to interdisciplinary collaboration and evaluation. In this chapter, we distinguish validity - the degree to which a research design is thought to support veracious inferences - from veracity itself. We propose a tridimensional framework of validity encompassing conceptual, internal, and external dimensions, reflecting concerns with measurement, causal inference, and generalizability. Although different fields emphasize different dimensions, often at the expense of at least one other, robust science requires attention to all three. For instance, developmental scientists do not always give external validity its due, instead more closely attending to the highest quality measurement and strengthened ability to support causal inferences at the expense of generalizability. We illustrate the dangers of ignoring issues of generalizability through a case study of loneliness and cognitive decline in later life. Analyses using the Survey of Health, Aging, and Retirement in Europe (SHARE) show how prioritizing precise measurement (conceptual validity) without sufficient regard for sampling (external validity) can yield misleading conclusions, even when internal validity is addressed. We argue that validity should not be viewed as a zero-sum competition among dimensions, but rather as an interdependent structure where higher-quality research simultaneously maximizes all three dimensions to the greatest extent possible, with risks to deprioritizing any dimension. Our tridimensional framework provides developmental scientists and other researchers with a tool to recognize and navigate tradeoffs, avoid false inferences, and advance interdisciplinary science in the quest for veracity.