Identifying the Optimal Methods for Controlling Contamination Bias in Prospective Research on Child Maltreatment

Contamination in child maltreatment research occurs when members of a comparison condition are exposed to child maltreatment prior to enrolling in a study or during longitudinal follow-up. This phenomenon presents a serious scientific concern, as contamination minimizes real differences in the risk for adverse child development between child maltreatment and comparison conditions. The current project addresses this concern by testing different methods for detecting and controlling contamination bias in child maltreatment research. Specifically, investigators will access existing data from independent prospective cohort studies and examine the performance of five different statistical modeling approaches to determine which has the optimal control of contamination. Two bias reduction modeling approaches, propensity score and augmented synthetic control models, will be executed with results compared against three conventional approaches: no control of contamination, controlling contamination by removing identified participants from the statistical model, and controlling contamination by testing it as a covariate or moderator of risk estimates. Investigators will then use data simulations to model the performance of these five methods across different research conditions, including variations in contamination prevalence, sample size, statistical power, and effect size magnitude. Ultimately, the best performing methods for detecting and controlling contamination in child maltreatment research will be disseminated to scientists so that future risk estimates are more accurate, something that can better inform child welfare policy through more reliable scientific data. While tested within child maltreatment research, results from the statistical models evaluated in this project will be available to any scientist conducting observational research on exposure variables, enhancing the reliability of scientific results across STEM domains. Finally, knowledge on how to use these methods will be incorporated into STEM education at the University-level by educating undergraduate, graduate, and post-doctoral fellows on how to control contamination in prospective cohort studies within and outside the area of child maltreatment.Contamination, when subjects recruited to a non-exposure comparison condition have been exposed to the event under investigation, is a methodological phenomenon in observational research that downwardly biases effect size estimates by minimizing group differences, thereby increasing replication and reproducibility failures. The current project will evaluate the performance of multiple strategies for detecting and controlling contamination bias in prospective cohort research with the child maltreatment population. This will be achieved through secondary analysis of existing data from the Longitudinal Studies of Child Abuse and Neglect (LONGSCAN; N=1354) and the National Survey of Child and Adolescent Well-being-II (NSCAWII; N=5873) cohorts, each of which are multi-wave, prospective cohort studies of child maltreatment from birth through age eighteen. A multi-method approach of official case reports, self-report, and caregiver-report assessments obtained across child development will be used to maximize sensitivity for detecting contamination and establishing its prevalence in these cohorts. Two innovative methods for controlling bias in observational research, doubly robust propensity score and augmented synthetic controls, will be used to control contamination bias in each of these cohorts. Results from these two models will be evaluated against alternative models that: 1) ignore contamination, 2) control contamination by removing subjects from statistical analysis, and 3) control contamination by estimating it as a covariate/moderator of child maltreatment effects. Following statistical efficiency principles, the performance of all five models will be benchmarked with respect to needed sample size, impact on statistical power, and extent of bias reduction in effect size estimates. Given the ultimate goal to identify which methods provide the most accurate estimates under different empirical conditions, this project will also evaluate results from all five models using simulations that both mimic the data structures of LONGSCAN and NSCAW-II and extend out to conditions commonly encountered in prospective cohort studies, including variations in contamination prevalence, sample size, and effect size magnitude. These results will have broader scientific and public impacts by: disseminating results to scientists outside the area of child maltreatment who conduct research where contamination is present, training the next generation of STEM scientists in the optimal methods for detecting and controlling contamination, and providing more accurate estimates of the risks associated with child maltreatment to better inform child welfare policy in the U.S.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.