A comparison of mean partialing and dual-hypothesis testing to evaluate stereotype effects when assessing profile similarity

Assessing profile similarity is an important task in research and clinical practice, but conclusions about profile similarity may be confounded by stereotype effects that create artificially similar profiles. In this article, we review the impact of stereotype effects on profile similarity and a conventional approach to addressing this confound (i.e., mean partialing). We argue that a dual-hypothesis testing approach distinguishing the no-effect null hypothesis (i.e., is the observed similarity different from zero?) from the chance-effect null hypothesis (i.e., is the observed similarity different from chance given the distribution of profile elements?) can provide a more nuanced understanding of profile similarity. To compare results from these 2 perspectives, we analyzed data from 2 samples using within-persons correlations as indexes of similar profile shapes. Results indicated that a dual-hypothesis testing approach led to more conservative conclusions about profile similarity (i.e., fewer Type 1 errors) than mean partialing and may be especially valuable when dealing with moderate-sized stereotype effects. Both approaches led to identical conclusions when stereotype effects were largest. Conclusions emphasize the relative merits and limitations of the dual-hypothesis testing approach as well as potential future applications in the personality assessment domain.