Inter-rater and test-retest reliability: Methods and results for the neighborhood observational checklist

The popularity of direct or systematic social observation as a method to evaluate the mechanisms by which neighborhood environments impact health and contribute to health disparities is growing. The development of measures with adequate inter-rater and test-retest reliability is essential for this research. In this paper, based on our experiences conducting direct observation of neighborhoods in Detroit, MI, we describe strategies to promote high inter-rater and test-retest reliability and methods to evaluate reliability. We then present the results and discuss implications for future research efforts using direct observation in four areas: methods to evaluate reliability, instrument content and design, observer training, and data collection.