The use of generalizability theory to estimate data reliability in single-subject observational research

Direct observation of behaviors is a data collection method customarily used in clinical and educational settings. Repeated measures and small samples are inherent characteristics of observational studies that pose challenges to the numerical estimation of reliability for observational data. In this article, we review some debates about the use of Generalizability Theory in estimating reliability of single-subject observational data. We propose that it could be used but under a clearly stated set of conditions. The conceptualization of facets and object of measurement for a common design of observational research is elucidated under a different light. We provide two numerical examples to illustrate the ideas. Limitations of using Generalizability Theory to estimate reliability of observational data are discussed.