Validation of an observation instrument for measuring student engagement in health professions settings

Kimberly J. O'Malley, Betty Jeanne Moran, Paul Haidet, Charles L. Seidel, Virginia Scheinder, Robert O. Morgan, P. Adam Kelly, Boyd Richards

Research output: Contribution to journalArticlepeer-review

63 Scopus citations


Documenting student engagement has received increased emphasis in medical schools, as teaching strategies are changing to include more student-to-student interactions. The purpose of this study was to develop and evaluate a measure of student engagement completed by independent observers that would not interfere with student learning time. Data from 3,182 observations completed by nine observers in 32 educational classroom settings with 23 different instructors were used to evaluate the interobserver reliability and gather validity evidence for our observational instrument, named the STROBE. Results indicated that interobserver agreement was good to excellent when observations were conducted simultaneously on randomly selected students in the same classroom (84% average agreement and 0.79 average kappa coefficient) and when observations were conducted on different randomly selected students (79% average agreement). Results also provided strong evidence for validity. Overall, findings indicate that the STROBE demonstrates promise for educational research and evaluation by documenting student engagement in medical education settings.

Original languageEnglish (US)
Pages (from-to)86-103
Number of pages18
JournalEvaluation and the Health Professions
Issue number1
StatePublished - 2003

All Science Journal Classification (ASJC) codes

  • Health Policy


Dive into the research topics of 'Validation of an observation instrument for measuring student engagement in health professions settings'. Together they form a unique fingerprint.

Cite this