Measuring fidelity of implementation in a large-scale research study (RTP)

When evaluating the implementation of an engineering curriculum, it's important to be able to measure the fidelity with which the curriculum is implemented by teachers. In this paper, we describe our instruments for and approaches to measuring fidelity of implementation of an elementary school engineering curriculum, and give evidence for reliability and validity of use of these instruments for an efficacy study of the curriculum. The most important instruments are the engineering lesson implementation logs, for which teachers were prompted to indicate (1) which portions of each engineering lesson they completed; (2) the duration and date of each lesson; and (3) indications of how they taught each portion of the lesson, to measure whether teachers were using a pedagogy more in line with the research study's treatment or comparison group-a measure of program differentiation. In this paper, we provide both qualitative and quantitative evidence for the suitability of use of the engineering logs to measure implementation fidelity.