“C”ing the light–assessing code comprehension in novice programmers using C code patterns

Background and Context: Feedback on one’s progress is essential to new programming language learners, particularly in out-of-classroom settings. Though many study materials offer assessment mechanisms, most do not examine the accuracy of the feedback they deliver, nor give evidence on its validity. Objective: We investigate the potential use of a preexisting set of C code snippets as the basis for a high-quality C programming ability assessment tool. Method: We utilize the Rasch Model and the Linear Logistic Test Model to evaluate the validity and accuracy of the code snippets and to determine which C operations contribute most to their overall difficulty. Findings: Our results show that these code snippets yield accurate assessments of programming ability and reveal the degree of difficulty associated with specific programming operations. Implications: Our results suggest that the code snippets could serve as the basis for sophisticated, valid, and fair code comprehension skill assessment tools.