Project CLASSIFIES: Common Language Assessment in Studying Statistics with Instructional Feedback and Increased Enrollment Scalability

This project aims to serve the national interest by developing new tools to support instructors' assessment of written problems in large enrollment statistics courses. Writing is particularly crucial in statistics, and short answer questions provide students with important opportunities to interpret problems and consider multiple paths to a correct solution. Because students must formulate their answers in their own words, short answer questions are more robust for authentic assessment of student competencies and misconceptions than other short-form problems, such as multiple choice or true/false questions. For all assessments, timely feedback is well established as important for learning. Unfortunately, providing timely, quality feedback on student writing can be challenging in large classes. This can lead to assessment methods being poorly aligned with learning goals in the courses that reach the most students. This Level 1 project in the Engaged Student Learning track of the IUSE: EDU program addresses this long-standing obstacle to the effective use of free-form questions in large-enrollment classes. The project will develop, test, and refine a grading support platform that combines human and machine assessment. Specifically, the tools developed will sort students' solutions to short-answer problems into one of three categories: essentially correct, partially correct, or incorrect. Partially correct solutions are then sent to the course instructor for further review. The project engages partner institutions throughout the country to collect diverse data on statistics question prompts and student answers. The project-developed tools will give students more opportunities to strengthen their reasoning skills through exposure to short-answer problems and will give instructors a tool to monitor learning more closely.This project’s goal is to advance understanding and study effectiveness of innovative tools that pair instructor expertise with natural language processing (NLP) techniques to provide students with rich feedback. Integrating the project platform into introductory statistics courses will allow for the frequent use of short-answer tasks in large enrollment statistics classes. Large-enrollment course instructors will be able to provide detailed feedback much like students might expect in a class with 25-30 students. The project utilizes a collaboration of prominent experts in NLP, statistics education, and assessment to pursue the three aims of 1) developing an interactive, scalable, human/software partnership, 2) field-testing the methods across diverse student populations, and 3) creating open access resources of software, assessment tasks, and student data to foster future research on free-form formative assessments. The project will utilize a mixed-methods evaluation plan to assess progress towards key goals and the project-developed platform will be made freely available to interested users. The NSF IUSE: EDU Program supports research and development projects to improve the effectiveness of STEM education for all students.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.