Using Computational Modeling to Transform Assessments of Creativity in Engineering Design

This collaborative project from research teams at Pennsylvania State University, University of Maryland, and Washington and Lee University focuses on measuring creativity in undergraduate engineering education. The ability to think creatively is essential for success in STEM fields, particularly engineering, which requires designing solutions to complex problems that often have no single or 'correct' solution. The Next Generation Science Standards identify creative thinking skills, such as problem solving and flexibility, as core competencies for modern STEM education. Yet educators are not currently equipped with adequate tools to assess creativity in their classrooms. To effectively prepare the STEM workforce, there is a critical need for assessment tools that educators and researchers can use to identify what works in STEM education to foster creativity. Current creativity tests present significant challenges for STEM educators, including (in-person) paper administration and, perhaps most problematically, manual scoring that requires teachers to count and code thousands of responses—a labor-intensive and often costly process, particularly for under-resourced schools. In light of the increasingly diverse student population, the availability of creativity tests that measure student ability fairly and consistently, regardless of race or ethnicity, is even more critical for equity of opportunity in STEM education. This project seeks to create an online platform for measuring creativity in engineering design that educators can use to cater to the needs of all their students. The tool will allow educators to administer a range of engineering creativity tasks and automatically calculate creativity scores. This project fits the intent of the ECR program to facilitate 'the development, refinement, and testing of new education research, measurement, and evaluation methodologies.' It addresses the ECR research track, 'Research on STEM Learning and Learning Environments,' and has additional impacts for 'Research on Broadening Participation in STEM Fields' by designing inclusive and culturally and linguistically diverse assessment tools targeted to students who remain underrepresented in the pursuit of STEM courses of study and English as second language speakers.

Two aims guide this project. First is to build an online platform for large-scale engineering design assessment — validating all platform tasks with undergraduate engineering students — to allow teachers and researchers to easily assess creativity, automatically compute creativity metrics, and generate customizable student reports. Second is to apply the platform in an undergraduate design course at Penn State that includes a 3-week Creativity Module (with lessons and exercises on creativity in engineering design) to obtain valuable platform usability data from both instructors and students, while evaluating a promising undergraduate course intended to promote creativity in engineering design. The team will apply recent advances in computational modeling and machine learning — including active learning of design sketches and distributional semantic modeling of text-based responses to creative problem solving tasks. It is expected that this approach will streamline educational assessment of creativity, resulting in a user-friendly technology to assist STEM educators in the classroom. The novel computational tools developed in this project will advance knowledge and understanding for creativity psychometric assessment and across different fields (not only engineering). The PI team will also design assessment tools that are culturally responsive and minimally biased — especially for the growing number of students who speak English as a second language — and collaborate with STEM educators to maximize the usability of the platform in their classrooms. The online platform and course materials will be publicly available, facilitating the national transition to remote education and research (accelerated by the current pandemic) by providing online resources for STEM teachers and researchers across the country.

This project is supported by NSF's EHR Core Research (ECR) program. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field. Investments are made in critical areas that are essential, broad and enduring: STEM learning and STEM learning environments, broadening participation in STEM, and STEM workforce development. The program supports the accumulation of robust evidence to inform efforts to understand, build theory to explain, and suggest intervention and innovations to address persistent challenges in education.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.