Utilization of Automatized Creativity Ratings in Linguistically Diverse Populations: Automated Scores Align with Human Ratings

Danielle S. Dickson, Gul E. Okudan Kremer, Zahed Siddique, Elif Elcin Gunay, Janet Van Hell

Research output: Contribution to journalConference articlepeer-review


Measuring an individual's creativity typically relies on labor-intensive subjective ratings of the quality of ideas and solutions to problems. In the Alternate Uses Task (AUT), frequently used in engineering design education for concept generation and to gauge creative function-object relationships, participants generate as many novel uses of everyday objects as possible within a given time frame. Unfortunately, objective and rapid evaluation of AUT responses for levels of originality and usefulness is difficult. Recently, an automatized method for generating scores has been developed, the freely accessible Semantic Distance (SemDis) tool [1]. Given the linguistic and cultural diversity of engineering students in the U.S., it seems fair to question how well this type of automatic rating system, based on prototypical language models, captures the creativity of engineering students who may be nonnative speakers of English. We extensively trained human raters to score the AUT responses of multilingual engineering students living in either a non-English environment or in the US, and the AUT responses of monolingual English engineering students. We found that the human ratings of all three groups of engineering students correlated strongly, and positively, with the automatic SemDis ratings. This forms proof of concept for using automatic rating systems such as SemDis in engineering classroom settings. In addition to saving evaluators' time, this method may also be preferred because it is unbiased to cultural and linguistic features of responders' answers that might reveal their gender, race, ethnic or linguistic background information.

Original languageEnglish (US)
JournalASEE Annual Conference and Exposition, Conference Proceedings
StatePublished - Aug 23 2022
Event129th ASEE Annual Conference and Exposition: Excellence Through Diversity, ASEE 2022 - Minneapolis, United States
Duration: Jun 26 2022Jun 29 2022

All Science Journal Classification (ASJC) codes

  • General Engineering


Dive into the research topics of 'Utilization of Automatized Creativity Ratings in Linguistically Diverse Populations: Automated Scores Align with Human Ratings'. Together they form a unique fingerprint.

Cite this