TY - JOUR
T1 - Utilization of Automatized Creativity Ratings in Linguistically Diverse Populations
T2 - 129th ASEE Annual Conference and Exposition: Excellence Through Diversity, ASEE 2022
AU - Dickson, Danielle S.
AU - Okudan Kremer, Gul E.
AU - Siddique, Zahed
AU - Gunay, Elif Elcin
AU - Van Hell, Janet
N1 - Funding Information:
This study has been supported by NSF grants DUE 1561660 to Zahed Siddique and Janet van Hell; NSF DUE IUSE-1726811 to Janet van Hell, NSF DUE IUSE-1726358 to Zahed Siddique, and NSF DUE IUSE-1726884 to Gul Okudan-Kremer.
Publisher Copyright:
© American Society for Engineering Education, 2022.
PY - 2022/8/23
Y1 - 2022/8/23
N2 - Measuring an individual's creativity typically relies on labor-intensive subjective ratings of the quality of ideas and solutions to problems. In the Alternate Uses Task (AUT), frequently used in engineering design education for concept generation and to gauge creative function-object relationships, participants generate as many novel uses of everyday objects as possible within a given time frame. Unfortunately, objective and rapid evaluation of AUT responses for levels of originality and usefulness is difficult. Recently, an automatized method for generating scores has been developed, the freely accessible Semantic Distance (SemDis) tool [1]. Given the linguistic and cultural diversity of engineering students in the U.S., it seems fair to question how well this type of automatic rating system, based on prototypical language models, captures the creativity of engineering students who may be nonnative speakers of English. We extensively trained human raters to score the AUT responses of multilingual engineering students living in either a non-English environment or in the US, and the AUT responses of monolingual English engineering students. We found that the human ratings of all three groups of engineering students correlated strongly, and positively, with the automatic SemDis ratings. This forms proof of concept for using automatic rating systems such as SemDis in engineering classroom settings. In addition to saving evaluators' time, this method may also be preferred because it is unbiased to cultural and linguistic features of responders' answers that might reveal their gender, race, ethnic or linguistic background information.
AB - Measuring an individual's creativity typically relies on labor-intensive subjective ratings of the quality of ideas and solutions to problems. In the Alternate Uses Task (AUT), frequently used in engineering design education for concept generation and to gauge creative function-object relationships, participants generate as many novel uses of everyday objects as possible within a given time frame. Unfortunately, objective and rapid evaluation of AUT responses for levels of originality and usefulness is difficult. Recently, an automatized method for generating scores has been developed, the freely accessible Semantic Distance (SemDis) tool [1]. Given the linguistic and cultural diversity of engineering students in the U.S., it seems fair to question how well this type of automatic rating system, based on prototypical language models, captures the creativity of engineering students who may be nonnative speakers of English. We extensively trained human raters to score the AUT responses of multilingual engineering students living in either a non-English environment or in the US, and the AUT responses of monolingual English engineering students. We found that the human ratings of all three groups of engineering students correlated strongly, and positively, with the automatic SemDis ratings. This forms proof of concept for using automatic rating systems such as SemDis in engineering classroom settings. In addition to saving evaluators' time, this method may also be preferred because it is unbiased to cultural and linguistic features of responders' answers that might reveal their gender, race, ethnic or linguistic background information.
UR - http://www.scopus.com/inward/record.url?scp=85138272391&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85138272391&partnerID=8YFLogxK
M3 - Conference article
AN - SCOPUS:85138272391
SN - 2153-5965
JO - ASEE Annual Conference and Exposition, Conference Proceedings
JF - ASEE Annual Conference and Exposition, Conference Proceedings
Y2 - 26 June 2022 through 29 June 2022
ER -