A novel method for providing global assessments of design concepts using single-word adjectives and semantic similarity

Engineering design idea-generation sessions often result in dozens, if not hundreds, of ideas. These ideas must be quickly evaluated and filtered in order to select a few candidate concepts to move forward in the design process. While creativity is often stressed in the conceptual phases of design, it receives little attention in these later phases - particularly during concept selection. This is largely because there are no methods for quickly rating or identifying worthwhile creative concepts during this process. Therefore, the purpose of this study was to develop and test a novel method for evaluating the creativity and feasibility of design concepts and compare this method to gold standards in our field. The SCAT method employed in this paper uses word selections and semantic similarity to quickly and effectively evaluate candidate concepts for their creativity and feasibility. This method requires little knowledge of the rating process by the evaluator. We tested this method with 10 engineering designers and three different design tasks. Our results revealed that SCAT ratings can be used as a proxy for measuring design concepts but there are modifications that could enhance its utility. This work contributes to our understanding of how to evaluate creativity after idea generation and provides a framework for further research in this field.