Collaborative Research: Improving the Validity and Reliability of Creativity Ratings in Engineering Design

Without creativity, there is no potential for innovation. Consequently, creativity is seen as an essential component of engineering design. Numerous creativity metrics have been developed to assess the creativity of designs people produce. Such metrics are important for assessing business and engineering practices, a key part of improving US innovation and economic competitiveness. However, existing metrics are often scattered across different, specialized domains and it is difficult to validate their ability to accurately measure the creativity of the wide variety of engineered solutions that are produced. In addition, past research provides limited guidance on how to use and validate creativity metrics for a given design problem. This has led to universally applying metrics without systematic assessment of where and when a given metric is appropriate for a given task. This award supports fundamental research into how to evaluate the effectiveness of different creativity metrics for different types of products or services. This project does so by unifying statistical models that can experimentally validate how well different creativity metrics perform across design domains. Thus, the work will impact society by providing a verifiable method for identifying what does and does not improve creativity. Because creativity and innovation are the drivers of economic success, the work has the opportunity to drive design innovation and, as a bi-product, help stimulate the economy. In addition, the unified statistical framework developed as part of the research will advance the field of engineering design and applied mathematics by providing evidence on the utility of this approach for measuring accuracy and precision. The research involves several disciplines including engineering, psychology and applied mathematics. The multidisciplinary approach will help broaden participation of underrepresented groups in research.

The technical objectives of this project are to: (1) evaluate the effectiveness of creativity metrics through the development of a unified statistical framework that combines two techniques--the minimax conditional entropy principle and the Lovasz-Bregman Divergence--to compare the accuracy and precision of different metrics for a given problem; and (2) experimentally validate a methodology for validating the transfer of creativity metrics across different design domains to identify metrics that are robust across different applications within design and systems engineering via variance and confidence measures of the Lovasz-Bregman Divergence. The results from this project will advance the field of engineering by developing a unified method of measuring and comparing mathematical, computational, and human-judgment models of creativity. This new knowledge will provide a rigorous foundation upon which to build and verify methods of design creativity across a wide variety of design disciplines (e.g., arts and architecture, psychology, engineering, business).