The development of a rubric to evaluate and promote students' integration of stakeholder considerations into the engineering design process

Approaches exist for assessing student performance in some activities during the design of complex systems such as aircraft. However, resources are limited for assessing students' abilities to consider design from a broad perspective and to account for a design's impact on its stakeholders. The purpose of this paper is to introduce a rubric to assess how students perceive and integrate stakeholders into the design of a complex system. Following a description of the rubric and its development, this paper describes results from the initial application and evaluation of the rubric by a panel of faculty, graduate students, and research scientists, as they used the rubric to assess aircraft design projects. This initial evaluation demonstrated the strengths of the rubric (particularly with regards to validity) and how the reliability of the ratings among raters was sensitive to the few, discrete scores that the initial version of the rubric allowed; potential improvements to improve reliability are noted. The paper concludes with discussion of how the rubric can be used by design instructors as both a formative assessment tool to identify and describe to students key aspects of the design process necessary to account for stakeholder considerations, and as a summative assessment tool.