Can process metrics predict product success? A pilot study of student design teams

Engineering projects typically revolve around producing a deliverable. That deliverable goes to a customer, who either deems it acceptable or in need of further work. The engineering analysis and components of whatever system is to be produced are the subject of much scrutiny. However, the human process composed of team interactions that lead up to creating that final product is frequently treated as a “black box” that simply produces an output. In order to identify what factors in that process are key to a successful product, this work seeks to identify what successful engineering design teams do differently than less successful teams. As part of our larger research project, metrics for measuring team performance during the process of design have also been created. In this paper, we use three of those metrics in a case study of 5 senior-level student design teams. These data are employed in conjunction with feedback from the instructor, acting as their customer, to identify which behaviors had strong links with more successful team results. We also investigate whether any of the behaviors exhibited by the teams correspond to worse results, in order to identify behaviors with the potential to be used to predict poorer performance in advance. This analysis is completed using data collected via a mid-term survey and an end-of-project survey (both completed by the team members), in addition to video and audio meeting data, and data collected from both midterm and final presentations. We present these results as an avenue to move us towards enabling engineers to choose to engage knowingly in behaviors that correlate with better project results.