Metamodel-driven interfaces for engineering design: Impact of delay and problem size on user performance

In this paper, we present results from a follow-up study to our previous research involving the use of a metamodel-driven, graphical design interface for sizing the plan view layout of an aircraft wing to minimize cost subject to three constraints. This study extends our work by examining the impact of (i) varying problem size, i.e., the number of design variables (2, 4, or 6) that the user can manipulate, and (ii) a longer response delay (1.5 seconds) on user performance. Design effectiveness is measured by calculating the percent error between a submitted design and the known optimum, and design efficiency is measured by recording the completion time for solving the wing design problem. Fortyeight users participated in the study. Response delay and problem size both had a significant effect on design effectiveness: the 1.5 second delay increased average error by 150% compared to the no delay case, and the average error more than doubled each time the problem size increased. Problem size also had a significant effect on design efficiency: the average completion times for the 4 and 6 variable problems were more than double the completion time for the 2 variable problem. Average completion time also increased as the response delay increased, but this effect was not significant at the 5% level. Although the interaction between delay and problem size was not significant, the impact of the response delay on design efficiency and design effectiveness did decrease as problem size increased. Correlations between user performance and responses to p re- and post-test questionnaires are also discussed along with the implications of our findings for the development of metamodel-driven, graphical design interfaces.