Application of SHM pattern recognition to assess decision making of humans in the loop

Structural health monitoring (SHM) techniques have traditionally been applied to mechanical, aerospace, and civil structures to identify loading and damage patterns. However, human operators in the loop play an important role in the operational performance of aircraft and other structural systems. The increased availability of sensors such as EEG, skin conductance, and eye-tracking systems are creating an opportunity to develop SHM techniques for assessing neuro-physiological factors that influence human decision-making. The parallels between the structural dynamic response of a system to an excitation source and the response of a human to the presentation of a scenario suggests that SHM algorithms can be used to interpret neurophysiological signals. As in traditional SHM, where the system's dynamic response is measured to characterize the system's state of health, the measured response of a human during decision-making can capture information about the human's mental state, including levels of fatigue, engagement, workload, and other human factors. The ability to monitor the human's mental state in real-time could also enable predictions of human susceptibility to poor decision-making and to trigger an appropriate intervention to prevent human errors. In this work, EEG, eye-tracking, and skin conductance data are acquired from multiple subjects while performing the Stroop test, a standard test designed to induce errors, under varying degrees of time pressure. Time pressure is induced by progressively reducing the time-to-answer allowed for each set of questions. The data is then analyzed using pattern recognition techniques including principal component analysis and the least squares complex exponential (LSCE) parameter estimation algorithm. Results from the principal component analysis identify the modes which dominate the response during decision-making. These modes are compared to the modes identified while the subject is at rest. Next, LSCE is applied to identify model parameters that can be used to perform a one-step-ahead prediction of the neurophysiological variables. The LSCE approach allows data from the different sensor types to be analyzed simultaneously. Results show that model error is reduced as the time pressure is increased.