Using eye movements to evaluate the effectiveness of the situation awareness rating technique scale in measuring situation awareness for smart manufacturing

Physiological indicators, including eye-tracking measures, may provide insight into human internal states in many domains, such as smart manufacturing. The Situation Awareness Rating Technique (SART) scale has been criticised for not assessing situation awareness (SA) accurately. In this study, we investigated the precision of the SART scale for assessing SA by comparing the scores to eye movement data. Thirty participants were recruited to complete a process plant monitoring task. Participants’ eye movements and SART scores were recorded and analysed. Our results moderately supported the idea that the SART scale did not accurately measure SA. We found that four dimensions in the SART scale need to be revised to reflect real SA, which may partially solve the divergence between objective and subjective SA measurements. Moreover, these findings provided solutions for designing a revised SART scale to measure the internal states of operators for smart manufacturing. Practitioner summary: Situation awareness (SA) is a critical component of decision-making and performance for smart manufacturing. The present study examines the relationships among eye movement, the SART scale, and SA for smart manufacturing in a refinery control room.