VRMN-bD: A Multi-modal Natural Behavior Dataset of Immersive Human Fear Responses in VR Stand-up Interactive Games

He Zhang, Xinyang Li, Yuanxi Sun, Xinyi Fu, Christine Qiu, John M. Carroll

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

Understanding and recognizing emotions are important and challenging issues in the metaverse era. Understanding, identifying, and predicting fear, which is one of the fundamental human emotions, in virtual reality (VR) environments plays an essential role in immersive game development, scene development, and next-generation virtual human-computer interaction applications. In this article, we used VR horror games as a medium to analyze fear emotions by collecting multi-modal data (posture, audio, and physiological signals) from 23 players. We used an LSTM-based model to predict fear with accuracies of 65.31% and 90.47% under 6-level classification (no fear and five different levels of fear) and 2-level classification (no fear and fear), respectively. We constructed a multi-modal natural behavior dataset of immersive human fear responses (VRMN-bD) and compared it with existing relevant advanced datasets. The results show that our dataset has fewer limitations in terms of collection method, data scale and audience scope. We are unique and advanced in targeting multi-modal datasets of fear and behavior in VR stand-up interactive environments. Moreover, we discussed the implications of this work for communities and applications. The dataset and pre-trained model are available at https://github.com/KindOPSTAR/VRMN-bD.

Original languageEnglish (US)
Title of host publicationProceedings - 2024 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2024
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages320-330
Number of pages11
ISBN (Electronic)9798350374025
DOIs
StatePublished - 2024
Event31st IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2024 - Orlando, United States
Duration: Mar 16 2024Mar 21 2024

Publication series

NameProceedings - 2024 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2024

Conference

Conference31st IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2024
Country/TerritoryUnited States
CityOrlando
Period3/16/243/21/24

All Science Journal Classification (ASJC) codes

  • Computer Graphics and Computer-Aided Design
  • Human-Computer Interaction
  • Media Technology
  • Modeling and Simulation

Fingerprint

Dive into the research topics of 'VRMN-bD: A Multi-modal Natural Behavior Dataset of Immersive Human Fear Responses in VR Stand-up Interactive Games'. Together they form a unique fingerprint.

Cite this