Collaborative Research: An Extended Reality Factory Innovation for Adaptive Problem-solving and Personalized Learning in Manufacturing Engineering

Rapid advances in technology increase the complexity and dynamic characteristics of problems and their solutions in industry. Problem solving is understood as the process required to achieve a goal in an uncertain environment. Understanding problem-solving entails exploring the processes used in conceptualizing the problem and in moving from the initial state to the goal. Traditional problem-solving approaches focus more on developing solutions in static situations, and are less concerned about the pace of dynamic changes and technological disruptions, which require adaptive problem-solving skills (APS). Thus, to succeed in this environment, future engineers should be equipped with APS skills. This project will design and develop a virtual factory, with physical sensors, to investigate the impact of technological advances on problem-solving skills and develop a personalized learning platform for manufacturing education to meet the needs of learners and educators. First, adaptive problem situations that relate to the past, present, and future of manufacturing will be designed. Second, extended reality (xR) environments will be developed and integrated with eye and motion tracking to provide real-time monitoring of learning behavior and dynamics. Third, analytical models will be created to enhance the proficiency of APS abilities. Research outcomes will be evaluated and disseminated via scholarly publications and educational outreach programs. This project integrates physical, virtual reality and augmented reality manufacturing simulations with sensing technology to characterize and quantify APS skills. The project will have a direct positive impact on teaching and learning of APS by simulating the industrial evolutions and dynamical changes in manufacturing settings. Rich data collected through eye, facial and motion tracking will be utilized to analyze nonlinear human behaviors, thereby providing dynamic models to improve the user learning experience and optimize APS skills. The research will be guided by the following questions: (1) To what extent do heterogeneous learning modes (physical, virtual, mixed) enhance the problem-solving experience? (2) What is the impact of integrating artificial intelligence and virtual agents on personalized learning? and, (3) How to leverage sensor signals to model and analyze the development process of APS skills? This research will also characterize multiple pathways of analyzing and solving problems, as well as the factors driving these pathways. The project will provide practical tools for xR simulations that will complement classroom instruction and help educators diagnose and tailor instruction to learner needs. Project outcomes will provide hands-on and immersive experiences to diverse learners, including undergraduate and graduate students, and better prepare them for the next industrial revolution. Integration of sensing technologies with xR environments will also facilitate communication and problem-solving for a diversity of learners.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.