NCS-FO: Functional and neural mechanisms of integrating multiple artificial somatosensory feedback signals in prosthesis control

After a limb amputation, both motor and sensory functions associated with the limb are lost. Substantial effort has been devoted to restore lost motor functions. In contrast, there has been less advancement in restoring sensory function. Although earlier works have used sensory substitution or nerve stimulation techniques to elicit a single type of artificial sensation, there is limited understanding of how the human brain integrates different sources of artificial sensation, when multiple sensory stimulations are provided. This project will help understand the integration principles of different artificially evoked sensations of joint movements. By identifying key factors that determine the integration principle of multiple sources of artificial sensation, this project can generate potentially transformative outcomes for human-robot interactions, specifically developing brain-inspired sensory stimulation strategies that can enable intuitive interactions of assistive devices. The project will provide educational opportunities. Different project components will be integrated into existing undergraduate courses. Summer projects incorporating the sensory stimulation techniques will be offered to local school and community college students. Outreach programs associated with the research outcomes will be developed targeting underrepresented students. The goal of this project is to understand the integration principles of different artificially evoked proprioceptive feedback. The research team will combine psychophysical testing, behavioral modeling, and brain signal recordings to understand the integration principle of artificial sensory signals. Proprioceptive feedback of the joint kinematics will be evoked using vibrotactile stimulation and peripheral nerve stimulation. Both upper and lower limbs will be investigated to evaluate whether the integration principle is task or end-effector dependent. The research team will use a Bayesian integration model and electroencephalogram (EEG) recordings to quantify how uncertainty and intuitiveness-associated attentional bias of artificial feedback impact sensory integration. The project outcomes will provide a theoretical basis for developing artificial sensory feedback for intuitive human-robot interactions, and will also provide a research platform for studying sensory perception. This project is funded by Integrative Strategies for Understanding Neural and Cognitive Systems (NCS), a multidisciplinary program jointly supported by the Directorates for Biology (BIO), Computer and Information Science and Engineering (CISE), Education and Human Resources (EHR), Engineering (ENG), and Social, Behavioral, and Economic Sciences (SBE).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.