TY - GEN
T1 - NaviSense
T2 - 27th International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS 2025
AU - Sridhar, Ajay Narayanan
AU - Qiao, Fuli
AU - Troncoso Aldas, Nelson Daniel
AU - Shi, Yanpei
AU - Mahdavi, Mehrdad
AU - Itti, Laurent
AU - Narayanan, Vijaykrishnan
N1 - Publisher Copyright:
© 2025 Copyright held by the owner/author(s).
PY - 2025/10/22
Y1 - 2025/10/22
N2 - People with visual impairments often face significant challenges in locating and retrieving objects in their surroundings. Existing assistive technologies present a trade-off: systems that offer precise guidance typically require pre-scanning or support only fixed object categories, while those with open-world object recognition lack spatial feedback for reaching the object. To address this gap, we introduce NaviSense, a mobile assistive system that combines conversational AI, vision-language models, augmented reality (AR), and LiDAR to support open-world object detection with real-time audio-haptic guidance. Users specify objects via natural language and receive continuous spatial feedback to navigate toward the target without needing prior setup. Designed with insights from a formative study and evaluated with 12 blind and low-vision participants, NaviSense significantly reduced object retrieval time and was preferred over existing tools, demonstrating the value of integrating open-world perception with precise, accessible guidance.
AB - People with visual impairments often face significant challenges in locating and retrieving objects in their surroundings. Existing assistive technologies present a trade-off: systems that offer precise guidance typically require pre-scanning or support only fixed object categories, while those with open-world object recognition lack spatial feedback for reaching the object. To address this gap, we introduce NaviSense, a mobile assistive system that combines conversational AI, vision-language models, augmented reality (AR), and LiDAR to support open-world object detection with real-time audio-haptic guidance. Users specify objects via natural language and receive continuous spatial feedback to navigate toward the target without needing prior setup. Designed with insights from a formative study and evaluated with 12 blind and low-vision participants, NaviSense significantly reduced object retrieval time and was preferred over existing tools, demonstrating the value of integrating open-world perception with precise, accessible guidance.
UR - https://www.scopus.com/pages/publications/105022605953
UR - https://www.scopus.com/pages/publications/105022605953#tab=citedBy
U2 - 10.1145/3663547.3759726
DO - 10.1145/3663547.3759726
M3 - Conference contribution
AN - SCOPUS:105022605953
T3 - ASSETS 2025 - Proceedings of the 27th International ACM SIGACCESS Conference on Computers and Accessibility
BT - ASSETS 2025 - Proceedings of the 27th International ACM SIGACCESS Conference on Computers and Accessibility
A2 - Shinohara, Kristen
A2 - Bennett, Cynthia L.
A2 - Mott, Martez
A2 - Kane, Shaun K.
PB - Association for Computing Machinery, Inc
Y2 - 26 October 2025 through 29 October 2025
ER -