Show Me What To Pick: Pointing Versus Spatial Gestures for Conveying Intent

Vidullan Surendran, Alan R. Wagner

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Gestures are a convenient modality of conveying human intent in collaborative human-robot tasks, and the pointing gesture is commonly used in pick-and-place tasks. But, it is hard to accurately detect the location pointed to with stereo cameras, and experiments in the literature tend to space out the objects of interest in order to make the task easier. We propose the use of gestures conveying spatial directions as an alternative to the pointing gesture when objects are closely packed, since inaccuracies in the detection of the spatial location pointed to can increase task completion difficulty. Using a human study, we confirmed that the gestures we propose are naturally used by humans collaborating with other humans when performing the task. Then, we develop a computer vision pipeline capable of generating a vector representing the pointing direction, and detecting specific spatial gestures from an RGB-D video stream. Using a self report survey, we show statistically significant evidence that subjects report higher satisfaction and better team performance when using spatial gestures instead of the pointing gesture to communicate with a robotic teammate. Finally, we show preliminary evidence that this trend holds true even when the accuracy of the pointing location detection is artificially inflated.

Original languageEnglish (US)
Title of host publication2023 32nd IEEE International Conference on Robot and Human Interactive Communication, RO-MAN 2023
PublisherIEEE Computer Society
Pages77-84
Number of pages8
ISBN (Electronic)9798350336702
DOIs
StatePublished - 2023
Event32nd IEEE International Conference on Robot and Human Interactive Communication, RO-MAN 2023 - Busan, Korea, Republic of
Duration: Aug 28 2023Aug 31 2023

Publication series

NameIEEE International Workshop on Robot and Human Communication, RO-MAN
ISSN (Print)1944-9445
ISSN (Electronic)1944-9437

Conference

Conference32nd IEEE International Conference on Robot and Human Interactive Communication, RO-MAN 2023
Country/TerritoryKorea, Republic of
CityBusan
Period8/28/238/31/23

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Computer Vision and Pattern Recognition
  • Human-Computer Interaction
  • Software

Cite this