Neural network-based learning from demonstration of an autonomous ground robot

Yiwei Fu, Devesh K. Jha, Zeyu Zhang, Zhenyuan Yuan, Asok Ray

Research output: Contribution to journalArticlepeer-review

13 Scopus citations


This paper presents and experimentally validates a concept of end-to-end imitation learning for autonomous systems by using a composite architecture of convolutional neural network (ConvNet) and Long Short Term Memory (LSTM) neural network. In particular, a spatio-temporal deep neural network is developed, which learns to imitate the policy used by a human supervisor to drive a car-like robot in a maze environment. The spatial and temporal components of the imitation model are learned by using deep convolutional network and recurrent neural network architectures, respectively. The imitation model learns the policy of a human supervisor as a function of laser light detection and ranging (LIDAR) data, which is then used in real time to drive a robot in an autonomous fashion in a laboratory setting. The performance of the proposed model for imitation learning is compared with that of several other state-of-the-art methods, reported in the machine learning literature, for spatial and temporal modeling. The learned policy is implemented on a robot using a Nvidia Jetson TX2 board which, in turn, is validated on test tracks. The proposed spatio-temporal model outperforms several other off-the-shelf machine learning techniques to learn the policy.

Original languageEnglish (US)
Article number24
Issue number2
StatePublished - Jun 1 2019

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Computer Science (miscellaneous)
  • Mechanical Engineering
  • Control and Optimization
  • Industrial and Manufacturing Engineering
  • Electrical and Electronic Engineering


Dive into the research topics of 'Neural network-based learning from demonstration of an autonomous ground robot'. Together they form a unique fingerprint.

Cite this