Trajectory Planning in an Urban Scenario with Heuristic Guided Reinforcement Learning

The use of artificial intelligence for planning safe and efficient trajectories for autonomous vehicles in dynamic and complex urban environments has grown rapidly. In this paper, we present a novel methodology for autonomous vehicle trajectory planning using Reinforcement Learning with a heuristic based reward function. The decision boundary formulated by a Support Vector Machine (SVM) classifier is used as a heuristic within the reward function to guide the agent along a smooth and collision-free trajectory toward a predefined goal position in a roundabout scenario. This heuristic based reward function is initially coupled with a Time to Collision (TTC) warning system, which is later replaced by dual SVM classifiers for trajectory and collision prediction of moving vehicles. The Soft Actor Critic (SAC) and Deep Deterministic Policy Gradient (DDPG) algorithms are used to train the agent to navigate safely through the dynamic roundabout scenario in minimum time to a goal position. The effectiveness of the proposed methodology is evaluated through simulations and compared against a Spatio-temporal lattice trajectory planner that uses an SVM based classifier as heuristic in its A∗search algorithm.