Model-Free Dynamic Operations Management for EV Battery Swapping Stations: A Deep Reinforcement Learning Approach

Battery swapping stations (BSS) have recently grabbed the attention of transportation firms as a viable method for accelerating the deployment of electric vehicles. However, most existing literature considered model-based approaches lacking dynamicity to optimize their swapping systems. To address this problem, this paper introduces a model-free optimal dynamic operations framework for BSS using novel deep reinforcement learning (DRL) approaches. The main goal is to minimize the BSS's running costs by controlling the charging/discharging and swapping actions. First, the system is formulated as a Markov decision process. Then, two DRL approaches, the double buffer deep deterministic policy gradient (DB-DDPG) and twin delayed DDPG (TD3), are utilized to achieve optimal control of the BSS operations. Unlike existing methods, the charging characteristics, the swapping time, and the battery degradation are considered in the proposed framework to obtain realistic results. The proposed approach is model-free and can adaptively learn an optimal policy. Hence, it is more robust to uncertainties than existing model-based methods that require previous knowledge of the uncertainty distribution. Simulation results considering real-world data verify the effectiveness of the proposed solution and show that the proposed DRL approaches outperform the classical DDPG method.