District-level bridge networks management with multi-agent reinforcement learning: from theory to real-world application

The practical use of multi-agent reinforcement learning (MARL) solutions for real-world bridge networks comprising thousands of structures and subject to multiple uncertainties and constraints remains an open optimization challenge. Further work is thus needed in this direction, particularly related to scalability and applicability issues. This paper adds to this discussion and efforts and provides MARL solutions to existing bridge networks in Pennsylvania, USA, through the Deep Decentralized Multi-Agent Actor-Critic with Centralized Training and Decentralized Execution (DDMAC-CTDE) framework. The presented approach integrates stochastic deterioration models, uncertain observations, several maintenance actions, and cost-risk assessments, optimizing the maintenance of aging bridge assets under both deterministic and stochastic resource and condition constraints, with all application aspects fully aligned with the methodologies and parameters followed by the Pennsylvania Department of Transportation for bridge asset management. To address scalability challenges, we categorize MARL learning paradigms and examine their limitations, introducing approximation-based techniques to handle networks with thousands of bridges. Optimization results on several examples, including a district-level Pennsylvania network with 3,000 bridges, showcase the effectiveness of our approach. Overall, this work represents a step forward regarding the real-world deployment of MARL for large-scale infrastructure management, aiming to bridge the gap between theoretical advancements and engineering implementations.