TY - GEN
T1 - Deep reinforcement learning-driven life-cycle management of bridge and pavement systems
AU - Bhattacharya, A.
AU - Saifullah, M.
AU - Papakonstantinou, K. G.
N1 - Publisher Copyright:
© 2024 The Author(s).
PY - 2024
Y1 - 2024
N2 - Optimal management of bridge systems and related transportation infrastructure poses multi-faceted challenges, requiring adept inspection and maintenance policies at both system and individual asset levels, to minimize life-cycle costs while considering various operational, risk, and performance constraints. This demanding type of optimization problems entails, among others, high-dimensional aspects, describing multi-component systems, long-planning horizons, diverse probabilistic and deterministic operational objectives and constraints, and inherent uncertainties associated with inspections and stochastic models. Effective coordination among individual component assets considering various inter-dependencies is also essential to enable a true system-based optimal solution. In this work, this optimization problem is formulated within the framework of Partially Observable Markov Decision Processes (POMDPs) and constrained Multi-Agent Deep Reinforcement Learning (MARL). POMDPs offer a principled mathematical approach for sequential decision-making under uncertainty, incorporating Bayesian inference to address the observation/monitoring data uncertainty, and can be suitably scaled to high-dimensional state and action spaces associated with multi-component systems, exploiting the rich representational capacities of deep learning and decentralized control settings of MARL. In this work, the recently developed DDMAC deep reinforcement learning (DRL) algorithm (Deep Decentralized Multi-Agent Actor-Critic) has been successfully deployed based on the Centralized Training and Decentralized Execution (CTDE) formulation. The efficacy and implementation aspects of the developed framework are originally studied in this work based on two existing real-world transportation networks in Virginia and Pennsylvania, USA, following all regulations imposed by the relevant agencies, as well as their overall practices, in an effort to investigate the use of the suggested framework in practical, actual settings. In both cases, DRL results significantly surpass the ones related to current state-of-practice and state-of-the-art policies, providing further support and insights toward the use of DRL-driven policies for infrastructure management.
AB - Optimal management of bridge systems and related transportation infrastructure poses multi-faceted challenges, requiring adept inspection and maintenance policies at both system and individual asset levels, to minimize life-cycle costs while considering various operational, risk, and performance constraints. This demanding type of optimization problems entails, among others, high-dimensional aspects, describing multi-component systems, long-planning horizons, diverse probabilistic and deterministic operational objectives and constraints, and inherent uncertainties associated with inspections and stochastic models. Effective coordination among individual component assets considering various inter-dependencies is also essential to enable a true system-based optimal solution. In this work, this optimization problem is formulated within the framework of Partially Observable Markov Decision Processes (POMDPs) and constrained Multi-Agent Deep Reinforcement Learning (MARL). POMDPs offer a principled mathematical approach for sequential decision-making under uncertainty, incorporating Bayesian inference to address the observation/monitoring data uncertainty, and can be suitably scaled to high-dimensional state and action spaces associated with multi-component systems, exploiting the rich representational capacities of deep learning and decentralized control settings of MARL. In this work, the recently developed DDMAC deep reinforcement learning (DRL) algorithm (Deep Decentralized Multi-Agent Actor-Critic) has been successfully deployed based on the Centralized Training and Decentralized Execution (CTDE) formulation. The efficacy and implementation aspects of the developed framework are originally studied in this work based on two existing real-world transportation networks in Virginia and Pennsylvania, USA, following all regulations imposed by the relevant agencies, as well as their overall practices, in an effort to investigate the use of the suggested framework in practical, actual settings. In both cases, DRL results significantly surpass the ones related to current state-of-practice and state-of-the-art policies, providing further support and insights toward the use of DRL-driven policies for infrastructure management.
UR - http://www.scopus.com/inward/record.url?scp=85200393673&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85200393673&partnerID=8YFLogxK
U2 - 10.1201/9781003483755-400
DO - 10.1201/9781003483755-400
M3 - Conference contribution
AN - SCOPUS:85200393673
SN - 9781032770406
T3 - Bridge Maintenance, Safety, Management, Digitalization and Sustainability - Proceedings of the 12th International Conference on Bridge Maintenance, Safety and Management, IABMAS 2024
SP - 3380
EP - 3388
BT - Bridge Maintenance, Safety, Management, Digitalization and Sustainability - Proceedings of the 12th International Conference on Bridge Maintenance, Safety and Management, IABMAS 2024
A2 - Jensen, Jens Sandager
A2 - Frangopol, Dan M.
A2 - Schmidt, Jacob Wittrup
PB - CRC Press/Balkema
T2 - 12th International Conference on Bridge Maintenance, Safety and Management, IABMAS 2024
Y2 - 24 June 2024 through 28 June 2024
ER -