Deep reinforcement learning-driven life-cycle management of bridge and pavement systems

A. Bhattacharya, M. Saifullah, K. G. Papakonstantinou

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Optimal management of bridge systems and related transportation infrastructure poses multi-faceted challenges, requiring adept inspection and maintenance policies at both system and individual asset levels, to minimize life-cycle costs while considering various operational, risk, and performance constraints. This demanding type of optimization problems entails, among others, high-dimensional aspects, describing multi-component systems, long-planning horizons, diverse probabilistic and deterministic operational objectives and constraints, and inherent uncertainties associated with inspections and stochastic models. Effective coordination among individual component assets considering various inter-dependencies is also essential to enable a true system-based optimal solution. In this work, this optimization problem is formulated within the framework of Partially Observable Markov Decision Processes (POMDPs) and constrained Multi-Agent Deep Reinforcement Learning (MARL). POMDPs offer a principled mathematical approach for sequential decision-making under uncertainty, incorporating Bayesian inference to address the observation/monitoring data uncertainty, and can be suitably scaled to high-dimensional state and action spaces associated with multi-component systems, exploiting the rich representational capacities of deep learning and decentralized control settings of MARL. In this work, the recently developed DDMAC deep reinforcement learning (DRL) algorithm (Deep Decentralized Multi-Agent Actor-Critic) has been successfully deployed based on the Centralized Training and Decentralized Execution (CTDE) formulation. The efficacy and implementation aspects of the developed framework are originally studied in this work based on two existing real-world transportation networks in Virginia and Pennsylvania, USA, following all regulations imposed by the relevant agencies, as well as their overall practices, in an effort to investigate the use of the suggested framework in practical, actual settings. In both cases, DRL results significantly surpass the ones related to current state-of-practice and state-of-the-art policies, providing further support and insights toward the use of DRL-driven policies for infrastructure management.

Original languageEnglish (US)
Title of host publicationBridge Maintenance, Safety, Management, Digitalization and Sustainability - Proceedings of the 12th International Conference on Bridge Maintenance, Safety and Management, IABMAS 2024
EditorsJens Sandager Jensen, Dan M. Frangopol, Jacob Wittrup Schmidt
PublisherCRC Press/Balkema
Pages3380-3388
Number of pages9
ISBN (Print)9781032770406
DOIs
StatePublished - 2024
Event12th International Conference on Bridge Maintenance, Safety and Management, IABMAS 2024 - Copenhagen, Denmark
Duration: Jun 24 2024Jun 28 2024

Publication series

NameBridge Maintenance, Safety, Management, Digitalization and Sustainability - Proceedings of the 12th International Conference on Bridge Maintenance, Safety and Management, IABMAS 2024

Conference

Conference12th International Conference on Bridge Maintenance, Safety and Management, IABMAS 2024
Country/TerritoryDenmark
CityCopenhagen
Period6/24/246/28/24

All Science Journal Classification (ASJC) codes

  • Building and Construction
  • Safety Research
  • Civil and Structural Engineering

Fingerprint

Dive into the research topics of 'Deep reinforcement learning-driven life-cycle management of bridge and pavement systems'. Together they form a unique fingerprint.

Cite this