POMDP and MOMDP solutions for structural life-cycle cost minimization under partial and mixed observability

Konstantinos G. Papakonstantinou, Charalampos P. Andriotis, Masanobu Shinozuka

Research output: Contribution to journalArticlepeer-review

30 Scopus citations


Scheduling of inspection and maintenance policies during the life-cycle of operating infrastructure necessitates optimization of long-term objectives in stochastic environments. Modern answers to the problem should focus on quantitative decision-making techniques, taking advantage of informative but uncertain data that become available in time. As such, the problem is efficiently addressed within the framework of stochastic dynamic programming by means of Partially Observable Markov Decision Processes (POMDPs) and Mixed Observability Markov Decision Processes (MOMDPs). Although these methodologies can provide very sophisticated solutions with optimality guarantees, important computational challenges often emerge, mainly due to the continuity of the multidimensional belief space on the probability simplex. In response, recent value iteration algorithms based on point-based approaches have been suggested, focusing on reachable belief points that can support an accurate value function. In this work, several POMDP and MOMDP point-based algorithms, with various characteristics regarding the exploration of the belief space and the value function update procedures, are rigorously analyzed. The algorithms are compared and evaluated in terms of accuracy and performance in stationary and nonstationary problems of structural inspection and maintenance for life-cycle cost minimization. Results are thoroughly discussed and several insights along with practical suggestions for similar problems are provided.

Original languageEnglish (US)
Pages (from-to)869-882
Number of pages14
JournalStructure and Infrastructure Engineering
Issue number7
StatePublished - Jul 3 2018

All Science Journal Classification (ASJC) codes

  • Civil and Structural Engineering
  • Building and Construction
  • Safety, Risk, Reliability and Quality
  • Geotechnical Engineering and Engineering Geology
  • Ocean Engineering
  • Mechanical Engineering


Dive into the research topics of 'POMDP and MOMDP solutions for structural life-cycle cost minimization under partial and mixed observability'. Together they form a unique fingerprint.

Cite this