Planning structural inspection and maintenance policies via dynamic programming and Markov processes. Part II: POMDP implementation

Research output: Contribution to journalArticlepeer-review

97 Scopus citations


The overall objective of this two part study is to highlight the advanced attributes, capabilities and use of stochastic control techniques, and especially Partially Observable Markov Decision Processes (POMDPs), that can address the conundrum of planning optimum inspection/monitoring and maintenance policies based on stochastic models and uncertain structural data in real time. In this second part of the study a distinct, advanced, infinite horizon POMDP formulation with 332 states is cast and solved, related to a corroding reinforced concrete structure and its minimum life-cycle cost. The formation and solution of the problem modernize and extend relevant approaches and motivate the use of POMDP methods in challenging practical applications. Apart from uncertain observations the presented framework can also support uncertain action outcomes, non-periodic inspections and choice availability of inspection/monitoring types and intervals, as well as maintenance actions and action times. It is thus no surprise that the estimated optimum policy consists of a complex combination of a variety of actions, which cannot be achieved by any other method. To be able to solve the problem we resort to a point-based value iteration solver and we evaluate its performance and solution quality for this type of applications. Simpler approximate solvers based on MDPs are also used and compared and the important notions of observation gathering actions and the value of information are briefly discussed.

Original languageEnglish (US)
Pages (from-to)214-224
Number of pages11
JournalReliability Engineering and System Safety
StatePublished - Oct 2014

All Science Journal Classification (ASJC) codes

  • Safety, Risk, Reliability and Quality
  • Industrial and Manufacturing Engineering


Dive into the research topics of 'Planning structural inspection and maintenance policies via dynamic programming and Markov processes. Part II: POMDP implementation'. Together they form a unique fingerprint.

Cite this