Understanding sequential decisions via inverse reinforcement learning

Siyuan Liu, Miguel Araujo, Emma Brunskill, Rosaldo Rossetti, Joao Barros, Ramayya Krishnan

Research output: Contribution to journalConference articlepeer-review

20 Scopus citations


The execution of an agent's complex activities, comprising sequences of simpler actions, sometimes leads to the clash of conflicting functions that must be optimized. These functions represent satisfaction, short-term as well as long-term objectives, costs and individual preferences. The way that these functions are weighted is usually unknown even to the decision maker. But if we were able to understand the individual motivations and compare such motivations among individuals, then we would be able to actively change the environment so as to increase satisfaction and/or improve performance. In this work, we approach the problem of providing high-level and intelligible descriptions of the motivations of an agent, based on observations of such an agent during the fulfillment of a series of complex activities (called sequential decisions in our work). A novel algorithm for the analysis of observational records is proposed. We also present a methodology that allows researchers to converge towards a summary description of an agent's behaviors, through the minimization of an error measure between the current description and the observed behaviors. This work was validated using not only a synthetic dataset representing the motivations of a passenger in a public transportation network, but also real taxi drivers' behaviors from their trips in an urban network. Our results show that our method is not only useful, but also performs much better than the previous methods, in terms of accuracy, efficiency and scalability.

Original languageEnglish (US)
Article number6569134
Pages (from-to)177-186
Number of pages10
JournalProceedings - IEEE International Conference on Mobile Data Management
StatePublished - 2013
Event14th International Conference on Mobile Data Management, MDM 2013 - Milan, Italy
Duration: Jun 3 2013Jun 6 2013

All Science Journal Classification (ASJC) codes

  • General Engineering


Dive into the research topics of 'Understanding sequential decisions via inverse reinforcement learning'. Together they form a unique fingerprint.

Cite this