Dynamic ensemble of contextual bandits to satisfy users' changing interests

Qingyun Wu, Huazheng Wang, Yanen Li, Hongning Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

16 Scopus citations


Recommender systems have to handle a highly non-stationary environment, due to users' fast changing interests over time. Traditional solutions have to periodically rebuild their models, despite high computational cost. But this still cannot empower them to automatically adjust to abrupt changes in trends caused by timely information. It is important to note that the changes of reward distributions caused by a non-stationary environment can also be context dependent. When the change is orthogonal to the given context, previously maintained models should be reused for better recommendation prediction. In this work, we focus on contextual bandit algorithms for making adaptive recommendations. We capitalize on the unique context-dependent property of reward changes to conquer the challenging non-stationary environment for model update. In particular, we maintain a dynamic ensemble of contextual bandit models, where each bandit model's reward estimation quality is monitored regarding given context and possible environment changes. Only the admissible models to the current environment will be used for recommendation. We provide a rigorous upper regret bound analysis of our proposed algorithm. Extensive empirical evaluations on both synthetic and three real-world datasets confirmed the algorithm's advantage against existing non-stationary solutions that simply create new models whenever an environment change is detected.

Original languageEnglish (US)
Title of host publicationThe Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019
PublisherAssociation for Computing Machinery, Inc
Number of pages11
ISBN (Electronic)9781450366748
StatePublished - May 13 2019
Event2019 World Wide Web Conference, WWW 2019 - San Francisco, United States
Duration: May 13 2019May 17 2019

Publication series

NameThe Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019


Conference2019 World Wide Web Conference, WWW 2019
Country/TerritoryUnited States
CitySan Francisco

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Software


Dive into the research topics of 'Dynamic ensemble of contextual bandits to satisfy users' changing interests'. Together they form a unique fingerprint.

Cite this