Index-aware reinforcement learning for adaptive video streaming at the wireless edge

Guojun Xiong, Xudong Qin, Bin Li, Rahul Singh, Jian Li

Research output: Chapter in Book/Report/Conference proceedingConference contribution

7 Scopus citations


We study adaptive video streaming for multiple users in wireless access edge networks with unreliable channels. The key challenge is to jointly optimize the video bitrate adaptation and resource allocation such that the users' cumulative quality of experience is maximized. This problem is a finite-horizon restless multi-armed multi-action bandit problem and is provably hard to solve. To overcome this challenge, we propose a computationally appealing index policy entitled Quality Index Policy, which is well-defined without the Whittle indexability condition and is provably asymptotically optimal without the global attractor condition. These two conditions are widely needed in the design of most existing index policies, which are difficult to establish in general. Since the wireless access edge network environment is highly dynamic with system parameters unknown and time-varying, we further develop an index-aware reinforcement learning (RL) algorithm dubbed QA-UCB. We show that QA-UCB achieves a sub-linear regret with a low-complexity since it fully exploits the structure of the Quality Index Policy for making decisions. Extensive simulations using real-world traces demonstrate significant gains of proposed policies over conventional approaches. We note that the proposed framework for designing index policy and index-aware RL algorithm is of independent interest and could be useful for other large-scale multi-user problems.

Original languageEnglish (US)
Title of host publicationMobiHoc 2022 - Proceedings of the 2022 23rd International Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and Mobile Computing
PublisherAssociation for Computing Machinery
Number of pages10
ISBN (Electronic)9781450391658
StatePublished - Oct 3 2022
Event23rd ACM International Symposium on Mobile Ad Hoc Networking and Computing, MobiHoc 2022 - Seoul, Korea, Republic of
Duration: Oct 17 2022Oct 20 2022

Publication series

NameProceedings of the International Symposium on Mobile Ad Hoc Networking and Computing (MobiHoc)


Conference23rd ACM International Symposium on Mobile Ad Hoc Networking and Computing, MobiHoc 2022
Country/TerritoryKorea, Republic of

All Science Journal Classification (ASJC) codes

  • Hardware and Architecture
  • Computer Networks and Communications
  • Software

Cite this