Optimal Control of Markov-Modulated Multiclass Many-Server Queues

We study multiclass many-server queues for which the arrival, service, and abandonment rates are all modulated by a common finite-state Markov process. We as-sume that the system operates in the “averaged” Halfin–Whitt regime, which means that it is critically loaded in the average sense although not necessarily in each state of the Markov process. We show that, under any static priority policy, the Markov-modulated diffusion-scaled queueing process is exponentially ergodic. This is accomplished by em-ploying a solution to an associated Poisson equation to construct a suitable Lyapunov function. We establish a functional central limit theorem for the diffusion-scaled queueing process and show that the limiting process is a controlled diffusion with piecewise linear drift and constant covariance matrix. We address the infinite-horizon discounted and long-run average (ergodic) optimal control problems and establish asymptotic optimality.