Adaptive service rate control of an M/M/1 queue with server breakdowns

We study service rate control problems for an M/M/1 queue with server breakdowns in which the breakdown rate is assumed to be a function of the service rate. Assuming that the queue has infinite capacity, we first establish the optimality equations for the discounted cost problem and characterize the optimal rate control policies. Then, we characterize the ergodicity of the controlled queue and establish the optimality conditions for the average-cost (ergodic) control problem using the vanishing discounted method. We next study the ergodic control problem when the queue has a finite capacity and establish a verification theorem by directly involving the stationary distribution of the controlled Markov process. For practical applications, we consider the adaptive service rate control problem for the model with finite capacity. Studying this problem is useful because the relationship between the server breakdown rate and the service rate is costly to observe in practice. We propose an adaptive (self-tuning) control algorithm, assuming that the relationship between the server breakdown rate and the service rate is linear with unknown parameters. We prove that the regret vanishes under the algorithm and the proposed policies are asymptotically optimal. In addition, numerical experiments are conducted to validate the algorithm.