Optimizing Input Data Acquisition for Ranking and Selection: A View Through the Most Probable Best

This paper concerns a Bayesian ranking and selection (R&S) problem under input uncertainty when all solutions are simulated with common input models estimated from data. We assume that there are multiple independent input data sources from which additional data can be collected at a cost to reduce input uncertainty. To optimize input data acquisition, we first show that the most probable best (MPB)-the solution with the largest posterior probability of being optimal (posterior preference)-is a strongly consistent estimator for the real-world optimum. We investigate the optimal asymptotic static sampling ratios from the input data sources that maximizes the exponential convergence rate of the MPB's posterior preference. We then create a sequential sampling rule that balances the simulation and input data collection effort. The proposed algorithm stops with posterior confidence in the solution quality.