A framework for comparing stochastic simulation models against multidimensional data using the Wasserstein distance

In many applications, there is a need to compare multiple simulation models or parameter configurations in order to choose the model(s) whose output best resembles the historical data available from the system under study. This paper presents a novel framework that enables such analysis for multidimensional output statistics such as those found in weather/climate, epidemic, swarm/crowd control, social systems, communication networks, and other simulation applications with spatial output statistics that are distributed across various locations or geographical regions. The proposed framework uses the Wasserstein distance from the target data as the primary performance metric and statistically compares the models to determine the best-performing candidate(s). The efficacy of the proposed framework is illustrated through three examples related to random walk of swarm particles on a two-dimensional space, Monte Carlo simulation with correlated bivariate output, and a realistic engineering application involving simulation of unmanned aerial vehicle (UAV) communication systems with spatial output.