Application of Reinforcement Learning in Geostatistical Modeling Workflows

Numerous geostatistical methods have been proposed to model spatial and temporal variations in earth science phenomena. There are several algorithms for modeling complex spatial heterogeneity, assimilating data, and quantifying residual uncertainty. The inference and modeling of variogram(s) are crucial for implementing many of these algorithms. Robust variogram inference is crucial for understanding the pattern of spatial (and temporal or spatiotemporal) variations and is a prerequisite for subsurface modeling and characterization. Even though the tasks of variogram inference and modeling are crucial for assessing the spatial structure of geologic attributes, the subjective decisions of the modeler play an oversized role in the process. The accuracy of the ultimate model depends strongly on user abilities and expert knowledge about the underlying phenomenon being modeled. In situations where the available data are sparse, the subjectivity of the modeling process becomes even more apparent. To render the variogram inference and modeling process more objective and somewhat automated, this paper proposes the use of reinforcement learning (RL) methods to gain a better understanding of the underlying spatial and/or temporal processes that manifest in the form of heterogeneities. The RL method may ultimately improve the accuracy of spatial predictions. RL can facilitate the inference of variogram parameters for various nested variogram structures, including the choice of model and the contribution of that structure to the overall spatial variability. The learning agent will develop an optimal policy for variogram modeling by iteratively adjusting variogram parameters and observing the resulting estimations at sampled locations that can be used to provide rewards on the basis of discrepancies between estimated and actual outcomes. The RL framework converges on an optimal policy that maximizes the cumulative reward and results in optimal variogram parameters.