A Double-Agent Reinforcement Learning Framework for Automated Variogram Parameter Estimation in Linear Models of Coregionalization

Objectives/Scope: This paper introduces a dual-agent reinforcement learning (RL) framework tailored to refine variogram parameters within linear models of coregionalization that can be used for assimilating multi-physics data in reservoir models. By targeting direct and cross-dependencies among reservoir variables, the approach aims to reduce trial-and-error calibration efforts while sustaining high accuracy in predictive geostatistical reservoir models, especially in settings with limited primary data. Methods, Procedures, Process: The methodology employs two dedicated RL agents operating in tandem. The first agent tunes the spatial correlation model for the primary variable using goodness of fit as the reward function for training the agent. Subsequently, a second agent focuses on adjusting cross-variogram parameters by exploiting intervariable correlations. Iterative feedback loops allow each agent to refine its parameter estimations based on goodness of fit calculations made at every iteration. The framework is evaluated through synthetic datasets, where multivariate conditional simulation compares the modelled reservoir properties against known references, assessing both convergence speed and predictive reliability. Results, Observations, Conclusions: Empirical evaluations on comprehensive synthetic datasets demonstrate that the proposed dualagent RL approach achieves exceptional performance with an average goodness score of 0.96 (range: 0.92-0.99), significantly exceeding typical values (0.70-0.85) reported for traditional calibration methods. This improvement translates to 20-30% better accuracy in uncertainty quantification for reserve estimation. The framework successfully handles diverse variogram structures including spherical, exponential, and Gaussian models with both single and nested anisotropic configurations. Calibrated models achieve target correlations with mean absolute error of 0.041 while maintaining all required mathematical constraints. Agent A efficiently optimizes diagonal sill parameters controlling individual variable variances, while Agent B manages cross-sill coefficients determining intervariable correlations. Computational efficiency enables practical application with calibration times ranging from 2.8 minutes (2D single structure) to 18.3 minutes (3D nested anisotropic structures), representing an 85% reduction compared to manual workflows that typically require 2-3 weeks of expert time. Novel/Additive Information: Unlike conventional techniques for assimilating data within geostatistical workflows that use subjective decisions and are tedious, this study's double-agent RL system orchestrates the inference and modeling of all measures for primary, secondary, and cross-correlations in a single cohesive process. This paper showcases the enhancement of the accuracy of simulation techniques by seamlessly integrating multiple data sources; the approach significantly enhances the simulation process and prediction accuracy of co-simulation techniques.