On the identification of model error through observations of time-varying parameters

When performing system identification, it can be possible to realise a deficient model (i.e. one that will make low fidelity predictions) that is able to closely represent a set of training data. For example, the parameters of linear dynamical models can often be tuned to realise a close match to training data that was generated from a system with strong nonlinearities. Despite this close match to available data, these same models may make very poor-quality predictions when shifted even slightly from the 'validation domain' (which could, for example, be a specific time window). In this paper we investigate the hypothesis that, by treating our model's parameters as being time-varying, we can identify key weaknesses in a model that would have been difficult to establish using other identification methods that do not consider the potentially time-varying nature of the model's parameters. Specifically, we use an Extended Kalman Filter to 'track' the parameters of a dynamical system, as a time history of training data is analysed. We then illustrate that this approach can reveal important information about the potential deficiencies of a model.