Discussion: The Best Input Approach.¶
Description and Background¶
As introduced in the variant thread on linking models to reality (ThreadVariantModelDiscrepancy), the most commonly used method of linking a model \(f\) to reality \(y\) is known as the Best Input approach. This page discusses the Best Input assumptions, highlights the strengths and weaknesses of such an approach and gives links to closely related discussions regarding the model discrepancy \(d\) itself, and to methods that go beyond the Best Input approach. Here we use the term model synonymously with the term simulator.
Notation¶
The following notation and terminology is introduced in ThreadVariantModelDiscrepancy. In accordance with standard toolkit notation, in this page we use the following definitions:
- \(x\) - inputs to the model
- \(f(x)\) - the model function
- \(y\) - the real system value
- \(z\) - an observation of reality \(y\)
- \(x^+\) - the ‘best input’ (see below)
- \(d\) - the model discrepancy (see below)
The full definitions of \(x^+\) and \(d\) are given in the discussion below.
Discussion¶
A model is an imperfect representation of reality, and hence there will always be a difference between model output \(f(x)\) and system value \(y\). Refer to ThreadVariantModelDiscrepancy where this idea is introduced, and to the discussion page on model discrepancy (DiscWhyModelDiscrepancy) for the importance of including this feature in our analysis.
It is of interest to try to represent this difference in the simplest possible manner, and this is achieved by a method known as the Best Input approach. This approach involves the notion that were we to know the actual value, \(x^+\), of the system properties, then the evaluation \(f(x^+)\) would contain all of the information in the model about system performance. In other words, we only allow \(x^+\) to vary because we do not know the appropriate value at which to fix the input. This does not mean that we would expect perfect agreement between \(f(x^+)\) and \(y\). Although the model could be highly sophisticated, it will still offer a necessarily simplified account of the physical system and will most likely approximate the numerical solutions to the governing equations (see DiscWhyModelDiscrepancy for more details). The simplest way to view the difference between \(f^+ = f(x^+)\) and \(y\) is to express this as:
Note that as we are uncertain as to the values of \(y\), \(f^+\) and \(d\), they are all taken to be random quantities. We consider \(d\) to be independent of \(x^+\) and uncorrelated with \(f\) and \(f^+\) (in the Bayes Linear Case) or independent of \(f\) (in the fully Bayesian Case). The Model Discrepancy \(d\) is the subject, in various forms, of ThreadVariantModelDiscrepancy. The definition of the model discrepancy given by the Best Input approach is simple and intuitive, and is widely used in computer modelling studies. It treats the best input \(x^+\) in an analogous fashion to the parameters of a statistical model and (in certain situations) is in accord with the view that \(x^+\) is an expression of the true but unknown properties of the physical system.
In the case where each of the elements of \(x\) correspond to a well defined physical property of the real system, \(x^+\) can simply be thought of as representing the actual values of these physical properties. For example, if the univariate input \(x\) represented the acceleration due to gravity at the Earths surface, \(x^+\) would represent the real system value of \(g=9.81 ms^{-2}\).
It is often the case that certain inputs do not have a direct physical counterpart, but are, instead, some kind of ‘tuning parameter’ inserted into the model to describe approximately some physical process that is either too complex to model accurately, or is not well understood. In this case \(x^+\) can be thought of as the values of these tuning parameters that gives the best performance of the model function \(f(x)\) in some appropriately defined sense (for example, the best agreement between \(f^+\) and the observed data). See the discussion pages on reification (DiscReification) and its theory (DiscReificationTheory) for further details about tuning parameters.
In general, inputs can be of many different types: examples include physical parameters, tuning parameters, aggregates of physical quantities, control (or variable) inputs, or decision parameters. The meaning of \(x^+\) can be different for each type: for a decision parameter or a control variable there might not even be a clearly defined \(x^+\), as we might want to simultaneously optimise the behaviour of \(f\) for all possible values of the decision parameter or the control variable.
The statement that the model discrepancy \(d\) is probabilistically independent of both \(f\) and \(x^+\) (or uncorrelated with \(f\) or \(x^+\) in the Bayes Linear case) is a simple and in many cases largely reasonable assumption, that helps ensure the tractability of subsequent calculations. It involves the idea that the modeller has made all the improvements to the model that s/he can think of, and that beliefs about the remaining inaccuracies of the model would not be altered by knowledge of the function \(f\) or the best input \(x^+\); see further discussions in DiscReification and DiscReificationTheory.
Additional Comments¶
Although useful for many applications, the Best Input approach does break down in certain situations. If we have access to two models, the second a more advanced version of the first, then we cannot use the Best Input assumptions for both models. In this case \(x^+\) would be different for each model, and it would be unrealistic to simultaneously impose the independence assumption on both models. An approach which resolves this issue by modelling relationships across models, known as Reification, is described in DiscReification with a more theoretical treatment given in DiscReificationTheory.