Discussion: The GP covariance function¶
Description and Background¶
In the fully Bayesian approach an emulator is created based on a Gaussian process (GP), and one of the principal steps in building the emulator is to specify the GP’s covariance function. The covariance function also has an analogous role in the Bayes linear approach. See the thread for the analysis of the core model using Gaussian process methods (ThreadCoreGP) and the thread for Bayes linear emulation for the core model (ThreadCoreBL) for emulation of the core problem from the fully Bayesian and Bayes linear approaches, respectively.
Discussion¶
The covariance function gives the prior covariance between the simulator output at any given vector of input values and the output at any other given input vector. When these two vectors are the same, the covariance function specifies the prior variance of the output at that input vector, and so quantifies prior uncertainty about the simulator output. In principal, this function can take a very wide variety of forms.
Within the MUCM toolkit, we make an assumption of constant prior variance, so that in the case of a single output (as in the core problem) the covariance function has the form \(\sigma^2 c(\cdot, \cdot)\), where \(c(\cdot, \cdot)\) is a correlation function having the property that, for any input vector \(x\), \(c(x,x)=1\). Hence the interpretation of \(\sigma^2\) is as the variance of the output (at any input). The correlation function generally depends on a set of hyperparameters denoted by \(\delta\), while \(\sigma^2\) is another hyperparameter.
When building a multivariate GP emulator for a simulator with multiple outputs, the covariance between the sets of outputs at inputs \(x\) and \(x'\) is a matrix which is assumed to have the separable form \(\Sigma c(\cdot, \cdot)\), where now \(\Sigma\) is a between-outputs covariance matrix but \(c(\cdot, \cdot)\) is a correlation function over the input space with the same interpretation for each output as in the single output case. See the variant thread for the analysis of a simulator with multiple outputs using Gaussian process methods (ThreadVariantMultipleOutputs).
Additional Comments¶
More generally, we might relax the assumption that the variance is constant across the input space. However, it is important for the resulting covariance function to be valid, in the sense that the implied variance for any linear combination of outputs at different input points remains positive.
One valid covariance function in which the variance is not constant takes the form \(u(x) u(x') c(x,x')\), where \(c(\cdot, \cdot)\) is a correlation function (see the alternatives page on emulator prior correlation function (AltCorrelationFunction)) as before, while \(u(x)^2\) is the variance of the output at inputs \(x\), and can now vary with \(x\). In general, \(u(\cdot)\) would be modelled using further hyperparameters.