Definition of Term: Multivariate Gaussian process

The univariate Gaussian process (GP) is a probability distribution for a function, in which the joint distribution of any set of points on that function is multivariate normal. In MUCM it is used to develop an emulator for the output of a simulator, regarding the simulator output as a function \(f(x)\) of its input(s) \(x\).

Most simulators in practice produce multiple outputs (e.g. temperature, pressure, wind speed, …) for any given input configuration. The multivariate GP is an extension of the univariate GP to the case where \(f(x)\) is not a scalar but a vector. If the simulator has \(r\) outputs then \(f(x)\) is \(r\times 1\).

Formally, the multivariate GP is a probability model over functions with multivariate values. It is characterised by a mean function \(m(\cdot) = \textrm{E}[f(\cdot)]\) and a covariance function \(v(\cdot,\cdot) = \textrm{Cov}[f(\cdot),f(\cdot)]\). Under this model, the function evaluated at a single input :math:` x` has a multivariate normal distribution \(f(x) \sim N\left(m(x),v(x,x)\right)\), where:

  • \(m(x)\) is the \(r \times 1\) mean vector of \(f(x)\) and
  • \(v(x,x)\) is the \(r\times r\) covariance matrix of \(f(x)\).

Furthermore, the joint distribution of \(f(x)\) and \(f(x')\) is also multivariate normal:

\[\begin{split}\left( \begin{array}{c}f(x) \\ f(x') \end{array} \right) \sim N \left( \,\left(\begin{array}{c}m(x) \\ m(x') \end{array} \right), \left(\begin{array}{cc}v(x,x) & v(x,x') \\ v(x',x) & v(x',x') \end{array} \right) \,\right)\end{split}\]

so that \(v(x,x')\) is the \(r\times r\) matrix of covariances between elements of \(f(x)\) and \(f(x')\). In general, the multivariate GP is characterised by the fact that if we stack the vectors \(f(x_1), f(x_2),\ldots,f(x_n)\) at an arbitrary set of \(n\) outputs \(D = (x_1,\ldots,x_n)\) into a vector of \(rn\) elements, then this has a multivariate normal distribution. The only formal constraint on the covariance function \(v(.,.)\) is that the \(rn\times rn\) covariance matrix for any arbitrary set \(D\) should always be non-negative definite.

The general multivariate GP is therefore very flexible. However, a special case which has the following more restrictive form of covariance function can be very useful:

\[v(\cdot,\cdot) = \Sigma c(\cdot, \cdot),\]

where \(\Sigma\) is a covariance matrix between outputs and \(c(\cdot, \cdot)\) is a correlation function between input points.

This restriction implies a kind of separability between inputs and outputs. With a covariance function of this form the multivariate GP has an important property, which is exploited in the procedure for emulating multiple outputs ProcBuildMultiOutputGPSep. One aspect of this separable property is that when we stack the vectors \(f(x_1),\ldots,f(x_n)\) into an \(rn\times 1\) vector its covariance matrix has the Kronecker product form \(\Sigma\otimes c(D,D)\), where \(c(D,D)\) is the between inputs correlation matrix with elements \(c(x_i,x_{i'})\) defined according to the notational conventions in the notation page.

Another aspect is that if instead of stacking the output vectors into a long vector we form instead the \(r\times n\) matrix \(f(D)\) (again following the conventions in the notation page) then \(f(D)\) has a matrix-variate normal distribution with mean matrix \(m(D)\), between-rows covariance matrix \(\Sigma\) and between-columns covariance matrix \(c(D,D)\).

References

Information on matrix variate distributions can be found in: