Gaussian process regression with functional covariates and multivariate response (original) (raw)
Related papers
Gaussian process methods for nonparametric functional regression with mixed predictors
Computational Statistics & Data Analysis, 2019
Gaussian process methods are proposed for nonparametric functional regression for both scalar and functional responses with mixed multidimensional functional and scalar predictors. The proposed models allow the response variables to depend on the entire trajectories of the functional predictors. They inherit the desirable properties of Gaussian process regression, and can naturally accommodate both scalar and functional variables as the predictors, as well as easy to obtain and express uncertainty in predictions. The numerical experiments show that the proposed methods significantly outperform the competing models, and their usefulness is also demonstrated by the application to two real datasets.
Gaussian process functional regression modeling for batch data
2007
Summary A Gaussian process functional regression model is proposed for the analysis of batch data. Covariance structure and mean structure are considered simultaneously, with the covariance structure modeled by a Gaussian process regression model and the mean structure modeled by a functional regression model. The model allows the inclusion of covariates in both the covariance structure and the mean structure.
Computer Physics Communications, 2022
We present an implementation for the RS-HDMR-GPR (Random Sampling High Dimensional Model Representation Gaussian Process Regression) method. The method builds representations of multivariate functions with lower-dimensional terms, either as an expansion over orders of coupling or using terms of only a given dimensionality. This facilitates, in particular, recovering functional dependence from sparse data. The method also allows for imputation of missing values of the variables and for a significant pruning of the useful number of HDMR terms. It can also be used for estimating relative importance of different combinations of input variables, thereby adding an element of insight to a general machine learning method, in a way that can be viewed as extending the automatic relevance determination approach. The capabilities of the method and of the associated Python software tool are demonstrated on test cases involving synthetic analytic
Gaussian Process Regression for Structured Data Sets
Approximation algorithms are widely used in many engineering problems. To obtain a data set for approximation a factorial design of experiments is often used. In such case the size of the data set can be very large. Therefore, one of the most popular algorithms for approximation — Gaussian Process regression — can hardly be applied due to its computational complexity. In this paper a new approach for a Gaussian Process regression in case of a factorial design of experiments is proposed. It allows to efficiently compute exact inference and handle large multidimensional and anisotropic data sets.
Details on Gaussian Process Regression (GPR) and Semi-GPR Modeling
2019
This report tends to provide details on how to perform predictions using Gaussian process regression (GPR) modeling. In this case, we represent proofs for prediction using non-parametric GPR modeling for noise-free predictions as well as prediction using semi-parametric GPR for noisy observations. 1. GAUSSIAN PROCESS REGRESSION Gaussian processes (GPs) are widely used for modeling a phenomenon based on the observed spatiotemporal data [12]. A GP can be used as a tool for either classification or regression [1, 2, 12]. GPs have been used for decades as a supervised learning tool for regression problems known as Gaussian process regression (GPR) models [1, 2], and are also referred to as kriging, named after the mining engineer D.G. Krige in the geostatistics literature [3-5]. GPR models and kriging methods are applicable to a variety of problems such as the prediction and estimation of temperature, precipitation, missing pixel and un-mixing of pixels in hyperspectral imaging (HSI), human head pose estimation, concentration of carbon dioxide in the atmosphere, etc. [2, 6-11]. In GP modeling, it is assumed that the phenomenon of interest (PoI) can be evaluated via an unknown and probably nonlinear function, which we denote by f (•). The arguments of the function comprise a variable set u referred to as the input data. For example, u can be defined as u = [u x , u y , u z , t] T , where (u x , u y , u z) and t denote the spatial and temporal information about the measurements, respectively. Unlike parametric models such as linear regression, GP is non-parametric. In GP one defines a probability distribution function as a prior over the unknown function f (•), directly. In other words, GP defines a distribution over functions in the function space and the inference is performed directly in this space [2]. This is more general than a parametric model such as Bayesian linear regression, where the prior distribution is defined over the space of parameters. The GP model treats any observation as an outcome of a Gaussian random variable, and all of these random variables are jointly Gaussian. With this setting, any well-defined GP model only needs a mean accompanied with a positive definite covariance function. Under this assumption, GP provides a posterior distribution over the unknown function f once data are observed. Therefore, for any set of N observations with the input data set {u 1 ,. .. , u N }, GP assumes that the distribution p(f (u 1),. .. , f (u N)) is jointly Gaussian with some mean µ(U) and a covariance matrix K(U), where U := [u 1 , ..., u N ]. The entry in row i and column j of K(U) is denoted by [K(U)] ij = κ(u i , u j), where κ(., .) is a positive definite kernel function. The kernel function specifies
Multiple output gaussian process regression
2005
Gaussian processes are usually parameterised in terms of their covariance functions. However, this makes it difficult to deal with multiple outputs, because ensuring that the covariance matrix is positive definite is problematic. An alternative formulation is to treat Gaussian processes as white noise sources convolved with smoothing kernels, and to parameterise the kernel instead. Using this, we extend Gaussian processes to handle multiple, coupled outputs. 1
Computationally Efficient Algorithm for Gaussian Process Regression in Case of Structured Samples 1
Surrogate modeling is widely used in many engineering problems. Data sets often have Cartesian product structure (for instance factorial design of experiments with missing points). In such case the size of the data set can be very large. Therefore, one of the most popular algorithms for approximation – Gaussian Process regression – can be hardly applied due to its computational complexity. In this paper a computationally efficient approach for constructing Gaussian Process regression in case of data sets with Cartesian product structure is presented. Efficiency is achieved by using a special structure of the data set and operations with tensors. Proposed algorithm has low computational as well as memory complexity compared to existing algorithms. In this work we also introduce a regular-ization procedure allowing to take into account anisotropy of the data set and avoid degeneracy of regression model.
A Framework for Evaluating Approximation Methods for Gaussian Process Regression
2012
Gaussian process (GP) predictors are an important component of many Bayesian approaches to machine learning. However, even a straightforward implementation of Gaussian process regression (GPR) requires O(n 2 ) space and O(n 3 ) time for a dataset of n examples. Several approximation methods have been proposed, but there is a lack of understanding of the relative merits of the different approximations, and in what situations they are most useful. We recommend assessing the quality of the predictions obtained as a function of the compute time taken, and comparing to standard baselines (e.g., Subset of Data and FITC). We empirically investigate four different approximation algorithms on four different prediction problems, and make our code available to encourage future comparisons.
Coupled Gaussian Processes for Functional Data Analysis
SIS2019: Smart Statistics for smart Applications Book of Short Papers, 2019
We present an approach for modelling multivariate dependent functional data. To account for the dominant structural features of the data, we rely on the theory of Gaussian Processes and extend hierarchical dynamic linear models for multivariate time series to the functional data setting. We illustrate the proposed methodology within the framework of bivariate functional data and discuss problems referring to detection of spatial patterns and curve prediction.
Hierarchical Gaussian process mixtures for regression
2005
As a result of their good performance in practice and their desirable analytical properties, Gaussian process regression models are becoming increasingly of interest in statistics, engineering and other fields. However, two major problems arise when the model is applied to a large data-set with repeated measurements. One stems from the systematic heterogeneity among the different replications, and the other is the requirement to invert a covariance matrix which is involved in the implementation of the model.