John Jakeman - Profile on Academia.edu (original) (raw)

Papers by John Jakeman

Deep Learning of Parameterized Equations with Applications to Uncertainty Quantification

International Journal for Uncertainty Quantification

The Future of Sensitivity Analysis: An essential discipline for systems modeling and policy support

Environmental Modelling & Software

Journal of Machine Learning for Modeling and Computing

Gaussian process regression is a popular Bayesian framework for surrogate modeling of expensive d... more Gaussian process regression is a popular Bayesian framework for surrogate modeling of expensive data sources. As part of a broader effort in scientific machine learning, many recent works have incorporated physical constraints or other a priori information within Gaussian process regression to supplement limited data and regularize the behavior of the model. We provide an overview and survey of several classes of Gaussian process constraints, including positivity or bound constraints, monotonicity and convexity constraints, differential equation constraints provided by linear PDEs, and boundary condition constraints. We compare the strategies behind each approach as well as the differences in implementation, concluding with a discussion of the computational challenges introduced by constraints.

MFNets: MULTI-FIDELITY DATA-DRIVEN NETWORKS FOR BAYESIAN LEARNING AND PREDICTION

International Journal for Uncertainty Quantification

LDRD #218317: Learning Hidden Structure in Multi-Fidelity Information Sources for Efficient Uncertainty Quantification

Arctic Tipping Points Triggering Global Change LDRD Final Report

LDRD Project Summary: Incorporating physical constraints into Gaussian process surrogate models

Environmental Modelling & Software

Neural Networks as Surrogates of Nonlinear High-Dimensional Parameter-to-Prediction Maps

SIAM/ASA Journal on Uncertainty Quantification

The recovery of approximately sparse or compressible coefficients in a polynomial chaos expansion... more The recovery of approximately sparse or compressible coefficients in a polynomial chaos expansion is a common goal in many modern parametric uncertainty quantification (UQ) problems. However, relatively little effort in UQ has been directed toward theoretical and computational strategies for addressing the sparse corruptions problem, where a small number of measurements are highly corrupted. Such a situation has become pertinent today since modern computational frameworks are sufficiently complex with many interdependent components that may introduce hardware and software failures, some of which can be difficult to detect and result in a highly polluted simulation result. In this paper we present a novel compressive sampling-based theoretical analysis for a regularized 1 minimization algorithm that aims to recover sparse expansion coefficients in the presence of measurement corruptions. Our recovery results are uniform (the theoretical guarantees hold for all compressible signals and compressible corruptions vectors), and prescribe algorithmic regularization parameters in terms of a user-defined a priori estimate on the ratio of measurements that are believed to be corrupted. We also propose an iteratively reweighted optimization algorithm that automatically refines the value of the regularization parameter, and empirically produces superior results. Our numerical results test our framework on several medium-to-high dimensional examples of solutions to parameterized differential equations, and demonstrate the effectiveness of our approach. * B. Adcock and A. Bao acknowledge the support of the Alfred P. Sloan Foundation and the Natural Sciences and Engineering Research Council of Canada through grant 611675. † J.D.Jakeman's work was supported by DARPA EQUiPS.

Journal of Computational Physics

In this paper we present a basis selection method that can be used with 1-minimization to adaptiv... more In this paper we present a basis selection method that can be used with 1-minimization to adaptively determine the large coefficients of polynomial chaos expansions (PCE). The adaptive construction produces anisotropic basis sets that have more terms in important dimensions and limits the number of unimportant terms that increase mutual coherence and thus degrade the performance of 1-minimization. The important features and the accuracy of basis selection are demonstrated with a number of numerical examples. Specifically, we show that for a given computational budget, basis selection produces a more accurate PCE than would be obtained if the basis is fixed a priori. We also demonstrate that basis selection can be applied with non-uniform random variables and can leverage gradient information.

A Generalized Sampling and Preconditioning Scheme for Sparse Approximation of Polynomial Chaos Expansions

SIAM Journal on Scientific Computing

Local Polynomial Chaos Expansion for Linear Differential Equations with High Dimensional Random Inputs

Siam Journal on Scientific Computing, Jan 8, 2015

In this paper we present a locally and dimension-adaptive sparse grid method for interpolation an... more In this paper we present a locally and dimension-adaptive sparse grid method for interpolation and integration of high-dimensional functions with discontinuities. The proposed algorithm combines the strengths of the generalised sparse grid algorithm and hierarchical surplus-guided local adaptivity. A high-degree basis is used to obtain a high-order method which, given sufficient smoothness, performs significantly better than the piecewise-linear basis. The underlying generalised sparse grid algorithm greedily selects the dimensions and variable interactions that contribute most to the variability of a function. The hierarchical surplus of points within the sparse grid is used as an error criterion for local refinement with the aim of concentrating computational effort within rapidly varying or discontinuous regions. This approach limits the number of points that are invested in 'unimportant' dimensions and regions within the high-dimensional domain. We show the utility of the proposed method for non-smooth functions with hundreds of variables.

Enhancing $ ell_1$-minimization estimates of polynomial chaos expansions using basis selection

Dimension reduction for PDE using local Karhunen Loeve expansions

Inundation Modelling of the December 2004 Indian Ocean Tsunami

Geoscience Australia, in an open collaboration with the Mathematical Sciences Institute, The Aust... more Geoscience Australia, in an open collaboration with the Mathematical Sciences Institute, The Australian National University, is developing a software ap-plication, ANUGA, to model the hydrodynamics of floods, storm surges and tsunamis. The free source software implements a finite volume central-upwind Godunov method to solve the non-linear depth-averaged shallow water wave equations. In light of the renewed interest in tsunami forecasting and mitigation, this paper explores the use of ANUGA to model the inundation of the Indian Ocean tsunami of December 2004. The Method of Splitting Tsunamis (MOST) was used to simulate the initial tsunami source and the tsunami's propagation at depths greater than 100m. The resulting output was used to provide boundary conditions to the ANUGA model in the shallow water. Data with respect to 4-minute bathymetry, 2-minute bathymetry, 3-arc second bathymetry and elevation were used in the open ocean, shallow water and on land, respectively. A parti...

Geoscience Australia, in an open collaboration with the Mathematical Sciences Institute, The Aust... more Geoscience Australia, in an open collaboration with the Mathematical Sciences Institute, The Australian National University, is developing a software ap- plication, ANUGA, to model the hydrodynamics of floods, storm surges and tsunamis. The free source software implements a finite volume central- upwind Godunov method to solve the non-linear depth-averagedshallowwater waveequations. Inlight of the renewed interest in tsunami forecasting and mitigation,

Local Polynomial Chaos Expansion for Linear Differential Equations with High Dimensional Random Inputs

SIAM Journal on Scientific Computing, 2015

Deep Learning of Parameterized Equations with Applications to Uncertainty Quantification

International Journal for Uncertainty Quantification

The Future of Sensitivity Analysis: An essential discipline for systems modeling and policy support

Environmental Modelling & Software

Journal of Machine Learning for Modeling and Computing

Gaussian process regression is a popular Bayesian framework for surrogate modeling of expensive d... more Gaussian process regression is a popular Bayesian framework for surrogate modeling of expensive data sources. As part of a broader effort in scientific machine learning, many recent works have incorporated physical constraints or other a priori information within Gaussian process regression to supplement limited data and regularize the behavior of the model. We provide an overview and survey of several classes of Gaussian process constraints, including positivity or bound constraints, monotonicity and convexity constraints, differential equation constraints provided by linear PDEs, and boundary condition constraints. We compare the strategies behind each approach as well as the differences in implementation, concluding with a discussion of the computational challenges introduced by constraints.

MFNets: MULTI-FIDELITY DATA-DRIVEN NETWORKS FOR BAYESIAN LEARNING AND PREDICTION

International Journal for Uncertainty Quantification

LDRD #218317: Learning Hidden Structure in Multi-Fidelity Information Sources for Efficient Uncertainty Quantification

Arctic Tipping Points Triggering Global Change LDRD Final Report

LDRD Project Summary: Incorporating physical constraints into Gaussian process surrogate models

Environmental Modelling & Software

Neural Networks as Surrogates of Nonlinear High-Dimensional Parameter-to-Prediction Maps

SIAM/ASA Journal on Uncertainty Quantification

The recovery of approximately sparse or compressible coefficients in a polynomial chaos expansion... more The recovery of approximately sparse or compressible coefficients in a polynomial chaos expansion is a common goal in many modern parametric uncertainty quantification (UQ) problems. However, relatively little effort in UQ has been directed toward theoretical and computational strategies for addressing the sparse corruptions problem, where a small number of measurements are highly corrupted. Such a situation has become pertinent today since modern computational frameworks are sufficiently complex with many interdependent components that may introduce hardware and software failures, some of which can be difficult to detect and result in a highly polluted simulation result. In this paper we present a novel compressive sampling-based theoretical analysis for a regularized 1 minimization algorithm that aims to recover sparse expansion coefficients in the presence of measurement corruptions. Our recovery results are uniform (the theoretical guarantees hold for all compressible signals and compressible corruptions vectors), and prescribe algorithmic regularization parameters in terms of a user-defined a priori estimate on the ratio of measurements that are believed to be corrupted. We also propose an iteratively reweighted optimization algorithm that automatically refines the value of the regularization parameter, and empirically produces superior results. Our numerical results test our framework on several medium-to-high dimensional examples of solutions to parameterized differential equations, and demonstrate the effectiveness of our approach. * B. Adcock and A. Bao acknowledge the support of the Alfred P. Sloan Foundation and the Natural Sciences and Engineering Research Council of Canada through grant 611675. † J.D.Jakeman's work was supported by DARPA EQUiPS.

Journal of Computational Physics

In this paper we present a basis selection method that can be used with 1-minimization to adaptiv... more In this paper we present a basis selection method that can be used with 1-minimization to adaptively determine the large coefficients of polynomial chaos expansions (PCE). The adaptive construction produces anisotropic basis sets that have more terms in important dimensions and limits the number of unimportant terms that increase mutual coherence and thus degrade the performance of 1-minimization. The important features and the accuracy of basis selection are demonstrated with a number of numerical examples. Specifically, we show that for a given computational budget, basis selection produces a more accurate PCE than would be obtained if the basis is fixed a priori. We also demonstrate that basis selection can be applied with non-uniform random variables and can leverage gradient information.

A Generalized Sampling and Preconditioning Scheme for Sparse Approximation of Polynomial Chaos Expansions

SIAM Journal on Scientific Computing

Local Polynomial Chaos Expansion for Linear Differential Equations with High Dimensional Random Inputs

Siam Journal on Scientific Computing, Jan 8, 2015

In this paper we present a locally and dimension-adaptive sparse grid method for interpolation an... more In this paper we present a locally and dimension-adaptive sparse grid method for interpolation and integration of high-dimensional functions with discontinuities. The proposed algorithm combines the strengths of the generalised sparse grid algorithm and hierarchical surplus-guided local adaptivity. A high-degree basis is used to obtain a high-order method which, given sufficient smoothness, performs significantly better than the piecewise-linear basis. The underlying generalised sparse grid algorithm greedily selects the dimensions and variable interactions that contribute most to the variability of a function. The hierarchical surplus of points within the sparse grid is used as an error criterion for local refinement with the aim of concentrating computational effort within rapidly varying or discontinuous regions. This approach limits the number of points that are invested in 'unimportant' dimensions and regions within the high-dimensional domain. We show the utility of the proposed method for non-smooth functions with hundreds of variables.

Enhancing $ ell_1$-minimization estimates of polynomial chaos expansions using basis selection

Dimension reduction for PDE using local Karhunen Loeve expansions

Inundation Modelling of the December 2004 Indian Ocean Tsunami

Geoscience Australia, in an open collaboration with the Mathematical Sciences Institute, The Aust... more Geoscience Australia, in an open collaboration with the Mathematical Sciences Institute, The Australian National University, is developing a software ap-plication, ANUGA, to model the hydrodynamics of floods, storm surges and tsunamis. The free source software implements a finite volume central-upwind Godunov method to solve the non-linear depth-averaged shallow water wave equations. In light of the renewed interest in tsunami forecasting and mitigation, this paper explores the use of ANUGA to model the inundation of the Indian Ocean tsunami of December 2004. The Method of Splitting Tsunamis (MOST) was used to simulate the initial tsunami source and the tsunami's propagation at depths greater than 100m. The resulting output was used to provide boundary conditions to the ANUGA model in the shallow water. Data with respect to 4-minute bathymetry, 2-minute bathymetry, 3-arc second bathymetry and elevation were used in the open ocean, shallow water and on land, respectively. A parti...

Geoscience Australia, in an open collaboration with the Mathematical Sciences Institute, The Aust... more Geoscience Australia, in an open collaboration with the Mathematical Sciences Institute, The Australian National University, is developing a software ap- plication, ANUGA, to model the hydrodynamics of floods, storm surges and tsunamis. The free source software implements a finite volume central- upwind Godunov method to solve the non-linear depth-averagedshallowwater waveequations. Inlight of the renewed interest in tsunami forecasting and mitigation,

Local Polynomial Chaos Expansion for Linear Differential Equations with High Dimensional Random Inputs

SIAM Journal on Scientific Computing, 2015

ArXiv, 2018

We describe and analyze a variance reduction approach for Monte Carlo (MC) sampling that accelera... more We describe and analyze a variance reduction approach for Monte Carlo (MC) sampling that accelerates the estimation of statistics of computationally expensive simulation models using an ensemble of models with lower cost. These lower cost models-which are typically lower fidelity with unknown statistics-are used to reduce the variance in statistical estimators relative to a MC estimator with equivalent cost. We derive the conditions under which our proposed approximate control variate framework recovers existing multi-model variance reduction schemes as special cases. We demonstrate that these existing strategies use recursive sampling strategies, and as a result, their maximum possible variance reduction is limited to that of a control variate algorithm that uses only a single low-fidelity model with known mean. This theoretical result holds regardless of the number of low-fidelity models and/or samples used to build the estimator. We then derive new sampling strategies within our framework that circumvent this limitation to make efficient use of all available information sources. In particular, we demonstrate that a significant gap can exist, of orders of magnitude in some cases, between the variance reduction achievable by using a single low-fidelity model and our non-recursive approach. We also present initial sample allocation approaches for exploiting this gap. They yield the greatest benefit when augmenting the high-fidelity model evaluations is impractical because, for instance, they arise from a legacy database. Several analytic examples and an example with a hyperbolic PDE describing elastic wave propagation in heterogeneous media are used to illustrate the main features of the methodology. 1. Introduction. Numerical evaluation of integrals is a foundational aspect of mathematics that has impact on diverse areas such as finance, uncertainty quantification, stochastic programming, and many others. Monte Carlo (MC) sampling is arguably the most robust means of estimating such integrals and can be easily applied to arbitrary integration domains and measures. The MC estimate of an integral is unbiased, and its rate of convergence is independent of the number of variables and the smoothness of the integrand. Nevertheless, obtaining a moderately accurate estimate of an integral with MC is computationally intractable for integrands that are expensive to evaluate, e.g., those arising from a high-fidelity simulation. This intractability arises because the variance of a MC estimator is proportional to the ratio of the variance of the integrand and inversely proportional to the number of samples used. As such, techniques that retain the benefits of MC estimation while reducing its variance are important for extending the applicability of these sampling-based approaches. Control variates (CV) are a class of such techniques that have a long history of reducing MC variance by introducing additional estimators that are correlated with the MC estimator [15, 17, 16, 13]. The use of CV methods has recently seen a resurgence for uncertainty quantification (UQ) problems where the integrands are computationally expensive to evaluate. In these cases, CV approaches can leverage multiple simulation models to accelerate the convergence of statistics for both forward [20, 11, 28, 4, 8] and inverse [1] UQ. These additional simulation models arise from either different sets of equations (i.e., the multifidelity case of differing model forms) and/or from varying temporal and spatial discretizations (i.e., the multilevel case of differing numerical resolutions for the same set of equations). The model ensemble could include reduced-order models [29], dimension-reduction or surrogate models [25] (e.g., active subspace approximations), and even data from physical experiments [14]. Multiple conceptual dimensions can exist within a modeling hierarchy , leading to multi-index constructions [12] in the case of independent resolution controls. Finally, both multi-physics and multi-scale simulations can contribute additional combinatorial richness to the associated modeling ensemble. Traditional CV methods [15] require explicit knowledge of the statistics (for instance the expected value) of their approximate information sources. However, these estimates are frequently unavailable a priori in the UQ simulation-based context. Consequently, CV methods must be modified to balance the computational cost of evaluating lower fidelity models and the reduction in error that they each provide. There exist several strategies that explicitely pursue the goal of estimating the unknown expected values [31, 5, 26, 21] within a control variate framework; however the analysis of these approaches is limited to the case of a single control variate only. As a result, they do not consider how ensembles of low-fidelity information sources could be