Bayesian parameter estimation via filtering and functional approximations (original) (raw)
Related papers
Nonlinear Bayesian Estimation: From Kalman Filtering to a Broader Horizon
This article presents an up-to-date tutorial review of nonlinear Bayesian estimation. State estimation for nonlinear systems has been a challenge encountered in a wide range of engineering fields, attracting decades of research effort. To date, one of the most promising and popular approaches is to view and address the problem from a Bayesian probabilistic perspective, which enables estimation of the unknown state variables by tracking their probabilistic distribution or statistics (e.g., mean and covariance) conditioned on a system's measurement data. This article offers a systematic introduction to the Bayesian state estimation framework and reviews various Kalman filtering (KF) techniques, progressively from the standard KF for linear systems to extended KF, unscented KF and ensemble KF for nonlinear systems. It also overviews other prominent or emerging Bayesian estimation methods including Gaussian filtering, Gaussian-sum filtering, particle filtering and moving horizon estimation and extends the discussion of state estimation to more complicated problems such as simultaneous state and parameter/input estimation.
Kalman filtering and sequential Bayesian analysis
WIREs Computational Statistics, 2018
In this paper we present an overview of the state of the art in Kalman filtering and dynamic Bayesian linear and nonlinear models. We present some of the basic results including the derivation of Kalman filtering equations as well as recent advances in Kalman filter models and their extensions including non-Gaussian state-space models. In so doing, we take a Bayesian perspective and discuss parameter learning in state-space models which typically involves Markov chain Monte Carlo and sequential Monte Carlo methods. We present particle filtering and Bayesian particle learning techniques for state space models and discuss recent advances.
IEEE Transactions on Signal Processing, 2021
We present a general nonlinear Bayesian filter for high-dimensional state estimation using the theory of reproducing kernel Hilbert space (RKHS). Applying kernel method and the representer theorem to perform linear quadratic estimation in a functional space, we derive a Bayesian recursive state estimator for a general nonlinear dynamical system in the original input space. Unlike existing nonlinear extensions of Kalman filter where the system dynamics are assumed known, the state-space representation for the Functional Bayesian Filter (FBF) is completely learned from measurement data in the form of an infinite impulse response (IIR) filter or recurrent network in the RKHS, with universal approximation property. Using positive definite kernel function satisfying Mercer's conditions to compute and evolve information quantities, the FBF exploits both the statistical and time-domain information about the signal, extracts higher-order moments, and preserves the properties of covariances without the ill effects due to conventional arithmetic operations. This novel kernel adaptive filtering algorithm is applied to recurrent network training, chaotic time-series estimation and cooperative filtering using Gaussian and non-Gaussian noises, and inverse kinematics modeling. Simulation results show FBF outperforms existing Kalman-based algorithms.
Particle Kalman Filtering: A Nonlinear Bayesian Framework for Ensemble Kalman Filters
2011
Abstract: This paper investigates an approximation scheme of the optimal nonlinear Bayesian filter based on the Gaussian mixture representation of the state probability distribution function. The resulting filter is similar to the particle filter, but is different from it in that, the standard weight-type correction in the particle filter is complemented by the Kalman-type correction with the associated covariance matrices in the Gaussian mixture.
Updating Noise Parameters of Kalman Filter using Bayesian Approach
In the present study, the Bayesian probabilistic approach is proposed to estimate the process noise and measurement noise parameters in the Kalman filter for the case when the input is a zero-mean Gaussian white noise process and limited output measurements are available. The methodology presented in this study is based on the idea that the process noise and measurement noise parameters directly affect the state estimate and the covariance matrix of the Kalman filter. Therefore, the noise parameters can be estimated by expressing the likelihood factor of the measurements as a function of the state estimate and the covariance matrix for a given set of process noise and measurement noise parameters. By evaluating the likelihood factor for all the noise parameters within the search domain, the optimal estimates of the noise parameters are chosen by the maximum likelihood criterion. Through the demonstration of an example, the optimal estimates of the noise parameters are close to the a...
A Bayesian Adaptive Ensemble Kalman Filter for Sequential State and Parameter Estimation
Monthly Weather Review, 2018
This paper proposes new methodology for sequential state and parameter estimation within the ensemble Kalman filter. The method is fully Bayesian and propagates the joint posterior distribution of states and parameters over time. To implement the method, the authors consider three representations of the marginal posterior distribution of the parameters: a grid-based approach, a Gaussian approximation, and a sequential importance sampling (SIR) approach with kernel resampling. In contrast to existing online parameter estimation algorithms, the new method explicitly accounts for parameter uncertainty and provides a formal way to combine information about the parameters from data at different time periods. The method is illustrated and compared to existing approaches using simulated and real data.
The parameters of temporal models, such as dynamic Bayesian networks, may be modelled in a Bayesian context as static or atemporal variables that influence transition probabilities at every time step. Particle filters fail for models that include such variables, while methods that use Gibbs sampling of parameter variables may incur a per-sample cost that grows linearly with the length of the observation sequence. Storvik (2002) devised a method for incremental computation of exact sufficient statistics that, for some cases, reduces the per-sample cost to a constant. In this paper, we demonstrate a connection between Storvik's filter and a Kalman filter in parameter space and establish more general conditions under which Storvik's filter works. Drawing on an analogy to the extended Kalman filter, we develop and analyze, both theoretically and experimentally, a Taylor approximation to the parameter posterior that allows Storvik's method to be applied to a broader class of models. Our experiments on both synthetic examples and real applications show improvement over existing methods.
Physica D: Nonlinear Phenomena, 2012
We present a fully deterministic method to compute sequential updates for stochastic state estimates of dynamic models from noisy measurements. It does not need any assumptions about the type of distribution for either data or measurement -in particular it does not have to assume any of them as Gaussian. It is based on a polynomial chaos expansion (PCE) of the stochastic variables of the model. We use a minimum variance estimator that combines an a priori state estimate and noisy measurements in a Bayesian way. For computational purposes, the update equation is projected onto a finite-dimensional PCE-subspace. The resulting Kalman-type update formula for the PCE coefficients can be efficiently computed solely within the PCE. As it does not rely on sampling, the method is deterministic, robust, and fast.
Particle filtering for Bayesian parameter estimation in a high dimensional state space model
2015 23rd European Signal Processing Conference (EUSIPCO), 2015
Researchers in some of the most active fields of science, including, e.g., geophysics or systems biology, have to deal with very-large-scale stochastic dynamic models of real world phenomena for which conventional prediction and estimation methods are not well suited. In this paper, we investigate the application of a novel nested particle filtering scheme for joint Bayesian parameter estimation and tracking of the dynamic variables in a high dimensional state space model-namely a stochastic version of the two-scale Lorenz 96 chaotic system, commonly used as a benchmark model in meteorology and climate science. We provide theoretical guarantees on the algorithm performance, including uniform convergence rates for the approximation of posterior probability density functions of the fixed model parameters.
On sequential Monte Carlo sampling methods for Bayesian filtering
Statistics and Computing, 2000
In this article, we present an overview of methods for sequential simulation from posterior distributions. These methods are of particular interest in Bayesian filtering for discrete time dynamic models that are typically nonlinear and non-Gaussian. A general importance sampling framework is developed that unifies many of the methods which have been proposed over the last few decades in several different