Bayesian model selection applied to artificial neural networks used for water resources modeling (original) (raw)
Related papers
Bayesian training of artificial neural networks used for water resources modeling
1] Artificial neural networks have proven to be superior prediction models in many hydrology-related areas; however, failure of ANN practitioners to account for uncertainty in the predictions has limited the wider use of ANNs as forecasting models. Conventional methods for quantifying parameter uncertainty are difficult to apply to ANN weights because of the complexity of these models, and complicated methods developed for this purpose have been not been adopted by water resources practitioners because of the difficulty in implementing them. This paper presents a relatively straightforward Bayesian training method that enables weight uncertainty to be accounted for in ANN predictions. The method is applied to a salinity forecasting case study, and the resulting ANN is shown to significantly outperform an ANN developed using standard approaches in a real-time forecasting scenario. Moreover, the Bayesian approach produces prediction limits that indicate the level of uncertainty in the predictions, which is extremely important if forecasts are to be used with confidence in water resources applications. Citation: Kingston, G. B., M. F. Lambert, and H. R. Maier (2005), Bayesian training of artificial neural networks used for water resources modeling, Water Resour. Res., 41, W12409,
Hydrological model selection: A Bayesian alternative
Water Resources Research, 2005
1] The evaluation and comparison of hydrological models has long been a challenge to the practicing hydrological community. No single model can be identified as ideal over the range of possible hydrological situations. With the variety of models available, hydrologic modelers are faced with the problem of determining which model is best applied to a catchment for a particular modeling exercise. The model selection problem is well documented in hydrologic studies, but a broadly applicable as well as theoretically and practically sound method for comparing model performance does not exist in the literature. Bayesian statistical inference, with computations carried out via Markov chain Monte Carlo (MCMC) methods, offers an attractive alternative to conventional model selection methods allowing for the combination of any preexisting knowledge about individual models and their respective parameters with the available catchment data to assess both parameter and model uncertainty. The aim of this study is to present a method by which hydrological models may be compared in a Bayesian framework. The study builds on previous work in which a Bayesian approach implemented using MCMC algorithms was presented as a simple and efficient basis for assessing parameter uncertainty in hydrological models. In this study, a model selection framework is developed in which an adaptive Metropolis algorithm is used to calculate the model's posterior odds. The model used to illustrate our approach is a version of the Australian Water Balance Model (Boughton, 1993) reformulated such that it can have a flexible number of soil moisture storages. To assess the model selection method in a controlled setting, artificial runoff data were created corresponding to a known model configuration. These data were used to evaluate the accuracy of the model selection method and its sensitivity to the size of the sample being used. An application of the Bayesian model identification methodology to 11 years of daily streamflow data from the Murrumbidgee River at Mittagang Crossing in southeastern Australia concludes our study.
Water Resources Research, 2014
Bayesian model selection or averaging objectively ranks a number of plausible, competing conceptual models based on Bayes' theorem. It implicitly performs an optimal trade-off between performance in fitting available data and minimum model complexity. The procedure requires determining Bayesian model evidence (BME), which is the likelihood of the observed data integrated over each model's parameter space. The computation of this integral is highly challenging because it is as high-dimensional as the number of model parameters. Three classes of techniques to compute BME are available, each with its own challenges and limitations: (1) Exact and fast analytical solutions are limited by strong assumptions. Numerical evaluation quickly becomes unfeasible for expensive models.
Bridging groundwater models and decision support with a Bayesian network
Water Resources Research, 2013
Resource managers need to make decisions to plan for future environmental conditions, particularly sea level rise, in the face of substantial uncertainty. Many interacting processes factor in to the decisions they face. Advances in process models and the quantification of uncertainty have made models a valuable tool for this purpose. Long-simulation runtimes and, often, numerical instability make linking process models impractical in many cases. A method for emulating the important connections between model input and forecasts, while propagating uncertainty, has the potential to provide a bridge between complicated numerical process models and the efficiency and stability needed for decision making. We explore this using a Bayesian network (BN) to emulate a groundwater flow model. We expand on previous approaches to validating a BN by calculating forecasting skill using cross validation of a groundwater model of Assateague Island in Virginia and Maryland, USA. This BN emulation was shown to capture the important groundwater-flow characteristics and uncertainty of the groundwater system because of its connection to island morphology and sea level. Forecast power metrics associated with the validation of multiple alternative BN designs guided the selection of an optimal level of BN complexity. Assateague island is an ideal test case for exploring a forecasting tool based on current conditions because the unique hydrogeomorphological variability of the island includes a range of settings indicative of past, current, and future conditions. The resulting BN is a valuable tool for exploring the response of groundwater conditions to sea level rise in decision support.
A sequential Bayesian approach for hydrologic model selection and prediction
Water Resources Research, 2009
When a single model is used for hydrologic prediction, it must be capable of estimating system behavior accurately at all times. Multiple-model approaches integrate several model behaviors and, when effective, they can provide better estimates than that of any single model alone. This paper discusses a sequential model fusion strategy that uses the Bayes rule. This approach calculates each model's transient posterior distribution at each time when a new observation is available and merges all model estimates on the basis of each model's posterior probability. This paper demonstrates the feasibility of this approach through case studies that fuse three hydrologic models, auto regressive with exogenous inputs, Sacramento soil moisture accounting, and artificial neural network models, to predict daily watershed streamflow.
Impacts of prior parameter distributions on Bayesian evaluation of groundwater model complexity
Water Science and Engineering, 2018
This study used the marginal likelihood and Bayesian posterior model probability for evaluation of model complexity in order to avoid using over-complex models for numerical simulations. It focused on investigation of the impacts of prior parameter distributions (involved in calculating the marginal likelihood) on the evaluation of model complexity. We argue that prior parameter distributions should define the parameter space in which numerical simulations are made. New perspectives on the prior parameter distribution and posterior model probability were demonstrated in an example of groundwater solute transport modeling with four models, each simulating four column experiments. The models had different levels of complexity in terms of their model structures and numbers of calibrated parameters. The posterior model probability was evaluated for four cases with different prior parameter distributions. While the distributions substantially impacted model ranking, the model ranking in each case was reasonable for the specific circumstances in which numerical simulations were made. For evaluation of model complexity, it is thus necessary to determine the parameter spaces for modeling, which can be done by conducting numerical simulation and using engineering judgment based on understanding of the system being studied.
Bayesian neural network for rainfall-runoff modeling
Water Resources Research, 2006
In this paper, a Bayesian learning approach is introduced to train a multilayer feedforward network for daily river flow and reservoir inflow simulation in a cold region river basin in Canada. In Bayesian approach, uncertainty about the relationship between inputs and outputs is initially taken care of by an assumed prior distribution of parameters (weights and biases). This prior distribution is updated to posterior distribution using a likelihood function following Bayes' theorem while data are observed. This posterior distribution is called the objective function of a network in the Bayesian learning approach. The objective function is maximized using a suitable optimization technique. Once the network is trained, the predictive distribution of the network outputs is obtained by integrating over the posterior distribution of weights. In this study, Gaussian prior distribution and a Gaussian noise model are used in defining posterior distribution. The network has been optimized using a scaled conjugate gradient technique. Posterior distribution of weights is approximated to Gaussian during prediction. Prediction performance of the Bayesian neural network (BNN) is compared with the results obtained from a standard artificial neural network (ANN) model and a widely used conceptual rainfall-runoff model, namely, HBV-96. The BNN model outperformed the conceptual model and slightly outperformed the standard ANN model in simulating mean, peak, and low river flows and reservoir inflows. The significant contribution of the Bayesian method over the conventional ANN approach, among others, is the uncertainty estimation of the outputs in the form of confidence intervals which are particularly needed in practical water resources applications. Prediction confidence limits (or intervals) indicate the extent to which one can rely on predictions for decision making. It is shown that the BNN can provide reliable streamflow and reservoir inflow forecasts without a loss in model prediction accuracy as compared to standard ANN and conceptual model HBV. Another significant advantage of BNN approach is that the overfitting and underfitting problems are automatically taken care of by the Bayesian learning algorithm, which conversely remain serious problems with conventional ANN learning algorithm.
Predictive modeling of hydrological time series is essential for groundwater resource development and management. Here, we examined the comparative merits and demerits of three modern soft computing techniques, namely, artificial neural networks (ANN) optimized by scaled conjugate gradient (SCG) (ANN.SCG), Bayesian neural networks (BNN) optimized by SCG (BNN.SCG) with evidence approximation and adaptive neuro-fuzzy inference system (ANFIS) in the predictive modeling of groundwater level fluctuations. As a first step of our analysis, a sensitivity analysis was carried out using automatic relevance determination scheme to examine the relative influence of each of the hydro-meteorological attributes on groundwater level fluctuations. Secondly, the result of stability analysis was studied by perturbing the underlying data sets with different levels of correlated red noise. Finally, guided by the ensuing theoretical experiments, the above techniques were applied to model the groundwater level fluctuation time series of six wells from a hard rock area of Dindigul in Southern India. We used four standard quantitative statistical measures to compare the robustness of the different models. These measures are (1) root mean square error, (2) reduction of error, (3) index of agreement (IA), and (4) Pearson’s correlation coefficient (R). Based on the above analyses, it is found that the ANFIS model performed better in modeling noise-free data than the BNN.SCG and ANN.SCG models. However, modeling of hydrological time series correlated with significant amount of red noise, the BNN.SCG models performed better than both the ANFIS and ANN.SCG models. Hence, appropriate care should be taken for selecting suitable methodology for modeling the complex and noisy hydrological time series. These results may be used to constrain the model of groundwater level fluctuations, which would in turn, facilitate the development and implementation of more effective sustainable groundwater management and planning strategies in semi-arid hard rock area of Dindigul, Southern India and alike.
Environmental Modelling & Software, 2014
The application of Artificial Neural Networks (ANNs) in the field of environmental and water resources modelling has become increasingly popular since early 1990s. Despite the recognition of the need for a consistent approach to the development of ANN models and the importance of providing adequate details of the model development process, there is no systematic protocol for the development and documentation of ANN models. In order to address this shortcoming, such a protocol is introduced in this paper. In addition, the protocol is used to critically review the quality of the ANN model development and reporting processes employed in 81 journal papers since 2000 in which ANNs have been used for drinking water quality modelling. The results show that model architecture selection is the best implemented step, while greater focus should be given to input selection considering input independence and model validation considering replicative and structural validity.
A Comparison of Scientific and Engineering Criteria for Model Selection
2000
Given a set of possible models for variables X and a set of possible parameters for each model, the Bayesian "estimate" of the probability distribution for X given observed data is obtained by averaging over the possible models and their parameters. An often-used approximation for this estimate is obtained by selecting a single model and averaging over its parameters. The approximation is useful because it is computationally efficient, and because it provides a model that facilitates understanding of the domain. A common criterion for model selection is the posterior probability of the model. Another criterion for model selection, proposed by San Martini and Spezzafari (1984), is the predictive performance of a model for the next observation to be seen. From the standpoint of domain understanding, both criteria are useful, because one identifies the model that is most likely, whereas the other identifies the model that is the best predictor of the next observation. To highlight the difference, we refer to the posterior-probability and alternative criteria as the scientific criterion (SC) and engineering criterion (EC), respectively. When we are interested in predicting the next observation, the model-averaged estimate is at least as good as that produced by EC, which itself is at least as good as the estimate produced by SC. We show experimentally that, for Bayesian-network models containing discrete variables only, the predictive performance of the model average can be significantly better than those of single models selected by either criterion, and that differences between models selected by the two criterion can be substantial.