Introducing Prior Information into the Forward Search for Regression (original) (raw)

Robust Bayesian regression with the forward search: theory and data analysis

TEST

The frequentist forward search yields a flexible and informative form of robust regression. The device of fictitious observations provides a natural way to include prior information in the search. However, this extension is not straightforward, requiring weighted regression. Bayesian versions of forward plots are used to exhibit the presence of multiple outliers in a data set from banking with 1903 observations and nine explanatory variables which shows, in this case, the clear advantages from including prior information in the forward search. Use of observation weights from frequentist robust regression is shown to provide a simple general method for robust Bayesian regression.

Building Regression Models with the Forward Search

Journal of Computing and Information Technology, 2007

We give an example of the use of the forward search in building a regression model. The standard backwards elimination of variables is supplemented by forward plots of added variable t statistics that exhibit the effect of each observation on the process of model building. Attention is also paid to the effect of individual observations on selection of a transformation. Variable selection using AIC is mentioned, as is the analysis of multivariate data.

fsdaSAS: A Package for Robust Regression for Very Large Datasets Including the Batch Forward Search

Stats

The forward search (FS) is a general method of robust data fitting that moves smoothly from very robust to maximum likelihood estimation. The regression procedures are included in the MATLAB toolbox FSDA. The work on a SAS version of the FS originates from the need for the analysis of large datasets expressed by law enforcement services operating in the European Union that use our SAS software for detecting data anomalies that may point to fraudulent customs returns. Specific to our SAS implementation, the fsdaSAS package, we describe the approximation used to provide fast analyses of large datasets using an FS which progresses through the inclusion of batches of observations, rather than progressing one observation at a time. We do, however, test for outliers one observation at a time. We demonstrate that our SAS implementation becomes appreciably faster than the MATLAB version as the sample size increases and is also able to analyse larger datasets. The series of fits provided by ...

A Robust and Diagnostic Information Criterion for Selecting Regression Models

JOURNAL OF THE JAPAN STATISTICAL SOCIETY, 2008

We combine the selection of a statistical model with the robust parameter estimation and diagnostic properties of the Forward Search. As a result we obtain procedures that select the best model in the presence of outliers. We derive distributional properties of our method and illustrate it on data on ozone concentration. The effect of outliers on the choice of a model is revealed. Although our example is for regression, the connection with AIC is stressed.

Models as Approximations --- Part II: A General Theory of Model-Robust Regression

2016

We discuss a model-robust theory for general types of regression in the simplest case of iid observations. The theory replaces the parameters of parametric models with statistical functionals, to be called “regression functionals” and defined on large non-parametric classes of joint x-y distributions without assuming a working model. Examples of regression functionals are the slopes of OLS linear equations at largely arbitrary x-y distributions (see Part I). More generally, regression functionals can be defined by minimizing objective functions or solving estimating equations at joint x-y distributions. The role of parametric models is reduced to heuristics for generating objective functions and estimating equations without assuming them as correct. In this framework it is possible to achieve the following: (1) explicate the meaning of mis/well-specification for regression functionals, (2) decompose sampling variability into two components, one due to the conditional response distri...

Applications of Robust Regression in Business, Economics and Social Sciences

Abstract: Robust regression techniques are rarely used in business, economics or in social sciences. It is a reliable alternative, where ordinary least squares (OLS) totally fails due to unusual observations and the violations of normality assumptions of error distributions. We demonstrate the importance of robust regression techniques by studying and comparing with OLS. Three examples are taken from the literature in areas of business, economics and social sciences.

Partially adaptive robust estimation of regression models and applications

European Journal of Operational Research, 2006

This paper provides an accessible exposition of recently developed partially adaptive estimation methods and their application. These methods are robust to thick-tailed or asymmetric error distributions and should be of interest to researchers and practitioners in data mining, agent learning, and mathematical modeling in a wide range of disciplines. In particular, partially adaptive estimation methods can serve as robust alternatives to ordinary regression analysis, as well as machine learning methods developed by the artificial intelligence and computing communities. Results from analysis of three problem domains demonstrate application of the theory.

Model robust regression: combining parametric, nonparametric, and semiparametric methods

Journal of Nonparametric …, 2001

In obtaining a regression tit to a set of data, ordinary least squares regression depends directly on the parametric model formulated by the researcher. Ifthis model is incorrect, a least squares analysis may be misleading. Altematively, nonparametric regression (kemel or local polynomial regression, for example) has no dependence on an underlying parametric model, but instead depends entirely on the distances between regressor coordinates and the prediction point of interest. This procedure avoids the necessity of a reliable model, but in using no information from the researcher, may Ht to irregular

The forward search: Theory and data analysis

2010

The paper by Atkinson, Riani and Ceroli, henceforth ARC, is concerned with detection of outliers and unsuspected structures which is rather important in practice. This is done through a Forward Search Algorithm. The statistical analysis of such algorithms poses many challenging problems, and we would like to contribute to the theory of the algorithm in this discussion.