A Parametric Framework for the Comparison of Methods of Very Robust Regression (original) (raw)
Related papers
Leverages, Outliers and the Performance of Robust Regression Estimators
British Journal of Mathematics & Computer Science, 2016
In this study, we assess the performance of some robust regression methods. These are the least-trimmed squares estimator (LTSE), Huber maximum likelihood estimator (HME), S-Estimator (SE) and modified maximum likelihood estimator (MME) which are compared with the ordinary least squares Estimator (OLSE) at different levels of leverages in the predictor variables. Anthropometric data from Komfo Anokye Teaching Hospital (KATH) was used and the comparison is done using root mean square error (RMSE), relative efficiencies (RE), coefficients of determination (R-squared) and power of the test. The results show that robust methods are as efficient as the OLSE if the assumptions of OLSE are met. OLSE is affected by low and high percentage of leverages, HME broke-down with leverages in data. MME and SE are robust to all percentage of aberrations, while LTSE is slightly affected by high percentage leverages perturbation. Thus, MME and SE are the most robust methods, while OLSE and HME are the least robust and the performance of the LTSE is affected by higher percentages of leverage in this study.
A New Robust Method for Estimating Linear Regression Model in the Presence of Outliers
Pacific Journal of Science and technology , 2018
Ordinary Least-Squares (OLS) estimators for a linear model are very sensitive to unusual values in the design space or outliers among response values. Even single atypical value may have a large effect on the parameter estimates. In this paper, we propose a new class of robust regression method for the classical linear regression model. The proposed method was developed using regularization methods that allow one to handle a variety of inferential problems where there are more covariates than cases. Specifically, each outlying point in the data is estimated using case-specific parameter. Penalized estimators are often suggested when the number of parameters in the model is more than the number of observed data points. In light of this, we propose the use of Ridge regression method for estimating the case-specific parameters. The proposed robust regression method was validated using Monte-Carlo datasets of varying proportion of outliers. Also, performance comparison was done for the proposed method with some existing robust methods. Assessment criteria results using breakdown point and efficiency revealed the supremacy of the proposed method over the existing methods considered.
Calibrated Very Robust Regression
2011
The behaviour of algorithms for very robust regression depends on the distance between the regression data and the outliers. We introduce a parameter λ that defines a parametric path in the space of models and enables us to study, in a systematic way, the properties of estimators as the groups of data move from being far apart to close together. We examine, as a function of λ, the variance and squared bias of five estimators and we also consider their power when used in the detection of outliers. This systematic approach provides tools for gaining knowledge and better understanding of the properties of robust estimators.
REVIEW OF SOME ROBUST ESTIMATORS IN MULTIPLE LINEAR REGRESSIONS IN THE PRESENCE OF OUTLIER(s
African Journal of Mathematics and Statistics Studies , 2023
Linear regression has been one of the most important statistical data analysis tools. Multiple regression is a type of regression where the dependent variable shows a linear relationship with two or more independent variables. OLS estimate is extremely sensitive to unusual observations (outliers), with low breakdown point and low efficiency. This paper reviews and compares some of the existing robust methods (Least Absolute Deviation, Huber M-Estimator, Bisquare M-Estimator, MM Estimator, Least Median Square, Least Trimmed Square, S-Estimator); a simulation method is used to compare the selected existing methods. It was concluded based on the results that for y direction outlier, the best estimator in terms of high efficiency and breakdown point of at most 0.3 is MM; for x direction outlier, the best estimator in term breakdown point of at most 0.4 is S; for x, y direction outlier, the best estimator in terms of high efficiency and breakdown point of at most 0.2 is MM.
Comparison of Robust Regression Methods in Linear Regression
Int. J. Contemp. Math. Sciences, 2011
In classical multiple regression, the ordinary least squares estimation is the best method if assumptions are met to obtain regression weights when analyzing data. However, if the data does not satisfy some of these assumptions, then sample estimates and results can be misleading. Especially, outliers violate the assumption of normally distributed residuals in the least squares regression. The danger of outlying observations, both in the direction of the dependent and explanatory variables, to the least squares regression is that they can have a strong adverse effect on the estimate and they may remain unnoticed. Therefore, statistical techniques that are able to cope with or to detect outlying observations have been developed. Robust regression is an important method for analyzing data that are contaminated with outliers. It can be used to detect outliers and to provide resistant results in the presence of outliers. The purpose of this study is to define behavior of outliers in linear regression and to compare some of robust regression methods via simulation study. The simulation study is used in determining which methods best in all of the linear regression scenarios.
A comparative study of some robust methods for coefficient-estimation in linear regression
Computational Statistics & Data Analysis, 1997
Robust regression estimators are known to perform well in the presence of outliers. Although theoretical properties of these estimators have been derived, there is always a need for empirical results to assist their implementation in practical situations. A simulation study of four robust alternatives to the least-squares method is presented within a set of error-distributions which includes many outlier-generating models. The robustness and efficiency features of the methods are exhibited, some finite-sample results are discussed in combination with asymptotic properties, and the relative merits of the estimators are viewed in connection with the tail-length of the underlying errordistribution.
This study compared the performance of some robust regression methods and the Ordinary Least Squares Estimator (OLSE). The estimators were compared using varied levels of leverages and vertical outliers in the predictors and the dependent variables. An anthropometric dataset on total body fat with height, Body Mass Index (BMI), Triceps Skin-fold(TS), and arm fat as percent composition of the body (parmfat), as the predictors. The effects of outliers and leverages on the estimators, were investigated at (5% leverages and 10% vertical outliers, 5% leverages with 15% vertical outliers). The criteria for the comparison: coefficients, Root Mean Square Error (RMSE), Relative Efficiencies (RE), coefficients of determination (R-squared) and power of the test. The findings from this study revealed that, OLSE was affected by both outliers and leverages whilst Huber Maximum likelihood Estimator (HME) was affected by leverages. The Least Trimmed Squares Estimator (LTSE) was slightly affected by high perturbations of outliers and leverages.
Regression Estimation in the Presence of Outliers: A Comparative Study
2016
In linear models, the ordinary least squares (OLS) estimators of parameters have always turned out to be the best linear unbiased estimators. However, if the data contain outliers, this may affect the least-squares estimates. So, an alternative approach; the so-called robust regression methods, is needed to obtain a better fit of the model or more precise estimates of parameters. In this article, various robust regression methods have been reviewed. The focus is on the presence of outliers in the y-direction (response direction). Comparison of the properties of these methods is done through a simulation study. The comparison's criteria were the efficiency and breakdown point. Also, the methods are applied to a real data set.
Benchmark testing of algorithms for very robust regression: FS, LMS and LTS
Computational Statistics & Data Analysis, 2012
The methods of very robust regression resist up to 50% of outliers. The algorithms for very robust regression rely on selecting numerous subsamples of the data. New algorithms for LMS and LTS estimators that have increased computational efficiency due to improved combinatorial sampling are proposed. These and other publicly available algorithms are compared for outlier detection. Timings and estimator quality are also considered. An algorithm using the forward search (FS) has the best properties for both size and power of the outlier tests.