The Comparing of S-estimator and M-estimators in Linear Regression (original) (raw)
Related papers
2020
Robust regression is a regression method used when the remainder's distribution is not reasonable, or there is an outreach to observational data that affects the model. One method for estimating regression parameters is the Least Squares Method (MKT). The method is easily affected by the presence of outliers. Therefore we need an alternative method that is robust to the presence of outliers, namely robust regression. Methods for estimating robust regression parameters include Least Trimmed Square (LTS) and Least Median Square (LMS). These methods are estimators with high breakdown points for outlier observational data and have more efficient algorithms than other estimation methods. This study aims to compare the regression models formed from the LTS and LMS methods, determine the efficiency of the model formed, and determine the factors that influence the production of community oil palm in Langkat District in 2018. The results showed that in testing, the estimated model of the...
Regression Estimation in the Presence of Outliers: A Comparative Study
2016
In linear models, the ordinary least squares (OLS) estimators of parameters have always turned out to be the best linear unbiased estimators. However, if the data contain outliers, this may affect the least-squares estimates. So, an alternative approach; the so-called robust regression methods, is needed to obtain a better fit of the model or more precise estimates of parameters. In this article, various robust regression methods have been reviewed. The focus is on the presence of outliers in the y-direction (response direction). Comparison of the properties of these methods is done through a simulation study. The comparison's criteria were the efficiency and breakdown point. Also, the methods are applied to a real data set.
REVIEW OF SOME ROBUST ESTIMATORS IN MULTIPLE LINEAR REGRESSIONS IN THE PRESENCE OF OUTLIER(s
African Journal of Mathematics and Statistics Studies , 2023
Linear regression has been one of the most important statistical data analysis tools. Multiple regression is a type of regression where the dependent variable shows a linear relationship with two or more independent variables. OLS estimate is extremely sensitive to unusual observations (outliers), with low breakdown point and low efficiency. This paper reviews and compares some of the existing robust methods (Least Absolute Deviation, Huber M-Estimator, Bisquare M-Estimator, MM Estimator, Least Median Square, Least Trimmed Square, S-Estimator); a simulation method is used to compare the selected existing methods. It was concluded based on the results that for y direction outlier, the best estimator in terms of high efficiency and breakdown point of at most 0.3 is MM; for x direction outlier, the best estimator in term breakdown point of at most 0.4 is S; for x, y direction outlier, the best estimator in terms of high efficiency and breakdown point of at most 0.2 is MM.
Combining Some Biased Estimation Methods with Least Trimmed Squares Regression and its Application
Revista Colombiana de EstadÃstica, 2015
In the case of multicollinearity and outliers in regression analysis, the researchers are encouraged to deal with two problems simultaneously. Biased methods based on robust estimators are useful for estimating the regression coefficients for such cases. In this study we examine some robust biased estimators on the datasets with outliers in x direction and outliers in both x and y direction from literature by means of the R package ltsbase. Instead of a complete data analysis, robust biased estimators are evaluated using capabilities and features of this package.
British Journal of Mathematics & Computer Science, 2016
The Ordinary Least Squares Estimator (OLSE) is the best method for linear regression if the classical assumptions are satisfied for estimating weights. When these assumptions are violated, the robust methods give more reliable estimates while the OLSE is strongly affected adversely. In order to assess the sensitivity of some estimators using more than five criteria, a secondary dataset on Anthropometric measurements from Komfo Anokye Teaching Hospital, Kumasi-Ghana, is used. In this study, we compare the performance of the Huber Maximum Likelihood Estimator (HMLE), Least Trimmed Squares Estimator (LTSE), S Estimator (SE) and Modified Maximum Likelihood Estimator (MMLE) relative to the OLSE when the dataset has normal errors; 10, 20 and 30 percent outliers; 20% error contamination and lognormal contamination in the response
Some Methods of Detection of Outliers in Linear Regression Model
An outlier is an observation that deviates markedly from the majority of the data. To know which observation has greater influence on parameter estimate, detection of outlier is very important. There are several methods for detection of outliers available in the literature. A good number of test-statistics for detecting outliers have been developed. In contrast to detection, outliers are also tackled through robust regression techniques like, M-estimator, Least Median of Square (LMS). Robust regression provides parameter estimates that are insensitive to the presence of outliers and also helps to detect outlying observations. Recently, Forward Search (FS) method has been developed, in which a small number of observations robustly chosen are used to fit a model through Least Square (LS) method. Then more number of observations are included in the subsequent steps. This forward search procedure provides a wealth of information not only for outlier detection but, much more importantly, on the effect of each observation on aspects of inferences about the model. It also reveals the masking problem, if present, very nicely in the data.
Asian Research Journal of Mathematics, 2022
Supplementary variables associated with the study variables have been identified to be helpful in improving the efficiency of ratio, product and regression estimators both at planning and estimation stages. The existing regression-based estimators are functions of regression slopes and known auxiliary variables which are sensitive to outliers. Zaman & Bulut [1] and Zaman [2] addressed the issue of regression slopes in the aforementioned estimators using robust regression slopes like Huber-M, Hampel-M, Least Trimmed Squares (LTS) and Least Absolute Deviation (LAD). However, their estimators still utilized known auxiliary functions which are also sensitive to outliers or extreme values. Similarly, Yadav and Zaman [3] suggested non-conventional robust parameters of auxiliary variable which are robust against outliers. However, the problems of effects of outliers on regression slopes were not considered. In this study, the estimators of Zaman & Bulut [1] and Zaman [2] estimators were mo...
A New Robust Method for Estimating Linear Regression Model in the Presence of Outliers
Pacific Journal of Science and technology , 2018
Ordinary Least-Squares (OLS) estimators for a linear model are very sensitive to unusual values in the design space or outliers among response values. Even single atypical value may have a large effect on the parameter estimates. In this paper, we propose a new class of robust regression method for the classical linear regression model. The proposed method was developed using regularization methods that allow one to handle a variety of inferential problems where there are more covariates than cases. Specifically, each outlying point in the data is estimated using case-specific parameter. Penalized estimators are often suggested when the number of parameters in the model is more than the number of observed data points. In light of this, we propose the use of Ridge regression method for estimating the case-specific parameters. The proposed robust regression method was validated using Monte-Carlo datasets of varying proportion of outliers. Also, performance comparison was done for the proposed method with some existing robust methods. Assessment criteria results using breakdown point and efficiency revealed the supremacy of the proposed method over the existing methods considered.
Robust model selection criteria for robust S and LTS estimators
Hacettepe Journal of Mathematics and Statistics, 2015
Outliers and multi-collinearity often have large influence in the model/variable selection process in linear regression analysis. To investigate this combined problem of multi-collinearity and outliers, we studied and compared Liu-type S (liuS-estimators) and Liu-type Least Trimmed Squares (liuLTS) estimators as robust model selection criteria. Therefore, the main goal of this study is to select subsets of independent variables which explain dependent variables in the presence of multi-collinearity, outliers and possible departures from the normality assumption of the error distribution in regression analysis using these models.
Leverages, Outliers and the Performance of Robust Regression Estimators
British Journal of Mathematics & Computer Science, 2016
In this study, we assess the performance of some robust regression methods. These are the least-trimmed squares estimator (LTSE), Huber maximum likelihood estimator (HME), S-Estimator (SE) and modified maximum likelihood estimator (MME) which are compared with the ordinary least squares Estimator (OLSE) at different levels of leverages in the predictor variables. Anthropometric data from Komfo Anokye Teaching Hospital (KATH) was used and the comparison is done using root mean square error (RMSE), relative efficiencies (RE), coefficients of determination (R-squared) and power of the test. The results show that robust methods are as efficient as the OLSE if the assumptions of OLSE are met. OLSE is affected by low and high percentage of leverages, HME broke-down with leverages in data. MME and SE are robust to all percentage of aberrations, while LTSE is slightly affected by high percentage leverages perturbation. Thus, MME and SE are the most robust methods, while OLSE and HME are the least robust and the performance of the LTSE is affected by higher percentages of leverage in this study.