Regression Estimators Related to Multinormal Distributions: Computer Experiences in Root Finding (original) (raw)
European Journal of Operational Research, 1998
Several linear regression estimators are presented, which approximate the distribution function of the m-dimensional normal distribution, or the distribution function along a line. These regression estimators are quadratic functions, or simple functions of quadratic functions and can be applied in numerical problems, arising during optimization of stochastic programming problems. A root ®nding procedure is developed, that can be used to ®nd the intersection of a line and the border of the feasible set. Directional derivatives and gradient of the normal distribution can be computed. Some numerical results are also presented. Ó
Linear and Second Order Cone Programming Approaches to Statistical Estimation Problems
2000
Estimation problems in statistics can often be formulated as nonlinear optimization problems where the approximating function is also constrained to satisfy certain shape constraints. For example, it may be required that the approximating function be nonnegative, nondecreasing or nonincreasing in some variables, unimodal in a variable, convex, or concave. We show how these shape constraints can be included in models where the approximating function is a polynomial spline of a given degree. Four classes of problems are considered: univariate nonparametric regression with nonnegativity constraints; isotonic, convex, and concave nonparametric regression; density estimation; and univariate arrival rate approximation. It is shown that all of these problems can be modeled by convex optimization models with linear and second-order conic constraints, which can be handled very efficiently both in theory and in practice. It is shown that in certain cases a linearly constrained model is enough, while in other cases nonlinear constraints are unavoidable. Extensive numerical computations show that these models often outperform or match the quality of results obtained using other popular nonparametric statistical methods, such as state-of-the-art kernel methods.
M-PROCEDURES IN THE GENERAL MULTIVARIATE NONLINEAR REGRESSION MODEL
Pakistan Journal of Statistics, 2010
In the multivariate nonlinear regression model, parameter estimators and test statistics based on least squares and maximum likelihood methods are usually nonrobust. For this type of models, we introduce M-estimators and M-tests, which are robust to departures from normality. In addition, we study the asymptotic properties and consider a computational algorithm for these estimators.
Section I: Computational Statistics
Notions of depth in regression have been introduced and studied in the literature. The most famous example is Regression Depth (RD), which is a direct extension of location depth to regression. The projection regression depth (PRD) is the extension of another prevailing location depth, the projection depth, to regression. The computation issues of the RD have been discussed in the literature. The computation issues of the PRD have never been dealt with before. The computation issues of the PRD and its induced median (maximum depth estimator) in a regression setting are addressed now. For a given β ∈ R p exact algorithms for the PRD with cost O(n 2 log n) (p = 2) and O(N (n, p)(p 3 + n log n + np 1.5 + npN Iter)) (p > 2) and approximate algorithms for the PRD and its induced median with cost respectively O(N v np) and O(RpN β (p 2 +nN v N Iter)) are proposed. Here N (n, p) is a number defined based on the total number of (p − 1) dimensional hyperplanes formed by points induced from sample points and the β; N v is the total number of unit directions v utilized; N β is the total number of candidate regression parameters β employed; N Iter is the total number of iterations carried out in an optimization algorithm; R is the total number of replications. Furthermore, as the second major contribution, three PRD induced estimators, which can be computed up to 30 times faster than that of the PRD induced median while maintaining a similar level of accuracy are introduced. Examples and simulation studies reveal that the depth median induced from the PRD is favorable in terms of robustness and efficiency, compared to the maximum depth estimator induced from the RD, which is the current leading regression median.
On the Relationship Between Regression Analysis and Mathematical Programming
Journal of Applied Mathematics and Decision Sciences, 2004
The interaction between linear, quadratic programming and regression analysis are explored by both statistical and operations research methods. Estimation and optimization problems are formulated in two different ways: on one hand linear and quadratic programming problems are formulated and solved by statistical methods, and on the other hand the solution of the linear regression model with constraints makes use of the simplex methods of linear or quadratic programming. Examples are given to illustrate the ideas.
Fast and robust estimation of the multivariate errors in variables model
2003
In the multivariate errors in variable models one wishes to retrieve a linear relationship of the form y = ß x + a, where both x and y can be multivariate. The variables y and x are not directly measurable, but observed with measurement error. The classical approach to estimate the multivariate errors in variable model is based on an eigenvector analysis of the joint covariance matrix of the observations. In this paper a projection-pursuit approach is proposed to estimate the unknown parameters. Focus is on projection indices based on half-samples. These will lead to robust estimators, which can be computed using fast algorithms. Consistency of the procedure is shown, without needing to make distributional assumptions on the x-variables. A simulation study gives insight in the robustness and the efficiency of the procedure.
Restricted expected multivariate least squares
Journal of Multivariate Analysis, 2006
A new approach of estimating parameters in multivariate models is introduced. A fitting function will be used. The idea is to estimate parameters so that the fitting function equals or will be close to its expected value. The function will be decomposed into two parts. From one part, which will be independent of the mean parameters, the dispersion matrix is estimated. This estimator is inserted in the second part which then yields the estimators of the mean parameters. The Growth Curve model, extended Growth Curve model and a multivariate variance components model will illustrate the approach.
The multilinear normal distribution: Introduction and some basic properties
Journal of Multivariate Analysis, 2013
In this paper, the multilinear normal distribution is introduced as an extension of the matrix-variate normal distribution. Basic properties such as marginal and conditional distributions, moments, and the characteristic function, are also presented. The estimation of parameters using a flip-flop algorithm is also briefly discussed.
Empirical Saddlepoint Approximations for Multivariate M-Estimators
Journal of the Royal Statistical Society: Series B (Methodological), 1994
In this paper. we investigate the use of the empirical distribution function in place of the underlying distribution function F to construct an empirical saddlepoint approximation to the density In of a general multivariate M-estimator. We obtain an explicit form for the error term in the approximation, investigate the effect of renormalizing the estimator, carry out some numerical comparisons and discuss the regression problem.
Multiple roots of estimating functions
Canadian Journal of Statistics, 1999
Estimating functions can have multiple roots. In such cases, the statistician must choose among the roots to estimate the parameter. Standard asymptotic theory shows that in a wide variety of cases, there exists a unique consistent root, and that this root will lie asymptotically close to other consistent (possibly inefficient) estimators for the parameter. For this reason, attention has largely focused on the problem of selecting this root and determining its approximate asymptotic distribution. In this paper, however, we shall concentrate on the exact distribution of the roots as a random set. In particular, we propose the use of higher order root intensity functions as a tool for examining the properties of the roots and determining their most problematic features. The use of root intensity functions of first and second order is illustrated by application to the score function for the Cauchy location model.
MULTIVARIATE REGRESSION S-ESTIMATORS FOR ROBUST ESTIMATION AND INFERENCE
2005
In this paper we consider S-estimators for multivariate regression. We study the robustness of the estimators in terms of their breakdown point and in- uence function. Our results extend results on S-estimators in the context of uni- variate regression and multivariate location and scatter. Furthermore we develop a fast and robust bootstrap method for the multivariate S-estimators to obtain in-
Possibilistic Linear Programming Problems involving Normal Random Variables
International Journal of Fuzzy System Applications, 2016
A new solution procedure of possibilistic linear programming problem is developed involving the right hand side parameters of the constraints as normal random variables with known means and variances and the objective function coefficients are considered as triangular possibility distribution. In order to solve the proposed problem, convert the problem into a crisp equivalent deterministic multi-objective mathematical programming problem and then solved by using fuzzy programming method. A numerical example is presented to illustrate the solution procedure and developed methodology.
Approximation of multivariate distribution functions
Mathematica Slovaca, 2008
In the paper the unknown distribution function is approximated with a known distribution function by means of Taylor expansion. For this approximation a new matrix operation — matrix integral — is introduced and studied in [PIHLAK, M.: Matrix integral, Linear Algebra Appl. 388 (2004), 315–325]. The approximation is applied in the bivariate case when the unknown distribution function is approximated with normal distribution function. An example on simulated data is also given.
Maximum Likelihood Estimation by Artificial Regression
Artificial regressions are developed, based on elementary zero functions, that exploit the fact that the normal distribution is completely characterised by its first two moments. These artificial regressions can be used as the basis of numerical algorithms for the maximum likelihood estimation of models with normally distributed random elements, and other estimation techniques based on the optimisation of criterion functions. The proposed algorithms are often simpler to program than many conventional algorithms for the optimisation of functions, and they have the advantage that an asymptotically correct estimate of the covariance matrix of the parameter estimates is computed as a by-product. Specific examples discussed include regression models with ARMA or (G)ARCH errors.