Extremum estimator (original) (raw)

From Wikipedia, the free encyclopedia

In statistics and econometrics, extremum estimators are a wide class of estimators for parametric models that are calculated through maximization (or minimization) of a certain objective function, which depends on the data. The general theory of extremum estimators was developed by Amemiya (1985).

An estimator θ ^ {\displaystyle \scriptstyle {\hat {\theta }}} {\displaystyle \scriptstyle {\hat {\theta }}} is called an extremum estimator, if there is an objective function Q ^ n {\displaystyle \scriptstyle {\hat {Q}}_{n}} {\displaystyle \scriptstyle {\hat {Q}}_{n}} such that

θ ^ = a r g m a x θ ∈ Θ Q ^ n ( θ ) , {\displaystyle {\hat {\theta }}={\underset {\theta \in \Theta }{\operatorname {arg\;max} }}\ {\widehat {Q}}_{n}(\theta ),} {\displaystyle {\hat {\theta }}={\underset {\theta \in \Theta }{\operatorname {arg\;max} }}\ {\widehat {Q}}_{n}(\theta ),}

where Θ is the parameter space. Sometimes a slightly weaker definition is given:

Q ^ n ( θ ^ ) ≥ max θ ∈ Θ Q ^ n ( θ ) − o p ( 1 ) , {\displaystyle {\widehat {Q}}_{n}({\hat {\theta }})\geq \max _{\theta \in \Theta }\,{\widehat {Q}}_{n}(\theta )-o_{p}(1),} {\displaystyle {\widehat {Q}}_{n}({\hat {\theta }})\geq \max _{\theta \in \Theta }\,{\widehat {Q}}_{n}(\theta )-o_{p}(1),}

where o p(1) is the variable converging in probability to zero. With this modification θ ^ {\displaystyle \scriptstyle {\hat {\theta }}} {\displaystyle \scriptstyle {\hat {\theta }}} doesn't have to be the exact maximizer of the objective function, just be sufficiently close to it.

The theory of extremum estimators does not specify what the objective function should be. There are various types of objective functions suitable for different models, and this framework allows us to analyse the theoretical properties of such estimators from a unified perspective. The theory only specifies the properties that the objective function has to possess, and so selecting a particular objective function only requires verifying that those properties are satisfied.

When the parameter space Θ is not compact (Θ = R in this example), then even if the objective function is uniquely maximized at _θ_0, this maximum may be not well-separated, in which case the estimator θ ^ {\displaystyle \scriptscriptstyle {\hat {\theta }}} {\displaystyle \scriptscriptstyle {\hat {\theta }}} will fail to be consistent.

If the parameter space Θ is compact and there is a limiting function _Q_0(θ) such that: Q ^ n ( θ ) {\displaystyle \scriptstyle {\hat {Q}}_{n}(\theta )} {\displaystyle \scriptstyle {\hat {Q}}_{n}(\theta )} converges to _Q_0(θ) in probability uniformly over Θ, and the function _Q_0(θ) is continuous and has a unique maximum at θ = _θ_0 then θ ^ {\displaystyle \scriptstyle {\hat {\theta }}} {\displaystyle \scriptstyle {\hat {\theta }}} is consistent for _θ_0.[1]

The uniform convergence in probability of Q ^ n ( θ ) {\displaystyle \scriptstyle {\hat {Q}}_{n}(\theta )} {\displaystyle \scriptstyle {\hat {Q}}_{n}(\theta )} means that

sup θ ∈ Θ | Q ^ n ( θ ) − Q 0 ( θ ) | → p 0. {\displaystyle \sup _{\theta \in \Theta }{\big |}{\hat {Q}}_{n}(\theta )-Q_{0}(\theta ){\big |}\ {\xrightarrow {p}}\ 0.} {\displaystyle \sup _{\theta \in \Theta }{\big |}{\hat {Q}}_{n}(\theta )-Q_{0}(\theta ){\big |}\ {\xrightarrow {p}}\ 0.}

The requirement for Θ to be compact can be replaced with a weaker assumption that the maximum of _Q_0 was well-separated, that is there should not exist any points θ that are distant from _θ_0 but such that _Q_0(θ) were close to _Q_0(_θ_0). Formally, it means that for any sequence {θi} such that _Q_0(θi) → _Q_0(_θ_0), it should be true that θi → _θ_0.

Asymptotic normality

[edit]

Assuming that consistency has been established and the derivatives of the sample Q n {\displaystyle Q_{n}} {\displaystyle Q_{n}} satisfy some other conditions,[2] the extremum estimator converges to an asymptotically Normal distribution.

  1. ^ Newey & McFadden (1994), Theorem 2.1
  2. ^ Shi, Xiaoxia. "Lecture Notes: Asymptotic Normality of Extremum Estimators" (PDF).
  3. ^ Hayashi, Fumio (2000). Econometrics. Princeton: Princeton University Press. p. 448. ISBN 0-691-01018-8.
  4. ^ Hayashi, Fumio (2000). Econometrics. Princeton: Princeton University Press. p. 447. ISBN 0-691-01018-8.