General regression neural network (original) (raw)

From Wikipedia, the free encyclopedia

Generalized regression neural network (GRNN) is a variation to radial basis neural networks. GRNN was suggested by D.F. Specht in 1991.[1]

GRNN can be used for regression, prediction, and classification. GRNN can also be a good solution for online dynamical systems.

GRNN represents an improved technique in the neural networks based on the nonparametric regression. The idea is that every training sample will represent a mean to a radial basis neuron.[2]

Mathematical representation

[edit]

Y ( x ) = ∑ k = 1 N y k K ( x , x k ) ∑ k = 1 N K ( x , x k ) {\displaystyle Y(x)={\frac {\sum _{k=1}^{N}y_{k}K(x,x_{k})}{\sum _{k=1}^{N}K(x,x_{k})}}} $Y(x)={\frac {\sum _{k=1}^{N}y_{k}K(x,x_{k})}{\sum _{k=1}^{N}K(x,x_{k})}}$

where:

K ( x , x k ) = e − d k / 2 σ 2 , d k = ( x − x k ) T ( x − x k ) {\displaystyle K(x,x_{k})=e^{-d_{k}/2\sigma ^{2}},\qquad d_{k}=(x-x_{k})^{T}(x-x_{k})} $K(x,x_{k})=e^{-d_{k}/2\sigma ^{2}},\qquad d_{k}=(x-x_{k})^{T}(x-x_{k})$

where d k {\displaystyle d_{k}} $d_{k}$ is the squared euclidean distance between the training samples x k {\displaystyle x_{k}} $x_{k}$ and the input x . {\displaystyle x.} $x.$

GRNN has been implemented in many computer languages including MATLAB,[3] R- programming language, Python (programming language) and Node.js.

Neural networks (specifically Multi-layer Perceptron) can delineate non-linear patterns in data by combining with generalized linear models by considering distribution of outcomes (sightly different from original GRNN). There have been several successful developments, including Poisson regression, ordinal logistic regression, quantile regression and multinomial logistic regression that described by Fallah in 2009.[4]

Advantages and disadvantages

[edit]

Similar to RBFNN, GRNN has the following advantages:

Single-pass learning so no backpropagation is required.
High accuracy in the estimation since it uses Gaussian functions.
It can handle noises in the inputs.
It requires relatively few data to train.

The main disadvantages of GRNN are:

Its size can be huge, which would make it computationally expensive.
There is no optimal method to improve it.

^ Specht, D. F. (1991-11-01). "A general regression neural network". IEEE Transactions on Neural Networks. 2 (6): 568–576. doi:10.1109/72.97934. PMID 18282872. S2CID 6266210.
^ https://minds.wisconsin.edu/bitstream/handle/1793/7779/ch2.pdf?sequence=14 [_bare URL PDF_]
^ "Generalized Regression Neural Networks - MATLAB & Simulink - MathWorks Australia".
^ Fallah, Nader; Gu, Hong; Mohammad, Kazem; Seyyedsalehi, Seyyed Ali; Nourijelyani, Keramat; Eshraghian, Mohammad Reza (2009). "Nonlinear Poisson regression using neural networks: A simulation study". Neural Computing and Applications. 18 (8): 939–943. doi:10.1007/s00521-009-0277-8. S2CID 18980875.