Robust Nonparametric Probability Density Estimation by Soft Clustering (original) (raw)

Abstract

A method to estimate the probability density function of multivariate distributions is presented. The classical Parzen window approach builds a spherical Gaussian density around every input sample. This choice of the kernel density yields poor robustness for real input datasets. We use multivariate Student-t distributions in order to improve the adaptation capability of the model. Our method has a first stage where hard neighbourhoods are determined for every sample. Then soft clusters are considered to merge the information coming from several hard neighbourhoods. Hence, a specific mixture component is learned for each soft cluster. This leads to outperform other proposals where the local kernel is not as robust and/or there are no smoothing strategies, like the manifold Parzen windows.

Preview

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Bezdek, J.C.: Numerical taxonomy with fuzzysets. J. Math. Biol. 1, 57–71 (1974)
    Article MATH MathSciNet Google Scholar
  2. Bishop, C.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1995)
    Google Scholar
  3. Hjort, N.L., Jones, M.C.: Locally Parametric Nonparametric Density Estimation. Annals of Statistics 24(4), 1619–1647 (1996)
    Article MATH MathSciNet Google Scholar
  4. Izenman, A.J.: Recent developments in nonparametric density estimation. Journal of the American Statistical Association 86(413), 205–224 (1991)
    Article MATH MathSciNet Google Scholar
  5. Kanzow, C., Yamashita, N., Fukushima, M.: Levenberg-Marquardt methods for constrained nonlinear equations with strong local convergence properties. Journal of Computational and Applied Mathematics 172, 375–397 (2004)
    Article MATH MathSciNet Google Scholar
  6. Lejeune, M., Sarda, P.: Smooth estimators of distribution and density functions. Computational Statistics and Data Analysis 14, 457–471 (1992)
    Article MATH MathSciNet Google Scholar
  7. McLachlan, G., Peel, D.: Finite Mixture Models. Wiley, Chichester (2000)
    MATH Google Scholar
  8. Newman, D.J., Hettich, S., Blake, C.L., Merz, C.J.: UCI Repository of machine learning databases. Department of Information and Computer Science, University of California, Irvine (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html
    Google Scholar
  9. Parzen, E.: On the Estimation of a Probability Density Function and Mode. Annals of Mathematical Statistics 33, 1065–1076 (1962)
    Article MATH MathSciNet Google Scholar
  10. Shoham, S.: Robust clustering by deterministic agglomeration EM of mixtures of multivariate t-distributions. Pattern Recognition 35, 1127–1142 (2002)
    Article MATH Google Scholar
  11. Silverman, B.: Density Estimation for Statistics and Data Analysis. Chapman and Hall, New York (1986)
    MATH Google Scholar
  12. Svensén, M., Bishop, C.M.: Robust Bayesian mixture modeling. Neurocomputing 64, 235–252 (2005)
    Article Google Scholar
  13. Tipping, M.E., Bishop, C.M.: Mixtures of Probabilistic Principal Components Analyzers. Neural Computation 11, 443–482 (1999)
    Article Google Scholar
  14. Vapnik, V.N.: Statistical Learning Theory. John Wiley and Sons, New York (1998)
    MATH Google Scholar
  15. Vincent, P., Bengio, Y.: Manifold Parzen Windows. Advances in Neural Information Processing Systems 15, 825–832 (2003)
    Google Scholar
  16. Wang, H., Zhang, Q., Luo, B., Wei, S.: Robust mixture modelling using multivariate t-distribution with missing information. Pattern Recognition Letters 25, 701–710 (2004)
    Article Google Scholar

Download references

Author information

Authors and Affiliations

  1. School of Computing, University of Málaga, Campus de Teatinos, s/n., 29071, Málaga, Spain
    Ezequiel López-Rubio, Juan Miguel Ortiz-de-Lazcano-Lobato, Domingo López-Rodríguez & María del Carmen Vargas-Gonzalez

Authors

  1. Ezequiel López-Rubio
  2. Juan Miguel Ortiz-de-Lazcano-Lobato
  3. Domingo López-Rodríguez
  4. María del Carmen Vargas-Gonzalez

Editor information

Véra Kůrková Roman Neruda Jan Koutník

Rights and permissions

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

López-Rubio, E., Ortiz-de-Lazcano-Lobato, J.M., López-Rodríguez, D., del Carmen Vargas-Gonzalez, M. (2008). Robust Nonparametric Probability Density Estimation by Soft Clustering. In: Kůrková, V., Neruda, R., Koutník, J. (eds) Artificial Neural Networks - ICANN 2008. ICANN 2008. Lecture Notes in Computer Science, vol 5163. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87536-9\_17

Download citation

Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Publish with us