Alberto piano soul | Universidad Carlos III de Madrid (original) (raw)

Papers by Alberto piano soul

Research paper thumbnail of Compound key word generation from document databases using a hierarchical clustering ART model

Intelligent Data Analysis, 1997

The growing availability of databases on the information highways motivates the development of ne... more The growing availability of databases on the information highways motivates the development of new processing tools able to deal with a heterogeneous and changing information environment. A highly desirable feature of data processing systems handling this type of information is the ability to automatically extract its own key words. In this paper we address the specific problem of creating semantic term associations from a text database. The proposed method uses a hierarchical model made up of Fuzzy Adaptive Resonance Theory (ART) neural networks. First, the system uses several Fuzzy ART modules to cluster isolated words into semantic classes, starting from the database raw text. Next, this knowledge is used together with coocurrence information to extract semantically meaningful term associations. These associations are asymmetric and one-to-many due to the polisemy phenomenon. The strength of the associations between words can be measured numerically. Besides this, they implicitly define a hierarchy between descriptors. The underlying algorithm is appropriate for employment on large databases. The operation of the system is illustrated on several real databases.

Research paper thumbnail of On the Fusion of Polynomial Kernels for Support Vector Classifiers

On the Fusion of Polynomial Kernels for Support Vector Classifiers

Lecture Notes in Computer Science, 2006

In this paper we propose some methods to build a kernel matrix for classification purposes using ... more In this paper we propose some methods to build a kernel matrix for classification purposes using Support Vector Machines (SVMs) by fusing polynomial kernels. The proposed techniques have been successfully evaluated on artificial and real data sets. The new methods outperform the best individual kernel under consideration and they can be used as an alternative to the parameter selection problem

Research paper thumbnail of One-Class Support Vector Machines and Density Estimation: The Precise Relation

Lecture Notes in Computer Science, 2004

One-Class Support Vector Machines (SVM) afford the problem of estimating high density regions fro... more One-Class Support Vector Machines (SVM) afford the problem of estimating high density regions from univariate or multivariate data samples. To be more precise, sets whose probability is specified in advance are estimated. In this paper the exact relation between One-Class SVM and density estimation is demonstrated. This relation provides theoretical background for the behaviour of One-Class SVM when the Gaussian kernel is used, the only case for which successful results are shown in the literature.

Research paper thumbnail of Level Sets Based Distances for Probability Measures and Ensembles with Applications

In this paper we study Probability Measures (PM) from a functional point of view: we show that PM... more In this paper we study Probability Measures (PM) from a functional point of view: we show that PMs can be considered as functionals (generalized functions) that belong to some functional space endowed with an inner product. This approach allows us to introduce a new family of distances for PMs, based on the action of the PM functionals on `interesting' functions of the sample. We propose a specific (non parametric) metric for PMs belonging to this class, based on the estimation of density level sets. Some real and simulated data sets are used to measure the performance of the proposed distance against a battery of distances widely used in Statistics and related areas.

Research paper thumbnail of Monitoring Nonlinear Profiles Using Support Vector Machines

Lecture Notes in Computer Science

In this work we focus on the use of SVMs for monitoring techniques applied to nonlinear profiles ... more In this work we focus on the use of SVMs for monitoring techniques applied to nonlinear profiles in the Statistical Process Control (SPC) framework. We develop a new methodology based on Functional Data Analysis for the construction of control limits for nonlinear profiles. In particular, we monitor the fitted curves themselves instead of monitoring the parameters of any model fitting the curves. The simplicity and effectiveness of the data analysis method has been tested against other statistical approaches using a standard data set in the process control literature.

Research paper thumbnail of Locally linear approximation for Kernel methods : the Railway Kernel

In this paper we present a new kernel, the Railway Kernel, that works properly for general (nonli... more In this paper we present a new kernel, the Railway Kernel, that works properly for general (nonlinear) classification problems, with the interesting property that acts locally as a linear kernel. In this way, we avoid potential problems due to the use of a general purpose kernel, like the RBF kernel, as the high dimension of the induced feature space. As a consequence, following our methodology the number of support vectors is much lower and, therefore, the generalization capability of the proposed kernel is higher than the obtained using RBF kernels. Experimental work is shown to support the theoretical issues.

Research paper thumbnail of Rejoinder to "Support Vector Machines with Applications

Rejoinder to ``Support Vector Machines with Applications'' [math.ST/0612817]

Research paper thumbnail of Improving Support Vector Classification via the Combination of Multiple Sources of Information

Lecture Notes in Computer Science, 2004

In this paper we describe several new methods to build a kernel matrix from a collection of kerne... more In this paper we describe several new methods to build a kernel matrix from a collection of kernels. This kernel will be used for classification purposes using Support Vector Machines (SVMs). The key idea is to extend the concept of linear combination of kernels to the concept of functional (matrix) combination of kernels. The functions involved in the combination take advantage of class conditional probabilities and nearest neighbour techniques. The proposed methods have been successfully evaluated on a variety of real data sets against a battery of powerful classifiers and other kernel combination techniques.

Research paper thumbnail of Fusion of Gaussian Kernels Within Support Vector Classification

Lecture Notes in Computer Science, 2006

In this paper we propose some methods to build a kernel matrix for classification purposes using ... more In this paper we propose some methods to build a kernel matrix for classification purposes using Support Vector Machines (SVMs) by fusing Gaussian kernels. The proposed techniques have been successfully evaluated on artificial and real data sets. The new methods outperform the best individual kernel under consideration and they can be used as an alternative to the parameter selection problem in Gaussian kernel methods.

Research paper thumbnail of Solving the One-Class Problem Using Neighbourhood Measures

Lecture Notes in Computer Science, 2004

The problem of estimating high density regions from univariate or multivariate data samples is st... more The problem of estimating high density regions from univariate or multivariate data samples is studied. To be more precise, we estimate minimum volume sets, whose probability is specified in advance. This problem arises in outlier detection and cluster analysis, and is strongly related to One-Class Support Vector Machines (SVM). In this paper we propose a new method to solve this problem, the Support Neighbour Machine (SNM). We show its properties and introduce a new class of kernels. Finally, numerical results illustrating the advantage of the new method are shown.

Research paper thumbnail of Self-organizing maps for outlier detection

Neurocomputing, 1998

In this paper we address the problem of multivariate outlier detection using the (unsupervised) s... more In this paper we address the problem of multivariate outlier detection using the (unsupervised) self-organizing map (SOM) algorithm introduced by Kohonen. We examine a number of techniques, based on summary statistics and graphics derived from the trained SOM, and conclude that they work well in cooperation with each other. Useful tools include the median interneuron distance matrix and the projection ofthe trained map (via Sammon's projection). SOM quantization errors provide an important complementary source of information for certain type of outlying behavior. Empirical results are reported on both artificial and real data.

Research paper thumbnail of Methods for the combination of kernel matrices within a support vector framework

Machine Learning, 2009

The problem of combining different sources of information arises in several situations, for insta... more The problem of combining different sources of information arises in several situations, for instance, the classification of data with asymmetric similarity matrices or the construction of an optimal classifier from a collection of kernels. Often, each source of information can be expressed as a similarity matrix. In this paper we propose a new class of methods in order to produce, for classification purposes, a single kernel matrix from a collection of kernel (similarity) matrices. Then, the constructed kernel matrix is used to train a Support Vector Machine (SVM). The key ideas within the kernel construction are twofold: the quantification, relative to the classification labels, of the difference of information among the similarities; and the extension of the concept of linear combination of similarity matrices to the concept of functional combination of similarity matrices. The proposed methods have been successfully evaluated and compared with other powerful classifiers and kernel combination techniques on a variety of artificial and real classification problems.

Research paper thumbnail of On the combination of kernels for support vector classifiers

Nº.: UC3M Working …, 2005

The problem of combining different sources of information arises in several situations, for insta... more The problem of combining different sources of information arises in several situations, for instance, the classification of data with asymmetric similarity matrices or the construction of an optimal classifier from a collection of kernels. Often, each source of information can be ...

Research paper thumbnail of Representing functional data in reproducing Kernel Hilbert spaces with applications to clustering, classification and time series problems

Functional data are difficult to manage for many traditional statistical techniques given their v... more Functional data are difficult to manage for many traditional statistical techniques given their very high (or intrinsically infinite) dimensionality. The reason is that functional data are essentially functions and most algorithms are designed to work with (low) finite-dimensional vectors. Within this context we propose techniques to obtain finitedimensional representations of functional data. The key idea is to consider each functional curve as a point in a general function space and then project these points onto a Reproducing Kernel Hilbert Space with the aid of Regularization theory. In this work we describe the projection method, analyze its theoretical properties and propose a model selection procedure to select appropriate Reproducing Kernel Hilbert spaces to project the functional data.

Research paper thumbnail of Functional Learning of Kernels for Information Fusion Purposes

Lecture Notes in Computer Science, 2008

When there are several sources of information available in pattern recognition problems, the task... more When there are several sources of information available in pattern recognition problems, the task of combining them is most interesting. In the context of kernel methods it means to design a single kernel function that collects all the relevant information of each kernel for the classification task at hand. The problem is then solved by training a Support Vector Machine (SVM) on the resulting kernel. Here we propose a consistent method to produce kernel functions from kernel matrices created by any given kernel combination technique. Once this fusion kernel function is available, it will be possible to evaluate the kernel at any data point. The performance of the proposed fusion Kernel is illustrated on several classification and visualization tasks.

Research paper thumbnail of From Indefinite to Positive Semi-Definite Matrices

Lecture Notes in Computer Science, 2006

Similarity based classification methods use positive semidefinite (PSD) similarity matrices. When... more Similarity based classification methods use positive semidefinite (PSD) similarity matrices. When several data representations (or metrics) are available, they should be combined to build a single similarity matrix. Often the resulting combination is an indefinite matrix and can not be used to train the classifier. In this paper we introduce new methods to build a PSD matrix from an indefinite matrix. The obtained matrices are used as input kernels to train Support Vector Machines (SVMs) for classification tasks. Experimental results on artificial and real data sets are reported.

Research paper thumbnail of Representing functional data using support vector machines

Pattern Recognition Letters, 2010

Functional data are difficult to manage for most classical statistical techniques, given the very... more Functional data are difficult to manage for most classical statistical techniques, given the very high (or intrinsically infinite) dimensionality. The reason lies in that functional data are functions and most algorithms are designed to work with low dimensional vectors. In this paper we propose a functional analysis technique to obtain finite-dimensional representations of functional data. The key idea is to consider each functional datum as a point in a general function space and then to project these points onto a Reproducing Kernel Hilbert Space with the aid of a support vector machine. We show some theoretical properties of the method and illustrate its performance in some classification examples.

Research paper thumbnail of Support Vector Machines with Applications

Statistical Science, 2006

Support vector machines (SVMs) appeared in the early nineties as optimal margin classifiers in th... more Support vector machines (SVMs) appeared in the early nineties as optimal margin classifiers in the context of Vapnik's statistical learning theory. Since then SVMs have been successfully applied to real-world data analysis problems, often providing improved results compared with other techniques. The SVMs operate within the framework of regularization theory by minimizing an empirical risk in a well-posed and consistent way. A clear advantage of the support vector approach is that sparse solutions to classification and regression problems are usually obtained: only a few samples are involved in the determination of the classification or regression functions. This fact facilitates the application of SVMs to problems that involve a large amount of data, such as text processing and bioinformatics tasks. This paper is intended as an introduction to SVMs and their applications, emphasizing their key features. In addition, some algorithmic extensions and illustrative real-world applications of SVMs are shown.

Research paper thumbnail of Compound key word generation from document databases using a hierarchical clustering ART model

Intelligent Data Analysis, 1997

The growing availability of databases on the information highways motivates the development of ne... more The growing availability of databases on the information highways motivates the development of new processing tools able to deal with a heterogeneous and changing information environment. A highly desirable feature of data processing systems handling this type of information is the ability to automatically extract its own key words. In this paper we address the specific problem of creating semantic term associations from a text database. The proposed method uses a hierarchical model made up of Fuzzy Adaptive Resonance Theory (ART) neural networks. First, the system uses several Fuzzy ART modules to cluster isolated words into semantic classes, starting from the database raw text. Next, this knowledge is used together with coocurrence information to extract semantically meaningful term associations. These associations are asymmetric and one-to-many due to the polisemy phenomenon. The strength of the associations between words can be measured numerically. Besides this, they implicitly define a hierarchy between descriptors. The underlying algorithm is appropriate for employment on large databases. The operation of the system is illustrated on several real databases.

Research paper thumbnail of On the Fusion of Polynomial Kernels for Support Vector Classifiers

On the Fusion of Polynomial Kernels for Support Vector Classifiers

Lecture Notes in Computer Science, 2006

In this paper we propose some methods to build a kernel matrix for classification purposes using ... more In this paper we propose some methods to build a kernel matrix for classification purposes using Support Vector Machines (SVMs) by fusing polynomial kernels. The proposed techniques have been successfully evaluated on artificial and real data sets. The new methods outperform the best individual kernel under consideration and they can be used as an alternative to the parameter selection problem

Research paper thumbnail of One-Class Support Vector Machines and Density Estimation: The Precise Relation

Lecture Notes in Computer Science, 2004

One-Class Support Vector Machines (SVM) afford the problem of estimating high density regions fro... more One-Class Support Vector Machines (SVM) afford the problem of estimating high density regions from univariate or multivariate data samples. To be more precise, sets whose probability is specified in advance are estimated. In this paper the exact relation between One-Class SVM and density estimation is demonstrated. This relation provides theoretical background for the behaviour of One-Class SVM when the Gaussian kernel is used, the only case for which successful results are shown in the literature.

Research paper thumbnail of Level Sets Based Distances for Probability Measures and Ensembles with Applications

In this paper we study Probability Measures (PM) from a functional point of view: we show that PM... more In this paper we study Probability Measures (PM) from a functional point of view: we show that PMs can be considered as functionals (generalized functions) that belong to some functional space endowed with an inner product. This approach allows us to introduce a new family of distances for PMs, based on the action of the PM functionals on `interesting' functions of the sample. We propose a specific (non parametric) metric for PMs belonging to this class, based on the estimation of density level sets. Some real and simulated data sets are used to measure the performance of the proposed distance against a battery of distances widely used in Statistics and related areas.

Research paper thumbnail of Monitoring Nonlinear Profiles Using Support Vector Machines

Lecture Notes in Computer Science

In this work we focus on the use of SVMs for monitoring techniques applied to nonlinear profiles ... more In this work we focus on the use of SVMs for monitoring techniques applied to nonlinear profiles in the Statistical Process Control (SPC) framework. We develop a new methodology based on Functional Data Analysis for the construction of control limits for nonlinear profiles. In particular, we monitor the fitted curves themselves instead of monitoring the parameters of any model fitting the curves. The simplicity and effectiveness of the data analysis method has been tested against other statistical approaches using a standard data set in the process control literature.

Research paper thumbnail of Locally linear approximation for Kernel methods : the Railway Kernel

In this paper we present a new kernel, the Railway Kernel, that works properly for general (nonli... more In this paper we present a new kernel, the Railway Kernel, that works properly for general (nonlinear) classification problems, with the interesting property that acts locally as a linear kernel. In this way, we avoid potential problems due to the use of a general purpose kernel, like the RBF kernel, as the high dimension of the induced feature space. As a consequence, following our methodology the number of support vectors is much lower and, therefore, the generalization capability of the proposed kernel is higher than the obtained using RBF kernels. Experimental work is shown to support the theoretical issues.

Research paper thumbnail of Rejoinder to "Support Vector Machines with Applications

Rejoinder to ``Support Vector Machines with Applications'' [math.ST/0612817]

Research paper thumbnail of Improving Support Vector Classification via the Combination of Multiple Sources of Information

Lecture Notes in Computer Science, 2004

In this paper we describe several new methods to build a kernel matrix from a collection of kerne... more In this paper we describe several new methods to build a kernel matrix from a collection of kernels. This kernel will be used for classification purposes using Support Vector Machines (SVMs). The key idea is to extend the concept of linear combination of kernels to the concept of functional (matrix) combination of kernels. The functions involved in the combination take advantage of class conditional probabilities and nearest neighbour techniques. The proposed methods have been successfully evaluated on a variety of real data sets against a battery of powerful classifiers and other kernel combination techniques.

Research paper thumbnail of Fusion of Gaussian Kernels Within Support Vector Classification

Lecture Notes in Computer Science, 2006

In this paper we propose some methods to build a kernel matrix for classification purposes using ... more In this paper we propose some methods to build a kernel matrix for classification purposes using Support Vector Machines (SVMs) by fusing Gaussian kernels. The proposed techniques have been successfully evaluated on artificial and real data sets. The new methods outperform the best individual kernel under consideration and they can be used as an alternative to the parameter selection problem in Gaussian kernel methods.

Research paper thumbnail of Solving the One-Class Problem Using Neighbourhood Measures

Lecture Notes in Computer Science, 2004

The problem of estimating high density regions from univariate or multivariate data samples is st... more The problem of estimating high density regions from univariate or multivariate data samples is studied. To be more precise, we estimate minimum volume sets, whose probability is specified in advance. This problem arises in outlier detection and cluster analysis, and is strongly related to One-Class Support Vector Machines (SVM). In this paper we propose a new method to solve this problem, the Support Neighbour Machine (SNM). We show its properties and introduce a new class of kernels. Finally, numerical results illustrating the advantage of the new method are shown.

Research paper thumbnail of Self-organizing maps for outlier detection

Neurocomputing, 1998

In this paper we address the problem of multivariate outlier detection using the (unsupervised) s... more In this paper we address the problem of multivariate outlier detection using the (unsupervised) self-organizing map (SOM) algorithm introduced by Kohonen. We examine a number of techniques, based on summary statistics and graphics derived from the trained SOM, and conclude that they work well in cooperation with each other. Useful tools include the median interneuron distance matrix and the projection ofthe trained map (via Sammon's projection). SOM quantization errors provide an important complementary source of information for certain type of outlying behavior. Empirical results are reported on both artificial and real data.

Research paper thumbnail of Methods for the combination of kernel matrices within a support vector framework

Machine Learning, 2009

The problem of combining different sources of information arises in several situations, for insta... more The problem of combining different sources of information arises in several situations, for instance, the classification of data with asymmetric similarity matrices or the construction of an optimal classifier from a collection of kernels. Often, each source of information can be expressed as a similarity matrix. In this paper we propose a new class of methods in order to produce, for classification purposes, a single kernel matrix from a collection of kernel (similarity) matrices. Then, the constructed kernel matrix is used to train a Support Vector Machine (SVM). The key ideas within the kernel construction are twofold: the quantification, relative to the classification labels, of the difference of information among the similarities; and the extension of the concept of linear combination of similarity matrices to the concept of functional combination of similarity matrices. The proposed methods have been successfully evaluated and compared with other powerful classifiers and kernel combination techniques on a variety of artificial and real classification problems.

Research paper thumbnail of On the combination of kernels for support vector classifiers

Nº.: UC3M Working …, 2005

The problem of combining different sources of information arises in several situations, for insta... more The problem of combining different sources of information arises in several situations, for instance, the classification of data with asymmetric similarity matrices or the construction of an optimal classifier from a collection of kernels. Often, each source of information can be ...

Research paper thumbnail of Representing functional data in reproducing Kernel Hilbert spaces with applications to clustering, classification and time series problems

Functional data are difficult to manage for many traditional statistical techniques given their v... more Functional data are difficult to manage for many traditional statistical techniques given their very high (or intrinsically infinite) dimensionality. The reason is that functional data are essentially functions and most algorithms are designed to work with (low) finite-dimensional vectors. Within this context we propose techniques to obtain finitedimensional representations of functional data. The key idea is to consider each functional curve as a point in a general function space and then project these points onto a Reproducing Kernel Hilbert Space with the aid of Regularization theory. In this work we describe the projection method, analyze its theoretical properties and propose a model selection procedure to select appropriate Reproducing Kernel Hilbert spaces to project the functional data.

Research paper thumbnail of Functional Learning of Kernels for Information Fusion Purposes

Lecture Notes in Computer Science, 2008

When there are several sources of information available in pattern recognition problems, the task... more When there are several sources of information available in pattern recognition problems, the task of combining them is most interesting. In the context of kernel methods it means to design a single kernel function that collects all the relevant information of each kernel for the classification task at hand. The problem is then solved by training a Support Vector Machine (SVM) on the resulting kernel. Here we propose a consistent method to produce kernel functions from kernel matrices created by any given kernel combination technique. Once this fusion kernel function is available, it will be possible to evaluate the kernel at any data point. The performance of the proposed fusion Kernel is illustrated on several classification and visualization tasks.

Research paper thumbnail of From Indefinite to Positive Semi-Definite Matrices

Lecture Notes in Computer Science, 2006

Similarity based classification methods use positive semidefinite (PSD) similarity matrices. When... more Similarity based classification methods use positive semidefinite (PSD) similarity matrices. When several data representations (or metrics) are available, they should be combined to build a single similarity matrix. Often the resulting combination is an indefinite matrix and can not be used to train the classifier. In this paper we introduce new methods to build a PSD matrix from an indefinite matrix. The obtained matrices are used as input kernels to train Support Vector Machines (SVMs) for classification tasks. Experimental results on artificial and real data sets are reported.

Research paper thumbnail of Representing functional data using support vector machines

Pattern Recognition Letters, 2010

Functional data are difficult to manage for most classical statistical techniques, given the very... more Functional data are difficult to manage for most classical statistical techniques, given the very high (or intrinsically infinite) dimensionality. The reason lies in that functional data are functions and most algorithms are designed to work with low dimensional vectors. In this paper we propose a functional analysis technique to obtain finite-dimensional representations of functional data. The key idea is to consider each functional datum as a point in a general function space and then to project these points onto a Reproducing Kernel Hilbert Space with the aid of a support vector machine. We show some theoretical properties of the method and illustrate its performance in some classification examples.

Research paper thumbnail of Support Vector Machines with Applications

Statistical Science, 2006

Support vector machines (SVMs) appeared in the early nineties as optimal margin classifiers in th... more Support vector machines (SVMs) appeared in the early nineties as optimal margin classifiers in the context of Vapnik's statistical learning theory. Since then SVMs have been successfully applied to real-world data analysis problems, often providing improved results compared with other techniques. The SVMs operate within the framework of regularization theory by minimizing an empirical risk in a well-posed and consistent way. A clear advantage of the support vector approach is that sparse solutions to classification and regression problems are usually obtained: only a few samples are involved in the determination of the classification or regression functions. This fact facilitates the application of SVMs to problems that involve a large amount of data, such as text processing and bioinformatics tasks. This paper is intended as an introduction to SVMs and their applications, emphasizing their key features. In addition, some algorithmic extensions and illustrative real-world applications of SVMs are shown.