Siamak Mehrkanoon | Maastricht University (original) (raw)

Papers by Siamak Mehrkanoon

Pattern Recognition, Jun 1, 2018

The use of indefinite kernels has attracted many research interests in recent years due to their ... more The use of indefinite kernels has attracted many research interests in recent years due to their flexibility. They do not possess the usual restrictions of being positive definite as in the traditional study of kernel methods. This paper introduces the indefinite unsupervised and semi-supervised learning in the framework of least squares support vector machines (LS-SVM). The analysis is provided for both unsupervised and semi-supervised models, i.e., Kernel Spectral Clustering (KSC) and Multi-Class Semi-Supervised Kernel Spectral Clustering (MSS-KSC). In indefinite KSC models one solves an eigenvalue problem whereas indefinite MSS-KSC finds the solution by solving a linear system of equations. For the proposed indefinite models, we give the feature space interpretation, which is theoretically important, especially for the scalability using Nyström approximation. Experimental results on several real-life datasets are given to illustrate the efficiency of the proposed indefinite kernel spectral learning.

Neural Computation, Jun 1, 2016

Robust support vector machines for classification with nonconvex and smooth losses

Mechanical Systems and Signal Processing, Jun 1, 2017

Structural health monitoring refers to the process of measuring damagesensitive variables to asse... more Structural health monitoring refers to the process of measuring damagesensitive variables to assess the functionality of a structure. In principle, vibration data can capture the dynamics of the structure and reveal possible failures, but environmental and operational variability can mask this information. Thus, an effective outlier detection algorithm can be applied only after having performed data normalization (i.e. filtering) to eliminate external influences. Instead, in this article we propose a technique which unifies the data normalization and damage detection steps. The proposed algorithm, called adaptive kernel spectral clustering (AKSC), is initialized and calibrated in a phase when the structure is undamaged. The calibration process is crucial to ensure detection of early damage and minimize the number of false alarms. After the calibration, the method can automatically identify new regimes which may be associated with possible faults. These regimes are discovered by means of two complementary damage (i.e. outlier) indicators. The proposed strategy is validated with a simulated example and with real-life natural frequency data from the Z24 pre-stressed concrete bridge, which was progressively damaged at the end of a one-year monitoring period.

A non-parallel semi-supervised algorithm based on kernel spectral clustering is formulated. The p... more A non-parallel semi-supervised algorithm based on kernel spectral clustering is formulated. The prior knowledge about the labels is incorporated into the kernel spectral clustering formulation via adding regularization terms. In contrast with the existing multi-plane classifiers such as Multisurface Proximal Support Vector Machine (GEPSVM) and Twin Support Vector Machines (TWSVM) and its least squares version (LSTSVM) we will not use a kernel-generated surface. Instead we apply the kernel trick in the dual. Therefore as opposed to conventional non-parallel classifiers one does not need to formulate two different primal problems for the linear and nonlinear case separately. Experimental results demonstrate the efficiency of the proposed method over existing methods.

Engineering Applications of Artificial Intelligence, Oct 1, 2016

Although time delay is an important element in both system identification and control performance... more Although time delay is an important element in both system identification and control performance assessment, its computation remains elusive. This paper proposes the application of a least squares support vector machines driven approach to the problem of determining constant time delay for a chemical process. The approach consists of two steps, where in the first step the state of the system and its derivative are approximated based on the LS-SVM model. The second stage consists of modeling the delay term and estimating the unknown model parameters as well as the time delay of the system. Therefore the proposed approach avoids integrating the given differential equation that can be computationally expensive. This time delay estimation method is applied to both simulation and experimental data obtained from a continuous, stirred, heated tank. The results show that the proposed method can provide accurate estimates even if significant noise or unmeasured additive disturbances are present.

Neurocomputing, Nov 1, 2014

This paper introduces a general framework of non-parallel support vector machines, which involves... more This paper introduces a general framework of non-parallel support vector machines, which involves a regularization term, a scatter loss and a misclassification loss. When dealing with binary problems, the framework with proper losses covers some existing non-parallel classifiers, such as multisurface proximal support vector machine via generalized eigenvalues, twin support vector machines, and its least squares version. The possibility of incorporating different existing scatter and misclassification loss functions into the general framework is discussed. Moreover, in contrast with the mentioned methods, which applies kernel-generated surface, we directly apply the kernel trick in the dual and then obtain nonparametric models. Therefore, one does not need to formulate two different primal problems for the linear and nonlinear kernel respectively. In addition, experimental results are given to illustrate the performance of different loss functions.

IEEE transactions on neural networks and learning systems, 2017

Domain adaptation learning is one of the fundamental research topics in pattern recognition and m... more Domain adaptation learning is one of the fundamental research topics in pattern recognition and machine learning. This paper introduces a Regularized Semi-Paired Kernel Canonical Correlation Analysis (RSP-KCCA) formulation for learning a latent space for the domain adaptation problem. The optimization problem is formulated in the primal-dual LS-SVM setting where side information can be readily incorporated through regularization terms. The proposed model learns a joint representation of the data set across different domains by solving a generalized eigenvalue problem or linear system of equations in the dual. The approach is naturally equipped with out-of-sample extension property which plays an important role for model selection. Furthermore, the Nyström approximation technique is used to make the computational issues due to the large size of the matrices involved in the eigendecomposition feasible. The learnt latent space of the source domain is fed to a Multi-Class Semi-Supervised Kernel Spectral Clustering model, MSS-KSC, that can learn from both labeled and unlabeled data points of the source domain in order to classify the data instances of the target domain. Experimental results are given to illustrate the effectiveness of the proposed approaches on synthetic and real-life datasets.

Neurocomputing, Jul 1, 2018

This paper introduces a novel hybrid deep neural kernel framework. The proposed deep learning mod... more This paper introduces a novel hybrid deep neural kernel framework. The proposed deep learning model makes a combination of a neural networks based architecture and a kernel based model. In particular, here an explicit feature map, based on random Fourier features, is used to make the transition between the two architectures more straightforward as well as making the model scalable to large datasets by solving the optimization problem in the primal. Furthermore, the introduced framework is considered as the first building block for the development of even deeper models and more advanced architectures. Experimental results show an improvement over shallow models and the standard non-hybrid neural networks architecture on several medium to large scale real-life datasets.

The European Symposium on Artificial Neural Networks, 2018

Manual labeling of sufficient training data for diverse application domains is a costly, laboriou... more Manual labeling of sufficient training data for diverse application domains is a costly, laborious task and often prohibitive. Therefore, designing models that can leverage rich labeled data in one domain and be applicable to a different but related domain is highly desirable. In particular, domain adaptation or transfer learning algorithms seek to generalize a model trained in a source domain to a new target domain. Recent years has witnessed increasing interest in these types of models due to their practical importance in real-life applications. In this paper we provide a brief overview of recent techniques with both shallow and deep architectures for domain adaptation models.

We live in the era of big data with dataset sizes growing steadily over the past decades. In addi... more We live in the era of big data with dataset sizes growing steadily over the past decades. In addition, obtaining expert labels for all the instances is time-consuming and in many cases may not even be possible. This necessitates the development of advanced semi-supervised models that can learn from both labeled and unlabeled data points and also scale at worst linearly with the number of examples. In the context of kernel based semisupervised models, constructing the training kernel matrix for the large training dataset is expensive and memory inefficient. This paper investigates the scalability of the recently proposed multiclass semi-supervised kernel spectral clustering model (MSSKSC) by means of random Fourier features. The proposed model maps the input data into an explicit low-dimensional feature space. Thanks to the explicit feature maps, one can then solve the MSSKSC optimization formation in the primal, making the complexity of the method linear in number of training data points. The performance of the proposed model is compared with that of recently introduced reduced kernel techniques and Nyström based MSSKSC approaches. Experimental results demonstrate the scalability, efficiency and faster training computation times of the proposed model over conventional large scale semi-supervised models on large scale real-life datasets.

Often in real-world applications such as web page categorization, automatic image annotations and... more Often in real-world applications such as web page categorization, automatic image annotations and protein function prediction, each instance is associated with multiple labels (categories) simultaneously. In addition, due to the labeling cost one usually deals with a large amount of unlabeled data while the fraction of labeled data points will typically be small. In this paper, we propose a multi-label semi-supervised kernel spectral clustering learning algorithm that learns from both labeled and unlabeled instances. The kernel spectral clustering algorithm (KSC) serves as a core model and the information of labeled data points is integrated into the model via regularization terms. The propagation of the multiple labels to unlabeled data points is achieved by incorporating the mutual correlation between (similarity across) labels as well as encouraging the model output to be as close as possible to the given ground-truth of the labeled data points. Thanks to the Nyström approximation method, an explicit feature map is constructed and the optimization problem is solved in the primal. Experimental results demonstrate the effectiveness of the proposed approaches on real multi-label datasets.

This paper introduces a software tool SYM-LS-SVM-SOLVER written in Maple to derive the dual syste... more This paper introduces a software tool SYM-LS-SVM-SOLVER written in Maple to derive the dual system and the dual model representation of LS-SVM based models, symbolically. SYM-LS-SVM-SOLVER constructs the Lagrangian from the given objective function and list of constraints. Afterwards it obtains the KKT (Karush-Kuhn-Tucker) optimality conditions and finally formulates a linear system in terms of the dual variables. The effectiveness of the developed solver is illustrated by applying it to a variety of problems involving LS-SVM based models.

The Canadian Journal of Chemical Engineering, 2017

The development and implementation of better control strategies to improve the overall performanc... more The development and implementation of better control strategies to improve the overall performance of a plant is often hampered by the lack of available measurements of key quality variables. One way to resolve this problem is to develop a soft sensor that is capable of providing process information as often as necessary for control. One potential area for implementation is in a hot steel rolling mill, where the final strip thickness is the most important variable to consider. Difficulties with this approach include the fact that the data may not be available when needed or that different conditions (operating points) will produce different process conditions. In this paper, a soft sensor is developed for the hot steel rolling mill process using least-squares support vector machines and a properly designed bias update term. It is shown that the system can handle multiple different operating conditions (different strip thickness setpoints, and input conditions).

2016 IEEE Symposium Series on Computational Intelligence (SSCI), 2016

2016 International Joint Conference on Neural Networks (IJCNN), 2016

2015 International Joint Conference on Neural Networks (IJCNN), 2015

This paper introduces a methodology to incorporate the label information in discovering the under... more This paper introduces a methodology to incorporate the label information in discovering the underlying clusters in a hierarchical setting using multi-class semi-supervised clustering algorithm. The method aims at revealing the relationship between clusters given few labels associated to some of the clusters. The problem is formulated as a regularized kernel spectral clustering algorithm in the primal-dual setting. The available labels are incorporated in different levels of hierarchy from top to bottom. As we advance towards the lowers levels in the tree all the previously added labels are used in the generation of the new levels of hierarchy. The model is trained on a subset of the data and then applied to the rest of the data in a learning framework. Thanks to the previously learned model, the out-of-sample extension property of the model allows then to predict the memberships of a new point. A combination of an internal clustering quality index and classification accuracy is used for model selection. Experiments are conducted on synthetic data and real image segmentation problems to show the applicability of the proposed approach.

IFAC Proceedings Volumes, 2012

This paper develops a new approach based on Least Squares Support Vector Machines (LS-SVMs) for p... more This paper develops a new approach based on Least Squares Support Vector Machines (LS-SVMs) for parameter estimation of time invariant as well as time varying dynamical SISO systems. Closed-form approximate models for the state and its derivative are first derived from the observed data by means of LS-SVMs. The time-derivative information is then substituted into the system of ODEs, converting the parameter estimation problem into an algebraic optimization problem. In the case of time invariant systems one can use least-squares to solve the obtained system of algebraic equations. The estimation of time-varying coefficients in SISO models, is obtained by assuming an LS-SVM model for it.