SKT: A Computationaly Efficient SUPANOVA: Spline Kernel Based Machine Learning Tool (original) (raw)
Related papers
A Computationally Efficient SUPANOVA: Spline Kernel Based Machine Learning Tool
Advances in Soft Computing, 2007
Many machine learning methods just consider the quality of prediction results as their final purpose. To make the prediction process transparent (reversible), spline kernel based methods were proposed by Gunn. However, the original solution method, termed SUpport vector Parsimonious ANOVA (SUPANOVA) was computationally very complex and demanding. In this paper, we propose a new heuristic to compute the optimal sparse vector in SUPANOVA that replaces the original solver for the convex quadratic problem of very high dimensionality. The resulting system is much faster without the loss of precision, as demonstrated in this paper on two benchmarks: the iris data set and the Boston housing market data benchmark.
Using efficient supanova kernel for heart disease diagnosis
proc. ANNIE, 2006
Many machine learning methods focus on the quality of prediction results as their final purpose. Spline kernel based methods attempt to provide also transparency to the prediction identifying features that are important in the decision process. In this paper, we present a new heuristic for computing efficiently sparse kernel in SUPANOVA. We applied it to a benchmark Boston housing market dataset and to socially important problem of improving the detection of heart diseases in the population using a novel, non-invasive measurement of the heart activities based on magnetic field produced by the human heart. On this data, 83.7% predictions were correct, exceeding the results obtained using the standard Support Vector Machine and equivalent kernels. Equally good results were achieved by the spline kernel on a benchmark Boston housing market dataset.
Penalised spline support vector classifiers: computational issues
Computational Statistics, 2008
We study computational issues for support vector classification with penalised spline kernels. We show that, compared with traditional kernels, computational times can be drastically reduced in large problems making such problems feasible for sample sizes as large as ∼10 6 . The optimisation technology known as interior point methods plays a central role. Penalised spline kernels are also shown to allow simple incorporation of low-dimensional structure such as additivity. This can aid both interpretability and performance.
Kernel Estimation: the Equivalent Spline Smoothing Method
SSRN Electronic Journal, 1994
Among nonparametric smoothers, there is a well-known correspondence between kernel and Fourier series methods, pivoted by t h e F ourier transform of the kernel. This suggests a similar relationship between kernel and spline estimators. A known special case is the result of Silverman (1984) on the e ective k ernel for the classical Reinsch-Schoenberg smoothing spline in the nonparametric regression model. We present an extension by showing that a large class of kernel estimators have a spline equivalent, in the sense of identical asymptotic local behaviour of the weighting coe cients. This general class of spline smoothers includes also the minimax linear estimator over Sobolev ellipsoids. The analysis is carried out for piecewise linear splines and equidistant design.
Canadian Journal of Statistics, 2005
The authors propose "kernel spline regression," a method of combining spline regression and kernel smoothing by replacing the polynomial approximation for local polynomial kernel regression with the spline basis. The new approach retains the local weighting scheme and the use of a bandwidth to control the size of local neighborhood. The authors compute the bias and variance of the kernel linear spline estimator, which they compare with local linear regression. They show that kernel spline estimators can succeed in capturing the main features of the underlying curve more effectively than local polynomial regression when the curvature changes rapidly. They also show through simulation that kernel spline regression often performs better than ordinary spline regression and local polynomial regression. La regression spline a noyau Rksurnk : Les auteurs proposent la "dgression spline B noyau," une m6thode qui permet d'allier la dgression spline et le lissage B noyau en remplapnt par des splines l'approximation polynomiale de la dgression polynomiale locale B noyau. La nouvelle approche conserve la flexibilid offerte par la ponderation locale et par l'emploi d'une fenetre pour le contr6le de la taille du voisinage local. Les auteurs determinent le biais et la variance de l'estimateur spline link& B noyau et comparent leurs dsultats avec la dgression lineaire locale. Ils montrent que lorsque la courbm change rapidement, les principales caracdristiques de la courbe sous-jacente peuvent etre mieux saisies par les estimateurs splines B noyau que par la dgression polynomiale locale. Ils montrent aussi par simulation que la dgression spline B noyau donne souvent de meilleurs dsultats que la dgression spline ordinaire ou la rdgression polynomiale locale.
The combination of spline and kernel estimator for nonparametric regression and its properties
Applied Mathematical Sciences, 2015
Consider additive nonparametric regression model with two predictor variables components. In the first predictor component, the regression curve is approached using Spline regression, and in the second predictor component, the regression curve is approached using Kernel regression. Random error of regression model is assumed to have independent normal distribution with zero mean and the same variance. This article provides an estimator of Spline regression curve, estimator of Kernel regression curve, and an estimator of a combination of Spline and Kernel regressions. The produced estimators are biased estimators, but all estimators are classified as linear estimators in observation. Estimator of a combination of Spline and Kernel regression depended on knot points and bandwith parameter. The best estimator of a combination of Spline and Kernel regression is found by minimizing Generalized Cross Validation (GCV) function.
Machines and Kernel Partial Least Squares. The hyperparameters
2008
Abstract- We describe the use of machine learning for pattern recognition in magnetocardiography (MCG) that measures magnetic fields emitted by the electrophysiological activity of the heart. We used direct kernel methods to separate abnormal MCG heart patterns from normal ones. For unsupervised learning, we
Nonlinear kernel-based statistical pattern analysis
IEEE Transactions on Neural Networks, 2001
The eigenstructure of the second-order statistics of a multivariate random population can be inferred from the matrix of pairwise combinations of inner products of the samples. Therefore, it can be also efficiently obtained in the implicit, high-dimensional feature spaces defined by kernel functions. We elaborate on this property to obtain general expressions for immediate derivation of nonlinear counterparts of a number of standard pattern analysis algorithms, including principal component analysis, data compression and denoising, and Fisher's discriminant. The connection between kernel methods and nonparametric density estimation is also illustrated. Using these results we introduce the kernel version of Mahalanobis distance, which originates nonparametric models with unexpected and interesting properties, and also propose a kernel version of the minimum squared error (MSE) linear discriminant function. This learning machine is particularly simple and includes a number of generalized linear models such as the potential functions method or the radial basis function (RBF) network. Our results shed some light on the relative merit of feature spaces and inductive bias in the remarkable generalization properties of the support vector machine (SVM). Although in most situations the SVM obtains the lowest error rates, exhaustive experiments with synthetic and natural data show that simple kernel machines based on pseudoinversion are competitive in problems with appreciable class overlapping
Nonlinear Support Vector Machines Through Iterative Majorization and I-Splines
Studies in Classification, Data Analysis, and Knowledge Organization, 2007
To minimize the primal support vector machine (SVM) problem, we propose to use iterative majorization. To do so, we propose to use iterative majorization. To allow for nonlinearity of the predictors, we use (non)monotone spline transformations. An advantage over the usual kernel approach in the dual problem is that the variables can be easily interpreted. We illustrate this with an example from the literature.