Evolutionary Optimization of Least-Squares Support Vector Machines (original) (raw)

The performance of Kernel Machines depends to a large extent on its kernel function and hyperparameters. Selecting these is traditionally done using intuition or a costly "trial-and-error" approach, which typically prevents these methods from being used to their fullest extent. Therefore, two automated approaches are presented for the selection of a suitable kernel function and optimal hyperparameters for the Least-Squares Support Vector Machine. The first approach uses Evolution Strategies, Genetic Algorithms, and Genetic Algorithms with floating point representation to find optimal hyperparameters in a timely manner. On benchmark data sets the standard Genetic Algorithms approach outperforms the two other evolutionary algorithms and is shown to be more efficient than grid search. The second approach aims to improve the generalization capacity of the machine by evolving combined kernel functions using Genetic Programming. Empirical studies show that this model indeed increases the generalization performance of the machine, although this improvement comes at a high computational cost. This suggests that the approach may be justified primarily in applications where prediction errors can have severe consequences, such as in medical settings.