Electromagnetism-like algorithm for support vector machine parameter tuning

Abstract

This paper introduces an electromagnetism-like (EM) approach for solving the problem of parameter tuning in the support vector machine (SVM). The proposed method is used to tune binary SVM classifiers in single and multiple kernel mode. The internal kernel structure is based on linear and radial basis functions (RBF). An appropriate encoding scheme of EM enables easy transformation of real-valued EM points directly to real-valued parameter combinations. Estimations of the generalization error based on the cross-validation and validation set error are used as objective functions. The efficient local search procedure uses variable size interval movement in order to improve the convergence of the method. The quality of the proposed method is tested on four collections of testing benchmarks through five separate experiments. The first three collections consist of small-size to medium-size classification data sets with up to 60 features and 1,300 training vectors, while the fourth collection is formed of large heterogeneous data sets with up to 1,554 features and 2,186 training vectors. The obtained results indicate that EM outperforms the comparison algorithms in 10 out of 13 instances from the first collection, 5 out of 5 instances from the second, and 13 out of 15 instances from the third collection. The last two experiments, conducted on the fourth collection, show that the proposed method outperforms 14 successful methods in 3 out of 5 data sets where RBF multiple kernel learning is used, and behaves competitively in cases when linear kernels are used.