Support Vector Machines

Let us imagine a linearly separable training data set, characterized by only two class labels. In such a situation, the binary classification task can be accomplished by determining a linear separator for the training points. Given that an infinite number of possible linear separators exist in general, methods such as the Perceptron algorithm find just any of the existing separators, while other methods search for the “best” linear separator, according to some criterion. Support Vector Machines (SVMs) aim to find a decision surface that is maximally far away from any data point, with the objective of maximizing classification accuracy and robustness, and generalization ability. In this chapter, SVMs are first introduced for binary classification and for linearly separable problems. Then, the concepts are extended to nonlinearly separable problems and multiclass classification.