Neuroevolution under unimodal error landscapes: An exploration of the semantic learning machine algorithm

Neuroevolution is a field in which evolutionary algorithms are applied with the goal of evolving Neural Networks (NNs). This paper studies different variants of the Semantic Learning Machine (SLM) algorithm, a recently proposed supervised learning neuroevolution method. Perhaps the most interesting characteristic of SLM is that it searches over unimodal error landscapes in any supervised learning problem where the error is measured as a distance to the known targets. SLM is compared with the NeuroEvolution of Augmenting Topologies (NEAT) algorithm and with a fixed-topology neuroevolution approach. Experiments are performed on a total of 9 real-world regression and classification datasets. The results show that the best SLM variants generally outperform the other neuroevolution approaches in terms of generalization achieved, while also being more efficient in learning the training data. The best SLM variants also outperform the common NN backpropagation-based approach under different topologies. The most efficient SLM variant used in combination with a recently proposed semantic stopping criterion is capable of evolving competitive neural networks in a few seconds on the vast majority of the datasets considered. A final comparison shows that a NN ensemble built with SLM is able to outperform the Random Forest algorithm in two classification datasets.