Fitness landscape analysis of convolutional neural network architectures for image classification

Nuno M. Rodrigues, Katherine M. Malan, Gabriela Ochoa, Leonardo Vanneschi, Sara Silva

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)
34 Downloads (Pure)

Abstract

The global structure of the hyperparameter spaces of neural networks is not well understood and it is therefore not clear which hyperparameter search algorithm will be most effective. In this paper we analyze the landscapes of convolutional neural network architecture search spaces to provide insight into appropriate search algorithms for these spaces. Using a classical fitness landscape analysis approach (fitness distance correlation) and a more recent tool (local optima networks) we study the global structure of these spaces. Our analysis on six image classification datasets reveals that the landscapes are multi-modal, but with relatively few local optima from which it is not hard to escape with a simple perturbation operator. This led us to explore the performance of iterated local search, which we found to more effectively search the training landscapes than three evolutionary algorithm variants. Evolutionary algorithms, however, outperformed iterated local search in terms of generalization on problems with larger discrepancies between the training and testing landscapes.
Original languageEnglish
Pages (from-to)711-726
Number of pages16
JournalInformation Sciences
Volume609
Issue numberSeptember
Early online date11 Jul 2022
DOIs
Publication statusPublished - 1 Sep 2022

Keywords

  • Neural architecture search
  • Convolutional neural networks
  • Fitness distance correlation
  • Local optima networks
  • Loss landscapes

Fingerprint

Dive into the research topics of 'Fitness landscape analysis of convolutional neural network architectures for image classification'. Together they form a unique fingerprint.

Cite this