LASIGE’s PhD student, Nuno Rodrigues authored the paper “Fitness landscape analysis of convolutional neural network architectures for image classification” published in the high-ranked journal Information Sciences (Scimago Q1, IF 8.233). The paper is co-authored by Katherine M. Malan (University of South Africa, South Africa), Gabriela Ochoa (University of Stirling, Scotland), Leonardo Vanneschi (NOVA Information Management School, Portugal), and LASIGE’s integrated researcher Sara Silva.

The global structure of the hyperparameter spaces of neural networks is not well understood and it is therefore not clear which hyperparameter search algorithm will be most effective. The authors analyzed the landscapes of convolutional neural network architecture search spaces to provide insight into appropriate search algorithms for these spaces. Using a classical fitness landscape analysis approach (fitness distance correlation) and a more recent tool (local optima networks) they studied the global structure of these spaces. The paper presents their analysis on six image classification datasets that reveal that the landscapes are multi-modal, but with relatively few local optima from which it is not hard to escape with a simple perturbation operator. This led them to explore the performance of iterated local search, which they found to more effectively search the training landscapes than three evolutionary algorithm variants. Evolutionary algorithms, however, outperformed iterated local search in terms of generalization on problems with larger discrepancies between the training and testing landscapes.

The article is available here.