Genetic programming needs better benchmarks

James McDermott, David R. White, Sean Luke, Luca Manzoni, Mauro Castelli, Leonardo Vanneschi, Wojciech Jaśkowski, Krzysztof Krawiec, Robin Harper, Kenneth De Jong, Una May O'Reilly

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

166 Citations (Scopus)

Abstract

Genetic programming (GP) is not a field noted for the rigor of its benchmarking. Some of its benchmark problems are popular purely through historical contingency, and they can be criticized as too easy or as providing misleading information concerning real-world performance, but they persist largely because of inertia and the lack of good alternatives. Even where the problems themselves are impeccable, comparisons between studies are made more difficult by the lack of standardization. We argue that the definition of standard benchmarks is an essential step in the maturation of the field. We make several contributions towards this goal. We motivate the development of a benchmark suite and define its goals; we survey existing practice; we enumerate many candidate benchmarks; we report progress on reference implementations; and we set out a concrete plan for gathering feedback from the GP community that would, if adopted, lead to a standard set of benchmarks.

Original languageEnglish
Title of host publicationGECCO'12 - Proceedings of the 14th International Conference on Genetic and Evolutionary Computation
Pages791-798
Number of pages8
DOIs
Publication statusPublished - 13 Aug 2012
Event14th International Conference on Genetic and Evolutionary Computation, GECCO'12 - Philadelphia, PA, United States
Duration: 7 Jul 201211 Jul 2012

Publication series

NameGECCO'12 - Proceedings of the 14th International Conference on Genetic and Evolutionary Computation

Conference

Conference14th International Conference on Genetic and Evolutionary Computation, GECCO'12
CountryUnited States
CityPhiladelphia, PA
Period7/07/1211/07/12

Keywords

  • benchmarks
  • genetic programming

Fingerprint

Dive into the research topics of 'Genetic programming needs better benchmarks'. Together they form a unique fingerprint.

Cite this