The Benchmark Design Challenge for Continuous Optimization
Deadline: 2026-06-29
Webpage: https://iohprofiler.github.io/competitions/benchdesign
Description
This competition challenges participants to design new and interesting benchmark functions for continuous optimization.
Unlike traditional competitions that focus on solving predefined problems, this challenge inverts the paradigm: participants submit problem sets that best differentiate the performance of a predefined set of optimization algorithms.
The objective is to advance our understanding of what makes optimization problems difficult, diverse, or distinctive with respect to algorithm performance.
Participation is simple: you design a set of 25 benchmark problems of the form $0,1^N\rightarrow\mathbb{R}$.
The quality of your problem set is then judged in terms of performance diversity of a set of 5 commonly-used algorithms (CMA-ES, DE, PSO, BFGS, COBYLA). The exact evaluation procedure is present in our getting started files. For details about the limitations on the problem dimensions, details about test instances, etc., please refer to the dedicated competition website.
We particularly welcome submissions that are accompanied by a short paper or technical report (up to 2 pages following the GECCO formatting requirements) describing the key approaches used to create the submitted problems. Each team can submit only a single problem set.
Abstract Submission
The competition allows 2-page contributions to the GECCO Companion to present short descriptions of the competition entry, focusing on algorithmic design, strengths and limitations. The 2-page abstract paper will require at least one author to register at the conference as a presenter. It is important to mention that these 2-page abstracts ARE NOT APC Eligible (no publication fee has to be paid by the authors) under the current ACM Open publishing guidelines. The following dates are relevant for these submissions:
- Submission opening: April 1, 2026
- Submission deadline: April 21, 2026
- Notification: April 28, 2026
- Camera-ready: May 5, 2026
- Author's mandatory registration: May 11, 2026
Organizers
Niki van Stein is an Associate Professor at the Leiden Institute of Advanced Computer Science (LIACS), Leiden University, specializing in Explainable Artificial Intelligence (XAI). Since January 2022, Dr. van Stein has led the XAI research group and is a member of the management team of the Natural Computing cluster. Her research focuses on the intersection of machine learning, LLMs, optimization, and XAI, with applications in predictive maintenance, time-series analysis, and engineering design. Dr. van Stein obtained a PhD in Computer Science from Leiden University in 2018, under the supervision of Prof. Dr. Thomas Bäck, with a thesis on data-driven modelling and optimization of industrial processes.
With over 90 peer-reviewed publications and multiple awards, including best paper recognitions at GECCO and the IEEE Symposium Series on Computational Intelligence, Dr. van Stein has made significant contributions to the fields of evolutionary computing and explainable artificial intelligence.