Loading...
 
Skip to main content

More than Tables: Visualizing Anytime Performance in Single- and Multiobjective Optimization

Description

Assessing the performance of optimization algorithms is important for their design, selection and recommendation, in both single- and multiobjective settings. Despite the recent progress in how performance is analyzed in single-objective optimization, studies in multiobjective optimization often still follow a practice established two decades ago: presenting long tables of quality indicator values at a (single) fixed evaluation budget. This format places the burden of
interpretation on the reader and makes it difficult to understand how algorithms behave over time.

In contrast, single-objective studies nowadays often rely on aggregation and visualization of performance data, highlighting *anytime* algorithm behavior using runtime profile plots (also known as empirical runtime distribution functions or data profiles). These visualizations provide an immediate overview of algorithm performance over time and across problems.

This tutorial will show how similar ideas can be applied seamlessly across both single- and multiobjective scenarios, shifting from budget-based to target-based and therefore anytime performance assessment. We will introduce coco-viz, a lightweight software package that produces runtime profile plots, otherwise available only in larger frameworks like COCO and IOHProfiler, directly from simple input data (pairs of quality indicator values and the corresponding number of evaluations) with only minimal required data preparation.

Participants are encouraged to bring their own algorithm results for an interactive hands-on session with coco-viz, where they can generate visualizations of their data and, if they wish, compare it with that of other participants.


Organizers

Image
Dimo Brockhoff

Inria and Ecole Polytechnique, France

Dimo Brockhoff received his diploma in computer science from University of Dortmund, Germany in 2005 and his PhD (Dr. sc. ETH) from ETH Zurich, Switzerland in 2009. After two postdocs at Inria Saclay Ile-de-France (2009-2010) and at Ecole Polytechnique (2010-2011), he joined Inria in November 2011 as a permanent researcher (first in its Lille - Nord Europe research center and since October 2016 in the Saclay - Ile-de-France one). His research interests are focused on evolutionary multiobjective optimization (EMO), in particular on theoretical aspects of indicator-based search and on the benchmarking of blackbox algorithms in general. Dimo has co-organized all BBOB workshops since 2013 and has been EMO track co-chair at GECCO in 2013, 2014, and 2023.


Image
Olaf Mersmann

Federal University of Applied Administrative Sciences, Germany

Olaf Mersmann is a Professor of Computer Science at the Federal University of Applied
Administrative Sciences in Germany and before that he was a Professor of Data Science at TH Köln - University of Applied Sciences. He received his BSc, MSc and PhD in Statistics from TU Dortmund. His research interests include applying statistical and machine learning methods to large benchmark databases to gain insight into the structure of the algorithm choice problem, the automated design of benchmark functions and benchmark function sets and using these methods on real world engineering problems.


Image
Tea Tušar

Jožef Stefan Institute, Slovenia

Tea Tušar is a senior research associate at the Department of Intelligent Systems of the Jozef Stefan Institute in Ljubljana, Slovenia. She was awarded the PhD degree in Information and Communication Technologies by the Jozef Stefan International Postgraduate School for her work on visualizing solution sets in multiobjective optimization. She has completed a one-year postdoctoral fellowship at Inria Lille in France where she worked on benchmarking multiobjective optimizers. Her research interests include evolutionary algorithms for singleobjective and multiobjective optimization with emphasis on visualizing and benchmarking their results and applying them to real-world problems.