More than Tables: Visualizing Anytime Performance in Single- and Multiobjective Optimization
Description
Assessing the performance of optimization algorithms is important for their design, selection and recommendation, in both single- and multiobjective settings. Despite the recent progress in how performance is analyzed in single-objective optimization, studies in multiobjective optimization often still follow a practice established two decades ago: presenting long tables of quality indicator values at a (single) fixed evaluation budget. This format places the burden of
interpretation on the reader and makes it difficult to understand how algorithms behave over time.
In contrast, single-objective studies nowadays often rely on aggregation and visualization of performance data, highlighting *anytime* algorithm behavior using runtime profile plots (also known as empirical runtime distribution functions or data profiles). These visualizations provide an immediate overview of algorithm performance over time and across problems.
This tutorial will show how similar ideas can be applied seamlessly across both single- and multiobjective scenarios, shifting from budget-based to target-based and therefore anytime performance assessment. We will introduce coco-viz, a lightweight software package that produces runtime profile plots, otherwise available only in larger frameworks like COCO and IOHProfiler, directly from simple input data (pairs of quality indicator values and the corresponding number of evaluations) with only minimal required data preparation.
Participants are encouraged to bring their own algorithm results for an interactive hands-on session with coco-viz, where they can generate visualizations of their data and, if they wish, compare it with that of other participants.
Organizers
Olaf Mersmann is a Professor of Computer Science at the Federal University of Applied
Administrative Sciences in Germany and before that he was a Professor of Data Science at TH Köln - University of Applied Sciences. He received his BSc, MSc and PhD in Statistics from TU Dortmund. His research interests include applying statistical and machine learning methods to large benchmark databases to gain insight into the structure of the algorithm choice problem, the automated design of benchmark functions and benchmark function sets and using these methods on real world engineering problems.