Performance Assessment of Population-Based Multiobjective Optimization Algorithms Using Composite Indicators

Description

Abstract

The performance of population-based multiobjective optimization algorithms is usually evaluated using indicators assessing the quality of the approximation set generated according to convergence, cardinality, spread, and uniformity (the combination of the last two known as diversity).

Since not all quality indicators can capture all these properties, we proposed to aggregate already-existing indicators into a single measure informing about the algorithm's performance from a general perspective. To synthesize the desired quality indicators, we built three composite quality indicators (weak, strong, and mixed) based on the reference point approach. This approach enabled the use of desirable value ranges for the aggregated quality indicators, defined by aspiration and reservation levels, that allow knowing which algorithms perform better, within, or worse than the desired limits.

Each of the composite quality indicators proposed enables a different compensation degree among the aggregated indicators, and their joint use permits a deep insight into the algorithms' performance. In addition, we showed that the weak and mixed composite indicators are Pareto-compliant, and the strong one is weakly Pareto-compliant if at least one of the aggregated indicators is Pareto-compliant. Finally, we demonstrated the benefits of our proposal when comparing many population-based algorithms on three-, five-, and eight-objective optimization problems.