Quality-Diversity Optimization: a novel branch of stochastic optimization

Traditional optimization algorithms search for a single global optimum
that maximizes (or minimizes) the objective function. Multimodal optimization algorithms
search for the highest peaks in the search space that can be more than one.
Quality-Diversity algorithms are a recent addition to the evolutionary computation
toolbox that do not only search for a single set of local optima, but instead try to illuminate
the search space. In effect, they provide a holistic view of how high-performing
solutions are distributed throughout a search space. The main differences with multimodal
optimization algorithms are that (1) Quality-Diversity typically works in the
behavioral space (or feature space), and not in the genotypic (or parameter) space,
and (2) Quality-Diversity attempts to fill the whole behavior space, even if the niche
is not a peak in the fitness landscape. In this chapter, we provide a gentle introduction
to Quality-Diversity optimization, discuss the main representative algorithms, and
the main current topics under consideration in the community. Throughout the chapter,
we also discuss several successful applications of Quality-Diversity algorithms,
including deep learning, robotics, and reinforcement learning.