Poster: Towards Reinforcement Learning for Vulnerability Detection in Power Systems and Markets

Future smart grids can and will be subject of systematic attacks that can result in monetary costs and reduced system stability. These attacks are not necessarily malicious, but can be economically motivated as well. Emerging flexibility markets are of interest here, because they can incite attacks. Since dimension and danger potential of such strategies are still uncertain, analysis tools are required to systematically search for unknown strategies and their respective countermeasures. We propose deep reinforcement learning to learn attack strategies autonomously to identify underlying systemic vulnerabilities this way. Exemplarily, we apply our approach to a reactive power market setting in a distribution grid. In our case study, the attacker learned to exploit the reactive power market by using controllable loads to induce constraint violations into the system and then providing remunerated countermeasures, thus finding a previously unknown economically motivated attack strategy.