Drift detection on feature attributions for monitoring visual reinforcement learning models in maritime port surveillance

This article shows how it is possible to monitor an AI model interpretation of the data it is being fed, rather than monitoring only the data itself, to improve our capability to understand how an AI model is behaving while it works. This allows us to detect changes in the way it is behaving, as well as understand these changes better, in order to detect more quickly when the model is not behaving correctly and fix it. In this article, we apply this approach to a maritime port surveillance environment, testing it under different simulated scenarios, such as overabundance of obstacles in the port or malicious attacks on the vessels, to see if the method meets its potential. We conclude that it does, with certain caveats such as the need of stronger hardware, resulting in a good balance between detection of irregular AI model behavior and interpretability of said behavior.