ODIN TS: a tool for the black-box evaluation of time series analytics

Abstract. The increasing availability of time series datasets enabled by the diffusion of IoT architectures and the progress in the analysis of temporal data fostered by Deep Learning methods are boosting the interest in anomaly detection and predictive maintenance applications. 
The analysis of performance for these tasks relies on standard metrics applied to the entire dataset. Such indicators provide a global performance assessment but might not help a deep understanding of the model weaknesses.
A complementary diagnostic approach exploits error categorization and ad-hoc visualizations. In this paper we present ODIN, an open source diagnosis framework for time series analysis that lets developers compute performance metrics, disaggregated by different criteria, and visualize diagnosis reports. ODIN is agnostic to the training platform and can be extended with application- and domain-specific meta-annotations and metrics with almost no coding. We show ODIN at work through two time series analytics examples.