Autoencoder-based Anomaly Detection in Streaming Data with Incremental Learning and Concept Drift Adaptation

In our digital universe nowadays, enormous amount of data are produced in a streaming manner in a variety of application areas. These data are usually unlabelled. In this case, identifying infrequent events, such as, anomalies, poses a great challenging. This problem becomes even more challenging when we consider the non-stationary environment (concept drift), which can result in deterioration of the predictive performance of a model. To address the above challenges, we propose a novel approach, which is an autoencoder-based incremental learning method with drift detection (strAEm++DD). Our proposed method strAEm++DD leverages on the advantages of both incremental learning and drift detection. To our knowledge, not only the application of AEs is limited within the online learning framework, but this is one of the very few studies that investigate the combination of incremental learning and concept drift detection for stream learning. We conduct an experimental study using real-world and synthetic datasets with severe or extreme class imbalance, and provide an empirical analysis of strAEm++ and strAEm++DD. We further conduct a comparative study, showing that the proposed method significantly outperforms existing baseline and advanced methods.