Effectively Detecting Operational Anomalies in Large-scale IoT Data Infrastructures by using a GAN-based Predictive Model

Quality of data services is crucial for operational large-scale internet-of-things

(IoT) research data infrastructure, in particular when serving large amounts of

distributed users. Eectively detecting runtime anomalies and diagnosing their

root cause helps to defend against adversarial attacks, thereby essentially boosting

system security and robustness of the IoT infrastructure services. However,

conventional anomaly detection methods are inadequate when facing the dynamic

complexities of these systems. In contrast, supervised machine learning methods

are unable to exploit large amounts of data due to the unavailability of labeled

data. This paper leverages popular GAN-based generative models and end-to-

end one-class classication to improve unsupervised anomaly detection. A novel

heterogeneous BiGAN-based anomaly detection model Heterogeneous Temporal

Anomaly-reconstruction GAN (HTA-GAN) is proposed to make better use of a

one-class classier and a novel anomaly scoring function. The Generator-Encoder-

Discriminator BiGAN structure can lead to practical anomaly score computation

and temporal feature capturing. We empirically compare the proposed approach

with several state-of-the-art anomaly detection methods on real-world datasets,

anomaly benchmarks, and synthetic datasets. The results show that HTA-GAN

outperforms its competitors and demonstrates better robustness.