Multi-Channel LSTM-Capsule Autoencoder Network for Anomaly Detection on Multivariate Data

Abstract

Copyright © 2022 by the authors. Deep learning techniques have recently shown promise in the field of anomaly detection, providing a flexible and effective method of modelling systems in comparison to traditional statistical modelling and signal processing-based methods. However, there are a few issues that Neural Networks (NN)s face, such as generalisation ability, requiring large volumes of labelled data to train effectively, and understanding spatial context in data. This paper introduces a novel NN architecture to tackle these problems, which utilises a Long-Short-Term-Memory (LSTM) encoder and Capsule decoder in a multi-channel input Autoencoder architecture for use on multivariate time series data. Experimental results show that using Capsule decoders increases the resilience of the model to overfitting and improves training efficiency, which is shown by the improvement of Mean Squared Error (MSE) on unseen data from an average of 10.61 to 2.08 for single channel architectures, and 10.08 to 2.05 for multi-channel architectures. Additionally, results also show that the proposed model can learn multivariate data more consistently, and was not affected by outliers in the training data. The proposed architecture was also tested on an open-source benchmark, where it achieved state-of-the-art performance in outlier detection, and performs best overall with a total accuracy of 0.494 over the metrics tested.