AVID: Adversarial Visual Irregularity Detection (original) (raw)

An Approach to Detect Anomaly in Video Using Deep Generative Network

IEEE Access, 2021

Anomaly detection in the video has recently gained attention due to its importance in the intelligent surveillance system. Even though the performance of the state-of-art methods has been competitive in the benchmark dataset, the trade-off between the computational resource and the accuracy of the anomaly detection should be considered. In this paper, we present a framework to detect anomalies in video. We proposed a ''multi-scale U-Net'' network architecture, the unsupervised learning for anomaly detection in video based on generative adversarial network (GAN) structure. Shortcut Inception Modules (SIMs) and residual skip connection are employed to the generator network to increase the ability of the training and testing of the neural network. An asymmetric convolution has been applied instead of traditional convolution layers to decrease the number of training parameters without performance penalty in terms of detection accuracy. In the training phase, the generator network was trained to generate the normal events and attempt to make the generated image and the ground truth to be similar. A multi-scale U-Net kept useful features of an image that were lost during training caused by the convolution operator. The generator network is trained by minimizing the reconstruction error on the normal data and then using the reconstruction error as an indicator of anomalies in the testing phase. Our proposed framework has been evaluated on three benchmark datasets, including UCSD pedestrian, CHUK Avenue, and ShanghaiTech. As a result, the proposed framework surpasses the state-of-the-art learning-based methods on all these datasets, which achieved 95.7%, 86.9%, and 73.0% in terms of AUC. Moreover, the numbers of training and testing parameters in our framework are reduced compared to the baseline network architecture, while the detection accuracy is still improved.

Enhanced Adversarial Learning Based Video Anomaly Detection with Object Confidence and Position

2019 13th International Conference on Signal Processing and Communication Systems (ICSPCS)

Video anomaly detection is to identify the abnormal objects, positions and behaviours during the video sequences. It is an important but challenging problem in intelligent video surveillance. Nowadays, there is much concern about the generative adversarial networks (GAN) to detect anomalies which contains two parts: generator and discriminator. However, the two networks of this model are hard to train well at the same time in practical use. In this paper, we propose to exploit object detection to enhance the adversarial learning model and to improve classification method to distinguish anomalies in a semi-supervised manner. We also detect object position anomaly in our proposed model which can not be done in generative adversarial learning models separately. The proposed framework is evaluated on dataset UCSD Ped1 and Ped2 using two criteria: area under the curve (AUC) and equal error rate (EER). The results confirm that our proposed method can effectively improve object variety anomaly performance and detect object position anomaly and is also superior to the baseline. Our approach also achieves improved performance compared with recent state-ofthe-art methods.

Reconstruction by inpainting for visual anomaly detection

Pattern Recognition, 2021

Visual anomaly detection addresses the problem of classification or localization of regions in an image that deviate from their normal appearance. A popular approach trains an auto-encoder on anomaly-free images and performs anomaly detection by calculating the difference between the input and the reconstructed image. This approach assumes that the auto-encoder will be unable to accurately reconstruct anomalous regions. But in practice neural networks generalize well even to anomalies and reconstruct them sufficiently well, thus reducing the detection capabilities. Accurate reconstruction is far less likely if the anomaly pixels were not visible to the auto-encoder. We thus cast anomaly detection as a self-supervised reconstruction-by-inpainting problem. Our approach (RIAD) randomly removes partial image regions and reconstructs the image from partial inpaintings, thus addressing the drawbacks of auto-enocoding methods. RIAD is extensively evaluated on several benchmarks and sets a new state-of-the art on a recent highly challenging anomaly detection benchmark.

Adversarially Learned Anomaly Detection

2018 IEEE International Conference on Data Mining (ICDM), 2018

Anomaly detection is a significant and hence wellstudied problem. However, developing effective anomaly detection methods for complex and high-dimensional data remains a challenge. As Generative Adversarial Networks (GANs) are able to model the complex high-dimensional distributions of real-world data, they offer a promising approach to address this challenge. In this work, we propose an anomaly detection method, Adversarially Learned Anomaly Detection (ALAD) based on bi-directional GANs, that derives adversarially learned features for the anomaly detection task. ALAD then uses reconstruction errors based on these adversarially learned features to determine if a data sample is anomalous. ALAD builds on recent advances to ensure data-space and latent-space cycle-consistencies and stabilize GAN training, which results in significantly improved anomaly detection performance. ALAD achieves state-of-the-art performance on a range of image and tabular datasets while being several hundred-fold faster at test time than the only published GAN-based method.

Interpreting Abnormality of a Complex Static Scene using Generative Adversarial Network

2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)

Anomaly detection remains a difficult task in the computer vision and image processing field. Although several studies have been done to address this challenge, most of these studies focused on analyzing temporal features to determine abnormality. Examples of temporal features include behavioral changes and new object appearance in the target scene. In this paper, we are interpreting abnormality from a new perspective, which is static and complex image scene that focused on one object (airplane) using generative adversarial networks (GANs). Our interpretation of abnormality in such image intended to test two research hypotheses: 1) whether GANs can capture the cognitive features of abnormality from within a complex scene. 2) whether GANs can be used to generate more reliable datasets of abnormal scenes. In this work, we chose an airplane as the object of our experiment. We defined abnormal and normal scenes as follow: The scene is abnormal if the airplane involved in accidents (such as crash or fire), and normal otherwise (such as flying or landed airplane). A custom dataset is built for this experiment and it consists of two classes; normal and abnormal. We augmented each class to the double of its size using GANs, and then we created three different sets of datasets: (DS1, DS2, and DS3) to test our hypotheses. We applied four different supervised machine learning classifiers on each of these three sets, we repeated this step 3 times as follow: 1) pixel-based, 2) with applying Principal Component Analysis (PCA), and 3) with applying Local Binary Pattern (LBP). The overall results showed that GANs possess the capability of generating images that capture the abnormality features from the static complex scene.

Unsupervised Adversarial Learning of Anomaly Detection in the Wild

2020

Unsupervised learning of anomaly detection in highdimensional data, such as images, is a challenging problem recently subject to intense research. Through careful modelling of the data distribution of normal samples, it is possible to detect deviant samples, so called anomalies. Generative Adversarial Networks (GANs) can model the highly complex, high-dimensional data distribution of normal image samples, and have shown to be a suitable approach to the problem. Previously published GAN-based anomaly detection methods often assume that anomaly-free data is available for training. However, this assumption is not valid in most real-life scenarios, a.k.a. in the wild. In this work, we evaluate the effects of anomaly contaminations in the training data on state-of-the-art GAN-based anomaly detection methods. As expected, detection performance deteriorates. To address this performance drop, we propose to add an additional encoder network already at training time and show that joint genera...

SVD-GAN for Real-Time Unsupervised Video Anomaly Detection

Proceedings of the British Machine Vision Conference (BMVC 2021), 2021

Real-time unsupervised anomaly detection from videos is challenging due to the uncertainty in occurrence and definition of abnormal events. To overcome this ambiguity, an unsupervised adversarial learning model is proposed to detect such unusual events. The proposed end-to-end system is based on a Generative Adversarial Network (GAN) architecture with spatiotemporal feature learning and a new Singular Value Decomposition (SVD) loss function for robust reconstruction and video anomaly detection. The loss employs efficient low-rank approximations of the matrices involved to drive the convergence of the model. During training, the model strives to learn the relevant normal data distribution. Anomalies are then detected as frames whose reconstruction error, based on such distribution, shows a significant deviation. The model is efficient and lightweight due to our adoption of depth-wise separable convolution. The complete system is validated upon several benchmark datasets and proven to be robust for complex video anomaly detection, in terms of both AUC and Equal Error Rate (EER).

CutPaste: Self-Supervised Learning for Anomaly Detection and Localization

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021

We aim at constructing a high performance model for defect detection that detects unknown anomalous patterns of an image without anomalous data. To this end, we propose a two-stage framework for building anomaly detectors using normal training data only. We first learn self-supervised deep representations and then build a generative one-class classifier on learned representations. We learn representations by classifying normal data from the CutPaste, a simple data augmentation strategy that cuts an image patch and pastes at a random location of a large image. Our empirical study on MVTec anomaly detection dataset demonstrates the proposed algorithm is general to be able to detect various types of real-world defects. We bring the improvement upon previous arts by 3.1 AUCs when learning representations from scratch. By transfer learning on pretrained representations on ImageNet, we achieve a new state-of-theart 96.6 AUC. Lastly, we extend the framework to learn and extract representations from patches to allow localizing defective areas without annotations during training.

Deep-anomaly: Fully convolutional neural network for fast anomaly detection in crowded scenes

Computer Vision and Image Understanding, 2018

The detection of abnormal behaviours in crowded scenes has to deal with many challenges. This paper presents an efficient method for detection and localization of anomalies in videos. Using fully convolutional neural networks (FCNs) and temporal data, a pre-trained supervised FCN is transferred into an unsupervised FCN ensuring the detection of (global) anomalies in scenes. High performance in terms of speed and accuracy is achieved by investigating the cascaded detection as a result of reducing computation complexities. This FCN-based architecture addresses two main tasks, feature representation and cascaded outlier detection. Experimental results on two benchmarks suggest that detection and localization of the proposed method outperforms existing methods in terms of accuracy.

Counterfeit Anomaly Using Generative Adversarial Network for Anomaly Detection

IEEE Access, 2020

Anomaly detection aims to detect anomaly with only normal data available for training. It attracts considerable attentions in the medical domain, as normal data is relatively easy to obtain but it is rather difficult to have abnormal data especially for some rare diseases, making training a standard classifier challenging or even impossible. Recently, generative adversarial networks (GANs) become prevalent for anomaly detection and most existing GAN-based methods detect outliers by the reconstruction error. In this paper, we propose a novel framework called adGAN for anomaly detection using GAN. Unlike existing GAN-based methods, adGAN is a discriminative model, which uses the fake data generated from GAN as an abnormal class, and then learns a boundary between normal data and simulated abnormal data. Thus it is able to output the anomaly scores directly similar as one-class SVM (OCSVM), without any reconstruction process. We explicitly design adGAN with two key elements, i.e., fake pool generation and concentration loss. The fake pool is created by incrementally collecting the fake data produced by intermediate-state GAN, which are likely surrounding the normal data distribution. The concentration loss is innovatively introduced to penalize large standard deviations of discriminator outputs for normal data, aiming to make the distribution of normal data more compact and more likely to be separated from the distribution of the potential abnormal data. The trained discriminator is finally used as an anomaly detector. We evaluated adGAN on three datasets, including ab-MNIST for synthetic anomaly detection, the ISIC'2016 for skin lesion detection, and the BraTS'2017 for brain lesion detection. The extensive experiments demonstrate that adGAN is consistently superior to its competitors on all three datasets. INDEX TERMS Anomaly detection, concentration loss, fake pool, GAN.