Anomaly Detection in Videos Using Two-Stream Autoencoder with Post Hoc Interpretability

Figure 1

The pipeline of our method. First, we need to decompose the video of datasets into continuous frames. Then we use the convolutional two-stream network to learn the feature of image frames. Finally, the video frame’s regularity score is calculated to decide this frame is normal or abnormal. From the initial image to the heatmap, we can see that the model is more interested in areas where abnormal events occur, such as the patrol car that appears here.