Anomaly Prediction and Analysis Using Deep Learning-Based Techniques

Abstract

In today's world, where public safety is a growing concern, intelligent surveillance systems play a crucial role in identifying abnormal behavior in public places. To build an effective surveillance system, early prediction of abnormal actions is crucial as it can help avoid potentially dangerous situations before they occur. This thesis focused on the development of a deep learning-based anomaly prediction model that can predict future abnormal actions based on partially observed information of actions from surveillance videos. These videos are often captured under real-world environmental conditions such as poor lighting, occlusions, dynamic backgrounds, visual noise, etc., and only partial information of an action is considered for prediction, which makes the task of forecasting more challenging. Although previous studies have shown progress in anomaly recognition, which focuses on recognizing abnormal action only after it occurs completely, rather than predicting future action. In addition, many existing methods rely on spatial information and the generation of image-based next anomaly frames, usually ignoring the importance of temporal information. This limits its effectiveness in real-world scenarios in which understanding the action sequence and progress of actions over time is crucial. To address these limitations, this research proposed a novel deep learning-based hybrid approach that combines spatial and temporal information. The proposed model uses a pre-trained ResNet50 to extract spatial features and BiLSTM with an attention mechanism to learn temporal dependencies throughout the video. The final classification is performed using a fully connected layer with softmax activation function. Extensive experiments are carried out on the real-world challenging UCF-Crime dataset, where the model is evaluated at various observation ratios, such as 20%, 40%, 60%, and 80% of the video, to simulate early prediction scenarios. The results show that the proposed model achieved 81.4% accuracy, significantly outperforming baseline predictive models, even when only a limited portion of the action was observed, confirming their effectiveness in the early prediction of anomalies. In addition, the model achieved 90.3% accuracy when tested on complete video (100% video observation ratio), acting as a conventional anomalous action recognition. Although the anomaly prediction task achieved lower accuracy than anomaly recognition due to limited information available at early stages, the proposed model can make accurate future predictions much earlier. This makes it suitable for time-critical surveillance scenarios, where early predictions of crimes are more valuable. This study contributes to the advancement of proactive surveillance systems, allowing timely responses to anomalous events before they occur