Human Action Recognition in Low-Resolution Videos Based on Spatio-Temporal Features

Abstract

In recent years, the field of computer vision and pattern recognition has devoted a lot of research focus to the study of human action. Most of the current action recognition research focuses on high-quality videos with clearly apparent actions. Most actions are of limited quality and take place at a distance, making it difficult to identify them. Therefore, the issue of low video quality is still under-researched and difficult to solve in practical implementation. The goal of this study is to establish a deep learning based framework for recognizing human actions in low-quality video by utilizing spatial-temporal features. This research focuses on the use of annotated dataset TinyVIRAT-v2 of action recognition videos recorded in low quality in order to examine the applicability of deep learning architecture for action recognition from videos of low perceptual quality. First, in the proposed framework datasets will be pre-processed and spatial-temporal features will be retrieved using a CNN-based feature extractor. The extracted features will then be fed into a 3D custom model for classification, which is based on resnet50 as backbone network and c3d with a SoftMax layer for multiclass prediction, allowing for its implementation as a real-world practical application. Upon classifying, every testing video is allocated a predicted category, and the cumulative outcomes are evaluated using performance measure F1-Score which is 0.68.