Spatio-temporal pedestrian detection in video: comparative evaluation of VGG16 with recurrent neural networks

Abstract

Pedestrian detection is a crucial application in video surveillance, autonomous driving, and traffic monitoring. Thus, reliable surveillance is required for individual decision making and safety. The study aims to compare two models, one based on VGG16 for feature extraction, coupled with a long short-term memory (LSTM), and the other simply a dense model, for pedestrian detection in video. The integration of an attention mechanism to improve feature discrimination across frames along with a lightweight structure for real-time processing that enables cross-domain generalization to diverse datasets is novelty of this work. We exploit the pre-trained VGG16 model on ImageNet, extracting spatial features from all the frames of the videos. We then feed these spatial features through an LSTM to capture temporal dependencies. The dense model uses just the spatial features and throws into the bin of information the time holds for them. We apply accuracy, precision, recall, and specificity as metrics in evaluation models on a labeled dataset of pedestrian video clips. Experimental results show that the VGG+LSTM model performs better than the dense model by giving a higher accuracy and performing better on temporal variations of frames. The LSTM-based approach achieves 0.96 accuracy over multivariate datasets.