Convolutional Neural Network and LSTM Applied to Abnormal Behaviour Detection from Highway Footage

Authors: Rafael Marinho de Andrade, Elcio Hideti Shiguemori, Rafael Duarte Coelho dos Santos

Abstract:

Relying on computer vision, many clever things are possible in order to make the world safer and optimized on resource management, especially considering time and attention as manageable resources, once the modern world is very abundant in cameras from inside our pockets to above our heads while crossing the streets. Thus, automated solutions based on computer vision techniques to detect, react, or even prevent relevant events such as robbery, car crashes and traffic jams can be accomplished and implemented for the sake of both logistical and surveillance improvements. In this paper, we present an approach for vehicles’ abnormal behaviors detection from highway footages, in which the vectorial data of the vehicles’ displacement are extracted directly from surveillance cameras footage through object detection and tracking with a deep convolutional neural network and inserted into a long-short term memory neural network for behavior classification. The results show that the classifications of behaviors are consistent and the same principles may be applied to other trackable objects and scenarios as well.

Keywords: artificial intelligence, behavior detection, computer vision, convolutional neural networks, LSTM, highway footage