Real‐time running detection system for UAV imagery based on optical flow and deep convolutional networks

Abstract

A fast‐running human detection system for the unmanned aerial vehicle (UAV) based on optical flow and deep convolution networks is proposed in this study. In the system, running humans can be detected in real‐time at the speed of 15 frames per second (fps) with an 81.1% detection accuracy. To fast locate the candidate targets, optical flow representing the motion information is calculated with every two successive frames. A series of prior‐processing operations, including spatial average filtering, morphological expansion and outer contour extraction, are performed to extract the regions of interest. A classification model based on small‐kernel convolution networks is proposed to achieve the accurate recognition of the running people in various backgrounds. In the model, small convolutional filters are adopted to accelerate the speed of the data representation. Moreover, a total of 60,000 samples are collected to enhance the robustness of the model to adapt to the complex outdoor UAV scenes. The proposed method is compared with other deep learning frameworks for object detection. Field experiments on UAV videos are performed to verify that the proposed system can effectively detect the running people targets in real‐time.

Keywords

• optical flow
• deep convolutional networks
• human detection system
• unmanned aerial vehicle
• deep convolution networks
• 81.1% detection accuracy