Method for Computing Dense Optical Flow on FPGA in Real Time

Abstract

One of the most actual problems in the technical vision systems is the problem of objects detection and selection in the field of the image sensor view. Applications of objects automatic detection and object allocation technologies include search and rescue tasks, space monitoring of the Earth, traffic analysis, air traffic control, etc. Objects can be stationary and moving. The movement in the video can be formalized in a term such as optical flow. Optical flow is the basic tool for processing and analyzing video sequences, and its calculation is the most common approach to the detection of moving objects. The article describes the features of the hardware implementation of the FPGA method for calculating the dense optical flow on high-definition video. The developed method is based on the correlation-extreme algorithm for finding the minimum sum of absolute differences in the brightness of the pixels of the windows in the vicinity of the current point of neighboring frames of the video. A detailed description of the main stages of this algorithm is given. The principle of parallelization and pipelization, which allows us to minimize the amount of calculations due to the reuse of intermediate results obtained in the previous steps, is considered. The described approach allows us to calculate the optical flow for each pixel at the rate of input video. The hardware implementation cost of the method on the Xilinx Zynq Ultra Scale + FPGA, which is part of the Xilinx Zynq Ultra Scale + MPSoC ZCU102 Evaluation Kit, is considered.

Keywords

• optical flow
• fpga
• criterion function
• motion detection
• absolute difference