Toward High-Quality Real-Time Video Denoising With Pseudo Temporal Fusion Network

Abstract

With the increasing availability of high-resolution video recording and streaming, there is a need for fast and high-quality video denoising methods that can handle high-resolution videos. However, many existing methods fail to achieve high-quality denoising performance and computationally effecient at the same time. This paper proposes a video denoising network, Pseudo Temporal Fusion Network (PTFN), that satisfies these requirements. PTFN adopts a new Pseudo Temporal Fusion (PTF) module that captures pseudo-temporal relationships between video frames in combination with the Temporal Shift Module. PTFN also adopts a modern ConvBlock paradigm that breaks away from the classical ConvBlock paradigm, contributing to denoising performance and computationally effecient. PTFN achieves better performance than existing video denoising methods in terms of both video quality and computational effeciency. Specifically, PTFN has only about 16.7% of the computational cost of existing lightweight methods, while it improves denoising performance. PTFN is also superior in terms of memory consumption. It can process 1080p videos with a GPU with 24 GB RAM. In addition, a lighter version (PTFN Half) can process 2K videos at high speed under the same conditions.

Keywords

• Deep learning
• video denoising
• light weight network
• pseudo temporal fusion
• modern ConvBlock