Non-Local Multi-Focus Image Fusion With Recurrent Neural Networks

Abstract

Previous Convolutional Neural Networks (CNNs) based multi-focus image fusion methods rely primarily on local information of images. In this paper, we propose a novel deep network architecture for multi-focus image fusion that is based on a non-local image model. The motivation of this paper stems from local and non-local self-similarity widely shown in nature images. We build on this concept and introduce a recurrent neural network (RNN) that performs non-local processing. The RNN captures global and local information by retrieving long distant dependencies, hence augmenting the representation of each pixel with contextual representations. The augmented representation is beneficial to detect accurately focused and defocused pixels. In addition, we design a regression loss to address the influences of texture information. Experimental results demonstrate that the proposed method outperforms the state-of-the-art methods, both qualitatively and quantitatively.

Keywords

• Multi-focus image fusion
• recurrent neural network
• non-local self-similarity
• texture information