Learning-Based Illuminant Estimation Model With a Persistent Memory Residual Network (PMRN) Architecture

Abstract

Since AlexNet, large deep convolutional neural networks (DCNNs) have been one of the major topics of interest in the field of computer vision, as well as bringing remarkable progress to the field. However, there has been little effort to use the DCNNs in realizing the mechanism of human memory. The human memory can be classified into three types: sensory memory, short-term memory and long-term memory. The short-term memory, also known as primary memory or active memory, is the information that humans are presently perceiving or thinking about, whereas the long-term memory refers to the persistent storage of information. In the mechanism of the human brain, the long-term memory enables the human vision to identify the actual color of an object effortlessly. In the computer vision, the DCNN-based illuminant estimation models are facing the long-term dependency problem as deeper networks encounter widening gaps between their earlier layers and later layers. Therefore, it is highly inspiring to apply the human long-term memory to the DCNN-based illuminant estimation models. The natural motivation of this article is to present a novel persistent memory residual network (PMRN) model which provides the DCNN with explicit access to persistent memory. The proposed PMRN architecture has two distinct units: a recursive unit and a gate unit. The two units combined serve to facilitate persistent memory access in a non-recursive fashion. The recursive unit has four residual blocks which are trained on the multiple-level image features on diverse receptive fields. The residual block outputs are concatenated and then fed into the gate unit. The proposed architecture keeps track of the recursive unit, deciding on how many of the earlier blocks to keep in reserve and how much of the image features to let the present block store. In this way, the proposed architecture contributes to solving the long-term dependency problem of conventional DCNNs. Comprehensive experiments support unparalleled performance of the proposed architecture in comparison to its counterparts and its potential to meet the needs of illumination estimation applications.

Keywords

• Human memory
• long-term dependency problem
• short-term memory
• long-term memory
• illumination estimation
• persistent memory residual network (PMRN)