Advances in Automated Pigment Mapping for 15th-Century Manuscript Illuminations Using 1-D Convolutional Neural Networks and Hyperspectral Reflectance Image Cubes

Abstract

Reflectance imaging spectroscopy (RIS) is invaluable in mapping and identifying artists’ materials in paintings. The analysis of the RIS image cube first involves classifying the cube into spatial regions, each having a unique reflectance spectrum (endmember). Second, endmember spectra are analyzed for spectral features useful to identify the pigments present to create labeled classes. The analysis process for paintings remains semi-automated because of the complex diffuse reflectance spectra due to the use of intimate pigment mixtures and optically thin paint layers by the artist. As a result, even when a group of related paintings are analyzed, each RIS cube is analyzed individually, which is time consuming. There is a need for new approaches to more efficiently analyze RIS cubes of related paintings to address the growing interest in the study of related paintings within a group of artists or artistic schools. This work builds upon prior investigations of 1-D spectral convolutional neural networks (CNNs) to address this need in two ways. First, an expanded training set was used—ten illuminated manuscripts created by artists stylistically grouped under the notname “Master of the Cypresses” (15th century Seville, Spain). Second, two 1-D CNN models were trained from the RIS cubes: reflectance and the first derivative. The results showed that the first derivative-trained CNN generally performed better than the reflectance-trained CNN in creating accurate labeled material maps for these illuminated manuscripts.

Keywords

• reflectance imaging spectroscopy
• neural networks
• pigment mapping
• illuminated manuscripts