Leveraging Artificial Intelligence for Large-Scale Plant Phenology Studies From Noisy Time-Lapse Images

Abstract

Phenology has become a field of growing importance due to the increasingly apparent impacts of climate change. However, the time-consuming, subjective and tedious nature of traditional human field observations have hindered the development of large-scale phenology networks. Such networks are rare and rely on time-lapse cameras and simplistic color indexes to monitor phenology. To automatize rapid, detailed and repeatable analyzes, we propose an Artificial Intelligence (AI) framework based on machine learning and computer vision techniques. Our approach extracts multiple ecologically-relevant indicators from time-lapse digital photography datasets. The proposed framework consists of three main components: (i) a random forest model to automatically select relevant images based on color information; (ii) a convolutional neural network (CNN) to identify and localize open tree buds; and (iii) a density-based spatial clustering algorithm to cluster open bud detections across the time-series. We tested this framework on a dataset including thousands of black spruce and balsam fir tree images captured using our phenological camera network. The performed experiments showed the efficiency of the proposed approach under challenging perturbation factors, such as significant image noise. Our framework is exceedingly faster and more accurate than human analysts, reducing the time-series processing time from multiple days to under an hour. The proposed methodology is particularly appropriate for large-scale and long-term analyzes of ecological imagery datasets. Our work demonstrates that the use of computer vision and machine learning methods represents a promising direction for the implementation of national, continental, or even global plant phenology networks.

Keywords

• Balsam fir
• black spruce
• computer vision
• convolutional neural network
• deep learning
• forest ecology