Machine learning to detect marine animals in UAV imagery: effect of morphology, spacing, behaviour and habitat

Abstract

Abstract Machine learning algorithms are being increasingly used to process large volumes of wildlife imagery data from unmanned aerial vehicles (UAVs); however, suitable algorithms to monitor multiple species are required to enhance efficiency. Here, we developed a machine learning algorithm using a low‐cost computer. We trained a convolutional neural network and tested its performance in: (1) distinguishing focal organisms of three marine taxa (Australian fur seals, loggerhead sea turtles and Australasian gannets; body size ranges: 0.8–2.5 m, 0.6–1.0 m, and 0.8–0.9 m, respectively); and (2) simultaneously delineating the fine‐scale movement trajectories of multiple sea turtles at a fish cleaning station. For all species, the algorithm performed best at detecting individuals of similar body length, displaying consistent behaviour or occupying uniform habitat (proportion of individuals detected, or recall of 0.94, 0.79 and 0.75 for gannets, seals and turtles, respectively). For gannets, performance was impacted by spacing (huddling pairs with offspring) and behaviour (resting vs. flying shapes, overall precision: 0.74). For seals, accuracy was impacted by morphology (sexual dimorphism and pups), spacing (huddling and creches) and habitat complexity (seal sized boulders) (overall precision: 0.27). For sea turtles, performance was impacted by habitat complexity, position in water column, spacing, behaviour (interacting individuals) and turbidity (overall precision: 0.24); body size variation had no impact. For sea turtle trajectories, locations were estimated with a relative positioning error of <50 cm. In conclusion, we demonstrate that, while the same machine learning algorithm can be used to survey multiple species, no single algorithm captures all components optimally within a given site. We recommend that, rather than attempting to fully automate detection of UAV imagery data, semi‐automation is implemented (i.e. part automated and part manual, as commonly practised for photo‐identification). Approaches to enhance the efficiency of manual detection are required in parallel to the development of effective implementation of machine learning algorithms.

Keywords

• aerial surveys
• demography
• satellite imagery
• deep learning
• artificial intelligence
• movement ecology