Biodiversity in NLP: modelling lexical meaning with the Fruit Fly Algorithm

Abstract

The natural world is very diverse in terms of biological organisation, and solves problems in a wide variety of efficient and creative manners. This biodiversity is in stark contrast with the landscape of artificial models in the field of Natural Language Processing (NLP). In the last years, NLP algorithms have clustered around a few very expensive architectures, the cost of which has many facets, including training times, storage, replicability, interpretability, equality of access to experimental paradigms, and even environmental impact. Inspired by the biodiversity of the real world, we argue for a methodology which promotes ‘artificial diversity’, and we further propose that cognitively-inspired algorithms are a good starting point to explore new architectures. As a case study, we investigate the fruit fly’s olfactory system as a distributional semantics model. We show that, even in its rawest form, it provides many of the features that we might require from a good model of meaning acquisition, and that the original architecture can serve as a basis for cognitively-inspired extensions. We focus on one such extension by implementing a mechanism of neural adaptation.