Hyena neural operator for partial differential equations

Abstract

Numerically solving partial differential equations typically requires fine discretization to resolve necessary spatiotemporal scales, which can be computationally expensive. Recent advances in deep learning have provided a new approach to solving partial differential equations that involves the use of neural operators. Neural operators are neural network architectures that learn mappings between function spaces and have the capability to solve partial differential equations based on data. This study utilizes a novel neural operator called Hyena, which employs a long convolutional filter that is parameterized by a multilayer perceptron. The Hyena operator is an operation that enjoys sub-quadratic complexity and enjoys a global receptive field at the meantime. This mechanism enhances the model’s comprehension of the input’s context and enables data-dependent weight for different partial differential equation instances. To measure how effective the layers are in solving partial differential equations, we conduct experiments on the diffusion–reaction equation and Navier–Stokes equation and compare it with the Fourier neural operator. Our findings indicate that the Hyena neural operator can serve as an efficient and accurate model for learning the partial differential equation solution operator. The data and code used can be found at https://github.com/Saupatil07/Hyena-Neural-Operator.