Multi-channel neural audio decorrelation using generative adversarial networks

Abstract

Abstract The degree of correlation between the sounds received by the ears significantly influences the spatial perception of a sound image. Audio signal decorrelation is, therefore, a commonly used tool in various spatial audio rendering applications. In this paper, we propose a multi-channel extension of a previously proposed decorrelation method based on generative adversarial networks. A separate generator network is employed for each output channel. All generator networks are optimized jointly to obtain a multi-channel output signal with the desired properties. The training objective includes a number of individual loss terms to control both the input-output and the inter-channel correlation as well as the quality of the individual output channels. The proposed approach is trained on music signals and evaluated both objectively and through formal listening tests. Thereby, a comparison with two classical signal processing-based multi-channel decorrelators is performed. Additionally, the influence of the number of output channels, the individual loss term weightings, and the employed training data on the proposed method’s performance is investigated.

Keywords

• Multi-channel audio decorrelation
• Generative adversarial networks
• Convolutional neural networks