An Mcformer encoder integrating Mamba and Cgmlp for improved acoustic feature extraction

Abstract

Abstract Currently, attention models based on the Conformer architecture have become mainstream in the field of speech recognition due to their integration of self-attention mechanisms and convolutional networks. However, further research indicates that Conformers still exhibit limitations in capturing global information. To address this limitation, the Mcformer encoder is introduced, which incorporates the Mamba module in parallel with multi-head attention blocks to enhance the model’s global context processing capabilities. Additionally, a Convolutional Gated Multilayer Perceptron (Cgmlp) structure is employed to improve the extraction of local features through deep convolutional layers. Experimental results on the Aishell-1, Common Voice zh 14 public datasets, and the TED-LIUM 3 English public dataset demonstrate that, without a language model, the Mcformer encoder achieves character error rates (CER) of 4.15%, 4.48%, and 13.28%, 13.06% on the validation and test sets of Aishell-1 and Common Voice zh 14, respectively. When incorporating a language model, the CER further decreases to 3.88%, 4.08%, and 11.89%, 11.29%. On the TED-LIUM 3 English public dataset, the word error rates (WER) for the validation and test sets are 7.26% and 6.95%, respectively, without a language model. These experimental outcomes substantiate the efficacy of Mcformer in enhancing speech recognition performance.

Keywords

• Speech recognition
• Deep learning
• Mamba
• Feature fusion