MetNet: A Novel Low-Complexity Neural Network-Aided Detection for Faster-Than-Nyquist (FTN) Signaling in ISI Channels

Abstract

This paper studies the application of neural networks to Viterbi detection of FTN signals in an intersymbol interference (ISI) channel. The main contribution of this paper is to propose a receiver structure for detecting FTN signals in unknown static ISI channel. In particular, we propose a novel low-complexity neural network structure for calculating the branch metrics, and we explore its suitability for FTN signalling with channel uncertainty. We compare the proposed network, which we call the Metric Net (MetNet), to a benchmark neural network-based technique for metric calculation, the ViterbiNet, which was originally designed for ISI channels. The simulation results confirm that the MetNet outperforms the ViterbiNet, with two orders of magnitude lower complexity, and is much more resilient to channel uncertainty than the traditional Viterbi detector, which uses Euclidean distance for metric calculations. We further show that the MetNet exhibits robustness to being trained at mismatched SNR values and FTN pulse acceleration factors, meaning that the number of trained models required can be significantly reduced. Additionally, the results show that the proposed MetNet remains a favorable alternative at much higher levels of channel uncertainties. The results also reflect that we can generalize the MetNet to work with different channel models defined by different decaying factors. Finally, we show that we succeed in achieving a bandwidth efficiency gain of 33% due to FTN by using the MetNet in the presence of channel uncertainty.

Keywords

• Faster-than-Nyquist
• maximum likelihood sequence estimation
• AI based signal detection
• spectral efficiency enhancement
• intersymbol interference