On the Design of Channel Shortening Demodulators for Iterative Receivers in Linear Vector Channels

Abstract

We consider designing demodulators for linear vector channels that use reduced-size trellis descriptions for the received signal. We assume an iterative receiver and use interference cancellation (IC) with soft-information provided by an outer decoder to mitigate the signal part that is not covered by a reduced-size trellis description. In order to reach a trellis description, a linear filter is applied as a front end to compress the signal structure into a small trellis. This process requires three parameters to be designed: (1) the front-end filter; (2) the feedback filter through which the IC is done; and (3) a target response which specifies the trellis. Demodulators of this form have been studied before under the name channel shortening (CS), but the interplay between CS, IC, and the trellis-search processes has not been adequately addressed in the literature. In this paper, we analyze two types of CS demodulators that are based on the Forney and Ungerboeck detection models, respectively. The parameters are jointly optimized with a generalized mutual information (GMI) function. We also introduce a third type of CS demodulator that is, in general, suboptimal, which has closed-form solutions. Furthermore, signal-to-noise ratio asymptotic properties are analyzed, and we show that the third CS demodulator asymptotically converges to the optimal CS demodulator in the sense of GMI maximization.

Keywords

• Channel shortening (CS)
• intersymbol interference (ISI)
• multi-input multi-output (MIMO)
• front-end filter
• feedback filter
• target response