Equalization Basics

When a digital signal travels through a real communication channel, it experiences frequency-dependent distortion that causes ISI at the receiver. An equaliser is a signal processing filter placed at the receiver whose purpose is to compensate for this channel distortion and restore the signal to a form that allows reliable symbol detection. Equalization is essential in high-speed wireline links, mobile communications, and any system where channel distortion would otherwise make detection unreliable.

Why Equalization Is Necessary

The received signal in a baseband system is the convolution of the transmitted signal with the channel impulse response. This convolution spreads energy across time, causing ISI. To undo this spreading, the receiver needs to apply a filter whose transfer function counteracts the channel's distortion. This inverse filtering operation is called equalization.

The challenge is that perfect channel inversion works well only when the channel has a relatively flat frequency response. Real channels often have spectral nulls or deep notches where the channel gain approaches zero. Inverting such a channel would amplify noise at those frequencies to extremely high levels, making the equalized output worse than the uncorrected signal. This fundamental trade-off between ISI removal and noise enhancement is the central problem in equalizer design.

Zero-Forcing Equalizer

The zero-forcing (ZF) equalizer completely inverts the channel by setting its transfer function W_ZF(f) = 1/H_c(f), where H_c(f) is the channel frequency response. The cascade of channel and equalizer then has a flat frequency response, eliminating all ISI. At the sampling instants, the ZF equalizer forces ISI to be exactly zero.

The drawback of the ZF equalizer is severe noise enhancement. At frequencies where |H_c(f)| is small (deep fades), 1/H_c(f) becomes very large, amplifying noise at those frequencies. In channels with spectral nulls, the noise power at the ZF equalizer output can be theoretically infinite, making the SNR very poor despite zero ISI. ZF equalization is therefore useful only in high SNR environments or channels with mild frequency selectivity.

Minimum Mean Square Error Equalizer

The minimum mean square error (MMSE) equalizer takes a different approach. Instead of forcing ISI to exactly zero, it minimises the total mean squared error between the equalizer output and the desired signal. The MMSE criterion balances ISI reduction and noise enhancement, resulting in a filter that accepts some residual ISI in exchange for not amplifying noise excessively.

The MMSE equalizer transfer function is W_MMSE(f) = H_c*(f) / (|H_c(f)|^2 + N0/Es), where N0 is the noise PSD and Es is the symbol energy. The noise regularisation term N0/Es prevents the denominator from becoming zero at spectral nulls. At high SNR where N0/Es is small, W_MMSE approaches the ZF equalizer. At low SNR where N0/Es is large, W_MMSE approaches the matched filter. MMSE always achieves better BER than ZF equalizer because it properly accounts for noise.

Adaptive Equalization

In wireless communications, the channel is time-varying due to user mobility and multipath changes. A fixed equalizer designed for one channel state will be suboptimal when the channel changes. Adaptive equalization solves this by continuously updating the equalizer coefficients to track the changing channel.

The most common adaptive algorithm is the least mean squares (LMS) algorithm. The LMS algorithm computes the error between the equalizer output and a reference (either a known training symbol or a decision), then adjusts the filter taps in the direction that reduces this error. The step size parameter controls the speed-accuracy tradeoff: a large step size converges fast but has more residual noise in steady state, while a small step size is more accurate but converges slowly.

Adaptive equalizers operate in two phases. During the training phase, a known sequence of symbols called a training sequence or pilot is transmitted. The receiver uses this known sequence as the reference to train the equalizer. After convergence, the system switches to decision-directed mode where the equalizer's own output decisions are used as the reference to continue tracking slow channel variations.

Given:
Channel frequency response magnitude: |H_c(f)| = 0.2 at frequency f1 (deep fade)
Signal energy Es = 1 (normalised)
Noise PSD N0 = 0.01

Why this formula applies:
Comparing ZF and MMSE noise amplification at the faded frequency.
ZF gain = 1/|H_c(f)|, MMSE gain = |H_c(f)| / (|H_c(f)|^2 + N0/Es)

Formula:
ZF equalizer gain at f1:   G_ZF = 1 / |H_c(f1)|
MMSE equalizer gain at f1: G_MMSE = |H_c(f1)| / (|H_c(f1)|^2 + N0/Es)

Substitution:
G_ZF = 1 / 0.2 = 5
G_MMSE = 0.2 / (0.04 + 0.01) = 0.2 / 0.05 = 4

Calculation:
ZF amplifies noise by factor 5 at this frequency.
MMSE amplifies by factor 4, limiting noise enhancement.

Final Answer:
ZF gain = 5, MMSE gain = 4 at the faded frequency.
MMSE reduces noise amplification by 20% compared to ZF at this spectral null.
For deeper fades, this difference becomes dramatically larger.

Exam Tip: GATE questions on equalization often ask which equalizer type completely removes ISI (ZF) versus which minimises BER (MMSE). Remember that at high SNR, MMSE approaches ZF. At low SNR, MMSE approaches matched filter. Never confuse zero-forcing with maximum likelihood detection.

• ZF equalizer completely removes ISI by inverting channel: W_ZF(f) = 1/H_c(f). Fails at spectral nulls due to massive noise amplification.
• MMSE equalizer minimises mean square error: W_MMSE(f) = H_c*(f) / (|H_c|^2 + N0/Es). Balances ISI and noise.
• At high SNR, MMSE approaches ZF behaviour. At low SNR, MMSE approaches matched filter behaviour.
• Adaptive equalizers use LMS or RLS algorithms to track time-varying channels using training sequences and then decision-directed mode.
• LMS algorithm: update taps by subtracting step-size times error times input. Simple but sensitive to step size choice.

Quick Revision

• Equalizer purpose: compensate for channel distortion (ISI) at the receiver by applying an inverse filter.
• ZF equalizer: W(f) = 1/H_c(f). Removes all ISI. Severely amplifies noise at spectral nulls.
• MMSE equalizer: W(f) = H_c*(f)/(|H_c|^2 + N0/Es). Minimises total error. Better BER than ZF in practice.
• MMSE limit at high SNR = ZF. MMSE limit at low SNR = matched filter. Always remember these two boundary cases.
• Adaptive equalizer: uses LMS algorithm to update filter taps in real time. Required for time-varying wireless channels.
• GATE trap: ZF equalizer does NOT maximise SNR; it only removes ISI. MMSE is the balanced optimal choice.
• Training phase uses known pilot symbols for initial convergence; decision-directed mode maintains tracking afterward.