Abstract : Artificial neural network (ANN)-based nonlinear equalizers (NLEs) have been gaining
popularity as powerful waveform equalizers for intensity-modulation (IM)/direct-detection (DD)
systems. On the other hand, the M-ary pulse amplitude modulation (PAM-M) format is now
widely used for high-speed IM/DD systems. For the training of ANN-NLE in PAM-M IM/DD
systems, it is common to employ pseudorandom binary sequences (PRBSs) for the generation of
PAM-M training sequences. However, when the PRBS orders used for training are not sufficiently
high, the ANN-NLE might suffer from the overfitting problem, where the equalizer can estimate
one or more of the constituent PRBSs from a part of PAM-M training sequence, and as a result,
the trained ANN-NLE shows poor performance for new input sequences. In this paper, we
provide a selection guideline of the PRBSs to train the ANN-NLE for PAM-M signals without
experiencing the overfitting. For this purpose, we determine the minimum PRBS orders required
to train the ANN-NLE for a given input size of the equalizer. Our theoretical analysis is confirmed
through computer simulation. The selection guideline is applicable to training the ANN-NLE for
the PAM-M signals, regardless of symbol coding.
