Imitation Learning with Bayesian Exploration (IL-BE) for Signal Integrity (SI) of PAM-4 based High-speed Serial Link: PCIe 6.0 (DesignCon 2022)
Authors: Jihun Kim, Minsu Kim, Hyunwook Park, Jiwon Yoon, Seonguk Choi, Joonsang Park, Haeyeon Kim, Keeyoung Son, Seongguk Kim, Daehwan Lho, Keunwoo Kim, Jinwook Song, Kyungsuk Kim, Jongkyu Park and Joungho Kim
Abstract: This paper proposes a novel imitation learning with Bayesian exploration (IL-BE) method to optimize via parameters of any given channel parameters for signal integrity (SI) of PAM-4 based high-speed serial link on PCIe 6.0. PCIe 6.0. is a crucial interconnect link for highspeed communication of processors, and PAM-4 signaling is a major component of PCIe 6.0 that can double bandwidth. However, the design space of PAM-4 based PCIe 6.0 is extremely complex. Moreover, because PAM-4 signaling reduces eye-margin 1/3 compared with NRZ signaling, it is more sensitive to optimize. Bayesian optimization (BO) is a candidate method because it shows powerful searching abilities on black-box continuous optimization space. However, BO has a significant limitation to apply via optimization for any given channel parameters because BO needs massive iterations to solve each problem (i.e., no adaptation to new tasks). Deep reinforcement learning is a promising method that deep neural network (DNN) agents learn to capture meta-features among problems interacting with the real-world environment. Therefore, learned DNN agents can adapt to a new problem by optimization with small iterations. However, DNN agents must learn through massive iterative trial and error; it is extremely complex to train DNN. We blend the benefit of the BO and DRL method. Firstly, we collect high-quality expert data using BO rather than relying on poor exploration of the initial DNN agent. Then we use the collected high-quality data to train DNN agents by using an imitation learning scheme. For verification, we target one pair differential PCIe 6.0 (64Gbps) interconnection of SSD board and three-layer transition; a task is formulated to optimize via given channel parameters. The statistical simulation method is used to evaluate SI performances, including PAM-4 eye-diagram. Proposed IL-BE shows 100× faster training speed than conventional DRL method, 16× reduced iterations for the via parameter optimization than BO having state-of-the-art SI performances.
