High Accuracy RRAM-Based Binary Neural Network Training

• Main Authors: Lin, Jia-Wei, 林佳緯
• Other Authors: Chang, Tian-Sheuan
• Format: Others
• Language: en_US
• Published: 2018
• Online Access: http://ndltd.ncl.edu.tw/handle/g5t559

碩士 === 國立交通大學 === 電子研究所 === 107 === Deep learning is getting popular at the edge devices to provide better services. However, its high computational complexity and memory bandwidth makes real time execution on the traditional Von Neuman architecture very challenging even with the low complexity binary neural network model. A promising solution, in-memory computing, computes directly in memory with high parallelism and reduces the data transmission time, especially with the emerging new device such as Resistive Random Access Memory (RRAM) due to its small area and low power nature. However, RRAM devices suffers from the non-ideal state switching and variation characteristics, which could seriously affects the performance of neural networks, even with the on-device training. To solve above problem, we first analyze the non-ideal device effect on the network training performance and propose to combine residual network architectures, more channel width, and 3x3 kernel size for better training on non-ideal devices. Furthermore, we systematically determine the hyper-parameters of RRAM-based BNNs with bad probability updating curve. This training strategy makes it get better training results. Besides, device variation due to non-uniformity of the process will make training process unstable and thus get a worse results. To overcome this problem, we propose hybrid RRAM-based BNNs to combine the benefits of both digital computation and RRAM-based computation. Moreover, we use a longer training period with the above determined hyper-parameters to further improve the tolerance on the C2C variation. With the designs mentioned above, we can improve the accuracy of RRAM-based BNN from 57.39% to 83.69% even though the devices are in a worst condition.