Artificial Neural Network and Resistive Random Access Memory for Handwritten Digit Recognition

• Main Authors: Zeng, Wan-Qi, 曾琬淇
• Other Authors: Hou, Tuo-Hung
• Format: Others
• Language: en_US
• Published: 2019
• Online Access: http://ndltd.ncl.edu.tw/handle/9q824a

碩士 === 國立交通大學 === 電子研究所 === 107 === “Artificial Intelligence” becomes a hot topic in recent years. The word of “Artificial Intelligence”, AI, was coined by John McCarthy at the Dartmouth Conference in 1956. Due to the financial setbacks and the bottleneck of the algorithm, there were two periods of declining funding and interest in AI, also called as “AI winter”. In 2012, AlexNet, an eight-layer convolutional neural network (CNN) algorithm won the champion in the ImageNet Large Scale Visual Recognition Challenge. In 2016, AlphaGo, a program developed by Google DeepMind, beat the world champion of the Go, Lee Sedol. These successes laid important fundaments for AI nowadays. Neuromorphic computing is a promising approach to implement AI, which imitates the computation in human brain for achieving low-power consumption, parallel computing and fault tolerance. The neural network comprise a large amount of synapses and neurons, and the signal transmits from pre-neurons to synapses and then to post-neurons. The connection strength between synapses is not fixed because of the synaptic plasticity. The weight is adjusted by the Spike-Timing Dependent Plasticity (STDP) to allow learning in the network. Coincidentally, the analog resistive random-access memory (RRAM) shares the similar characteristics with the synapse. That is, the conductance change of RRAM can be modulated by different input voltages. Additionally, thanks to the simple structure of RRAM, it can be used to realize high density array and accelerate the neuromorphic computing. In this thesis, we implement the two-layer back propagation algorithm to recognize the enlarged patterns by using analog RRAM. According to the algorithm, we design the hardware to test the enlarged patterns. Besides, we discuss the device characteristic and its impact on the neural network. Finally, we demonstrate the recognition results of the enlarged patterns. There are five chapters in this thesis, and the main content of the research is described from Chapter 2 to Chapter 4. In chapter 2, we introduce the system architecture and the corresponding application of each sub-module. In a circuit system corresponding to a neural network, the RRAM crossbar array acts as the synaptic network, the pulse generator acts as the pre-neuron, the leaky integrate-and-fire circuit acts as the post-neuron, and the FPGA is the controller to control the clock or adjust the weight in the network. Then, we will introduce the sequence of each operation, the relationship between the RRAM and the peripheral circuits, and the precise waveforms in each operation. In chapter3, we analyze the characteristics of the analog RRAM. First, we test the characteristics of the single cell in DC and AC mode, the operation condition, the endurance and the line resistance. Second, because numerous cells of RRAM with acceptable characteristics are needed to implement the neural network, we design a rapid device screening method before and after device package. Then, we test the impact of breakdown devices in the array and discuss how to use the array without a perfect yield. Finally, we discuss the impacts on the RRAM when exposed to high-energy X-ray. In chapter 4, we implement the training and testing of enlarged patterns into the hardware, and modify the non-ideal peripheral circuits to improve the resolution of the system. Then, we demonstrate the results of pattern recognition from the two-layer neural network (100ｘ10ｘ 3). In this research, we implement the two-layer back propagation algorithm to recognize the enlarged patterns in the hardware. This makes the implement of the neuromorphic computing hardware closer to real applications. We believe the neuromorphic computing is capable of learning more complex tasks and making accurate decisions in the future.