Architecture Design for Real-Time CNN-Based Super Resolution Using Multi-Layer Computation

• Main Authors: Chung-Yan Chih, 池忠諺
• Other Authors: 陳良基
• Format: Others
• Language: en_US
• Published: 2017
• Online Access: http://ndltd.ncl.edu.tw/handle/e3h95p

碩士 === 國立臺灣大學 === 電子工程學研究所 === 106 === In recent years, the display resolution on end devices is enhancing rapidly. The super resolution aims to enhance visual quality when low-resolution contents play at high-resolution displays. Besides, high frame-rate is also an important issue to make the scene smooth. On the other hand, deep learning is growing rapidly in recent years, it can achieve better performance compared to the original methods in many computer vision domain, including super resolution. In super resolution domain, we find that the learning based methods are better than the non-learning-based methods as expected. But learning-based methods have massive computation, real-time performance is hard to achieve. The truth is, the architectures with non-learning based methods which can achieve real-time performance are rare. In upscaling option, most of the hardware can only perform upscale 2 and most of the architectures'' output resolution is below full-HD. We proposed a customized ASIC implementation of super resolution which can perform 60 fps upscaled full-HD or even higher output by different upscaling factors. Our architecture is learning based super resolution algorithm. We first compare the difference between high-resolution (HR) network and low-resolution (HR) network and decide to use LR network for computation cost concern. We then decide to use "Fast Super Resolution Convolutional Neural Network" (FSRCNN) to implement. Since the original FSRCNN has deconvolution layer but the major part is convolution, we decide to modify the deconvolution filter into multi-size convolution filters to make the hardware has higher utilization. We propose our architecture for hardware friendly FSRCNN and we define the bandwidth problem and use a different strategy to reduce the bandwidth and memory cost. Instead of layer-wise computation, which is commonly used in convolutional neural network hardware accelerator, we use multi-layer computation to reduce bandwidth requirement and enhance throughput. We arrange data flow and store reuse data by using level B data reuse to make our work a memory efficient architecture We implement an FSRCNN accelerator with a lower bandwidth, smaller on-chip memory, and high frame-rate.