McDRAM v2: In-Dynamic Random Access Memory Systolic Array Accelerator to Address the Large Model Problem in Deep Neural Networks on the Edge
The energy efficiency of accelerating hundreds of MB-large deep neural networks (DNNs) in a mobile environment is less than that of a server-class big chip accelerator because of the limited power budget, silicon area, and smaller buffer size of static random access memory associated with mobile sys...
Main Authors: | , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
IEEE
2020-01-01
|
Series: | IEEE Access |
Subjects: | |
Online Access: | https://ieeexplore.ieee.org/document/9146167/ |
id |
doaj-2483953c4df9456586dedd6f0d2eba5e |
---|---|
record_format |
Article |
spelling |
doaj-2483953c4df9456586dedd6f0d2eba5e2021-03-30T03:24:33ZengIEEEIEEE Access2169-35362020-01-01813522313524310.1109/ACCESS.2020.30112659146167McDRAM v2: In-Dynamic Random Access Memory Systolic Array Accelerator to Address the Large Model Problem in Deep Neural Networks on the EdgeSeunghwan Cho0https://orcid.org/0000-0001-5373-329XHaerang Choi1Eunhyeok Park2https://orcid.org/0000-0002-7331-9819Hyunsung Shin3https://orcid.org/0000-0001-8104-3368Sungjoo Yoo4https://orcid.org/0000-0002-5853-0675Department of Computer Science and Engineering, Seoul National University, Seoul, South KoreaDepartment of Computer Science and Engineering, Seoul National University, Seoul, South KoreaInter-University Semiconductor Research Center (ISRC), Seoul National University, Seoul, South KoreaMemory Division, Samsung Electronics, Hwaseong, South KoreaDepartment of Computer Science and Engineering, Seoul National University, Seoul, South KoreaThe energy efficiency of accelerating hundreds of MB-large deep neural networks (DNNs) in a mobile environment is less than that of a server-class big chip accelerator because of the limited power budget, silicon area, and smaller buffer size of static random access memory associated with mobile systems. To address this challenge and provide powerful computing capability for processing large DNN models in power/resource-limited mobile systems, we propose McDRAM v2, which is a novel in-dynamic random access memory (DRAM) systolic array accelerator architecture. McDRAM v2 makes the best use of large in-DRAM bandwidths for accelerating various DNN applications. It can handle large DNN models without off-chip memory accesses, in a fast and efficient manner, by exposing the large DRAM capacity and large in-DRAM bandwidth directly to an input systolic array of a processing element matrix. Additionally, it maximizes data reuse using a systolic multiply-accumulate (MAC) structure. The proposed architecture maximizes the utilization of large-scale MAC units by judiciously exploiting the DRAM's internal bus and buffer structure. An evaluation of large DNN models in the fields of image classification, natural language processing, and recommendation systems shows that it achieves 1.7 times tera operations per second (TOPS), 3.7 times TOPS/watt, and 8.6 times TOPS/mm<sup>2</sup> improvements over a state-of-the-art mobile graphics processing unit accelerator, and 4.1 times better energy efficiency over a state-of-the-art server-class accelerator. Moreover, it incurs a minimal overhead, i.e., a 9.7% increase in area, and uses less than 4.4 W of peak operating power.https://ieeexplore.ieee.org/document/9146167/Acceleratorconvolutional neural networkdeep neural networkdynamic random access memoryedge inferencemulti-layer perceptron |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Seunghwan Cho Haerang Choi Eunhyeok Park Hyunsung Shin Sungjoo Yoo |
spellingShingle |
Seunghwan Cho Haerang Choi Eunhyeok Park Hyunsung Shin Sungjoo Yoo McDRAM v2: In-Dynamic Random Access Memory Systolic Array Accelerator to Address the Large Model Problem in Deep Neural Networks on the Edge IEEE Access Accelerator convolutional neural network deep neural network dynamic random access memory edge inference multi-layer perceptron |
author_facet |
Seunghwan Cho Haerang Choi Eunhyeok Park Hyunsung Shin Sungjoo Yoo |
author_sort |
Seunghwan Cho |
title |
McDRAM v2: In-Dynamic Random Access Memory Systolic Array Accelerator to Address the Large Model Problem in Deep Neural Networks on the Edge |
title_short |
McDRAM v2: In-Dynamic Random Access Memory Systolic Array Accelerator to Address the Large Model Problem in Deep Neural Networks on the Edge |
title_full |
McDRAM v2: In-Dynamic Random Access Memory Systolic Array Accelerator to Address the Large Model Problem in Deep Neural Networks on the Edge |
title_fullStr |
McDRAM v2: In-Dynamic Random Access Memory Systolic Array Accelerator to Address the Large Model Problem in Deep Neural Networks on the Edge |
title_full_unstemmed |
McDRAM v2: In-Dynamic Random Access Memory Systolic Array Accelerator to Address the Large Model Problem in Deep Neural Networks on the Edge |
title_sort |
mcdram v2: in-dynamic random access memory systolic array accelerator to address the large model problem in deep neural networks on the edge |
publisher |
IEEE |
series |
IEEE Access |
issn |
2169-3536 |
publishDate |
2020-01-01 |
description |
The energy efficiency of accelerating hundreds of MB-large deep neural networks (DNNs) in a mobile environment is less than that of a server-class big chip accelerator because of the limited power budget, silicon area, and smaller buffer size of static random access memory associated with mobile systems. To address this challenge and provide powerful computing capability for processing large DNN models in power/resource-limited mobile systems, we propose McDRAM v2, which is a novel in-dynamic random access memory (DRAM) systolic array accelerator architecture. McDRAM v2 makes the best use of large in-DRAM bandwidths for accelerating various DNN applications. It can handle large DNN models without off-chip memory accesses, in a fast and efficient manner, by exposing the large DRAM capacity and large in-DRAM bandwidth directly to an input systolic array of a processing element matrix. Additionally, it maximizes data reuse using a systolic multiply-accumulate (MAC) structure. The proposed architecture maximizes the utilization of large-scale MAC units by judiciously exploiting the DRAM's internal bus and buffer structure. An evaluation of large DNN models in the fields of image classification, natural language processing, and recommendation systems shows that it achieves 1.7 times tera operations per second (TOPS), 3.7 times TOPS/watt, and 8.6 times TOPS/mm<sup>2</sup> improvements over a state-of-the-art mobile graphics processing unit accelerator, and 4.1 times better energy efficiency over a state-of-the-art server-class accelerator. Moreover, it incurs a minimal overhead, i.e., a 9.7% increase in area, and uses less than 4.4 W of peak operating power. |
topic |
Accelerator convolutional neural network deep neural network dynamic random access memory edge inference multi-layer perceptron |
url |
https://ieeexplore.ieee.org/document/9146167/ |
work_keys_str_mv |
AT seunghwancho mcdramv2indynamicrandomaccessmemorysystolicarrayacceleratortoaddressthelargemodelproblemindeepneuralnetworksontheedge AT haerangchoi mcdramv2indynamicrandomaccessmemorysystolicarrayacceleratortoaddressthelargemodelproblemindeepneuralnetworksontheedge AT eunhyeokpark mcdramv2indynamicrandomaccessmemorysystolicarrayacceleratortoaddressthelargemodelproblemindeepneuralnetworksontheedge AT hyunsungshin mcdramv2indynamicrandomaccessmemorysystolicarrayacceleratortoaddressthelargemodelproblemindeepneuralnetworksontheedge AT sungjooyoo mcdramv2indynamicrandomaccessmemorysystolicarrayacceleratortoaddressthelargemodelproblemindeepneuralnetworksontheedge |
_version_ |
1724183493472157696 |