Communication Usage Optimization of Gradient Sparsification with Aggregation
碩士 === 國立臺灣大學 === 資訊工程學研究所 === 106 === Communication usage is a bottleneck of scaling workers for distributed deep learning. One solution is to compress the exchanged gradients into sparse format with gradient sparsification. We found that the send cost of server, which is the aggregated size of spa...
| Main Authors: | , |
|---|---|
| Other Authors: | |
| Format: | Others |
| Language: | en_US |
| Published: |
2018
|
| Online Access: | http://ndltd.ncl.edu.tw/handle/ppyyqb |