Improving Classification Performance of Softmax Loss Function Based on Scalable Batch-Normalization

Improving Classification Performance of Softmax Loss Function Based on Scalable Batch-Normalization

Convolutional neural networks (CNNs) have made great achievements on computer vision tasks, especially the image classification. With the improvement of network structure and loss functions, the performance of image classification is getting higher and higher. The classic Softmax + cross-entropy los...

Full description

Bibliographic Details
Main Authors: Qiuyu Zhu, Zikuang He, Tao Zhang, Wennan Cui
Format: Article
Language:English
Published: MDPI AG 2020-04-01
Series:Applied Sciences
Subjects:
convolutional neural network
loss function
gradient decent
Online Access:https://www.mdpi.com/2076-3417/10/8/2950
id doaj-4102fd05c8b64f638758a7f95c81e350
record_format Article
spelling doaj-4102fd05c8b64f638758a7f95c81e3502020-11-25T02:00:30ZengMDPI AGApplied Sciences2076-34172020-04-01102950295010.3390/app10082950Improving Classification Performance of Softmax Loss Function Based on Scalable Batch-NormalizationQiuyu Zhu0Zikuang He1Tao Zhang2Wennan Cui3School of Communication & Information Engineering, Shanghai University, Shanghai 200444, ChinaSchool of Communication & Information Engineering, Shanghai University, Shanghai 200444, ChinaKey Laboratory of intelligent infrared perception, Chinese Academy of Sciences, Shanghai 200083, ChinaKey Laboratory of intelligent infrared perception, Chinese Academy of Sciences, Shanghai 200083, ChinaConvolutional neural networks (CNNs) have made great achievements on computer vision tasks, especially the image classification. With the improvement of network structure and loss functions, the performance of image classification is getting higher and higher. The classic Softmax + cross-entropy loss has been the norm for training neural networks for years, which is calculated from the output probability of the ground-truth class. Then the network’s weight is updated by gradient calculation of the loss. However, after several epochs of training, the back-propagation errors usually become almost negligible. For the above considerations, we proposed that batch normalization with adjustable scale could be added after network output to alleviate the problem of vanishing gradient problem in deep learning. The experimental results show that our method can significantly improve the final classification accuracy on different network structures, and is also better than many other improved classification Loss.https://www.mdpi.com/2076-3417/10/8/2950convolutional neural networkloss functiongradient decent
collection DOAJ
language English
format Article
sources DOAJ
author Qiuyu Zhu
Zikuang He
Tao Zhang
Wennan Cui
spellingShingle Qiuyu Zhu
Zikuang He
Tao Zhang
Wennan Cui
Improving Classification Performance of Softmax Loss Function Based on Scalable Batch-Normalization
Applied Sciences
convolutional neural network
loss function
gradient decent
author_facet Qiuyu Zhu
Zikuang He
Tao Zhang
Wennan Cui
author_sort Qiuyu Zhu
title Improving Classification Performance of Softmax Loss Function Based on Scalable Batch-Normalization
title_short Improving Classification Performance of Softmax Loss Function Based on Scalable Batch-Normalization
title_full Improving Classification Performance of Softmax Loss Function Based on Scalable Batch-Normalization
title_fullStr Improving Classification Performance of Softmax Loss Function Based on Scalable Batch-Normalization
title_full_unstemmed Improving Classification Performance of Softmax Loss Function Based on Scalable Batch-Normalization
title_sort improving classification performance of softmax loss function based on scalable batch-normalization
publisher MDPI AG
series Applied Sciences
issn 2076-3417
publishDate 2020-04-01
description Convolutional neural networks (CNNs) have made great achievements on computer vision tasks, especially the image classification. With the improvement of network structure and loss functions, the performance of image classification is getting higher and higher. The classic Softmax + cross-entropy loss has been the norm for training neural networks for years, which is calculated from the output probability of the ground-truth class. Then the network’s weight is updated by gradient calculation of the loss. However, after several epochs of training, the back-propagation errors usually become almost negligible. For the above considerations, we proposed that batch normalization with adjustable scale could be added after network output to alleviate the problem of vanishing gradient problem in deep learning. The experimental results show that our method can significantly improve the final classification accuracy on different network structures, and is also better than many other improved classification Loss.
topic convolutional neural network
loss function
gradient decent
url https://www.mdpi.com/2076-3417/10/8/2950
work_keys_str_mv AT qiuyuzhu improvingclassificationperformanceofsoftmaxlossfunctionbasedonscalablebatchnormalization
AT zikuanghe improvingclassificationperformanceofsoftmaxlossfunctionbasedonscalablebatchnormalization
AT taozhang improvingclassificationperformanceofsoftmaxlossfunctionbasedonscalablebatchnormalization
AT wennancui improvingclassificationperformanceofsoftmaxlossfunctionbasedonscalablebatchnormalization
_version_ 1724960105925967872

Similar Items