Towards better understanding and improving optimization in recurrent neural networks

Towards better understanding and improving optimization in recurrent neural networks

Recurrent neural networks (RNN) are known for their notorious exploding and vanishing gradient problem (EVGP). This problem becomes more evident in tasks where the information needed to correctly solve them exist over long time scales, because it prevents important gradient components from being bac...

Full description

Bibliographic Details
Main Author: Kanuparthi, Bhargav
Other Authors: Bengio, Yoshua
Language:English
Published: 2021
Subjects:
Machine Learning
Deep Learning
Recurrent Neural Networks
Long Term Dependencies
Exploding Vanishing Gradients Problem
Self Attentive Networks
Scalability
Apprentissage automatique
L'apprentissage en profondeur
Réseaux de neurones récurrents
Dépendances à long terme
Problème d'explosion des dégradés de fuite
Réseaux auto-attentifs
Évolutivité
Applied Sciences - Artificial Intelligence / Sciences appliqués et technologie - Intelligence artificielle (UMI : 0800)
Online Access:http://hdl.handle.net/1866/24319

Internet

http://hdl.handle.net/1866/24319

Similar Items