GAUSSIAN MIXTURE MODELS FOR ADAPTATION OF DEEP NEURAL NETWORK ACOUSTIC MODELS IN AUTOMATIC SPEECH RECOGNITION SYSTEMS

• Main Authors: Natalia A. Tomashenko, Yuri Yu. Khokhlov, Anthony Larcher, Yannick Estève, Yuri N. Matveev
• Format: Article
• Language: English
• Published: Saint Petersburg National Research University of Information Technologies, Mechanics and Optics (ITMO University) 2016-11-01
• Series: Naučno-tehničeskij Vestnik Informacionnyh Tehnologij, Mehaniki i Optiki
• Subjects: automatic speech recognition (ASR); acoustic models; speaker adaptation; deep neural networks (DNN); GMM-derived features; GMMD; maximum a posteriori (MAP); fMLLR; GMM; acoustic model adaptation; fusion
• Online Access: http://ntv.ifmo.ru/file/article/16176.pdf
• ISSN: 2226-1494; 2500-0373

Subject of Research. We study speaker adaptation of deep neural network (DNN) acoustic models in automatic speech recognition systems. The aim of speaker adaptation techniques is to improve the accuracy of the speech recognition system for a particular speaker. Method. A novel method for training and adaptation of deep neural network acoustic models has been developed. It is based on using an auxiliary GMM (Gaussian Mixture Models) model and GMMD (GMM-derived) features. The principle advantage of the proposed GMMD features is the possibility of performing the adaptation of a DNN through the adaptation of the auxiliary GMM. In the proposed approach any methods for the adaptation of the auxiliary GMM can be used, hence, it provides a universal method for transferring adaptation algorithms developed for GMMs to DNN adaptation.Main Results. The effectiveness of the proposed approach was shown by means of one of the most common adaptation algorithms for GMM models – MAP (Maximum A Posteriori) adaptation. Different ways of integration of the proposed approach into state-of-the-art DNN architecture have been proposed and explored. Analysis of choosing the type of the auxiliary GMM model is given. Experimental results on the TED-LIUM corpus demonstrate that, in an unsupervised adaptation mode, the proposed adaptation technique can provide, approximately, a 11–18% relative word error reduction (WER) on different adaptation sets, compared to the speaker-independent DNN system built on conventional features, and a 3–6% relative WER reduction compared to the SAT-DNN trained on fMLLR adapted features.