Multimodal Fusion Based on LSTM and a Couple Conditional Hidden Markov Model for Chinese Sign Language Recognition

Multimodal Fusion Based on LSTM and a Couple Conditional Hidden Markov Model for Chinese Sign Language Recognition

A novel multimodal fusion approach is proposed for Chinese sign language (CSL) recognition. This framework, the LSTM2+CHMM model, uses dual long short-term memory (LSTM) and a couple hidden Markov model (CHMM) to fuse hand and skeleton sequence information. Novel contributions, first, include a uniq...

Full description

Bibliographic Details
Main Authors:	Qinkun Xiao, Minying Qin, Peng Guo, Yidan Zhao
Format:	Article
Language:	English
Published:	IEEE 2019-01-01
Series:	IEEE Access
Subjects:	Multimodal fusion LSTM CHMM CNN CSL recognition
Online Access:	https://ieeexplore.ieee.org/document/8750875/

Similar Items

Favorite Video Classification Based on Multimodal Bidirectional LSTM
by: Takahiro Ogawa, et al.
Published: (2018-01-01)

Multimodal Gesture Recognition Using 3-D Convolution and Convolutional LSTM
by: Guangming Zhu, et al.
Published: (2017-01-01)

Fault diagnosis of rolling element bearing based on wavelet kernel principle component analysis-coupled hidden Markov model
by: Hongchao Wang, et al.
Published: (2017-12-01)

Multi-Information Spatial–Temporal LSTM Fusion Continuous Sign Language Neural Machine Translation
by: Qinkun Xiao, et al.
Published: (2020-01-01)

Recognizing Emotions Evoked by Music Using CNN-LSTM Networks on EEG Signals
by: Sobhan Sheykhivand, et al.
Published: (2020-01-01)

Convolutional neural networks approach for multimodal biometric identification system using the fusion of fingerprint, finger-vein and face images
by: El mehdi Cherrat, et al.
Published: (2020-01-01)

Continuous Gesture Recognition Based on Time Sequence Fusion Using MIMO Radar Sensor and Deep Learning
by: Wentai Lei, et al.
Published: (2020-05-01)

Design of vocieprint recognition system based on CNN-LSTM neural network
by: Mu Junjie, et al.
Published: (2021-03-01)

Attention Alignment Multimodal LSTM for Fine-Gained Common Space Learning
by: Sijia Chen, et al.
Published: (2018-01-01)

End-to-End Multimodal 16-Day Hatching Eggs Classification
by: Lei Geng, et al.
Published: (2019-06-01)

A Hybrid of Deep CNN and Bidirectional LSTM for Automatic Speech Recognition
by: Passricha Vishal, et al.
Published: (2019-03-01)

A 3D-CNN and LSTM Based Multi-Task Learning Architecture for Action Recognition
by: Xi Ouyang, et al.
Published: (2019-01-01)

Domain Fusion CNN-LSTM for Short-Term Power Consumption Forecasting
by: Xiaorui Shao, et al.
Published: (2020-01-01)

Robust Multimodal Emotion Recognition from Conversation with Transformer-Based Crossmodality Fusion
by: Baijun Xie, et al.
Published: (2021-07-01)

Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
by: Francisco Javier Ordóñez, et al.
Published: (2016-01-01)

Cross-Subject Multimodal Emotion Recognition Based on Hybrid Fusion
by: Yucel Cimtay, et al.
Published: (2020-01-01)

British Sign Language Recognition via Late Fusion of Computer Vision and Leap Motion with Transfer Learning to American Sign Language
by: Jordan J. Bird, et al.
Published: (2020-09-01)

A Multimodal Sensor Fusion Architecture for Audio-Visual Speech Recognition
by: Makkook, Mustapha
Published: (2007)

A Multimodal Sensor Fusion Architecture for Audio-Visual Speech Recognition
by: Makkook, Mustapha
Published: (2007)

CNN-Based Multimodal Human Recognition in Surveillance Environments
by: Ja Hyung Koo, et al.
Published: (2018-09-01)

ANALYSIS OF MULTIMODAL FUSION TECHNIQUES FOR AUDIO-VISUAL SPEECH RECOGNITION
by: D.V. Ivanko, et al.
Published: (2016-05-01)

Supervised ECG wave segmentation using convolutional LSTM
by: Aman Malali, et al.
Published: (2020-09-01)

AHWR-Net: offline handwritten amharic word recognition using convolutional recurrent neural network
by: Fetulhak Abdurahman, et al.
Published: (2021-07-01)

A Hybrid Latent Space Data Fusion Method for Multimodal Emotion Recognition
by: Shahla Nemati, et al.
Published: (2019-01-01)

Graph Fusion for Finger Multimodal Biometrics
by: Haigang Zhang, et al.
Published: (2019-01-01)

Detecting Deception from Gaze and Speech Using a Multimodal Attention LSTM-Based Framework
by: Ascensión Gallardo-Antolín, et al.
Published: (2021-07-01)

Deep Learning Framework for Alzheimer’s Disease Diagnosis via 3D-CNN and FSBi-LSTM
by: Chiyu Feng, et al.
Published: (2019-01-01)

Rating Algorithm for Pronunciation of English Based on Audio Feature Pattern Matching
by: Li Kun, et al.
Published: (2015-01-01)

Multimodal Attention Network for Continuous-Time Emotion Recognition Using Video and EEG Signals
by: Dong Yoon Choi, et al.
Published: (2020-01-01)

MindLink-Eumpy: An Open-Source Python Toolbox for Multimodal Emotion Recognition
by: Ruixin Li, et al.
Published: (2021-02-01)

A Novel DAS Signal Recognition Method Based on Spatiotemporal Information Extraction With 1DCNNs-BiLSTM Network
by: Huijuan Wu, et al.
Published: (2020-01-01)

Toward Transportation Mode Recognition Using Deep Convolutional and Long Short-Term Memory Recurrent Neural Networks
by: Yanjun Qin, et al.
Published: (2019-01-01)

A Scalable and Hybrid Intrusion Detection System Based on the Convolutional-LSTM Network
by: Muhammad Ashfaq Khan, et al.
Published: (2019-04-01)

Dance Emotion Recognition Based on Laban Motion Analysis Using Convolutional Neural Network and Long Short-Term Memory
by: Simin Wang, et al.
Published: (2020-01-01)

Towards Automatic Depression Detection: A BiLSTM/1D CNN-Based Model
by: Lin Lin, et al.
Published: (2020-12-01)

Implicit Discourse Relation Recognition via a BiLSTM-CNN Architecture With Dynamic Chunk-Based Max Pooling
by: Fengyu Guo, et al.
Published: (2019-01-01)

Explainable Multimodal Fusion
by: Alvi, Jaweriah
Published: (2021)

CNN and LSTM-Based Emotion Charting Using Physiological Signals
by: Muhammad Najam Dar, et al.
Published: (2020-08-01)

Attention-LSTM-Attention Model for Speech Emotion Recognition and Analysis of IEMOCAP Database
by: Yeonguk Yu, et al.
Published: (2020-04-01)

An Investigation of Deep Learning Models for EEG-Based Emotion Recognition
by: Yaqing Zhang, et al.
Published: (2020-12-01)