American Sign Language Recognition Using Principal Component Analysis and Dynamic Time Warping

• Main Author: Achmad Fiqhi Ibadillah
• Other Authors: Chang Hong Lin
• Format: Others
• Language: en_US
• Published: 2013
• Online Access: http://ndltd.ncl.edu.tw/handle/34087519704141211915

碩士 === 國立臺灣科技大學 === 電子工程系 === 101 === Sign language recognition research has made significant progresses in recent years. The present progresses provide the basis method for future implementations with the objective of supporting the integration of deaf people into the hearing society. Translation systems, for example, could facilitate communication between deaf and hearing people in public situations. Further applications, such as user interfaces and automatic indexing of signed videos, become feasible. The current state in sign language recognition is roughly 30 years behind [1] speech recognition. Research efforts were mainly focused on robust feature extraction or statistical modeling of signs. However, current recognition systems are still designed for signer-dependent operation under laboratory conditions. This thesis proposes a scheme to recognize isolated American Sign Language (ASL) by using RGB-D camera. We utilize 10 signs database that distinguished become 5 signs for one-handed sign language and 5 signs for two-handed sign language. The main idea of our proposed method is to recognize sign language by detecting and recognizing hand shapes. After we get the detected hand shape, then we utilize the distance trajectory feature of hand against the signer face center to classify the hand gesture. In this scheme, the hand shape detection is introduced first for further recognition process. The hand shape detection segments one or two hands according to the number of hands appearing in the image frame sequence by using background subtraction and skin color detection. Face detection is utilized to detect the signer and signer face center as threshold point. However, the skin color methods may fail in insufficient light conditions. Therefore, the adaptive lighting compensation is applied to help the skin color detection method become more accurate. Before performing hand shape recognition process, image enhancement is performed first because the obtained blobs consist of holes inside them. After obtaining masked enhanced image in gray scale color space then recognition process is performed. Principal Component Analysis (PCA) is applied to train and recognize obtained hand shapes. The hand distance trajectory is classified by using Dynamic Time Warping (DTW). To analyze the performance of proposed method the datasets established from the RGB-D camera under indoor environment are tested. The experimental results show the effectiveness of our proposed method with high hand shape recognition rate and good hand distance trajectory classification result.