A Sign Language Translation System on Mobile Devices

• Main Authors: Kun-YuTsai, 蔡坤佑
• Other Authors: Tzone-I Wang
• Format: Others
• Language: zh-TW
• Published: 2016
• Online Access: http://ndltd.ncl.edu.tw/handle/97329618013356290968

碩士 === 國立成功大學 === 工程科學系 === 104 === Sign language is used by hearing-impaired people to communicate with; through the hand sign gestures and body motions to convey the information to other disabled people. But normal people usually do not learn the sign language, thus are unable to understand what the disabled people say. With the advance of information technology, especially the image recognition and processing technology, mobile communication hardware and software, together should be able to build a sign language translation system that enable normal people to communicate with disabled people using sign language. The main purpose of this study is to design and implement a mobile-device-based translation system that, using a mobile device, such as a smart phone, takes video of sign language form disabled people, translates them into texts for normal people, and shows texts typed by normal people to disabled ones, repeating this process to finish the communication. The system is targeted for daily life usages, such as that in a hospital or a bank. The techniques involve capturing sign language images from a mobile device, then making use of a sequence of image processing to detect hand regions, and recognizing the information of hand gestures, in three parts. The first part is to get the information of hand shapes, by using Features from Accelerated Segment Test (FAST) and Binary Robust Independent Elementary Features (BRIEF) algorithms to detect interested points and generate characteristic values. Through calculating Hamming distance and comparing it to the image base to find the most similar hand shape. The second part is to figure out the directions of both hands’ movements by calculating the angles between two successively captures of the hands’ image center points. The third part is to get both hand positions by comparing their center points with a threshold point to decide if a hand is in the low or high position. The recognition precision of the sign language translation system, from experiments, can reach 91.1% and the average time for recognizing a hand shape is 90.59ms.