Natural Language Processing for Taiwanese Sign Language to Speech Conversion

• Main Authors: Chi-Shiang Guo, 郭啟祥
• Other Authors: Chung-Hsien Wu
• Format: Others
• Language: zh-TW
• Published: 2000
• Online Access: http://ndltd.ncl.edu.tw/handle/46258767179664734946

碩士 === 國立成功大學 === 資訊工程研究所 === 88 === Individuals with hearing/speech impairment are unable to communicate with others through natural ways in daily activities, education and vocation. Since 1970, developed countries have designed extensively Augmentative and Alternative Communication (AAC) system. The underlying principle is to facilitate the disabled with minimum efforts of alternative manual-sign input to generate comprehensive information output and greatly improve expressive communication abilities. Unfortunately the developed AAC technology cannot be directly applied to the people using Chinese language. In addition, the usage and characteristics of sign language varies with the time and the place of native circumstance. The purpose of this thesis is to investigate the methods for improving the input rate and accuracy of communication aid for Taiwanese Sign Language (TSL) AAC system. More specifically, the study focuses on: 1) developing an effective TSL virtual keyboard (VK), 2) investigating TSL prediction strategies for input rate enhancement, 3) integrating bottom-up parsing with top-down filtering strategy and Variable N-grams language modeling for sentence generation and selection. In order to evaluate the performance of our approach, a pilot study was undertaken. 1000 Chinese sentences, in which the average length of sentences is 4.9 words, were selected as the training and testing database. A 26.25% keystroke saving rate from function word deletion was achieved. The scanning rate enhancement using keyword prediction, sentence pattern prediction and Chinese phonetic abbreviation input are 67.71%, 79.50%, and 96.87%, respectively. For the assessment of practical communication aid, a single case with speech-impaired was asked to conduct the experiments. After training, adaptation and evaluation phases, the sentence construction consistency compared to user transcription is 47.37%, 65.0% and 68.38%, respectively. The generation rate and subjective satisfactory level of sentence construction have been significantly improved. Finally, a computer-aided instruction (CAI) program is also developed to improve the language learning and speech communication ability for communication impaired people.