A Portable Speaking-Aid System for Taiwanese Sign Language

• Main Authors: Yu-Xiang Zhao, 趙于翔
• Other Authors: Mu-Chun Su
• Format: Others
• Language: zh-TW
• Published: 2002
• Online Access: http://ndltd.ncl.edu.tw/handle/26567206052890681505

碩士 === 淡江大學 === 電機工程學系 === 90 === Successful rehabilitation of the deaf in a hearing world must provide a mechanism by which meaningful communication between the deaf and hearing persons is possible. For this to happen, it first requires a hand gesture recognition system to recognize sign language and a speech synthesizer to provide speech to the hearing person. The goal of this thesis is to implement a portable Taiwanese Sign Language (TSL) recognition system as a speaking aid for the deaf in Taiwan. TSL is the language most commonly used by the Taiwan deaf community for interpersonal communication. It uses hands, body and facial expressions to make signs. In TSL, each sign word can be described by the following four attributes: (1) the shape of the hands, (2) the location of the hands relative to the signer, (3) the orientation of the palms, and (4) the trajectory of the arms. The proposed speaking aid consists of four main components: input devices, a FPGA-based system, a recognition system, and a speech synthesizer. Input devices consist of a pair of sensing gloves which measure fingers’ flex angles and a 3-dimensional tracking system which senses the 3D absolute positions of the hands in space. The FPGA-based system consists of five modules: (1) an input module for the gloves, (2) an input module for the 3-dimensional tracking system, (3) a control module, (4) a coordinating and communication module, and (5) a blue tooth module. The gestural information measured by the two input devices is transmitted to the FPGA-based system via the two input modules. The main mission of the control module is to integrate the gestural information. The recognition system can control and communicate with the control module via the coordinating and communication module. Finally, the blue tooth module works as a wireless interface between the FPGA-based system and the recognition system. In the recognition system, each attribute (e.g. the shape of the hands, the trajectory of the arms etc) has its corresponding recognition module. In this thesis, we focus on recognizing 3D arm movements. A neural-network-based approach and a fuzzy-rule-based approach are proposed. Each has its own merits and disadvantages. To make the speaking aid portable, we decide to adopt a hand-held computer as the working platform of the recognition system and the speech synthesizer since the weight of a hand-held computer is not as heavy as a notebook and it can also output sounds. We really hope the deaf in Taiwan can benefit from the proposed portable speaking aid.