Development of Infrared Lip Movement Sensor for Spoken Word Recognition

• Main Authors: Takahiro Yoshida, Seiichiro Hangai
• Format: Article
• Language: English
• Published: International Institute of Informatics and Cybernetics 2007-12-01
• Series: Journal of Systemics, Cybernetics and Informatics
• Online Access: http://www.iiisci.org/Journal/CV$/sci/pdfs/S054ZB.pdf

Lip movement of speaker is very informative for many application of speech signal processing such as multi-modal speech recognition and password authentication without speech signal. However, in collecting multi-modal speech information, we need a video camera, large amount of memory, video interface, and high speed processor to extract lip movement in real time. Such a system tends to be expensive and large. This is one reasons of preventing the use of multi-modal speech processing. In this study, we have developed a simple infrared lip movement sensor mounted on a headset, and made it possible to acquire lip movement by PDA, mobile phone, and notebook PC. The sensor consists of an infrared LED and an infrared photo transistor, and measures the lip movement by the reflected light from the mouth region. From experiment, we achieved 66% successfully word recognition rate only by lip movement features. This experimental result shows that our developed sensor can be utilized as a tool for multi-modal speech processing by combining a microphone mounted on the headset.