Interaction Design for Augmented Reality by Using Dynamic Hand Gesture and Speech Recognition

• Main Authors: Cheng-Ying Han, 韓承穎
• Other Authors: Shuenn-Jyi Wang
• Format: Others
• Language: zh-TW
• Published: 2014
• Online Access: http://ndltd.ncl.edu.tw/handle/41940268987778656071

碩士 === 國防大學理工學院 === 資訊工程碩士班 === 102 === The technique of Augmented Reality (AR) augment virtual objects or information to the real scene using a visible camera with schemes of object detection and tracking. To interact with virtual objects intuitively, bared-hand gestures can be used to control virtual objects. However, the depth information of hand gesture is not easy obtained using a single visible camera. Therefore, we cannot control the virtual objects with three dimensional variations. Though depth information can be acquired via Kinect equipment, the price is higher than a Webcam and it is not popular especially in mobile devices. In this thesis, we proposed a scheme to join dynamic hand gesture recognition and speech recognition to control virtual objects with three dimensional activities using a visible camera. First, we design an iterative decision tree modeling method to extract two dynamic hand skin regions. Then, the finger points and palm centers are computed via contour approximate, convex-hull, and maximum inscribed circle algorithms. Hand gestures are recognized and speech recognition is assisted to obtain depth information. Finally, the accurate scales, directions, and positions of virtual objects are computed and proper presented. Moreover, the proposed scheme can be applied to mobile devices such as iPad, smart phone, or Google Glass.