An Interactive Augmented Reality System with Occlusion Handling using Kinect

• Main Authors: ZI-QUAN YANG, 楊子權
• Other Authors: 陳湘鳳
• Format: Others
• Language: zh-TW
• Published: 2015
• Online Access: http://ndltd.ncl.edu.tw/handle/10304263520978062796

碩士 === 國立臺灣大學 === 機械工程學研究所 === 103 === Augmented Reality is a technology which combines virtual object with real scene captured from camera using computer graphic rendering and calculation of camera position technology. This study solved occlusion problem in augmented reality by camera calibration method and propose a method that let user directly interact with virtual object in augmented reality system. The result shows the method proposed by this study is effective. Kinect from Microsoft was used in this method. Infrared and color camera were calibrated by a stereo calibration method. Infrared and depth image’s translation parameters were calibrated by a calibration board made by this study. Occlusion problem was solved through the developed calibration procedures which can be used in dynamic indoor scenes and have accurate and real time results. Using the calibration board made in this study to correct the translation error between the depth map and infrared image. This study applied edge detection, ellipse fitting, and image filtering to search for correspondence points of the center of ellipse. Then calculated the least square solution of translation error. The result indicated the method to be effective. This study solved occlusion problem by the camera calibration parameters. The approach overwrites Z-buffer in the drawing library with the calibration depth map. Per pixel comparing method was conducted to show the partial occlusion images. This study proposes a method which transform user skeleton coordinate into model coordinate system and designs series of methods to interact with virtual object using natural user interface. Finally, this study created an augmented reality system for furniture display and conducted a user test. The result shows that users think occlusion handling is effective and must be used in the system; system runs smoothly and interaction with virtual object is enjoyable; design of interaction is intuitive. The result of the user test can be used as an example for developing future natural user interfaces in augmented reality systems.