High-resolution 360◦Video Foveated Stitching for Real-time VR

• Main Authors: Wei-Tse Lee, 李維哲
• Other Authors: Bing-Yu Chen
• Format: Others
• Language: en_US
• Published: 2016
• Online Access: http://ndltd.ncl.edu.tw/handle/76610318956930127902

碩士 === 國立臺灣大學 === 資訊網路與多媒體研究所 === 104 === In recent years, virtual reality (VR) becomes one of the most fascinating technologies where real-time immersion experience is in the spotlight. In those applications, the contents are usually generated by creating a 360◦video panorama of a real-world scene. Despite that many capture devices are released, getting high-resolution panoramas and display of a virtual world at real-time update rates are still very challenging since it is a computationally demanding paradigm. In this paper, we proposed a real-time 360◦video foveated stitching framework, indicating what objects in a scene should be represented at what detail level. Our foveated stitching consists of two major parts; the acuity map and the saliency map. The acuity map is calculated taking into considerations the characteristics of the human visual system, while the saliency map is calculated using theories from the field of visual attention. Our innovative solution takes multiple cameras inputs and creates a high-resolution panoramic video in real-time that can be streamed directly to the client. We parallelize the algorithm on a GPU to achieve a responsive interface and validate our results using user study. Our system accelerate graphics computation by a factor of 6 on a Google Cardboard display.