MARKER-LESS MOBILE AUGMENTED REALITY APPLICATION FOR MASSIVE 3D POINT CLOUDS AND SEMANTICS

• Main Authors: A. Kharroubi, R. Billen, F. Poux
• Format: Article
• Language: English
• Published: Copernicus Publications 2020-08-01
• Series: The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
• Online Access: https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLIII-B2-2020/255/2020/isprs-archives-XLIII-B2-2020-255-2020.pdf

Mobile Augmented Reality (MAR) attracts significant research and development efforts from both the industry and academia, but rarely integrate massive 3D dataset’s interactions. The emergence of dedicated AR devices and powerful Software Development Kit (ARCore for android and ARKit for iOS) improves performance on mobile devices (Smartphones and tablets). This is aided by new sensor integration and advances in computer vision that fuels the development of MAR. In this paper, we propose a direct integration of massive 3D point clouds with semantics in a web-based marker-less mobile Augmented Reality (AR) application for real-time visualization. We specifically investigate challenges linked to point cloud data structure and semantic injection. Our solution consolidates some of the overarching principles of AR, of which pose estimation, registration and 3D tracking. The developed AR system is tested on mobile phones web-browsers providing clear insights on the performance of the system. Promising results highlight a number of frame per second varying between 27 and 60 for a real-time point budget of 4.3 million points. The point cloud tested is composed of 29 million points and shows how our indexation strategy permits the integration of massive point clouds aiming at the point budget. The results also gives research directions concerning the dependence and delay related to the quality of the network connection, and the battery consumption since portable sensors are used all the time.