Creating the Internet of Augmented Things: An Open-Source Framework to Make IoT Devices and Augmented and Mixed Reality Systems Talk to Each Other

Augmented Reality (AR) and Mixed Reality (MR) devices have evolved significantly in the last years, providing immersive AR/MR experiences that allow users to interact with virtual elements placed on the real-world. However, to make AR/MR devices reach their full potential, it is necessary to go furt...

Full description

Bibliographic Details
Main Authors: Óscar Blanco-Novoa, Paula Fraga-Lamas, Miguel A. Vilar-Montesinos, Tiago M. Fernández-Caramés
Format: Article
Language:English
Published: MDPI AG 2020-06-01
Series:Sensors
Subjects:
IoT
Online Access:https://www.mdpi.com/1424-8220/20/11/3328
id doaj-1e4bac84b0de44078b4a3c2a174daa33
record_format Article
spelling doaj-1e4bac84b0de44078b4a3c2a174daa332020-11-25T02:52:42ZengMDPI AGSensors1424-82202020-06-01203328332810.3390/s20113328Creating the Internet of Augmented Things: An Open-Source Framework to Make IoT Devices and Augmented and Mixed Reality Systems Talk to Each OtherÓscar Blanco-Novoa0Paula Fraga-Lamas1Miguel A. Vilar-Montesinos2Tiago M. Fernández-Caramés3Department of Computer Engineering, Faculty of Computer Science, Universidade da Coruña, 15071 A Coruña, SpainDepartment of Computer Engineering, Faculty of Computer Science, Universidade da Coruña, 15071 A Coruña, SpainNavantia S. A., Astillero de Ferrol, 15403 Ferrol, SpainDepartment of Computer Engineering, Faculty of Computer Science, Universidade da Coruña, 15071 A Coruña, SpainAugmented Reality (AR) and Mixed Reality (MR) devices have evolved significantly in the last years, providing immersive AR/MR experiences that allow users to interact with virtual elements placed on the real-world. However, to make AR/MR devices reach their full potential, it is necessary to go further and let them collaborate with the physical elements around them, including the objects that belong to the Internet of Things (IoT). Unfortunately, AR/MR and IoT devices usually make use of heterogeneous technologies that complicate their intercommunication. Moreover, the implementation of the intercommunication mechanisms requires involving specialized developers with have experience on the necessary technologies. To tackle such problems, this article proposes the use of a framework that makes it easy to integrate AR/MR and IoT devices, allowing them to communicate dynamically and in real time. The presented AR/MR-IoT framework makes use of standard and open-source protocols and tools like MQTT, HTTPS or Node-RED. After detailing the inner workings of the framework, it is illustrated its potential through a practical use case: a smart power socket that can be monitored and controlled through Microsoft HoloLens AR/MR glasses. The performance of such a practical use case is evaluated and it is demonstrated that the proposed framework, under normal operation conditions, enables to respond in less than 100 ms to interaction and data update requests.https://www.mdpi.com/1424-8220/20/11/3328augmented realitymixed realityIoTInternet of Thingsopen-source frameworkmicrosoft holoLens
collection DOAJ
language English
format Article
sources DOAJ
author Óscar Blanco-Novoa
Paula Fraga-Lamas
Miguel A. Vilar-Montesinos
Tiago M. Fernández-Caramés
spellingShingle Óscar Blanco-Novoa
Paula Fraga-Lamas
Miguel A. Vilar-Montesinos
Tiago M. Fernández-Caramés
Creating the Internet of Augmented Things: An Open-Source Framework to Make IoT Devices and Augmented and Mixed Reality Systems Talk to Each Other
Sensors
augmented reality
mixed reality
IoT
Internet of Things
open-source framework
microsoft holoLens
author_facet Óscar Blanco-Novoa
Paula Fraga-Lamas
Miguel A. Vilar-Montesinos
Tiago M. Fernández-Caramés
author_sort Óscar Blanco-Novoa
title Creating the Internet of Augmented Things: An Open-Source Framework to Make IoT Devices and Augmented and Mixed Reality Systems Talk to Each Other
title_short Creating the Internet of Augmented Things: An Open-Source Framework to Make IoT Devices and Augmented and Mixed Reality Systems Talk to Each Other
title_full Creating the Internet of Augmented Things: An Open-Source Framework to Make IoT Devices and Augmented and Mixed Reality Systems Talk to Each Other
title_fullStr Creating the Internet of Augmented Things: An Open-Source Framework to Make IoT Devices and Augmented and Mixed Reality Systems Talk to Each Other
title_full_unstemmed Creating the Internet of Augmented Things: An Open-Source Framework to Make IoT Devices and Augmented and Mixed Reality Systems Talk to Each Other
title_sort creating the internet of augmented things: an open-source framework to make iot devices and augmented and mixed reality systems talk to each other
publisher MDPI AG
series Sensors
issn 1424-8220
publishDate 2020-06-01
description Augmented Reality (AR) and Mixed Reality (MR) devices have evolved significantly in the last years, providing immersive AR/MR experiences that allow users to interact with virtual elements placed on the real-world. However, to make AR/MR devices reach their full potential, it is necessary to go further and let them collaborate with the physical elements around them, including the objects that belong to the Internet of Things (IoT). Unfortunately, AR/MR and IoT devices usually make use of heterogeneous technologies that complicate their intercommunication. Moreover, the implementation of the intercommunication mechanisms requires involving specialized developers with have experience on the necessary technologies. To tackle such problems, this article proposes the use of a framework that makes it easy to integrate AR/MR and IoT devices, allowing them to communicate dynamically and in real time. The presented AR/MR-IoT framework makes use of standard and open-source protocols and tools like MQTT, HTTPS or Node-RED. After detailing the inner workings of the framework, it is illustrated its potential through a practical use case: a smart power socket that can be monitored and controlled through Microsoft HoloLens AR/MR glasses. The performance of such a practical use case is evaluated and it is demonstrated that the proposed framework, under normal operation conditions, enables to respond in less than 100 ms to interaction and data update requests.
topic augmented reality
mixed reality
IoT
Internet of Things
open-source framework
microsoft holoLens
url https://www.mdpi.com/1424-8220/20/11/3328
work_keys_str_mv AT oscarblanconovoa creatingtheinternetofaugmentedthingsanopensourceframeworktomakeiotdevicesandaugmentedandmixedrealitysystemstalktoeachother
AT paulafragalamas creatingtheinternetofaugmentedthingsanopensourceframeworktomakeiotdevicesandaugmentedandmixedrealitysystemstalktoeachother
AT miguelavilarmontesinos creatingtheinternetofaugmentedthingsanopensourceframeworktomakeiotdevicesandaugmentedandmixedrealitysystemstalktoeachother
AT tiagomfernandezcarames creatingtheinternetofaugmentedthingsanopensourceframeworktomakeiotdevicesandaugmentedandmixedrealitysystemstalktoeachother
_version_ 1724728142640185344