Exploratory Application of Augmented Reality/Mixed Reality Devices for Acute Care Procedure Training

Introduction Augmented reality (AR), mixed reality (MR), and virtual reality devices are enabling technologies that may facilitate effective communication in healthcare between those with information and knowledge (clinician/specialist; expert; educator) and those seeking understanding and insight (...

Full description

Bibliographic Details
Main Authors: Leo Kobayashi, Xiao Chi Zhang, Scott A. Collins, Naz Karim, Derek L. Merck
Format: Article
Language:English
Published: eScholarship Publishing, University of California 2017-12-01
Series:Western Journal of Emergency Medicine
Online Access:https://escholarship.org/uc/item/5n6993rt
id doaj-aa7f25501b1349cf99919e95cf30bf35
record_format Article
spelling doaj-aa7f25501b1349cf99919e95cf30bf352020-11-24T23:38:04ZengeScholarship Publishing, University of CaliforniaWestern Journal of Emergency Medicine1936-90182017-12-0119110.5811/westjem.2017.10.35026wjem-19-158Exploratory Application of Augmented Reality/Mixed Reality Devices for Acute Care Procedure TrainingLeo Kobayashi0Xiao Chi Zhang1Scott A. Collins2Naz Karim3Derek L. Merck4Alpert Medical School of Brown University, Department of Emergency Medicine, Providence, Rhode IslandAlpert Medical School of Brown University, Department of Emergency Medicine, Providence, Rhode IslandRhode Island Hospital, CT Scan Department, Providence, Rhode IslandAlpert Medical School of Brown University, Department of Emergency Medicine, Providence, Rhode IslandAlpert Medical School of Brown University, Department of Diagnostic Imaging, Providence, Rhode IslandIntroduction Augmented reality (AR), mixed reality (MR), and virtual reality devices are enabling technologies that may facilitate effective communication in healthcare between those with information and knowledge (clinician/specialist; expert; educator) and those seeking understanding and insight (patient/family; non-expert; learner). Investigators initiated an exploratory program to enable the study of AR/MR use-cases in acute care clinical and instructional settings. Methods Academic clinician educators, computer scientists, and diagnostic imaging specialists conducted a proof-of-concept project to 1) implement a core holoimaging pipeline infrastructure and open-access repository at the study institution, and 2) use novel AR/MR techniques on off-the-shelf devices with holoimages generated by the infrastructure to demonstrate their potential role in the instructive communication of complex medical information. Results The study team successfully developed a medical holoimaging infrastructure methodology to identify, retrieve, and manipulate real patients’ de-identified computed tomography and magnetic resonance imagesets for rendering, packaging, transfer, and display of modular holoimages onto AR/MR headset devices and connected displays. Holoimages containing key segmentations of cervical and thoracic anatomic structures and pathology were overlaid and registered onto physical task trainers for simulation-based “blind insertion” invasive procedural training. During the session, learners experienced and used task-relevant anatomic holoimages for central venous catheter and tube thoracostomy insertion training with enhanced visual cues and haptic feedback. Direct instructor access into the learner’s AR/MR headset view of the task trainer was achieved for visual-axis interactive instructional guidance. Conclusion Investigators implemented a core holoimaging pipeline infrastructure and modular open-access repository to generate and enable access to modular holoimages during exploratory pilot stage applications for invasive procedure training that featured innovative AR/MR techniques on off-the-shelf headset devices.https://escholarship.org/uc/item/5n6993rt
collection DOAJ
language English
format Article
sources DOAJ
author Leo Kobayashi
Xiao Chi Zhang
Scott A. Collins
Naz Karim
Derek L. Merck
spellingShingle Leo Kobayashi
Xiao Chi Zhang
Scott A. Collins
Naz Karim
Derek L. Merck
Exploratory Application of Augmented Reality/Mixed Reality Devices for Acute Care Procedure Training
Western Journal of Emergency Medicine
author_facet Leo Kobayashi
Xiao Chi Zhang
Scott A. Collins
Naz Karim
Derek L. Merck
author_sort Leo Kobayashi
title Exploratory Application of Augmented Reality/Mixed Reality Devices for Acute Care Procedure Training
title_short Exploratory Application of Augmented Reality/Mixed Reality Devices for Acute Care Procedure Training
title_full Exploratory Application of Augmented Reality/Mixed Reality Devices for Acute Care Procedure Training
title_fullStr Exploratory Application of Augmented Reality/Mixed Reality Devices for Acute Care Procedure Training
title_full_unstemmed Exploratory Application of Augmented Reality/Mixed Reality Devices for Acute Care Procedure Training
title_sort exploratory application of augmented reality/mixed reality devices for acute care procedure training
publisher eScholarship Publishing, University of California
series Western Journal of Emergency Medicine
issn 1936-9018
publishDate 2017-12-01
description Introduction Augmented reality (AR), mixed reality (MR), and virtual reality devices are enabling technologies that may facilitate effective communication in healthcare between those with information and knowledge (clinician/specialist; expert; educator) and those seeking understanding and insight (patient/family; non-expert; learner). Investigators initiated an exploratory program to enable the study of AR/MR use-cases in acute care clinical and instructional settings. Methods Academic clinician educators, computer scientists, and diagnostic imaging specialists conducted a proof-of-concept project to 1) implement a core holoimaging pipeline infrastructure and open-access repository at the study institution, and 2) use novel AR/MR techniques on off-the-shelf devices with holoimages generated by the infrastructure to demonstrate their potential role in the instructive communication of complex medical information. Results The study team successfully developed a medical holoimaging infrastructure methodology to identify, retrieve, and manipulate real patients’ de-identified computed tomography and magnetic resonance imagesets for rendering, packaging, transfer, and display of modular holoimages onto AR/MR headset devices and connected displays. Holoimages containing key segmentations of cervical and thoracic anatomic structures and pathology were overlaid and registered onto physical task trainers for simulation-based “blind insertion” invasive procedural training. During the session, learners experienced and used task-relevant anatomic holoimages for central venous catheter and tube thoracostomy insertion training with enhanced visual cues and haptic feedback. Direct instructor access into the learner’s AR/MR headset view of the task trainer was achieved for visual-axis interactive instructional guidance. Conclusion Investigators implemented a core holoimaging pipeline infrastructure and modular open-access repository to generate and enable access to modular holoimages during exploratory pilot stage applications for invasive procedure training that featured innovative AR/MR techniques on off-the-shelf headset devices.
url https://escholarship.org/uc/item/5n6993rt
work_keys_str_mv AT leokobayashi exploratoryapplicationofaugmentedrealitymixedrealitydevicesforacutecareproceduretraining
AT xiaochizhang exploratoryapplicationofaugmentedrealitymixedrealitydevicesforacutecareproceduretraining
AT scottacollins exploratoryapplicationofaugmentedrealitymixedrealitydevicesforacutecareproceduretraining
AT nazkarim exploratoryapplicationofaugmentedrealitymixedrealitydevicesforacutecareproceduretraining
AT dereklmerck exploratoryapplicationofaugmentedrealitymixedrealitydevicesforacutecareproceduretraining
_version_ 1725517880270782464