Multimodal Interaction of Contextual and Non-Contextual Sound and Haptics in Virtual Simulations

• Main Authors: Mohammed Melaisi, David Rojas, Bill Kapralos, Alvaro Uribe-Quevedo, Karen Collins
• Format: Article
• Language: English
• Published: MDPI AG 2018-11-01
• Series: Informatics
• Subjects: multimodal interactions; haptics; sound; virtual simulation; task performance; virtual environment
• Online Access: https://www.mdpi.com/2227-9709/5/4/43

Touch plays a fundamental role in our daily interactions, allowing us to interact with and perceive objects and their spatial properties. Despite its importance in the real-world, touch is often ignored in virtual environments. However, accurately simulating the sense of touch is difficult, requiring the use of high-fidelity haptic devices that are cost-prohibitive. Lower fidelity consumer-level haptic devices are becoming more widespread, yet are generally limited in perceived fidelity and the range of motion (degrees of freedom) required to realistically simulate many tasks. Studies into sound and vision suggest that the presence or absence of sound can influence task performance. Here, we explore whether the presence or absence of contextually relevant sound cues influences the performance of a simple haptic drilling task. Although the results of this study do not show any statistically significant difference in task performance with general (task-irrelevant) sound, we discuss how this is a necessary step in understanding the role of sound on haptic perception.