MuMIA: Multimodal Interactions to Better Understand Art Contexts

• Main Authors: George E. Raptis, Giannis Kavvetsos, Christina Katsini
• Format: Article
• Language: English
• Published: MDPI AG 2021-03-01
• Series: Applied Sciences
• Subjects: human–computer interaction; multimodal interactions; eye tracking; voice; cultural heritage; museum
• Online Access: https://www.mdpi.com/2076-3417/11/6/2695

Cultural heritage is a challenging domain of application for novel interactive technologies, where varying aspects in the way that cultural assets are delivered play a major role in enhancing the visitor experience, either onsite or online. Technology-supported natural human–computer interaction that is based on multimodalities is a key factor in enabling wider and enriched access to cultural heritage assets. In this paper, we present the design and evaluation of an interactive system that aims to support visitors towards a better understanding of art contexts through the use of a multimodal interface, based on visual and audio interactions. The results of the evaluation study shed light on the dimensions of evoking natural interactions within cultural heritage environments, using micro-narratives for self-exploration and understanding of cultural content, and the intersection between human–computer interaction and artificial intelligence within cultural heritage. We expect our findings to provide useful insights for practitioners and researchers of the broad human–computer interaction and cultural heritage communities on designing and evaluating multimodal interfaces to better support visitor experiences.