Four-Features Evaluation of Text to Speech Systems for Three Social Robots

The success of social robotics is directly linked to their ability of interacting with people. Humans possess verbal and non-verbal communication skills, and, therefore, both are essential for social robots to get a natural human−robot interaction. This work focuses on the first of them si...

Full description

Bibliographic Details
Main Authors: Fernando Alonso Martin, María Malfaz, Álvaro Castro-González, José Carlos Castillo, Miguel Ángel Salichs
Format: Article
Language:English
Published: MDPI AG 2020-02-01
Series:Electronics
Subjects:
Online Access:https://www.mdpi.com/2079-9292/9/2/267
id doaj-0dc89a6d949e4b71bd016a20d3aa9117
record_format Article
spelling doaj-0dc89a6d949e4b71bd016a20d3aa91172020-11-25T02:05:44ZengMDPI AGElectronics2079-92922020-02-019226710.3390/electronics9020267electronics9020267Four-Features Evaluation of Text to Speech Systems for Three Social RobotsFernando Alonso Martin0María Malfaz1Álvaro Castro-González2José Carlos Castillo3Miguel Ángel Salichs4Department of Robotics, University Carlos III of Madrid, Avda de la Universidad, 30, 28911 Leganés (Madrid), SpainDepartment of Robotics, University Carlos III of Madrid, Avda de la Universidad, 30, 28911 Leganés (Madrid), SpainDepartment of Robotics, University Carlos III of Madrid, Avda de la Universidad, 30, 28911 Leganés (Madrid), SpainDepartment of Robotics, University Carlos III of Madrid, Avda de la Universidad, 30, 28911 Leganés (Madrid), SpainDepartment of Robotics, University Carlos III of Madrid, Avda de la Universidad, 30, 28911 Leganés (Madrid), SpainThe success of social robotics is directly linked to their ability of interacting with people. Humans possess verbal and non-verbal communication skills, and, therefore, both are essential for social robots to get a natural human&#8722;robot interaction. This work focuses on the first of them since the majority of social robots implement an interaction system endowed with verbal capacities. In order to do this implementation, we must equip social robots with an artificial voice system. In robotics, a Text to Speech (TTS) system is the most common speech synthesizer technique. The performance of a speech synthesizer is mainly evaluated by its similarity to the human voice in relation to its intelligibility and expressiveness. In this paper, we present a comparative study of eight off-the-shelf TTS systems used in social robots. In order to carry out the study, 125 participants evaluated the performance of the following TTS systems: <i>Google</i>, <i>Microsoft</i>, <i>Ivona</i>, <i>Loquendo</i>, <i>Espeak</i>, <i>Pico</i>, <i>AT&amp;T</i>, and <i>Nuance</i>. The evaluation was performed after observing videos where a social robot communicates verbally using one TTS system. The participants completed a questionnaire to rate each TTS system in relation to four features: <i>intelligibility</i>, <i>expressiveness</i>, <i>artificiality</i>, and <i>suitability</i>. In this study, four research questions were posed to determine whether it is possible to present a ranking of TTS systems in relation to each evaluated feature, or, on the contrary, there are no significant differences between them. Our study shows that participants found differences between the TTS systems evaluated in terms of intelligibility, expressiveness, and artificiality. The experiments also indicated that there was a relationship between the physical appearance of the robots (embodiment) and the suitability of TTS systems.https://www.mdpi.com/2079-9292/9/2/267text to speech systemsuser studiesspeech-basedaccessibility technologiesnatural language generation
collection DOAJ
language English
format Article
sources DOAJ
author Fernando Alonso Martin
María Malfaz
Álvaro Castro-González
José Carlos Castillo
Miguel Ángel Salichs
spellingShingle Fernando Alonso Martin
María Malfaz
Álvaro Castro-González
José Carlos Castillo
Miguel Ángel Salichs
Four-Features Evaluation of Text to Speech Systems for Three Social Robots
Electronics
text to speech systems
user studies
speech-based
accessibility technologies
natural language generation
author_facet Fernando Alonso Martin
María Malfaz
Álvaro Castro-González
José Carlos Castillo
Miguel Ángel Salichs
author_sort Fernando Alonso Martin
title Four-Features Evaluation of Text to Speech Systems for Three Social Robots
title_short Four-Features Evaluation of Text to Speech Systems for Three Social Robots
title_full Four-Features Evaluation of Text to Speech Systems for Three Social Robots
title_fullStr Four-Features Evaluation of Text to Speech Systems for Three Social Robots
title_full_unstemmed Four-Features Evaluation of Text to Speech Systems for Three Social Robots
title_sort four-features evaluation of text to speech systems for three social robots
publisher MDPI AG
series Electronics
issn 2079-9292
publishDate 2020-02-01
description The success of social robotics is directly linked to their ability of interacting with people. Humans possess verbal and non-verbal communication skills, and, therefore, both are essential for social robots to get a natural human&#8722;robot interaction. This work focuses on the first of them since the majority of social robots implement an interaction system endowed with verbal capacities. In order to do this implementation, we must equip social robots with an artificial voice system. In robotics, a Text to Speech (TTS) system is the most common speech synthesizer technique. The performance of a speech synthesizer is mainly evaluated by its similarity to the human voice in relation to its intelligibility and expressiveness. In this paper, we present a comparative study of eight off-the-shelf TTS systems used in social robots. In order to carry out the study, 125 participants evaluated the performance of the following TTS systems: <i>Google</i>, <i>Microsoft</i>, <i>Ivona</i>, <i>Loquendo</i>, <i>Espeak</i>, <i>Pico</i>, <i>AT&amp;T</i>, and <i>Nuance</i>. The evaluation was performed after observing videos where a social robot communicates verbally using one TTS system. The participants completed a questionnaire to rate each TTS system in relation to four features: <i>intelligibility</i>, <i>expressiveness</i>, <i>artificiality</i>, and <i>suitability</i>. In this study, four research questions were posed to determine whether it is possible to present a ranking of TTS systems in relation to each evaluated feature, or, on the contrary, there are no significant differences between them. Our study shows that participants found differences between the TTS systems evaluated in terms of intelligibility, expressiveness, and artificiality. The experiments also indicated that there was a relationship between the physical appearance of the robots (embodiment) and the suitability of TTS systems.
topic text to speech systems
user studies
speech-based
accessibility technologies
natural language generation
url https://www.mdpi.com/2079-9292/9/2/267
work_keys_str_mv AT fernandoalonsomartin fourfeaturesevaluationoftexttospeechsystemsforthreesocialrobots
AT mariamalfaz fourfeaturesevaluationoftexttospeechsystemsforthreesocialrobots
AT alvarocastrogonzalez fourfeaturesevaluationoftexttospeechsystemsforthreesocialrobots
AT josecarloscastillo fourfeaturesevaluationoftexttospeechsystemsforthreesocialrobots
AT miguelangelsalichs fourfeaturesevaluationoftexttospeechsystemsforthreesocialrobots
_version_ 1724937301933424640