Four-Features Evaluation of Text to Speech Systems for Three Social Robots
The success of social robotics is directly linked to their ability of interacting with people. Humans possess verbal and non-verbal communication skills, and, therefore, both are essential for social robots to get a natural human−robot interaction. This work focuses on the first of them si...
Main Authors: | , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
MDPI AG
2020-02-01
|
Series: | Electronics |
Subjects: | |
Online Access: | https://www.mdpi.com/2079-9292/9/2/267 |
id |
doaj-0dc89a6d949e4b71bd016a20d3aa9117 |
---|---|
record_format |
Article |
spelling |
doaj-0dc89a6d949e4b71bd016a20d3aa91172020-11-25T02:05:44ZengMDPI AGElectronics2079-92922020-02-019226710.3390/electronics9020267electronics9020267Four-Features Evaluation of Text to Speech Systems for Three Social RobotsFernando Alonso Martin0María Malfaz1Álvaro Castro-González2José Carlos Castillo3Miguel Ángel Salichs4Department of Robotics, University Carlos III of Madrid, Avda de la Universidad, 30, 28911 Leganés (Madrid), SpainDepartment of Robotics, University Carlos III of Madrid, Avda de la Universidad, 30, 28911 Leganés (Madrid), SpainDepartment of Robotics, University Carlos III of Madrid, Avda de la Universidad, 30, 28911 Leganés (Madrid), SpainDepartment of Robotics, University Carlos III of Madrid, Avda de la Universidad, 30, 28911 Leganés (Madrid), SpainDepartment of Robotics, University Carlos III of Madrid, Avda de la Universidad, 30, 28911 Leganés (Madrid), SpainThe success of social robotics is directly linked to their ability of interacting with people. Humans possess verbal and non-verbal communication skills, and, therefore, both are essential for social robots to get a natural human−robot interaction. This work focuses on the first of them since the majority of social robots implement an interaction system endowed with verbal capacities. In order to do this implementation, we must equip social robots with an artificial voice system. In robotics, a Text to Speech (TTS) system is the most common speech synthesizer technique. The performance of a speech synthesizer is mainly evaluated by its similarity to the human voice in relation to its intelligibility and expressiveness. In this paper, we present a comparative study of eight off-the-shelf TTS systems used in social robots. In order to carry out the study, 125 participants evaluated the performance of the following TTS systems: <i>Google</i>, <i>Microsoft</i>, <i>Ivona</i>, <i>Loquendo</i>, <i>Espeak</i>, <i>Pico</i>, <i>AT&T</i>, and <i>Nuance</i>. The evaluation was performed after observing videos where a social robot communicates verbally using one TTS system. The participants completed a questionnaire to rate each TTS system in relation to four features: <i>intelligibility</i>, <i>expressiveness</i>, <i>artificiality</i>, and <i>suitability</i>. In this study, four research questions were posed to determine whether it is possible to present a ranking of TTS systems in relation to each evaluated feature, or, on the contrary, there are no significant differences between them. Our study shows that participants found differences between the TTS systems evaluated in terms of intelligibility, expressiveness, and artificiality. The experiments also indicated that there was a relationship between the physical appearance of the robots (embodiment) and the suitability of TTS systems.https://www.mdpi.com/2079-9292/9/2/267text to speech systemsuser studiesspeech-basedaccessibility technologiesnatural language generation |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Fernando Alonso Martin María Malfaz Álvaro Castro-González José Carlos Castillo Miguel Ángel Salichs |
spellingShingle |
Fernando Alonso Martin María Malfaz Álvaro Castro-González José Carlos Castillo Miguel Ángel Salichs Four-Features Evaluation of Text to Speech Systems for Three Social Robots Electronics text to speech systems user studies speech-based accessibility technologies natural language generation |
author_facet |
Fernando Alonso Martin María Malfaz Álvaro Castro-González José Carlos Castillo Miguel Ángel Salichs |
author_sort |
Fernando Alonso Martin |
title |
Four-Features Evaluation of Text to Speech Systems for Three Social Robots |
title_short |
Four-Features Evaluation of Text to Speech Systems for Three Social Robots |
title_full |
Four-Features Evaluation of Text to Speech Systems for Three Social Robots |
title_fullStr |
Four-Features Evaluation of Text to Speech Systems for Three Social Robots |
title_full_unstemmed |
Four-Features Evaluation of Text to Speech Systems for Three Social Robots |
title_sort |
four-features evaluation of text to speech systems for three social robots |
publisher |
MDPI AG |
series |
Electronics |
issn |
2079-9292 |
publishDate |
2020-02-01 |
description |
The success of social robotics is directly linked to their ability of interacting with people. Humans possess verbal and non-verbal communication skills, and, therefore, both are essential for social robots to get a natural human−robot interaction. This work focuses on the first of them since the majority of social robots implement an interaction system endowed with verbal capacities. In order to do this implementation, we must equip social robots with an artificial voice system. In robotics, a Text to Speech (TTS) system is the most common speech synthesizer technique. The performance of a speech synthesizer is mainly evaluated by its similarity to the human voice in relation to its intelligibility and expressiveness. In this paper, we present a comparative study of eight off-the-shelf TTS systems used in social robots. In order to carry out the study, 125 participants evaluated the performance of the following TTS systems: <i>Google</i>, <i>Microsoft</i>, <i>Ivona</i>, <i>Loquendo</i>, <i>Espeak</i>, <i>Pico</i>, <i>AT&T</i>, and <i>Nuance</i>. The evaluation was performed after observing videos where a social robot communicates verbally using one TTS system. The participants completed a questionnaire to rate each TTS system in relation to four features: <i>intelligibility</i>, <i>expressiveness</i>, <i>artificiality</i>, and <i>suitability</i>. In this study, four research questions were posed to determine whether it is possible to present a ranking of TTS systems in relation to each evaluated feature, or, on the contrary, there are no significant differences between them. Our study shows that participants found differences between the TTS systems evaluated in terms of intelligibility, expressiveness, and artificiality. The experiments also indicated that there was a relationship between the physical appearance of the robots (embodiment) and the suitability of TTS systems. |
topic |
text to speech systems user studies speech-based accessibility technologies natural language generation |
url |
https://www.mdpi.com/2079-9292/9/2/267 |
work_keys_str_mv |
AT fernandoalonsomartin fourfeaturesevaluationoftexttospeechsystemsforthreesocialrobots AT mariamalfaz fourfeaturesevaluationoftexttospeechsystemsforthreesocialrobots AT alvarocastrogonzalez fourfeaturesevaluationoftexttospeechsystemsforthreesocialrobots AT josecarloscastillo fourfeaturesevaluationoftexttospeechsystemsforthreesocialrobots AT miguelangelsalichs fourfeaturesevaluationoftexttospeechsystemsforthreesocialrobots |
_version_ |
1724937301933424640 |