Att utvärdera AdApt, ett multimodalt konverserande dialogsystem, med PARADISE

• Main Author: Hjalmarsson, Anna
• Format: Others
• Language: Swedish
• Published: Linköpings universitet, Institutionen för datavetenskap 2003
• Subjects: Interdisciplinary studies; PARADISE; AdApt; system evaluation; dialogue system; multi-modal; TVÄRVETENSKAP; Social Sciences Interdisciplinary; Tvärvetenskapliga studier inom samhällsvetenskap
• Online Access: http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-1831

This master’s thesis presents experiences from an evaluation of AdApt, a multi- modal, conversational dialogue system, using PARADISE, PARAdigm for Dialogue System Evaluation, a general framework for evaluation. The purpose of this master’s thesis was to assess PARADISE as an evaluation tool for such a system. An experimental study with 26 subjects was performed. The subjects were asked to interact with one of three different system versions of AdApt. Data was collected through questionnaires, hand tagging of the dialogues and automatic logging of the interaction. Analysis of the results suggests that further research is needed to develop a general framework for evaluation which is easy to apply and can be used for varying kinds of spoken dialogue systems. The data collected in this study can be used as starting point for further research.