A Study on the Evaluation Method of the Emotional Expression in Chatbot Dialogue

• Main Authors: SHIH, WEN-FENG, 施文烽
• Other Authors: WU, SHIH-HUNG
• Format: Others
• Language: zh-TW
• Published: 2018
• Online Access: http://ndltd.ncl.edu.tw/handle/s544c7

碩士 === 朝陽科技大學 === 資訊工程系 === 106 === Along with the advancement of technology and popularity, people’s communication way is influenced by Instant Messaging. The Chatbot technology is also rising. Most studies of Chatbot is focused on engineering aspect, that is: how to make better Chatbot. But the research on evaluation aspect is few. However, it is also important on how to evaluate the degree of different aspects in responses that replied by Chatbot. There are two major evaluation methods: automatic evaluation method and the evaluation method that judge by human. Although the automatic evaluation method can judge the dialogue system quickly, it has low correlation with human in many cases. On the other hand, judgement by human can avoid this situation. Therefore, the evaluation method can get more effective result at the evaluation method that judge by human. We will adopt the evaluation method that judge by human in our research. The evaluation rules in STC task of NTCIR are designed for general reply evaluation method. So it do not evaluate the expression degree in each aspects of responses. To design an evaluation method that can effective judge dialogue system is our research objective. By observing and analyze the evaluation methods of STC task in NTCIR, we designed a process of designing questionnaire, and find out factors with different aspects to design evaluation questions with different aspects for different question and answer types. We use consistency to judge our evaluation questions. In the experiments, we picked the dialog data with emotional expression type and analyzed it. In general, about the emotional expression type is venting his emotion. People might have different thoughts, and then the content of responses contains of sympathy, comfort and ridicule. So it needs to consider these different aspects in evaluation method. We designed the new evaluation questionnaire with different aspects, and verify it whether easily achieve consistency or not. In questionnaire survey result, we found out that the different aspects are easily achieve consistency. Therefore, our presented the process of design questionnaire that can make a new questionnaire to evaluate dialogues system with different aspects. In different corpus also suitable for our process of design questionnaire.