Interactive Question-Posing System for Reminiscing about Personal Photos

• Main Authors: Yi-Luen Wu, 吳宜倫
• Other Authors: 傅立成
• Format: Others
• Language: en_US
• Published: 2017
• Online Access: http://ndltd.ncl.edu.tw/handle/00885564573565249911

碩士 === 國立臺灣大學 === 資訊工程學研究所 === 105 === Reminiscence is a lifelong activity that happens throughout our lifespan. While memories can serve as the topics in people''s chit-chat, recalling the past can also help people to build self-esteem and increase the level of happiness. In this thesis, we aim to develop a companion robot that helps people to recollect the memories from the personal photos. We focus on how a robot could associate concepts relevant to the content in the photos and evoke people''s memory by asking related and engaging questions. To understand the content in the photo, we apply deep learning techniques to recognize events, objects, and scenes in the image. Then, the observations from the photo and the user utterance are considered in the Markov random field that contains commonsense knowledge of events, whereby the loopy belief propagation is used to infer possible associated concepts and topics. Afterwards, appropriate questions about the selected topics are posed to guide the user to remember the memories in the interaction. Finally, we evaluate our system through well designed experiments. The results show that the proposed system can pose proper and related questions to interact with the user, and has a potential to help and guide the user to recall the past in an organized way.