Learning from Noisy Labels for Visual Question Answering

• Main Authors: Pan, Wei-Zhi, 潘韋志
• Other Authors: Peng, Wen-Hsiao
• Format: Others
• Language: zh-TW
• Published: 2018
• Online Access: http://ndltd.ncl.edu.tw/handle/jnaynq

碩士 === 國立交通大學 === 多媒體工程研究所 === 107 === This thesis conducts a study of learning algorithms to address noisy label issues inherent in Visual Question Answering (VQA) tasks. The noisy labelling in VQA tasks refers to the phenomenon of possibly collecting different answers to an image-question pair from different human subjects. This often arises because some image-question pairs may create an ambiguous context that leads to indefinite answers. When trained with such noisy supervision, the performance of the VQA model suffers. To address noisy label issues, we first survey three mainstream algorithms for learning from noisy labels, including (1) loss-correction, (2) label cleansing and (3) graphical models. We then implement these algorithms based on a dual attention VQA network (which we call the base VQA model) and test their performance on VirginiaTech VQA dataset. Experimental results show that (1) the performances of the loss-correction algorithms rely heavily on accurate estimation of label transition probabilities due to noise or accurate detection of noise level, that (2) the label cleansing algorithms require enough verified labels to perform effectively, and that (3) the graphical models need to differentiate the noise level of each QA input to work well. In addition, the capability of the base VQA model can have a profound effect on the performances of these noisy label learning algorithms.