Deep Visual Semantic Transform Model Learning from Multi-Label Images

• Main Authors: Lee, Yi-Nan, 李奕男
• Other Authors: Yeh, Mei-Chen
• Format: Others
• Language: zh-TW
• Published: 2017
• Online Access: http://ndltd.ncl.edu.tw/handle/48kv54

碩士 === 國立臺灣師範大學 === 資訊工程學系 === 105 === Learning the relation between images and text semantics has been an important problem in the field of machine learning and computer vision. This paper addresses this problem. We observe that there is a semantic relation between texts, for example, “sky” and “cloud” have a close semantic relation, and “sky” and “car” have a weak semantic relation. We suppose the semantic relation between texts can be different depending on images. For example, an image contains both sky and car. The word “sky” and “car” are initially semantically irrelevant, but may have a connection because of the image containing these concepts. Therefore, we propose a Convolutional Neural Network based model to link the semantic relation between an image and its text labels. The main difference between our work and existing visual semantic embedding models is that the output of our model is a linear transformation function. In other words, each input image is treated as a function to determine the relation between each word and the image, and to predict the possible labels for the image. Finally, this model is validated on the NUS-WIDE dataset and the experimental results show that the model has a great performance on predicting labels for images.