Mobile Product Recognition with Deep Learning of Category Selection, Fine-grained Appearance and Part Configuration

• Main Authors: Huang, Chia Wan, 黃家琬
• Other Authors: Hsu, Chiou Ting
• Format: Others
• Language: en_US
• Published: 2016
• Online Access: http://ndltd.ncl.edu.tw/handle/85024293032152018215

碩士 === 國立清華大學 === 資訊工程學系 === 104 === The goal of product recognition is to retrieve database images similar to the query image and then determine the product of the query based on the retrieved images. In product recognition, there are two issues, inter-product similarity and intra-product diversity, concerning the recognition performance. To address these two issues, we first introduce an intuitive multi-stage method which consists of three convolutional neural networks (CNN) and the proposed similarity measurement. Since the similarity estimation with all images in the database costs a great amount of time, we further propose a two-stage method which estimates similarity with the images under the predicted category of the query. In this scenario, we need to offline cluster products into categories. However, because traditional clustering methods extract features and cluster images separately without coordination, they usually end up with improper cluster assignment. To tackle the weakness of these methods, we propose two clustering schemes that iteratively refine the last few layers of Faster RCNN toward the best category assignment and feature representation. Next, to achieve an efficient recognition process, we execute the repetitive steps of part configuration comparison as well as feature extraction once in the two-stage method. For validation, we conduct experiments on SHORT and our own collected dataset, PRODUCT-100, taken under different variations. The experimental results show that the proposed two-stage method demonstrates promising performance with regard to both accuracy and efficiency.