Automatic Album Summarization Based on 4W: Who, When, Where, and What

• Main Authors: Hsiao, Fan Kai, 蕭凡凱
• Other Authors: Lin, Chia Wen
• Format: Others
• Language: en_US
• Published: 2015
• Online Access: http://ndltd.ncl.edu.tw/handle/70180349620791986344

碩士 === 國立清華大學 === 通訊工程研究所 === 103 === In this paper, we present an automatic personal album summarization system. Given a query viewer, the proposed method combines 4W: who, when, where, and what to calculate the optimization function and output the summary. Our method includes two parts: the first part is the album structure extraction. We extract following structures: for “who” is in the album, we use the correlation between the characters, calculate the correlation matrix and the appearance frequency. Furthermore, we combine the community detection to find close relationship. “When” were the photos taken helps to split the time events. “Where” were the photos taken can be extract from GPS which helps maintain the diversity of geolocation. The last one is “what” the image contents are about. The image contents are analyzed by the local and global features, which form the hierarchical structures named shots and scenes in the album. In the second part of the method, given viewer query, we combine 4W and design an optimization function. Solving the optimization function by greedy algorithm, we can get the summary result which is depended on who the viewer is. We build the online album viewing system, model how user used to view the album. The experiment is evaluated by the subjective assessment. We let the viewer input who he/she is, system shows the corresponding results. The experiment results show the effectiveness of the proposed method personal album summary. Compare to other methods, ours can let viewers understand more about the album content and is preferable by viewers.