Digital Comics Image Indexing Based on Deep Learning

• Main Authors: Nhu-Van Nguyen, Christophe Rigaud, Jean-Christophe Burie
• Format: Article
• Language: English
• Published: MDPI AG 2018-07-01
• Series: Journal of Imaging
• Subjects: comics analysis; image indexing; deep learning; CNN; LSTM; handwritten text recognition
• Online Access: http://www.mdpi.com/2313-433X/4/7/89

The digital comic book market is growing every year now, mixing digitized and digital-born comics. Digitized comics suffer from a limited automatic content understanding which restricts online content search and reading applications. This study shows how to combine state-of-the-art image analysis methods to encode and index images into an XML-like text file. Content description file can then be used to automatically split comic book images into sub-images corresponding to panels easily indexable with relevant information about their respective content. This allows advanced search in keywords said by specific comic characters, action and scene retrieval using natural language processing. We get down to panel, balloon, text, comic character and face detection using traditional approaches and breakthrough deep learning models, and also text recognition using LSTM model. Evaluations on a dataset composed of online library content are presented, and a new public dataset is also proposed.