Context-Based Structure Mining Methodology for Static Object Re-Identification in Broadcast Content

• Main Authors: Krishna Kumar Thirukokaranam Chandrasekar, Steven Verstockt
• Format: Article
• Language: English
• Published: MDPI AG 2021-08-01
• Series: Applied Sciences
• Subjects: object detection; logical story unit detection (LSU); object re-ID
• Online Access: https://www.mdpi.com/2076-3417/11/16/7266

Technological advancement, in addition to the pandemic, has given rise to an explosive increase in the consumption and creation of multimedia content worldwide. This has motivated people to enrich and publish their content in a way that enhances the experience of the user. In this paper, we propose a context-based structure mining pipeline that not only attempts to enrich the content, but also simultaneously splits it into shots and logical story units (LSU). Subsequently, this paper extends the structure mining pipeline to re-ID objects in broadcast videos such as SOAPs. We hypothesise the object re-ID problem of SOAP-type content to be equivalent to the identification of reoccurring contexts, since these contexts normally have a unique spatio-temporal similarity within the content structure. By implementing pre-trained models for object and place detection, the pipeline was evaluated using metrics for shot and scene detection on benchmark datasets, such as RAI. The object re-ID methodology was also evaluated on 20 randomly selected episodes from broadcast SOAP shows <i>New Girl</i> and <i>Friends</i>. We demonstrate, quantitatively, that the pipeline outperforms existing state-of-the-art methods for shot boundary detection, scene detection, and re-identification tasks.