Spatio-temporal image inpainting for video applications

• Main Authors: Voronin Viacheslav, Marchuk Vladimir, Makov Sergey, Mladenović Vladimir, Cen Yigang
• Format: Article
• Language: English
• Published: Faculty of Technical Sciences in Cacak 2017-01-01
• Series: Serbian Journal of Electrical Engineering
• Subjects: inpainting; patching; masking; spatio-temporal; restoring of missing pixels; video; dynamic textures
• Online Access: http://www.doiserbia.nb.rs/img/doi/1451-4869/2017/1451-48691700004V.pdf

Video inpainting or completion is a vital video improvement technique used to repair or edit digital videos. This paper describes a framework for temporally consistent video completion. The proposed method allows to remove dynamic objects or restore missing or tainted regions presented in a video sequence by utilizing spatial and temporal information from neighboring scenes. Masking algorithm is used for detection of scratches or damaged portions in video frames. The algorithm iteratively performs the following operations: achieve frame; update the scene model; update positions of moving objects; replace parts of the frame occupied by the objects marked for remove by using a background model. In this paper, we extend an image inpainting algorithm based texture and structure reconstruction by incorporating an improved strategy for video. Our algorithm is able to deal with a variety of challenging situations which naturally arise in video inpainting, such as the correct reconstruction of dynamic textures, multiple moving objects and moving background. Experimental comparisons to state-of-the-art video completion methods demonstrate the effectiveness of the proposed approach. It is shown that the proposed spatio-temporal image inpainting method allows restoring a missing blocks and removing a text from the scenes on videos.