Robust semi-automatic 2D-to-3D image conversion via residual-driven optimization

• Main Author: Hongxing Yuan
• Format: Article
• Language: English
• Published: SpringerOpen 2018-08-01
• Series: EURASIP Journal on Image and Video Processing
• Subjects: 3D video; 2D-to-3D conversion; Depth; Cross-boundary scribbles; Optimization
• Online Access: http://link.springer.com/article/10.1186/s13640-018-0310-x

Abstract Semi-automatic 2D-to-3D conversion provides a cost-effective solution to the problem of 3D content shortage. The performance of most methods degrades significantly when cross-boundary scribbles are present due to their inability to remove unwanted input. To address this problem, a residual-driven energy function is proposed to remove unwanted input introduced by cross-boundary scribbles while preserving expected user input. Firstly, confidence of user input is computed from residuals between the estimation and user-specified depth values, and it is applied to the data fidelity term. Secondly, the residual-driven optimization is performed to estimate dense depth from user scribbles. The procedure is repeated until a maximum number of iterations is exceeded. Input confidence based on residuals avoids the propagation of unwanted scribbles and thus enables to generate high-quality depth even with cross-boundary input. Experimental results demonstrate that the proposed method removes unwanted scribbles successfully while preserving expected input, and it outperforms the state-of-the-art when presented with cross-boundary scribbles.