Deep Learning for Transient Image Reconstruction from ToF Data

• Main Authors: Enrico Buratto, Adriano Simonetto, Gianluca Agresti, Henrik Schäfer, Pietro Zanuttigh
• Format: Article
• Language: English
• Published: MDPI AG 2021-03-01
• Series: Sensors
• Subjects: Time-of-Flight; multi-path interference; depth estimation; transient imaging; denoising; deep learning
• Online Access: https://www.mdpi.com/1424-8220/21/6/1962

In this work, we propose a novel approach for correcting multi-path interference (MPI) in Time-of-Flight (ToF) cameras by estimating the direct and global components of the incoming light. MPI is an error source linked to the multiple reflections of light inside a scene; each sensor pixel receives information coming from different light paths which generally leads to an overestimation of the depth. We introduce a novel deep learning approach, which estimates the structure of the time-dependent scene impulse response and from it recovers a depth image with a reduced amount of MPI. The model consists of two main blocks: a predictive model that learns a compact encoded representation of the backscattering vector from the noisy input data and a fixed backscattering model which translates the encoded representation into the high dimensional light response. Experimental results on real data show the effectiveness of the proposed approach, which reaches state-of-the-art performances.