Deep Learning for Transient Image Reconstruction from ToF Data

In this work, we propose a novel approach for correcting multi-path interference (MPI) in Time-of-Flight (ToF) cameras by estimating the direct and global components of the incoming light. MPI is an error source linked to the multiple reflections of light inside a scene; each sensor pixel receives i...

Full description

Bibliographic Details
Main Authors: Enrico Buratto, Adriano Simonetto, Gianluca Agresti, Henrik Schäfer, Pietro Zanuttigh
Format: Article
Language:English
Published: MDPI AG 2021-03-01
Series:Sensors
Subjects:
Online Access:https://www.mdpi.com/1424-8220/21/6/1962
Description
Summary:In this work, we propose a novel approach for correcting multi-path interference (MPI) in Time-of-Flight (ToF) cameras by estimating the direct and global components of the incoming light. MPI is an error source linked to the multiple reflections of light inside a scene; each sensor pixel receives information coming from different light paths which generally leads to an overestimation of the depth. We introduce a novel deep learning approach, which estimates the structure of the time-dependent scene impulse response and from it recovers a depth image with a reduced amount of MPI. The model consists of two main blocks: a predictive model that learns a compact encoded representation of the backscattering vector from the noisy input data and a fixed backscattering model which translates the encoded representation into the high dimensional light response. Experimental results on real data show the effectiveness of the proposed approach, which reaches state-of-the-art performances.
ISSN:1424-8220