DETERMINING PLANE-SWEEP SAMPLING POINTS IN IMAGE SPACE USING THE CROSS-RATIO FOR IMAGE-BASED DEPTH ESTIMATION

With the emergence of small consumer Unmanned Aerial Vehicles (UAVs), the importance and interest of image-based depth estimation and model generation from aerial images has greatly increased in the photogrammetric society. In our work, we focus on algorithms that allow an online image-based dense...

Full description

Bibliographic Details
Main Authors: B. Ruf, B. Erdnuess, M. Weinmann
Format: Article
Language:English
Published: Copernicus Publications 2017-08-01
Series:The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Online Access:https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLII-2-W6/325/2017/isprs-archives-XLII-2-W6-325-2017.pdf
id doaj-4a83d4ae76b94a6891e8612aefc0e1e4
record_format Article
spelling doaj-4a83d4ae76b94a6891e8612aefc0e1e42020-11-25T00:42:44ZengCopernicus PublicationsThe International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences1682-17502194-90342017-08-01XLII-2-W632533210.5194/isprs-archives-XLII-2-W6-325-2017DETERMINING PLANE-SWEEP SAMPLING POINTS IN IMAGE SPACE USING THE CROSS-RATIO FOR IMAGE-BASED DEPTH ESTIMATIONB. Ruf0B. Ruf1B. Erdnuess2B. Erdnuess3M. Weinmann4Fraunhofer IOSB, Video Exploitation Systems, 76131 Karlsruhe, GermanyInstitute of Photogrammetry and Remote Sensing, Karlsruhe Institute of Technology, 76131 Karlsruhe, GermanyFraunhofer IOSB, Video Exploitation Systems, 76131 Karlsruhe, GermanyInstitute of Photogrammetry and Remote Sensing, Karlsruhe Institute of Technology, 76131 Karlsruhe, GermanyInstitute of Photogrammetry and Remote Sensing, Karlsruhe Institute of Technology, 76131 Karlsruhe, GermanyWith the emergence of small consumer Unmanned Aerial Vehicles (UAVs), the importance and interest of image-based depth estimation and model generation from aerial images has greatly increased in the photogrammetric society. In our work, we focus on algorithms that allow an online image-based dense depth estimation from video sequences, which enables the direct and live structural analysis of the depicted scene. Therefore, we use a multi-view plane-sweep algorithm with a semi-global matching (SGM) optimization which is parallelized for general purpose computation on a GPU (GPGPU), reaching sufficient performance to keep up with the key-frames of input sequences. One important aspect to reach good performance is the way to sample the scene space, creating plane hypotheses. A small step size between consecutive planes, which is needed to reconstruct details in the near vicinity of the camera may lead to ambiguities in distant regions, due to the perspective projection of the camera. Furthermore, an equidistant sampling with a small step size produces a large number of plane hypotheses, leading to high computational effort. To overcome these problems, we present a novel methodology to directly determine the sampling points of plane-sweep algorithms in image space. The use of the perspective invariant cross-ratio allows us to derive the location of the sampling planes directly from the image data. With this, we efficiently sample the scene space, achieving higher sampling density in areas which are close to the camera and a lower density in distant regions. We evaluate our approach on a synthetic benchmark dataset for quantitative evaluation and on a real-image dataset consisting of aerial imagery. The experiments reveal that an inverse sampling achieves equal and better results than a linear sampling, with less sampling points and thus less runtime. Our algorithm allows an online computation of depth maps for subsequences of five frames, provided that the relative poses between all frames are given.https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLII-2-W6/325/2017/isprs-archives-XLII-2-W6-325-2017.pdf
collection DOAJ
language English
format Article
sources DOAJ
author B. Ruf
B. Ruf
B. Erdnuess
B. Erdnuess
M. Weinmann
spellingShingle B. Ruf
B. Ruf
B. Erdnuess
B. Erdnuess
M. Weinmann
DETERMINING PLANE-SWEEP SAMPLING POINTS IN IMAGE SPACE USING THE CROSS-RATIO FOR IMAGE-BASED DEPTH ESTIMATION
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
author_facet B. Ruf
B. Ruf
B. Erdnuess
B. Erdnuess
M. Weinmann
author_sort B. Ruf
title DETERMINING PLANE-SWEEP SAMPLING POINTS IN IMAGE SPACE USING THE CROSS-RATIO FOR IMAGE-BASED DEPTH ESTIMATION
title_short DETERMINING PLANE-SWEEP SAMPLING POINTS IN IMAGE SPACE USING THE CROSS-RATIO FOR IMAGE-BASED DEPTH ESTIMATION
title_full DETERMINING PLANE-SWEEP SAMPLING POINTS IN IMAGE SPACE USING THE CROSS-RATIO FOR IMAGE-BASED DEPTH ESTIMATION
title_fullStr DETERMINING PLANE-SWEEP SAMPLING POINTS IN IMAGE SPACE USING THE CROSS-RATIO FOR IMAGE-BASED DEPTH ESTIMATION
title_full_unstemmed DETERMINING PLANE-SWEEP SAMPLING POINTS IN IMAGE SPACE USING THE CROSS-RATIO FOR IMAGE-BASED DEPTH ESTIMATION
title_sort determining plane-sweep sampling points in image space using the cross-ratio for image-based depth estimation
publisher Copernicus Publications
series The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
issn 1682-1750
2194-9034
publishDate 2017-08-01
description With the emergence of small consumer Unmanned Aerial Vehicles (UAVs), the importance and interest of image-based depth estimation and model generation from aerial images has greatly increased in the photogrammetric society. In our work, we focus on algorithms that allow an online image-based dense depth estimation from video sequences, which enables the direct and live structural analysis of the depicted scene. Therefore, we use a multi-view plane-sweep algorithm with a semi-global matching (SGM) optimization which is parallelized for general purpose computation on a GPU (GPGPU), reaching sufficient performance to keep up with the key-frames of input sequences. One important aspect to reach good performance is the way to sample the scene space, creating plane hypotheses. A small step size between consecutive planes, which is needed to reconstruct details in the near vicinity of the camera may lead to ambiguities in distant regions, due to the perspective projection of the camera. Furthermore, an equidistant sampling with a small step size produces a large number of plane hypotheses, leading to high computational effort. To overcome these problems, we present a novel methodology to directly determine the sampling points of plane-sweep algorithms in image space. The use of the perspective invariant cross-ratio allows us to derive the location of the sampling planes directly from the image data. With this, we efficiently sample the scene space, achieving higher sampling density in areas which are close to the camera and a lower density in distant regions. We evaluate our approach on a synthetic benchmark dataset for quantitative evaluation and on a real-image dataset consisting of aerial imagery. The experiments reveal that an inverse sampling achieves equal and better results than a linear sampling, with less sampling points and thus less runtime. Our algorithm allows an online computation of depth maps for subsequences of five frames, provided that the relative poses between all frames are given.
url https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLII-2-W6/325/2017/isprs-archives-XLII-2-W6-325-2017.pdf
work_keys_str_mv AT bruf determiningplanesweepsamplingpointsinimagespaceusingthecrossratioforimagebaseddepthestimation
AT bruf determiningplanesweepsamplingpointsinimagespaceusingthecrossratioforimagebaseddepthestimation
AT berdnuess determiningplanesweepsamplingpointsinimagespaceusingthecrossratioforimagebaseddepthestimation
AT berdnuess determiningplanesweepsamplingpointsinimagespaceusingthecrossratioforimagebaseddepthestimation
AT mweinmann determiningplanesweepsamplingpointsinimagespaceusingthecrossratioforimagebaseddepthestimation
_version_ 1725280611113893888