DETERMINING PLANE-SWEEP SAMPLING POINTS IN IMAGE SPACE USING THE CROSS-RATIO FOR IMAGE-BASED DEPTH ESTIMATION
With the emergence of small consumer Unmanned Aerial Vehicles (UAVs), the importance and interest of image-based depth estimation and model generation from aerial images has greatly increased in the photogrammetric society. In our work, we focus on algorithms that allow an online image-based dense...
Main Authors: | , , |
---|---|
Format: | Article |
Language: | English |
Published: |
Copernicus Publications
2017-08-01
|
Series: | The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
Online Access: | https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLII-2-W6/325/2017/isprs-archives-XLII-2-W6-325-2017.pdf |
id |
doaj-4a83d4ae76b94a6891e8612aefc0e1e4 |
---|---|
record_format |
Article |
spelling |
doaj-4a83d4ae76b94a6891e8612aefc0e1e42020-11-25T00:42:44ZengCopernicus PublicationsThe International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences1682-17502194-90342017-08-01XLII-2-W632533210.5194/isprs-archives-XLII-2-W6-325-2017DETERMINING PLANE-SWEEP SAMPLING POINTS IN IMAGE SPACE USING THE CROSS-RATIO FOR IMAGE-BASED DEPTH ESTIMATIONB. Ruf0B. Ruf1B. Erdnuess2B. Erdnuess3M. Weinmann4Fraunhofer IOSB, Video Exploitation Systems, 76131 Karlsruhe, GermanyInstitute of Photogrammetry and Remote Sensing, Karlsruhe Institute of Technology, 76131 Karlsruhe, GermanyFraunhofer IOSB, Video Exploitation Systems, 76131 Karlsruhe, GermanyInstitute of Photogrammetry and Remote Sensing, Karlsruhe Institute of Technology, 76131 Karlsruhe, GermanyInstitute of Photogrammetry and Remote Sensing, Karlsruhe Institute of Technology, 76131 Karlsruhe, GermanyWith the emergence of small consumer Unmanned Aerial Vehicles (UAVs), the importance and interest of image-based depth estimation and model generation from aerial images has greatly increased in the photogrammetric society. In our work, we focus on algorithms that allow an online image-based dense depth estimation from video sequences, which enables the direct and live structural analysis of the depicted scene. Therefore, we use a multi-view plane-sweep algorithm with a semi-global matching (SGM) optimization which is parallelized for general purpose computation on a GPU (GPGPU), reaching sufficient performance to keep up with the key-frames of input sequences. One important aspect to reach good performance is the way to sample the scene space, creating plane hypotheses. A small step size between consecutive planes, which is needed to reconstruct details in the near vicinity of the camera may lead to ambiguities in distant regions, due to the perspective projection of the camera. Furthermore, an equidistant sampling with a small step size produces a large number of plane hypotheses, leading to high computational effort. To overcome these problems, we present a novel methodology to directly determine the sampling points of plane-sweep algorithms in image space. The use of the perspective invariant cross-ratio allows us to derive the location of the sampling planes directly from the image data. With this, we efficiently sample the scene space, achieving higher sampling density in areas which are close to the camera and a lower density in distant regions. We evaluate our approach on a synthetic benchmark dataset for quantitative evaluation and on a real-image dataset consisting of aerial imagery. The experiments reveal that an inverse sampling achieves equal and better results than a linear sampling, with less sampling points and thus less runtime. Our algorithm allows an online computation of depth maps for subsequences of five frames, provided that the relative poses between all frames are given.https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLII-2-W6/325/2017/isprs-archives-XLII-2-W6-325-2017.pdf |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
B. Ruf B. Ruf B. Erdnuess B. Erdnuess M. Weinmann |
spellingShingle |
B. Ruf B. Ruf B. Erdnuess B. Erdnuess M. Weinmann DETERMINING PLANE-SWEEP SAMPLING POINTS IN IMAGE SPACE USING THE CROSS-RATIO FOR IMAGE-BASED DEPTH ESTIMATION The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
author_facet |
B. Ruf B. Ruf B. Erdnuess B. Erdnuess M. Weinmann |
author_sort |
B. Ruf |
title |
DETERMINING PLANE-SWEEP SAMPLING POINTS IN IMAGE SPACE USING THE CROSS-RATIO FOR IMAGE-BASED DEPTH ESTIMATION |
title_short |
DETERMINING PLANE-SWEEP SAMPLING POINTS IN IMAGE SPACE USING THE CROSS-RATIO FOR IMAGE-BASED DEPTH ESTIMATION |
title_full |
DETERMINING PLANE-SWEEP SAMPLING POINTS IN IMAGE SPACE USING THE CROSS-RATIO FOR IMAGE-BASED DEPTH ESTIMATION |
title_fullStr |
DETERMINING PLANE-SWEEP SAMPLING POINTS IN IMAGE SPACE USING THE CROSS-RATIO FOR IMAGE-BASED DEPTH ESTIMATION |
title_full_unstemmed |
DETERMINING PLANE-SWEEP SAMPLING POINTS IN IMAGE SPACE USING THE CROSS-RATIO FOR IMAGE-BASED DEPTH ESTIMATION |
title_sort |
determining plane-sweep sampling points in image space using the cross-ratio for image-based depth estimation |
publisher |
Copernicus Publications |
series |
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
issn |
1682-1750 2194-9034 |
publishDate |
2017-08-01 |
description |
With the emergence of small consumer Unmanned Aerial Vehicles (UAVs), the importance and interest of image-based
depth estimation and model generation from aerial images has greatly increased in the photogrammetric society. In
our work, we focus on algorithms that allow an online image-based dense depth estimation from video sequences, which
enables the direct and live structural analysis of the depicted scene. Therefore, we use a multi-view plane-sweep algorithm
with a semi-global matching (SGM) optimization which is parallelized for general purpose computation on a GPU
(GPGPU), reaching sufficient performance to keep up with the key-frames of input sequences. One important aspect
to reach good performance is the way to sample the scene space, creating plane hypotheses. A small step size between
consecutive planes, which is needed to reconstruct details in the near vicinity of the camera may lead to ambiguities in
distant regions, due to the perspective projection of the camera. Furthermore, an equidistant sampling with a small step
size produces a large number of plane hypotheses, leading to high computational effort. To overcome these problems,
we present a novel methodology to directly determine the sampling points of plane-sweep algorithms in image space.
The use of the perspective invariant cross-ratio allows us to derive the location of the sampling planes directly from the
image data. With this, we efficiently sample the scene space, achieving higher sampling density in areas which are close
to the camera and a lower density in distant regions. We evaluate our approach on a synthetic benchmark dataset for
quantitative evaluation and on a real-image dataset consisting of aerial imagery. The experiments reveal that an inverse
sampling achieves equal and better results than a linear sampling, with less sampling points and thus less runtime. Our
algorithm allows an online computation of depth maps for subsequences of five frames, provided that the relative poses
between all frames are given. |
url |
https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLII-2-W6/325/2017/isprs-archives-XLII-2-W6-325-2017.pdf |
work_keys_str_mv |
AT bruf determiningplanesweepsamplingpointsinimagespaceusingthecrossratioforimagebaseddepthestimation AT bruf determiningplanesweepsamplingpointsinimagespaceusingthecrossratioforimagebaseddepthestimation AT berdnuess determiningplanesweepsamplingpointsinimagespaceusingthecrossratioforimagebaseddepthestimation AT berdnuess determiningplanesweepsamplingpointsinimagespaceusingthecrossratioforimagebaseddepthestimation AT mweinmann determiningplanesweepsamplingpointsinimagespaceusingthecrossratioforimagebaseddepthestimation |
_version_ |
1725280611113893888 |