GLOBAL MESSAGE PASSING IN NETWORKS VIA TASK-DRIVEN RANDOM WALKS FOR SEMANTIC SEGMENTATION OF REMOTE SENSING IMAGES
The capability of globally modeling and reasoning about relations between image regions is crucial for complex scene understanding tasks such as semantic segmentation. Most current semantic segmentation methods fall back on deep convolutional neural networks (CNNs), while their use of convolutions w...
Main Authors: | , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
Copernicus Publications
2020-08-01
|
Series: | ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
Online Access: | https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/V-2-2020/533/2020/isprs-annals-V-2-2020-533-2020.pdf |
id |
doaj-1cacc52923cb409ea1d927f5f342c064 |
---|---|
record_format |
Article |
spelling |
doaj-1cacc52923cb409ea1d927f5f342c0642020-11-25T01:56:07ZengCopernicus PublicationsISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences2194-90422194-90502020-08-01V-2-202053354010.5194/isprs-annals-V-2-2020-533-2020GLOBAL MESSAGE PASSING IN NETWORKS VIA TASK-DRIVEN RANDOM WALKS FOR SEMANTIC SEGMENTATION OF REMOTE SENSING IMAGESL. Mou0Y. Hua1Y. Hua2P. Jin3X. X. Zhu4X. X. Zhu5Remote Sensing Technology Institute (IMF), German Aerospace Center (DLR), Wessling, GermanyRemote Sensing Technology Institute (IMF), German Aerospace Center (DLR), Wessling, GermanySignal Processing in Earth Observation, Technical University of Munich (TUM), Munich, GermanySignal Processing in Earth Observation, Technical University of Munich (TUM), Munich, GermanyRemote Sensing Technology Institute (IMF), German Aerospace Center (DLR), Wessling, GermanySignal Processing in Earth Observation, Technical University of Munich (TUM), Munich, GermanyThe capability of globally modeling and reasoning about relations between image regions is crucial for complex scene understanding tasks such as semantic segmentation. Most current semantic segmentation methods fall back on deep convolutional neural networks (CNNs), while their use of convolutions with local receptive fields is typically inefficient at capturing long-range dependencies. Recent works on self-attention mechanisms and relational reasoning networks seek to address this issue by learning pairwise relations between each two entities and have showcased promising results. But such approaches have heavy computational and memory overheads, which is computationally infeasible for dense prediction tasks, particularly on large size images, i.e., aerial imagery. In this work, we propose an efficient method for global context modeling in which at each position, a sparse set of features, instead of all features, over the spatial domain are adaptively sampled and aggregated. We further devise a highly efficient instantiation of the proposed method, namely learning RANdom walK samplIng aNd feature aGgregation (RANKING). The proposed module is lightweight and general, which can be used in a plug-and-play fashion with the existing fully convolutional neural network (FCN) framework. To evaluate RANKING-equipped networks, we conduct experiments on two aerial scene parsing datasets, and the networks can achieve competitive results at significant low costs in terms of the computational and memory.https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/V-2-2020/533/2020/isprs-annals-V-2-2020-533-2020.pdf |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
L. Mou Y. Hua Y. Hua P. Jin X. X. Zhu X. X. Zhu |
spellingShingle |
L. Mou Y. Hua Y. Hua P. Jin X. X. Zhu X. X. Zhu GLOBAL MESSAGE PASSING IN NETWORKS VIA TASK-DRIVEN RANDOM WALKS FOR SEMANTIC SEGMENTATION OF REMOTE SENSING IMAGES ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
author_facet |
L. Mou Y. Hua Y. Hua P. Jin X. X. Zhu X. X. Zhu |
author_sort |
L. Mou |
title |
GLOBAL MESSAGE PASSING IN NETWORKS VIA TASK-DRIVEN RANDOM WALKS FOR SEMANTIC SEGMENTATION OF REMOTE SENSING IMAGES |
title_short |
GLOBAL MESSAGE PASSING IN NETWORKS VIA TASK-DRIVEN RANDOM WALKS FOR SEMANTIC SEGMENTATION OF REMOTE SENSING IMAGES |
title_full |
GLOBAL MESSAGE PASSING IN NETWORKS VIA TASK-DRIVEN RANDOM WALKS FOR SEMANTIC SEGMENTATION OF REMOTE SENSING IMAGES |
title_fullStr |
GLOBAL MESSAGE PASSING IN NETWORKS VIA TASK-DRIVEN RANDOM WALKS FOR SEMANTIC SEGMENTATION OF REMOTE SENSING IMAGES |
title_full_unstemmed |
GLOBAL MESSAGE PASSING IN NETWORKS VIA TASK-DRIVEN RANDOM WALKS FOR SEMANTIC SEGMENTATION OF REMOTE SENSING IMAGES |
title_sort |
global message passing in networks via task-driven random walks for semantic segmentation of remote sensing images |
publisher |
Copernicus Publications |
series |
ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
issn |
2194-9042 2194-9050 |
publishDate |
2020-08-01 |
description |
The capability of globally modeling and reasoning about relations between image regions is crucial for complex scene understanding tasks such as semantic segmentation. Most current semantic segmentation methods fall back on deep convolutional neural networks (CNNs), while their use of convolutions with local receptive fields is typically inefficient at capturing long-range dependencies. Recent works on self-attention mechanisms and relational reasoning networks seek to address this issue by learning pairwise relations between each two entities and have showcased promising results. But such approaches have heavy computational and memory overheads, which is computationally infeasible for dense prediction tasks, particularly on large size images, i.e., aerial imagery. In this work, we propose an efficient method for global context modeling in which at each position, a sparse set of features, instead of all features, over the spatial domain are adaptively sampled and aggregated. We further devise a highly efficient instantiation of the proposed method, namely learning RANdom walK samplIng aNd feature aGgregation (RANKING). The proposed module is lightweight and general, which can be used in a plug-and-play fashion with the existing fully convolutional neural network (FCN) framework. To evaluate RANKING-equipped networks, we conduct experiments on two aerial scene parsing datasets, and the networks can achieve competitive results at significant low costs in terms of the computational and memory. |
url |
https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/V-2-2020/533/2020/isprs-annals-V-2-2020-533-2020.pdf |
work_keys_str_mv |
AT lmou globalmessagepassinginnetworksviataskdrivenrandomwalksforsemanticsegmentationofremotesensingimages AT yhua globalmessagepassinginnetworksviataskdrivenrandomwalksforsemanticsegmentationofremotesensingimages AT yhua globalmessagepassinginnetworksviataskdrivenrandomwalksforsemanticsegmentationofremotesensingimages AT pjin globalmessagepassinginnetworksviataskdrivenrandomwalksforsemanticsegmentationofremotesensingimages AT xxzhu globalmessagepassinginnetworksviataskdrivenrandomwalksforsemanticsegmentationofremotesensingimages AT xxzhu globalmessagepassinginnetworksviataskdrivenrandomwalksforsemanticsegmentationofremotesensingimages |
_version_ |
1724981439598952448 |