GLOBAL MESSAGE PASSING IN NETWORKS VIA TASK-DRIVEN RANDOM WALKS FOR SEMANTIC SEGMENTATION OF REMOTE SENSING IMAGES

The capability of globally modeling and reasoning about relations between image regions is crucial for complex scene understanding tasks such as semantic segmentation. Most current semantic segmentation methods fall back on deep convolutional neural networks (CNNs), while their use of convolutions w...

Full description

Bibliographic Details
Main Authors: L. Mou, Y. Hua, P. Jin, X. X. Zhu
Format: Article
Language:English
Published: Copernicus Publications 2020-08-01
Series:ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Online Access:https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/V-2-2020/533/2020/isprs-annals-V-2-2020-533-2020.pdf
id doaj-1cacc52923cb409ea1d927f5f342c064
record_format Article
spelling doaj-1cacc52923cb409ea1d927f5f342c0642020-11-25T01:56:07ZengCopernicus PublicationsISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences2194-90422194-90502020-08-01V-2-202053354010.5194/isprs-annals-V-2-2020-533-2020GLOBAL MESSAGE PASSING IN NETWORKS VIA TASK-DRIVEN RANDOM WALKS FOR SEMANTIC SEGMENTATION OF REMOTE SENSING IMAGESL. Mou0Y. Hua1Y. Hua2P. Jin3X. X. Zhu4X. X. Zhu5Remote Sensing Technology Institute (IMF), German Aerospace Center (DLR), Wessling, GermanyRemote Sensing Technology Institute (IMF), German Aerospace Center (DLR), Wessling, GermanySignal Processing in Earth Observation, Technical University of Munich (TUM), Munich, GermanySignal Processing in Earth Observation, Technical University of Munich (TUM), Munich, GermanyRemote Sensing Technology Institute (IMF), German Aerospace Center (DLR), Wessling, GermanySignal Processing in Earth Observation, Technical University of Munich (TUM), Munich, GermanyThe capability of globally modeling and reasoning about relations between image regions is crucial for complex scene understanding tasks such as semantic segmentation. Most current semantic segmentation methods fall back on deep convolutional neural networks (CNNs), while their use of convolutions with local receptive fields is typically inefficient at capturing long-range dependencies. Recent works on self-attention mechanisms and relational reasoning networks seek to address this issue by learning pairwise relations between each two entities and have showcased promising results. But such approaches have heavy computational and memory overheads, which is computationally infeasible for dense prediction tasks, particularly on large size images, i.e., aerial imagery. In this work, we propose an efficient method for global context modeling in which at each position, a sparse set of features, instead of all features, over the spatial domain are adaptively sampled and aggregated. We further devise a highly efficient instantiation of the proposed method, namely learning RANdom walK samplIng aNd feature aGgregation (RANKING). The proposed module is lightweight and general, which can be used in a plug-and-play fashion with the existing fully convolutional neural network (FCN) framework. To evaluate RANKING-equipped networks, we conduct experiments on two aerial scene parsing datasets, and the networks can achieve competitive results at significant low costs in terms of the computational and memory.https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/V-2-2020/533/2020/isprs-annals-V-2-2020-533-2020.pdf
collection DOAJ
language English
format Article
sources DOAJ
author L. Mou
Y. Hua
Y. Hua
P. Jin
X. X. Zhu
X. X. Zhu
spellingShingle L. Mou
Y. Hua
Y. Hua
P. Jin
X. X. Zhu
X. X. Zhu
GLOBAL MESSAGE PASSING IN NETWORKS VIA TASK-DRIVEN RANDOM WALKS FOR SEMANTIC SEGMENTATION OF REMOTE SENSING IMAGES
ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences
author_facet L. Mou
Y. Hua
Y. Hua
P. Jin
X. X. Zhu
X. X. Zhu
author_sort L. Mou
title GLOBAL MESSAGE PASSING IN NETWORKS VIA TASK-DRIVEN RANDOM WALKS FOR SEMANTIC SEGMENTATION OF REMOTE SENSING IMAGES
title_short GLOBAL MESSAGE PASSING IN NETWORKS VIA TASK-DRIVEN RANDOM WALKS FOR SEMANTIC SEGMENTATION OF REMOTE SENSING IMAGES
title_full GLOBAL MESSAGE PASSING IN NETWORKS VIA TASK-DRIVEN RANDOM WALKS FOR SEMANTIC SEGMENTATION OF REMOTE SENSING IMAGES
title_fullStr GLOBAL MESSAGE PASSING IN NETWORKS VIA TASK-DRIVEN RANDOM WALKS FOR SEMANTIC SEGMENTATION OF REMOTE SENSING IMAGES
title_full_unstemmed GLOBAL MESSAGE PASSING IN NETWORKS VIA TASK-DRIVEN RANDOM WALKS FOR SEMANTIC SEGMENTATION OF REMOTE SENSING IMAGES
title_sort global message passing in networks via task-driven random walks for semantic segmentation of remote sensing images
publisher Copernicus Publications
series ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences
issn 2194-9042
2194-9050
publishDate 2020-08-01
description The capability of globally modeling and reasoning about relations between image regions is crucial for complex scene understanding tasks such as semantic segmentation. Most current semantic segmentation methods fall back on deep convolutional neural networks (CNNs), while their use of convolutions with local receptive fields is typically inefficient at capturing long-range dependencies. Recent works on self-attention mechanisms and relational reasoning networks seek to address this issue by learning pairwise relations between each two entities and have showcased promising results. But such approaches have heavy computational and memory overheads, which is computationally infeasible for dense prediction tasks, particularly on large size images, i.e., aerial imagery. In this work, we propose an efficient method for global context modeling in which at each position, a sparse set of features, instead of all features, over the spatial domain are adaptively sampled and aggregated. We further devise a highly efficient instantiation of the proposed method, namely learning RANdom walK samplIng aNd feature aGgregation (RANKING). The proposed module is lightweight and general, which can be used in a plug-and-play fashion with the existing fully convolutional neural network (FCN) framework. To evaluate RANKING-equipped networks, we conduct experiments on two aerial scene parsing datasets, and the networks can achieve competitive results at significant low costs in terms of the computational and memory.
url https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/V-2-2020/533/2020/isprs-annals-V-2-2020-533-2020.pdf
work_keys_str_mv AT lmou globalmessagepassinginnetworksviataskdrivenrandomwalksforsemanticsegmentationofremotesensingimages
AT yhua globalmessagepassinginnetworksviataskdrivenrandomwalksforsemanticsegmentationofremotesensingimages
AT yhua globalmessagepassinginnetworksviataskdrivenrandomwalksforsemanticsegmentationofremotesensingimages
AT pjin globalmessagepassinginnetworksviataskdrivenrandomwalksforsemanticsegmentationofremotesensingimages
AT xxzhu globalmessagepassinginnetworksviataskdrivenrandomwalksforsemanticsegmentationofremotesensingimages
AT xxzhu globalmessagepassinginnetworksviataskdrivenrandomwalksforsemanticsegmentationofremotesensingimages
_version_ 1724981439598952448