Deriving Non-Cloud Contaminated Sentinel-2 Images with RGB and Near-Infrared Bands from Sentinel-1 Images Based on a Conditional Generative Adversarial Network

Deriving Non-Cloud Contaminated Sentinel-2 Images with RGB and Near-Infrared Bands from Sentinel-1 Images Based on a Conditional Generative Adversarial Network

Sentinel-2 images have been widely used in studying land surface phenomena and processes, but they inevitably suffer from cloud contamination. To solve this critical optical data availability issue, it is ideal to fuse Sentinel-1 and Sentinel-2 images to create fused, cloud-free Sentinel-2-like imag...

Full description

Bibliographic Details
Main Authors: Quan Xiong, Liping Di, Quanlong Feng, Diyou Liu, Wei Liu, Xuli Zan, Lin Zhang, Dehai Zhu, Zhe Liu, Xiaochuang Yao, Xiaodong Zhang
Format: Article
Language:English
Published: MDPI AG 2021-04-01
Series:Remote Sensing
Subjects:
Sentinel-1
Sentinel-2
generative adversarial network
non-cloud contamination
data fusion
Online Access:https://www.mdpi.com/2072-4292/13/8/1512
id doaj-5d93d65bd1b94b9eabe305e64b494977
record_format Article
spelling doaj-5d93d65bd1b94b9eabe305e64b4949772021-04-14T23:03:58ZengMDPI AGRemote Sensing2072-42922021-04-01131512151210.3390/rs13081512Deriving Non-Cloud Contaminated Sentinel-2 Images with RGB and Near-Infrared Bands from Sentinel-1 Images Based on a Conditional Generative Adversarial NetworkQuan Xiong0Liping Di1Quanlong Feng2Diyou Liu3Wei Liu4Xuli Zan5Lin Zhang6Dehai Zhu7Zhe Liu8Xiaochuang Yao9Xiaodong Zhang10College of Land Science and Technology, China Agricultural University, Beijing 100083, ChinaCenter for Spatial Information Science and Systems, George Mason University, 4400 University Dr., Fairfax, VA 22030, USACollege of Land Science and Technology, China Agricultural University, Beijing 100083, ChinaCollege of Land Science and Technology, China Agricultural University, Beijing 100083, ChinaCollege of Land Science and Technology, China Agricultural University, Beijing 100083, ChinaCollege of Land Science and Technology, China Agricultural University, Beijing 100083, ChinaCollege of Land Science and Technology, China Agricultural University, Beijing 100083, ChinaCollege of Land Science and Technology, China Agricultural University, Beijing 100083, ChinaCollege of Land Science and Technology, China Agricultural University, Beijing 100083, ChinaCollege of Land Science and Technology, China Agricultural University, Beijing 100083, ChinaCollege of Land Science and Technology, China Agricultural University, Beijing 100083, ChinaSentinel-2 images have been widely used in studying land surface phenomena and processes, but they inevitably suffer from cloud contamination. To solve this critical optical data availability issue, it is ideal to fuse Sentinel-1 and Sentinel-2 images to create fused, cloud-free Sentinel-2-like images for facilitating land surface applications. In this paper, we propose a new data fusion model, the Multi-channels Conditional Generative Adversarial Network (MCcGAN), based on the conditional generative adversarial network, which is able to convert images from Domain A to Domain B. With the model, we were able to generate fused, cloud-free Sentinel-2-like images for a target date by using a pair of reference Sentinel-1/Sentinel-2 images and target-date Sentinel-1 images as inputs. In order to demonstrate the superiority of our method, we also compared it with other state-of-the-art methods using the same data. To make the evaluation more objective and reliable, we calculated the root-mean-square-error (RSME), R<inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msup><mrow></mrow><mn>2</mn></msup></semantics></math></inline-formula>, Kling–Gupta efficiency (KGE), structural similarity index (SSIM), spectral angle mapper (SAM), and peak signal-to-noise ratio (PSNR) of the simulated Sentinel-2 images generated by different methods. The results show that the simulated Sentinel-2 images generated by the MCcGAN have a higher quality and accuracy than those produced via the previous methods.https://www.mdpi.com/2072-4292/13/8/1512Sentinel-1Sentinel-2generative adversarial networknon-cloud contaminationdata fusion
collection DOAJ
language English
format Article
sources DOAJ
author Quan Xiong
Liping Di
Quanlong Feng
Diyou Liu
Wei Liu
Xuli Zan
Lin Zhang
Dehai Zhu
Zhe Liu
Xiaochuang Yao
Xiaodong Zhang
spellingShingle Quan Xiong
Liping Di
Quanlong Feng
Diyou Liu
Wei Liu
Xuli Zan
Lin Zhang
Dehai Zhu
Zhe Liu
Xiaochuang Yao
Xiaodong Zhang
Deriving Non-Cloud Contaminated Sentinel-2 Images with RGB and Near-Infrared Bands from Sentinel-1 Images Based on a Conditional Generative Adversarial Network
Remote Sensing
Sentinel-1
Sentinel-2
generative adversarial network
non-cloud contamination
data fusion
author_facet Quan Xiong
Liping Di
Quanlong Feng
Diyou Liu
Wei Liu
Xuli Zan
Lin Zhang
Dehai Zhu
Zhe Liu
Xiaochuang Yao
Xiaodong Zhang
author_sort Quan Xiong
title Deriving Non-Cloud Contaminated Sentinel-2 Images with RGB and Near-Infrared Bands from Sentinel-1 Images Based on a Conditional Generative Adversarial Network
title_short Deriving Non-Cloud Contaminated Sentinel-2 Images with RGB and Near-Infrared Bands from Sentinel-1 Images Based on a Conditional Generative Adversarial Network
title_full Deriving Non-Cloud Contaminated Sentinel-2 Images with RGB and Near-Infrared Bands from Sentinel-1 Images Based on a Conditional Generative Adversarial Network
title_fullStr Deriving Non-Cloud Contaminated Sentinel-2 Images with RGB and Near-Infrared Bands from Sentinel-1 Images Based on a Conditional Generative Adversarial Network
title_full_unstemmed Deriving Non-Cloud Contaminated Sentinel-2 Images with RGB and Near-Infrared Bands from Sentinel-1 Images Based on a Conditional Generative Adversarial Network
title_sort deriving non-cloud contaminated sentinel-2 images with rgb and near-infrared bands from sentinel-1 images based on a conditional generative adversarial network
publisher MDPI AG
series Remote Sensing
issn 2072-4292
publishDate 2021-04-01
description Sentinel-2 images have been widely used in studying land surface phenomena and processes, but they inevitably suffer from cloud contamination. To solve this critical optical data availability issue, it is ideal to fuse Sentinel-1 and Sentinel-2 images to create fused, cloud-free Sentinel-2-like images for facilitating land surface applications. In this paper, we propose a new data fusion model, the Multi-channels Conditional Generative Adversarial Network (MCcGAN), based on the conditional generative adversarial network, which is able to convert images from Domain A to Domain B. With the model, we were able to generate fused, cloud-free Sentinel-2-like images for a target date by using a pair of reference Sentinel-1/Sentinel-2 images and target-date Sentinel-1 images as inputs. In order to demonstrate the superiority of our method, we also compared it with other state-of-the-art methods using the same data. To make the evaluation more objective and reliable, we calculated the root-mean-square-error (RSME), R<inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msup><mrow></mrow><mn>2</mn></msup></semantics></math></inline-formula>, Kling–Gupta efficiency (KGE), structural similarity index (SSIM), spectral angle mapper (SAM), and peak signal-to-noise ratio (PSNR) of the simulated Sentinel-2 images generated by different methods. The results show that the simulated Sentinel-2 images generated by the MCcGAN have a higher quality and accuracy than those produced via the previous methods.
topic Sentinel-1
Sentinel-2
generative adversarial network
non-cloud contamination
data fusion
url https://www.mdpi.com/2072-4292/13/8/1512
work_keys_str_mv AT quanxiong derivingnoncloudcontaminatedsentinel2imageswithrgbandnearinfraredbandsfromsentinel1imagesbasedonaconditionalgenerativeadversarialnetwork
AT lipingdi derivingnoncloudcontaminatedsentinel2imageswithrgbandnearinfraredbandsfromsentinel1imagesbasedonaconditionalgenerativeadversarialnetwork
AT quanlongfeng derivingnoncloudcontaminatedsentinel2imageswithrgbandnearinfraredbandsfromsentinel1imagesbasedonaconditionalgenerativeadversarialnetwork
AT diyouliu derivingnoncloudcontaminatedsentinel2imageswithrgbandnearinfraredbandsfromsentinel1imagesbasedonaconditionalgenerativeadversarialnetwork
AT weiliu derivingnoncloudcontaminatedsentinel2imageswithrgbandnearinfraredbandsfromsentinel1imagesbasedonaconditionalgenerativeadversarialnetwork
AT xulizan derivingnoncloudcontaminatedsentinel2imageswithrgbandnearinfraredbandsfromsentinel1imagesbasedonaconditionalgenerativeadversarialnetwork
AT linzhang derivingnoncloudcontaminatedsentinel2imageswithrgbandnearinfraredbandsfromsentinel1imagesbasedonaconditionalgenerativeadversarialnetwork
AT dehaizhu derivingnoncloudcontaminatedsentinel2imageswithrgbandnearinfraredbandsfromsentinel1imagesbasedonaconditionalgenerativeadversarialnetwork
AT zheliu derivingnoncloudcontaminatedsentinel2imageswithrgbandnearinfraredbandsfromsentinel1imagesbasedonaconditionalgenerativeadversarialnetwork
AT xiaochuangyao derivingnoncloudcontaminatedsentinel2imageswithrgbandnearinfraredbandsfromsentinel1imagesbasedonaconditionalgenerativeadversarialnetwork
AT xiaodongzhang derivingnoncloudcontaminatedsentinel2imageswithrgbandnearinfraredbandsfromsentinel1imagesbasedonaconditionalgenerativeadversarialnetwork
_version_ 1721526790607863808

Similar Items