Recommendations on benchmarks for numerical air quality model applications in China – Part 1: PM<sub>2.5</sub> and chemical species

<p>Numerical air quality models (AQMs) have been applied more frequently over the past decade to address diverse scientific and regulatory issues associated with deteriorated air quality in China. Thorough evaluation of a model's ability to replicate monitored conditions (i.e., a model pe...

Full description

Bibliographic Details
Main Authors: L. Huang, Y. Zhu, H. Zhai, S. Xue, T. Zhu, Y. Shao, Z. Liu, C. Emery, G. Yarwood, Y. Wang, J. Fu, K. Zhang, L. Li
Format: Article
Language:English
Published: Copernicus Publications 2021-02-01
Series:Atmospheric Chemistry and Physics
Online Access:https://acp.copernicus.org/articles/21/2725/2021/acp-21-2725-2021.pdf
id doaj-924bda534d714edcbf973bc55fea4cc5
record_format Article
collection DOAJ
language English
format Article
sources DOAJ
author L. Huang
Y. Zhu
H. Zhai
S. Xue
T. Zhu
Y. Shao
Z. Liu
C. Emery
G. Yarwood
Y. Wang
J. Fu
K. Zhang
L. Li
spellingShingle L. Huang
Y. Zhu
H. Zhai
S. Xue
T. Zhu
Y. Shao
Z. Liu
C. Emery
G. Yarwood
Y. Wang
J. Fu
K. Zhang
L. Li
Recommendations on benchmarks for numerical air quality model applications in China – Part 1: PM<sub>2.5</sub> and chemical species
Atmospheric Chemistry and Physics
author_facet L. Huang
Y. Zhu
H. Zhai
S. Xue
T. Zhu
Y. Shao
Z. Liu
C. Emery
G. Yarwood
Y. Wang
J. Fu
K. Zhang
L. Li
author_sort L. Huang
title Recommendations on benchmarks for numerical air quality model applications in China – Part 1: PM<sub>2.5</sub> and chemical species
title_short Recommendations on benchmarks for numerical air quality model applications in China – Part 1: PM<sub>2.5</sub> and chemical species
title_full Recommendations on benchmarks for numerical air quality model applications in China – Part 1: PM<sub>2.5</sub> and chemical species
title_fullStr Recommendations on benchmarks for numerical air quality model applications in China – Part 1: PM<sub>2.5</sub> and chemical species
title_full_unstemmed Recommendations on benchmarks for numerical air quality model applications in China – Part 1: PM<sub>2.5</sub> and chemical species
title_sort recommendations on benchmarks for numerical air quality model applications in china – part 1: pm<sub>2.5</sub> and chemical species
publisher Copernicus Publications
series Atmospheric Chemistry and Physics
issn 1680-7316
1680-7324
publishDate 2021-02-01
description <p>Numerical air quality models (AQMs) have been applied more frequently over the past decade to address diverse scientific and regulatory issues associated with deteriorated air quality in China. Thorough evaluation of a model's ability to replicate monitored conditions (i.e., a model performance evaluation or MPE) helps to illuminate the robustness and reliability of the baseline modeling results and subsequent analyses. However, with numerous input data requirements, diverse model configurations, and the scientific evolution of the models themselves, no two AQM applications are the same and their performance results should be expected to differ. MPE procedures have been developed for Europe and North America, but there is currently no uniform set of MPE procedures and associated benchmarks for China. Here we present an extensive review of model performance for fine particulate matter (PM<span class="inline-formula"><sub>2.5</sub></span>) AQM applications to China and, from this context, propose a set of statistical benchmarks that can be used to objectively evaluate model performance for PM<span class="inline-formula"><sub>2.5</sub></span> AQM applications in China. We compiled MPE results from 307 peer-reviewed articles published between 2006 and 2019, which applied five of the most frequently used AQMs in China. We analyze influences on the range of reported statistics from different model configurations, including modeling regions and seasons, spatial resolution of modeling grids, temporal resolution of the MPE, etc. Analysis using a random forest method shows that the choices of emission inventory, grid resolution, and aerosol- and gas-phase chemistry are the top three factors affecting model performance for PM<span class="inline-formula"><sub>2.5</sub></span>. We propose benchmarks for six frequently used evaluation metrics for AQM applications in China, including two tiers – “goals” and “criteria” – where goals represent the best model performance that a model is currently expected to achieve and criteria represent the model performance that the majority of studies can meet. Our results formed a benchmark framework for the modeling performance of PM<span class="inline-formula"><sub>2.5</sub></span> and its chemical species in China. For instance, in order to meet the goal and criteria, the normalized mean bias (NMB) for total PM<span class="inline-formula"><sub>2.5</sub></span> should be within 10 % and 20 %, while the normalized mean error (NME) should be within 35 % and 45 %, respectively. The goal and criteria values of correlation coefficients for evaluating hourly and daily PM<span class="inline-formula"><sub>2.5</sub></span> are 0.70 and 0.60, respectively; corresponding values are higher when the index of agreement (IOA) is used (0.80 for goal and 0.70 for criteria). Results from this study will support the ever-growing modeling community in China by providing a more objective assessment and context for how well their results compare with previous studies and to better demonstrate the credibility and robustness of their AQM applications prior to subsequent regulatory assessments.</p>
url https://acp.copernicus.org/articles/21/2725/2021/acp-21-2725-2021.pdf
work_keys_str_mv AT lhuang recommendationsonbenchmarksfornumericalairqualitymodelapplicationsinchinapart1pmsub25subandchemicalspecies
AT yzhu recommendationsonbenchmarksfornumericalairqualitymodelapplicationsinchinapart1pmsub25subandchemicalspecies
AT hzhai recommendationsonbenchmarksfornumericalairqualitymodelapplicationsinchinapart1pmsub25subandchemicalspecies
AT sxue recommendationsonbenchmarksfornumericalairqualitymodelapplicationsinchinapart1pmsub25subandchemicalspecies
AT tzhu recommendationsonbenchmarksfornumericalairqualitymodelapplicationsinchinapart1pmsub25subandchemicalspecies
AT yshao recommendationsonbenchmarksfornumericalairqualitymodelapplicationsinchinapart1pmsub25subandchemicalspecies
AT zliu recommendationsonbenchmarksfornumericalairqualitymodelapplicationsinchinapart1pmsub25subandchemicalspecies
AT cemery recommendationsonbenchmarksfornumericalairqualitymodelapplicationsinchinapart1pmsub25subandchemicalspecies
AT gyarwood recommendationsonbenchmarksfornumericalairqualitymodelapplicationsinchinapart1pmsub25subandchemicalspecies
AT ywang recommendationsonbenchmarksfornumericalairqualitymodelapplicationsinchinapart1pmsub25subandchemicalspecies
AT jfu recommendationsonbenchmarksfornumericalairqualitymodelapplicationsinchinapart1pmsub25subandchemicalspecies
AT kzhang recommendationsonbenchmarksfornumericalairqualitymodelapplicationsinchinapart1pmsub25subandchemicalspecies
AT lli recommendationsonbenchmarksfornumericalairqualitymodelapplicationsinchinapart1pmsub25subandchemicalspecies
_version_ 1724253112137416704
spelling doaj-924bda534d714edcbf973bc55fea4cc52021-02-24T07:45:09ZengCopernicus PublicationsAtmospheric Chemistry and Physics1680-73161680-73242021-02-01212725274310.5194/acp-21-2725-2021Recommendations on benchmarks for numerical air quality model applications in China – Part 1: PM<sub>2.5</sub> and chemical speciesL. Huang0Y. Zhu1H. Zhai2S. Xue3T. Zhu4Y. Shao5Z. Liu6C. Emery7G. Yarwood8Y. Wang9J. Fu10K. Zhang11L. Li12School of Environmental and Chemical Engineering, Shanghai University, Shanghai, 200444, ChinaSchool of Environmental and Chemical Engineering, Shanghai University, Shanghai, 200444, ChinaSchool of Environmental and Chemical Engineering, Shanghai University, Shanghai, 200444, ChinaSchool of Environmental and Chemical Engineering, Shanghai University, Shanghai, 200444, ChinaSchool of Environmental and Chemical Engineering, Shanghai University, Shanghai, 200444, ChinaSchool of Environmental and Chemical Engineering, Shanghai University, Shanghai, 200444, ChinaSchool of Environmental and Chemical Engineering, Shanghai University, Shanghai, 200444, ChinaRamboll, Novato, CA 94945, USARamboll, Novato, CA 94945, USASchool of Environmental and Chemical Engineering, Shanghai University, Shanghai, 200444, ChinaDepartment of Civil and Environmental Engineering, University of Tennessee, Knoxville, TN 37996, USASchool of Environmental and Chemical Engineering, Shanghai University, Shanghai, 200444, ChinaSchool of Environmental and Chemical Engineering, Shanghai University, Shanghai, 200444, China<p>Numerical air quality models (AQMs) have been applied more frequently over the past decade to address diverse scientific and regulatory issues associated with deteriorated air quality in China. Thorough evaluation of a model's ability to replicate monitored conditions (i.e., a model performance evaluation or MPE) helps to illuminate the robustness and reliability of the baseline modeling results and subsequent analyses. However, with numerous input data requirements, diverse model configurations, and the scientific evolution of the models themselves, no two AQM applications are the same and their performance results should be expected to differ. MPE procedures have been developed for Europe and North America, but there is currently no uniform set of MPE procedures and associated benchmarks for China. Here we present an extensive review of model performance for fine particulate matter (PM<span class="inline-formula"><sub>2.5</sub></span>) AQM applications to China and, from this context, propose a set of statistical benchmarks that can be used to objectively evaluate model performance for PM<span class="inline-formula"><sub>2.5</sub></span> AQM applications in China. We compiled MPE results from 307 peer-reviewed articles published between 2006 and 2019, which applied five of the most frequently used AQMs in China. We analyze influences on the range of reported statistics from different model configurations, including modeling regions and seasons, spatial resolution of modeling grids, temporal resolution of the MPE, etc. Analysis using a random forest method shows that the choices of emission inventory, grid resolution, and aerosol- and gas-phase chemistry are the top three factors affecting model performance for PM<span class="inline-formula"><sub>2.5</sub></span>. We propose benchmarks for six frequently used evaluation metrics for AQM applications in China, including two tiers – “goals” and “criteria” – where goals represent the best model performance that a model is currently expected to achieve and criteria represent the model performance that the majority of studies can meet. Our results formed a benchmark framework for the modeling performance of PM<span class="inline-formula"><sub>2.5</sub></span> and its chemical species in China. For instance, in order to meet the goal and criteria, the normalized mean bias (NMB) for total PM<span class="inline-formula"><sub>2.5</sub></span> should be within 10 % and 20 %, while the normalized mean error (NME) should be within 35 % and 45 %, respectively. The goal and criteria values of correlation coefficients for evaluating hourly and daily PM<span class="inline-formula"><sub>2.5</sub></span> are 0.70 and 0.60, respectively; corresponding values are higher when the index of agreement (IOA) is used (0.80 for goal and 0.70 for criteria). Results from this study will support the ever-growing modeling community in China by providing a more objective assessment and context for how well their results compare with previous studies and to better demonstrate the credibility and robustness of their AQM applications prior to subsequent regulatory assessments.</p>https://acp.copernicus.org/articles/21/2725/2021/acp-21-2725-2021.pdf