A run control framework to streamline profiling, porting, and tuning simulation runs and provenance tracking of geoscientific applications
<p>Geoscientific modeling is constantly evolving, with next-generation geoscientific models and applications placing large demands on high-performance computing (HPC) resources. These demands are being met by new developments in HPC architectures, software libraries, and infrastructures. In...
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Copernicus Publications
2018-07-01
|
| Series: | Geoscientific Model Development |
| Online Access: | https://www.geosci-model-dev.net/11/2875/2018/gmd-11-2875-2018.pdf |
| Summary: | <p>Geoscientific modeling is constantly evolving, with next-generation geoscientific models and applications placing large demands on
high-performance computing (HPC) resources. These demands are being met by
new developments in HPC architectures, software libraries, and
infrastructures. In addition to the challenge of new massively parallel HPC
systems, reproducibility of simulation and analysis results is of great
concern. This is due to the fact that next-generation geoscientific models
are based on complex model implementations and profiling, modeling, and data
processing workflows. Thus, in order to reduce both the duration and the cost
of code migration, aid in the development of new models or model components,
while ensuring reproducibility and sustainability over the complete data life
cycle, an automated approach to profiling, porting, and provenance tracking
is necessary. We propose a run control framework (RCF) integrated with a
workflow engine as a best practice approach to automate profiling, porting,
provenance tracking, and simulation runs. Our RCF encompasses all stages of
the modeling chain: (1) preprocess input, (2) compilation of code (including
code instrumentation with performance analysis tools), (3) simulation run,
and
(4) postprocessing and analysis, to address these issues. Within this RCF, the
workflow engine is used to create and manage benchmark or simulation
parameter combinations and performs the documentation and data organization
for reproducibility. In this study, we outline this approach and highlight
the subsequent developments scheduled for implementation born out of the
extensive profiling of ParFlow. We show that in using our run control
framework, testing, benchmarking, profiling, and running models is less time
consuming and more robust than running geoscientific applications in an ad
hoc fashion, resulting in more efficient use of HPC resources, more strategic
code development, and enhanced data integrity and reproducibility.</p> |
|---|---|
| ISSN: | 1991-959X 1991-9603 |