Comparison of neural closure models for discretised PDEs

• Main Authors: Crommelin, D. (Author), Koren, B. (Author), Melchers, H. (Author), Menkovski, V. (Author), Sanderse, B. (Author)
• Format: Article
• Language: English
• Published: Elsevier Ltd 2023
• Subjects: Accurate modeling; Closure model; Closure models; Loss functions; Multiscale modeling; Multiscale modelling; Multi-scale system; Neural networks; Neural ODE; Neural-networks; Ordinary differential equations; Partial differential equations; Simple++; Small scale; Training procedures; Trajectories
• Online Access: View Fulltext in Publisher; View in Scopus
• Physical Description: 14
• ISBN: 08981221 (ISSN)
• DOI: 10.1016/j.camwa.2023.04.030

Neural closure models have recently been proposed as a method for efficiently approximating small scales in multiscale systems with neural networks. The choice of loss function and associated training procedure has a large effect on the accuracy and stability of the resulting neural closure model. In this work, we systematically compare three distinct procedures: “derivative fitting”, “trajectory fitting” with discretise-then-optimise, and “trajectory fitting” with optimise-then-discretise. Derivative fitting is conceptually the simplest and computationally the most efficient approach and is found to perform reasonably well on one of the test problems (Kuramoto-Sivashinsky) but poorly on the other (Burgers). Trajectory fitting is computationally more expensive but is more robust and is therefore the preferred approach. Of the two trajectory fitting procedures, the discretise-then-optimise approach produces more accurate models than the optimise-then-discretise approach. While the optimise-then-discretise approach can still produce accurate models, care must be taken in choosing the length of the trajectories used for training, in order to train the models on long-term behaviour while still producing reasonably accurate gradients during training. Two existing theorems are interpreted in a novel way that gives insight into the long-term accuracy of a neural closure model based on how accurate it is in the short term. © 2023 The Author(s)