Geoscientific Model Development (Oct 2020)
Optimizing high-resolution Community Earth System Model on a heterogeneous many-core supercomputing platform
- S. Zhang,
- S. Zhang,
- S. Zhang,
- H. Fu,
- H. Fu,
- H. Fu,
- L. Wu,
- L. Wu,
- Y. Li,
- H. Wang,
- H. Wang,
- H. Wang,
- Y. Zeng,
- X. Duan,
- X. Duan,
- W. Wan,
- L. Wang,
- Y. Zhuang,
- H. Meng,
- K. Xu,
- K. Xu,
- P. Xu,
- P. Xu,
- L. Gan,
- L. Gan,
- Z. Liu,
- Z. Liu,
- S. Wu,
- Y. Chen,
- H. Yu,
- S. Shi,
- L. Wang,
- L. Wang,
- S. Xu,
- W. Xue,
- W. Xue,
- W. Liu,
- W. Liu,
- Q. Guo,
- J. Zhang,
- G. Zhu,
- Y. Tu,
- J. Edwards,
- J. Edwards,
- A. Baker,
- A. Baker,
- J. Yong,
- M. Yuan,
- Y. Yu,
- Q. Zhang,
- Q. Zhang,
- Z. Liu,
- M. Li,
- M. Li,
- M. Li,
- D. Jia,
- G. Yang,
- G. Yang,
- G. Yang,
- Z. Wei,
- J. Pan,
- P. Chang,
- P. Chang,
- G. Danabasoglu,
- G. Danabasoglu,
- S. Yeager,
- S. Yeager,
- N. Rosenbloom,
- N. Rosenbloom,
- Y. Guo
Affiliations
- S. Zhang
- International Laboratory for High-Resolution Earth System Model and Prediction (iHESP), Qingdao, China
- S. Zhang
- Laboratory for Ocean Dynamics and Climate, Qingdao Pilot National Laboratory for Marine Science and Technology, Qingdao, China
- S. Zhang
- Key Laboratory of Physical Oceanography, Institute for Advanced Ocean Study, Frontiers Science Center for Deep Ocean Multispheres and Earth System (FDOMES), College of Oceanic and Atmospheric Sciences, Ocean University of China, Qingdao, China
- H. Fu
- International Laboratory for High-Resolution Earth System Model and Prediction (iHESP), Qingdao, China
- H. Fu
- Ministry of Education Key Laboratory for Earth System Modeling, Department of Earth System Science, Tsinghua University, Beijing, China
- H. Fu
- National Supercomputing Center in Wuxi, Wuxi, China
- L. Wu
- Laboratory for Ocean Dynamics and Climate, Qingdao Pilot National Laboratory for Marine Science and Technology, Qingdao, China
- L. Wu
- Key Laboratory of Physical Oceanography, Institute for Advanced Ocean Study, Frontiers Science Center for Deep Ocean Multispheres and Earth System (FDOMES), College of Oceanic and Atmospheric Sciences, Ocean University of China, Qingdao, China
- Y. Li
- Department of Computer Science and Technology, Tsinghua University, Beijing, China
- H. Wang
- International Laboratory for High-Resolution Earth System Model and Prediction (iHESP), Qingdao, China
- H. Wang
- Laboratory for Ocean Dynamics and Climate, Qingdao Pilot National Laboratory for Marine Science and Technology, Qingdao, China
- H. Wang
- Key Laboratory of Physical Oceanography, Institute for Advanced Ocean Study, Frontiers Science Center for Deep Ocean Multispheres and Earth System (FDOMES), College of Oceanic and Atmospheric Sciences, Ocean University of China, Qingdao, China
- Y. Zeng
- Computer Science Center & National Supercomputer Center in Jinan, Jinan, China
- X. Duan
- National Supercomputing Center in Wuxi, Wuxi, China
- X. Duan
- School of Software, Shandong University, Jinan, China
- W. Wan
- National Supercomputing Center in Wuxi, Wuxi, China
- L. Wang
- Computer Science Center & National Supercomputer Center in Jinan, Jinan, China
- Y. Zhuang
- Computer Science Center & National Supercomputer Center in Jinan, Jinan, China
- H. Meng
- National Supercomputing Center in Wuxi, Wuxi, China
- K. Xu
- National Supercomputing Center in Wuxi, Wuxi, China
- K. Xu
- School of Software, Shandong University, Jinan, China
- P. Xu
- National Supercomputing Center in Wuxi, Wuxi, China
- P. Xu
- Department of Computer Science and Technology, Tsinghua University, Beijing, China
- L. Gan
- National Supercomputing Center in Wuxi, Wuxi, China
- L. Gan
- Department of Computer Science and Technology, Tsinghua University, Beijing, China
- Z. Liu
- National Supercomputing Center in Wuxi, Wuxi, China
- Z. Liu
- Department of Computer Science and Technology, Tsinghua University, Beijing, China
- S. Wu
- National Supercomputing Center in Wuxi, Wuxi, China
- Y. Chen
- Department of Supercomputing, Qingdao Pilot National Laboratory for Marine Science and Technology, Qingdao, China
- H. Yu
- National Supercomputing Center in Wuxi, Wuxi, China
- S. Shi
- National Supercomputing Center in Wuxi, Wuxi, China
- L. Wang
- National Supercomputing Center in Wuxi, Wuxi, China
- L. Wang
- College of Global Change and Earth System Science, Beijing Normal University, Beijing, China
- S. Xu
- Ministry of Education Key Laboratory for Earth System Modeling, Department of Earth System Science, Tsinghua University, Beijing, China
- W. Xue
- National Supercomputing Center in Wuxi, Wuxi, China
- W. Xue
- Department of Computer Science and Technology, Tsinghua University, Beijing, China
- W. Liu
- National Supercomputing Center in Wuxi, Wuxi, China
- W. Liu
- School of Software, Shandong University, Jinan, China
- Q. Guo
- Computer Science Center & National Supercomputer Center in Jinan, Jinan, China
- J. Zhang
- Computer Science Center & National Supercomputer Center in Jinan, Jinan, China
- G. Zhu
- Computer Science Center & National Supercomputer Center in Jinan, Jinan, China
- Y. Tu
- Computer Science Center & National Supercomputer Center in Jinan, Jinan, China
- J. Edwards
- International Laboratory for High-Resolution Earth System Model and Prediction (iHESP), Qingdao, China
- J. Edwards
- National Center for Atmospheric Research, Boulder, Colorado, USA
- A. Baker
- International Laboratory for High-Resolution Earth System Model and Prediction (iHESP), Qingdao, China
- A. Baker
- National Center for Atmospheric Research, Boulder, Colorado, USA
- J. Yong
- Key Laboratory of Physical Oceanography, Institute for Advanced Ocean Study, Frontiers Science Center for Deep Ocean Multispheres and Earth System (FDOMES), College of Oceanic and Atmospheric Sciences, Ocean University of China, Qingdao, China
- M. Yuan
- Key Laboratory of Physical Oceanography, Institute for Advanced Ocean Study, Frontiers Science Center for Deep Ocean Multispheres and Earth System (FDOMES), College of Oceanic and Atmospheric Sciences, Ocean University of China, Qingdao, China
- Y. Yu
- Key Laboratory of Physical Oceanography, Institute for Advanced Ocean Study, Frontiers Science Center for Deep Ocean Multispheres and Earth System (FDOMES), College of Oceanic and Atmospheric Sciences, Ocean University of China, Qingdao, China
- Q. Zhang
- International Laboratory for High-Resolution Earth System Model and Prediction (iHESP), Qingdao, China
- Q. Zhang
- Department of Oceanography, Texas A&M University, College Station, Texas, USA
- Z. Liu
- Department of Supercomputing, Qingdao Pilot National Laboratory for Marine Science and Technology, Qingdao, China
- M. Li
- International Laboratory for High-Resolution Earth System Model and Prediction (iHESP), Qingdao, China
- M. Li
- Laboratory for Ocean Dynamics and Climate, Qingdao Pilot National Laboratory for Marine Science and Technology, Qingdao, China
- M. Li
- Key Laboratory of Physical Oceanography, Institute for Advanced Ocean Study, Frontiers Science Center for Deep Ocean Multispheres and Earth System (FDOMES), College of Oceanic and Atmospheric Sciences, Ocean University of China, Qingdao, China
- D. Jia
- Department of Supercomputing, Qingdao Pilot National Laboratory for Marine Science and Technology, Qingdao, China
- G. Yang
- International Laboratory for High-Resolution Earth System Model and Prediction (iHESP), Qingdao, China
- G. Yang
- National Supercomputing Center in Wuxi, Wuxi, China
- G. Yang
- Department of Computer Science and Technology, Tsinghua University, Beijing, China
- Z. Wei
- Department of Supercomputing, Qingdao Pilot National Laboratory for Marine Science and Technology, Qingdao, China
- J. Pan
- Computer Science Center & National Supercomputer Center in Jinan, Jinan, China
- P. Chang
- International Laboratory for High-Resolution Earth System Model and Prediction (iHESP), Qingdao, China
- P. Chang
- Department of Oceanography, Texas A&M University, College Station, Texas, USA
- G. Danabasoglu
- International Laboratory for High-Resolution Earth System Model and Prediction (iHESP), Qingdao, China
- G. Danabasoglu
- National Center for Atmospheric Research, Boulder, Colorado, USA
- S. Yeager
- International Laboratory for High-Resolution Earth System Model and Prediction (iHESP), Qingdao, China
- S. Yeager
- National Center for Atmospheric Research, Boulder, Colorado, USA
- N. Rosenbloom
- International Laboratory for High-Resolution Earth System Model and Prediction (iHESP), Qingdao, China
- N. Rosenbloom
- National Center for Atmospheric Research, Boulder, Colorado, USA
- Y. Guo
- Computer Science Center & National Supercomputer Center in Jinan, Jinan, China
- DOI
- https://doi.org/10.5194/gmd-13-4809-2020
- Journal volume & issue
-
Vol. 13
pp. 4809 – 4829
Abstract
With semiconductor technology gradually approaching its physical and thermal limits, recent supercomputers have adopted major architectural changes to continue increasing the performance through more power-efficient heterogeneous many-core systems. Examples include Sunway TaihuLight that has four management processing elements (MPEs) and 256 computing processing elements (CPEs) inside one processor and Summit that has two central processing units (CPUs) and six graphics processing units (GPUs) inside one node. Meanwhile, current high-resolution Earth system models that desperately require more computing power generally consist of millions of lines of legacy code developed for traditional homogeneous multicore processors and cannot automatically benefit from the advancement of supercomputer hardware. As a result, refactoring and optimizing the legacy models for new architectures become key challenges along the road of taking advantage of greener and faster supercomputers, providing better support for the global climate research community and contributing to the long-lasting societal task of addressing long-term climate change. This article reports the efforts of a large group in the International Laboratory for High-Resolution Earth System Prediction (iHESP) that was established by the cooperation of Qingdao Pilot National Laboratory for Marine Science and Technology (QNLM), Texas A&M University (TAMU), and the National Center for Atmospheric Research (NCAR), with the goal of enabling highly efficient simulations of the high-resolution (25 km atmosphere and 10 km ocean) Community Earth System Model (CESM-HR) on Sunway TaihuLight. The refactoring and optimizing efforts have improved the simulation speed of CESM-HR from 1 SYPD (simulation years per day) to 3.4 SYPD (with output disabled) and supported several hundred years of pre-industrial control simulations. With further strategies on deeper refactoring and optimizing for remaining computing hotspots, as well as redesigning architecture-oriented algorithms, we expect an equivalent or even better efficiency to be gained on the new platform than traditional homogeneous CPU platforms. The refactoring and optimizing processes detailed in this paper on the Sunway system should have implications for similar efforts on other heterogeneous many-core systems such as GPU-based high-performance computing (HPC) systems.
