A Hadoop cloud-based surrogate modelling framework for approximating complex hydrological models

Abstract

Hydrological simulation has long been a challenge because of the computationally intensive and expensive nature of complex hydrological models. In this paper, a surrogate modelling (SM) framework is presented based on the Hadoop cloud for approximating complex hydrological models. The substantial model runs required by the design of the experiment (DOE) of SM were solved using the Hadoop cloud. Polynomial chaos expansion (PCE) was fitted and verified using the high-fidelity model DOE and was then used as a case study to investigate the approximation capability in a Soil and Water Assessment Tool (SWAT) surrogate model with regard to the accuracy, fidelity, and efficiency. In experiments, the Hadoop cloud reduced the computation time by approximately 86% when used in a global sensitivity analysis. PCE achieved results equivalent to those of the standard Monte Carlo approach, with a flow variance coefficient of determination of 0.92. Moreover, PCE proved to be as reliable as the Monte Carlo approach but significantly more efficient. The proposed framework greatly decreases the computational costs through cloud computing and surrogate modelling, making it ideal for complex hydrological model simulation and optimization. HIGHLIGHTS Our surrogate modelling framework reduces the computational cost of simulations.; The design of the experiment was parallelized on a Hadoop cloud.; PCE was fitted and verified using a high-fidelity model.; The approximation ability of PCE in the SWAT surrogate model was investigated.;

Keywords

• chaospy
• hadoop cloud
• polynomial chaos expansion
• surrogate modelling
• swat