HIERARCHICAL DATAFLOW MODEL WITH AUTOMATED FILE MANAGEMENT FOR ENGINEERING AND SCIENTIFIC APPLICATIONS

Abstract

Solving modern scientific and engineering problems typically implies using multiple task-specific software applications and often a complex sequence of computations must be performed. Adopted approach to achieve the required level of automation is to use one of the many available scientific and engineering workflow systems, which can be based on different workflow models. This paper introduces a workflow model targeted to provide natural automation and distributed execution of complex iterative computation processes, where the calculation chain contains multiple task-specific software applications which exchange files during the process.The proposed workflow model addresses a wide range of applications and targets complex cases when a single iteration of a top-level process may contain multiple nested execution loops. Typical requirements to process automation are considered as well: execution isolation, data re-use and caching, parallel execution, data provenance tracking.

Keywords

• automation of scientific processes
• automation of engineering processes
• distributed computing
• workflow
• dataflow
• pseven