Brazil Data Cube Workflow Engine: a tool for big Earth observation data processing

Abstract

ABSTRACTEarth Observation (EO) satellites have produced vast image collections that are freely accessible to society. However, handling these images often surpasses the capabilities of traditional hardware and software for EO data storage and processing, posing challenges for traditional Spatial Data Infrastructure (SDI). To overcome these challenges, innovative cloud computing and distributed systems have been developed, such as matrix databases, MapReduce systems, and web services. These technologies are now integrated into leading-edge platforms, forming a new generation of SDI for big EO data. These platforms have different characteristics in terms of governance, technologies, data access, infrastructure abstractions, data processing, and flexibility to extend their functionality. Our work contributes to the area of SDI for big EO data by proposing a server-side data-processing tool called Brazil Data Cube Workflow Engine (BDC-WE), based on workflow orchestration approach. BDC-WE provides a high-level interface using the openEO API for big EO data accessing and processing, allowing SDI maintainers to easily describe sequences of processes and integrate new algorithms. The architecture proposed in this study was implemented and the prototype was evaluated in two case studies described in this paper.

Keywords

• Big data
• Earth observation data
• spatial data infrastructure
• openEO
• workflow orchestration