spatial.IO - An integrated cloud-ready geospatial data management system

Spatial Data Infrastructure

A Spatial Data Infrastructure (SDI) is a combination of policies, standards and software to manage and deliver geospatial data (Simmons, 2018). A good SDI follows policies and standards that are (widely) accepted in the communities (e.g. FAIR, OGC). Although often providing new functionality, the main advantage of an SDI is the connection of different tools and software products to build (mostly) automated workflows. This allows for less manual processing (and therefore fewer errors) as well as standardized data products due to fixed workflows. For this to work flawlessly, extensive documentation and user instructions are key. An SDI can contain (but is not limited to) data storage, metadata catalogue, tools for data processing, WebGIS and a form of data access (e.g. download, web service).

Environmental research produces an increasing amount of spatial data (e.g. climate, hydrological, socio-economic) by using a variety of methods (e.g. modelling, remote sensing, data-driven). This leads to the demand for effective management of such data. This application aims to provide automated workflows and user-friendly applications/interfaces in a cloud-based environment to easily manage, share and visualize standardized spatial data. The standard specifications follow the Binding Regulations for Storing Data as netCDF Files. Other vector and raster data formats can be included in the workflows with manual work-steps and will be automated in the next versions. The application will be expanded continuously into a self-service platform to create custom WebGIS, automated workflows and various (meta-)data provision interfaces for a wide range of spatial data formats.

Requirements

• Simply FAIR
• Science- and Management friendly: Provide interoperable and reliable spatial data enriched by metadata and with provenance information.
• User friendly: Easy to use user interface for people that manage spatial data or create WebGIS for netCDF data, without requiring knowledge about underlying technologies like databases.
• Admin friendly: A scalable and transferable container based solution that will smoothly integrate into typical scientific IT landscapes.
• Developer friendly: Common open source solutions structured by microservice architecture to keep it open and simple to extend for developers.

Features

• S3 cloud-storage with MinIO
• FROST®-Server to store and access sensor data (in combination with time.IO and SaQC)
• Creation of custom interactive WebGIS components for netCDF, STA and GeoTIFF data
• Extendable processes to get spatially aggregated values for netCDF and GeoTIFF data
• Use of django framework to make configuration of data and WebGIS user-friendly
• Workflow for automated creation of OGC web services with GeoServer of new spatial data
• Workflow for automated creation of metadata entries in GeoNetwork
• THREDDS Data Server (TDS) to provide netCDF data with OPeNDAP

Quickstart

1. Install Docker Engine (Community Edition - CE is enough) and Docker Compose
2. Install a git client and checkout the spatialIO repository or unzip the attached archive
3. Follow step-by-step instructions in README.md to startup

Techstack, dependencies and third party open source products

• MinIO
• FROST®-Server
• GeoServer
• GeoNetwork
• THREDDS Data Server (TDS)
• django
• PostgreSQL
• Vue
• Bulma
• pygeoapi

Acknowledgements