Spatial data in Puhti

Spatial data in Puhti#

  • Large commonly used Finnish geospatial datasets with open licenses

  • Removes transfer bottleneck

  • Located in: /appl/data/geo/

  • All Puhti users have read access

  • ~13 TB of datasets available:

    • Paituli data, with virtual mosaics for raster data

    • Finnish Environmental Institute (SYKE) open datasets: CORINE land use etc

    • Forest center: canopy height, forest mask, gridcells, forest resource plots

  • LUMI and Mahti do not have spatial data on local disk.

  • CSC Docs: Spatial data in CSC computing environment

Paituli#

  • Spatial data download service

  • ~300 Finnish datasets

  • All datasets open to everyone

  • Also historical versions of several datasets

  • Possibility to publish own datasets for universities and research institutes

  • Ministry of education and culture supports financially, CSC maintains

  • Paituli

"Paituli"

Paituli data access in Puhti:#

  • As files in /appl/data/geo/

  • Raster datasets via STAC

  • Rasters divided to mapsheets as virtual rasters

  • Majority via OGC APIs, both old and new standards

Virtual rasters

Virtual rasters are a useful GDAL concept for managing large raster datasets that are split into map sheets that do not overlap. Technically a virtual raster is just a small XML file that tells GDAL where the actual data files are, but from user’s point of view virtual rasters can be treated much like any other raster format. Virtual rasters can include raster data in any file format GDAL supports. Virtual rasters are useful because they allow handling of large datasets as if they were a single file eliminating the need for locating correct files.

  • Virtual rasters are not useful for managing time-series or overlapping rasters, for example remote sensing tiles.

  • Supported by any GDAL based tool, including Python and R spatial packages, ArcGIS, FME, GrassGIS, MapInfo, QGIS, and SagaGIS.

  • CSC Docs: Virtual rasters tutorial, inc code examples for R and Python.

Contents