DIASPARA - Habitat database

Continental and marine habitat for diadromous Fishes in Europe, United States and Canadian East coast. These files contain the referentials for river, lakes and catchments from hydrosheds, plus referentials based on hydroshed and ICES and GFCM referentials in the marine area to describe a hierarchy of habitats, for ICES diadromous fish working groups WGNAS, WGBAST and WGEEL. The main product is table ref.tr_area_area which contains the code for all working groups. It can be downloaded as excel, shapefile, parquet file (in refnas, refbast, refeel.zip) or best postgres file. The table contains two geometry columns as the reference area can be a stock unit, or country, or EMU (eel management unit), or basin and in this case it is a polygon (column geom_polygon), but it can correpond to a river segment, and in that case it's a line (column geom_line). For that reason, parquet file is incomplete (no rivers) and we have downloaded two shapefile, ref_tr_area_are_geom_polyon_shp.zip exports the polygons, ref_tr_area_are_geom_line_shp.zip it export the line. Otherwise the two tables are identicaL. Finally the content of the file without geometry can be downloaded as an excel file ref.tr_area_are.xlsx .

Please have a look in our report to understand what we have done.

Versions

Version 0.0.6 (May 2026)(THIS RELEASE)

• Added a file, amended the R code for parquet.

Version 0.0.5 (March 2026) 

• Added shapefile with both polygon and lines
• Added excel table of ref.tr_area_area the common vocab for the three working groups

Version 0.0.4 (March 2026) 

• Added excel file with named rivers (WGNAS, WGBAST)

Version 0.0.3 (March 2026)

• fixed issue of corrupted format when trying to restore files (for gpc files) tested on server postgres 17.0

Version 0.0.2 (January 2026)

•   fixed geometry format in parquet for lakes

Version 0.0.1 (December 2025)

• Eventual structural modifications depending on feedback from WGNAS and WGBAST
• Adding referential table for WGEEL

Version 0.0.1-beta (July 2025)

• Beta release for view and test by DIASPARA and WORKING GROUPS.
• Integration with migdb vocabulary.
• Starting the integration of specific tables.
• Adding names to rivers corresponding to ICES vocabularies

Version 0.0.1-pre-beta (mid June 2025)

• Adding referential tables for hierarchical structure
• Structure for WGBAST and WGNAS

Version 0.0.1-alpha (mid February 2025)

• Alpha release for view and test by DIASPARA
• work here : Git issue using the hydrosheds

Download postgres database

If you don't have postgres, download it, make sure to use the stackbuilder program to download postgis. open a shell with command CMD Move to the place where you have downloaded the file using the following command

cd c:/path/to/my/folder

Note psql must be accessible, in windows you can add the path to the postgres bin folder, otherwise you need to add the full path to the postgres bin folder see link to instructions below below we assume that the user you are using to postgres

createdb -U postgres diaspara psql -U postgres diaspara

This will open a command with # where you can launch the following SQL command

CREATE EXTENSION postgis;


You will need this because we only restore part of the database : the schema corresponding to different locations.You will probably want to create roles, if not it's not a problem you'll get some warnings that grants were not created for this user

You need to have these schema ready to restore the content of some tables

Now you will have saved the files in the c:/path/to/my/folder location where you are currently working.

You need to use pg_restore to restore the files

If you want to restore everything in one go use :

This will restore all schemas for different ICES areas plus the habitat tables in schema ref, refbast, refnas and refeel

If you are just interested in a limited set of areas :

The first file to restore is habitat. This is necessary as all tables inherit from habitat.

Then if for instance if you want to use h_med_central.pgc use :

all_ref.gpc contains the code for schema ref, refnas, refeel, refbast (habitats for the three working groups) :

Download Parquet format 

You can use the files straight from R or Qgis. To use with R you will need Arrow and DuckDB packages.

Tutorials

If you need any help to use those, please see the webinar videos and presentation documents that can be found here.

Project page

https://diaspara.bordeaux-aquitaine.inrae.fr

Specific work done by WP3 can be found here.

Description and details about the creation of the Habitat Database can be found here.

Acknowlegment

hydrosheds :

Linke, S., Lehner, B., Ouellet Dallaire, C., Ariwi, J., Grill, G., Anand, M., Beames, P., Burchard-Levine, V., Maxwell, S., Moidu, H., Tan, F., Thieme, M. (2019). Global hydro-environmental sub-basin and river reach characteristics at high spatial resolution. Scientific Data 6: 283. doi: https://doi.org/10.1038/s41597-019-0300-6

# install packages (uncomment to use)
#install.packages("zen4R")
#install.packages("purrr")
#install.packages("arrow")
#install.packages("sfarrow")
library(zen4R)
library(arrow)
library(sfarrow)
library(purrr)
library(ggplot2)
# this will create a directory, use setwd() if you want to
# change the directory to put the files
dir.create("download_zenodo")
# you can list the files in Zenodo using the following script
# z <- get_zenodo(doi= "10.5281/zenodo.18155125")
#names(z$files)
# download the reference files containing the structure for eel and salmon
download_zenodo(doi= "10.5281/zenodo.18155125",
path = "download_zenodo",
files = c("refbast.zip", "refnas.zip", "refeel.zip"))
downloaded_files <- list.files("download_zenodo", full.name = TRUE)
downloaded_path <- file.path(getwd(),"download_zenodo")
# sequentially unzip all files in the same folder
downloaded_files |> purrr::map(\(x)unzip(zipfile=x, exdir =downloaded_path))  
# Open with arrow
ds <- arrow::open_dataset("download_zenodo/refeel_tr_area_are_polygon.parquet")
# you need to use collect() at the end of the query to put the data to memory.
# here I want to know which level will be used. Assessment unit will change to substock
# in the next zenodo version.
ds |> select(are_lev_code) |> unique() |> collect()
mymap <- ds|>
  filter(are_lev_code == "Assessment_unit") |>
    collect()|>
  mutate(geom_polygon = st_as_sfc(geom_polygon, EWKB = TRUE))

#mymap

ggplot(mymap) + geom_sf(aes(geometry = geom_polygon, fill =are_code),color="white",size=.2)+
  scale_fill_viridis_d()+
  theme_minimal()+
  theme(panel.background = element_rect(fill = "light blue"))

# GET ADDITIONAL DATA FROM ICES (ECOREGIONS) -------------
library(httr2)   # or use download.file if you prefer base R
library(sf)      # to read the shapefile or GeoPackage
library(fs)
ecoregions_url <- "https://gis.ices.dk/shapefiles/ICES_ecoregions.zip"   # common historical location
out_dir <- path_temp("ices_ecoregions_dl")
dir_create(out_dir, recurse = TRUE)
zip_path <- path(out_dir, "ICES_ecoregions.zip")
unzip_dir <- path(out_dir, "unzipped")
req <- request(ecoregions_url) |>
  req_user_agent("R (httr2) - ICES ecoregions download") |>
  req_timeout(120)
resp <- req_perform(req, path = zip_path)
dir_create(unzip_dir, recurse = TRUE)
unzipped_files <- unzip(zipfile = zip_path, exdir = unzip_dir)
shp  <- unzipped_files[grepl("\.shp$",  unzipped_files, ignore.case = TRUE)]
# Read with sf
ecoregions_sf <- sf::st_read(shp, quiet = TRUE)
plot(ecoregions_sf["Ecoregion"], main = "ICES Ecoregions")

Get parents in Postgres : recursive function