Technical note on the harmonization workflow

The SCARFACE - Methodological note.pdf file contains a technical note on the statistical methodologies used to spatially-align gridded datasets via spatial block kriging (Section S1), the post-stratification procedure adopted to generate the spatio-temporal weighting system for FADN survey data (Section S2) and the Generalized Variance Function (GVF) methodology adopted to regularize direct estimates of the variance for FADN survey data.

Update

This dataset was initially compiled for the "Sequestering CARbon through Forests, AgriCulture, and land usE (SCARFACE)" research initiative funded by the University of Milano-Bicocca and will be systematically updated whenever new information, for instance new releases of the underlying raw dataset, will become available.

Note about data and replication code

The file SCARFACE - Data and replication code.zip contains the following

• A separate folder for each source-specific dataset (i.e., Animals, CAMSgrid, EDGARgrid, EDOgrid, EEAconc, EnvironmentalVars, LandCover, ISTATSocioEconomicData and CREA)
• A folder with auxiliary functions to apply the spatial block kriging algorith used to upscale gridded data (i.e., AuxFuns_Kriging)
• Other files

Each folder contains:

• R and Python code to generate (potential) intermediate output (e.g., spatially-aligned data not yet aggregated at the areal level) and the final dataset associated with each thematic domain (e.g., yearly georeferenced panel data with area-level values)
• Intermediate output(s)
• Pre-processed final dataset associated with each thematic domain

Notice the following:

1. The folders DO NOT contain the original data for EDGAR emissions inventories, CAMS concentrations inventories, EEA concentration inventories, EDO extreme events variables and ERA5 environmental variables (oversized files). Data can be retrieved through the corresponding scripts. Intermediate dataset (i.e., temporally and spatially interpolated data) are provided;
2. The folders CONTAINS the original data for ISTAT socio-economic variables, BDN livestock, land cover, ISTAT agricultural census 2020 and 2010. Data can also be retrieved through the corresponding scripts. Intermediate dataset are provided;
3. Before running R and Python scripts is necessary to change the working directory (notice that codes are setup to work inside the same folder where the original data are located)

Note on the availability of FADN data

While all datasets included in SCARFACE are provided by national and international public institutions and are, in principle, publicly available, farm activity and agro-economic indicators derived from the FADN represent the only exception. In fact, they are accessed under a research data agreement with the data provider, that is, the Italian Council for Agricultural Research and Economics -- Research Centre for Agricultural Policies and Bioeconomy (CREA-PB). Specifically, the analysis relies on farm-level microdata that cannot be publicly disclosed without appropriate anonymization and spatial aggregation. The methodology adopted to ensure confidentiality is described in detail in Section Methods of the companion paper.

Usage notes

The dataset is ready to be used as is by any user interested in studying the complex interactions among agricultural systems, air quality management, environmental processes, and human activities in a critical area of Europe. However, users should take into account some potential caveats.

Potential missing values

Due to structural heterogeneity of the data sources (i.e., different spatial and temporal coverages) several missing values could arise (e.g., socio-economic data cover the period 2014--2023, while emissions data cover the whole period 2011--2024), especially across the temporal domain. We recall that no missing data treatment was enforced during the dataset building. For transparency purposes, in file SCARFACE_MissingAnalysis_April2026.xlsx we provide a descriptive analysis about the missing values detected for each individual dataset, year, variable and ASR.

Uncertainty quantification and interpolation diagnostics

In addition to the main data products, the SCARFACE framework provides a comprehensive set of uncertainty-related information, including metrics associated with temporal aggregation, spatial interpolation, and the tuning of interpolation hyperparameters.

First, temporal aggregation uncertainty arises from the harmonization of input datasets originally available at monthly or higher temporal resolution (e.g., meteorological and drought indicators). Prior to spatial alignment via BK, these data were summarized at the grid-cell level using annual and seasonal statistics (mean, minimum, maximum, and standard deviation). The standard deviation captures infra-annual variability and provides an indirect measure of temporal aggregation uncertainty.

Second, for gridded data interpolated using local spatial block kriging (in addition to the harmonized dataset) we provide both the estimated parameters, the covariance function used to interpolate and the corresponding cross-validation results (see the objects '*_BKinterpolated.RData' and '* _BK_output.RData' inside each 'InterpolationResults' folder and the corresponding scripts '* - Application of Xval spatial block kriging.R' and '* - Stacking block kriging estimates') . Similarly, for FADN survey data we provide both the original non-smoothed variances and the GVF smoothed variances (as well as the empirical smoothing model) allowing a fair comparison of the pre-regularization and post-regularization distributions (see folder CREA/GVFDiagPlots).

Thrid, for every variable spatially-interpolated using local spatial block kriging, estimates of the ASR-level predicted kriging average (i.e., '*_BKm') and the corresponding ASR-level kriging prediction variance (i.e., '*_BKmv') are provided. Also, FADN variables aggregated at the ASR-level using the Horvitz-Thompson (HT) expansion estimator, estimates of the average value (HTm) and the total value (HTt), as well the corresponding variances (HTmv and HTtv) are provided. In both cases, the estimated area-level variances quantify the interpolation uncertainty and are intended to support downstream analyses in which estimation precision is critical, such as weighted regression models, uncertainty propagation, sensitivity analyses, and the identification of domains with lower inferential reliability. In this regard, for modeling purposes, such uncertainty should be carefully taken into account through specific model-based adjustments, such as multi-stage bootstrap inference, in order to correct estimates and inference for the prior knowledge of the underlying data interpolation process.