Initial Sample of HYPERNETS Hyperspectral Surface Reflectance Measurements for Satellite Validation from the agricultural land at Demmin, Germany

The HYPERNETS project (www.hypernets.eu) aims to ensure that high-quality in situ measurements are available to support the (VNIR/SWIR) optical Copernicus products. Therefore, it established a new autonomous hyperspectral spectroradiometer (HYPSTAR® - www.hypstar.eu) dedicated to land and water surface reflectance validation with instrument-pointing capabilities. In the prototype phase, the instrument is being deployed at 24 sites covering a range of water and land types and a range of climatic and logistic conditions. This dataset provides the first published data for the HYPERNETS site in Demmin, Germany [53°52'5.80"N,13°16'6.80"E] (DEGE). It is a subset of the complete data record, consisting of the measurements withEthaturements which could be used for satellite validation. 

The provided NetCDF files are the L2A hypernets products with surface reflectances, their associated uncertainties and error-correlation information. The reflectance in the L2A products is the Hemispherical-directional Reflectance Factor (HDRF) defined as HDRF = π L / E where L is the directional upwelling radiance (with the field o, view of 5 degrees), and E is the (hemispherical) downwelling irradiance (i.e. including both direct solar and diffuse sky irradiance). These reflectances have dimensions of wavelength and series, where each series is a set of measurements for a given geometry (combination of viewing zenith and azimuth angle). In addition to variables for wavelength and bandwidth, the files also contain variables that provide for each series the acquisition time, viewing and solar angles, number of valid scans used, and quality flags (typically, no flags are set in the data provided in this dataset). These NetCDF files also contain further relevant metadata as attributes. See https://hypernets-processor.readthedocs.io/ for further info.

The HYPSTAR®-XR sensor was installed on 22 July 2021 at the top of a 10m mast on an extended 5 m horizontal boom to minimise interruption of the field of view. The boom faces South at the right angle towards bare soil. The mast is located at 53.868278°N, 13.268556°E. Data are collected every 30 minutes between 9:00 and 17:00 (UTC) from different zenith and azimuth angles.

The HYPSTAR®-XR (eXtended Range) instruments deployed at each land HYPERNETS site consist of a VNIR and a SWIR sensor and autonomously collect data between 380-1700 nm at various viewing geometries and send it to a central server for quality control and processing. The VNIR sensor spans 1330 channels between 380 and 1000 nm with an FWHM of 3 nm, and the SWIR sensor has 220 channels between 1000 and 1700 nm with an FWHM of 10 nm. The hypernets_processor (Goyens et al. 2021; De Vis et al. in prep.) automatically processes all this data into various products, including the L2A surface reflectance product provided here. All products have associated uncertainties (divided into random and systematic uncertainties, including error-correlation information)  propagated using the CoMet toolkit (www.comet-toolkit.org). 

To obtain this dataset, we start from the full DEGE data record and omit all the data that do not pass all quality checks performed as part of the hypernets_processor. In addition, an additional screening procedure was also developed to remove outliers and supply the best quality data suitable for satellite validation. To remove the outliers, a sigma-clipping method is used. First, reflectances are extracted in separate 2-hour windows throughout the day (to account for BRDF differences due to different solar positions) for four different wavelengths (500, 900, 1100 and 1600 nm). Outliers in these reflectances are then identified by iteratively calculating the mean reflectance trend with time (by binning the data per maximum of 30 data points), calculating the standard deviation from this trend, and masking any data that is more than three standard deviations away from the trend. This process is repeated on the unmasked data until the standard deviation does not vary by more than 5% between two iterations. The masks for the four different wavelengths are then combined (keeping only measurements for which none of the four wavelengths is an outlier). The reflectances and associated uncertainties for any masked series (i.e. a geometry that is masked either by the sigma-clipping procedure or from the masks of the hypernets_processor) are replaced by NaNs. Any sequence that has more than half of its series masked is removed entirely.