A systematic synthesis of sky image enhancement techniques for ground-based solar irradiance forecasting

Data files and deep learning models supporting a study on image enhancement methods for improving ground-based solar irradiance forecasting using sky images and pyranometer readings.

Code repository

The source code repository used to study the effects of image enhancement methods and tricks on solar irradiance forecasting, and to generate the models below, is available at https://github.com/PUTvision/solar-irradiance. Additionally, Zenodo contains the latest version of the compressed source code in ZIP format.

The README.md file in the source code repository describes dependencies, requirements and installation steps.

Models

Pretrained models checkpoints for a 15-minute forecasting horizon with sky image enhancement pipeline applied for solar irradiance forecasting using ground-based sky images.

Irradiance forecasting models available in the literature.

[1] H. Wen, Y. Du, X. Chen, E. Lim, H. Wen, L. Jiang, W. Xiang, Deep learning based multistep solar forecasting for pv ramp-rate control using sky im-
ages, IEEE Transactions on Industrial Informatics 17 (2) (2021) 1397–1406. doi:10.1109/TII.2020.2987916.
[2] E. A. Papatheofanous, V. Kalekis, G. Venitourakis, F. Tziolos, D. Reisis, Deep learning-based image regression for short-term solar irradiance forecasting on the edge, Electronics 11 (22) (2022). doi:10.3390/electronics11223794.
[3] G. Venitourakis, C. Vasilakis, A. Tsagkaropoulos, T. Amrou, G. Konstantoulakis, P. Golemis, D. Reisis, Neural network-based solar irradiance forecast for edge computing devices, Information 14 (11) (2023). doi:10.3390/info14110617.

Dataset

The dataset used in this study is based on the Folsom dataset. To make the data preparation and follow-up steps easier to reproduce, these steps were described as a directed acyclic graph (DAG) using the DVC package. In the source code repository, this pipeline is stored in a dvc.yaml file.

Data files

• cleaned_irradiance.csv - cleaned version of the original Folsom_irradiance.csv files with timestamps converted to US/Pacific and added column with corresponding sky image, which is available in the Folsom dataset and is not corrupted
• periods_{FORECASTING HORIZON}m.pickle - data (irradiance readings and corresponding image filenames) grouped into periods of historical measurements at times (t-15, t-10, t-5, t), and the corresponding target irradiance at time t+H (H represents the forecasting horizon)
• solar-irradiance-{RELEASE TAG}.zip - release of the source code related to this study

Forecasting horizons

Dataset splits used in the study for different forecasting horizons.

Data splits

The training, validation and test splits were generated using the following code sample, which is defined in the solar_irradiance/datamodules/forecasting.py file int the source code repository.

from pathlib import Path

import numpy as np
import pandas as pd


periods_path = Path("./periods_15m.pickle")

with periods_path.open("rb") as f:
    periods = pd.read_pickle(f)

test_periods = list(filter(lambda p: p["history"][-1]["image_name"].startswith("2014"), periods))
train_val_periods = list(filter(lambda p: not p["history"][-1]["image_name"].startswith("2014"), periods))
size = 170  # number of days for validatation dataset to get 80-20 ratio of train-val datasets

np.random.seed(42)
val_dates = [
    str(y) + str(m).zfill(2) + str(d).zfill(2)
    for y, m, d in zip(
        np.random.randint(2015, 2017, size=size),
        np.random.randint(1, 13, size=size),
        np.random.randint(1, 29, size=size),
        strict=False,
    )
]
val_periods = list(filter(lambda p: p["history"][-1]["image_name"][:8] in val_dates, train_val_periods))
train_periods = list(filter(lambda p: p["history"][-1]["image_name"][:8] not in val_dates, train_val_periods))