FiN: A Smart Grid and Powerline Communication Dataset

# FiN: A Smart Grid and Powerline Communication Dataset

Within the Fühler-im-Netz (FiN) project 38 BPL modems were distributed in three different areas of a German city with about 150.000 inhabitants. Over a period of 22 months, an SNR spectrum of each connection between adjacent BPL modems was generated every quarter of an hour. The availability of this data from actual practical use opens up new possibilities to face the increasing complex challenges in smart grids.

~~ For detailed information we would like to refer to the full paper. ~~

Attributs | FiN 1
-------- | --------
SNR measurements   | 3.3 Mio
Timespan   | ~2.5yrs
*Metadata*   |
Sleeve count per section   | ☑
Cable length, typ, cross section   | ☑
Number of conductors   | ☑
Year of installation  | ☑
Weather by openweather     | ☑

## Paper abstract
The increasing complexity of low-voltage networks poses a growing challenge for the reliable and fail-safe operation of power grids. The reasons for this are, for example, a more decentralized energy generation (photovoltaic systems, wind power, ...) and the emergence of new types of consumers (e-mobility, domestic electricity storage, ...). At the same time, the low-voltage grid is largely unmonitored and local power failures are sometimes detected only when consumers report the outage. To end the blind flight within the low voltage network, the use of a broadband over power line (BPL) infrastructure is a possible solution. In addition to the purpose of establishing a communication infrastructure, BPL also offers the possibility of evaluating the cables themselves, as well as the connection quality between individual cable distributors based on their Signal-to-Noise-Ratio (SNR). Within the Fühler-im-Netz pilot project 38 BPL modems were distributed in three different areas of a German city with about 100.000 inhabitants. Over a period of 21 months, an SNR spectrum of each connection between adjacent BPL modems was generated every quarter of an hour. The availability of this data from actual practical use opens up new possibilities to react agilely to the increasingly complex challenges.

# FiN-Dataset release 1.0

### Content
- 68 data .npz files
- 3 weather csv files
- 2 metadata csv files
- this readme

### Summary
The dataset contains ~3.7B SNR measurements divided into 68 1-to-1 connections. Each of the 1-to-1 connections can split into additional segments, e.g. if part of a cable was replaced due to a cable break.
All 68 connections are formed by 38 different nodes distributed over three different locations. Due to data protection regulations, the exact location of the nodes cannot be given. Therefore, each of the 38 nodes is uniquely identified by an ID.

### Data
The filename specifies the location, the ID of the source node and the destination ID.
Example: "loc03_from26_to27.npz"
    -> Node is in lcation 3
    -> Source node is 26
    -> Destination node is 27

The .npz file contains a Python dict that is structured as follows:
    
    data_dict = {"timestamps": np.array(...),        --> Nx1 Timestamps
             "spectrum_rx": np.array(...),        --> Nx1536    SNR assesments on 1536 channels in RX directions. Range is 0.00dB...40.00dB
             "tonemap_rx": np.array(...),        --> Nx1536    Tonemaps in RX directions. Range is 0...7
             "tonemap_tx": np.array(...)}        --> Nx1536    Tonemaps in TX directions. Range is 0...7

### Weather
In addition to the measured data, we add weather data provided by https://openweathermap.org for all three locations. The weather data is stored in CSV format and contains many different weather attributes. Detailed information on the weather data can be found in the official documentation: https://openweathermap.org/history-bulk

### Metadata
    --> nodes.csv
        Contains in overview of all nodes, their id, corresponding location and voltage level.

    --> connections.csv
        Contains all available metadata for the 68 1-to-1 connections and their individual segements.

        + year_of_installation            -> year in which the cable was installed
        + year_approximated            -> Indicates whether the year was approximated or not (e.g. due to missing records)
        + cable_section                -> identifies the segment or section described by the metadata
        + length                 -> length in meters
        + number_of_conductors             -> identifier for the conductor structure in the cable
        + cross-section             -> cross-section of the conductors
        + voltage_level                -> identifier for the voltage level (MV=mid voltage; LV=low voltage)
        + t_sleeves                -> number of T-sleeves installed within a section
        + type                    -> cable type
        + src_id                -> id of the source node
        + dst_id                -> id of the destination node