Scope and limitations of ad-hoc neural network reconstructions of solar wind parameters; analysis and computation tools

This code recreates the results shown in the paper:
Scope and limitations of ad-hoc neural network reconstructions of solar wind parameters
by Maximilian Hecht, Verena Heidrich-Meisner, Lars Berger, and Robert F. Wimmer-Schweingruber

DOI: 10.1051/0004-6361/202346714

-----------------------------------------------------------------

Solar wind properties are determined by the conditions of their solar source region and transport history. Solar wind parameters, such as proton speed, proton density, proton temperature, magnetic field strength, and the charge state composition of oxygen, are used as proxies to investigate the solar source region of the solar wind. The solar source
region of the solar wind is  relevant to both the interaction of this latter with the Earth's magnetosphere and to our understanding of the underlying plasma processes, but the effect of the transport history of the wind is also important. The transport and conditions in the solar source region affect several solar wind parameters simultaneously. Therefore, the typically considered solar wind properties (e.g. proton density and oxygen charge-state composition) carry redundant information. Here, we are interested in exploring this redundancy. 

The observed redundancy could be caused by a set of hidden variables that determine the solar wind properties. We test this assumption by determining how well a (arbitrary, non-linear) function of four of the selected solar wind parameters can model the fifth solar wind parameter. If such a function provided a perfect model, then this solar wind parameter would be uniquely determined from hidden variables of the other four parameters and would therefore be redundant. If no reconstruction were possible, this parameter would be likely to contain information unique to the parameters evaluated here. In addition, isolating redundant or unique information contained in these properties guides requirements for in situ measurements and development of computer models. Sufficiently accurate measurements are necessary to understand the solar wind and its origin, to meaningfully classify solar wind types, and to predict space weather effects. 

We employed a neural network as a function approximator to model unknown, arbitrary, non-linear relations between the considered solar wind parameters. 
This approach is not designed to reconstruct the temporal structure of the observations. Instead a time-stable model is assumed and  each point of measurement is treated separately. This approach is applied to solar wind data from the Advanced Composition Explorer (ACE).
The neural network reconstructions are evaluated in comparison to observations, and the resulting reconstruction accuracies for each reconstructed solar wind parameter are compared while differentiating between different solar wind conditions (i.e. different solar wind types) and between different phases in the solar activity cycle. Therein, solar wind types are identified according to two proton plasma property based solar-wind classification schemes.

Within the limits defined by the measurement uncertainties, the proton density and proton temperature can be reconstructed well. Each parameter was evaluated with multiple criteria. Overall proton speed was the parameter with the most accurate reconstruction, while the oxygen charge-state ratio and magnetic field strength were most difficult to recover.

For all considered solar wind parameters, but in particular the proton density, proton temperature, and the oxygen charge-state ratio, parameter reconstruction is hindered by measurement uncertainties. The proton speed, while being one of the easiest to measure, also seems to carry the highest degree of redundancy with the combination of the four other solar wind parameters. Nevertheless, the reconstruction accuracy for the proton speed is limited negatively affected by the large measurement uncertainties on the respective input parameters. The reconstruction accuracy of sector reversal plasma is noticeably lower than that of streamer belt or coronal hole plasma. We suspect that this is a result of the effect of stream interaction regions, which strongly influence the proton plasma properties and are typically assigned to sector reversal plasma. The fact that the oxygen charge-state ratio ---a non-transport-affected property--- is difficult to reconstruct may imply that recovering source-specific information from the transport-affected proton plasma properties is challenging. This underlines the importance of measuring the heavy ion charge-state composition.

-----------------------------------------------------------------
-----------------------------------------------------------------

How to use:

-----------------------------------------------------------------
-----------------------------------------------------------------

0. Requirements

tested with:
python >= v3.11
numpy >= v1.25.0
matplotlib >= 3.7.1
scikit-learn >= v1.3.0
progressbar2 >= 4.2.0

-----------------------------------------------------------------

1. Training
-----------------------------------------------------------------
We assume your python installation is called with "python". You may need to change this (in run.sh and/or run.bat) to "python3" or the name of your python installation.

The easiest way to start your training is by using run.sh (or run.bat) .
in linux terminal: sh run.sh
in windows powershell: .\run.bat

Using run.sh first downloads the necessary data sets (solar wind data from https://zenodo.org/record/7568673 and monthly sun spot data needed for Figure 3 from https://sidc.be/SILSO/DATA/SN_m_tot_V2.0.txt). Then, it starts training and creates files in the results directory that you can evaluate.
After the download you can track the progress of training in the logs directory.

For a quick first demo,  run.sh executes a reduced number of trials and Monte Carlo simulations. To reproduce the full results from the paper, the following steps are necessary:
1. change "--trials 2 --iters 51" to "--trials 100 --iters 201" in run.sh (or run.bat)
2. adjust the for loop to produce 100 Monte Carlo runs:
    Example in run.sh:        |    Example in run.bat:
    MCnumber=20            |    set MCnumber=20
    for n in 1 21 41 61 81 ; do    |    for %%n in (1 21 41 61 81) do (
        ...            |        ...
    done                |    )

Under the demo settings, the expected runtime of run.sh on a Ryzen 5700x is approximately 20 min and will result in ~300 MB of data in the results directory and ~200 MB in the data directory.

You can adjust your training values by customising the prompts for sw_neural_any.py .
To see which training values you can adjust use: python sw_neural_any.py -h .

To create Monte Carlo runs add noise values to your training prompt by adding the option "--noise 1 11" . This creates 10 Monte Carlo runs. 

2. Evaluation
-----------------------------------------------------------------
To evaluate your data use evaluation.py .
If you used the provided run.sh continue with evaluation.py with --demo 1 as argument.
in commandline: python evaluation.py --demo 1

This creates all plots used in the paper except for the learning curves of the model selection (which require results for all combinations of the hyper-parameters for the solar wind speed as reconstructed parameter to be available).
The plots are saved in the graphs directory.

Otherwise you can use python evaluation.py -h to select which plots you want to generate with which settings.

-----------------------------------------------------------------
-----------------------------------------------------------------

Acknowledgments

This work was supported by the Deutsches Zentrum für Luft- und Raumfahrt (DLR) as SOHO/CELIAS 50 OC 2104.  
We further thank the science teams of  ACE/SWEPAM, ACE/MAG as well as ACE/SWICS for providing the respective level 2 and level 1 data products.

-----------------------------------------------------------------
-----------------------------------------------------------------

necessary Files:
sw_neural_any.py    This file is used to train the neural network.
evaluation.py    This file is used to evaluate the results of training.
load_data_new_format.py    This file contains the loader for the ACE data from https://zenodo.org/record/7568673 and the sunspot data.
custom_utils.py        This file contains some utilities that are used in the preprocessing stages of training.    
beautify.py        This file makes plots prettier with custom formatting.
datetools.py        Transforms dates between different formats.

run.sh            This is a bash script to run multiple training runs. Current values are adjusted so training takes less time. To reconstruct data in the paper change --trials 2 and --iters 51 arguments to --trials 100 --iters 201. This takes considerably more time to complete.
run.bat

data/ACE_data_2001-2010.dat    Solar wind ACE data from https://zenodo.org/record/7568673 .
data/SN_m_tot_V2.0.txt        Sunspot number data from https://sidc.be/SILSO/DATA/SN_m_tot_V2.0.txt .

graphs/        Empty folder to save plots.
logs/        Empty folder to save logs from training.
results/    Empty folder to save trained networks from sw_neural_any.py .

settings_sorted_gridsearchperc_vsw.txt        Sorted settings from hyperparameter selection for proton speed.
settings_sorted_gridsearchperc_tsw.txt        Sorted settings from hyperparameter selection for proton temperature.
settings_sorted_gridsearchperc_dsw.txt        Sorted settings from hyperparameter selection for proton density.
settings_sorted_gridsearchperc_B.txt        Sorted settings from hyperparameter selection for magnetic field strength.
settings_sorted_gridsearchperc_dO7_6.txt    Sorted settings from hyperparameter selection for charge state ratio of oxygen.

-----------------------------------------------------------------

tested with:
python >= v3.11
numpy >= v1.25.0
matplotlib >= 3.7.1
scikit-learn >= v1.3.0
progressbar2 >= 4.2.0