10.1016/j.neunet.2021.11.020
https://zenodo.org/records/5776561
oai:zenodo.org:5776561
Kateryna Chumachenko
Kateryna Chumachenko
Faculty of Information Technology and Communication Sciences, Tampere University, Tampere, Finland
Alexandros Iosifidis
Alexandros Iosifidis
Department of Electrical and Computer Engineering, Aarhus University, Aarhus, Denmark
Moncef Gabbouj
Moncef Gabbouj
Faculty of Information Technology and Communication Sciences, Tampere University, Tampere, Finland
Feedforward neural networks initialization based on discriminant learning
Zenodo
2021
neural networks initialization
discriminant learning
2021-11-25
eng
https://github.com/AlekosIosifidis/nnet_discriminant_init
https://zenodo.org/communities/marvel_project
https://zenodo.org/communities/eu
Creative Commons Attribution 4.0 International
In this paper, a novel data-driven method for weight initialization of Multilayer Perceptrons andConvolutional Neural Networks based on discriminant learning is proposed. The approach relaxes some of the limitations of competing data-driven methods, including unimodality assumptions, limitations on the architectures related to limited maximal dimensionalities of the corresponding projection spaces, as well as limitations related to high computational requirements due to the need of eigendecomposition on high-dimensional data. We also consider assumptions of the method on the data and propose a way to account for them in a form of a new normalization layer. The experiments on three large-scale image datasets show improved accuracy of the trained models compared to competing random-based and data-driven weight initialization methods, as well as better convergence properties in certain cases.
This work is supported by Business Finland under project 5GVertical Integrated Industry for Massive Automation (5G-VIIMA). A. Iosifidis acknowledges funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 957337 (MARVEL).
European Commission
10.13039/501100000780
957337
Multimodal Extreme Scale Data Analytics for Smart Cities Environments