Software Open Access
Vo, Huynh Quang Nguyen
These files contain the proposed implementation for benchmarking to evaluate whether a setup of hardware is feasible for complex deep learning projects.
3. Evaluation metrics
There are various metrics to benchmark the performance capabilities of a setup for deep learning purposes. Here, the following metrics are used:
Benchmark Type A.ipynb
Benchmark Type B.ipynb
S. Deitsch, V. Christlein, S. Berger, C. Buerhop-Lutz, A. Maier, F. Gallwitz, and C. Riess, "Automatic classification of defective photovoltaic module cells in electroluminescence images," Solar Energy, vol. 185, p. 455–468, 06-2019
M. Rahimzadeh and A. Attar, "A modified deep convolutional neural network for detecting COVID-19 and pneumonia from chest X-ray images based on the concatenation of Xception and ResNet50V2," Informatics in MedicineUnlocked, vol. 19, p. 100360, 2020.
H. Vo, "Realization and Verification of Deep Learning Models for FaultDetection and Diagnosis of Photovoltaic Modules," Master's Thesis, Aalto University. School of Electrical Engineering, 2021.
P. Warden, "Why GEMM is at the heart of deep learning," Pete Warden's Blog, 2015. Available at: https://petewarden.com/2015/04/20/why-gemm-is-at-the-heart-of-deep-learning/
Baidu Research, "Benchmarking Deep Learning operations on different hardware". Available at: https://github.com/baidu-research/DeepBench
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, 1998.
Xiao, K. Rasul, and R. Vollgraf, "A Novel Image Dataset for Benchmarking Machine Learning Algorithms," 2017. https://github.com/zalandoresearch/fashion-mnist
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion,O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vander-plas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay,"Scikit-learn: Machine learning in Python," Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011.
F. Chollet, "Keras," 2015. Available at: https://github.com/fchollet/keras
ML Commons. Availabke at: https://mlcommons.org/en/
W. Dai and D. Berleant, "Benchmarking contemporary deep learn-ing hardware and frameworks: A survey of qualitative metrics," 2019 IEEE First International Conference on Cognitive Machine Intelligence (CogMI), Dec 2019.