Development of software and algorithms of parallel learning of artificial neural networks using CUDA technologies
- 1. Lviv Polytechnic National University
- 2. Ivan Franko National University of Lviv
- 3. Ukrainian National Forestry University
Description
The object of research is to parallelize the learning process of artificial neural networks to automate the procedure of medical image analysis using the Python programming language, PyTorch framework and Compute Unified Device Architecture (CUDA) technology. The operation of this framework is based on the Define-by-Run model. The analysis of the available cloud technologies for realization of the task and the analysis of algorithms of learning of artificial neural networks is carried out. A modified U-Net architecture from the MedicalTorch library was used. The purpose of its application was the need for a network that can effectively learn with small data sets, as in the field of medicine one of the most problematic places is the availability of large datasets, due to the requirements for data confidentiality of this nature. The resulting information system is able to implement the tasks set before it, contains the most user-friendly interface and all the necessary tools to simplify and automate the process of visualization and analysis of data. The efficiency of neural network learning with the help of the central processor (CPU) and with the help of the graphic processor (GPU) with the use of CUDA technologies is compared. Cloud technology was used in the study. Google Colab and Microsoft Azure were considered among cloud services. Colab was first used to build a prototype. Therefore, the Azure service was used to effectively teach the finished architecture of the artificial neural network. Measurements were performed using cloud technologies in both services. The Adam optimizer was used to learn the model. CPU duration measurements were also measured to assess the acceleration of CUDA technology. An estimate of the acceleration obtained through the use of GPU computing and cloud technologies was implemented. CPU duration measurements were also measured to assess the acceleration of CUDA technology. The model developed during the research showed satisfactory results according to the metrics of Jaccard and Dyce in solving the problem. A key factor in the success of this study was cloud computing services.
Files
Development of software and algorithms of parallel learning of artificial neural networks using CUDA technologies.pdf
Files
(194.0 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:298d787eb22bd97c247871a0513425f9
|
194.0 kB | Preview Download |
Additional details
References
- Sokolovskyy, Y. I., Shymanskyi, V. M., Mokrytska, O. V., Kharko, Y. V. (2019). Neural network model for identification of material creep curves using CUDA technologies. Ukrainian Journal of Information Technology, 1 (1), 11–16. doi: http://doi.org/10.23939/ujit2019.01.011
- Manokhin, D. (2021) Prohramno alhorytmichne zabezpechennia rozparalelennia protsesu navchannia shtuchnykh neironnykh merezh z vykorystanniam tekhnolohii CUDA. Mizhnarodna studentska naukova konferentsiia z pytan prykladnoi matematyky ta kompiuternykh nauk (MSNKPMK-2021). Lviv. Available at: https://ami.lnu.edu.ua/wp-content/uploads/2021/05/Ministerstvo-osvity-i-nauky-Ukrainy.docx
- Ambros, R., Waltham, R. et. al. (2021). Godfrey Hounsfield. Available at: https://radiopaedia.org/articles/godfrey-hounsfield?lang=us
- Bell, D. J., Mirjan, Pr., Nadrljanski, M. et. al. (2021). Computed tomography. Available at: https://radiopaedia.org/articles/computed-tomography
- Bell, D. J., Greenway, K. et. al. (2021). Hounsfield unit. Available at: https://radiopaedia.org/articles/hounsfield-unit
- Goodfellow, I., Bengio, Yo., Courville, A. (2016). Deep Learning. MIT Press, 781.
- Ciresan, D. C., Gambardella, L. M., Giusti, A. (2012). Deep neural networks segment neuronal membranes in electron microscopy images. NIPS, 2852–2860.
- Ronnenbergerm, O., Fischerm, P., Broxm, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention (MICCAI). Springer, LNCS, 9351, 234–241. doi: http://doi.org/10.1007/978-3-319-24574-4_28
- Chilamkurthy, S., Ghosh, R., Tanamala, S., Biviji, M., Campeau, N. G., Venugopal, V. K., Warier, P. (2018). Development and validation of deep learning algorithms for detection of critical findings in head CT scans. arXiv preprint. Available at: https://arxiv.org/abs/1803.05854
- RSNA Intracranial Hemorrhage Detection (2019). Radiological Society of North Ameriaca. Available at: https://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/overview
- Hssayeni, M. D., Croock, M. S., Salman, A. D., Al-khafaji Hassan Falah, Yahya, Z. A., Ghoraani, B. (2020). Intracranial Hemorrhage Segmentation Using a Deep Convolutional Model. Data, 5 (1), 14. doi: http://doi.org/10.3390/data5010014
- Goldberger, A. L., Amaral, L. A., Glass, L., Hausdorff, J. M., Ivanov, P. C., Mark, R. G. et. al. (2000). PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals. Circulation, 101 (23), E215–E220. doi: http://doi.org/10.1161/01.cir.101.23.e215
- Perone, C. S., Cclauss, Saravia, E., Ballester, P. L., Tare, M. (2018). Perone/medicaltorch: Release v0.2 (v0.2). doi: https://doi.org/10.5281/zenodo.1495335
- Hssayeni, M. (2020). Computed Tomography Images for Intracranial Hemorrhage Detection and Segmentation. PhysioNet. 1.3.1. doi: https://doi.org/10.13026/4nae-zg36
- Tokui, S., Oono, K. (2015). Chainer: a Next-Generation Open Source Framework for Deep Learning. Available at: http://learningsys.org/papers/LearningSys_2015_paper_33.pdf
- Hu, W., Miyato, T., Tokui, S., Matsumoto, E., Sugiyama, M.. (2017). Learning Discrete Representations via Information Maximizing Self-Augmented Training. Proceedings of the 34th International Conference on Machine Learning Proceedings of Machine Learning Research, 70, 1558–1567. Available at: https://proceedings.mlr.press/v70/hu17b.html
- PyTorch Documentation (2021) Available at: https://pytorch.org/docs/stable/index.html
- Colaboratory Frequently Asked Questions (2021). Available at: https://research.google.com/colaboratory/faq.html
- How Azure Machine Learning works: Architecture and concepts (2020). Available at: https://docs.microsoft.com/en-us/azure/machine-learning/concept-azure-machine-learning-architecture