PreNeT: Leveraging Computational Features to Predict Deep Neural Network Training Time
Description
We have developed a framework that allows users to predict the training time of various deep learning models and architectures. This is achieved by gathering execution times of different deep learning layer types under diverse configurations.
After collecting this data, which is shown in the figure below, we trained and evaluated multiple machine learning models for the task of predicting training times for individual layers. The best-performing model for each layer type was selected and used to provide accurate predictions of training times. This approach helps users estimate model performance and make informed decisions when designing or optimizing neural networks.
You can find more details about this framework and the experiments in our paper published in ACM/SPEC ICPE 2025. You can access the paper here: arxiv link
Setup
To get started, follow the steps below:
Clone the repository:
cd PreNet
Create and activate a virtual environment:
source ./venv/bin/activate
python -m pip install -r requirements.txt
To gather training data for a specific GPU, navigate to the data_collection directory and run a benchmark. For example, to test RNN models, use the following command:
cd data_collection
python run.py --testRNN --num_val=20000 --repetitions=5 --logdir=../data
This command will run the benchmark for RNN models with 20,000 validation examples and repeat the process 5 times to gather data. The results will be saved in the data directory.
Sample Data
The data directory contains a sample of the data (in the sample_dataset folder) we collected for different layer types and GPUs. The data is stored in CSV files, with each row representing a different configuration. It is one-fourth of the original data we collected for the paper. The full dataset is available in the full_dataset folder.
Files
PreNeT.zip
Files
(40.5 MB)
| Name | Size | Download all |
|---|---|---|
|
md5:2b2a514cf8cae863384dd427563e5980
|
40.5 MB | Preview Download |
Additional details
Dates
- Accepted
-
2024-12-1916th ACM/SPEC International Conference on Performance Engineering (ICPE 2025)
Software
- Repository URL
- https://github.com/pacslab/PreNeT
- Programming language
- Jupyter Notebook , Python
- Development Status
- Active