Published May 21, 2025 | Version v1
Dataset Open

Replication Package for: Enhancing Task Prioritization in Software Development Issues Tracking System

Authors/Creators

  • 1. EDMO icon University of Oslo

Description

This Zenodo record provides the code, scripts, notebooks, and links to datasets and models used to support the findings for Research Questions 1 through 5 (RQ1-RQ5) in the paper "Enhancing Task Prioritization in Software Development Issues Tracking system." Modern software development faces a critical bottleneck in manually prioritizing issues. This paper investigates automated issue priority classification using Transformer models. We evaluate models like BERT, DeBERTa, and a specialized ModernBERT, comparing them against general Large Language Models (LLMs) such as Qwen2.5-3B and Llama-3.2-3B, using curated datasets from Jira and GitHub. This package contains the necessary components to replicate the in-distribution classification (RQ1), out-of-distribution generalization (RQ2), fine-tuning impact assessments (RQ3), detailed performance analysis of ModernBERT across priority levels (RQ4), and the comparative performance of LLMs against ModernBERT (RQ5). The main paper demonstrates that Transformer models, particularly ModernBERT, achieve high classification performance (e.g., accuracy > 81%, AUC > 0.90, MCC > 0.62), significantly outperforming the evaluated general LLMs for this task.

Other

Related Hugging Face Repositories

Hugging Face Model Repositories (Source of pre-trained models/fine-tuned models used/evaluated in the paper):

Hugging Face Dataset Repositories (Source of datasets used/generated as per paper contribution):

Other

  • Priority_RQ1toRQ3/ (Folder containing files for RQ1-RQ3)
    • .vscode/ (VSCode editor specific settings - optional)
    • extra_code/ (Supplementary code or notebooks)
    • jira/ (Scripts or data related to Jira dataset processing/analysis)
    • priority/ (Scripts or notebooks for priority classification tasks)
    • priority_model_deberta/ (Scripts/notebooks specific to DeBERTa model experiments for RQ1-RQ3)
    • scripts_shared/ (Shared utility scripts)
    • .gitignore (Git ignore file)
    • github_script.sh (Shell script for running RQ1-RQ3 experiments, e.g., Papermill)
    • README.md (README specific to RQ1-RQ3 components)
    • requirements_local.txt (Python dependencies for local execution)
    • requirements_ml_nodes.txt (Python dependencies for HPC/ML Nodes environment)
    • 01_train_high_vs_med_low_top50_hp.ipynb (Example Jupyter notebook for RQ1-RQ3 training)
  • High_priority_llm_classification.ipynb (Jupyter notebook for LLM classification - High Priority - RQ5)
  • Low_priority_llm_classification.ipynb (Jupyter notebook for LLM classification - Low Priority - RQ5)
  • Medium_priority_llm_classification.ipynb (Jupyter notebook for LLM classification - Medium Priority - RQ5)
  • ModernBERT.ipynb (Jupyter notebook for ModernBERT analysis - RQ4)

Files

ModernBERT.ipynb

Files (41.2 MB)

Name Size Download all
md5:252b2b82969a9823c16412e8d96e6d9b
1.7 MB Preview Download
md5:95cacb074021cf8a656cc24235c0aeaf
1.0 MB Preview Download
md5:167cfbc537868e381f71d172d06330dc
1.2 MB Preview Download
md5:674f370d2776b619df773faa4f28ac17
25.2 kB Preview Download
md5:ce3c0666a24d0e318713f889bd96c8d2
37.2 MB Preview Download