BalancedCommitBench: A Language-Balanced Commit Messages Subset Extracted from CommitBench:

BalancedCommitBench: A Language-Balanced Commit Messages Subset Extracted from CommitBench:

We base our experiments on CommitBench, a large-scale benchmark for commit message generation introduced by Schall et al.(2024) . CommitBench aggregates more than one million real commits collected from thousands of open-source repositories across six major programming languages: Python, Java, JavaScript, Go, PHP, and Ruby. Each instance contains a git diff, the corresponding human-written commit message, and metadata such as author, timestamp, and repository name.

From this corpus, we derive BalancedCommitBench, a language-balanced subset obtained through systematic preprocessing. The dataset is cleaned via language normalization, de-duplication, removal of bot-generated and low-information commits, and length-based filtering, followed by uniform sampling across the six languages to ensure balanced representation.

M. Schall, T. Czinczoll and G. De Melo, "CommitBench: A Benchmark for Commit Message Generation," 2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), Rovaniemi, Finland, 2024, pp. 728-739, doi: 10.1109/SANER60148.2024.00080.