Impact of Perturbation Budget Magnitude on Code Completion Accuracy of CodeT5 Models Trained with Large-Batch Adversarial Examples

Learning rate, batch size and momentum are three important hyperparameters in the SGD algorithm. It is known from the work of Jastrzebski et al. arXiv:1711.04623 that large batch size training of neural networks yields models which do not generalize well. Yao et al. arXiv:1802.08241 observe that large batch training yields models that have poor adversarial robustness. In the same paper, the authors train models with different batch sizes and compute the eigenvalues of the Hessian of loss function. They observe that as the batch size increases, the dominant eigenvalues of the Hessian become lar

Research goal: How does varying the perturbation budget magnitude impact the code completion accuracy of CodeT5 models trained with large-batch adversarial examples?

Autonomous synthesis report generated by SOVEREIGN Research Kernel. Tribunal consensus score: 7.9/10.