Impact of Intermediate-Task Training on Inference Latency in XTREME-R Code Generation

Deep learning (DL) techniques have been used to support several code-related tasks such as code summarization and bug-fixing. In particular, pre-trained transformer models are on the rise, also thanks to the excellent results they achieved in Natural Language Processing (NLP) tasks. The basic idea behind these models is to first pre-train them on a generic dataset using a self-supervised task (e.g, filling masked words in sentences). Then, these models are fine-tuned to support specific tasks of interest (e.g, language translation). A single model can be fine-tuned to support multiple tasks, p

Research goal: How does intermediate-task training on code-related tasks (e.g., code summarization, bug-fixing) affect inference latency on XTREME-R code generation tasks while maintaining accuracy?

Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 7.7/10.