Llama3-70B and Gemini 1.5 Pro Performance on Adversarial CodeXGLUE Security Benchmarks

This report synthesises findings from 1 peer-reviewed paper addressing the following research question: How does the performance of Llama3-70B and Gemini 1.5 Pro compare on the CodeXGLUE security subset under adversarial perturbations when evaluated using BLEU score instead of EM (Exact Match). Background: Retrieval-augmented generation (RAG) aims to reduce hallucinations and outdated knowledge by grounding LLM outputs in retrieved evidence, but empirical results are scattered across tasks, systems, and metrics, limiting cumulative insight. Objective: We aimed to. 10 claims were extracted from source literature; 10 were independently verified against retrieved documents. An automated multi-reviewer quality assessment produced a score of 8.4/10. This report is a machine-generated literature synthesis and does not constitute original research.

Research goal: How does the performance of Llama3-70B and Gemini 1.5 Pro compare on the CodeXGLUE security subset under adversarial perturbations when evaluated using BLEU score instead of EM (Exact Match)?

Autonomous literature synthesis. Automated review score: 8.4/10. Full text and citation available at Assignee Research.