FRANC: A Framework for Improving the Quality of Automatically Generated Code

In recent years, the use of automated source code generation utilizing transformer-based generative models has expanded, and these models can generate functional code according to the requirements of the developers. However, recent research revealed that these automatically generated source codes contain quality flaws and vulnerabilities. Despite researchers and practitioners attempting to enhance code generation quality, retraining large language models is time-consuming and resource-intensive. As a result, our work presents FRANC, a framework for recommending more secure and high-quality source code derived from transformer-based code generation models. FRANC includes a static filter to make generated code compilable with heuristics and a quality filter to rank the suggestions based on quality (i.e., code and security smells). In addition, FRANC contains techniques for refactoring vulnerable source code. The framework has been instantiated with five Python and Java code generation models and six prompt datasets, including a newly proposed one in this work, SOEVAL. The static filter improves 9% to 46% Java suggestions and 10% to 43% Python suggestions regarding compilability. The average improvement over the NDCG score for the ranking system is 0.0763, and the repairing techniques repair the highest 80% of prompts. The framework, FRANC takes, on average, 1.9845 seconds for Java; for Python, it takes 0.0846.