Abstract—Today's software systems are becoming more complex, and automatic detection and code remediation of software vulnerabilities becomes increasingly important for ensuring secure and functionally correct software code. Detecting security flaws is currently fairly effective, but creating secure alternative is not very effective. To fill this gap, here is an integrated framework that, on the test set of 262 examples, achieves a BLEU score of 30.65 and a CodeBERT embedding similarity score of 0.9643 that is impressively high in terms of semantic similarity to reference secure code. Qualitative results also illustrate the system's ability to successfully mitigate against various types of vulnerability within several programming languages such as SQL Injection, XSS, Path Traversal, ServerSide Request Forgery (SSRF) and Command Injection. We call our method a hybrid fine-tuning and RAG approach by adding a fine-tuning step to the pre-trained CodeLlama-13b model.To obtain these results, we propose a hybrid method consisting of fine-tuning a pre-trained CodeLlama-13b and a RetrievalAugmented Generation (RAG) pipeline. Fine-tuning was performed through Quantized Low-Rank Adaptation (QLoRA) on a multilingual vulnerable and corrected code snippet dataset by quantizing to 4 bits.Fine-tuning is done using Quantized Low-Rank Adaptation (QLoRA) with a multilingual dataset of vulnerable code snippets and corrected code snippets in 4 bits quantization. The RAG pipeline uses cross-encoder reranking along with the BGE encoder, which is a feature of ChromaDB, to fetch relevant contextual knowledge from large CVE and CWE databases to prevent hallucinations and induce generation.

Parameter-Efficient Fine-Tuning,QLoRA,Large Language Models,CodeLlama,RAG,Secure Code Generation,Vulnerability Detection