Abstract

Manual security code reviews are slow and often miss subtle vulnerabilities due to human error and inconsistent skill levels. They aren’t scalable, making them costly as the codebase grows, and they typically happen periodically, leaving gaps in security. This reliance on individual expertise makes the process less reliable, underscoring the need for efficient and consistent AI-assisted reviews.

We have implemented a system for enhancing the current security code review practice, leveraging the capabilities of Codellama 13B, a locally hosted Large Language Model (LLM). Codellama 13B is optimized for deep understanding of programming languages and excels in identifying security vulnerabilities within codebases.

Key to our approach is the use of Retrieval-Augmented Generation (RAG). This technique integrates Codellama 13B’s generative capabilities with embeddings stored in ChromaDB, a specialized Vector Database designed for efficient retrieval of numerical representations (embeddings) of code snippets. These embeddings encode semantic relationships and context, enabling Codellama 13B to retrieve and analyze relevant information swiftly during reviews.

Facilitating interaction with Codellama 13B and ChromaDB is AnythingLLM, an interface that provides a prompt-based environment for querying and receiving responses from the model. The configuration includes a token context window of 25,000 tokens, allowing Codellama 13B to consider extensive code segments and dependencies, which enhances its ability to provide detailed security assessments.

By integrating Codellama 13B, ChromaDB, and AnythingLLM with RAG, we have significantly enhanced our capacity to conduct thorough and efficient security reviews. This setup not only enhances the accuracy and depth of vulnerability detection but also accelerates remediation efforts, thereby enhancing the security posture of their software.