Google has created a new framework called Project Naptime, which allows a large language model (LLM) to conduct vulnerability research and enhance automated discovery methods.

According to Google Project Zero, researchers Sergei Glazunov and Mark Brand, the Naptime architecture revolves around the interaction between an AI agent and a target codebase. The agent is equipped with a collection of specialized tools specifically designed to replicate the workflow of a human security researcher.

The initiative is titled "Naptime" because it enables humans to rest regularly while aiding in vulnerability research and automating variant analysis. The research found that refining the testing methodology to take advantage of modern LLM capabilities can significantly improve vulnerability discovery performance. They proposed guiding principles to facilitate the effective evaluation of LLMs for vulnerability discovery.

The team implemented these principles in their LLM-powered vulnerability research framework, which increased CyberSecEval2 benchmark performance by up to 20x from the original paper. This approach achieves new top scores of 1.00 on the "Buffer Overflow" tests (from 0.05) and 0.76 on the "Advanced Memory Corruption" tests (from 0.24).

Project Naptime

The system includes various elements, such as a Code Browser tool that allows the AI agent to navigate through the target codebase, a Python tool for running Python scripts in a controlled environment for fuzzing, a Debugger tool for observing program behaviour with different inputs, and a Reporter tool for monitoring task progress.

According to CYBERSECEVAL 2 benchmarks, Google stated that Naptime is both model-agnostic and backend-agnostic. Additionally, it is more effective at identifying buffer overflow and advanced memory corruption issues.

Proposed principles

Reviewing the existing publications on using LLMs for vulnerability discovery, the team found that many approaches contradict their intuition and experience. Over the last couple of years, they have been thinking extensively about how expertise can be used in "human-powered" vulnerability research to help adapt LLMs to this task. They also learned much about what does and doesn't work well (at least with current models). While modelling a human workflow is not necessarily an optimal way for an LLM to solve a task, it provides a soundness check for the approach. It allows for the possibility of collecting a comparative baseline in the future.

The principles proposed by the team are designed to condense the most important parts of what they have learned. They are designed to enhance the LLMs' performance by leveraging their strengths while addressing their limitations. The principles include:

• Space for reasoning
• Interactive Environment
• Specialized tools
• Perfect Verification
• Sampling strategy

The researchers have remarked that Naptime allows an LLM to conduct vulnerability research that closely emulates the iterative and hypothesis-driven process of human security experts. This architectural design improves the agent's capacity to detect and analyze vulnerabilities and ensures that the outcomes are precise and can be replicated.

Project Zero Blog