Cisco opens source Project Quarry tool to detect AI agents on hacking sprees

This tool, named Project Quarry, intends to establish a “shared language” for assessing agent conduct, much like Snort addressed malicious network traffic or YARA handled rule-based malware identification.

Most current detection methods depend on specific interactions with an AI model or agent, such as whether it attempted prompt injection or called an attack tool.

Amy Chang, head of AI threat and security research at Cisco, noted in an exclusive interview with The Register that “what is missing is the entire context under which something may occur.” She compared this to detecting credit card fraud, where isolated transactions might appear benign until further activity occurs.

Quarry addresses this gap by examining the complete agent session. Chang explained that while one tool call might seem acceptable, the subsequent step—like sending gathered information to an attacker-controlled server—can signal trouble.

Data theft itself is a sequence of events, not a singular action. This concept was demonstrated by Invariant Labs, which used a proof-of-concept attack against GitHub's MCP server to coerce a user's agent into leaking private repository data.

Chang and fellow Cisco researchers Adam Swanda, Ravikumar Balakrishnan, and Konstantin Berlin wrote that “A policy can produce a finding when the trace shows a sensitive file read followed within a defined window by an external send.” They emphasized that the risk stems from the relationship between actions, so findings must preserve evidence from both.

Quarry supports multi-turn conversations, agents, tool usage, MCP interactions, and multimodal inputs. The GitHub repository provides installation guides, supported Python versions, and plugins for agents compatible with Claude Code and Codex.

For example, using the provided rule pack, Chang stated, “If a base-64 encoded malware is somewhere in the tool call, that would result in a block.” The rule format mirrors YARA or Snort rules, detailing traffic, matching content, and tracking metadata.

The Cisco AI researchers believe that if the OpenAI-Hugging Face incident had been viewed as a complete session record, Quarry rules could have flagged several points. Chang mentioned that “The rule can flag specific tool calls that reach an external server asset that wasn’t in the task’s allow list.”

This capability is relevant because, as OpenAI’s report showed, an agent validated 14 exposed Hugging Face tokens with write access on July 10 and shared them. Chang noted, “The rule would flag that there was a credential in the tool result that then called an outside service, which is likely not within the parameters of what you would want to do in this case.”

The Cisco team clarified that these rules are not a replacement for human oversight like chain-of-thought monitors, but they do help capture patterns for other teams to “review, test, and apply to their own agents.” Chang concluded, “We all see slices of the problem, and it’s important for us to share what we see.”