· ai-devtools

Anthropic Claude writes, reviews 90% of code, uses automated checks to verify work

Claude Code's team utilizes an intricate system to validate the vast amount of code generated by the model, even though Claude itself is responsible for authoring and reviewing approximately 90% of…


Claude Code's team utilizes an intricate system to validate the vast amount of code generated by the model, even though Claude itself is responsible for authoring and reviewing approximately 90% of the code base. This verification process, detailed in a recent engineering post by Anthropic staff member Addy Osmani, is engineered to support numerous concurrent agents through automated checks that furnish engineers with the necessary evidence for judgment, rather than requiring manual inspection of every modification.

The system has prompted discussion among developers, with much of the focus landing on the financial implications of using a model for verification and whether less expensive, deterministic tooling could assume some of that responsibility. In response to inquiries from The New Stack, Anthropic provided a walkthrough detailing how it manages inference expenditures, prevents agents from manipulating tests, and assesses the efficacy of its automated reviewers.

The organization has Claude execute its proposed alterations within a live application environment; this involves leveraging browser automation for web applications, simulators linked to MCP for mobile builds, and local services for backend tasks. These directives reside within repository-specific verification skills located under .claude/skills/, allowing an agent to run and validate its own changes prior to committing them. Furthermore, Osmani directs developers toward employing property-based testing to uncover potential edge cases, alongside differential testing to reveal unexpected behavioral shifts between an existing implementation and a newly created one.

Not all observers accept the underlying cost calculations. Some developers expressed doubt regarding the justification for the added inference expenses, with Hacker News commenter OtherShrezzing suggesting that reusable Playwright tests could manage a significant portion of verification without necessitating repeated calls to a model. Anthropic informed The New Stack that inference expenditures fluctuate substantially between different modifications, and they keep these costs manageable by executing the inexpensive, deterministic checks before any model involvement occurs. The company stated, "Deterministic checks like the build, linters, unit tests and property tests go first, because once Claude writes them they run as ordinary software and cost very little to rerun." They added, "Model review is for what a test can’t answer, like whether a change takes the right approach or misses a case nobody wrote a test for." Anthropic declined to provide a precise figure for the increased inference cost associated with model-based validation.

A different concern was raised by Hacker News user xnorswap: the possibility of agents subtly altering code specifically to make a failing test pass. xnorswap noted, "It’s not the per-run costs I worry about, it’s the costs of churn," cautioning that an agent might implement a workaround or rewrite the test instead of addressing the root issue. Anthropic has established protocols governing how Claude addresses failed tests, and equally crucial, it dictates which sections of the repository its reviewer is permitted to trust. The company asserted, "Our rule is that if a change can modify the check that judges it, the check is broken." In practice, Code Review references REVIEW.md from the base branch rather than the changes submitted in a pull request, and any assessment is valid only for the precise commit it examined. Claude is prohibited from bypassing flaky tests, inserting sleep functions, or repeatedly running a test until it achieves a pass; if the agent cannot determine the source of the problem, it deletes the test entirely and documents the reason for its removal. The team also maintains a log detailing how every commit was verified, including instances where no verification took place whatsoever.

Regarding the 39 reliability checks the Claude Code team has gathered, Claude typically executes only between three and ten checks per modification. For instance, when changes involve session storage, Claude verifies if older transcripts still load with the –resume parameter. Commit gates mandate that developers validate changes either through /code-review or /verify. Teams possess the ability to monitor adherence before making these checks mandatory, and developers may opt to bypass a gate provided they offer a justification. Anthropic shared with The New Stack that between June and September, the volume of merged pull requests into Claude Code’s codebase more than doubled, while the percentage of changes that required being reverted decreased by over 40%. The firm did not specify how much of this improvement stemmed from its verification loops, nor did it offer separate statistics for production defects versus regressions.

To evaluate its own reviewers, Anthropic scores them against past incidents and reverts, reconstructing the repository as it existed at the time of review and creating a one-sentence answer key for each scenario; a reviewer only earns credit when their identification accurately describes the bug in question. Previously merged changes that remained stable serve as benchmarks for identifying false positives, and the company maintains a dedicated set of tests, isolated from the examples used to fine-tune the reviewers. Anthropic measures recall at various points throughout the review pipeline, and even if a reviewer successfully identifies a genuine bug, that finding is counted as a miss if it never reaches an engineer. The company stated it currently does not release the false-positive or false-negative rates derived from these Code Review assessments.

Anthropic’s methodology provides a structure for determining which checks necessitate model inference versus which can be handled more economically by deterministic tools, yet without published reviewer error rates or detailed inference cost data, developers lack substantial information against which to measure their own implementations. The company firmly stated to The New Stack, "A person is accountable for every change that ships, whoever typed it." Engineers are required to examine the verification evidence, which encompasses screenshots and test outcomes. Should a check fail after three attempts, Claude escalates the issue to an engineer.

Why it matters

This showcases a significant shift toward AI-driven development tooling, where LLMs handle the bulk of coding and review. Developers should watch how Anthropic balances high automation with rigorous, multi-layered verification to ensure code quality.

Related coverage

Also worth reading

  1. next.js 16.5.0-canary.59 released
  2. Masullo's GRH proof draft released after Conrad's feedback, Vakil notes
  3. Bell Canada confirms agentic hacker stole enterprise customer data on November 17 2026
  4. Cyber leaders find clear communication a front-line defense capability
  5. Meta releases Triton TLX, a Native GPU Compiler Extension for advanced hardware control