· ai-devtools

DeepSeek’s coding agent is too tidy for its own good — and benchmarks are punishing it

DeepSeek's open-source coding agent is losing points on a major benchmark for a habit developers usually treat as good practice: cleaning up after itself.


The problem surfaced when Hugging Face harness optimization researcher Joël Niklaus added DeepSeek Harness to his coding-agent leaderboard, which compares ten harnesses running the same underlying models. DeepSeek Harness is statistically tied for first across the board's five scored evaluations, yet on Terminal-Bench alone it drops to sixth, because one built-in behavior caused it to fail tasks that competing agents completed. DeepSeek Harness is statistically tied for first across the board's five scored evaluations, yet on Terminal-Bench alone it drops to sixth… "One habit explains much of that gap," Niklaus writes on X, referring to the agent's weaker Terminal-Bench performance.

A tidy habit backfires DeepSeek Harness shuts down background processes when a task finishes, which is a problem for three Terminal-Bench tasks that need those processes to keep running. Niklaus found that DeepSeek Harness passed just two of 36 attempts using DeepSeek-V4.1-Flash and GLM-5.3-Flash, while OpenCode, Pi, and Nous Research's Hermes Agent passed all 36 on those same tasks. DeepSeek Harness passed just two of 36 attempts … while OpenCode, Pi and Nous Research's Hermes Agent passed all 36 on those same tasks.

Related coverage

Also worth reading

  1. next.js 16.5.0-canary.50 released
  2. Node.js Brings Debounce, Throttle, and PKCS#12 Parsing Into Core Modules
  3. Google Cloud Moves the Compute Engine API to Immutable Date-Based Versions
  4. Open Source DocumentDB 1.0 Reaches General Availability
  5. You could have invented transformers from memory