· ai-devtools
Cloudflare Auto Router, AI Gateway, rates requests by complexity, ambiguity, stakes, context dependence
The Auto Router, now in public beta via AI Gateway, evaluates each incoming request by assigning a stake, ranging from the most recent interactions.
Cloudflare's own testing revealed a benchmark where its router falls short compared to Claude Opus 5.5, yet it achieves this with about a third of the expense. Practitioners have begun to question how the concept of "stakes" remains viable across extended agent sessions.
The routing process involves two phases. First, AI Gateway establishes a pool of potential models, filtering those unable to handle the request format while applying constraints like access control, billing configuration, and spend limits.
Subsequently, a multi-head classification model operates across Cloudflare’s edge GPUs. This model assigns probabilities across 14 task categories and rates the request on four metrics: complexity, ambiguity, stakes, and reliance on prior context, using a formula defined as "utility = expected quality - adaptive cost penalty."
The Auto Router focuses its classification only on the newest turns, limited to 8 KiB of text. It then scores the candidate models based on task strength and anticipated gain for more difficult work, while factoring in price less as the task's difficulty increases.
Jacques B, a CTO engaged in agentic engineering, highlighted the critical nature of this metric on LinkedIn, stating, "A cheaper model isn’t much of a saving if someone has to spend another hour fixing the result." Cloudflare provided the data supporting this claim.
Regarding the classification itself, Igor Pyrin, a tech lead experienced with healthcare and banking platforms, pointed out a concern: stakes are scored from a concise view favoring recent turns. He noted that in a protracted agent session, a user might mention "this is a customer refund" early on, but later simply say "go ahead," suggesting an application-owned risk label should persist across turns.
The documentation offers two additions for auditability: the response includes `cf-aig-routed-model` and `cf-aig-routing-reason`, which can show values like `cost_optimal_within_pool` or `pinned_by_turn`. Furthermore, the pool can be restricted per request using `cf-aig-allowed-models` and `cf-aig-allowed-providers` headers.
A caution for developers reviewing the figures involves the default model pool, which currently includes Claude Opus 5, Sonnet 5, Fable 5, GPT-5.6 Luna, Sol and Terra, and Grok 4.5. The two baselines in the table, Opus 5.5 and GPT-6 Sol, are only accessible via the opt-in headers.
Session management also incurs a cost; without a session ID, requests within the same conversation might utilize different models, precluding prompt cache reuse. Cloudflare imposes a switching penalty that scales with context length because a new model might re-process reasoning tokens at output prices.
