TL;DR
- An AI gateway (or model router) is a layer that sits between your tools and the AI providers, giving you one API, one bill, and one place to switch models — instead of wiring every provider's SDK into every tool by hand.
- They exist because the number of usable frontier and open-weight models has exploded, and because per-developer AI spend keeps climbing: the top AI-adopting firms now spend roughly $90,000/year per employee on AI, more than 650x the median firm.
- Automatic routing is useful when you want strong results without evaluating every model yourself. Picking a specific model still makes sense when you need predictable behavior, a particular capability, or direct control over cost and latency.
- Betting a whole workflow on one provider is a real risk — models get repriced, throttled, or pulled entirely with no notice. A neutral, provider-agnostic routing layer is what protects you from that.
- Kilo's answer is to stay open source and vendor-agnostic: bring your own gateway, bring your own keys (BYOK), or use Kilo's own Auto Model routing — across VS Code, JetBrains, and the CLI.
What is an AI gateway or model router?
An AI gateway is a single API endpoint that sits in front of many different AI model providers — Anthropic, OpenAI, Google, Mistral, Moonshot AI, DeepSeek, and dozens of others — and normalizes them behind one interface. Instead of your application or coding agent integrating a separate SDK, auth scheme, and pricing model for every provider, it talks to the gateway once, and the gateway forwards the request to whichever backend model you (or a routing policy) selected.
A model router does the harder part on top of that: it decides which model should handle a given request. That decision can be as simple as "always use the model the user picked," or as sophisticated as classifying the task, checking live pricing and provider capacity, and sending the request to whichever model gives the best result for the cost.
OpenRouter is the best-known example of this category — a marketplace that gives developers a single API to reach hundreds of models from dozens of providers. It became infrastructure important enough that Stripe reportedly explored a $10 billion acquisition of it. Other gateways and routers worth knowing include cloud-native options like Amazon Bedrock and Google Vertex AI (which route within one cloud's model catalog), self-hosted proxies like LiteLLM, and product-level routers built into coding tools themselves — Kilo's Kilo Gateway and Auto Model among them.
Why this category exploded
Two years ago, "which model should I use" was a short conversation: pick the best closed frontier model you could afford, and move on. That question has gotten genuinely hard. In the last few months alone, teams have had to weigh Claude Sonnet 5, Claude Opus 5, GPT-5.6, Grok 4.5, Gemini 3.6 Flash, Kimi K3, and GLM-5.2 against each other — each with different pricing, different strengths per task, and different availability depending on the provider and region. As Kilo's Ari Messer put it, the industry has moved from asking whether a model is smart enough to do the job to asking which model is the right fit for this specific task, at what cost.
Trying to track that by hand — reading model release notes, re-benchmarking, and manually switching your tools every time a new model ships — is not a sustainable habit for an individual developer, let alone an engineering org with hundreds of them. A gateway solves the integration problem (one API, one key, one bill). A router solves the decision problem (which model, for this task, right now). Together they're the reason "model choice" stopped being a one-time setup decision and became something that can adapt automatically, request by request.
The economics: why routing is now a budget line, not a nice-to-have
The case for gateways and routers isn't just architectural convenience — it's increasingly a cost-control necessity. Ramp's AI Index, tracking real corporate card spend, found that the top 1% of "AI-pilled" firms now spend about $7,500 per employee per month on AI — roughly $90,000 a year, versus about $11.38/month at the median firm. That's a gap of more than 650x, and it's widening: top-spender per-employee AI spend grew over 14% in a single month.
Ramp's own data shows the heaviest spenders aren't the ones defaulting every task to the most expensive model available — they're the ones who "mix and match," bouncing between multiple frontier and open-weight models depending on the job. The spend is high because usage is high, not because routing failed to control it. The firms in actual budget trouble are the ones that locked their whole workflow to a single vendor and got hit when that vendor's pricing or availability changed under them — GitHub's move to usage-based Copilot billing being the clearest recent example, and Uber reportedly burning through its entire annual AI coding budget in four months as a result.
There's a second economic force behind this, separate from per-developer spend: the compute bill sitting underneath all of it. The hyperscalers funding most of today's AI compute buildout have seen their combined forward free cash flow fall by roughly 90% over 18 months, even as usage and headcount at AI-adopting companies keep growing. That gap between money spent building capacity and money coming back is exactly the kind of pressure that gets passed down to customers as price changes, tighter limits, or new billing models with little warning. None of that is hypothetical — it already happened with Copilot. A gateway or router that isn't tied to one vendor is the mechanism that lets a team absorb a pricing shift on one provider by shifting traffic elsewhere, rather than eating the increase or ripping out tooling under deadline pressure.
Routing also has a more direct effect on the bill: the most expensive part of a request usually isn't the per-token rate, it's how many tokens a model burns to finish the job. Frontier models tend to reason longer and write more, so a premium rate times a premium token count compounds fast. Sending routine work to a cheaper or open-weight model that's good enough for the task, and reserving frontier reasoning for the steps that actually need it, is where most of the savings in KiloBench testing has come from.
Auto routing vs. picking your own model
The honest answer to "should I let a router pick the model, or choose it myself?" is: it depends on the step.
Kilo ran a controlled experiment on this — planning and building the same backend service three ways: with Auto Model routing (frontier tier for planning, efficient tier for implementation), with one fixed frontier model end-to-end, and with one fixed mid-tier model end-to-end. The results, detailed in full here, were telling:
- All three setups produced functionally identical services, passing the same logic checks. Once a solid plan exists, the model that implements it barely changes the correctness of the result.
- The Auto Model pipeline was the cheapest of the three — it spent up on the planning step, where model quality matters most, and spent down on implementation, where it didn't need to.
- The Auto Model run was also the only one that matched the fixed API contract exactly. The fixed frontier model produced sound logic but drifted from the specified field names and response shapes — a reminder that "smarter" doesn't automatically mean "more compliant with the spec you gave it."
The pattern that keeps showing up across these tests: automatic routing earns its keep on high-volume, well-specified work where "good enough, cheap, and fast" beats "maximal capability, expensive, and slow." Picking a specific model by hand still matters when you need a particular model's known behavior.
Why model choice became existential, not optional
Two dynamics make this more than an optimization exercise.
Commoditization is real, but it doesn't mean compute gets cheap. After DeepSeek's January 2025 release rattled markets, the assumption flipped from "closed frontier models are an unbreakable moat" to "open models make the frontier worthless." Both readings miss what's actually happening. Recent open-weight releases like Kimi K3 and GLM-5.2 are genuinely competitive on intelligence benchmarks, but serving a model with hundreds of billions of parameters at scale still requires real GPU capacity — reportedly millions of dollars for the hardware alone for a competitively-provisioned rack. When Kimi K3 got popular, its own provider had to pause new subscriptions and split capacity between chat and coding traffic because throughput fell off a cliff. Open weights don't eliminate the capacity constraint — they just mean more parties can compete to serve a given model well, which is precisely the situation routing across providers is built to take advantage of.
Capacity itself is volatile, independent of pricing. The clearest cautionary example: Anthropic's Fable 5 was, for a short window, the best coding model on the market — and then a US export-control directive pulled it for every customer, including paying enterprise accounts, with no notice. A model you've built a workflow around can become unavailable for reasons that have nothing to do with your relationship with the vendor. This is also the backdrop for the OpenRouter acquisition interest and Anthropic's own Opus 5 pricing move (undercutting its own Fable 5 pricing by half) — the market is visibly repricing around the assumption that buyers won't stay loyal to one model or provider by default. As Kilo's Ari Messer wrote when Opus 5 landed alongside the OpenRouter news, the walled gardens are falling, and a routing layer only pays off if it's free to route across the whole market — not just one vendor's catalog.
That's also why the routing layer itself should be independent of any single model vendor. A router built by a lab will, understandably, tend to favor that lab's own models. A router with no models of its own to protect has no reason to do anything but match the work to whichever model actually wins it.
Kilo's approach: flexible by design, not locked to a gateway
Kilo doesn't ask teams to bet on one routing strategy. The platform is built so you can combine approaches depending on what a given team or task needs:
- Bring your own gateway. If your org already routes traffic through OpenRouter, Bedrock, Vertex AI, or an internal LiteLLM proxy, Kilo works with it — you're not required to rip out existing infrastructure to adopt Kilo.
- BYOK (bring your own key). Connect your own Anthropic, OpenAI, Google, or other provider keys directly, at zero markup on tokens, and keep using existing provider agreements and credits.
- Kilo's own Auto Model routing. For teams that want the decision handled for them, Auto Model offers Frontier, Balanced, Efficient, and Free tiers, routing each request to whatever model its public benchmarks — KiloBench — show is the right fit for the task, at the tier's price point. Auto Efficient in particular is built to deliver a large share of frontier-level completion at a fraction of the cost.
- Local models. Ollama and LM Studio support means teams with strict data-handling requirements can route sensitive work to a model that never leaves their infrastructure.
The common thread is that none of it locks you in. Kilo's extensions and CLI are MIT-licensed, with a 500+ model catalog spanning every major closed and open-weight family. You can mix all of the above in the same organization: BYOK for one team, Auto Model for another, an external gateway for a third — and the surface you work in doesn't change that flexibility. Kilo runs as a genuine VS Code extension, a native JetBrains plugin, and a standalone CLI, so the same routing choices are available wherever your team actually writes code.
Data sovereignty and where your requests actually run
Model and gateway choice isn't only a cost and reliability question — for a growing number of teams, it's a compliance question. Which provider handles a request also determines which jurisdiction that data touches, which matters for teams operating under GDPR, sector-specific data residency rules, or internal policies about where source code and prompts are allowed to be processed.
This is part of why routing flexibility and sovereign AI in Europe have become linked conversations. A gateway or router that can only reach models hosted in one region gives compliance-conscious teams no lever to pull. One that can route to EU-hosted infrastructure when it matters — without abandoning access to the rest of the model market for everything else — gives teams a way to satisfy data residency requirements on the workloads that need it, while keeping full model choice on the ones that don't. Kilo's EU offering is built around exactly that: European teams get sovereign hosting options without giving up the same open, multi-provider routing available everywhere else Kilo runs.
Enterprise controls: freedom within approved boundaries
Model choice does not have to mean every developer can send code to every available provider. With Kilo Enterprise, administrators can approve specific providers, restrict model access, and keep unsupported routes out of engineering workflows. Developers retain the freedom to use the models that fit their work, while security and compliance teams control which options are available.
These controls help organizations enforce their provider and data-handling policies consistently across teams. By limiting access to approved models, providers, and regional endpoints, enterprises can keep code and prompts within the routes covered by their privacy, security, and compliance requirements. The resulting guarantees still depend on the selected providers, endpoints, and agreements.
The bottom line
The number of usable models has grown faster than any individual developer's ability to track them by hand, and per-developer AI spend is rising fast enough that routing has become a real budget lever, not a technical curiosity. Auto routing is a strong default for the high-volume, well-understood share of the work; deliberate model selection still earns its place on the hard reasoning and the steps where exact compliance to a spec matters most. What ties both together is avoiding a bet on any single model or provider — because pricing, capacity, and even access can all change with no warning, as several of the last year's biggest AI headlines have shown.
Kilo is built to be the neutral layer underneath that decision: open source, provider-agnostic, and usable with your own gateway, your own keys, or its own Auto Model routing — in VS Code, JetBrains, or the CLI. Get started with Kilo for free and route your next task to whatever model actually earns it.
Frequently asked questions
What's the difference between an AI gateway and a model router?
A gateway gives you one API and one bill across multiple AI providers — it solves integration. A router decides which model handles a given request — it solves the model-selection problem. Many products, including OpenRouter and Kilo's Gateway and Auto Model, combine both: a single endpoint plus routing logic on top of it.
How is Kilo Gateway different from OpenRouter?
Both provide a single interface for accessing models from multiple providers. Kilo Gateway is integrated directly into Kilo's coding tools, with unified billing, no markup on model usage, and access to 500+ models. OpenRouter is a general-purpose model marketplace and API that can be used by many different applications, including Kilo.
How is Kilo's Auto Model different from OpenRouter?
Auto Model is Kilo's coding-specific routing policy. It selects models based on the task and is evaluated against Kilo's agent harness, KiloBench. OpenRouter primarily provides access to models and provider-level routing, while Auto Model decides which model is best suited to handle a coding task.
Should I always let a router pick the model automatically?
In most cases, yes. Let automatic routing choose the right model for the task at hand. For larger tasks, break the work into stages: use Auto Frontier to create the plan, then Auto Efficient to implement it. Select a model by hand only when you need that model's specific behavior.
Why does model and provider lock-in matter for AI coding tools?
Because pricing, availability, and even access to a specific model can change with little or no notice — a provider can raise prices, throttle capacity, or have a model pulled entirely by regulatory action. Teams that build their workflow around a single model or vendor absorb that risk directly. A provider-agnostic gateway or router like Kilo Gateway lets a team shift traffic elsewhere when that happens, rather than being stuck.
Does using a gateway or router affect data sovereignty and compliance?
Yes — which provider and region processes a request determines which jurisdiction's rules apply to that data. Teams with GDPR or other data residency requirements benefit from a routing layer that can direct specific workloads to compliant infrastructure (such as EU-hosted options) while keeping broader model choice for everything else.
Is Kilo tied to a specific gateway or model provider?
No. Kilo is open source and designed to be provider-agnostic. You can bring your own gateway, bring your own API keys (BYOK) across 500+ models, run local models via Ollama or LM Studio, or use Kilo's own Auto Model routing — in any combination, across VS Code, JetBrains, and the CLI.