Ollama
A lightweight runtime with command-line model management and a local background service.
Best for: Scriptable local development and terminal-first workflows
Setup guideLocal inference, your hardware
Local AI Coding with Kilo
Connect Kilo in VS Code, JetBrains, or the CLI to Ollama, LM Studio, or a compatible endpoint. Keep model inference on hardware you control without giving up agentic coding workflows or model choice.
Choose a local runtime
The runtime manages and serves the model. Kilo connects that model to coding agents, repository context, file edits, and terminal tools.
LM Studio
A desktop model browser with download, quantization, memory, and local server controls.
Best for: Visual model discovery and desktop runtime management
Setup guideOpenAI-compatible endpoints
Connect a compatible local, LAN, or on-prem endpoint, including runtimes such as vLLM or SGLang.
Best for: Team-shared inference and controlled infrastructure
Setup guideWhat determines success
Hardware fit
Artifact size, quantization, context, runtime, and workload determine fit. Active parameter count alone is not enough.
Agent reliability
Chat quality does not guarantee reliable file edits or tool calls. Validate the exact model on bounded coding tasks.
Data boundaries
Local inference keeps model requests on the endpoint you configure. Other enabled integrations can still use the network.
Total cost
There is no hosted token bill, but you still provide hardware, power, storage, setup, and maintenance.
Where local models work in Kilo
Local model support applies to clients that can reach the endpoint. It does not make every cloud or integration feature local.
| Surface | Local model support | Boundary |
|---|---|---|
| VS Code extension | Yes | Runs local coding workflows against the configured local provider. |
| JetBrains plugin | Yes | Uses the local provider configured for the plugin. |
| Kilo CLI | Yes | Runs terminal workflows against the configured local provider. |
| Cloud Agents and hosted services | No direct localhost access | Cloud execution cannot call a runtime bound only to your development machine. |
Local inference is a boundary
Prompts sent to your local model stay on the endpoint you configure. That statement does not automatically cover sign-in, software updates, remote MCP servers, telemetry choices, browser tools, Cloud Agents, or hosted review services.
For isolated environments, inventory each enabled integration, bind the runtime appropriately, and test the complete workflow rather than assuming a local model makes every dependency offline.
Local, hosted, or hybrid
Local models work well for bounded tasks, sensitive repositories, experimentation, and predictable workloads. Hosted frontier models often provide better latency, context, and tool reliability for demanding agents.
Kilo keeps the workflow portable: use local inference for one task, BYOK for another, and Kilo Gateway when managed access is the better operational tradeoff.
Compare inference optionsLocal model FAQ
What is a local LLM coding assistant?
A local LLM coding assistant sends model requests to inference software running on hardware you control instead of a hosted model API. Kilo provides the agent workflow while Ollama, LM Studio, or another compatible endpoint serves the model.
Does Kilo support local models in VS Code, JetBrains, and the CLI?
Yes. Kilo supports local model providers in the VS Code extension, JetBrains plugin, and MIT-licensed CLI. Cloud-hosted workflows have different network and inference requirements.
Does a local model mean no data ever leaves my machine?
Local inference keeps prompts sent to that model on the endpoint you configure. Account access, updates, remote MCP servers, telemetry choices, Cloud Agents, hosted code review, and other integrations have separate data paths. Review or disable each networked dependency for an offline or isolated workflow.
Are local coding models free?
They avoid hosted token charges when you operate them on your own hardware. Hardware, electricity, storage, setup, maintenance, and slower developer time can still make local inference costly.
Start with the runtime you prefer
Install Kilo, connect Ollama or LM Studio, then validate the exact model on a small coding task before increasing context or autonomy.