8 GB
No registry-backed coding recommendation is currently published for this tier.
Hardware-first local coding guide
Best Local LLM for Coding in 2026
Choose a local coding model by hardware tier, not by model family hype. Each pick lists the exact artifact, runtime, quantization or precision, memory assumption, usable context, and whether the hardware fit is tested, official, or estimated.
Quick Picks By Hardware Tier
16 GB
No registry-backed coding recommendation is currently published for this tier.
24 GB
Devstral Small 2505
Mistral documents single-RTX-4090 or 32 GB Mac deployment; usable context still depends on quantization and memory headroom.
- Runtime
- Ollama
- Quantization
- 4-bit quantization
- Memory
- 32 GB unified memory or a 24 GB RTX 4090 class GPU
- Evidence
- official
48 GB
Devstral Small 2505
Use a smaller context if latency or memory pressure is high.
- Runtime
- LM Studio
- Quantization
- 4-bit or 8-bit quantization
- Memory
- 48 GB unified memory or GPU VRAM class
- Evidence
- estimated
Server-class
GLM 5.2
Use hosted or server deployment for practical coding-agent latency.
- Runtime
- vLLM
- Quantization
- BF16 or deployment-specific quantization
- Memory
- Server-class GPU memory
- Evidence
- official
Qwen3 Coder 480B-A35B Instruct
The full checkpoint is a server-class deployment; use a hosted route or reviewed quantization for smaller hardware.
- Runtime
- vLLM
- Quantization
- BF16 or deployment-specific quantization
- Memory
- Multi-GPU/server memory required
- Evidence
- official
Kimi K3
The official model card recommends vLLM, SGLang, or TokenSpeed; this is not a consumer-hardware recommendation.
- Runtime
- SGLang
- Quantization
- Native MXFP4 weights with MXFP8 activations (quantization-aware training)
- Memory
- Server-class GPU memory
- Evidence
- official
DeepSeek V4
Confirm access, license, precision, and memory requirements for the exact artifact before self-hosting.
- Runtime
- vLLM
- Quantization
- Deployment-specific quantization
- Memory
- Server-class GPU memory
- Evidence
- estimated
Laguna S 2.1
Poolside documents running quantized builds on a single NVIDIA DGX Spark; confirm exact memory requirements per quantization before self-hosting.
- Runtime
- vLLM
- Quantization
- FP8, NVFP4, INT4, or GGUF quantization (BF16 also published)
- Memory
- TBD - needs verification (public sources cite conflicting figures)
- Evidence
- estimated
GPU, Apple Silicon, CPU
NVIDIA GPUs usually give the best latency for vLLM or SGLang. Apple Silicon unified memory can be practical for quantized models when the OS and IDE have headroom. CPU or heavy offload is best treated as experimentation unless latency is acceptable.
Chat vs Agent Workloads
Code chat can work with smaller context and slower tokens. Tool-using agents need more context, better instruction following, lower latency, and memory headroom for repeated file reads and command output.
Hosted Fallback
Server-class open-weight artifacts are usually better consumed through Kilo hosted models or your own on-prem endpoint. Local does not mean free; it moves cost to hardware, power, and operations.
Kilo Setup
- 1. Install Ollama, LM Studio, vLLM, or SGLang for the selected artifact.
- 2. Load the exact quantization or precision that matches your hardware tier.
- 3. Connect Kilo to the local or OpenAI-compatible endpoint and switch models per task.
Local Model FAQ
Can active parameter count tell me if a model fits locally?
No. Local fit depends on the exact artifact, precision or quantization, runtime, context length, memory headroom, and workload. This guide labels estimates and avoids deriving fit from active parameters alone.
Is local inference always free?
Local inference avoids hosted token bills, but you still pay hardware, power, setup, maintenance, and latency costs.
Related Model Guides
Best open-weight coding models
Evidence-backed coding ranking and use-case picks.
Best free AI coding models
Live zero-price hosted models and tested free winners.
New open-weight models
Chronological verified release tracker.
Open source vs open weight
Definitions, licensing, and procurement checklist.
State of open-weight AI code review
Production data from 10,643 AI code reviews across 13 models.
Local guidance covers 7 curated artifacts. Last verified 2026-07-30.