Fully self-hosted AI coding works today, for a narrower set of jobs than the enthusiast threads suggest. Autocomplete, boilerplate, docstrings, test scaffolds, small refactors within a file: a Qwen2.5-Coder-class model running on your own GPU handles all of it, and you will stop noticing that it is local. Multi-file agentic work — "read this codebase, find the bug, fix it across four modules" — is where the gap with frontier models stops being academic and starts costing you an afternoon.
The realistic stack is Continue plus Ollama for in-editor assistance, Tabby if you want a team-wide self-hosted server, and Cline or Aider pointed at a local endpoint if you insist on agentic work locally. The realistic hardware floor is more VRAM than most people have.
That is the whole answer. The rest of this explains where each boundary sits, so you do not spend a weekend discovering it yourself.
What "self-hosted" has to mean
Vendors have made this word soft. Three different things get sold under it, and only one of them is what you are asking for:
- Local weights, local inference. The model runs on hardware you control. No token leaves your network. This is the real thing.
- Self-hosted client, hosted model. You run the IDE extension or the gateway; the completions still come from someone's API. This is most "private AI coding" marketing.
- Zero-retention cloud. A contractual promise that your code is not stored or trained on. Not self-hosting, but — and this matters — it is what most people who think they need self-hosting should actually buy.
If your requirement is legal or contractual rather than a preference, keep reading. If it is a preference, skip to the last section first, because the honest recommendation may not be the one you came for.
The quality gap, stated plainly
Local models have improved enormously and remain meaningfully behind frontier models at hard reasoning over large contexts. Both are true, and pretending otherwise is how people waste weekends.
Where local is genuinely fine:
- Fill-in-the-middle autocomplete. A 7B coding model is fast, private, and good enough that you will forget it is local. This is the highest-value, lowest-difficulty win.
- Boilerplate. CRUD handlers, config, DTOs, test scaffolds, regex you would otherwise google.
- Explaining and documenting code you already have, and single-file refactors with clear instructions.
Where it falls down:
- Agentic loops. Local models are less reliable at tool-calling — they produce malformed calls, loop, or give up. Agentic coding is a chain of tool calls, and small errors compound. This is the single biggest practical gap.
- Long context. You can technically feed a local model 100k tokens. Prompt processing on consumer hardware will make you regret it, and quality degrades toward the middle of a long context faster than it does for frontier models.
- Genuinely hard problems. The subtle concurrency bug, the design decision with three trade-offs. This is what you are paying Cursor or GitHub Copilot for, and it is not what you are getting locally.
Hardware: the part that decides everything
Model size is a VRAM question. A useful rule of thumb for 4-bit quantised weights is roughly half a gigabyte of VRAM per billion parameters, plus headroom for the KV cache, which grows with context length.
Which puts you here, approximately:
- 8 GB VRAM: 7B models, quantised. Fine for autocomplete. Do not expect agentic work.
- 16 GB: 7B comfortably with real context, or a 14B tightly. This is the sensible entry point.
- 24 GB (a 3090 or 4090): 32B quantised. The first tier where a local model feels like a real assistant.
- 48 GB+ or a 64 GB unified-memory Mac: larger models at usable speed. Apple Silicon is quietly the most cost-effective way to get a lot of usable memory, though prompt processing is slower than a discrete GPU.
No configuration on a normal desk runs a frontier-scale model. That is the constraint, and no amount of quantisation argues with it.
The stack
Ollama — the runtime everyone starts with
Ollama makes local models a one-line pull and exposes an OpenAI-compatible endpoint, which is why every tool below can point at it. For a team server, vLLM is the more serious choice — better batching and throughput. Start with Ollama on your own machine.
Continue — the editor extension
Open-source, VS Code and JetBrains, configured to talk to whatever endpoint you tell it. Autocomplete, chat, and edit-in-place, all pointed at your own runtime. Who should use it: anyone starting out. It is the shortest path from zero to a working local assistant. Who should not: nobody, really — it costs nothing to try. Trade-off: its agentic capabilities lag Cursor's by a wide margin, and the local model is what makes it lag further.
Tabby — the self-hosted server
Tabby is a self-hosted coding assistant designed to be deployed for a team: one GPU server, everyone's editor connects to it, no per-seat cloud spend and no code leaving the network. Who should use it: an engineering org with a compliance requirement and a GPU budget. Who should not: a solo developer, for whom it is infrastructure you have to maintain. Trade-off: the most enterprise-shaped answer here, and it inherits the local quality ceiling like everything else.
Cline and Aider — agentic, locally
Cline (in VS Code) and Aider (in the terminal) both accept a local OpenAI-compatible endpoint. Both are excellent with a frontier model. Both are noticeably fragile with a small local one, for the tool-calling reasons above. Who should use it: developers with 24 GB+ who want to run agentic edits privately and can tolerate babysitting. Who should not: anyone who expects a local Devin. Trade-off: this is where the gap hurts most, so set expectations before you install.
The models
- Qwen2.5-Coder is the default recommendation across sizes, with fill-in-the-middle support that makes autocomplete work properly. Most sizes ship under Apache 2.0 — check the individual model card, because the licence is not uniform across the family.
- Devstral (Mistral) is Apache 2.0 and specifically trained for agentic coding, which makes it the more interesting choice if you want local Cline to actually behave.
- Codestral (Mistral) is strong, and its licence is non-production. Read that again before you deploy it at work. This trips people up constantly.
- DeepSeek-Coder models are excellent and the good ones are large. The flagship sizes are not self-hostable on a workstation, whatever the benchmark charts imply.
- StarCoder2 remains a reasonable permissive option, and is showing its age.
The licence is part of the model. "Open weights" does not mean "commercially usable", and a non-commercial licence on a company inference server is a real problem, not a technicality.
Local vs. cloud, honestly
| Setup | Good at | Weak at | Hardware floor | Licence catch |
|---|---|---|---|---|
| Continue + Ollama + Qwen2.5-Coder | Autocomplete, boilerplate, single-file edits | Agentic multi-file work | 8–16 GB VRAM | Check the model card per size |
| Tabby (team server) | Org-wide private completion | Deep reasoning; you maintain it | A dedicated GPU box | Verify before internal deployment |
| Cline / Aider + local model | Private agentic edits, with supervision | Tool-call reliability | 24 GB+ | Devstral (Apache 2.0) is the safe pick |
| Codestral, locally | Quality | Non-production licence | 16 GB+ | Not for commercial use |
| Cursor / GitHub Copilot | Everything, well | Nothing leaves your network — because it does | None | Read the retention terms |
Swipe the table sideways to see every column →
A pragmatic setup that takes an evening
- Install Ollama. Pull a Qwen2.5-Coder at whatever size your VRAM allows. Bigger is better right up to the point where it stops fitting.
- Install Continue in VS Code or your JetBrains IDE. Point it at Ollama.
- Use the small model for autocomplete and the largest one you can fit for chat. Different jobs, different latency budgets. This is the single most impactful configuration choice and almost nobody makes it.
- Measure it on your actual work for a week. Not on a toy prompt. Whether local is "good enough" depends entirely on what you build.
- Keep a cloud model for the hard problems unless your rules forbid it. A hybrid — local for the constant small stuff, frontier for the genuinely difficult ones — is the setup most people who try this land on, and there is no shame in it.
When you should not self-host
If the reason is cost, do the arithmetic. A capable GPU plus electricity plus the hours you will spend maintaining it exceeds a year of GitHub Copilot by a comfortable margin. Self-hosting is not the cheap option; it is the private one.
If the reason is vague discomfort with sending code to a vendor, look at zero-retention enterprise agreements first. The major coding assistants offer them, they are contractually meaningful, and they get you frontier-model quality without a GPU under your desk. Our comparison of Claude Code, Cursor and Copilot covers what you would be giving up.
Self-host when you must: air-gapped environments, defence and regulated industries, client contracts that forbid third-party processing, or genuinely sensitive proprietary code where the answer to "can this leave the network" is no. In those cases the quality gap is not a trade-off you are choosing. It is the price of the constraint, and the stack above is how you pay the least for it.
FAQ
What is the best self-hosted AI coding assistant?
Continue paired with Ollama is the best starting point for an individual — open source, editor-native, and it will run whatever model your hardware supports. For a team that needs one private server everyone connects to, Tabby is the purpose-built answer. Neither will match a frontier cloud model on hard problems; both are entirely adequate for autocomplete and boilerplate.
Can a local AI coding model replace Copilot?
For autocomplete, effectively yes — a 7B coding model with fill-in-the-middle support is fast and close enough that most developers stop noticing. For agentic, multi-file, whole-repo reasoning, no. That is where the gap is widest and where it is most visible in daily use.
How much VRAM do I need for local AI coding?
As a rough guide, 8 GB gets you a quantised 7B model, which is enough for competent autocomplete. 16 GB is the sensible entry point. 24 GB opens up 32B models, which is the first tier where a local assistant feels like a real one. Apple Silicon Macs with 32–64 GB of unified memory are a cost-effective route to the larger sizes.
Is self-hosted AI coding actually cheaper?
Almost never. A GPU capable of running a useful model costs more than several years of a coding-assistant subscription, before electricity or your time. Self-hosting buys privacy and control, not savings. If someone tells you they are doing it to save money, they have not counted the hardware.
Which local coding models can I use commercially?
Check the licence per model, not per family. Most Qwen2.5-Coder sizes are Apache 2.0, and Devstral is Apache 2.0, so both are commercially usable. Codestral ships under a non-production licence and is not. "Open weights" and "open source" are not the same claim, and the difference is enforceable.
Do these tools work offline?
Yes, once the weights are downloaded — that is the point. A local model with a local editor extension needs no network at all, which also makes it one of the more useful entries in our list of AI tools that work offline.
Next steps
Compare what you would be trading away in the AI coding tools and AI developer tools categories, or start from the developers use-case page for the wider stack. If you are self-hosting because code review is the sensitive part, our AI code review tools piece covers which of those can run privately — and if you are running a self-hosted setup we should know about, submit it.