Most small businesses now want an AI assistant that knows their products, their policies and their documents — but they do not want to ship that knowledge to a third-party cloud, pay a per-token meter forever, or wait for an IT department they do not have. The good news in 2026: you can run a capable AI on a box that sits in your own office, on your own network, with nothing leaving the building.
This is the starting point for the whole topic. Each section below is deliberately short and links to a deeper post when you want the detail. Read it top to bottom for the full picture, or jump to the question you came with.
What "local AI hosting" means
Local AI hosting means the two things that matter — the model (the language model that generates answers) and the knowledge (your documents, indexed so the model can cite them) — both live on hardware you own, inside your own network. A customer asks a question; it is answered by a machine on your premises; the question and your internal knowledge never travel to someone else's data centre.
That is the whole idea, and it is what separates this from "using ChatGPT for work." There is no external API in the answer path, no per-message cost, and no copy of your knowledge sitting on a platform you do not control. The hardware is small — often a mini PC — and it is yours.
Why host AI locally
Three reasons drive small businesses to host their own AI, and it is worth being precise about each.
- Sovereignty. Your knowledge stays on equipment you control. Nothing is uploaded, nothing is used to train someone else's model, and you are not exposed if a cloud vendor changes its terms or its prices. Sovereignty-as-hardware is the bet this product is built on — it is our conviction, not a settled industry fact, but for many firms it is the deciding one.
- Cost shape. Local hosting trades a recurring per-token cloud bill for a one-time box plus a thin subscription. Whether that wins depends on your usage, and prices in 2026 are volatile — we work the numbers honestly in local AI vs cloud: the real cost.
- Compliance posture. Keeping personal data and processing on-premises strengthens your DSGVO and EU AI Act posture by design — there is no cross-border transfer to justify and no third-party processor to contract. It does not by itself settle compliance, but it removes whole categories of risk before you start.
We lay out the underlying argument — why local, owned, governed AI is a defensible position — in sovereign AI, explained.
What it costs
The cost of local AI hosting has two parts: a one-time hardware purchase and a thin software subscription for the governed core that runs on it. There is no per-token charge, which is the line item that makes cloud AI hard to budget.
As an indication, the hardware ranges from around ~€490 for an entry mini PC to ~€2,400+ for a performance box (both indicative, and 2026 prices move — see current prices on the hardware page). The subscription is deliberately light because the heavy lifting runs on your own machine; the pricing page has the current plans.
The real question is not the sticker price but the crossover against a recurring cloud bill. We work a concrete example, including where local does and does not win, in local AI vs cloud: the real cost.
Which hardware you need
Hardware splits into three tiers by the size of model — and therefore the amount of memory — you need. One number drives almost everything: a 7–14B model needs roughly 8–16 GB, while a 70B model needs about 40 GB, which is why the bigger boxes ship 128 GB of unified memory.
Here is one box from each tier as a snapshot:
| Machine | Runs | Memory | Compute | Price |
|---|---|---|---|---|
| Minisforum AI X1 Pro | 7–14B + long context | up to 96 GB (upgradeable) | Ryzen AI 9 HX 370 · Radeon 890M · 50 TOPS | from €739 |
| Minisforum MS-S1 MAX | 30–70B (clusters higher) | 128 GB unified | Ryzen AI Max+ 395 · Radeon 8060S | from €2,679 |
| NVIDIA DGX Station GB300 | 70B → 1T · 7 isolated tenants | 748 GB coherent | GB300 Grace Blackwell Ultra | On request |
- The entry tier (7–14B, one knowledge base) is where most SMEs start — an affordable mini PC you can run in an afternoon.
- The performance tier (30–70B, several knowledge bases) is a 128 GB unified-memory box for sharper answers or multiple departments.
- The high tier (70B+, many tenants) moves into workstation territory, sold on a quote basis.
For the full shortlist with current prices and disclosed buy links, see the hardware page; for the comparison of every box in the first two tiers read the best mini PCs for local AI in 2026, and to choose between platforms see AMD vs Intel vs Apple vs NVIDIA for local LLMs.
How much hardware you actually need
The honest answer is usually "less than you think." A 14B model on an entry box already answers customer questions from your documents, drafts replies and searches your own knowledge — that covers the majority of SME use cases. You step up to a 70B-class machine when you want noticeably sharper reasoning, several knowledge bases, or higher concurrency, not as a default.
The trap is buying a workstation for a job a mini PC does. We size memory to model and walk through how to right-size for your real workload in what hardware you need to run an LLM locally. When in doubt, start smaller — the knowledge and the governance move with you to a bigger box later.
Deployment shapes: appliance, self-managed, or managed
Owning your AI does not have to mean running it yourself. There are three deployment shapes, and they map to how much you want to operate:
- Appliance — a pre-imaged box arrives ready to run. You plug it in; it is yours; we (or your consultant) keep the software current. This is the turnkey route most SMEs choose.
- Self-managed — you run the open core on your own hardware and operate it yourself, with full control and no managed-service layer.
- Managed — a trusted partner operates the box on your behalf, on-premises, so you get the sovereignty without the ops.
All three keep the model and knowledge on hardware you own — the difference is who turns the wrench. We explain the model behind these shapes in sovereign AI, explained, and you can put the options side by side on the comparison page.
Staying DSGVO- and EU AI Act-ready
Local hosting is a strong foundation for compliance, but it is a posture, not a finish line. Because personal data and inference stay on your premises, you addressed the hardest parts — cross-border transfer and third-party processing — by design. The remaining work is governance: knowing what the agent is allowed to say, keeping a boundary between internal knowledge and public answers, and being able to show how the system behaves.
That boundary is what Cloud Shield enforces — the layer that decides what crosses from your private knowledge into a public-facing answer. It is on-premises by design, which is exactly what an EU compliance posture wants. For a plain-language walk through the obligations that apply to a small business, read the EU AI Act: what SMEs actually need to do. And for the data-protection terms themselves, the glossary entry on DSGVO is a quick reference.
Local hosting strengthens your DSGVO and EU AI Act posture. It does not, on its own, guarantee compliance — that always depends on how you govern the system.
The part people miss: governance is the product
Here is the thing most buying guides skip. A box running a raw model is a confidential database that will read its own secrets aloud to anyone who asks the right question. The hardware is only half the product. What makes a local AI safe to put in front of customers is the governance layer — the boundary between your internal knowledge and the public agent, plus the loop that improves it over time.
That governance layer, not the silicon, is what you are actually buying. We make the full case in the product is governance, and you can see how the governed core works on the platform page. Pick the right box, then put a governed core on it — never a bare model.
How to get it installed without an IT team
You do not need an in-house IT department to run local AI. The common route is a trusted consultant who supplies and installs a pre-imaged appliance — the box arrives configured, the consultant connects it to your knowledge, and you are answering questions the same day. The consultant stays in the loop to tune the agent and keep it current.
If you already work with an IT partner, business consultant or agency, they can become your supplier — see for consultants for how that works. If you would rather buy the turnkey route directly, the hardware page shows the appliance options with installation handled.
When you are ready to see a governed local AI answering questions on your own documents, book a demo — that is the fastest way to judge whether this fits your business.