Both are private AI — the question is which price you want to pay: self-hosting costs hardware, time and responsibility; a specialist provider costs trust in their architecture plus ongoing fees. The short decision rule: if you need a system completely disconnected from the internet, or you already own GPU infrastructure and an ops team, self-host. If you want GDPR-compliant AI without an infrastructure project of your own — a contract rather than a home-built solution — you are better off with a zero-retention provider from the EU. This guide gives you the honest maths for both routes.
What private AI really means — two architectures, one goal
When people say private AI they mean two very different things: local (a model runs on your own hardware — Ollama, GPT4All, a Llama or Mistral model on your own server) or contractually private (a provider processes your inputs but does not store them, does not train on them and guarantees that contractually). The goal is identical: your data should not become raw material for someone else's systems. The route there — and what it costs — differs fundamentally.
When self-hosting is the right answer
Honestly: there are scenarios in which we would recommend self-hosting to you.
- Air-gap requirements. If your system has to be completely disconnected from the internet (classified material, critical infrastructure, extreme compliance), there is no way around local hardware. No provider — including us — can offer "offline".
- Existing GPU infrastructure + ops team. If you already have data-centre capacity and people who can deploy, patch and monitor models, you would be paying a provider's premium for convenience you don't need.
- Extreme volume with constant load. Above a permanently high utilisation, your own hardware can become cheaper per token than any API — if you count the staff costs honestly (most people don't).
- Learning and experimenting. For developers and tinkerers, a local model is the best playground. Ollama on a MacBook costs nothing and teaches a lot.
The honest maths of self-hosting
What forum recommendations ("just use Ollama!") routinely leave out:
- Hardware: production-ready setups for usable models range from a few thousand euros (entry level) to five-figure sums (multiple users, larger models) — plus electricity, plus replacement in 2–3 years.
- Model quality: open models have become strong, but on your own consumer hardware it is usually the smaller variants that run — noticeably below the level of current frontier models. The quality gap is real and matters day to day.
- Operations: updates, security patches, availability, monitoring, backup strategy, user management. This is an IT project with ongoing operations, not a weekend setup.
- Compliance is NOT automatically handled: local AI also processes personal data — GDPR obligations (access control, deletion concepts, TOM documentation) remain; they simply have a different addressee: you. "It runs in our basement" is not a data protection concept, it is the start of one.
When the specialist provider wins
- You need contracts, not an architecture debate. Clients, data protection officers and professional chambers ask about the DPA, TOMs and responsibilities — not about your Docker config. A provider with a public DPA and documented measures answers those questions in minutes.
- Professionals bound by secrecy: for law firms, tax advisors and medical practices, self-hosting means there is no service provider who could be bound under § 203 StGB — instead you carry the entire technical responsibility personally. A provider with a confidentiality undertaking reverses that relationship: responsibility is shared contractually and documented.
- Frontier model quality without frontier hardware. Current large models on professional inference infrastructure, paid by usage rather than by purchase.
- Zero retention as architecture: at PrivatAI, inputs and outputs are processed and discarded — no content logs, no training, core processing in Germany and France. The practical effect is the same as with a local model — no body of data arises — only without the server cabinet.
- Ready to go immediately: chat for the team today, OpenAI-compatible API for your own integrations tomorrow. No procurement project.
The decision matrix
| Criterion | Self-hosting | Managed (e.g. PrivatAI) |
|---|---|---|
| Upfront cost | Hardware: thousands to five figures | None |
| Ongoing effort | Your own IT project (operations, patches, monitoring) | None (provider's duty, contractually) |
| Model quality | Usually smaller open models | Current large models |
| Data location | Your premises | EU (DE/FR), fixed contractually |
| Body of data | None (if configured correctly) | None — processed and discarded |
| DPA / § 203 chain | Not applicable / your own responsibility | DPA + Annex 4 |
| Air-gap possible | ✅ | ❌ |
| Time to start | Weeks | Minutes |
Frequently asked questions
Is local AI automatically GDPR-compliant?
No. Processing personal data still has to be governed — access controls, deletion concepts and documentation simply move entirely into your own responsibility.
Is hosted AI automatically insecure?
No — what matters is the provider's architecture: is anything stored? is anything trained on? where does processing happen? are there verifiable contracts? Those are exactly the four questions you should put to every provider (including us — the answers are public in the DPA).
Can we run a hybrid setup?
Yes, and it is a proven pattern: uncritical experiments locally, productive work with client data through the contractually secured provider. The OpenAI-compatible API makes switching between the two trivial.
What about Ollama and friends for personal use?
Excellent for individuals and for learning — recommended without reservation. The boundary is where client, patient or customer data and multiple users come into play: at that point the tool becomes an operations and compliance project.
