Blog · Private AI
What a private AI deployment actually includes
Key Takeaways
- Sizing comes first: a 1–4 GPU on-prem box matched to real user count and workloads, not a spec sheet
- Open-weight models run 100% locally — Ollama / SGLang / vLLM behind self-hosted Open WebUI
- Zero egress is blocked at the firewall, then verified by log audit and documented for your compliance file
- Delivery includes staff training, architecture diagrams, and runbooks — not just the install
- Fixed-scope pricing: $3,000–$5,000 small, $5,500–$10,000 mid, plus a $500–$1,500/mo managed retainer
When a regulated business asks "what does a private AI deployment actually include?", the honest answer is: a lot more than a chatbot on a laptop. It is a sized piece of infrastructure, a security baseline, and an operating relationship — not a one-time install.
A typical engagement starts with sizing the on-premises box to your actual usage: how many users will really sit at the console every day, which workloads matter, and what your network can host. That usually lands at a single-GPU machine for a small team or a 1–4 GPU server for a department. The models themselves are open-weight (Qwen, DeepSeek, Llama-class), running locally through Ollama, SGLang, or vLLM behind a self-hosted Open WebUI — so your staff get a familiar ChatGPT-class experience without a single request leaving the building.
The part clients are usually most surprised by is the verification, not the install. Outbound model calls are blocked at the firewall, and we log-audit that zero egress actually happens — then document it in an evidence pack your compliance file can use. Staff training, architecture diagrams, and runbooks are part of the delivery, because a box nobody knows how to run is just expensive furniture.
Pricing is fixed-scope, not per-token: a small deployment (1 GPU, roughly 2–15 users) runs $3,000–$5,000; a mid-size build (2–4 GPUs, 15–60 users) runs $5,500–$10,000. From there, a managed retainer — $500–$1,500/month depending on models, users, and SLA — covers health monitoring, model upgrades, patching, and 24–48h incident response.
The numbers above are the headline. The full deep-dive — the actual sizing worksheet, the firewall ruleset we verify against, the model-selection matrix, and the retainer's monthly checklist — goes to subscribers in our newsletter. Subscribe below and it lands in your inbox.