Private AI chat for a company: what it is, when it pays, and the decisions that matter

This guide comes from building a private AI workspace on Open WebUI for a mid-sized company’s staff, written up as a case study. The client is not named, and no figures from that project are published here.

Decide whether a private AI workspace is worth it

People in most companies already use public AI assistants, often on personal accounts, with company documents pasted in. A private workspace gives them the same convenience under the company’s rules: work login, chosen models, known costs, and conversations on infrastructure you control.

It is not the right answer for everyone.

Situation	Private workspace?	Why
Many people need AI help every day	Worth considering	One place, one login, shared assistants and knowledge
Sensitive documents end up in personal AI accounts	Worth considering	It replaces the workaround with something sanctioned
Spend must be split by department and capped	Worth considering	A gateway gives budgets and reports per team
A handful of people, occasional use	Probably not	A business plan of a public assistant is simpler
Nobody can own it after launch	Not yet	It needs an owner for upgrades, users and budgets

Choose where it runs and which models answer

Two choices are easy to mix up: where the workspace runs, and where the models run. They are separate decisions with separate trade-offs.

Choice	Option	What you gain	What it costs
Where the workspace runs	Your own server or datacentre	Conversations, files and users stay with you	Your IT runs and upgrades it
	A hosted environment	Faster to start; good for a pilot	Another party holds the data
Where the models run	Cloud models, with data collection switched off	The best models, no hardware, pay per use	Prompts leave your building
	Your own models on your own GPU	Prompts never leave the building	Expensive hardware, weaker models, more upkeep

On that project the client chose to start with cloud models and keep local models as a later step, prepared in the configuration but switched off. That got people using the product quickly. What it cost was privacy, not money, and we wrote that down in exactly those words, so nobody later believed prompts stayed on the premises.

Read the diagram as text

1. Your team: in a browser or on a phone, signed in with their work account
2. Chat workspace: Open WebUI: chat, custom assistants, knowledge bases, projects
3. Model gateway: holds the provider keys, enforces budgets, routes to the right model
4. Models: cloud models with data collection off, or your own models later
5. Underneath: Company directory: who may sign in, and what each group may do; leaving the company removes access
6. Underneath: Company knowledge: knowledge bases, and mail and documents read as the signed-in person
7. Underneath: Budgets and reports: a budget per team and per person, and spend reported by team

Put a gateway between the chat and the models

The single most useful design choice we made was a model gateway (we used LiteLLM) between the chat and the providers. Checklist for what it should do:

• Hold the only provider key. The chat gets a limited key of its own, so a problem in the chat does not expose the billing account.
• Give every team and person a budget, and refuse requests once it is spent, rather than only sending an alert.
• Report spend by team, so the AI bill becomes something a manager can read and approve.
• Route by need. Everyday questions go to a capable, cheaper model; a stronger model is available to the people and tasks that need it.
• Fall back on errors. If a model fails or times out, the next one answers. A fallback catches failures, not weak answers; deciding when an answer is not good enough is a separate step.

Connect it to your directory, and separate the groups

Accounts cannot be created and removed by hand for long. Sign people in through the company directory, so joining and leaving are handled where they already are, and map directory groups to permissions in the workspace.

Group	Typically gets	Typically does not get
Everyone	Chat, shared assistants, team knowledge bases	Admin settings, developer tools
Developers	Stronger models, coding tools, their own gateway keys	Admin settings
Administrators	Users, models, budgets, usage reports	Nothing extra by default

Two lessons from doing this:

• Test with a real login before launch. A group name that matches nothing produces a user with no permissions and no error message. Only a real person signing in proves the mapping works.
• Treat powerful features as decisions, not defaults. Running code from the chat, a shell for developers, connectors to mail: each one should be switched on or off deliberately and written down, not inherited from whatever the software ships with.

Bring company knowledge in carefully

A private workspace becomes useful when it knows your company. Most of what is needed already exists in the product and only has to be switched on, configured and explained.

• Knowledge bases for policies, product documentation and procedures, with search that works in the language your documents are written in.
• Projects, so today’s work, its instructions and its files are still there tomorrow, and can be shared with colleagues.
• Memory, so the assistant learns how a person likes to work, with the person able to see and change what it remembers.
• Mail and documents read as the signed-in person, so the assistant only ever sees what that person could already open. We connected Microsoft 365 mail, OneDrive and SharePoint this way.
• Custom assistants built by the people who know the work, shared with their team.

Plan the rollout like an integration, not an install

A pilot in a hosted environment proves the product works. Moving it into the company’s own network is a second integration, not a redeploy: the network, the certificates, the directory groups and the firewall rules are all new.

Checklist for a rollout the client’s IT can run without us in the room:

• A written list of prerequisites to send to IT before the day: server, addresses, certificates, directory access, firewall openings.
• One install document, in order, with the reason for each step.
• Deployment through an audited pipeline into the company’s network, rather than someone improvising over a remote shell.
• An address that works on phones outside the office, chosen before launch.
• A short list, kept up to date, of what has and has not been verified on the real system.

Teach people what it can do

The surprise on this project was how much was already there. Memory, projects, sharing and scheduled tasks existed in the product; several were simply switched off or never explained. A feature nobody knows about is indistinguishable from a feature that does not exist.

So we put the guide inside the product: a help page in the workspace itself, an assistant that answers questions about how to use it, and a simple way to report a problem. Adoption follows from people discovering, on their own, that the tool does what they need.

If you want a private AI workspace for your team, with your data and your models under your control, book a call. We will help you decide whether it is worth it, and how to set it up so your team can run it.