Running AI on your own data without sending it everywhere
Using AI on company data does not mean uploading everything to a chatbot. The architecture that keeps data in your control, layer by layer.
The short answer
Using AI on company data is often imagined as uploading everything to a chatbot and hoping the terms of service are kind. It does not have to work that way, and it should not. The design that keeps data in your control leaves documents where they live, builds an index in your own account, retrieves only the passages relevant to a question, and sends those passages with the question to a model under a business agreement that forbids training on them and limits retention. The application and its logs stay yours. What crosses to a provider is minimal, per call and contractually bounded. It is more private by construction, and it costs the index plus one model call per question instead of the whole document set on every conversation.
The four layers
| Layer | Options | What we recommend by default |
|---|---|---|
| Documents | Where they already live: drive, CMS, systems of record | Leave them; connect with read access that respects permissions |
| Index | Your own database or search service, in your cloud account and region | Yours, in your region, permission-aware |
| Model | Provider API on business terms; regional endpoint; self-hosted for rare cases | Business-tier API with no-training terms, regional where required |
| Application and logs | Your code, your accounts, your logging | Yours, with logs that omit document contents |
What leaves the building, precisely
- A person asks a question. It goes to your application.
- Your index is searched, in your account, for passages the person may see. Nothing has left yet.
- The question and the retrieved passages are sent to the model provider for one call, under the business agreement.
- The answer returns and is shown with citations. Your logs record the event, not the passages.
- Provider retention is whatever the agreement states: often zero for business tiers, or days for abuse monitoring. Verify, do not assume.
Self-hosting: when it is worth it
Running a model on your own infrastructure removes the provider from the picture at the cost of hardware, expertise and ongoing operations, and usually at lower quality than the best hosted models. It is justified for data that may not leave your environment under any contract, at volumes that make the hardware pay, with a team to run it. That describes few small businesses. The architecture above, with a business-tier provider and regional endpoints, meets most obligations at a fraction of the cost.
What this means for you
You can use AI on your company’s knowledge without handing that knowledge to anyone. Keep documents and the index in your accounts, send only the relevant passages per question to a provider on business terms, and keep the application and its logs yours. Provide that to staff so the unsanctioned pasting stops. Self-host only when a real obligation demands it. The private design is also the well-engineered one, and it costs less.
Frequently asked questions
Do we need to run the model ourselves to keep data private?
Rarely. Self-hosting a model is expensive and demanding, and it solves a problem a small business does not have. What matters is that only the minimal relevant text reaches the provider, under a business agreement that excludes training and limits retention, in a suitable region, with your own index and logs under your control. That design keeps the documents in your accounts, and you pay for the index plus a model call per question instead of for uploading everything.
What actually leaves our systems when someone asks a question?
The question and the few passages retrieved to answer it, sent to the model provider for that one call, plus whatever your logging captures. Not the whole document set, not the index, not the history, unless you design it that way. Being able to state this precisely is the point of the architecture.
Can staff still use general AI tools for everyday tasks?
Yes, under a policy: approved tools on business plans, no customer personal data or confidential documents, and a company-provided assistant for the tasks that need company data. People use unsanctioned tools when no sanctioned option exists; providing one is the effective control.