Local vs Cloud vs Hybrid AI Assistants: Where Does Your Data Go?
Compare local, cloud and hybrid AI assistants by tracing every component: interface, model, documents, tools, logs and backups. Retention, training, cost and a decision tree.
Contents
- The same task, three architectures
- Trace every place your data can go
- Training, retention and access are three different questions
- What local actually controls
- Compare quality, hardware, cost and maintenance
- Decide what may leave your device
- Verify the configuration you chose
- Choose a starting architecture
- Frequently asked questions
- Does a local user interface mean local inference?
- Does a no-training policy mean zero retention?
- Can a local assistant still send data through tools?
- Can I change architectures later?
Image: Kevin Ache on Unsplash.
Whether an AI assistant is private depends on where each of its parts runs and what each provider keeps, not on whether the chat window is on your laptop. A local interface can sit on top of a cloud model. A local model can sit behind a cloud mail connector. Map every component before you choose.
This guide traces one task through three architectures and gives you a way to decide. It is part of the AI harness engineering guide.
The same task, three architectures
The task: prepare tomorrow's meeting brief from your calendar and project notes.
| Component | Local | Cloud | Hybrid (one common split) |
|---|---|---|---|
| Interface | Your machine | Provider's app or website | Your machine |
| Model inference | Your machine | Provider | Provider for hard tasks, local for sensitive ones |
| Notes and search index | Your machine | Provider's storage | Your machine |
| Calendar connector | Calendar provider's API | Calendar provider's API | Calendar provider's API |
| Logs and receipts | Your machine | Provider | Your machine |
| Backups | Wherever you put them | Provider | Wherever you put them |
Notice the calendar row. Every column sends something to the calendar provider, because that is where the calendar lives. "Fully local" almost never means fully local once real accounts are connected.
Local is a property of a component, not of an assistant.
Trace every place your data can go
For each component, write down three things: what data it handles, where that data goes, and whether you have checked it yourself or are relying on documentation.
The obvious paths:
your prompts,
the documents you attach or index,
the model's responses.
The paths people forget:
embeddings and search indexes built from your documents,
tool and connector calls that carry document text,
telemetry and crash reports,
logs that store full prompts,
backups and sync folders,
a fallback provider that takes over when the main one is down.
Label each row observed (you checked the traffic or the files), documented (the provider says so) or unknown. Unknown rows are your to-do list.
Training, retention and access are three different questions
Providers answer these separately, and so should you.
Training. Is your data used to improve the provider's models? OpenAI's API documentation says API data is not used to train or improve its models unless you explicitly opt in (OpenAI).
Retention. How long is it kept, and why? The same page says abuse-monitoring logs may be retained for up to 30 days, unless longer retention is required by law or needed to protect services or others from harm. Zero Data Retention and modified abuse monitoring are available only with prior approval (OpenAI).
Access. Who can see it, under what conditions? This depends on the provider's terms, your account settings and any connector in between.
No training is not no retention. A consumer app and an API from the same company can have different policies. Read the policy for the exact product and endpoint you use, and date your notes, because policies change.
What local actually controls
Running the model locally does keep model calls on your hardware. Ollama, for example, binds to 127.0.0.1 on port 11434 by default, and documents a local-only mode that disables its cloud models and web search, set with OLLAMA_NO_CLOUD=1 or disable_ollama_cloud in ~/.ollama/server.json (Ollama FAQ).
That setting governs Ollama. It does not govern:
the interface you put in front of it,
a plugin that searches the web,
a connector that reads your mail,
the sync service backing up your notes folder.
Check each one. After changing any of them, check again.
Compare quality, hardware, cost and maintenance
| Factor | Local | Cloud | Hybrid |
|---|---|---|---|
| Data leaving the machine | Least, if every component is local | Most | You choose per task |
| Model capability | Limited by your hardware | Strongest available models | Both, with routing rules |
| Upfront cost | Hardware | Little | Some hardware |
| Running cost | Electricity and your time | Usage fees | Both, usually lower than all-cloud for routine work |
| Maintenance | You patch and update everything | Provider maintains the model | Split |
| Offline use | Yes | No | Partial |
Start from the work, not the hardware. If your tasks need the strongest available model, a fully local setup may not do the job well, however private it is. If your tasks are routine summaries of sensitive notes, a modest local model may be enough. The local hardware guide explains how to test that on your own machine, and the cost worksheet shows how to compare total cost honestly.
Decide what may leave your device
Write a small routing policy. It can be four lines:
| Data class | Example | Allowed destinations |
|---|---|---|
| Public | Published articles, documentation | Any configured model |
| Personal, low risk | Meeting titles, reading notes | Chosen cloud provider, under its API terms |
| Sensitive | Health, finances, legal, family | Local model only |
| Secrets | Passwords, API keys, recovery codes | Never in a model's context |
Then define fallback behaviour. If the local model is down, does a sensitive task wait, or does it quietly move to the cloud? Silent rerouting is the most common way a careful policy fails. Make the harness stop and say so instead.
Verify the configuration you chose
Configuration is intent. Verification is evidence.
- List expected destinations for each component.
- Run a synthetic task containing a unique made-up string, a canary such as
PURPLE-HARBOR-7731. - Check where the canary appears: network logs, provider dashboards, local files, logs, backups.
- Record the result with the date and versions.
- Repeat after changes to models, connectors or settings.
This tests what your setup sends. It cannot verify what a provider does after receiving data. For that you rely on its documentation and contract, so keep those dated and linked.
Choose a starting architecture
- Choose local-first if your notes are sensitive, your tasks are routine and you are willing to maintain the setup.
- Choose cloud if you need the strongest models, your data is low risk under the provider's terms, and you prefer less maintenance.
- Choose hybrid if you have both kinds of work and can enforce a routing policy in the harness.
- Choose a managed product if you cannot commit time to patching, checking and restoring. Judge it with the private AI assistant guide.
You can change later, if your documents, memory and workflows are portable. The provider exit test shows how to prove that. For the bigger picture of the local movement, read the rise of local AI.
Frequently asked questions
Does a local user interface mean local inference?
No. Many local apps call cloud models. Check where the model actually runs.
Does a no-training policy mean zero retention?
No. Training and retention are separate. OpenAI's API, for example, does not train on API data by default but may retain abuse-monitoring logs for up to 30 days.
Can a local assistant still send data through tools?
Yes. Every connector to an online service sends data to that service. That is often the point of the connector.
Can I change architectures later?
Yes, if your data and workflows are in portable formats and you have tested the move.