Local AI

Local or cloud? The answer is usually both.

This is framed as a choice more often than it needs to be. The businesses handling it well do not pick a side — they decide, workflow by workflow, which side each one belongs on. What follows is the comparison that decision rests on.

The comparison

	Local	Cloud
Quality on hard tasks	Good, below frontier	Best available
Cost per request	Zero after hardware	Per token, accumulates
Upfront cost	Hardware	None
Data exposure	None — never leaves	Sent to the provider
Works offline	Yes	No
Latency	Fast, no network round trip	Network dependent
Maintenance	Yours	Theirs
Improves over time	When you update it	Continuously, automatically
Availability risk	Your hardware	Provider outages, policy or pricing changes

The case for cloud

The best models are there. For genuinely hard reasoning, the frontier models are better and pretending otherwise helps nobody.
Nothing to maintain. No hardware, no updates, no capacity planning.
It improves without you. Capabilities arrive continuously.
Elastic. A thousand requests in an hour costs a thousand requests; locally it costs a queue.

The case for local

Data never leaves. For client files, health records, legal documents and financial data, this is not a preference — it is often the requirement.
Cost stops scaling. After hardware, volume is free. High-throughput repetitive work is where this becomes decisive.
No dependency. No outages, no deprecated models, no pricing changes, no terms-of-service updates that alter what you are permitted to do.
It works offline — which matters more in places with unreliable connectivity than most vendors, writing from cities with fibre, appreciate.

How to split it

Sensitive data stays local. Client documents, personal records, anything you would need to disclose a breach of.

High-volume repetitive work goes local. Classification, extraction, routing — the tasks where per-call costs quietly accumulate.

Hard reasoning goes to the cloud. Complex analysis, difficult code, work where the quality gap changes the result.

Anything customer-facing and latency-sensitive goes wherever it responds faster, which is often local.

Write the boundary down. A rule your team understands beats a habit nobody can articulate — and it is the thing an auditor or a client will ask about.

The direction of travel is worth noting when you plan. Open-weight models have been closing the gap with frontier models steadily, and consumer hardware capable of running them keeps getting cheaper. A workflow that needs the cloud today may not in a year — which argues for building so you can move it, rather than hard-wiring one provider into everything.

Common questions

Is local AI private by definition?

Local means the data does not leave your infrastructure, which removes the third-party exposure entirely. It does not automatically make you compliant or secure: access control, storage, backups and who inside your business can query the system are all still your responsibility. Local removes one risk and hands you the others.

Will local models catch up with cloud models?

The gap has narrowed considerably and open-weight releases have repeatedly reached capability levels that were frontier a year or two earlier. Whether that continues is not knowable, so the sensible position is to build systems where the model can be swapped rather than betting on either side.

What is the biggest hidden cost of running AI locally?

Maintenance and the expertise to do it. Cloud providers absorb updates, uptime and scaling. Locally that is yours — and for a business with no technical staff, that ongoing burden is a larger factor than hardware price. It is the reason some businesses that could run locally sensibly choose not to.

Can I switch later?

Yes, if you design for it. Keep the model behind an interface your software calls rather than embedding one provider throughout. Tools that support both — like our own outreach system, which runs on either a cloud provider or a local model — make the decision reversible, which is worth more than getting it right first time.

Keep reading

Running AI locally AI and business data privacy Hardware requirements