Local AI

Ollama for business. The shortest path to local AI.

Ollama is the reason running a language model on your own hardware stopped being a systems engineering project. It downloads models, runs them, and exposes a local API that behaves much like the cloud ones — which means most software that talks to an AI provider can be pointed at it with a configuration change.

What it actually does

Downloads and manages models with a single command, handling the formats and quantisation for you.
Runs them efficiently on Mac, Windows and Linux, using GPU acceleration where available.
Exposes a local HTTP API on your machine, so your own tools can call it like any other AI service.
Keeps everything local. No account, no telemetry of your prompts, no data leaving the machine.

Getting started

# install from ollama.com, then:
ollama pull llama3.1
ollama run llama3.1

# it now also serves an API at localhost:11434

That is genuinely the whole setup for a single machine. The interesting work starts afterwards, in what you point at it.

Choosing a model

Size	Memory needed	Suited to
3B – 8B	8 – 16GB	Summarising, classifying, drafting, fast responses
13B – 14B	16 – 32GB	General work with noticeably better quality
30B – 70B	32 – 64GB+	Closest to cloud quality; slower without a strong GPU

Start smaller than you think you need. A fast small model you actually use beats a large one that takes thirty seconds per response and quietly stops being used. Hardware detail here.

Using it in real work

Point existing tools at it. Anything that lets you set a custom API endpoint — including our own Cold Outreach Machine — can use a local model instead of a cloud provider.

Build a document assistant. Combine Ollama with a local search index over your own files so the model can answer from your material without any of it leaving.

Automate a repetitive task. Classification, extraction, first-draft writing. High volume is where local economics win decisively.

Serve a small team. Run it on one machine on your network and let colleagues use it. Add authentication before you do — the default configuration assumes a single trusted user.

A security point people miss: Ollama's API has no authentication by default. On a personal machine that is fine. Exposed to a network — and certainly to the internet — it is an open endpoint anyone can query. Put it behind a reverse proxy with authentication, or keep it bound to localhost.

Honest limits

Quality below the frontier. Real, and it matters most on complex reasoning.
Speed depends entirely on hardware. A large model without GPU acceleration is slow enough to change how you use it.
No built-in multi-user management. It is a runtime, not a platform. Team access is something you build around it.
You own the maintenance. Updates, model changes and uptime are yours.

Common questions

Is Ollama free for commercial use?

Ollama itself is open source and free. The models it runs have their own licences, and those vary — most open-weight models permit commercial use, some restrict it above certain scale thresholds. Check the licence of the specific model you intend to rely on rather than assuming.

Can I use Ollama with my existing software?

Usually yes. Its API is close enough to common cloud APIs that most tools accepting a custom endpoint will work with a configuration change. That compatibility is the main reason it became the default way to run models locally.

How many people can share one Ollama instance?

Technically several, practically it depends on hardware — requests queue, so a handful of concurrent users on a well-specified machine is comfortable and a whole department is not. For team-scale use you want a dedicated GPU machine and a queueing layer in front.

Which model should a business start with?

An 8B-class model on whatever machine you already own. It runs comfortably in 16GB, responds fast enough to be usable, and is good enough to tell you within a week which of your workflows local AI can handle. Scale up only once you know what you actually need.

Keep reading

Running AI locally Hardware requirements Self-hosted cold email