AI on your own machine, explained like you're new here

Most guides about running AI yourself start with hardware specifications. This one starts with the only three ideas you actually need. By the end you will understand what people mean by "self-hosting", "running a model locally", and "chatting with your documents" — and you will know whether any of it is for you.

Idea one: the model is a file, and a program that runs it

When people say "an AI model", the practical answer is: a very large file, plus a program that reads it. The file holds patterns learned from enormous amounts of text. The program — on my machines it is Ollama — loads that file into memory and answers questions using the patterns inside it.

That is the entire trick. No magic, no connection required while it runs. The model never learns from your conversations unless you deliberately feed it back into training, which these local programs do not do.

Idea two: "locally" is a claim about whose computer

The same question can travel two very different roads. Either it goes to a program on a machine in your room, or it crosses the internet to a machine owned by a company. Both roads lead to an answer. The difference is who holds the question on the way.

The same question can take two doors: to a model in your room, or to one in a datacentre. The trade is privacy versus capability

The trade is simple to state and easy to feel. On your own machine: nothing leaves the room, but the model is smaller and slower on hard tasks. In the cloud: the strongest models available, but your words cross a boundary you do not control. I use both, and this essay rounds up the arguments for each side.

Idea three: a container is a sealed box for one program

A server that runs many programs the old way is a drawer where every cable tangles with every other cable. Upgrade one thing, break another. A container puts each program in its own sealed box: the program, its files, its settings, all in one unit. The boxes share the machine underneath, but they cannot reach into each other.

One server, many sealed boxes: every service is a container with its own files, all sharing the machine underneath

This website is a container. So is the database behind its newsletter, the automation that runs my print shop's order intake, and the local models. About thirty boxes on one machine. When one misbehaves, the others do not notice — and I can replace any box without touching the rest. That is the whole argument of owning your own stack.

One idea people oversell: "chat with your documents"

You will see tools promising to answer questions about your PDFs. The honest mechanics are unglamorous: the tool cuts your pages into small passages, turns them into searchable coordinates, and — when you ask something — hands the model only the few passages closest to your question. The model answers from those, not from having read your whole folder.

This is why the choice of tool matters: some tools cut pages badly (complex tables fall apart), and the model can only answer from what it is handed. The head-to-heads guide compares the three main options, including where each one hurts.

So is this for you?

Three honest signals that you would enjoy this:

Something you work with is private enough that sending it to a cloud service makes you hesitate — client documents, financials, personal notes.
You have hardware that mostly idles — a decent laptop, a desktop with a graphics card.
You have been burned once by a service changing its price, closing an API, or losing your data — and the phrase "I'd rather own it" means something to you.

And two honest signals that you should wait:

You want the strongest possible answers on hard problems, today, with zero setup. Use the cloud; that is the right tool for that job right now.
Nobody involved can tolerate an evening of setup. Local AI has never been click-and-run. It is closer to an afternoon of tinkering that pays you back slowly.

Where to go next, in order

The essays stand alone, but they were written as a path:

1. Why I chose to own my own stack — the why, before any how. 2. Run AI on your own box — the practical map of what runs where. 3. Picking self-hosted tools without regret — the method for choosing between good options. 4. Building teams of software agents — what happens when the programs start doing work on a schedule.

If you want the numbers first: six models measured on my own hardware, including the ones that get simple arithmetic wrong.