Local Models Are Catching Up. It Depends What You Ask Them.

A model you rent and a model you keep are different kinds of things. One lives on somebody else’s servers and can be shut off on a Tuesday. The other sits on your hard drive and works in a blackout.

For years the second kind was a novelty. That has changed. But “you can run AI on your own computer now” is a sentence doing a lot of quiet work, because these models have nearly caught up at some jobs and are nowhere close at others. Which one you get depends entirely on what you ask.

What it actually is

A local model is a file. A big one, usually five to twenty-five gigabytes, about the size of a couple of movies. You download it once. After that it runs on your own machine.

No account. No monthly fee. No usage limit. Nothing you type leaves your computer.

There are two easy ways in.

Ollama is the one you drive by typing commands. You tell it which model you want, it downloads, and you are talking to it a few minutes later.

LM Studio is the one with buttons. You browse models in a window like an app store, click download, and chat in a normal-looking chat box.

Same models underneath. Pick based on whether you would rather click or type.

One thing to know going in: Ollama recently changed so that its main command opens an assistant that, by default, is powered by a model in the cloud. The local-first tool now arrives set to not-local. You can change it. But you have to know to.

“Free to download” is not the same as “your computer can run it”

This is the part that catches everyone, and it is worth understanding before you waste an afternoon.

When an AI lab releases a model publicly, that means you are allowed to have it. It says nothing about whether your machine can hold it.

Kimi is the clearest example. Its makers publish it openly and anyone can download it. The download is 595 gigabytes, and to actually run it you need more memory than an extremely expensive gaming desktop has. It is a free grand piano. You still need a room with a reinforced floor.

There is no smaller version of it either. It is that or nothing.

You can use Kimi through a tool on your laptop, and it works well. But that request is flying out to their servers and back. Your computer is being a keyboard.

The models you can genuinely run at home are smaller and less famous. Right now the best of them is a Qwen model from Alibaba, around 17 gigabytes, and it is legitimately good.

Notice what that means, though. Alibaba’s newest and best models are not the ones they let you download. The free ones are a generation or two behind. That is true across the whole industry.

Rough rule if you want to check your own machine: a model needs a little over half a gigabyte of memory for every billion parameters in its name. A “7B” model wants about five gigabytes free. A “14B” wants around eight. If your laptop has 16 gigabytes of memory total and Windows is already using six, you can see how the math goes.

What they are genuinely good at now

For anything you ask in one go, small local models are close enough that most people would not notice the difference.

Writing a first draft. Summarizing something long. Rewriting an email to sound less annoyed. Answering questions. Translating. Pulling the useful bits out of a messy document. Two years ago none of that was true on a home computer.

And then there are the reasons that have nothing to do with how smart it is.

Privacy that actually holds up. If you handle client work under an NDA, medical or legal paperwork, or anything you are not allowed to hand to a third party, a local model means the text never leaves your machine. This is the strongest argument for local, and it is the one that survives every objection.

It cannot be taken away. Cloud models get retired. Companies announce that the version you built your workflow around stops working next quarter, and it does. A file on your drive still works in five years.

No meter running. This one is psychological and I think it is underrated. When every request costs a fraction of a penny, you still feel it, and you try fewer things. When nothing is being counted, you try the stupid idea. Sometimes the stupid idea works.

It works with no internet. On a plane. In a dead zone.

One caution on the privacy point, because this has actually happened to people. There are documented cases where a tool loses track of its local setup, quietly falls back to a cloud service instead, and hands you a perfectly good answer with no sign that your text just left the building. Local is a setting, and settings fail silently. Check what your tools are really doing rather than trusting the label on the box.

Where they fall apart

Ask for one thing, and a small local model does fine. Ask it to carry out a long job with a lot of steps, checking its own work as it goes, and it comes apart.

There is a test that measures exactly this: real multi-step work, the kind where the model has to do something, look at the result, and decide what to do next. The best cloud models finish about eight out of ten of those tasks. The best model that fits on a good home computer finishes about two out of ten. A small model that runs on an ordinary laptop finishes fewer than one in ten.

The reason is not that the model is stupid. It is that small errors multiply.

Say it gets each individual step right 95% of the time, which sounds excellent. Over a job with forty steps, the chance it gets all the way through without derailing is about 13%. Improve it to 99% per step and that same job succeeds two times out of three.

Four percentage points, completely different tool. This is why something can feel impressive in a chat window and be useless the moment you turn it loose on a real task.

Worth knowing when you read AI claims: the company that makes that Qwen model publishes a score for itself roughly twice as high as what independent testers get running the same model. Neither side is lying. The company tested under conditions it chose. The gap between those two numbers is about the gap between a demo and a Tuesday.

“It’s free” is not quite true either

Running a model at home burns electricity. A decent graphics card working steadily costs about six cents an hour to run.

Do the math on what it produces in that hour, and it comes out to roughly fifty cents’ worth of writing. Google and DeepSeek both currently charge less than that for the same amount of output from their cheapest models.

So before you have spent a single dollar on hardware, the cheapest cloud AI is already undercutting your power bill.

Against the pricier cloud models, local does save money per use. But a $600 graphics card takes about fifty days of nonstop, around-the-clock generating to pay for itself. For a normal person that is a payback measured in years.

If you already own a good graphics card for gaming or video editing, that changes completely. The card is bought. You are only paying for electricity, and experimenting freely has real value.

So which one do you need?

Run it on your own machine if the material legally cannot leave your computer, if you need it to work without internet, if you are doing the same small repetitive task over and over, or if you already own the hardware and want to tinker without watching a meter.

Pay for the cloud if the job is long and complicated, if it needs to hold a lot of material in its head at once, or if you need it right the first time.

Or both. Keep the private, repetitive work at home and send the hard thinking out. That is genuinely how I expect most people will end up working. A year ago that split was not practical. Now it is a settings change.

Forward → Upward ↑ Onward ↗︎
Mstimaj

Sources and Further Reading

• Terminal-Bench 2.0 Leaderboard. Independent testing of AI models on real multi-step computer tasks.
• Qwen3.6 model card, Alibaba.
• Kimi K2.7-Code model card, Moonshot AI.
• Ollama and LM Studio.
• Sebastian Raschka, “Using Local Coding Agents”, June 2026.
• Gemini API pricing and DeepSeek API pricing.