Your hardware gets older
while the cloud gets cheaper.

The workload

Every comparison below uses one unit of work: 1M input tokens and 100k output tokens. That ratio is typical of agentic and document-heavy use, where you feed the model far more than it writes back.

Electricity is $0.30/kWh throughout. Machines are powered twelve hours a day. Utilization is 50%, which is optimistic for a team and generous for an individual.

First, about tokens per second

Nearly every throughput figure here is aggregate across concurrent requests, not what you watch in a chat window. This is the single most common misreading of local-inference benchmarks, so it's worth being blunt.

If you've run a model locally, you're used to single-stream numbers: you send a prompt, tokens appear at some readable pace, and that pace feels like what the GPU "does." It isn't. At batch size 1 the card spends most of its time waiting on memory reads and throws away nearly all its compute. Add concurrent requests and those same weight reads get amortized across the whole batch, so total output climbs far faster than per-user speed falls.

So when you read "3,000 tok/s" further down, that is not one conversation outrunning your eyes. It is roughly twenty conversations at ~145 tok/s each, running at once. This distinction decides the entire article — it is exactly why the first scenario fails.

Four DGX Sparks: no break-even exists

$18,400 of hardware, about 800 W, running DeepSeek V4.1 Flash at 50 tok/s. That figure is both the single-stream and the aggregate rate, because there isn't enough spare memory to batch — the machine can only ever serve one request at a time. One job takes over an hour. In a twelve-hour day it clears roughly ten.

Capex never enters the equation. The machine loses money every day before you account for a cent of the $18,400. You could have been given these boxes for free and still be behind.

Eight RTX PRO 6000: it works, barely

Around $83,000, drawing 5.5 kW under load. 768 GB of VRAM holds the ~510 GB checkpoint with room left for a large KV cache, and that headroom is what makes continuous batching possible. Batching is the whole game.

At 4,000 tok/s aggregate decode and 40,000 tok/s prefill, a job takes 50 seconds. At 50% utilization the node clears 432 jobs a day.

Payback: 2.9 years. Tight against a three-year depreciation schedule, but real. Hold that thought.

Two RTX 5090s: three months

Qwen3.8-27B is Apache 2.0, and at NVFP4 it fits on a single 32 GB card with the full 262k context. That fact determines the architecture. The 5090 talks over PCIe with no NVLink, so tensor parallelism across cards performs badly — but because the model fits on one card, you don't need it. Run two independent replicas and throughput scales almost linearly.

Two cards plus a host: about $7,000, drawing ~1.4 kW. Call the pair 3,000 tok/s aggregate decode and 16,000 tok/s prefill. Single-stream on one card, this model runs about 93 tok/s plain, or 130–160 with speculative decoding — the 3,000 figure is twenty of those streams at once.

A job takes 96 seconds, so 225 jobs a day at 50% utilization.

Payback: 3.3 months. Nearly the same daily margin as the $83,000 node, on a twelfth of the capital.

You are not capturing an efficiency advantage here. You are capturing a market that hasn't been squeezed yet. Nobody serves Qwen3.8-27B at DeepSeek's scale, so its price never got competed down toward marginal cost.

The numbers side by side

One caveat the table hides: these models are not interchangeable. Qwen3.8-27B is a capable small model; DeepSeek V4.1 Flash is a different weight class. The faster payback belongs to the model you might not have wanted.

What actually decides it

Throughput per watt, however you get there

Batching for the big node, quantization and replica parallelism for the small one. The DGX Spark scenario fails because it has neither — and because single-stream benchmarks made it look fine before purchase.

Sustained utilization

Below roughly 40%, nothing on this page pays back inside the hardware's life. A box that is busy two hours a day is a box you bought at retail to leave idle.

How competitive the hosted market is for your model

DeepSeek's $0.15/M is not a subsidy. It is what near-perfect utilization costs at scale, and it leaves you almost no margin to capture. Niche models face no such pressure, which is precisely why self-hosting them wins.

The clock nobody prices in

Every payback number above assumes today's API price holds until you break even. It won't.

Qwen3.8-27B's cheapest input rate fell 40% in ninety days, from $0.400 to $0.240 per million tokens. That is not an anomaly, it is the direction the whole market moves. Apply even half that rate of decline to a payback period and the target runs away from you faster than you approach it.

Run that against the two working scenarios and they separate completely:

Hardware ages in two directions at once. The cards lose resale value on a schedule you can predict. But the models move too: three years from now the checkpoint you bought 768 GB of VRAM to hold will be superseded by something with different memory requirements, and there is no guarantee your rack is the right shape for it. You optimized for a target that doesn't stand still either.