Local LLM Cost per Right Answer Jumps Up to 64x on Hard Questions

Image
Local LLM Cost per Right Answer Jumps Up to 64x on Hard Questions

Southern California – October 14, 2026 -- The cheapest local language model for easy questions costs 37 times more per correct answer on hard questions, according to power-metered tests of 27 models on a Strix Halo system. The model, qwen3-30b-a3b-instruct, delivers right answers at 3.6 cents per thousand on easy work. On hard questions the price rises to $1.35 per thousand. Two other models in the 11-model comparison only doubled in cost over the same jump.

Difficulty tax ranges from 2x to 64x across 11 models

The tester ran 11 of the 27 models on both an easy suite and hard questions, and calls the cost ratio between the two the "difficulty tax." It runs from 2x to 64x. The easy suite has 190 checkable tasks across seven categories, including a 20-step stack-and-total exercise. The hard set is the first 100 of 198 GPQA Diamond questions, which are expert-written physics, chemistry and biology problems.

All models ran at Q4_K_M quantization with four concurrent requests unless noted. Energy was integrated from 1Hz wall-power telemetry. Costs use on-peak Southern California rates of 58 cents per kilowatt-hour; the off-peak rate is 34 cents.

Seven models clear 97% on easy tasks, but costs differ 36x

Seven of the 27 models score at least 97% on the easy suite, making them interchangeable on accuracy. Running devstral-2-123b, a 125B dense model, on a 20-step running-total task costs 36 times what the cheapest correct model charges for the same result.

Qwen3-coder-next and qwen3.6-35b-a3b tie for top hard-task score at 79%

Qwen3.6-35b-a3b and qwen3-coder-next each score 79% on GPQA, with ornith-35b at 78%. Qwen3-coder-next costs 4.8 cents per thousand on easy work, second-cheapest overall. Its hard-task cost is inflated because it ran at half the concurrency of other models; the 46GB model is treated as an upper bound. Nemotron-3-super-120b (77GB) ran at a quarter concurrency, with the same caveat.

Qwen3-30b-a3b-instruct scores 63% on GPQA. Qwen3.6-35b-a3b, the tester's daily driver, sits in the expensive third of the easy-work cost table. On hard questions it uses about an eighth of the tokens the qwen3-30b model spends to get fewer answers right.

Gemma-4-26b-a4b with reasoning off costs 36 cents per thousand hard answers

With reasoning disabled, gemma-4-26b-a4b is the cheapest hard-task model measured at 36 cents per thousand right answers. It scores 100% on easy work and carries a 3x difficulty tax. It costs 30% less per right answer than gemma-4-12b with no lower score, on a third of the active parameters.

Gemma-3-4b is the cheapest model overall at 2.5 cents per thousand, with 72% accuracy on easy work.

Default reasoning mode cuts accuracy by up to 48 points

Gemma 4 ships with reasoning enabled. Leaving it on cost up to 48 accuracy points and 6x to 8x more money on easy work. Twelve of the tested models also failed to stop generating when told to.

The tester notes that two task sets are enough to show the difficulty tax is real, but not enough to choose a model. The results are meant as a shortlist to validate against a user's own workload.

Published by:
fairsonline_team
Industries
Products
News Type