Local LLM Testing: A 2015 GPU vs 15 Cloud Models

The short version

Same three jobs, 18 models: three compressed models on a 2015 graphics card, six open models and nine closed ones in the cloud.
Every closed model drew a valid pelican first time, and five of the six open ones did. None of the local ones did.
Open models are now close behind: GLM-5.3, Kimi K3, DeepSeek V3.2 and Qwen3 235B passed every check.
Local still wins on one thing: nothing leaves the building. Play every result below.

Every model's pelican

The test Simon Willison made famous: draw a pelican riding a bicycle, in SVG. These are the real files, unedited.

Never a valid drawing

Q2_0Local, 2015 GPU

Never a valid drawing

IQ2_XSLocal, 2015 GPU

Coder IQ1_M's pelican riding a bicycle — **Coder IQ1_M**Local, needed an extra turn

GLM-5.3's pelican riding a bicycle — **GLM-5.3**Open model, Z.ai

Kimi K3's pelican riding a bicycle — **Kimi K3**Open model, Moonshot AI

DeepSeek V3.2's pelican riding a bicycle — **DeepSeek V3.2**Open model, DeepSeek

Qwen3 235B's pelican riding a bicycle — **Qwen3 235B**Open model, Alibaba

Devstral 2's pelican riding a bicycle — **Devstral 2**Open model, Mistral

Never a valid drawing

MiniMax M3Open model, MiniMax

Claude Opus 5.5's pelican riding a bicycle — **Claude Opus 5.5**Closed model, Anthropic

Claude Sonnet 5.5's pelican riding a bicycle — **Claude Sonnet 5.5**Closed model, Anthropic

Claude Haiku 5.5's pelican riding a bicycle — **Claude Haiku 5.5**Closed model, Anthropic

GPT-6.1-Sol's pelican riding a bicycle — **GPT-6.1-Sol**Closed model, OpenAI

GPT-6-Astra's pelican riding a bicycle — **GPT-6-Astra**Closed model, OpenAI

GPT-6-Luna's pelican riding a bicycle — **GPT-6-Luna**Closed model, OpenAI

Gemini 3.8 Flash's pelican riding a bicycle — **Gemini 3.8 Flash**Closed model, Google

Gemini 4 Argon's pelican riding a bicycle — **Gemini 4 Argon**Closed model, Google

Muse Spark 1.3's pelican riding a bicycle — **Muse Spark 1.3**Closed model, Muse

Play the results yourself

Pick a job. The selected model's file runs right here; swap models to compare. The chart and scoreboard follow.

Try it yourself

The models' real, unedited files. It starts by itself; pick another model to swap.

Version

Press Play to open the game in its own window: arrows or A/D to move, P to pause, ✕ to close. Open in a new tab.

Scoreboard:

Local models got our detailed browser checks; hosted models got the automated ones (loads, starts, valid SVG). Tap a model for details.

Show the exact prompt for this job

Create a complete playable browser game called Meteor Dodge. Return ONLY one self-contained HTML document with inline CSS and JavaScript, no Markdown or explanation. No libraries, network calls or external assets. Use a canvas: a player spaceship moves with arrow keys or A/D, meteors fall, collisions cost one of 3 lives, surviving increases score, difficulty increases over time, and a game-over screen offers a working restart button. Include a Start button, pause/resume with P, visible score and lives, instructions, and touch left/right buttons. Use a polished dark space theme. Keep the code compact and complete, under 1700 tokens.

What we tested

Local: a 2015 NVIDIA GTX TITAN X (12 GB) running three Qwen models squeezed to about 2 bits per weight, on our own CUDA inference engine.
Open models, hosted: GLM-5.3, MiniMax M3, Kimi K3, DeepSeek V3.2, Qwen3 235B and Devstral 2, through opencode (our subscription, or Amazon Bedrock). These are the weights a business could run on its own bigger hardware.
Closed models, hosted: Claude Opus, Sonnet and Haiku 5.5; GPT-6.1-Sol, GPT-6-Astra and GPT-6-Luna; Gemini 3.8 Flash and Gemini 4 Argon; Muse Spark 1.3. Each through a command-line tool, at low effort.
Same for everyone: the exact prompts, one attempt, no code fixed by hand.

What we found

Closed models are the safest bet. All nine drew a valid pelican and all nine games start. Two games (Opus 5.5 and GPT-6-Luna) throw one JavaScript error but keep running.
Open models are close behind. GLM-5.3, Kimi K3, DeepSeek V3.2 and Qwen3 235B passed every check. Devstral 2 had errors in its game and dashboard; MiniMax M3's game never starts and its pelican is broken.
The old card is slow but not useless. It writes 14 to 29 tokens a second; hosted models managed 33 to 260 even counting tool overhead. IQ2_XS still built a dashboard that passed every check.
Valid isn't the same as good. Every open pelican but one is valid SVG, but Qwen3 235B's is lying on the ground next to its bike and Devstral 2's is a tangle. Look at the output, not just the checks.
Bigger isn't always better. Claude Sonnet 5.5 was the most consistent: 16 to 17 seconds a job, every check passed.

What this means for you

Open models are the middle path. The best ones match closed models on these jobs, and you can host them yourself on a modern GPU.
Test on your own work. Ten real tasks with a pass/fail check beat any leaderboard.
Run the output. Two of the prettiest games had bugs only a browser found.

One run per job: a smoke test, not a ranking. Want to size hardware or test models on your own tasks? That's our private AI and local LLM hosting.

All the numbers in one table

Job	Model	First attempt (s)	Tokens/s	Output tokens	Cut off?	Browser checks passed
Game	Q2_0	45.6	22.9	965	No	6 of 9
Game	IQ2_XS	90.5	17.6	1,530	No	5 of 7
Game	Coder IQ1_M	159.5	14.2	2,200	Yes, finished in one extra turn	3 of 6
Dashboard	Q2_0	63.9	22.0	1,313	No	6 of 7
Dashboard	IQ2_XS	99.7	17.1	1,633	No	9 of 9
Dashboard	Coder IQ1_M	135.9	16.8	2,200	Yes, finished in one extra turn	1 of 6
Pelican SVG	Q2_0	190.5	28.7*	2,400	Yes, still invalid after an extra turn	—
Pelican SVG	IQ2_XS	246.1	25.6*	2,400	Yes, still invalid after an extra turn	—
Pelican SVG	Coder IQ1_M	116.5	23.0*	2,400	Yes, valid after an extra turn	—
Game	Claude Opus 5.5 (cloud)	28.3	80.0†	2,263	No	Loads, starts, 1 JS error
Dashboard	Claude Opus 5.5 (cloud)	18.6	117.7†	2,190	No	Loads, no JS errors
Pelican SVG	Claude Opus 5.5 (cloud)	20.0	84.2†	1,685	No	Valid SVG
Game	Claude Sonnet 5.5 (cloud)	16.1	136.7†	2,201	No	Loads, starts, no JS errors
Dashboard	Claude Sonnet 5.5 (cloud)	16.7	153.0†	2,555	No	Loads, no JS errors
Pelican SVG	Claude Sonnet 5.5 (cloud)	15.6	120.3†	1,877	No	Valid SVG
Game	Claude Haiku 5.5 (cloud)	32.5	241.4†	7,846	No	Loads, starts, no JS errors
Dashboard	Claude Haiku 5.5 (cloud)	15.2	259.9†	3,951	No	Loads, no JS errors
Pelican SVG	Claude Haiku 5.5 (cloud)	29.4	190.8†	5,609	No	Valid SVG
Game	GPT-6.1-Sol (cloud)	53.8	35.7†	1,918	No	Loads, starts, no JS errors
Dashboard	GPT-6.1-Sol (cloud)	47.1	39.6†	1,867	No	Loads, no JS errors
Pelican SVG	GPT-6.1-Sol (cloud)	65.1	35.5†	2,308	No	Valid SVG
Game	GPT-6-Astra (cloud)	44.0	39.7†	1,747	No	Loads, starts, no JS errors
Dashboard	GPT-6-Astra (cloud)	44.0	43.1†	1,898	No	Loads, no JS errors
Pelican SVG	GPT-6-Astra (cloud)	62.4	33.4†	2,086	No	Valid SVG
Game	GPT-6-Luna (cloud)	18.4	93.5†	1,721	No	Loads, starts, 1 JS error
Dashboard	GPT-6-Luna (cloud)	20.4	96.1†	1,961	No	Loads, no JS errors
Pelican SVG	GPT-6-Luna (cloud)	18.7	84.2†	1,574	No	Valid SVG
Game	Gemini 3.8 Flash (cloud)	23.0	138.4†	3,183	No	Loads, starts, no JS errors
Dashboard	Gemini 3.8 Flash (cloud)	25.5	138.7†	3,537	No	Loads, no JS errors
Pelican SVG	Gemini 3.8 Flash (cloud)	48.1	124.3†	5,980	No	Valid SVG
Game	Gemini 4 Argon (cloud)	46.1	105.4†	4,861	No	Loads, starts, no JS errors
Dashboard	Gemini 4 Argon (cloud)	39.2	103.1†	4,043	No	Loads, no JS errors
Pelican SVG	Gemini 4 Argon (cloud)	79.0	94.9†	7,500	No	Valid SVG
Game	GLM-5.3 (open)	35.2	57.8†	2,033	No	Loads, starts, no JS errors
Dashboard	GLM-5.3 (open)	31.1	71.8†	2,237	No	Loads, no JS errors
Pelican SVG	GLM-5.3 (open)	211.7	60.1†	12,731	No	Valid SVG
Game	MiniMax M3 (open)	32.4	102.3†	3,318	No	Loads, never starts
Dashboard	MiniMax M3 (open)	53.6	139.8†	7,486	No	Loads, no JS errors
Pelican SVG	MiniMax M3 (open)	48.5	134.5†	6,517	No	Invalid
Game	Kimi K3 (open)	27.7	82.1†	2,274	No	Loads, starts, no JS errors
Dashboard	Kimi K3 (open)	91.8	55.5†	5,096	No	Loads, no JS errors
Pelican SVG	Kimi K3 (open)	89.4	106.8†	9,544	No	Valid SVG
Game	DeepSeek V3.2 (open)	90.1	47.4†	4,270	No	Loads, starts, no JS errors
Dashboard	DeepSeek V3.2 (open)	87.8	48.5†	4,257	No	Loads, no JS errors
Pelican SVG	DeepSeek V3.2 (open)	52.0	36.4†	1,893	No	Valid SVG
Game	Qwen3 235B (open)	59.5	56.7†	3,374	No	Loads, starts, no JS errors
Dashboard	Qwen3 235B (open)	43.8	57.0†	2,493	No	Loads, no JS errors
Pelican SVG	Qwen3 235B (open)	36.3	50.4†	1,829	No	Valid SVG
Game	Devstral 2 (open)	32.4	64.9†	2,101	No	Loads, starts, 1 JS error
Dashboard	Devstral 2 (open)	66.3	71.8†	4,758	No	Loads, 1 JS error
Pelican SVG	Devstral 2 (open)	32.1	54.9†	1,765	No	Valid SVG
Game	Muse Spark 1.3 (cloud)	29.8	105.0†	3,127	No	Loads, starts, no JS errors
Dashboard	Muse Spark 1.3 (cloud)	20.0	113.0†	2,261	No	Loads, no JS errors
Pelican SVG	Muse Spark 1.3 (cloud)	12.1	184.8†	2,232	No	Valid SVG

Hosted times are end-to-end through each command-line tool. †Hosted speed is effective speed: reported output (and reasoning) tokens divided by that end-to-end time. Hosted token counts include reasoning where the tool reports it. Local check counts aren't a score: each output got the checks that fitted what it built. Local pelican times can include loading the model. *Pelican speeds come from a separate streaming re-run on 9 October with the same prompt and settings; its drawings differed slightly but failed the same way.

Photo credits. Hero: two NVIDIA Titan X cards, Back2Gaming on Unsplash, used under the Unsplash License.

Sources. Our own test runs on 8 and 9 October 2026. Hugging Face GGUF docs for bits per weight. GeForce 900 series for the card. OAIC guidance on commercial AI products on privacy.