GPT-6 Astra’s best run on CelesteBench, a new test of how well AI models play the platformer Celeste Classic in real time, reached 5.17% progress in five minutes, developer Luc Robert-Villanueva wrote in a blog post on 15 November. The post puts the average for new human players at ~10% under the same time budget.
By our arithmetic, the best model run reached about half (52%) of the new-player average. Across its four runs at low thinking, GPT-6 Astra averaged 1.96%, about a fifth of it.
Business Pill · The Model and the Harness
An evergreen Business Engineer explainer on the difference between an AI model and the harness that runs it. It teaches the general idea only and does not discuss CelesteBench or these results.
The key insight: As we read it, CelesteBench separates knowing from doing: the best model run reached 5.17% of Celeste Classic in five minutes against ~10% for new players, and higher thinking settings mostly lowered real-time scores.
The game has 30 rooms, and the score is progress through them, so clearing the first room is worth 3.33%. The post says “Only two models cleared the first room in RTC”: GPT-6.1 Sol, in a single run at medium thinking, and GPT-6 Astra, in 1 run out of 4. It adds that “Most human players clear it in less than a minute”.
Every other model in the main chart scored 1.07% or 0.53%, still inside room 1. That group includes Claude Opus 5.5, Claude Sonnet 5.5, GPT-6 Sol, GPT-6 Luna, DeepSeek V4.1 Flash, Muse Spark 1.3, Claude Haiku 5.5, GLM 5.3 Flash and Grok 4.6. By our arithmetic, 1.07% is about a third of the way up the first room.
The post also compares cost. Claude Opus 5.5 “lands at the exact same score as DeepSeek V4.1 Flash, at roughly 260 times the cost”, $6.81 a run against $0.03 in its chart.
CelesteBench real-time score, the share of Celeste Classic’s 30 rooms cleared in a 300-second budget, for each model’s best setup, with GPT-6 Astra’s best run and the ~10% new-player average given in the developer’s blog post of 15 November 2026. Clearing room 1 is worth 3.33%.
What Was Posted on X
Haha, this is both funny and fascinating.
Someone built CelesteBench to see how well frontier AI models can play Celeste Classic in real time.
And they’re surprisingly bad at it.
In the developer’s tests, GPT-6 Astra’s best run reached just 5.17% progress in five minutes. Beginner human players averaged around 10%.
Only two models managed to finish even the first room in any tested run.
And giving models more time to reason often made their real-time performance worse.
It’s a small benchmark, so I wouldn’t draw sweeping conclusions from it. But I really like experiments like this. There’s still a huge difference between knowing what to do and actually doing it quickly and reliably.
An X user posted on 15 November about CelesteBench, with a screenshot of the Results section of the developer’s blog post. The post’s figures, GPT-6 Astra’s best run at 5.17% and new players at around 10%, match the blog post we read, and it calls the benchmark small. We take the benchmark’s figures from the developer’s own post and leaderboard data, not from this post.
More Thinking Made Real-Time Play Worse
CelesteBench has two modes. In Real-Time Control (RTC) the game runs at 30 frames a second and keeps moving while the model thinks. In Lite mode the game pauses during thinking, but the 300-second wall-clock budget still applies.
In RTC, the post says, “higher levels of thinking/reasoning (more budget for CoT tokens) worsen the scores” for almost every model. GPT-6 Astra fell from 1.96% at low to 1.07% at high and extra-high. DeepSeek V4.1 Flash fell from 1.07% at low to 0.53% at high and 0% at max. Muse Spark 1.3 fell from 1.07% at low to 0.27% at high.
The developer explains it this way: “They spend more time thinking and sometimes overthinking, and don’t have enough time to actually play the game”. GPT-6.1 Sol is the one case where a higher setting helped: 3.33% at medium, against 1.07% at both low and high.
CelesteBench real-time score at low and high thinking settings, 300-second budget, from the thinking-level table in the developer’s blog post of 15 November 2026. GPT-6 Astra’s low figure is a mean of four runs; GPT-6.1 Sol scored 3.33% at medium and DeepSeek V4.1 Flash 0% at max.
Pausing the Clock Did Not Fix It
If speed were the whole problem, Lite mode should have lifted scores. The post finds Lite scores “surprisingly close to RTC, and sometimes worse”. GPT-6 Astra’s single Lite run scored 5.17%, the same as its best RTC run.
Its conclusion: “So thinking longer does cost wall-clock time in RTC. But removing that cost doesn’t unlock better play either. The bottleneck is somewhere else”. Some models wrote correct strategies for the game in their reasoning, the post says, yet “They know what to do, but they confuse themselves or fail to deduce how to do it”.
At each step a model sees the latest three frames and its own text context, then sends one batch of button presses with a duration for each. GPT models played through Codex, Claude models through Claude Code and the rest through the CelesteBench harness. The one model run through two harnesses, Mistral Large 4, scored 1.07% in both, at $0.13 a run in the CelesteBench harness and $0.34 in Mistral Vibe.
Why Real-Time Agents Are a Different Test
Most model rankings score answers with no clock running. CelesteBench scores what a model gets done while the world keeps moving, the condition that robots, voice agents and computer-use agents work under. The developer frames it that way, writing that models “suck hard at robotic tasks, compared to humans” and that robots are expensive to test on.
Our earlier piece on a robot test put Claude Fable 5.2 at 2.2 times GPT-6 Astra there. On Cursor’s coding benchmark, in our Gemini 4 Hydrogen piece, a higher effort setting lifted Hydrogen’s score from 27.9% to 57.8%. On CelesteBench, with the clock running, higher thinking settings mostly lowered scores.
The Structural Read
As we read the post, the result is about control under a running clock rather than knowledge of the game: models wrote workable strategies, scored about the same with the clock paused, and lost ground when given more thinking time in real-time mode.
Luc Robert-Villanueva, CelesteBench blog post, 15 November 2026
“Even when Astra succeeds, it wastes ~2 minutes on average (in successful runs of course).”
Three Implications
FOR TEAMS BUILDING REAL-TIME AGENTS On CelesteBench, a higher thinking setting lowered the real-time score for GPT-6 Astra, DeepSeek V4.1 Flash, Muse Spark 1.3 and GLM 5.3 Flash, per the developer’s table.
FOR TEAMS COMPARING MODEL COST The post puts Claude Opus 5.5 at the same 1.07% as DeepSeek V4.1 Flash at roughly 260 times the cost per run.
FOR READING THE RESULT Most setups in the leaderboard data have one run, and the post lists the number of runs as its first limitation.
The analysis argues: “The surface, not the model, governs autonomy. The agentic surface — the harness — is becoming where value pools in the entire AI stack.”
As we read it, CelesteBench puts that question under a running clock. In its one same-model test, Mistral Large 4 scored 1.07% through both the CelesteBench harness and Mistral Vibe, at $0.13 against $0.34 a run, and the developer concludes the harness matters. The analysis predates CelesteBench and does not discuss it.
What Is Not Established
CelesteBench is a one-developer benchmark with few runs. The post lists “Number of runs. I would like to do more runs but it’s too expensive” as its first limitation. In the leaderboard data we read, most setups have a single run, and GPT-6 Astra’s 5.17% is the best of four.
Scores cluster on a few values, so many models tie. Harnesses differ by model maker, and the post says external harnesses can keep more images in context. The ~10% human figure is given as an average for new players, without the number of players behind it in the post. We did not run the benchmark.
This Week In The AI Supercycle
AI Lost Its Central Banker, the Lab Oligopoly, and the Future of Open Source
This week’s issue: the Nvidia leadership change, read as the AI build-out losing its central banker: who now finances the GPU landlords, backstops the data centres and sponsors open models, and why two labs are becoming the bank.
On CelesteBench, GPT-6 Astra’s best five-minute run reached 5.17% of Celeste Classic, against ~10% for new human players, about half by our arithmetic. Only GPT-6.1 Sol and GPT-6 Astra cleared the first room, and higher thinking settings mostly lowered real-time scores.
A note on sourcing. Scores, runs, thinking levels, costs, harness details and quotations are from Luc Robert-Villanueva’s CelesteBench blog post of 15 November 2026, including the ~10% average for new human players, which the post states under the same 300-second budget. Run counts per setup are from the CelesteBench leaderboard data file as we read it on 15 November; the modes and scoring are also described on the benchmark’s about page and GitHub repository.
The benchmark drew attention through an X post by @choblin29, embedded above. Shares and ratios are our arithmetic. We did not run the benchmark, and the developer was not contacted for this piece.
Gennaro Cuofano is a CRO and tech executive who has worked in AI since late 2015, starting with bringing NLP, natural-language generation, voice and chatbot products to the marketing industry. His background is in law and finance: he holds a Master's degree in Law and an International MBA with an emphasis on corporate finance (LUISS Business School and the University of San Diego, 2012), and worked as a financial analyst at a real-estate investment firm in San Diego and as an assistant controller. His work focuses on business model strategy, business engineering and, more broadly, structural analysis: how companies make money, read from their own filings. He created FourWeekMBA and leads research there and at The Business Engineer, his newsletter on AI and business strategy, with over 95,000 subscribers and more than 1,000 published analyses. His writing has also appeared on Entrepreneur (for example, How to Digitize a Family Business in Europe) and HackerNoon. Find him on LinkedIn and Substack. See how we source.
