LeCun Paper Lifts Robot ‘What If’ Scores From 46% to 96%

A new paper co-written by Yann LeCun shows how an AI world model can answer a “what if” question it could not answer before. On one simulated robot task, its score rose from 45.6% to 95.7%, the paper posted on arXiv on 18 November shows.

The authors are Randall Balestriero of Brown University, LeCun of New York University and Kerda Varaku. The paper lists Balestriero and LeCun with AMI Labs, LeCun’s start-up.

The scores are R²: the share of the variation in the real outcome that a prediction captures, where 100% is perfect. We track results like these on our AI model benchmarks page.

Business Pill: The Cost of Checking

An evergreen Business Engineer explainer on the cost of checking work. It teaches the general idea only and does not discuss this paper.

The key insight: A paper by Balestriero, LeCun and Varaku lifts counterfactual R² from 38-46% to 86-96% on four simulators with one extra calculation and no retraining, and finds goal-reaching actions in 35-69% of events against 14-23% for an oracle planner.

The Question World Models Could Not Answer

A world model predicts what happens after an action. The paper’s example is a boat pushed off course by a hidden current. The question is where the boat would have ended up with a different heading, in that same current.

The authors write that the answer “separates bad decisions from bad luck”. They prove that world models trained only on what actually happened cannot answer it on their own, “whatever their data or capacity”.

With the same world model, the paper's correction lifted counterfactual R² on four simulated tasks from 39.9-4
With the same world model, the paper’s correction lifted counterfactual R² on four simulated tasks from 39.9-45.6% to 86.1-95.7% (DINOv2-S encoder; Table 5 of arXiv:2611.24778).

The One-Line Fix

Their fix takes the outcome that was observed and adds the difference the model predicts between the two actions. The paper says this needs “no counterfactual data, extra parameters, or retraining”.

It rests on an old principle the authors quote: “nature makes no leaps”. Over a short time step, the push from the action and the push from the disturbance barely interact, so the disturbance’s effect carries over to the other action.

What the Tests Show

The authors tested JEPA world models working from images, on frozen DINOv2 and masked-autoencoder features. On four rendered simulators, the method lifted counterfactual R² from 38–46% to 86–96% with the same world models, the abstract says.

With the DINOv2-S encoder, the paper’s Table 5 shows FetchReach rising from 45.6% to 95.7%, HandReach from 39.9% to 86.1%, Ball-in-Cup from 42.4% to 91.6% and liquid transport from 43.6% to 93.8%. By our arithmetic, that is a gain of 46 to 50 points.

The authors also asked which action would have reached a goal in the event that had just happened. Their method found it in 35–69% of events, against 14–23% for an oracle planner that knows the simulator but not the outcome.

On FetchReach with the MAE-B encoder, Table 15 shows 69% against 14%. By our arithmetic, that is about 4.9 times the oracle planner’s rate.

Grouped bar chart: in hindsight planning with the MAE-B encoder, the counterfactual world model found the goal-reaching action in 37-69% of events across five simulated tasks, against 14-23% for an oracle planner that knows the simulator but not the outcome
In hindsight planning, the counterfactual world model found the action that would have reached the goal in 37-69% of events across five simulated tasks, against 14-23% for an oracle planner without the outcome (MAE-B encoder; Table 15 of arXiv:2611.24778).

The Structural Read

As we read it, the result matters for robots and agents that must learn from failures: the paper’s own framing is telling a bad decision from bad luck.

As we read it, the limits are as important as the gains: the paper names contacts and disturbances that scale the command as cases where the method can fail, and every test ran in simulation.

Balestriero, LeCun and Varaku, arXiv:2611.24778

“Its answer separates bad decisions from bad luck.”

Three Implications

FOR ROBOTICS TEAMS the paper says the method needs no counterfactual data, extra parameters or retraining.

FOR AI RESEARCHERS the authors prove factual training alone cannot answer individual what-if questions.

FOR READERS every result is from simulation, and the paper has not been peer reviewed.

What Is Not Established

The paper sets out the limits. The authors say the experiments recover the predicted failure modes, “including contacts, folded coordinates, and disturbances that multiply the command”.

All tests ran in simulation. The paper reports no test on a physical robot.

This Week In The AI Supercycle

A World Without an Order, and an AI Central Bank Without a Successor

This week’s issue: the week AI memory stayed largely sold out while the people holding the shares paid: why the AI Supercycle is not a dot-com replay, read through geopolitics, macro and technology, with the gap between building and financing the widest we’ve recorded.

Read the full issue →

Business Engineer Framework

The AI Supercycle

The Business Engineer map of where AI capability turns into business leverage.

Read the Map of AI →

The Bottom Line

A paper by Balestriero, LeCun and Varaku shows a world model answering “what if” questions with one extra calculation and no retraining. Counterfactual scores rose from 38–46% to 86–96% on four simulators, and it beat an oracle planner at finding the action that would have worked.

97,000+ executives read Business Engineer for the AI strategy frameworks cited by ChatGPT, Claude, and Perplexity.

A note on sourcing. Every result comes from the paper itself: its abstract, Table 5 (counterfactual prediction) and Table 15 (one-shot planning). The paper has not been peer reviewed. The point gains and the 4.9 times figure are our arithmetic on the paper’s tables.

Sources: Balestriero, LeCun and Varaku, ‘Counterfactual World Models: JEPAs That Reason About “What If”‘, arXiv:2611.24778, 18 Nov 2026 (abstract, Table 5 and Table 15) · Post on X by @alex_verem, 21 Nov 2026, which pointed us to the paper

Scroll to Top
FourWeekMBA