RoboJEPA

Scaling Robotic Latent World Models

Artem Zholus, Nicolas Beltran-Velez, Jianhao Yuan, Sarath Chandar, Tushar Nagarajan, Daniel Severo, Koustuv Sinha, Michal Drozdzal, Adriana Romero Soriano, Jeannette Bohg†, Nicolas Ballas†, Mahmoud Assran†

FAIR at Meta·Mila – Quebec AI Institute·Chandar Research Lab·Polytechnique Montréal

† Joint last author

In the background: imaginations by RoboJEPA-8B. First three DROID camera views, then many robots, cameras and resolutions. Each starts from recorded frames and follows a sequence of actions; everything after the first frame is imagined.

Abstract

Latent world models have shown a remarkable ability to predict future states and to plan in the real world. In practice, however, we lack a principled way to estimate how their capabilities scale with model size, data, and compute, an open problem that slows progress in the field. In this work we present RoboJEPA, a world model based on the Joint Embedding Predictive Architecture (JEPA) and trained on a large-scale dataset spanning 12 robotic embodiments. We show that RoboJEPA's imagination error, the error of its latent rollouts, follows a second-order power law in compute, allowing us to predict model quality well beyond the scale at which the law is fit. We further show that downstream robotic planning performance improves predictably with compute, and that imagination error is strongly correlated with it, making it a reliable proxy for real-robot evaluation. Finally, we demonstrate that latent world models can be deployed zero-shot as robotic agents, planning toward a single goal image to solve tasks requiring long-horizon planning on real hardware. We will release all model checkpoints together with our training and robot deployment code. To our knowledge, this is the first work to establish scaling laws for multi-embodiment robotic world models trained on real robot data, and RoboJEPA, at 8B parameters, is the largest JEPA predictor model trained to date.

Visual abstract

Forward-prediction error of RoboJEPA models from 22M to 8B parameters falls with training compute along a dashed scaling-law fit. Success on four RoboCasa planning tasks rises with training compute. Real-robot success on grasp, object lift and pick-and-place rises with model size.
Three plots. Left: forward-prediction error of RoboJEPA models from 22M to 8B parameters falls with training compute along a dashed scaling-law fit. Middle: success on four RoboCasa planning tasks rises with training compute. Right: real-robot success on grasp, object lift and pick-and-place rises with model size.
RoboJEPA scales predictably from offline prediction to real-robot control. Left: the world model's forward-prediction (ℓ1) error, averaged over the DROID and RoboCasa held-out sets, follows a scaling law in training compute. Middle: downstream planning success in RoboCasa improves predictably with compute across tasks of increasing difficulty. Right: on the real Franka/DROID robot, success rate improves with model size across Grasp, Object Lift, and Pick and Place. Actions are sampled from a uniform distribution and prioritized using the world model during planning.