Olaf-World Demo: Latent Actions That Transfer Across Video Worlds (Action Mixing + Better Control)

Olaf-World tackles a core problem in video world modeling: we want action-controllable world models, but real action labels are scarce. Latent-action methods exist, but the learned “actions” often don’t transfer across contexts — they get entangled with scene-specific cues and lack a shared coordinate system.

The key insight here is neat: even if actions are unobserved, their effects are observable. Olaf-World aligns latent actions to these observable effects, so the same action stays meaningful when the scene changes.

How they do it (high level):
• They introduce SeqΔ-REPA, a sequence-level control–effect alignment objective that anchors the latent action to temporal feature differences extracted by a frozen self-supervised video encoder.
• Built on top of that, Olaf-World pretrains an action-conditioned video world model from large-scale passive video (no explicit action labels).
• In experiments, they report a more structured latent action space, enabling stronger zero-shot action transfer and more data-efficient adaptation to new control interfaces than prior baselines.

Why this matters for “interactive AI worlds”: once actions are aligned by effects, you can reuse and even mix actions from different references in a more stable way — the kind of control interface world-model demos need to stop being pretty videos and start being interactive. (That’s the direction the paper is clearly aiming at.)

Code status: the repo and website are live, but the authors say the code release is coming soon.

Links:
Project page: https://showlab.github.io/Olaf-World/
Paper (arXiv): https://arxiv.org/abs/2602.10104
GitHub: https://github.com/showlab/Olaf-World

Видео Olaf-World Demo: Latent Actions That Transfer Across Video Worlds (Action Mixing + Better Control) канала ABV — AI · Books · Validation