A functional taxonomy of world models
On 3 June 2026 World Labs and Fei-Fei Li published an essay that divides everything called a “world model” into three functions by what it outputs: a renderer outputs observations (pixels), a simulator the state of the world, a planner actions; the essay calls simulation “the linchpin”.
Why it matters
The term “world model” had come to cover video models, physics engines and language models that improvise a game; the essay offers a yardstick, what the model outputs, against which they can be compared. An editorial assessment: this is one interested party’s proposal, and it was being discussed within a week; whether it sticks is unknown.
The essay derives three kinds from the agent-action-state-observation loop of a partially observable Markov decision process. A renderer outputs observations for the human eye; examples are Genie 3 and RTFM. A simulator outputs state: geometry, physics, dynamics that both people and programs can work with; Marble is named as the company’s first move that way (Gaussian splats together with collision meshes). A planner outputs actions: vision-language-action models and the new “world action models”. The authors take simulation to be the linchpin: from it one can derive both the pixel view and the consequences of actions. The boundaries between the three, they say, are blurring, and the logical endpoint is a single model. The essay traces the phrase “world model” to Kenneth Craik’s 1943 proposal. A market of more than a trillion dollars for NVIDIA Omniverse is given as NVIDIA’s own estimate. What the record does not state. The essay has no experiment or measurement; it is a classification, not a result. Its author is a co-founder of the company that sells Marble, and it illustrates the renderer and simulator categories with its own products. A HackerNoon commentator of 10 June, who called the framing needed, remarks that it is a taxonomy of outputs, not of internal representations; that is one view, not a survey. Whether the classification was adopted in the field the record does not establish.