\n<\/p>\n
You\u2019ve probably seen an artificial intelligence system go off track. You ask for a video of a dog, and as the dog runs behind the love seat, its collar disappears. Then, as the camera pans back, the love seat becomes a sofa.<\/p>\n
Part of the problem lies in the predictive nature of many AI models. Like the models that power ChatGPT, which are trained to predict text, video generation models predict what is statistically most plausible to look right next. In neither case does the AI hold a clearly defined model of the world that it continuously updates to make more informed decisions.<\/p>\n
But that\u2019s starting to change as researchers across many AI domains work on creating \u201cworld models,\u201d with implications that extend beyond video generation and chatbot use to augmented reality, robotics, autonomous vehicles and even humanlike intelligence\u2014or artificial general intelligence (AGI).<\/p>\n
If you’re enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.<\/p>\n
A simple way to understand world modeling is through four-dimensional, or 4D, models (three dimensions plus time). To do this, let\u2019s think back to 2012, when Titanic, 15 years after its theatrical release, was painstakingly converted into stereoscopic 3D. If you were to freeze any frame, you would have an impression of distance between characters and objects on the ship. But if Leonardo DiCaprio had his back to the camera, you wouldn\u2019t be able to walk around him to see his face. Cinema\u2019s illusion of 3D is made using stereoscopy\u2014two slightly different images often projected in rapid alternation, one for the left eye and one for the right. Everyone in the cinema sees the same pair of images and thus a similar perspective.<\/p>\n
Multiple perspectives are, however, increasingly possible thanks to the past decade of research. Imagine realizing you should have shot a photo from a different angle and then having AI make that adjustment, giving the same scene with a new perspective. Starting in 2020, NeRF (neural radiance field) algorithms offered a path to create \u201cphotorealistic novel views\u201d but required combining many photos so that an AI system could generate a 3D representation. Other 3D approaches use AI to fill in missing information predictively, deviating more from reality.<\/p>\n
Now, imagine that every frame in Titanic were represented in 3D so that the movie existed in 4D. You could scroll through time to see different moments or scroll through space to watch it from different perspectives. You could also generate new versions of it. For instance, a recent preprint, \u201cNeoVerse: Enhancing 4D World Model with in-the-Wild Monocular Videos,\u201d describes one way of turning videos into 4D models to generate new videos from different perspectives.<\/p>\n
But 4D techniques can also help generate new video content. Another recent preprint, \u201cTeleWorld: Towards Dynamic Multimodal Synthesis with a 4D World Model,\u201d applies to the scenario with which we began: the dog running behind the love seat. The authors argue that the stability of AI video systems improves when a continuously updated 4D world model guides generation. The system\u2019s 4D model would help to prevent the love seat from becoming a couch and the dog from losing its collar.<\/p>\n
These are early results, but they hint at a broader trend: models that update an internal scene map as they generate. Yet 4D modeling has applications far beyond video generation. For augmented reality (AR)\u2014think Meta\u2019s Orion prototype glasses\u2014a 4D world model is an evolving map of the user\u2019s world over time. It allows AR systems to keep virtual objects stable, to make lighting and perspective believable and to have a spatial memory of what recently happened. It also allows for occlusions\u2014when digital objects disappear behind real ones. A 2023 paper puts the requirement bluntly: \u201cTo achieve occlusion, a 3D model of the physical environment is required.\u201d<\/p>\n
Being able to rapidly convert videos into 4D also provides rich data for training robots and autonomous vehicles on how the real world works. And by generating 4D models of the space they\u2019re in, robots could navigate it better and predict what might happen next. Today\u2019s general-purpose vision-language AI models\u2014which understand images and text but do not generate clearly defined world models\u2014often make errors; a benchmark paper presented at a 2025 conference reports \u201cstriking limitations\u201d in their basic world-modeling abilities, including \u201cnear-random accuracy when distinguishing motion trajectories.\u201d<\/p>\n
Here\u2019s the catch: \u201cworld model\u201d means much more to those pursuing AGI. For instance, today\u2019s leading large language models (LLMs), such as those powering ChatGPT, have an implicit sense of the world from their training data. \u201cIn a way, I would say that the LLM already has a very good world model; it\u2019s just we don\u2019t really understand how it\u2019s doing it,\u201d says Angjoo Kanazawa, an assistant professor of electrical engineering and computer sciences at University of California, Berkeley. These conceptual models, though, aren\u2019t a real-time physical understanding of the world because LLMs can\u2019t update their training data in real time. Even OpenAI\u2019s technical report notes that, once deployed, its model GPT-4 \u201cdoes not learn from experience.\u201d<\/p>\n
\u201cHow do you develop an intelligent LLM vision system that can actually have streaming input and update its understanding of the world and act accordingly?\u201d Kanazawa says. \u201cThat\u2019s a big open problem. I think AGI is not possible without actually solving this problem.\u201d<\/p>\n
Though researchers debate whether LLMs could ever attain AGI, many see LLMs as a component of future AI systems. The LLM would act as the layer for \u201clanguage and common sense to communicate,\u201d Kanazawa says; it would serve as an \u201cinterface,\u201d whereas a more clearly defined underlying world model would provide the necessary \u201cspatial temporal memory\u201d that current LLMs lack.<\/p>\n
In recent years a number of prominent AI researchers have turned toward world models. In 2024 Fei Fei Li founded World Labs, which recently launched its Marble software to create 3D worlds from \u201ctext, images, video, or coarse 3D layouts,\u201d according to the start-up\u2019s promotional material. And last November AI researcher Yann LeCun announced on LinkedIn that he was leaving Meta to launch a start-up, now called Advanced Machine Intelligence (AMI Labs), to build \u201csystems that understand the physical world, have persistent memory, can reason, and can plan complex action sequences.\u201d He seeded these ideas in a 2022 position paper in which he asked why humans can act well in situations they\u2019ve never encountered and argued the answer \u201cmay lie in the ability… to learn world models, internal models of how the world works.\u201d Research increasingly shows the benefits of internal models. An April 2025 Nature paper reported results on DreamerV3, an AI agent that, by learning a world model, can improve its behavior by \u201cimagining\u201d future scenarios.<\/p>\n
So while in the context of AGI, \u201cworld model\u201d refers more closely to an internal model of how reality works, not just 4D reconstructions, advances in 4D modeling could provide components that help with understanding viewpoints, memory and even short-term prediction. And meanwhile, on the path to AGI, 4D models can provide rich simulations of reality in which to test AIs to ensure that when we do let them operate in the real world, they know how to exist in it.<\/p>\n","protected":false},"excerpt":{"rendered":"
You\u2019ve probably seen an artificial intelligence system go off track. You ask for a video of a dog, and as the dog runs behind the love seat, its collar disappears. Then, as the camera pans back, the love seat becomes a sofa. Part of the problem lies in the predictive nature of many AI models.<\/p>\n","protected":false},"author":1,"featured_media":41988,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[58],"tags":[1564,1443,4112,3508,6765,550],"class_list":{"0":"post-41987","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-science","8":"tag-artificial","9":"tag-intelligence","10":"tag-models","11":"tag-revolution","12":"tag-unlock","13":"tag-world"},"_links":{"self":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/posts\/41987","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=41987"}],"version-history":[{"count":0,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/posts\/41987\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/media\/41988"}],"wp:attachment":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=41987"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=41987"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=41987"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}