Heading Toward an Omniverse of Extensions and Microservices
New tools are being developed to help creators generate amazing content to populate the metaverse
By Damien Fagnou, Senior Director of Software, Omniverse, NVIDIA
The new Metaverse — a virtual shared space — will require creating a large quantity of content to populate the worlds that people want to visit and experience. But to create the amazing content that will be required to populate the Metaverse, new tools will need to be created.
In the metaverse, all of the content will be 3D, and this requires an incredible amount of new data to be generated. Making assets using the current creation tools can be time-consuming and hard to do, which won’t allow artists to get to that critical mass of content that is required to make those new worlds rich and immersive. To address this, we need to create new tools. Tools that are more approachable by a larger number of people, tools that leverage AI and big data to generate lots of content rapidly. And finally, all the content needs to be stored in a format that is open and enables easy interoperability.
Making the Computers Work, Instead of You
Even with great tools, generating and updating content will still require lots of time. But with the advances in microservices and cloud scale compute, creators need to transition to do a lot more automation, so the “computers” are constantly doing the work for them in the background. This won’t require programming knowledge as the innovation in AI and no-code workflows will allow the content creators to use graphical interfaces, or even natural language, to define automated workflow and creation tasks.
The Need for Distributed Systems
Doing and storing all this work at a Metaverse-scale will require a completely distributed system that runs in the cloud and data centers. The incredible amount of required compute is only possible if the Metaverse is a connected global infrastructure of compute and storage spread around the world. The software running on those infrastructures must be designed to be distributable and working as small services talking to each other via services interface across fast networks.
Asset Interchange and Smart Assets
As the world of content makers set their sight on new Metaverse opportunities, they need to be able to create assets that are generally useful everywhere, and in a format that is open to all. USD represents that format, and although it will continue evolving to support all workflows, it already fundamentally represents a generic and open format for storage and interchange. Other important technologies come to align with USD to cover other aspects of the assets needs like MDL for material and OpenVDB for Fluid, only to cite a few.
But the assets will also need to be “smart” — they will need to move, encapsulate behavior and configurability, and for that, we need more innovation in the proceduralism around USD. This, too, needs to be open and flexible. NVIDIA is contributing OmniGraph to that space and this would enable content makers to build smart assets that can leverage proceduralism and AI, and run on large distributed supercomputers or local systems alike.
Tools for the Maker
NVIDIA Omniverse is an open platform that addresses the current needs of artists and developers. With all this content to be made, we need to empower the tool makers to build new algorithms and distribute them to the artists of this new world. This needs a language that is easy to understand and learn, one that is already well established in many development areas.
Python is that language, and it is already used by all the AI community, as well as the web and cloud microservices. Omniverse makes Python its primary language for the developer to build their end user tools, still providing access to lower level core C++ and GPU compute for ultimate speed. The Omniverse Kit Extensions system enables anybody with basic programming knowledge to build powerful tools quickly and distribute them to the content makers, or to package them into micro-services to empower new distributed workflows.
Using the Cloud and Data Center
All these tools and technology need to be running in a cloud and data center environment. They need to be designed with modern constructs and small atomic pieces that talk to each other over advanced networking layers. The applications need to be extremely modular to fit into that model and be designed to be distributed at the core. From day one, Omniverse was designed with that premise in mind, and the expertise of NVIDIA in the data center and networking space is key to make it incredibly efficient and powerful.
Along with the tools, the storage of the assets and their “live” nature is key to make the “one world” a reality. Omniverse Nucleus brings lots of important innovations in this area, both by being USD native and designed to be collaborative at the lowest level.
AI Shapes the Future
We have all seen the revolutionary advancement the AI has brought to industries and workflows over the past 10 years — from being able to find or recognize things quickly, to being able to think and plan more precisely than humans in many tasks. It is now starting to create amazing art or content with new skills learned from the vast amount of data that humans have created that the AI algorithms can use to learn from
As the future of AI takes shape, it is clear that content creators will become assisted by AI to build their creations, and the AI will finish the work, propose variation, help on making the content physically correct, or explore new design areas, similar to how someone could get help from a master painter or builder to execute on their vision.
NVIDIA has been at the forefront of this revolution with innovations like GauGAN and Audio2Face, and all those technologies will come into the Omniverse platform to link with the tools and creation pipelines, helping both artists and developers alike to create their vision faster, with more freedom and accuracy.
Learn more about the latest innovations from NVIDIA, including updates on NVIDIA Omniverse, at the GPU Technology Conference, which begins November 8. Make sure to also join us for a special NVIDIA Omniverse Developer Day on November 10, where we’ll share more about the latest tools and techniques that empower artists and developers worldwide, from building new Omniverse extensions to leveraging AI. Join us and explore everything possible in the Metaverse. Register here.
Meet the Author
Damien Fagnou is currently senior director, software at NVIDIA, working on the NVIDIA Omniverse group, where he brings together his expertise in software and VFX production to help build the Omniverse Kit Application platform. Before that, Damien was the senior vice president of technology for Technicolor, where he oversaw technology and software strategy for the VFX group, including MPC Film, for which he was the CTO for many years, working directly on the Oscar-winning films “The Jungle Book” and “1917.” During that time MPC also created the VFX for the groundbreaking “Lion King” remake. With almost 20 years of experience in VFX, many-time GTC contributor Damien looks forward to contributing to the future of graphics software at NVIDIA, built around compute, raytracing, and AI.
