The metaverse was first brought into the international limelight last year as Facebook rebranded itself, but it wasn’t till recently that it started to gain traction. With advances in AI, AR, and VR tech, the metaverse is evolving from an abstract concept into something that can rightfully be the future of the internet.
Meta founder and CEO Mark Zuckerberg feels that AI is the key to unlocking the metaverse, and now at the company’s live-streamed “Inside the Lab” event, he unveiled AI projects and Meta’s work on AI research to make the metaverse a reality.
Meta is working on a new class of generative AI models that will allow people to create entire worlds through speech alone, improve how people interact with voice assistants, and bypass the linguistics barrier by translating across all languages.
If you are wondering how you can create a world with words, Meta has an answer in the form of the Builder Bot. In a demonstration, he appeared as a legless 3D avatar on an island and gave speech commands to create a beach and then add clouds, trees, and even a picnic blanket. This feature will be developed further, of course, and in the future, you will be able to create nuanced worlds to explore and share experiences with others, with just your voice.
Coming to how people will interact with voice assistants, the company’s Project CAIRaoke will help them communicate with AI in the metaverse. It, according to Zuckerberg, is a fully end-to-end neural model for building on-device assistants.
With Builder Bot and CAIRaoke, AI assistants will be improved and allow “AI to see the world from our experience” as people entered virtual reality via headsets or glasses, Zuckerberg said.
Meta will also work on self-supervised learning to prepare for how AI could interpret and predict the types of interactions that would occur in the metaverse. It is also working on egocentric data, which involves seeing worlds from a first-person perspective.
But wait, there is more. Project CAIRaoke will be used with Meta’s video-calling Portal device and Meta aimed to integrate it into devices with AR and VR. Meta’s VP for AI Jérôme Pesenti said it was tightly restricting the responses of its new CAIRaoke-based assistant until it could ensure that the system did not generate offensive language.
AI will also be used to moderate activity and content in the metaverse, and Meta is exploring this avenue. The company has been investing in AI for the past 10 years and today, a lot of AI is used for moderation on its main platforms.
Additionally, the metaverse may remove the barriers of languages completely. This will be possible with the universal speech translator the company is working on, the one that is capable of translating between all written languages. The company is aiming to provide instant speech-to-speech translation across all languages. “The ability to communicate with anyone in any language is a superpower that was dreamt of forever,” Zuckerberg said.
