March 24, 2022

Extending product experiences from the physical world into digital twins in the Metaverse

If you read our previous blog on the topic, you’ll know that we see the sense of hearing as an important part of AR and VR experiences in the Metaverse. As new generations of devices and form factors change the way we experience the world around us, it gives us an opportunity to enhance that experience and even overcome barriers that impact our sense of presence in the physical, augmented and virtual worlds.

One way that a greater sense of hearing can enhance that experience is through the way that users will interact with objects, and more specifically products.

As we start to embrace new experiences there is an opportunity for brands to be part of that experience, rather than interrupting through adverts or sponsorship. The reality is that advertising and brand tie-ins will be an important part of the Metaverse business model, as they are in most other areas of entertainment.

However, if we can make products a fun part of the experience then everybody wins: the user has some fun with it, the brand reaches consumers in an acceptable way and drives sales, and the companies building the software or wider ecosystem can drive revenue to fuel further product development.

The technology already exists to recognize certain objects using a camera or by scanning markers placed on products. They enable the product to have a ‘digital twin’ which has some uses like product information or links, however, where they fall short is the ability to create a digital twin of the product experience. A lot of our feelings of enjoyment start with the sound that product makes when we use it – the pop of a champagne cork, the twist of a beer bottle being opened, a packet of chips being rustled and opened, a games console turning on, the sound of a soda can being opened, etc.

For example, imagine wearing your AR glasses during the Super Bowl. A big promotion involving a major soft drinks brand enables you to open one of their products during the half time show and at that moment when the can is opened, the half time entertainer is projected in your lounge. This enables the user to take part in an experience that feels special but is tied to the brand’s sponsorship.

It promotes the experience of using the product, rather than just its proximity to the user. And by understanding the sound of a product being used, consumer brands can extend the physical enjoyment of their products into the augmented or virtual world and make them part of the experience.

We are developing the fundamental building blocks that place this greater sense of hearing at the centre of the xR world. We see this ability to recognize the sounds of products being used as a key enabler of digital twins and we call it Brand Sonification. It presents a great opportunity to synchronise our experiences and enjoyment in both our digital and physical worlds.

Our patent application covering this area has recently been allowed.