Artificial intelligence in acoustics could open up a multimodal metaverse

News » Science and Technology

 Artificial intelligence in acoustics can open a multimodal metaverse

Researchers at the Massachusetts Institute of Technology and the IBM Watson Artificial Intelligence Lab have created a model that can predict what a user will hear in various places in 3D space. In the paper, co-authored with Yilun Du, a graduate student at MIT, the researchers show how techniques like visual 3D modeling can be applied to acoustics. For example, changing the location of a listener in a room can create a completely different sound experience due to obstructions, the shape of the room, and the nature of the sound, making it difficult to predict the result.

To overcome this problem, the researchers built features of the physical laws of acoustics into their models. The resulting NAF neural acoustic field model was able to randomly select points on this grid in order to study features at specific locations. For example, proximity to a doorway greatly affects what a person hears compared to other geometric features that are on the other side of the room.
The model was then able to predict what a listener might hear from a particular acoustic stimulus based on their relative location in the room.

“By modeling sound propagation, NAF continuously learns to apply the resulting models to arbitrary sounds. We demonstrate that NAF allows us to map spatial acoustics to a listener at an arbitrary location and can predict sound propagation in new locations”,– according to the paper.

Chuang Gan, chief scientist at MIT-IBM's Watson AI Lab, who also worked on the project, said, “This new technique could open up new possibilities in the metaverse.”< /p> Follow us on Telegram

Add a Comment

Your email address will not be published. Required fields are marked *