A new wave of tech could dramatically change how we hear a digitally augmented world. Thanks to the new Bose Frames, anyone can hear personal stories from pilgrims hiking the famous Camino de Santiago trail in Spain—just as if they were walking alongside them. And Tónandi, a collaboration between the mixed-reality company Magic Leap and the Icelandic band Sigur Ros, allows users to create music by interacting with sounds and digital visuals in the room around them.
Augmented reality audio—also known as spatial audio—makes these experiences possible by using a smartphone’s accelerometer, gyroscope and compass to navigate sounds that are sometimes digitally pinned to geolocation. AR audio experiences are giving creators and brands another platform to reach consumers, while others work to find more practical opportunities for hospitals, restaurants and even workout routines.
“Without audio, you can’t lead people to different directions,” said Brian Glasscock, a project manager at Sennheiser, which last fall debuted a pair of AR audio headphones. “You don’t have the same control that you have to guide a film. You can’t cut, you can’t pan, you have to use audio and motion to guide a story. Audio can make or break an experience.”
At South by Southwest last month, Bose debuted its Frames product, which lets people interact with digital sounds placed in their real-world surroundings simply by moving their head or walking around. The stylish sunglasses, which cost $200, will soon help with everything from navigating a city, playing audio-driven games and podcasts, and assisting with workouts. And instead of headphones that go over the ear or inside of the ear, the Bose Frames send sounds toward it—letting users hear both real and digital sounds together.
“What we found that was kind of magical was that when you’re listening in this way nothing is blocking your ear,” said Mehul Trivedi, director of Bose Frames. “You get this clarity, but it’s also not blocking you from the outside world.”
Bose isn’t the first company to create AR headphones. Last fall, Sennheiser, in a partnership with Magic Leap, debuted its own in-ear pair—the Ambeo AR One headphones.
According to Glasscock, the technology that enables spatial audio relies on head-related transfer function, which is how our ears receive sound from various points in space. How sound is perceived depends on a variety of inputs including the position of a person’s head, which allows it to filter time, amplitude and frequency. However, if sounds are artificially imposed, they trick the brain into thinking they’re closer or farther away than they appear.
AR sound is important not just on its own—but also when paired with visuals from virtual and mixed-reality devices. That’s one reason Magic Leap built a software developer kit for its headset that is specifically for spatial audio. Dan Lehrich, Magic Leap’s senior director of production, said spatial audio is key to making visuals appear believable, adding that rendering audio is just as important as rendering graphics.
“Spatial audio is really important for content because it’s a key part of selling the experience,” Lehrich said. “If we think of spatial computing as this seamless blend between the digital and physical world, audio is a key part of the way we perceive the world.”
Agencies are also working with Bose. One of them is Brooklyn-based Huge, which has been testing the sunglasses in the agency’s R&D coffee shop in Atlanta to help baristas listen to instructions in an unobtrusive way. Gela Fridman, president of technology at Huge, said she thinks AR audio might be beneficial for hospitals, navigators and industries where workers need access to information without the distraction of a device with a screen.
“You’re moving around and going about your regular business, but at the same time you can get context that’s important to you,” she said. “There’s no screen, there’s nothing buzzing in your pocket. It’s like walking with a friend—potentially.”