To be a game audio designer in the industry these days means having a lot more skills than just creating epic game audio content. It also means that you need to be up on changes and additions to the technology, and being on the lookout for new methods and techniques of immersive audio that can enhance the player’s experience.
We’ll share some more insight below, but if you’re interested in taking a deep dive, check out the 8 week course in audio for XR at Game Audio Institute. This will be a hybrid course of Zoom meetings and lecture material. You’ll learn everything you need to know about the industry and what audio tool sets are available to use for different project types. You’ll also be working with Steam Audio and Unity, implementing sounds and music for a simple VR based GAI Game Lesson project that you can then use for portfolio work.
Enroll now: Seats are limited to 10 students in order to give maximum attention to each attendee. Right now we have an awesome special - 20% off the course if you use the code gai-vr2022-20 at checkout. Go here for more information.
So, you’ve likely heard lots of fancy new acronyms like VR, AR, MR, and most recently XR describing the fields of Virtual, Augmented, Mixed and Extended or Expanded Reality (nobody agrees on what XR means currently). And it’s possible you’ve come across articles talking all about audio for these mediums and how it’s completely different from audio you’ve experienced in games or other interactive media.
Well, yes and no. Let’s de-mystify how audio for XR works and how it’s both similar and different to audio you’ve already experienced in interactive media and especially, games.
Let’s get some basic facts out about how XR based audio works:
All right, now that we’ve laid the groundwork for a bit of what hasn’t changed, let’s talk about the additions to the audio arsenal that have been added. Some of this is also based on functionality in XR-based games and media.
Because XR devices are currently personally oriented, audio is sent to each individual device being used. XR currently doesn’t utilize a multi speaker setup, so the most common situation encountered is small speakers on the device or a separate set of headphones or earphones.
Preparing spatialized audio for headphones requires binaural encoding to be done. So in XR based situations, the spatialized game audio is sent to a binaural decoder or renderer which processes the sounds so they will sound convincing on headphones.
Okay now that we’ve covered our ears, let’s next discuss the head and why it’s significant for XR.
In XR based games and apps there’s commonly more attention paid to what the hands and the head do in the game space. This isn’t as much a concern for traditional 3D FPS games but head rotation in particular is kept track of, and spatial audio relates in one of two ways to this action:
Now let’s cover the two new spatial audio technologies emerging in XR.
Ambisonic audio is the older of these spatial audio technologies. It was originally adapted from the M/S (Mid Side) miking technique. The Ambisonic audio pioneers added two more microphones, which over time led to the development of the Soundfield microphone with four capsules, and then expansion of the technology to encode recorded audio into the format.
Since the patent expired the four capsule (or tetrahedral) mic also became popular and with several products out there, and it is now the choice for anyone wanting to record audio for 360 film projects.
Ambisonic audio achieves better soundstage accuracy, especially in regards to accurate depiction of height, than standard stereo audio does. The level of detail is governed by the order number, with first order ambisonic files providing the lowest level, and progressing upwards to higher orders, offering greater detail, although at greater computation cost.
Explore Ambisonics libraries from Pro Sound Effects here.
Strengths for Game Audio:
Weaknesses for Game Audio:
Head Related Transfer Function - this is a form of binaural audio filtering based on a group of subjects who had binaural microphones placed in their ears and were subjected to a battery of tests over the entire frequency range at different angles (azimuth) and at different heights (elevation). The result of these tests is compiled into an Impulse Response filter and then real-time convolution is performed on the signal to make it seem as if the sound is arriving right at your eardrum.
With HRTF, having all of this frequency response data recorded at different elevations, we now have all the benefits of standard binaural audio but with the addition of elevation added to the mix. The data is compiled into an impulse response (HRIR), then convolution is performed on the original signal and the sound is modified or filtered to sound like it’s coming directly at our ears.
Strengths for Game Audio:
Weaknesses for Game Audio:
Although both of these new spatial audio technologies can help sound designers in particular achieve more immersive results in their game’s audio, they should not be thought of as replacements for current game audio, but rather considered as an extension to the current audio palette. Instead professional designers should always be focusing on what serves the game best, and what can also be managed in terms of pipeline production for the developer or publisher.
Hope this article was helpful to you! We’ve just scratched the surface on XR audio in game engines. Again, if you’d like to learn more, consider taking the Game Audio Institute course in audio for XR. We only admit 10 students, and right now GAI is offering 20% off with the code gai-vr2022-20 at checkout. Go here for more information.