The idea for this article came after I had a conversation with a client who was in planning process of shooting a 360 video. He asked me, if he should bother with doing an ambisonics location record and a 'special' final mix. In the first moment, I thought, what a dumb question is this, of course, it's a 360 video it needs a spatial sound mix. But after few seconds, I realised, that is a valid question.
And here's the problem. There're very few 360 videos with good binaural mixes available for people to listen to (apart from big production video game showreels or similar). And because people don't hear them, they don't know what actually is 360 spatial sound mix, how it has to sound, what value it provides to video, what full experience can viewer have and most importantly, what it takes to produce one. The whole point of 360 videos is that it allows a person to be fully immersed in the location and space. And sound is a crucial factor in achieving this. Sound has never been more important than it is in this medium. So, let's not fuck this up.
As client explained, in previous 360 video shoot, they used Zoom H2N recorder placed on the floor, under the camera (this can work in certain setups, and if the video is for Youtube which uses 1st order ambisonics ) and married the sound with the picture in post. And yes, you guessed, it sounded shit, as most of the 360 videos in Youtube and Facebook. It's just a mush of sounds without any directionality. In best case of scenario, there are stereo music track laid under.
With ambisonics, it doesn't work like it does with mono and stereo recordings. First, you have to encode the ambisonics recordings to B-Format, before you can actually do anything with it. Then you do the sound design and object based mixing in a pretty complicated Pro Tools or Reaper session and when that's done, you have to know, what platform the mix is for (Youtube, Facebook, Oculus Rift, Gear VR, Google Daydream etc), because they all need to be encoded differently.
It's a new format, so be patient.
Because this technology has started to pick up fairly recently, there're lots of unknown grey areas. First of all, the lack of standards. As mentioned before, final mixes needs to be encoded differently for each playback platform. What about loudness? There're no any criteria or guidelines. But there's a light at the end of a tunnel. Audio tech companies are embracing this technology, releasing more affordable ambisonic microphones, recorders and plugins. Which eventually will lead to more sound engineers engaging this technology, offering the service to production companies, who will then offer this to their clients. Companies like Igloo Visions, for whom I recently did an immersive spatial sound mix, offers a good environment to have a full 360 audiovisual experience. The feedback from them was that the spatial sound mix provided a 'perfect enhancement to the content. Spatial audio really takes the sharedVR to a new dimension'. Even they are a vision focused company, they admit that sound is crucial in providing the full immersive 360 experience. I'm expecting more sporting events to embrace this, too. This year's World Cup Final's will offer 360 experiences. For instance, you can hang out with the footballers in the tunnel before they come on the pitch, or be amongst the winning team when the trophy is presented. And adding a spacial sound to all this, imagine what experience you can have?
However, it'll always be a niche product. I'm no Nostradamus, but I can't see it going mainstream in near future. And there are two reasons for this- the luck of standards and the complexity of it's process - from recording to deliverables. It's gonna take a while before we have any common standards for ambisonic and binaural mixes, like Dolby set for cinema and TV.
Let's get this right from the beginning.
But most importantly, the production companies, producers, directors, brands or whoever decides to embrace and work with this technology, have to understand that they have to have a budget for sound and it's gonna be bigger than they've used to. Forget about self shooting DP with G3 stuck on top of his D5. You'll need more than that - a good ambisonics microphone, you will need a Zoom F8, a sound recordist and your audio post will cost more. But we all want to make this right from the beginning, so we can enjoy this amazing technology and what it offers. We can't afford to think of 360 videos and have a sound as an afterthought. If we're embracing 360 videos, let's use it to full potential and having an ambisonics or binaural mix, is totally worth it. Let's not fuck this up.