Binaural blast (from the past?)
Recently I was doing more experiments in lightweight binaural spatialization for VR. Using the Ambisonic Toolkit (aka ATK), in SuperCollider, I implemented a binaural mixer for arbitrary number of 3D sources that includes Doppler shift and a faked reverb cue as audio sources move further away.
Here’s a quick test, with some spaceship-like sounds whizzing past (mostly from side to side, because the effect is more obvious). Listen with headphones if possible:
When running this with head tracking on the Oculus display, it’s quite convincing as the sources fly toward and past you in all directions and at different elevations. You can easily guess the direction to an approaching source, even with closed eyes. The graphics part is done in openFrameworks, with that app sending binaural source and listener information to SuperCollider via OSC messages. The graphics are very minimalistic – spheres – enough to ensure the sounds match their source positions as you look around in the Oculus DK2.
I also tried some other approaches, doing my own convolution (in SC) with the HRIR data directly (e.g. taken from the CIPIC or Listen datasets). I was able to support hundreds of simultaneous sources (more than 250 on my 2012 MacBook Air without dropping audio frames), but it’s lighter and simpler to use the ATK, because the decoding (HRTF convlution) only needs to happen once.
I also tried a few experiments with the Oculus Audio SDK, but just in a DAW, not with programmatic control. I prefer something that runs in SuperCollider, where I like to generate my sounds, and this unfortunately rules out VST or AU solutions (at least, without bending over backwards to make a plugin host UGen, or piping audio in and out using Jack or SoundFlower).
Over 20 years ago, we did binaural audio for VR using Crystal River Engineering’s Convolvatron (or Acoustetron), running on a dedicated PC and usually providing eight spatialized sources. Those turnkey systems were great (and still exist in some form), but it’s nice to be able to support hundreds of simple sources – plus the graphic rendering – on a single machine! And be able to check my mail at the same time. (-;