[Microphone Array PIC] Kinect Chapter 15.   Using the Kinect's Microphone Array


This chapter is not included in the book.

[Beamforming PIC]

The previous chapter was about speech recognition, with audio recorded from the PC's microphone rather than the Kinect. Frankly, a bit of a cheat, but still useful.

This chapter shows how to capture sound from the Kinect's microphone array, so there's no longer any need to feel guilty about using an extra microphone. The trick is to install audio support from Microsoft's Kinect SDK which lets Windows 7 treat the array as a standard multichannel recording device. Care must be taken when mixing this audio driver with OpenNI, but the payoff is that the microphone array becomes visible to Java's sound API. The main drawback is that this only works on Windows 7 since Microsoft's SDK only supports that OS.

[Spoken Kinect Breakout PIC]

My Java examples start with several tools for listing audio sources and their capabilities, such as the PC's microphone and the Kinect array. I'll also describe a simple audio recorder that reads from a specified source and saves the captured sound to a WAV file.

One of the novel features of Microsoft's SDK is support for beamforming the ability to calculate the direction of an audio source. It's possible to duplicate this using the Java SoundLocalizer application developed by Laurent Calmes.

I finish with another version of my audio-controlled Breakout game, this time employing the Kinect's microphone array instead of the PC's mike.




Dr. Andrew Davison
E-mail: ad@fivedots.coe.psu.ac.th
Back to my home page