Fascinating article by Microsoft Engineers on how the Kinect actually tracks your body and how they came up with the interface that they did. It’s a must read if you’re interested in the technology behind the magic.
At the heart of the skeletal tracking pipeline is a CMOS infrared sensor which allows Kinect to perceive the world, regardless of ambient lighting conditions. Think of this as seeing the environment in a monochrome spectrum of black and white: black being infinitely far away and white being infinitely close. The shades of gray in between these two extremes correspond to a physical distance from the sensor. The sensor gathers each point in its field of view and forms it into a depth image that represents the world. A stream of these depth images is produced at a rate of 30 frames per second, creating a real-time 3-D representation of the environment. Another way to think of this is like those pinpoint impression toys that used to be all the rage. By pushing up with your hands (or your face if you were really adventurous), you could create a simple 3-D model of a piece of your body.
