The Kinect You Need to Know

Basic principle: to construct a peripheral that would allow users to control a video game using nothing but an advanced motion-detecting camera.

One team was comprised of Microsoft employees, the other, we're told, was run by Hidden Path, a studio made up of many former Microsoft employees (and who would go on to release the excellent Defense Grid).

These guys weren't working with "exotic gyroscopic and accelerometer-based controller prototypes". They were working on a proto-Kinect.

The internal team was looking at technology developed by 3DV, a company Microsoft would in 2009 actually purchase, the other tech from PrimeSense, who would eventually go on to provide the brains behind Kinect. But that's now.

Back then, the technologies went head-to-head, their teams given a brief to present a selection of tech demos to be presented to Moore, who would make the ultimate decision whether to show them at E3 2007 or not.

Some of the demos revealed to Kotaku included "air drumming", a game where you hit balls (similar to one of Kinect's earliest demonstrations), painting, basketball shooting, a lightsaber fighting game and a number of interface ideas and visualisation tricks, like a Minority Report-type menu system and a tool that could make the user look like they were in an iPod commercial.

Some demos performed well (the Minority Report stuff), others not so well (the drumming), but one concept in particular really stole the show. It came from the 3DV team, and had a working title of "Ghost Grid", because it was able to generate a 3D "mesh" of everything the camera could see and incorporate it into a virtual environment, with your real-world actions influencing the on-screen world via a modelled physics system.

Rendering the player as a "ghost" (described as being similar in on-screen appearance to the silhouette in Punch-Out, only capturing the outline of the actual player), this meant that the player could interact with virtual objects and environments in a manner that current Kinect hardware can only dream of.

To give you an idea of how this worked, imagine Kinect's ball-whacking demo. Only, instead of reacting to a single type of object flying at you along a pre-determined path, you could properly interact with a whole range of items at varying speeds and locations. Something pushed slowly would roll/move away slowly. A brick wall built around the player could be smashed down brick by brick. You could even "pick up" a virtual item, and either use it or throw it away, the speed at which you flung it being recreated in the game world.

This demo would work with a single player or, if somebody else joined in, would work just as well with two, the prototype hardware was - unlike the "neutered" Kinect shipping later this year - able to easily handle the physics of two players interacting with the game world.

While impressive, our sources tell us that though it was an amazing experience to take part in, it did not demo well for the bystander. Which considering it was vying for a chance to be shown off at a trade show with the world's media watching was a problem. With the remaining demo concepts all also plagued by technical problems (or the fact they just turned out a bit rubbish), it was decided that for 2007 the technology simply wasn't quite there, and so proto-Kinect was not shown off.

Peter Moore left the company shortly after E3 2007, and upon his leaving, the projects drifted apart as well, with Mattrick having to issue a "call to arms" to get things back on track once he'd taken over from Moore, a process that would eventually lead to Project Natal and the Kinect many of you are putting through its paces today.

We can only wonder, then, what would have happened had Moore stuck around a bit longer and these teams continued their research. Would Kinect have been out sooner, beating Sony's PlayStation Move to market?

It's also interesting to consider just what Kinect could have been capable of had internal hardware not been sacrificed for cost; a beefier Kinect may have been more expensive, but if it could have pulled off the kind of interactivity found in "Ghost Grid", would that have been worth it?

We've checked with Microsoft to see if they can provide any more details on these early prototypes, and will update if we hear back. Likewise, we reached out to Peter Moore for further information, but was told he could "not comment on non-public events that occurred while I was at Microsoft".

Via Gizmodo