Kinect Tutorial (C++)

Hi everyone, below is the step-by-step from the Kinect workshop before break! Hit the jump for the full tutorial.

More than a year later, I have returned with more experience to make a few comments. This tutorial is extremely simple- the idea is just to jump in, get the Kinect hooked up and talking to you, and to understand the basics of how it handles data without making you learn everything about Windows programming. It’s really not a good basis for bigger projects, especially ones with visuals. That said, I still wish I had something like this when I started. On to the tutorial!

First, you’ll need a few things:

Visual Studio: I used VS Express in the workshop.

Kinect SDK: With which to program. When you get it downloaded, be sure to open up some of the samples, like Kinect Explorer, so you know you have everything hooked up correctly.

A Kinect: Yep, that’s important.

IF YOU ARE USING AN XBOX KINECT, USB Adapter: If you have an Xbox Kinect and you do not have one of these, you will not play with Kinect today.

Let’s get started, then!

Step One:

Let’s make sure everything with Visual Studio is working. Open up Visual Studio and go to File>New Project. You want a Win32 Console Project:

So make your screen look like that and hit “OK.” Hit “Next” on the welcome page, and then STOP WHAT YOU ARE DOING AND PAY ATTENTION TO THIS NEXT BIT. UNCHECK “Precompiled Header” or you will be a sad panda.

Now, we can go on. Hit “Finish” and your project will be created. You should get a premade project that looks like this:

// KinectProj.cpp : Defines the entry point for the console application.
//

#include "stdafx.h"

int _tmain(int argc, _TCHAR* argv[])
{
    return 0;
}

You did? Excellent! Compile that thang by going up to the top of your screen and hitting “Local Windows Debugger,” with the little green arrow:

Hit “Yes” when the dialog pops up, and watch as a little black console pops up and goes away. This is good. Now, at the bottom of your screen, you should see “The program ‘[6668] <yourprojectname>.exe’ has exited with code 0 (0x0).” You do? Fabulous, move on to the next step! You don’t? Deal with your sadness however you like to handle it (I like to make a cup of tea, look wistfully out the window and sigh while sad music plays in the background) and then go back through the tutorial and see where you went wrong. If you can’t figure it out, I may be able to help at mrachael@umich.edu.

Step Two:

Link up the Kinect. Go up to the top and hit Project>Properties. At the top, switch  the “Configuration” drop-down box to “All Configurations.” Now, in the left-hand window, open up Configuration Properties>C/C++>General, like so: 

Click where it says “<different options>,” the drop down arrow, and then “<Edit…>.” Click in the white space at the top of a new dialog and add the path to your Kinect include files. If you didn’t screw around with the install location during installation and don’t have any drive-related witchcraft going on (in which case I expect you can figure this bit out), it’ll be “C:\Program Files\Microsoft SDKs\Kinect\v1.6\inc”. See below:

Awesome. Visual Studio can be sort of frustrating, so we still have more to do. Hit “OK,” and then in the left-hand window switch to Linker>General. Follow the same procedure to add, under “Additional Library Directories,” the location of your library files, which should be something like “C:\Program Files\Microsoft SDKs\Kinect\v1.6\lib\x86”:

Now hit “OK.”

EDIT: But wait folks, there’s more! I forgot something last time, because yes, there are three separate places you need to do this. In this same window, switch to Linker>Input (you should be under General at the moment) and hit the <Edit…> under Additional Dependencies. Now, we need to add the actual file: “C:\Program Files\Microsoft SDKs\Kinect\v1.6\lib\x86\Kinect10.lib” or something very similar. Hit “Ok” twice to get back to your editing environment.

Hit the Local Windows Debugger again, “Yes,” and that same black box should come up. Make sure the message at the bottom of your screen says “The program ‘[6668] <yourprojectname>.exe’ has exited with code 0 (0x0).” Follow aforementioned procedure for handling sadness if it doesn’t.

Step Three:

We are almost ready to write some actual code! Under the line

#include <stdafx.h>

we need to add a few more things:

#include <iostream>
#include <Windows.h>
#include <NuiApi.h>

Windows.h and NuiApi.h are the Kinect-related things. iostream just lets us print to the screen. I am sort of a weirdo and don’t like the default “main” line they give you (because I am a second year programmer and I haven’t gotten into the stuff it’s used for yet), so I’ve changed the line

int _tmain(int argc, _TCHAR* argv[])

to

using namespace std;
int main()

because that’s just what worked for me. (Hey I snuck a using namespace std in there, that’s also important) The whole file should now be:

// KinectProj.cpp : Defines the entry point for the console application.
//
#include "stdafx.h"
#include <iostream>
#include <Windows.h>
#include <NuiApi.h>
int main()
{
 return 0;

}

Step Four:

We are at the code-writing step! The first thing we need to do here is turn on the Kinect with this line, added right under

int main()
{

NuiInitialize(NUI_INITIALIZE_FLAG_USES_SKELETON);

The flag NUI_INITIALIZE_FLAG_USES_SKELETON tells the software what we need from it- there are more flags and general info here. Next line:

NUI_SKELETON_FRAME ourframe;

which will create a place in memory to hold the data the Kinect sends to us. This isn’t a programming tutorial, so I won’t go into detail about C++ mechanics; the next blob of code, commented with the functions of each line, is:

while (1) //For all of time
 {
   NuiSkeletonGetNextFrame(0, &ourframe); //Get a frame and stuff it into ourframe
   for (int i = 0; i < 6; i++) //Six times, because the Kinect has space to track six people
   {
     if (ourframe.SkeletonData[i].eTrackingState == NUI_SKELETON_TRACKED) //See more on this line below
       cout << "Right Hand: "; //Print "Right hand:"
       cout << ourframe.SkeletonData[i].SkeletonPositions[NUI_SKELETON_POSITION_HAND_RIGHT].y << endl; //See more on this line below
     }
   system("cls");//Clear the screen
 }

So, that may have seemed a little abrupt. The first interesting thing that happens here is we call

NuiSkeletonGetNextFrame(0, &ourframe);

So, what does that mean? NuiSkeletonGetNextFrame is a function that stuffs data into its second argument. The first argument is a buffer, milliseconds to wait, that you don’t need to worry about right now. You want to put “&ourframe” in the second argument- the ampersand tells it “go to this variable’s location in memory and stuff the data in there.” Your frame is now full of data, even if it’s all zeros.

Let’s talk about the line:

if (ourframe.SkeletonData[i].eTrackingState == NUI_SKELETON_TRACKED)

ourframe is a struct with a number of members, the most important of which to you (right now) is the member “SkeletonData.” It’s an array of six NUI_SKELETON_DATA structures. There are six because the Kinect can track up to six people at a time. Each of these has a member, NUI_SKELETON_TRACKING _STATE eTrackingState, that is one of a number of values. The one we’re interested in is NUI_SKELETON_TRACKED, which tells us there’s data on the location of a person in this slot. So, to put that all together, we want to look in ourframe.SkeletonData[i].eTrackingState and see if it’s equal to NUI_SKELETON_TRACKED. If it is, proceed to the next part of the code.

Now, check out

cout << ourframe.SkeletonData[i].SkeletonPositions[NUI_SKELETON_POSITION_HAND_RIGHT].y << endl;

Here, we’re looking in the same place, ourframe.SkeletonData[i], but now we want a different member of SkeletonData. Inside SkeletonData is an array of 20 Vector4 structures, which contain data about a point in 3D space (x, y, z, w). (Chances are you’re only interested in (x, y, z).) This array is indexed by an enumerated type, NUI_SKELETON_POSITION_INDEX, which gives you an easy way to refer to any one joint. We’re just looking at the right hand, NUI_SKELETON_POSITION_HAND_RIGHT, right now, but you could swap that out for anything in the list. We add a .y to get the y coordinate, or a .x for the x, etc. Knowing all that, we can put it together again to get the line

ourframe.SkeletonData[i].SkeletonPositions[NUI_SKELETON_POSITION_HAND_RIGHT].y

The rest is just the syntax to print it to the screen.

That’s all we need to get started. At the end, we need to add one more line to shut down the Kinect. So, the whole program will look like this:

// KinectProject.cpp : Defines the entry point for the console application.
#include "stdafx.h"
#include <iostream>
#include <Windows.h>
#include <NuiApi.h>
using namespace std;
int main()
{
   cout << "Hello world" << endl;
   NuiInitialize(NUI_INITIALIZE_FLAG_USES_SKELETON);
   NUI_SKELETON_FRAME ourframe;
   while (1)
   {
     NuiSkeletonGetNextFrame(0, &ourframe);
     for (int i = 0; i < 6; i++)
     {
       if (ourframe.SkeletonData[i].eTrackingState == NUI_SKELETON_TRACKED)
         cout << "Right Hand: " ;
         cout << ourframe.SkeletonData[i].SkeletonPositions[NUI_SKELETON_POSITION_HAND_RIGHT].y << endl;
     }
     system("cls");
   }
  NuiShutdown();
  return 0;
}

And when this is run, it’ll spit out numbers that correspond to the distance in y from the user’s right hand to the Kinect, in meters. So, let’s try it! Hit the Local Windows Debugger button and “Yes” again, and get up in front of your Kinect. Make sure you don’t have any other Kinect-based programs running. To stop the program, hit ctrl+c on the keyboard and close the window.

Congratulations (hopefully)! That wasn’t so bad, now was it? Let’s recap the important takeaways:

• Start with NuiInitialize. This turns the Kinect on, so it’s sort of important.
• Create a NUI_SKELETON_FRAME to stuff your data into.
• Every time you want new data, call NuiSkeletonGetNextFrame.
• Look in your frame for some people. Remember to check all six slots in the SkeletonData array.
• When you find a person, go into their NUI_SKELETON_DATA and get whatever you need from there, usually information from SkeletonPositions.
• When you’re done, close everything with NuiShutdown.

You’ll find a lot of useful resources in the Programmer’s Guide and Reference. Good luck!

(I’ll try and come back to polish this later, when I get a moment)

---