Reprinted to: https://kheresy.wordpress.com/2015/09/04/vgb-cpp-api/
Heresy before " the establishment of Kinect gesture recognition library: Visual Gesture Builder tool (a) " and " the establishment of Kinect gesture recognition library: Visual Gesture Builder (two) " These two articles have the Visual Gesture Builder tool program the basic operations are done a certain extent introduced.
Then this one, it is concerned about how to be in your own C ++ program inside, Gesture Builder provides a C ++ API, and the attitude of the database build out their own through Visual, to identify the gesture.
Visual Gesture Builder files
VGB to be used in C ++ programs inside the C ++ API, it's necessary to introduce the VGB Header file, "Kinect.VisualGestureBuilder.h." The same location where the file location and other K4W SDKv2 of Header files are under "$ (KINECTSDK20_DIR) \ inc."
In the build phase required lib file, it is "Kinect20.VisualGestureBuilder.lib", also located in the "$ (KINECTSDK20_DIR) \ Lib" under this folder. (X86 and x64 placed in different folders within)
As required file is executed, it is in the "$ (KINECTSDK20_DIR) \ Redist \ VGB" under this folder. C ++ program in addition to the "Kinect20.VisualGestureBuilder.dll" this file, but also need to file "vgbtechs" in this folder; two DLL files within a folder This information should be continuity, respectively, discrete gesture detection the algorithm implementation.
VGB's C ++ interface
VGB when using the C ++ API, and will be used basically six major interface it provides, including:
- Posture-related database:
- IVisualGestureBuilderDatabase (original link here are not open)
- Detection of posture:
Wherein, IVisualGestureBuilderDatabase which is used to store read by a database file (.gba or .gbd) out of the posture of use, there is a fundamental need only in the initialization phase.
And IVisualGestureBuilderFrameSource, IVisualGestureBuilderFrameReader and IVisualGestureBuilderFrame these three, it is K4W SDK v2 and other information, like the three-tier architecture interface.
Finally, the interface IVisualGestureBuilderFrame result, you can then, for each posture according to the posture taken IDiscreteGestureResult IContinuousGestureResult or category, as the detection result of each gesture.
Here is the response to these interfaces, to do a more detailed description.
Read VGB repository
Since the posture when using the VGB detection, need to start with the database file (.gba or .gbd) inside, read the information required posture, so before you start to detect posture, we must first pose data library reading come out.
In VGB's C ++ API, you need to pass through CreateVisualGestureBuilderDatabaseInstanceFromFile () to read the file functions, create an object repository, IVisualGestureBuilderDatabase .
The program generally written here will be like this:
wstring sDatabaseFile = L“test.gbd";
IVisualGestureBuilderDatabase* pGestureDatabase = nullptr;
CreateVisualGestureBuilderDatabaseInstanceFromFile(
sDatabaseFile.c_str(), &pGestureDatabase);
In this example, which is to make VGB to read "test.gbd" The database files generated by the VGB tools, and the establishment of a database of objects, pGestureDatabase.
After establishing pGestureDatabase, the next step is through the members of his functions, to read the information inside posture. And because a database which may contain multiple positions, so here must first pass through get_AvailableGesturesCount () this function to obtain the number of available positions, and then through get_AvailableGestures () this function, to each posture to IGesture this type do not extracted.
Here's the basic program can be written in the following way:
// Get how many gestures in database
UINT iGestureCount = 0;
pGestureDatabase->get_AvailableGesturesCount(&iGestureCount);
// get the list of gestures
IGesture** aGestureList = new IGesture*[iGestureCount];
pGestureDatabase->get_AvailableGestures(iGestureCount, aGestureList);
Wherein, iGestureCount This information is pGestureDatabase library, the number of available posture; and then is taken out after aGestureList for individual operation posture (IGesture) array.
However, in practice IGesture this type and other objects can not be directly used for detection, but can only read his basic information only. If you want to read his basic information, it can be written as follows:
GestureType mType;
wchar_t sName[260];
for (int i = 0; i < iGestureCount; ++i)
{
if (aGestureList[i]->get_GestureType(&mType) == S_OK)
{
if (mType == GestureType_Discrete)
cout << “\t[D] “;
else if (mType == GestureType_Continuous)
cout << “\t[C] “;
if (aGestureList[i]->get_Name(260, sName) == S_OK)
wcout << sName << endl;
}
}
It can be seen here, the type of posture can be achieved through get_GestureType () (discrete or continuous), the name of the gesture may be acquired through get_Name ().
However, little is to be noted that, at the time Heresy own testing, is found if used to read the name strings (sName) length than 260 small, the name of the read fails.
Of course, here also note that all functions VGB C ++ API and K4W SDK v2 are the API, will return the results; it is safe, it is best to perform each function are checked back pass if the value is S_OK, to ensure the normal operation of the program there.
VGB read data initialization
Well, read in the library after VGB also need to use other elements can be detected posture, for reading data.
Before being first, VGB to be used to detect posture, in addition to the IVisualGestureBuilderFrameSource VGB interfaces, etc., also need K4W SDK v2 human body detection (IBodyFrameSource) related to the job program, which some of the proposals may refer to " K4W v2 C ++ Part 7: detection, tracking human skeleton . "
As part of the VGB, in fact, and also use other similar frame source, is first through CreateVisualGestureBuilderFrameSource () this function to establish IVisualGestureBuilderFrameSource object, and then through IVisualGestureBuilderFrameSource of OpenReader () this function to get IVisualGestureBuilderFrameReader objects.
But more specifically, is a IVisualGestureBuilderFrameSource object corresponds to only one operator, so if you want to take a posture for multiple operators to detect, then you need to generate more job objects.
Side of the program, can be written as:
IVisualGestureBuilderFrameSource ** ages tered sources = new IVisualGestureBuilderFrameSource * [iBodyCount];
IVisualGestureBuilderFrameReader aGestureReaders ** = new IVisualGestureBuilderFrameReader * [iBodyCount];
for (int i = 0; i <iBodyCount; ++ i)
{
CreateVisualGestureBuilderFrameSource (pSensor, 0, & ages tered sources [i]);
ages tered sources [i] -> AddGestures (iGestureCount, ages tered list);
ages tered sources [i] -> Open Reader (& aGestureReaders [i]);
}
In the above code inside, iBodyCount is the number of human IBodyFrameSource track, basically it should be six.
And here is aimed at the six possible the human body, have each established a IVisualGestureBuilderFrameSource and IVisualGestureBuilderFrameReader, subsequent to the posture detection, and data read.
After that, it is also for each IVisualGestureBuilderFrameSource, are set to be detected posture; here is the use AddGestures () this function to the IGesture array (aGestureList) through get_AvailableGestures () acquired the entire feed into it. If you do not detect each posture, you can also use AddGesture () this function, set one by one.
(When the detected person) Of course, in fact, does not have to start for all potential users, have established a corresponding VGB objects on implementation is also possible, if necessary, before going to the establishment of these items; however, as in the management of resources will be relatively trouble, so here on the use of such a relatively simple method of pre-configured.
Read data back to the main circle
When the program are initialization is complete, the next step is to be inside the main loop, through IVisualGestureBuilderFrameReader provided CalculateAndAcquireLatestFrame () this function, to obtain new data, that is, IVisualGestureBuilderFrame other data type.
Inside the main loop, first of all have to get to the track currently have information about the human body through the IBodyFrameReader. Side of the architecture program can basically written in the following way:
IBodyFrame* pFrame = nullptr;
if (pFrameReader->AcquireLatestFrame(&pFrame) == S_OK)
{
if (pFrame->GetAndRefreshBodyData(iBodyCount, aBody) == S_OK)
{
for (int i = 0; i < iBodyCount; ++i)
{
IBody* pBody = aBody[i];
BOOLEAN bTracked = false;
if ((pBody->get_IsTracked(&bTracked) == S_OK) && bTracked)
{
// Process Body Information
}
}
}
pFrame->Release();
}
After that is read new data (pFrame), then through GetAndRefreshBodyData () to update aBody IBody inside information, and then for each body (pBody), to check if there is to be tracked, if any, before processing continues to go (to perform "// process Body Information" content).
And if you want to use VGB, then here is the need to go get the body tracking number (Tracking ID), and told IVisualGestureBuilderFrameSource, let him go about doing this posture for the human body detection. This part of the program can be written in the following way:
UINT64 uTrackingId = 0;
if (pBody->get_TrackingId(&uTrackingId) == S_OK)
{
UINT64 uGestureId = 0;
if (aGestureSources[i]->get_TrackingId(&uGestureId) == S_OK)
{
if (uGestureId != uTrackingId)
aGestureSources[i]->put_TrackingId(uTrackingId);
}
}
Here, is the first to obtain the number of the body (uTrackingId) through the IBody get_TrackingID (), and then to obtain the corresponding IVisualGestureBuilderFrameSource object (aGestureSources [i]) of the reference current set (uGestureId); if the two values are not same, the representative needs through put_TrackingId () this function to set the number.
After it, you can through CalculateAndAcquireLatestFrame () this function to get a new IVisualGestureBuilderFrame, that is, the following code in pGestureFrame.
IVisualGestureBuilderFrame* pGestureFrame = nullptr;
if (aGestureReaders[i]->CalculateAndAcquireLatestFrame(&pGestureFrame) == S_OK)
{
BOOLEAN bGestureTracked = false;
if (pGestureFrame->get_IsTrackingIdValid(&bGestureTracked) == S_OK && bGestureTracked)
{
for (int j = 0; j < iGestureCount; ++j)
{
// Process each gesture
}
}
pGestureFrame->Release();
}
In the above code inside, is once again confirmed that there are currently posture detection elements have successfully traced to the user. And if it is determined no problem, then the next step is for each posture again, do individual deal.
Front mentioned, VGB posture, there are two discrete and continuous, both of which pose detection result of the data type is different. Discrete gesture detection result (IDiscreteGestureResult) will mainly be a Boolean variable, does it mean that there is detected, then there is a floating point number, representing his reliability (confidence), Alternatively, you can know at present whether it is the first a picture is detected; and continuous gesture detection result (IContinuousGestureResult) there is only one amplitude points, representing the current progress (progress), but be sure to get value.
Since data types vary widely, so the two types of results have to be handled separately. Side of the code can be written in the following form:
// get gesture information
GestureType mType;
aGestureList[j]->get_GestureType(&mType);
const UINT uTextLength = 260;
wchar_t sName[uTextLength];
aGestureList[j]->get_Name(uTextLength, sName);
if (mType == GestureType_Discrete)
{
IDiscreteGestureResult* pGestureResult = nullptr;
if (pGestureFrame->get_DiscreteGestureResult(aGestureList[j], &pGestureResult) == S_OK)
{
BOOLEAN bDetected = false;
if (pGestureResult->get_Detected(&bDetected) == S_OK && bDetected)
{
float fConfidence = 0.0f;
pGestureResult->get_Confidence(&fConfidence);
wcout << L“Detected Gesture “ << sName << L" @" << fConfidence << endl;
}
pGestureResult->Release();
}
}
else if (mType == GestureType_Continuous)
{
IContinuousGestureResult* pGestureResult = nullptr;
if (pGestureFrame->get_ContinuousGestureResult(aGestureList[j], &pGestureResult) == S_OK)
{
float fProgress = 0.0f;
if (pGestureResult->get_Progress(&fProgress) == S_OK)
{
if (fProgress > 0.5f)
wcout << L“Detected Gesture “ << sName << L" “ << fProgress << endl;
}
pGestureResult->Release();
}
}
In the above code inside, mainly to get the type of gesture, as well as name.
Then, if it is discrete gesture, then, is through pGestureFrame of get_DiscreteGestureResult () this function to get the results posture Detection, pGestureResult. The IDiscreteGestureResult itself provides a get_Detected (), get_Confidence (), get_FirstFrameDetected () These three functions can be used to read the results of detection.
In general, first through the first get_Detected () to determine whether this position is detected, reading the other information, or do other processing. In the example above, Heresy is written in this position when there is detected when the corresponding message will be printed out.
If continuity posture, then to use get_ContinuousGestureResult () this function to get the results of the posture detection. The information IContinuousGestureResult can provide essentially simpler, only get_Progress () This function only.
Because of the continuity of gesture detection is basically will give the value of a progress, so if you want to use continuity posture, it would be more trouble, it may be necessary with discrete position, is set to only certain discrete type posture condition is detected only under treatment continuity posture; through this type of approach, basically to deal with in order to truly continuity posture, so he has practical applications.
However, because Heresy here is just a simple example program, it is simply the time when the progress of more than 0.5, the corresponding message printed on it.
Instructions VGB C ++ API is basically to sort out here, and you can operate the complete example code, please refer Heresy put files on GitHub .
This sample program is basically no graphical interface, only text interface. After execution, the database will first pose posture are listed, and then begin to detect; when there is detected posture, it will output the information on the printed.
There are several things you may want to note that:
- Posture database files read is written in the code where the dead, the file name is "test.gbd '; so if you use your own database files do the test, you need to make corresponding changes, otherwise it will not read data.
- When the program execution, in addition to the runtime DLL can read Kinect20.VisualGestureBuilder.dll outside, in the folder also need to have "vgbtechs" this folder. So here, please recommend this DLL file, as well as "vgbtechs" in this folder is copied to the executable file is located.
In short, some of the first so the VGB think about how to use this system to look after it ~ ~