#Aix development board uses OpenVinoSharp
Link to the original text of the project: [Aix Development Board Trial] Aix Development Board uses OpenVinoSharp
Intel
distribution version of the OpenVINO™ toolkit developed based on oneAPI, which can speed up the development of high-performance computer vision and deep learning vision application toolkit, suitable for from edge to cloud , helping users deploy more accurate real-world results to production systems faster. Through a simplified development workflow, OpenVINO™ empowers developers to deploy high-performance applications and algorithms in the real world.
C# is a safe, stable, simple, and elegant object-oriented programming language derived from C and C++. C# combines the simple visual operation of VB and the high operating efficiency of C++. With its powerful operation ability, elegant syntax style, innovative language features and convenient component-oriented programming support, it has become the preferred language for .NET development. However, OpenVINO™ does not provide a C# language interface, which has brought a lot of trouble to the use of OpenVINO™ in C#. In our previous work, we launched OpenVinoSharp, which aims to promote the application of OpenVINO™ in the field of C#. It has been successfully used in Window platform implementation. In this article, we will introduce how to implement OpenVinoSharp on the AIxBoard development board based on the Linux system.
1. Introduction to Intel Development Kit AIxBoard
1. Product positioning
The Intel Development Kit AIxBoard is a member of the official series of Intel Development Kits , designed for entry-level artificial intelligence applications and edge smart devices. Xboard can perfectly compete in different application scenarios such as artificial intelligence learning, development, training, and application. The kit comes pre-installed with the Intel OpenVINO™ toolkit, model repository, and demo cases, allowing you to quickly and easily start application development.
The main interface of the kit is compatible with the Jetson Nano carrier board, and the GPIO is compatible with the Raspberry Pi, which can maximize the reuse of mature ecological resources. This enables the kit to be used as an edge computing engine to provide strong support for artificial intelligence product verification and development; at the same time, it can also be used as a domain control core to provide technical support for robot product development.
Using the AIxBoard (AixBoard) development kit, you will be able to build an excellent artificial intelligence application in a short time. Whether it is used for scientific research, education or business, X-Board can provide you with good support. With the help of the OpenVINO™ toolkit, both the CPU and iGPU have powerful AI reasoning capabilities, and support the parallel operation of multiple neural networks in applications such as image classification, object detection, segmentation, and speech processing.
2. Product parameters
| master control | Intel Celeron N5105 2.0-2.9GHz (formerly Jasper Lake) |
|---|---|
| Memory | Onboard LPDDR4x 2933MHz, 4GB/6GB/8GB |
| storage | Onboard 64GB eMMC storage |
| storage expansion | 1 M.2 Key-M 2242 expansion slot, support SATA&NVME protocol |
| BIOS | AMI UEFI BIOS |
| System Support | Ubuntu20.04 LTS |
| Winodws 10/11 |
3. AI reasoning unit
With the help of OpenVINO tools, CPU+iGPU heterogeneous computing reasoning can be realized, and the computing power of IGPU is about 0.6TOPS
| CPU | INT8/FP16/FP32 |
|---|---|
| iGPU | INT8/FP16 0.6TOPS |
| GNA | Gaussian and Neural Accelerator |
2. Configure the .NET environment
.NET is a free, cross-platform, open source developer platform for building a wide variety of applications. The following will demonstrate how AIxBoard installs .NET on Ubuntu 20.04 and supports .NET 6. If your AIxBoard uses other Linux systems, you can refer to Installing .NET on Linux distributions - .NET | Microsoft Learn .
1. Add the Microsoft package repository
Installation using APT can be accomplished with a few commands. Before installing .NET, run the following commands to add the Microsoft package signing key to the list of trusted keys and add the package repository.
Open a terminal and run the following command:
wget https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
sudo dpkg -i packages-microsoft-prod.deb
rm packages-microsoft-prod.deb
The following figure shows the output of the console after entering the above command:
2. Install the SDK
The .NET SDK enables you to develop applications through .NET. If you install the .NET SDK, you don't need to install the corresponding runtime. To install the .NET SDK, run the following command:
sudo apt-get update
sudo apt-get install -y dotnet-sdk-6.0
The following figure shows the console output after installation:
 |
 |
3. Test the installation
You can check the SDK version and runtime version through the command line.
dotnet --list-sdks
dotnet --list-runtimes
The following figure shows the output of the console after entering the test command:
4. Test console project
In the linux environment, we can create and compile projects through the dotnet command. The project creation command is:
dotnet new <project_type> -o <project name>
Here we create a simple test console project:
dotnet new console -o test_net6.0
cd test_net6.0
dotnet run
The figure below shows the output of the console and the project folder file after inputting the test command. The C# project will automatically create a Program.cs program file, which contains the main function of the program running entry, and will also create a ***.csproj* * file, which is responsible for specifying some configurations in project compilation.
The above are the configuration steps for the .NET environment. If your environment does not match this article, you can get more installation steps through .NET Documentation | Microsoft Learn .
3. Install OpenVINO Runtime
There are two ways to install OpenVINO™: OpenVINO Runtime and OpenVINO Development Tools. OpenVINO Runtime contains core libraries for running model deployment inference on processor devices. OpenVINO Development Tools is a set of tools for processing OpenVINO and OpenVINO models, including model optimizer, OpenVINO Runtime, model downloader, etc. Here we only need to install OpenVINO Runtime.
1. Download OpenVINO Runtime
Visit the Download the Intel Distribution of OpenVINO Toolkit page, and select the corresponding installation option according to the following process. On the download page, since our device uses Ubuntu20.04 , download according to the specified compiled version when downloading.
2. Unzip the installation package
The OpenVINO Runtime we downloaded is essentially a C++ dependency package, so we put it in our system directory, so that dependencies will be obtained according to the set system variables during compilation. First create a folder under the system folder:
sudo mkdir -p /opt/intel
Then unzip the installation file we downloaded and move it to the specified folder:
tar -xvzf l_openvino_toolkit_ubuntu20_2023.0.1.11005.fa1c41994f3_x86_64.tgz
sudo mv l_openvino_toolkit_ubuntu20_2023.0.1.11005.fa1c41994f3_x86_64 /opt/intel/openvino_2022.3.0
3. Install dependencies
Next, we need to install the dependencies of the OpenVINO Runtime, just enter the following command through the command line:
cd /opt/intel/openvino_2022.3.0/
sudo -E ./install_dependencies/install_openvino_dependencies.sh
4. Configure environment variables
After the installation is complete, we need to configure the environment variables to ensure that the system can obtain the corresponding files when calling. Enter the following command through the command line:
source /opt/intel/openvino_2022.3.0/setupvars.sh
The above are the configuration steps of the OpenVINO Runtime environment. If your environment does not match this article, you can get more installation steps through Install OpenVINO™ Runtime — OpenVINO™ documentation — Version (2023.0) .
4. Implementation of OpenVinoSharp on AIxBoard
In the previous steps, we have installed **.NET**, OpenVINO Runtime environment, write down that we are entangled with the above two tools to implement OpenVinoSharp on AIxBoard.
1. Create and configure a test project
First we create a C# console project through dotnet :
dotnet new console -o test_openvinosharp
test_openvinosharp
Edit the Program.cs file with a text compiler , first add the following code to the file to build the program structure:
using System;
using System.Runtime.InteropServices;
namespace test_openvinosharp
{
internal class Program{
static void Main(string[] args)
{
}
}
}
2. Add OpenVINO interface
In this project, the implementation idea is to call the C API interface in OpenVINO through the dynamic link library file, so we first implement the API interface we use. Since this is a test project, we only read some related interfaces , the implementation principle is as follows:
[DllImport(string dll_path, EntryPoint = string func_name, CharSet = CharSet.Unicode, CallingConvention = CallingConvention.Cdecl)]
public extern static void func(Val val);
The reading of the C API interface through the dynamic link interface needs to meet the following three points: function name correspondence, function parameter correspondence, and function return value correspondence. Due to the inconsistency of the programming language, there will be corresponding differences in data types, so it is necessary to ensure that the data between corresponding to the format.
Below we read several interfaces, mainly for testing whether OpenVINO is imported successfully.
class NativeMethods{
const string dll_extern = "libopenvino_c.so";
[DllImport(dll_extern, EntryPoint = "ov_get_openvino_version", CharSet = CharSet.Unicode, CallingConvention = CallingConvention.Cdecl)]
public extern static int ov_get_openvino_version(IntPtr version);
[DllImport(dll_extern, EntryPoint = "ov_version_free", CharSet = CharSet.Unicode, CallingConvention = CallingConvention.Cdecl)]
public extern static int ov_version_free(IntPtr version);
[DllImport(dll_extern, EntryPoint = "ov_core_create", CharSet = CharSet.Unicode, CallingConvention = CallingConvention.Cdecl)]
public extern static int ov_core_create(ref IntPtr core);
[DllImport(dll_extern, EntryPoint = "ov_core_free", CharSet = CharSet.Unicode, CallingConvention = CallingConvention.Cdecl)]
public extern static int ov_core_free(IntPtr core);
[DllImport(dll_extern, EntryPoint = "ov_core_get_versions_by_device_name", CharSet = CharSet.Unicode, CallingConvention = CallingConvention.Cdecl)]
public extern static int ov_core_get_versions_by_device_name(IntPtr core, ref sbyte device_name, IntPtr versions);
[DllImport(dll_extern, EntryPoint = "ov_core_versions_free", CharSet = CharSet.Unicode, CallingConvention = CallingConvention.Cdecl)]
public extern static int ov_core_versions_free(IntPtr versions);
}
3. Add Core Version
A Version structure is defined below, which is mainly used to obtain the Core Version, read the OpenVINO version, and mainly call the ov_get_openvino_versioninterface implementation. The specific implementation process will not be described here.
public struct Version
{
public string buildNumber;
public string description;
public Version(string buildNumber, string description) {
this.buildNumber = buildNumber;
this.description = description;
}
public string to_string()
{
string str = "";
str += description;
str += "\r\n Version : ";
str += buildNumber.Substring(0, buildNumber.IndexOf("-"));
str += "\r\n Build : ";
str += buildNumber;
return str;
}
public static Version get_openvino_version()
{
int l = Marshal.SizeOf(typeof(Version));
IntPtr ptr = Marshal.AllocHGlobal(l);
int status = NativeMethods.ov_get_openvino_version(ptr);
if (status != 0)
{
System.Diagnostics.Debug.WriteLine("ov get_openvino_version() error!");
return new Version();
}
var temp = Marshal.PtrToStructure(ptr, typeof(Version));
Version version = (Version)temp;
string build = String.Copy(version.buildNumber);
string description = String.Copy(version.description);
Version new_version = new Version(build, description);
NativeMethods.ov_version_free(ptr);
return new_version;
}
}
This structure can realize the acquisition of the OpenVINO version, and it can be used directly by calling the corresponding method:
Version v = Version.get_openvino_version();
Console.WriteLine(v.to_string());
4. Add Core
In the third step, we have obtained the version of OpenVINO Runtime through Version. Next, we obtain the version of the specified device information through Core, which mainly calls the ov_core_get_versions_by_device_nameinterface implementation. The specific implementation process will not be described here.
public struct CoreVersion
{
public string device_name;
public Version version;
}
public struct CoreVersionList
{
public IntPtr core_version;
public ulong size;
}
public class Core
{
private IntPtr ptr = IntPtr.Zero;
public IntPtr Ptr { get { return ptr; } set { ptr = value; } }
public Core(string xml_config_file = null)
{
int status;
status = NativeMethods.ov_core_create(ref ptr);
if (status != 0) {
ptr = IntPtr.Zero;
}
}
~Core() { dispose(); }
public void dispose()
{
if (ptr == IntPtr.Zero)
{
return;
}
NativeMethods.ov_core_free(ptr);
ptr = IntPtr.Zero;
}
public KeyValuePair<string, Version> get_versions(string device_name)
{
int status;
int l = Marshal.SizeOf(typeof(CoreVersionList));
IntPtr ptr_core_version_s = Marshal.AllocHGlobal(l);
sbyte[] c_device_name = (sbyte[])((Array)System.Text.Encoding.Default.GetBytes(device_name));
status = NativeMethods.ov_core_get_versions_by_device_name(ptr, ref c_device_name[0], ptr_core_version_s);
if (status != 0)
{
System.Diagnostics.Debug.WriteLine("Core get_versions() error : " + status.ToString());
return new KeyValuePair<string, Version>();
}
var temp1 = Marshal.PtrToStructure(ptr_core_version_s, typeof(CoreVersionList));
CoreVersionList core_version_s = (CoreVersionList)temp1;
var temp2 = Marshal.PtrToStructure(core_version_s.core_version, typeof(CoreVersion));
CoreVersion core_version = (CoreVersion)temp2;
KeyValuePair<string, Version> value = new KeyValuePair<string, Version>(core_version.device_name, core_version.version);
status = NativeMethods.ov_core_versions_free(ptr_core_version_s);
if (status != 0)
{
System.Diagnostics.Debug.WriteLine("Core get_versions() error : " + status.ToString());
return new KeyValuePair<string, Version>();
}
return value;
}
}
This structure can obtain the version information of OpenVINO by specifying the device name, and directly call the corresponding method when using it:
Core core = new Core();
KeyValuePair<string, Version> vs = core.get_versions("CPU");
Console.WriteLine(vs.Key);
Console.WriteLine(vs.Value.to_string());
vs = core.get_versions("GPU.0");
Console.WriteLine(vs.Key);
Console.WriteLine(vs.Value.to_string());
core.dispose();
The following figure shows the two methods we defined above to obtain the version information of OpenVINO:
Finally, list all the codes in this project, which is convenient for everyone to compile and implement. Since this project is only a test project, there is no division of files. All codes in this project are stored in the Program.cs file .
using System;
using System.Runtime.InteropServices;
namespace test_openvinosharp
{
internal class Program{
static void Main(string[] args)
{
Version v = Version.get_openvino_version();
Console.WriteLine(v.to_string());
Core core = new Core();
KeyValuePair<string, Version> vs = core.get_versions("CPU");
Console.WriteLine(vs.Key);
Console.WriteLine(vs.Value.to_string());
vs = core.get_versions("GPU.0");
Console.WriteLine(vs.Key);
Console.WriteLine(vs.Value.to_string());
core.dispose();
}
}
class NativeMethods{
const string dll_extern = "libopenvino_c.so";
[DllImport(dll_extern, EntryPoint = "ov_get_openvino_version", CharSet = CharSet.Unicode, CallingConvention = CallingConvention.Cdecl)]
public extern static int ov_get_openvino_version(IntPtr version);
[DllImport(dll_extern, EntryPoint = "ov_version_free", CharSet = CharSet.Unicode, CallingConvention = CallingConvention.Cdecl)]
public extern static int ov_version_free(IntPtr version);
[DllImport(dll_extern, EntryPoint = "ov_core_create", CharSet = CharSet.Unicode, CallingConvention = CallingConvention.Cdecl)]
public extern static int ov_core_create(ref IntPtr core);
[DllImport(dll_extern, EntryPoint = "ov_core_free", CharSet = CharSet.Unicode, CallingConvention = CallingConvention.Cdecl)]
public extern static int ov_core_free(IntPtr core);
[DllImport(dll_extern, EntryPoint = "ov_core_get_versions_by_device_name", CharSet = CharSet.Unicode, CallingConvention = CallingConvention.Cdecl)]
public extern static int ov_core_get_versions_by_device_name(IntPtr core, ref sbyte device_name, IntPtr versions);
[DllImport(dll_extern, EntryPoint = "ov_core_versions_free", CharSet = CharSet.Unicode, CallingConvention = CallingConvention.Cdecl)]
public extern static int ov_core_versions_free(IntPtr versions);
}
public struct Version
{
public string buildNumber;
public string description;
public Version(string buildNumber, string description) {
this.buildNumber = buildNumber;
this.description = description;
}
public string to_string()
{
string str = "";
str += description;
str += "\r\n Version : ";
str += buildNumber.Substring(0, buildNumber.IndexOf("-"));
str += "\r\n Build : ";
str += buildNumber;
return str;
}
public static Version get_openvino_version()
{
int l = Marshal.SizeOf(typeof(Version));
IntPtr ptr = Marshal.AllocHGlobal(l);
int status = NativeMethods.ov_get_openvino_version(ptr);
if (status != 0)
{
System.Diagnostics.Debug.WriteLine("ov get_openvino_version() error!");
return new Version();
}
var temp = Marshal.PtrToStructure(ptr, typeof(Version));
Version version = (Version)temp;
string build = String.Copy(version.buildNumber);
string description = String.Copy(version.description);
Version new_version = new Version(build, description);
NativeMethods.ov_version_free(ptr);
return new_version;
}
}
public struct CoreVersion
{
public string device_name;
public Version version;
}
public struct CoreVersionList
{
public IntPtr core_version;
public ulong size;
}
public class Core
{
private IntPtr ptr = IntPtr.Zero;
public IntPtr Ptr { get { return ptr; } set { ptr = value; } }
public Core(string xml_config_file = null)
{
int status;
status = NativeMethods.ov_core_create(ref ptr);
if (status != 0) {
ptr = IntPtr.Zero;
}
}
~Core() { dispose(); }
public void dispose()
{
if (ptr == IntPtr.Zero)
{
return;
}
NativeMethods.ov_core_free(ptr);
ptr = IntPtr.Zero;
}
public KeyValuePair<string, Version> get_versions(string device_name)
{
int status;
int l = Marshal.SizeOf(typeof(CoreVersionList));
IntPtr ptr_core_version_s = Marshal.AllocHGlobal(l);
sbyte[] c_device_name = (sbyte[])((Array)System.Text.Encoding.Default.GetBytes(device_name));
status = NativeMethods.ov_core_get_versions_by_device_name(ptr, ref c_device_name[0], ptr_core_version_s);
if (status != 0)
{
System.Diagnostics.Debug.WriteLine("Core get_versions() error : " + status.ToString());
return new KeyValuePair<string, Version>();
}
var temp1 = Marshal.PtrToStructure(ptr_core_version_s, typeof(CoreVersionList));
CoreVersionList core_version_s = (CoreVersionList)temp1;
var temp2 = Marshal.PtrToStructure(core_version_s.core_version, typeof(CoreVersion));
CoreVersion core_version = (CoreVersion)temp2;
KeyValuePair<string, Version> value = new KeyValuePair<string, Version>(core_version.device_name, core_version.version);
status = NativeMethods.ov_core_versions_free(ptr_core_version_s);
if (status != 0)
{
System.Diagnostics.Debug.WriteLine("Core get_versions() error : " + status.ToString());
return new KeyValuePair<string, Version>();
}
return value;
}
}
}
V. Summary
The above is the entire process of using OpenVinoSharp for the Aix development board. In this project, we only conducted a feasibility test for the Aix development board using OpenVinoSharp, and have not used the model for experiments. In the future, I will continue to carry out tests to verify the performance of the Aix development board, so stay tuned.
The project has been implemented in Window, and a NuGet package has been made, which can be used through one-click installation. If you are interested, you are welcome to visit my GitHub OpenVinoSharp to get the source code of the project. If you are interested in this work, you are also welcome to join the project In development.