Author: Huang Mingming, Intel Edge Computing Innovation Ambassador
The Intel® Distribution OpenVINO™ tool suite [1] is developed based on oneAPI and can accelerate the development of high-performance computer vision and deep learning vision applications. The tool suite is suitable for various Intel platforms from edge to cloud. Help users deploy more accurate real-world results into production systems faster. Through streamlined development workflows, OpenVINO™ empowers developers to deploy high-performance applications and algorithms in the real world.
Java is an object-oriented programming language. It not only absorbs the various advantages of the C++ language, but also abandons the difficult-to-understand concepts such as multiple inheritance and pointers in C++. Therefore, the Java language is powerful and simplicity of use. As a representative of static object-oriented programming languages, Java language perfectly implements object-oriented theory and allows programmers to perform complex programming with an elegant way of thinking.
Although OpenVINO™ provides the Ubuntu version of the API in [OpenVINO Contrilb](https://github.com/openvinotoolkit/openvino_contrib/tree/master/modules/java_api), due to the use of JNI technology , which is not particularly friendly to developers who are not involved in C/C++ programming, and later maintenance and updates also bring a lot of trouble.
In previous work, we launched the OpenVINO™ Java API[2], aiming to promote the application of OpenVINO™ in the Java field. It has been successfully used on Mac, Windows, and Linux platforms. In this article, we will introduce how to implement OpenVINO™ Java API on Linux based on Intel Development Kit AlxBoard.
The code used in the project has been uploaded to the OpenVINO™ Java API repository, and the GitHub URL is:
https://github.com/Hmm466/OpenVINO- Java-API
1. Introduction to Intel Development Kit AlxBoard
1.1 Product positioning
The Intel Development Kit AlxBoard is a member of the official family of Intel Development Kits, designed for entry-level artificial intelligence applications and edge smart devices. Intel's development kit AlxBoard can perfectly handle different application scenarios such as artificial intelligence learning, development, training, and application. The kit comes pre-installed with the Intel OpenVINO™ tool suite, model repository and demos.
The main interface of the kit is compatible with the Jetson Nano carrier board, and the GPIO is compatible with the Raspberry Pi, which can maximize the reuse of mature ecological resources. This enables the suite to serve as an edge computing engine to provide strong support for artificial intelligence product verification and development; at the same time, it can also serve as a domain control core to provide technical support for robot product development.
Using the Intel development kit AlxBoard, you will be able to build a stunning artificial intelligence application in a short time. Whether you're using it for research, education, or business, the Intel Development Kit AlxBoard has you covered. With the OpenVINO™ tool suite, both CPU and iGPU have powerful AI inference capabilities, supporting the parallel running of multiple neural networks in applications such as image classification, object detection, segmentation and speech processing.
1.2 Product parameters
1.3 AI reasoning unit
With the OpenVINO™ tool, CPU+iGPU heterogeneous computing inference can be realized, and the IGPU computing power is about 0.6TOPS
2. Preparation
2.1 Configure java environment
Download and configure JDK:
JDK (Java Development Kit) is called Java development kit or Java development tool. It is a program development environment for writing Java Applet applets and applications. . JDK is the core of the entire Java, including the Java Runtime Environment (Java Runtime Environment), some Java tools and Java's core class library (Java API). No matter what Java application server is, it essentially has a certain version of JDK built-in. The mainstream JDK is the JDK released by Sun. In addition to Sun, there are many companies and organizations that have developed their own JDK.
2.1.1 Add api to local maven
Add OpenVINO™ Java API to Maven (currently not published in the Meven central repository, so manual installation is required)
2.1.2 Clone OpenVINO™ Java API project to local
git clone https://github.com/Hmm466/OpenVINO-Java-API2.1.3 Open through IDEA or Eclipse
Through maven install to the local maven library
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 14.647 s
[INFO] Finished at: 2023-11-02T21:34:49+08:00
[INFO] ------------------------------------------------------------------------
The jar package will be placed in
/{userHome}/.m2/repository/org/openvino/java-api/1.0-SNAPSHOT/java-api-1.0-SNAPSHOT.pom
2.2 Install OpenVINO™ Runtime
OpenVINO™ has two installation methods: OpenVINO™ Runtime and OpenVINO™ Development Tools. OpenVINO™ Runtime contains core libraries for running model deployment inference on processor devices. OpenVINO™ Development Tools is a set of tools for working with OpenVINO™ and OpenVINO™ models, including a model optimizer, OpenVINO™ Runtime, model downloader, and more. Here we just need to install OpenVINO™ Runtime.
2.2.1 Download OpenVINO™ Runtime
Visit the Download the Intel Distribution of OpenVINO™ Toolkit[5] page and select the corresponding installation option according to the following process. On the download page, since AIxBoard uses Ubuntu20.04, you can download it according to the specified compiled version.
2.2.2 Unzip the installation package
The OpenVINO™ Runtime we downloaded is essentially a C++ dependency package, so we put it in our system directory so that the dependencies will be obtained according to the set system variables during compilation.
cd ~/Downloads/
tar -xvzf l_openvino_toolkit_ubuntu20_2022.3.1.9227.cf2c7da5689_x86_64.tgz
sudo mv l_openvino_toolkit_ubuntu20_2022.3.1.9227.cf2c7da5689_x86_64/runtime/lib/intel64/* /usr/lib/
2.3 Compile OpenCV java library
2.3.1 Download ANT
Since OpenCV compiled libopencv_java{version}.[so|dll|dylib] requires the support of apache ant, you need to manually download ant and add environment variables
Official website: ant.apache .org
export ANT_HOME={ant_home}
export PATH=$ANT_HOME/bin:$PATH
2.3.2 OpenCV download source code
opencv official websiteDownload source code
After unzipping, go to the folder
mkdir build
cd build
cmake -DBUILD_SHARED_LIBS=OFF -DWITH_IPP=OFF -DBUILD_ZLIB=OFF -DCMAKE_INSTALL_PREFIX=你的opencv目录
-DJAVA_INCLUDE_PATH={jdk 所在位置}/include -DJAVA_AWT_INCLUDE_PATH={jdk 所在位置}/include
-DJAVA_INCLUDE_PATH2={jdk 所在位置}/include -DBUILD_JAVA=ON ../
Pay attention to see if there is any output
-- Java:
-- ant:
-- JNI:
-- Java wrappers:
-- Java tests:
Needs to be NO or have a directory
Then compile and install
make -j 8
make install3. Test on AlxBoard
3.1 Direct testing of source code
git clone https://github.com/Hmm466/OpenVINO-Java-API- Open the project using IDEA / Eclipse
- Run src/test/java/org.openvino.java.test.YoloV8Test
3.2 Create other projects for testing
Create an AlxBoardDeployYolov8 Maven project
After the creation is completed, reference the OpenVINO-Java-API we just installed, or directly clone the project to directly modify the experience
maven reference
<dependency>
<groupId>org.openvino</groupId>
<artifactId>java-api</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>[Note: If you only use maven dependencies, you need to pay attention to the opencv library reference problem. You can reference the opencv library of OpenVINO-Java-API/libs to your project]
Write test code:
OpenVINO vino = OpenVINO.load();
OvVersion version = vino.getVersion();
Console.println("---- OpenVINO INFO----");
Console.println("Description : %s", version.description);
Console.println("Build number: %s", version.buildNumber);The result will be output:
--- OpenVINO INFO----
Description : OpenVINO Runtime
Build number: 2023.2.0-12538-e7c1344d3c3the
Description : OpenVINO Runtime
Build number: 2023.2.0-12538-e7c1344d3c3
[INFO] Loading model files: model/yolov8/yolov8s.xml
[INFO] model name: torch_jit
[INFO] inputs:
[INFO] input name: images
[INFO] input type: Node
[INFO] input shape: Shape{, rank=4, dims=1,3,640,640}
[INFO] outputs:
[INFO] output name: output0
[INFO] output type: Node
[INFO] output shape: Shape{, rank=3, dims=1,84,8400}
[INFO] Read image files: dataset/image/demo_2.jpg
Detection result :
1: 0 0.88585377 {744, 43, 388x667}
2: 0 0.87911075 {149, 202, 954x507}
3: 27 0.7225534 {435, 433, 98x284}
themselves
---- OpenVINO INFO----
Description : OpenVINO Runtime
Build number: 2023.2.0-12538-e7c1344d3c3
[INFO] Loading model files: model/yolov8/yolov8s-seg.xml
[INFO] model name: torch_jit
[INFO] inputs:
[INFO] input name: images
[INFO] input type: Node
[INFO] input shape: Shape{, rank=4, dims=1,3,640,640}
[INFO] outputs:
[INFO] output name: output0
[INFO] output type: Node
[INFO] output shape: Shape{, rank=3, dims=1,116,8400}
[INFO] Read image files: dataset/image/demo_2.jpg
Segmentation result :
1: 0 0.8988322 {745, 41, 402x671}
2: 0 0.8630947 {118, 196, 1011x515}
3: 27 0.6970458 {434, 436, 90x280}
pose
---- OpenVINO INFO----
Description : OpenVINO Runtime
Build number: 2023.2.0-12538-e7c1344d3c3
[INFO] Loading model files: model/yolov8/yolov8s.xml
[INFO] model name: torch_jit
[INFO] inputs:
[INFO] input name: images
[INFO] input type: Node
[INFO] input shape: Shape{, rank=4, dims=1,3,640,640}
[INFO] outputs:
[INFO] output name: output0
[INFO] output type: Node
[INFO] output shape: Shape{, rank=3, dims=1,84,8400}
[INFO] Read image files: dataset/image/demo_2.jpg
Classification result :
1: 1 0.88585377 {744, 43, 388x667} Nose: (0.0 ,0.0 ,4.446762E-6) Left Eye: (0.0 ,0.0 ,5.1098637E-6) Right Eye: (0.0 ,0.0 ,2.5492784E-6) Left Ear: (0.0 ,0.0 ,3.8548264E-6) Right Ear: (0.0 ,0.0 ,5.493899E-6) Left Shoulder: (0.0 ,0.0 ,2.8905094E-6) Right Shoulder: (0.0 ,0.0 ,4.185275E-6) Left Elbow: (0.0 ,0.0 ,4.1954772E-6) Right Elbow: (0.0 ,0.0 ,3.3010822E-6) Left Wrist: (0.0 ,0.0 ,9.217298E-6) Right Wrist: (0.0 ,0.0 ,5.601134E-6) Left Hip: (0.0 ,0.0 ,2.5226416E-6) Right Hip: (0.0 ,0.0 ,4.9324813E-6) Left Knee: (0.0 ,0.0 ,5.6728954E-6) Right Knee: (0.0 ,0.0 ,3.1320922E-6) Left Ankle: (0.0 ,0.0 ,7.081372E-7) Right Ankle: (0.0 ,0.0 ,4.169948E-6)
2: 1 0.87911075 {149, 202, 954x507} Nose: (0.0 ,0.0 ,2.6389546E-6) Left Eye: (0.0 ,0.0 ,2.7424064E-6) Right Eye: (0.0 ,0.0 ,2.0069933E-6) Left Ear: (0.0 ,0.0 ,2.9333999E-6) Right Ear: (0.0 ,0.0 ,4.0347936E-6) Left Shoulder: (0.0 ,0.0 ,4.200782E-6) Right Shoulder: (0.0 ,0.0 ,2.5432491E-6) Left Elbow: (0.0 ,0.0 ,1.7569737E-6) Right Elbow: (0.0 ,0.0 ,1.9186296E-6) Left Wrist: (0.0 ,0.0 ,3.1343661E-6) Right Wrist: (0.0 ,0.0 ,2.889939E-6) Left Hip: (0.0 ,0.0 ,1.7592807E-6) Right Hip: (0.0 ,0.0 ,1.5011567E-6) Left Knee: (0.0 ,0.0 ,3.6549225E-6) Right Knee: (0.0 ,0.0 ,1.9932615E-6) Left Ankle: (0.0 ,0.0 ,1.012553E-6) Right Ankle: (0.0 ,0.0 ,8.3442E-6) 
cls
---- OpenVINO INFO----
Description : OpenVINO Runtime
Build number: 2023.2.0-12538-e7c1344d3c3
[INFO] Loading model files: model/yolov8/yolov8s.xml
[INFO] model name: torch_jit
[INFO] inputs:
[INFO] input name: images
[INFO] input type: Node
[INFO] input shape: Shape{, rank=4, dims=1,3,640,640}
[INFO] outputs:
[INFO] output name: output0
[INFO] output type: Node
[INFO] output shape: Shape{, rank=3, dims=1,84,8400}
[INFO] Read image files: dataset/image/demo_2.jpg
Classification Top 10 result :
classid probability
------- -----------
{14789} {635.549438}
{3679} {635.543701}
{14788} {635.522583}
{14731} {635.518616}
{14730} {635.513428}
{3839} {635.502441}
{14790} {635.497314}
{14732} {635.489258}
{14781} {635.486694}
{14739} {635.484985}4. Summary
In this project, we implemented Java on the Ubuntu 22.04 system based on AlxBoard and successfully used the OpenVINO™ Java API on the hardware basis, and successfully allowed the Yolov8 model, verified the feasibility of Java running, and simplified Java How difficult it is for developers to get started with AI projects.
At the same time, the OpenVINO™ Java API has completed testing on Mac, Linux, and Windows, and the documentation for the Windows platform is also being output. In the future, I will continue to use the OpenVINO™ Java API to deploy more deep learning models on the Intel Development Kit AlxBoard.