Article directory
development environment
- win10 professional version
- Unity2019.4
- VS2017 ( Be sure to use this version of VS! )
- Hololens2 (turn on developer mode)
- win10 SDK 10.0.19041.0
1 Introduction
Project source: YoloDetectionHoloLens-Unity
This is a project using Hololens for target detection. The Hololens side is responsible for collection and display, and the computer side processes the image and annotates it, and sends the annotation frame back to the Holo side for display.
The specific project release process is introduced in the project. This article mainly records the specific steps that the author did not mention, the errors I encountered during the release process, and the solutions.
The general release process is as follows:
1. "HoloLensForCV"Generate the project solution and export ".dll"文件和".winmd"the file to the Unity project
2. Deploy the unity project to Hololens2
3. "ComputeOnDesktopYolo"Deploy the project on your own computer and generate ComputeOnDesktopYoloan executable program named
4. Place Holo and the computer in Under the same LAN, start the Holo program and enter the IP connection on the computer.
2.Debug records
2.1 Project download
Download the compressed package of the entire project from Github (do not use clone method). After downloading, right-click the compressed package>Properties>check Unlock
After decompression, I found that HololensForCVthis folder is empty, because the contents of this folder need to be downloaded separately.
Click to enter the HololensForCV project file shown below
Use the same method to download and decompress ( unlock it as well ). It is best to decompress HololensForCV in the location shown in the figure below according to its original project file directory, so as to avoid unnecessary trouble caused by the path and the entire project
. Just download it.
There is a point to note here. Once an error occurs during previous operations such as deploying the project,Re-unzipThe contents of the downloaded two packages and compressed packages will not change due to misoperation, version changes, etc. If it can't be solved, just reopen it. Unzip it again and delete the previously unzipped one.
2.2 Generate HololensForCV solution
It is written in the project introduction that it can be used Prebuilt文件夹, 预先生成的dll文件和winmd文件but for the sake of safety, I still generate it myself. And even if pre-generated files are used here, you will inevitably have to deploy other projects yourself later, so it is recommended to generate the files yourself .
Open the file under (the following string of characters is omitted, the same below) with VS2017 .HololensForCV文件夹HololensForCV.sln
Note that you must use VS2017 here. I just started using the 2019 version. Various errors occurred when generating the solution, and I couldn't find a solution. (WindowsSDK can use higher versions, but you must use VS2017.)
After opening the file, first make 重定结局方案目标it compatible with your own SDK version.
Next, when you open it 工具>Nuget包管理器>管理解决方案的nuget包
, you will find that there are two packages .
We need to install another one. OpenCV.Hololens包For three projects, they are under sample HololensForCV, Shared ComputeOnDesktopYoloand OpenCVHelpers
There are two methods:
Method 1:
You You can install the nuget package according to the method provided in the project. Open the nuget package console and enter the following command. You need to replace the address in the command with the absolute address of the file. The nuget package is under the main project file. Method 2:
Here 
I
provide Another installation method to avoid some people making mistakes due to paths.
Add a local nuget package source as shown below, change the address to the downloaded project address, click
Browse, switch the package to the source just added,
check the three projects just mentioned on the right, and click install (I have installed it before) After the installation is successful, the version number will appear in the "Version" column)
and OpenCV.Hololens nuget包the installation will be successful.
Set the HololensForCV project as the startup project. Select release ARM (if you have 1st generation glasses, select X86 here) to generate the project, and then generate the file
Several key points: Version VS2017, reorder the solution, import the nuget package (two methods), and generate the solution
2.3Unity project deployment
Copy all the generated files to the YoloDetectionHoloLensUnity project's folder mentioned by the author Assets->Plugins->x86/ARM 文件夹.
If you only copy the dll and winmd files here as the author said, Unity will report an error that the namespace is missing, so you need to copy all these generated files to solve the problem.
Then open the Unity project and convert the platform to UWP.
Here you need to make some modifications to the imported files.
This is not mentioned in the author's article
. As shown in the picture, change both of the files to the state on the right, and then click Apply ( If you are a generation glasses, replace ARM with X86).
Then open the scene
and replace the IP in this script with the IP of your computer on the same LAN. Check as follows:
If you build directly, there will be a certificate expiration error.
Open player setting>Other Settings >Certificate
and click on the certificate shown in the picture above to cancel the certificate mounting of the expired certificate.
Then perform Build to generate the solution and deploy it to Holo (the deployment process is not explained in detail)
Several key points: Platform conversion, dll and winmd file settings, IP address modification, certificate expiration.
2.4 Deploy computer-side applications
Open HololensForCV.slnthe file and set ComputeOnDesktopYolothe project as the startup project
Because the nuget package has been imported to this project before, there is no need to import it again here, just deploy it directly.
Set it up like the picture above Release或者Debug( x64the computer is 64-bit). Then deploy.
Warnings about converting float to int type may appear during the deployment process and can be ignored.
After deployment, the computer will generate an application.
Afterwards, you can turn off Unity, VS, etc., without leaving them hanging.
Open the application and there will be an interface like this.
This means the deployment is successful.
2.5 Debugging
First open the Holo application, and then follow the text prompts above to perform the double tap gesture. In the middle, you will be prompted to apply for camera permission.
Then enter Holo’s IP address into the computer program and click connect to receive the video from Holo.
Follow Holo's prompts and perform the double AirTap gesture, and Holo can display the callout box. Because the camera is occupied, the result pictures are not shown here.