Azure Kinect DK + ROS1 Noetic tutorial

News 2023-09-09 03:22:09 views: null

Author: Herman Ye @Galbot @Auromix
Version: V1.0
Test environment: Ubuntu20.04
Update date: 2023/08/16
Note 1 : The hardware in this article is supported by @Galbot.
Note 2 : @Auromix is ​​an open source organization for robotics enthusiasts.
Note 3 : This article was tested and confirmed valid as of the update date.

One-click installation script

You can refer to the one-click installation script and perform the following steps to install Azure Kinect DK on Ubuntu18/20:

wget -O $HOME/azure_kinect_dk_install.sh https://raw.githubusercontent.com/auromix/ros-install-one-click/main/azure_kinect_dk_install.sh && sudo chmod +x $HOME/azure_kinect_dk_install.sh && bash $HOME/azure_kinect_dk_install.sh

Note: Ubuntu22.04 has not been tested, you can also try it.

Azure Kinect DK usage tutorial

Azure Kinect DK is a developer toolkit equipped with advanced AI sensors that provide sophisticated computer vision and speech models. Kinect combines a depth sensor, a spatial microphone array with a video camera, and an orientation sensor into a small, all-in-one device that offers a variety of modes, options, and software development kits (SDKs).

Insert image description here

DK hardware

  • The RGB camera
    Azure Kinect DK contains the OV12A10 12MP CMOS rolling shutter sensor.
  • Depth Camera
    Azure Kinect DK integrates a 1-megapixel time-of-flight (ToF) depth camera designed by Microsoft.
  • The IMU
    embedded inertial measurement unit (IMU) is LSM6DSMUS, which includes an acceleration sensor and a gyroscope. The accelerometer and gyroscope are sampled simultaneously at 1.6 kHz. Samples are reported to the host at 208 Hz and can be read via the SDK.
  • Seven-microphone array
    Regular hexagon x6+ center x1 Premium seven-microphone circular array for standard USB Audio Class 2.0 devices

Insert image description here

DK power supply and data transmission

Method 1. Split input lines for power and data

DK->Type-C转Type-A -> PC
DK->DC5V转Type-A-> PC

Method 2. Power and data merge line

DK->Type-C转Type-C-> PC
Insert image description here

1. Install SDK related software packages

Warning: For Ubuntu20.04users, please refer directly 安装软件包-> 5.安装k4a-tools->解决方法and do not perform the following steps.

1. Check the system release version and version number

lsb_release -a

The result is similar to the following:

galbot@ubuntu:~$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.6 LTS
Release:	20.04
Codename:	focal

2. Configure Microsoft package source

Download Microsoft's package configuration file and write it to /etc/apt/sources.list.d/microsoft-prod.lista file to configure the system to obtain software updates or install packages from Microsoft's software sources.

curl -sSL https://packages.microsoft.com/config/<distribution>/<version>/prod.list | sudo tee /etc/apt/sources.list.d/microsoft-prod.list

Replace <distribution>and <version>with Distributor ID: Ubuntuand Release:20.04

Examples are as follows:

curl -sSL https://packages.microsoft.com/config/ubuntu/20.04/prod.list | sudo tee /etc/apt/sources.list.d/microsoft-prod.list

3. Configure the Microsoft repository GPG public key

Download public key

Download the GPG public key of the Microsoft repository, and then write it to the trusted GPG directory of the system so that the public key can be used for verification during the subsequent software package verification process to ensure the security and authenticity of the downloaded software package.
/etc/apt/trusted.gpg.d/microsoft.asc: is the target file to which data is to be written. As a file in the system's trusted GPG directory, it stores the trusted GPG public key.

curl -sSL https://packages.microsoft.com/keys/microsoft.asc | sudo tee /etc/apt/trusted.gpg.d/microsoft.asc

Register, install, and delete authentication-related software packages

Note: You need to download it first packages-microsoft-prod.deb , it may be a package related to Microsoft security certification, which is not officially mentioned.

wget https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.deb

Install Microsoft security certification related packages

sudo dpkg -i packages-microsoft-prod.deb

Delete the installation package files after installation to free up disk space

rm packages-microsoft-prod.deb

4. Update package index

sudo apt update

Now, Microsoft dependent product packages can be sudo apt install <microsoft_package_name>installed correctly.

5.Install k4a-tools

k4a-toolsPackage includes Azure Kinect Viewer, Azure Kinect Recorder, and Azure Kinect Firmware Tools

sudo apt install k4a-tools -y

Report an error

galbot@ubuntu:~$ sudo apt install k4a-tools
Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package k4a-tools

Related Issues raised by other users: Unable to install k4a-tools in Ubuntu 20.04
This issue was raised on August 24, 2020, and has not been resolved on August 8, 2023. The reason is that the SDK package was only available on August 8, 2023 . Support Ubuntu18.04 .

According to the solution in the Issue proposed by another user , the user proposed the method on October 17, 2020, but the specific method is partially outdated and unavailable. By referring to the user's ideas, the solution is re-proposed as follows.

Solution

For Ubuntu20.04 users, the solution and comments are as follows:

Clear old configuration

If unfortunately you have configured it according to the Ubuntu20.04 SDK installation method officially provided by Microsoft, you need to clean up related resources first.

  • Delete the content about Microsoft in sources.list
# Delete microsoft related line
sudo nano /etc/apt/sources.list
  • remove-packages-microsoft-prod
# remove packages-microsoft-prod
sudo apt remove packages-microsoft-prod
  • Delete the content about Microsoft in sources.list.d
# remove old 20.04 microsoft-prob.list
sudo rm /etc/apt/sources.list.d/microsoft-prod.list
sudo rm /etc/apt/sources.list.d/microsoft-prod.list.save

Execute new configuration

If you have cleared the old configuration or jumped directly to this step due to a warning prompt, follow the installation instructions below:

By simulating the 18.04 software download source, Ubuntu 20.04 is tricked into using 18.04 files, and an older version is specified to avoid problems such as the latest sensor dependency affecting basic use. For example, the Azure Kinect ROS driver requires the v1.3.0 version of Azure Kinect. Sensor SDK can be used to compile.

# Download key and add to system key list
curl -sSL https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -

# Add Microsoft Ubuntu18.04 packages list to source
curl -sSL https://packages.microsoft.com/config/ubuntu/18.04/prod.list | sudo tee /etc/apt/sources.list.d/microsoft-prod.list

# Update software source
sudo apt update

# Upgrade software
sudo apt upgrade

# Install package of Microsoft
sudo apt install libk4a1.3-dev -y
sudo apt install libk4abt1.0-dev -y
sudo apt install k4a-tools=1.3.0 -y

Note: Another feasible way is to manually download the Microsoft source files directly and perform deb manual installation. Although the Microsoft server 23.99.120.248is in Yadong, it seems that the speed of using Magic is still very slow. The reason is not clear at the moment. If the download time is too long, it is recommended to do it manually and choose an old version of the software.

Note: You can use apt-mark to prevent the corresponding software package from being updated to avoid problems caused by the new version not supporting ubuntu20.04 during the update.

sudo apt-mark hold <app_name>

The files that need to be downloaded are as follows:
libk4a1.3-dev
libk4abt1.0-dev
k4a-tools

For ARM architecture devices
libk4abt1.1-dev
libk4a1.4-dev
k4a-tools
libk4a1.4-dev ARM

2. Set Udev rules

  • Create new udev rule
sudo nano /etc/udev/rules.d/99-k4a.rules
  • Write udev rule content
# Bus 002 Device 116: ID 045e:097a Microsoft Corp.  - Generic Superspeed USB Hub
# Bus 001 Device 015: ID 045e:097b Microsoft Corp.  - Generic USB Hub
# Bus 002 Device 118: ID 045e:097c Microsoft Corp.  - Azure Kinect Depth Camera
# Bus 002 Device 117: ID 045e:097d Microsoft Corp.  - Azure Kinect 4K Camera
# Bus 001 Device 016: ID 045e:097e Microsoft Corp.  - Azure Kinect Microphone Array

BUS!="usb", ACTION!="add", SUBSYSTEM!=="usb_device", GOTO="k4a_logic_rules_end"

ATTRS{
    
    idVendor}=="045e", ATTRS{
    
    idProduct}=="097a", MODE="0666", GROUP="plugdev"
ATTRS{
    
    idVendor}=="045e", ATTRS{
    
    idProduct}=="097b", MODE="0666", GROUP="plugdev"
ATTRS{
    
    idVendor}=="045e", ATTRS{
    
    idProduct}=="097c", MODE="0666", GROUP="plugdev"
ATTRS{
    
    idVendor}=="045e", ATTRS{
    
    idProduct}=="097d", MODE="0666", GROUP="plugdev"
ATTRS{
    
    idVendor}=="045e", ATTRS{
    
    idProduct}=="097e", MODE="0666", GROUP="plugdev"

LABEL="k4a_logic_rules_end"
  • Restart udev service
sudo service udev restart
  • Set user group permissions
sudo usermod -aG plugdev $USER
  • Reboot the device

Unplug the device USB cable and plug it in again.

3. SDK basic testing

Use the official k4aviewer to confirm that the infrared image, depth image, color image, microphone array, and IMU are functioning properly.

# Open a new terminal
k4aviewer

Insert image description here

4. DK ROS basic test

Note: It ROS1 Noetichas been installed and configured correctly by default. If ROS1 has not been installed yet, please refer to the ROS1 Noetic one-click installation script.

  • Download, dependency installation, compilation, configuration
mkdir -p galbot_ws/src
cd galbot_ws/src
git clone https://github.com/microsoft/Azure_Kinect_ROS_Driver.git
cd ..
rosdep install --from-paths src --ignore-src -r -y
catkin_make
echo "source ~/galbot_ws/devel/setup.bash" >> ~/.bashrc
source ~/.bashrc
  • run
# Terminal 1
roslaunch azure_kinect_ros_driver driver.launch

# Terminal 2
rviz

Add DK publishable topic data and visualize it in RViz as follows:
Insert image description here

5. Important flaws

  • SDK does not support Ubuntu20.04 and Ubuntu22.04 by default
  • Azure Kinect ROS Driver does not support its own microphone array
  • Azure Kinect ROS Driver officially only supports ROS1 Melodic by default.
  • Using a USB docking station instead of a direct plug may cause ROS startup problems

6. References

official

DK official docs
DK official docs Chinese
Azure Kinect Sensor SDK Github
Sensor SDK Official Download
Sensor SDK API Document
Sensor SDK Sample Code
Sensor SDK Sample Code 2
Update DK Firmware
SDK Installation Document and Corresponding Version Firmware Download
Azure Kinect ROS Driver
Microphone Usage Reference
Voice Service Case
DK Hardware Specification
Microphone Hardware Specification Part

third party

image_proc
setup-azure-kinect-on-jetson-x-nx_18.04
Azure Kinect SDK and ROS driver installation
Azure Kinect DK Python package

Guess you like

Origin blog.csdn.net/m0_56661101/article/details/132160408
Recommended
Ranking
Goldman: Microsoft is winning the war cloud services
Change admob ids on existing apps
【C++】Introduction to Object-Oriented Programming③ (Process-Oriented Programming Structured Programming Method | Structured Programming Method Concept / Features / Advantages and Disadvantages | Object-Oriented Programming Introduction )
2021-02-26 Luogu P1966 digging landmines-dfs violent search (backtracking method)
jqgrid seleccione todo cancelar una sola fila haga clic en cancelar evento
InstructTTS: Modeling Expressive TTS in Discrete Latent Space with Natural Language Style Prompt
hdu1312 “Red and Black”——DFS与递归的应用
【Linux】孤儿进程、僵死进程和信号
OCRアーティファクト、PDF、数式を変換可能
Linux kernel Tcp performance tuning
Daily
More
2024-06-04(0)
2024-06-03(1)
2024-06-02(0)
2024-06-01(1)
2024-05-31(1)
2024-05-30(0)
2024-05-29(1)
2024-05-28(1)
2024-05-27(1)
2024-05-26(0)

Code World

© 2018 codetd.com