Jetson has the most complete jetpack5.1 installation records, just follow along (including bug error handling)

Table of contents

written in front

background

1 system pre-configuration

1.1 Burning

1.2 Configure environment variables

1.3 use vnc

1.4 pip source change and installation

1.5 pycharm installation

2 pytorch environment construction

2.1 Install miniforge to create a virtual environment

2.2 torch, torchvision installation

2.3 Installation of opencv

2.4 Install tensorrt

2.5 Install torch2trt

3 Camera Processing

3.1csi camera

3.2 usb camera

---

written in front

Jetpack5.1 uses cuda11 and cuda11.4 (the system has two built-in), tensorRT8.5.2, built-in opencv4.5.4. Although there are some bugs, it is a good version overall. As of April 23, tensorrt8.5 is still the latest stable version Version, friends who want to use trt to accelerate should not miss it.

This article will start from scratch, from burning to environment configuration, and will involve the installation of torch, torchvision, tensorRT, torch2trt, opencv-python. 

background

Realize torch-based face detection and recognition on jetson xavier nx and call trt for accelerated reasoning, use csi2 camera, and several sd cards (I also recommend that you back up the environment to a new sd card)

1 system pre-configuration

1.1 Burning

JetPack SDK 5.1 | NVIDIA developers please go to this link to download the sd card image according to the jetson model

Prepare the sd card, if the sd card has already been wiped by the system, please:

Right-click this computer --> Manage --> Disk Management The bottom is the sd card, right-click to delete the largest partition, then right-click to create a new volume, assign a drive letter and you can find it on the computer

SD Memory Card Formatter for Windows/Mac | SD Association (sdcard.org) download sdformatter from this link, open it, select the sd card and format it;

Win32 Disk Imager - Download [Official Site] Install win32diskimager, select the image file downloaded earlier to write

 Insert the written sd card into the jetson, turn it on, and complete the initial setup

1.2 Configure environment variables

#cuda
sudo vi ~/.bashrc
#添加
export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
export CUDA_HOME=$CUDA_HOME:/usr/local/cuda
#保存并刷新
source ~/.bashrc

Note that there are cuda11 and 11.4 in the 5.1 image, you can

export CUDA_HOME=$CUDA_HOME:/usr/local/cuda-ver #ver=11 or 11.4

I don't recommend changing the source here. The domestic source may have a little problem with the support of the new version. If you feel that the running is slow or there is a timeout, please run to the source by yourself.

The jtops that was available in the previous versions does not seem to work in this version. I can’t execute the following code, but it doesn’t matter if it doesn’t exist. Just check the version of what you want to check separately.

sudo -H pip install -U jetson-stats

1.3 use vnc

It is much more convenient to operate with a computer, and you can also copy and paste directly from the computer, so here we use ssh to establish a connection. First connect the computer and jetson with a network cable, and enter arp -a in cmd to find the IP of jetson (usually 192.168.137. xxx format, but not the 192.168.137.255 static full f address)

Device Manager->Network and Sharing Center->Change Adapter Options->wlan Right-click Properties->

Allow the sharing of the port that your jetson network cable accesses, here I am Ethernet 2

install vnc viewer

The vnc software vino-server has been installed in the jetson official image, if not, install it yourself, the installation command is as follows:

sudo apt-get install vino-server

If you don’t have a monitor, you need to use putty to connect to the ip address mentioned earlier, but I suggest that you must have a monitor, so I won’t talk about how to operate without a monitor.

Configure vinoserver to start automatically at boot

mkdir -p ~/.config/autostart
cp /usr/share/applications/vino-server.desktop ~/.config/autostart

make some settings 

gsettings set org.gnome.Vino prompt-enabled false
gsettings set org.gnome.Vino require-encryption false

Then set a password, I suggest that no password is required, if you need:

gsettings set org.gnome.Vino authentication-methods "['vnc']"
gsettings set org.gnome.Vino vnc-password $(echo -n 'password'|base64)
#password是你的密码

then reboot

sudo reboot

Turn on automatic login 

sudo gedit /etc/gdm3/custom.conf

Comment out these two lines of code, but the default should be commented out 

#  AutomaticLoginEnable = true
#  AutomaticLogin = user1

Then adjust the resolution, otherwise it will only be 800*600 (low resolution and low delay, it depends on your needs)

open a file 

sudo gedit /etc/X11/xorg.conf

 Enter this, but 5.1 should already be high resolution by default

Section "Screen"
   Identifier    "Default Screen"
   Monitor       "Configured Monitor"
   Device        "Tegra0"
   SubSection "Display"
       Depth    24
       Virtual 1280 800 # 改为自己需要的分辨率
   EndSubSection
EndSection

Then go back to windows and install vncviewer. Download VNC Viewer | VNC® Connect (realvnc.com)

Open, file->newconnection in the upper left corner

 Enter the previous jetson IP to connect.

Do not restart or start jetson with the cat that switches to yellow in windows!!!!!!!!!!!!!!!!!!!!!

Or anything like that cat that switches between black and yellow!!!!!!!!!!!!!!!!!!

If you want to use it, wait for vnc to establish a connection before opening it

Jetson's wired connection will be broken, and I can't tell how to solve it (I reinstalled jetson's vnc and reset the computer's network), arp -a directly lost jetson's ip.

If you need to transfer files, you can install another mobaxterm, the usage is the same,

 Click the upward arrow on the upper left to upload the file

1.4 pip source change and installation

5.1 does not seem to have pip, if you don't use a virtual environment, install a pip locally

sudo python3 get-pip.py 

Also install packaging

pip3 install packaging

Pip can change the source faster

sudo mkdir .pip
cd .pip

Create a pip.config file and enter

sudo touch pip.conf
sudo gedit pip.conf

  Just write the following content

[global]
timeout = 6000
index-url = https://pypi.doubanio.com/simple
trusted-host = pypi.doubanio.com

1.5 pycharm installation

Download PyCharm: Python IDE for Professional Developers by JetBrains

Download the compressed package for aarch64

Pycharm depends on java, and is divided into openjdk-8-jdk or openjdk-11-jdk according to the version of pycharm you installed

I installed a new version here, so I need openjdk-11-jdk

Although it's a bit stuck, but buy new rather than old, right? 

apt-cache search openjdk

sudo apt-get install openjdk-11-jdk

Next, configure the environment variables, the instructions are as follows:

sudo gedit ~/.bashrc

Then write the following code at the bottom of the configuration file:

export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-arm64
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH

At this point, the configuration of open-jdk is complete, and then unzip pycharm, either through the command or the decompression tool of the graphical interface, and then locate the decompressed folder in the terminal, find the bin folder in the pycharm folder, and cd to bin folder, execute in the bin folder:

$ sh ./pycharm.sh

After execution, ubuntu20.04 used by jetpack5.1 does not support one-click locking of pycharm to the menu, so you need to open pycharm and click the setting in the lower left corner

 Just click create desktop entry to create a shortcut

2 pytorch environment construction

There have been many episodes in the installation of pytorch, here is a summary

First, please install the torch under this link

PyTorch for Jetson - Jetson & Embedded Systems / Jetson Nano - NVIDIA Developer Forums

According to this link, we can use torch1.11 and 1.12. If you need a higher version, the official container of torch2.0.0 is provided:

NVIDIA L4T PyTorch | NVIDIA NGC

After installing torch1.12, I found the mismatch problem of torchvision0.13.0 (yes, he came again, and the version given by github was wrong again):

torchvision 0.13.0 requires requests, which is not installed.

But there is no problem putting torch1.11 and torchvision0.12.0 together. It is recommended to use this

If you need to install a lower version of pytorch on jetson, then you need to install python3.6 and face such a bug:

libmpi.so.20: cannot open shared object file: No such file or directory

Someone on the Internet tried to solve it, but I went to the library website to find this file and couldn’t find it. It should be a problem with python3.6

So use torch1.11 and torchvision0.12.0

If you want to go your own way and want to specify torch and python, here are

Releases · KumaTea/pytorch-aarch64 (github.com)

The whl package created by the individual, but the torch of this package cannot use cuda, that is, torch.cuda.is_available() returns false

The author also gave a solution: pytorch-aarch64/torch.sh at main KumaTea/pytorch-aarch64 (github.com)

Compile your own library for yourself, I have not tried

2.1 Install miniforge to create a virtual environment

Here we still need to use a virtual environment, it is best to save a file after installing a library

conda create -n 环境名 --clone 需要备份的环境名

Install miniforge now: just download the latest miniforge-pypy3-linux-aarch64

Releases · conda-forge/miniforge · GitHub

After downloading, install it, cd the path you downloaded in the terminal,

sh Miniforge-pypy3-version-Linux-aarch64.sh
# version看文件名

Enter all, it is not initialized by default, let's initialize manually: vim will not use its own search

# 编辑环境变量
vim ~/.bashrc
# 增加环境变量, 将<username>换成你的用户名
export PATH=/home/<username>/miniforge-pypy3/bin:$PATH
# 激活环境变量
source ~/.bashrc
# 显示(base)
source activate  # 没有base显示（base）时使用

Let's not use base:

conda config --set auto_activate_base false

In this way, you open the terminal without base

Then create a python3.8 virtual environment

conda create -n 环境名 python=3.8

We will install it in this virtual environment later.

Here conda is a bit slow if you don’t change the source, I think it’s okay to change it or not

#更换下载源

conda config --prepend channels https://mirrors.ustc.edu.cn/anaconda/pkgs/main/
conda config --prepend channels https://mirrors.ustc.edu.cn/anaconda/pkgs/free/
conda config --set show_channel_urls yes

2.2 torch, torchvision installation

Activate the virtual environment

source activate 
conda activate 环境名

Preprocessing: 

# 先把能更新的都更新了
sudo apt-get update
sudo apt-get upgrade
# 装上我们需要的环境包
# 最好使用pip3，不然出错了不好排查
sudo apt-get install python3-pip libopenblas-base libopenmpi-dev
pip3 install Cython

Download torch's whl file, cd his path

pip3 install numpy torch-1.11.0-cp38-cp38m-linux_aarch64.whl

In this way, we installed torch, and checked it after installation

python
>> import torch
>> torch.__version
>> torch.cuda.is_available()

 Then we install torchvision

$ sudo apt-get install libjpeg-dev zlib1g-dev libpython3-dev libavcodec-dev libavformat-dev libswscale-dev

$ git clone --branch <version> https://github.com/pytorch/vision torchvision   # <version>我这里是0.12.0

$ cd torchvision

$ export BUILD_VERSION=0.x.0      # where 0.x.0 is the torchvision version  我这里是0.12.0

$ python3 setup.py install --user

After installing torchvision, import it to see if there is any error

Remember that there is a problem here that when installing pillow, he can't find your cuda path:

FileNotFoundError: [Errno 2] No such file or directory: 
‘:/usr/local/cuda:/usr/local/cuda’: 
‘:/usr/local/cuda:/usr/local/cuda-’

Since it has two built-in cuda, you only need to:

# 命令行输入
export CUDA_HOME=/usr/local/cuda-11 #写11或者11.4,我选11.4

 just reinstall

It’s timed out for me to install numpy, I understand the solution, and csdn won’t allow it to be posted. But don’t install it directly in jetson, apt-get update will have a wonderful little bug. Error:

E: The repository ‘https://repo.download.nvidia.com/jetson/common 20 r32.5 Release’ no longer has a Release file.
N: Updating from such a repository can’t be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: The repository ‘https://repo.download.nvidia.com/jetson/t210 11 r32.5 Release’ no longer has a Release file.
N: Updating from such a repository can’t be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

I posted a blog alone, and went there to see how to solve it. This bad bug has ruined my whole day. So please install that thing on your computer and share it with jetson using LAN

If everything goes well, that's basically it

2.3 Installation of opencv

The system here comes with opencv and python3.8 (this is why I said it is best to install torch with py3.8, everything is complete)

If you dislike opencv without cuda acceleration, there are tutorials on the Internet, but using gstreamer to call the csi camera will cause problems. It is not so easy to get gstreamer into the virtual environment under the opencv you installed yourself (you can try the usb camera, I don’t have one try)

If you don't have such a high pursuit of speed, just copy the opencv of the system to the virtual environment and you're done.

sudo cp /usr/lib/python3.8/dist-packages/cv2/python-3.8/cv2.cpython-38-aarch64-linux-gnu.so ~/miniforge-pypy3/envs/Torch8/lib/python3.8/site-packages/
#自己根据虚拟环境位置调整指令

Remember to import cv2 to try

This opencv comes with gstreamer, I will talk about how to deal with the warning of gstreamer later

Try it and you will find:

ImportError: /lib/x86_64-linux-gnu/libp11-kit.so.0: undefined symbol: ffi_type_pointer, version LIBFFI_BASE_7.0

There is such an error!

The simple way is: but you still can't use pycharm

export LD_PRELOAD=/usr/lib/aarch64-linux-gnu/libffi.so.7

Thanks to the big guy for giving the best solution: (63 messages) libp11-kit.so.0: undefined symbol: ffi_type_pointer...Problem Solved_Destinycjk's blog-CSDN blog gives a solution in the Conda virtual environment:

#打开至conda虚拟环境下lib文件夹中
cd /home/用户名/miniforge-pypy3/envs/虚拟环境名/lib

Then ls, you can see a file called libffi.so.7

mv libffi.so.7 libffi_bak.so.7

Just change the name and save it, and then create a new libffi.so.7

sudo ln -s /usr/lib/aarch64-linux-gnu/libffi.so.7.1.0 libffi.so.7
sudo ldconfig

In this way, there will be no problem with importing.

2.4 Install tensorrt

Jetpack5.1 has a little magic bug, that is... you can't find trt under /usr/lib/python3.8/dist-packages/

So what should I do? Nvidia also felt a little embarrassed and gave a post

How to use tensorrt in python of AGX Xavier JetPack5.1 - Jetson AGX Xavier - NVIDIA Developer Forums

sudo apt install python3-libnvinfer*

 In this way, your trt will appear. jetpack5.1.1 seems to have changed this problem

2023.4 If a new tensorrt is released later, trt8.5.2 is currently the latest, so you can directly

sudo cp -r /usr/lib/python3.8/dist-packages/tensorrt* /home/用户名/miniforge-pypy3/envs/Torch8/lib/python3.8/site-packages/

Of course, it's the same when you soft link in.

2.5 Install torch2trt

this simple

git clone https://github.com/NVIDIA-AI-IOT/torch2trt
cd torch2trt
python setup.py install

can

If you encounter No module named packaging

So

pip3 install packaging

 that's it

3 Camera Processing

jetson can use csi camera and usb camera, it is recommended to use usb, it is easy and convenient to use

3.1csi camera

sudo apt install v4l-utils 

Install v4l-utils, then

ls /dev/video* 

Find the camera. If you plug in both csi and usb at the same time, generally video0 is csi. usb may have multiple video streams, like this:

 The latter two are actually a usb camera.

cap = cv2.VideoCapture(dev)

When dev=1, use the following command to see if csi can be opened

nvgstcapture

Install gstreamer, but 5.1 seems to be installed

sudo add-apt-repository universe
sudo add-apt-repository multiverse
sudo apt-get update
sudo apt-get install gstreamer1.0-tools gstreamer1.0-alsa gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav
sudo apt-get install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libgstreamer-plugins-good1.0-dev libgstreamer-plugins-bad1.0-dev

Test the following code 

import cv2

# 设置gstreamer管道参数
def gstreamer_pipeline(
    capture_width=1280, #摄像头预捕获的图像宽度
    capture_height=720, #摄像头预捕获的图像高度
    display_width=1280, #窗口显示的图像宽度
    display_height=720, #窗口显示的图像高度
    framerate=60,       #捕获帧率
    flip_method=0,      #是否旋转图像
):
    return (
        "nvarguscamerasrc ! "
        "video/x-raw(memory:NVMM), "
        "width=(int)%d, height=(int)%d, "
        "format=(string)NV12, framerate=(fraction)%d/1 ! "
        "nvvidconv flip-method=%d ! "
        "video/x-raw, width=(int)%d, height=(int)%d, format=(string)BGRx ! "
        "videoconvert ! "
        "video/x-raw, format=(string)BGR ! appsink"
        % (
            capture_width,
            capture_height,
            framerate,
            flip_method,
            display_width,
            display_height,
        )
    )

if __name__ == "__main__":
    capture_width = 1280
    capture_height = 720

    display_width = 1280
    display_height = 720

    framerate = 60			# 帧数
    flip_method = 0			# 方向

    # 创建管道
    print(gstreamer_pipeline(capture_width,capture_height,display_width,display_height,framerate,flip_method))

    #管道与视频流绑定
    cap = cv2.VideoCapture(gstreamer_pipeline(flip_method=0), cv2.CAP_GSTREAMER)

    if cap.isOpened():
        window_handle = cv2.namedWindow("CSI Camera", cv2.WINDOW_AUTOSIZE)

        # 逐帧显示
        while cv2.getWindowProperty("CSI Camera", 0) >= 0:
            ret_val, img = cap.read()
            CV2.imshow("CSI Camera", img)

            keyCode = cv2.waitKey(30) & 0xFF
            if keyCode == 27:# ESC键退出
                break

        cap.release()
        cv2.destroyAllWindows()
    else:
        print("打开摄像头失败") 

3.2 usb camera

Just use it as a normal camera, the parameter of VideoCapture is the video number you checked earlier

You will be surprised to find many warnings of gstreamer, this warning accompanied me from jetpack4.4dp to 5.1

GStreamer警告:无法查询视频位置: status=0，值=-1，时长=-1
[ warn:0] global /home/ubuntu/build_opencv/opencv/modules/videoio/src/cap_gstreamer.cpp (1390) setproperty opencv | gstreamer warning: #然后balabala

Solution:

capture = VideoCapture(0, cv.CAP_V4L2)

Enough

Brothers who need to back up the sd card can directly search for the sd card system image clone. If this thing collapses, it will be easy to waste all previous efforts. It is best to save an img system image to the computer

There are some other bugs. If I think about it, I will update it one after another. It should be easy to deal with, because I have no impression. If you have any questions, please contact me