1 words written in front

Before reading this blog, you need to be familiar with the following blog content:

[Play with Jetson TX2 NX] (8) TX2 NX Visual Studio Code (VSCode) installation (detailed tutorial)
[Play with Jetson TX2 NX] (9) TX2 NX install onnx-tensorrt tool (detailed tutorial + error resolution)
[Play with Jetson TX2 NX] (10) TX2 NX installs Archiconda3 + creates a pytorch environment (detailed tutorial + error resolution)
[Play with Jetson TX2 NX] (11) TX2 NX achieves significant target segmentation based on UNet network (detailed tutorial + error resolution)

2 jetcam running onboard CSI camera

First enter the command to install traitlets, refer to the article: Jetson nano uses jetcam to run CSI camera

pip3 install traitlets==4.3.3

as the picture shows:

insert image description here
Then, install jetcam and enter the commands in sequence

git clone https://github.com/NVIDIA-AI-IOT/jetcam
cd jetcam
sudo python3 setup.py install

as the picture shows:

insert image description here
The installation is successful, as shown in the figure:

insert image description here
Run the program and test the camera

from jetcam.csi_camera import CSICamera
import cv2

camera0 = CSICamera(capture_device=0, width=224, height=224)
image0 = camera0.read()
print(image0.shape)

print(camera0.value.shape)
while 1:
    image0 = camera0.read()
    cv2.imshow("CSI Camera0", image0)
    kk = cv2.waitKey(1)
    if kk == ord('q'):  # 按下 q 键，退出
        break

as the picture shows:

insert image description here

3 Real-time salient target segmentation based on UNet algorithm

The real-time salient target segmentation code using the onboard camera is as follows:

from jetcam.csi_camera import CSICamera
import cv2
import tensorrt as trt
import numpy as np
import inference as inference_utils 

camera0 = CSICamera(capture_device=0, width=480, height=320)

image0 = camera0.read()
print(image0.shape)

print(camera0.value.shape)

# 1. 网络构建
# Precision command line argument -> TRT Engine datatype
TRT_PRECISION_TO_DATATYPE = {
    
    
    16: trt.DataType.HALF,
    32: trt.DataType.FLOAT
}
# datatype: float 32
trt_engine_datatype = TRT_PRECISION_TO_DATATYPE[16]
# batch size = 1
max_batch_size = 1
engine_file_path = "/home/tx2_lei/WHL/unet_lei/models/unet_deconv.trt"
onnx_file_path = "/home/tx2_lei/WHL/unet_lei/models/unet_deconv.onnx"
new_width, new_height = 480, 320
output_shapes = [(1, new_height, new_width)]
trt_inference_wrapper = inference_utils.TRTInference(
    engine_file_path, onnx_file_path,
    trt_engine_datatype, max_batch_size,
)

while 1:

    image0 = camera0.read()
 
    # 2. 图像预处理
    img = image0
    # inference
    trt_outputs = trt_inference_wrapper.infer(img, output_shapes, new_width, new_height)[0]
    # 输出后处理
    out_threshold = 0.5
    print("the size of tensorrt output : {}".format(trt_outputs.shape))
    output = trt_outputs.transpose((1, 2, 0))
    # # 0/1像素值
    output[output > out_threshold] = 255
    output[output <= out_threshold] = 0
    
    output = output.astype(np.uint8)


    cv2.imshow("CSI Camera0", output)
    kk = cv2.waitKey(1)
    if kk == ord('q'):  # 按下 q 键，退出
        break

The operation was killed, as shown in the figure:

insert image description here
To solve the above problems, increase the size of the swap partition, first enter the command to view the swp partition

sudo jtop

As shown, the partition size is 2GB

insert image description here
The next two are to increase the Swap memory, and enter the commands in turn

#1）新增swapfile文件大小自定义
sudo fallocate -l 6G /var/swapfile
#2）配置该文件的权限
sudo chmod 600 /var/swapfile
#3）建立交换分区
sudo mkswap /var/swapfile
#4）启用交换分区
sudo swapon /var/swapfile

as the picture shows:

insert image description here
Enter the command again to view the swp partition size

sudo jtop

As shown in the figure, the swap memory size is 8GB

insert image description here
Set to self-start swapfile, enter the command

sudo bash -c 'echo "/var/swapfile swap swap defaults 0 0" >> /etc/fstab'

as the picture shows:

insert image description here
Then go back to vscode, run the program, and succeed, as shown in the figure, use the onboard camera to segment the significant target in real time

insert image description here
Press qto exit, as shown in the figure:

insert image description here
Video link at station B: [hand in hand to deploy UNet real-time salient object segmentation on Jetson TX2 NX]

I hope this article is helpful to everyone. If there is anything wrong with the above, please correct me.

Sharing determines the height, and learning widens the gap

[Play with Jetson TX2 NX] (12) TX2 NX onboard camera + UNet algorithm for real-time salient target segmentation (detailed tutorial + error resolution)

1 words written in front

2 jetcam running onboard CSI camera

3 Real-time salient target segmentation based on UNet algorithm

Guess you like