1. Description

This article is a series of tutorials, which specifically introduces the whole process of point cloud processing and is an introductory tool. The "Point Cloud Processing" tutorial is beginner-friendly, in which we briefly introduce the point cloud processing pipeline from data preparation to data segmentation and classification.

2. Introduction

Point cloud applications are everywhere: robotics, self-driving cars, assistance systems, healthcare, and more. Point clouds are a suitable 3D representation for working with real-world data, especially when scene/object geometry is required, such as object distance, shape, and size.

A point cloud is a set of points that represent a scene or object in space in the real world. It is a discrete representation of geometric objects and scenes. More formally, a point cloud PCD is a set of n points, where each point Pi is represented by its 3D coordinates:

Note that some other features can be added to describe the point cloud, such as RGB colors, normals, etc. For example, RGB colors can be added to provide color information.

3. Point cloud generation

Point clouds are typically generated using 3D scanners (laser scanners, time-of-flight scanners, and structured light scanners) or computer-aided design (CAD) models. In this tutorial, we will first create random point clouds and visualize them. We will then generate it from the 3D model by sampling points from the 3D surface using the Open3D library. Finally, we'll see how to create them from RGB-D data.

Let's start by importing the library:

import numpy as np
import matplotlib.pyplot as plt
import open3d as o3d

3.1 Random point cloud

The easiest way is to randomly create point clouds. Note that we usually don't create random points to process, e.g. create noise for GANs (Generative Adversarial Networks).

Typically, point clouds are represented by (n × 3) arrays, where n is the number of points. Let's create a point cloud with 5 random points:

number_points = 5
pcd = np.random.rand(number_points, 3)  # uniform distribution over [0, 1)
print(pcd)

We could just print the points, but it's not very efficient, especially if the number of points is large like most applications. A better approach is to display them in 3D space. Let's visualize it using the Matplotlib library:

# Create Figure:
fig, ax = plt.subplots(subplot_kw={"projection": "3d"})
ax.scatter3D(pcd[:, 0], pcd[:, 1], pcd[:, 2])
# label the axes
ax.set_xlabel("X")
ax.set_ylabel("Y")
ax.set_zlabel("Z")
ax.set_title("Random Point Cloud")
# display:
plt.show()

Random point cloud visualization

3.2 Sampling Point Cloud

Sometimes working directly with 3D models takes time and memory. Therefore, sampling point clouds from their 3D surfaces is a potential solution. Let's import the rabbit model from the Open3D dataset:

bunny = o3d.data.BunnyMesh()
mesh = o3d.io.read_triangle_mesh(bunny.path)

Or import it after downloading from the link below :

mesh = o3d.io.read_triangle_mesh("data/bunny.ply")

Next, display the 3D model to see what it looks like. You can move the mouse to view from different viewpoints.

# Visualize:
mesh.compute_vertex_normals() # compute normals for vertices or faces
o3d.visualization.draw_geometries([mesh])

To sample a point cloud, there are several methods. In this example, we uniformly sample 1000 points from the imported mesh and visualize them:

# Sample 1000 points:
pcd = mesh.sample_points_uniformly(number_of_points=1000)

# visualize:
o3d.visualization.draw_geometries([pcd])

rabbit point cloud

We can save the created point cloud in .ply format as follows:

# Save into ply file:
o3d.io.write_point_cloud("output/bunny_pcd.ply", pcd)

3.3 Point cloud from RGB-D data

RGB-D data is collected using RGB-D sensors such as the Microsoft Kinect, which provide both RGB and depth images. RGB-D sensors are involved in many applications such as indoor navigation, obstacle avoidance, etc. Since RGB images provide pixel colors, each pixel of a depth image indicates its distance from the camera.

Open3D provides a set of functions for RGB-D image processing. To create a point cloud from RGB-D data using Open3D functions, simply import two images, create an RGB-D image object, and finally calculate the point cloud, as follows:

# read the color and the depth image:
color_raw = o3d.io.read_image("../data/rgb.jpg")
depth_raw = o3d.io.read_image("../data/depth.png")

# create an rgbd image object:
rgbd_image = o3d.geometry.RGBDImage.create_from_color_and_depth(
    color_raw, depth_raw, convert_rgb_to_intensity=False)
# use the rgbd image to create point cloud:
pcd = o3d.geometry.PointCloud.create_from_rgbd_image(
    rgbd_image,
    o3d.camera.PinholeCameraIntrinsic(
        o3d.camera.PinholeCameraIntrinsicParameters.PrimeSenseDefault))

# visualize:
o3d.visualization.draw_geometries([pcd])

Colored point clouds generated from RGB-D images

4. Open3D and NumPy

Sometimes you need to switch between Open3D and NumPy representations. For example, suppose we want to convert a NumPy point cloud into an object for visualization, and use Matplotlib to visualize a 3D model of a rabbit.Open3D.PointCloud

4.1 From NumPy to Open3D

In this example, we create 2000 random points using the function that creates a random sample from a uniform distribution on . Then we create an object and use a function to set its characteristics to random points.NumPy.random.rand()[0,1[Open3D.PointCloudOpen3D.PointCloud.pointsOpen3D.utility.Vector3dVector()

# Create numpy pointcloud:
number_points = 2000
pcd_np = np.random.rand(number_points, 3)

# Convert to Open3D.PointCLoud:
pcd_o3d = o3d.geometry.PointCloud()  # create point cloud object
pcd_o3d.points = o3d.utility.Vector3dVector(pcd_np)  # set pcd_np as the point cloud points

# Visualize:
o3d.visualization.draw_geometries([pcd_o3d])

Open3D Visualization of Random Point Clouds

4.2 From Open3D to NumPy.

Here we first read the point cloud from the .ply file using a function that returns an object. Afterwards, we just need to use functions to convert the features representing points into NumPy arrays. Finally, we display the obtained array like above.Open3D.io.read_point_cloud()Open3D.PointCloudOpen3D.PointCloud.pointsNumPy.asarray()

# Read the bunny point cloud file:
pcd_o3d = o3d.io.read_point_cloud("../data/bunny_pcd.ply")

# Convert the open3d object to numpy:
pcd_np = np.asarray(pcd_o3d.points)

# Display using matplotlib:
fig, ax = plt.subplots(subplot_kw={"projection": "3d"})
ax.scatter3D(pcd_np[:, 0], pcd_np[:, 2], pcd_np[:, 1])
# label the axes
ax.set_xlabel("X")
ax.set_ylabel("Y")
ax.set_zlabel("Z")
ax.set_title("Bunny Point Cloud")
# display:
plt.show()

Rabbit point cloud displayed using Matplotlib

V. Conclusion

In this tutorial, we learned how to create and visualize point clouds. In the next tutorial, we will learn how to deal with them. In the next tutorial we will see how to calculate point clouds in detail from depth images and RGB-D data without using Open3D functions.

Reference source:

Introduction to Point Cloud Processing | by Chayma Zatout | Better Programming

[Point Cloud Processing Tutorial] 01 How to Create and Visualize Point Cloud