基于Open3D的点云处理12-体素化

体素化Voxelization

体素（voxel）是像素（pixel）、体积（volume）和元素（element）的组合词，相当于3D空间中的像素;

体素化是通过用空间均匀大小的体素网格(voxel grid)来模拟模型或者点云的几何形态的过程; 表示3D模型的体素跟表示2D图像的像素相似，只不过从二维的点扩展到三维的立方体单元。体素化能够对模型进行简化，得到均匀一致的网格，在求模型的切片，物理仿真分析过程中有较好的应用。

实现模型体素化的方式有很多，比如基于八叉树的三模网格模型体素化，基于GPU并利用渲染管线中fragment shader部分实现的栅格化插值。

体素化优点：

• 点云数据将在内存中有序存储;
• 数据有序存储和降采样，能够处理大规模的数据3、可以将二维的技术用到三维上
  体;

体素化缺点：

• 信息丢失，与分辨率有关;
• 内存占用与分辨率有关;
• 稀疏的点云体素化会构建很多空体素, 若不采用稀疏卷积, 将有大量的无意义运算，降低运算效率

算法流程:

• 采用模型的AABB包围盒构建体素空间$N^3$;
• 根据分辨率$N$将整个AABB包围盒按照长宽高划分为NxNxN个体素cell;
• 标记每个cell与三维模型的关系(在模型内部,在模型外部,在模型上);(对于表面体素空间，每个体素cell的flag则只需分辨与模型是相交还是分离即可，不必再区分实在模型内部还是外部);

测试用例

参考：http://www.open3d.org/docs/latest/tutorial/geometry/voxelization.html

Open3d中从三角网构建体素

接口 1: 从给定的TriangleMesh 创建VoxelGrid。创建的VoxelGrid 的边界是根据TriangleMesh 计算的。

接口2: 从给定的TriangleMesh 创建VoxelGrid。创建的VoxelGrid 的边界是根据输入参数确定的。

接口测试：

# 从三角网构建体素
import open3d as o3d
import numpy as np

mesh = o3d.io.read_triangle_mesh("data//BunnyMesh.ply")
# 缩放到单位尺寸
mesh.scale(1 / np.max(mesh.get_max_bound() - mesh.get_min_bound()), center=mesh.get_center())
mesh.compute_vertex_normals()
o3d.visualization.draw_geometries([mesh], window_name="mesh",
                                  mesh_show_back_face=False)  # 显示mesh


voxel_grid = o3d.geometry.VoxelGrid.create_from_triangle_mesh(mesh, voxel_size=0.01)
o3d.visualization.draw_geometries([voxel_grid], window_name="体素",
                                  mesh_show_back_face=False)  # 显示体素

Open3d中从点云构建体素

**接口1：**从给定的 PointCloud 创建 VoxelGrid。给定体素的颜色值是落入其中的点的平均颜色值（如果PointCloud 有颜色）。创建的 VoxelGrid 的边界是根据 PointCloud 计算的。

**接口2：**从给定的 PointCloud 创建 VoxelGrid。给定体素的颜色值是落入其中的点的平均颜色值（如果PointCloud 有颜色）。创建的VoxelGrid 的边界是根据输入参数确定的。

接口测试：

# 从点云构建体素

import open3d as o3d
import numpy as np

mesh = o3d.io.read_triangle_mesh("data//BunnyMesh.ply")

N = 5000
pcd = mesh.sample_points_poisson_disk(N)
# fit to unit cube
pcd.scale(1 / np.max(pcd.get_max_bound() - pcd.get_min_bound()), center=pcd.get_center())
pcd.colors = o3d.utility.Vector3dVector(np.random.uniform(0, 1, size=(N, 3)))
o3d.visualization.draw_geometries([pcd])

voxel_grid = o3d.geometry.VoxelGrid.create_from_point_cloud(pcd, voxel_size=0.05)
o3d.visualization.draw_geometries([voxel_grid])

体素内外测试

接口：

测试：

# 从点云构建体素
import open3d as o3d
import numpy as np

mesh = o3d.io.read_triangle_mesh("data//BunnyMesh.ply")

N = 5000
pcd = mesh.sample_points_poisson_disk(N)
# fit to unit cube
pcd.scale(1 / np.max(pcd.get_max_bound() - pcd.get_min_bound()), center=pcd.get_center())
pcd.colors = o3d.utility.Vector3dVector(np.random.uniform(0, 1, size=(N, 3)))
# o3d.visualization.draw_geometries([pcd])

voxel_grid = o3d.geometry.VoxelGrid.create_from_point_cloud(pcd, voxel_size=0.05)
# o3d.visualization.draw_geometries([voxel_grid])

# 测试体素内外
queries = np.asarray(pcd.points)
# print(queries)
queries = np.row_stack((range(1,4), queries))
output = voxel_grid.check_if_included(o3d.utility.Vector3dVector(queries))
print(output[:10])

结果：[False, True, True, True, True, True, True, True, True, True]

雕刻一个体素（solid）

方法create_from_point_cloud和create_from_triangle_mesh只能够在几何体的表面创造体素网格。然而从大量的深度图或者轮廓中雕刻一个体素网格是有可能的。Open3d提供了carve_depth_map和 carve_silhouette方法用于体素雕刻。
测试用例：首先从几何图形渲染深度图并使用这些深度图雕刻密集体素网格的用法。结果是给定形状的填充体素网格。

import open3d as o3d
import  numpy as np

def xyz_spherical(xyz):
    x = xyz[0]
    y = xyz[1]
    z = xyz[2]
    r = np.sqrt(x * x + y * y + z * z)
    r_x = np.arccos(y / r)
    r_y = np.arctan2(z, x)
    return [r, r_x, r_y]


def get_rotation_matrix(r_x, r_y):
    rot_x = np.asarray([[1, 0, 0], [0, np.cos(r_x), -np.sin(r_x)],
                        [0, np.sin(r_x), np.cos(r_x)]])
    rot_y = np.asarray([[np.cos(r_y), 0, np.sin(r_y)], [0, 1, 0],
                        [-np.sin(r_y), 0, np.cos(r_y)]])
    return rot_y.dot(rot_x)


def get_extrinsic(xyz):
    rvec = xyz_spherical(xyz)
    r = get_rotation_matrix(rvec[1], rvec[2])
    t = np.asarray([0, 0, 2]).transpose()
    trans = np.eye(4)
    trans[:3, :3] = r
    trans[:3, 3] = t
    return trans


def preprocess(model):
    min_bound = model.get_min_bound()
    max_bound = model.get_max_bound()
    center = min_bound + (max_bound - min_bound) / 2.0
    scale = np.linalg.norm(max_bound - min_bound) / 2.0
    vertices = np.asarray(model.vertices)
    vertices -= center
    model.vertices = o3d.utility.Vector3dVector(vertices / scale)
    return model


def voxel_carving(mesh,
                  cubic_size,
                  voxel_resolution,
                  w=300,
                  h=300,
                  use_depth=True,
                  surface_method='pointcloud'):
    mesh.compute_vertex_normals()
    camera_sphere = o3d.geometry.TriangleMesh.create_sphere()

    # setup dense voxel grid
    voxel_carving = o3d.geometry.VoxelGrid.create_dense(
        width=cubic_size,
        height=cubic_size,
        depth=cubic_size,
        voxel_size=cubic_size / voxel_resolution,
        origin=[-cubic_size / 2.0, -cubic_size / 2.0, -cubic_size / 2.0],
        color=[1.0, 0.7, 0.0])

    # rescale geometry
    camera_sphere = preprocess(camera_sphere)
    mesh = preprocess(mesh)

    # setup visualizer to render depthmaps
    vis = o3d.visualization.Visualizer()
    vis.create_window(width=w, height=h, visible=False)
    vis.add_geometry(mesh)
    vis.get_render_option().mesh_show_back_face = True
    ctr = vis.get_view_control()
    param = ctr.convert_to_pinhole_camera_parameters()

    # carve voxel grid
    pcd_agg = o3d.geometry.PointCloud()
    centers_pts = np.zeros((len(camera_sphere.vertices), 3))
    for cid, xyz in enumerate(camera_sphere.vertices):
        # get new camera pose
        trans = get_extrinsic(xyz)
        param.extrinsic = trans
        c = np.linalg.inv(trans).dot(np.asarray([0, 0, 0, 1]).transpose())
        centers_pts[cid, :] = c[:3]
        ctr.convert_from_pinhole_camera_parameters(param)

        # capture depth image and make a point cloud
        vis.poll_events()
        vis.update_renderer()
        depth = vis.capture_depth_float_buffer(False)
        pcd_agg += o3d.geometry.PointCloud.create_from_depth_image(
            o3d.geometry.Image(depth),
            param.intrinsic,
            param.extrinsic,
            depth_scale=1)

        # depth map carving method
        if use_depth:
            voxel_carving.carve_depth_map(o3d.geometry.Image(depth), param)
        else:
            voxel_carving.carve_silhouette(o3d.geometry.Image(depth), param)
        print("Carve view %03d/%03d" % (cid + 1, len(camera_sphere.vertices)))
    vis.destroy_window()

    # add voxel grid survace
    print('Surface voxel grid from %s' % surface_method)
    if surface_method == 'pointcloud':
        voxel_surface = o3d.geometry.VoxelGrid.create_from_point_cloud_within_bounds(
            pcd_agg,
            voxel_size=cubic_size / voxel_resolution,
            min_bound=(-cubic_size / 2, -cubic_size / 2, -cubic_size / 2),
            max_bound=(cubic_size / 2, cubic_size / 2, cubic_size / 2))
    elif surface_method == 'mesh':
        voxel_surface = o3d.geometry.VoxelGrid.create_from_triangle_mesh_within_bounds(
            mesh,
            voxel_size=cubic_size / voxel_resolution,
            min_bound=(-cubic_size / 2, -cubic_size / 2, -cubic_size / 2),
            max_bound=(cubic_size / 2, cubic_size / 2, cubic_size / 2))
    else:
        raise Exception('invalid surface method')
    voxel_carving_surface = voxel_surface + voxel_carving

    return voxel_carving_surface, voxel_carving, voxel_surface

if __name__ == "__main__":
    armadillo = o3d.data.ArmadilloMesh()
    mesh = o3d.io.read_triangle_mesh(armadillo.path)

    visualization = True
    cubic_size = 2.0
    voxel_resolution = 128.0

    voxel_grid, voxel_carving, voxel_surface = voxel_carving(
        mesh, cubic_size, voxel_resolution)

    print("surface voxels")
    print(voxel_surface)
    o3d.visualization.draw_geometries([voxel_surface])

    print("carved voxels")
    print(voxel_carving)
    o3d.visualization.draw_geometries([voxel_carving])

    print("combined voxels (carved + surface)")
    print(voxel_grid)
    o3d.visualization.draw_geometries([voxel_grid])

从体素中构建八叉树

Open3D从 VoxelGrid几何构造八叉树create_from_voxel_grid。输入的每个体素都VoxelGrid被视为 3D 空间中的一个点，其坐标对应于体素的原点。
接口：

octree = o3d.geometry.Octree(max_depth=4)
octree.create_from_voxel_grid(voxel_grid)

测试：

import open3d as o3d
import numpy as np

mesh = o3d.io.read_triangle_mesh("data//BunnyMesh.ply")

N = 5000
pcd = mesh.sample_points_poisson_disk(N)
# fit to unit cube
pcd.scale(1 / np.max(pcd.get_max_bound() - pcd.get_min_bound()), center=pcd.get_center())
pcd.colors = o3d.utility.Vector3dVector(np.random.uniform(0, 1, size=(N, 3)))
# o3d.visualization.draw_geometries([pcd])

# 构建八叉树分布
voxel_grid = o3d.geometry.VoxelGrid.create_from_point_cloud(pcd,
                                                            voxel_size=0.05)
o3d.visualization.draw_geometries([voxel_grid])

octree = o3d.geometry.Octree(max_depth=4)
octree.create_from_voxel_grid(voxel_grid)
o3d.visualization.draw_geometries([octree])

体素下采样

接口：

downpcd = pcd.voxel_down_sample(voxel_size=0.01)

测试：

# 体素下采样
import open3d as o3d
pcd = o3d.io.read_point_cloud("data//bunny.pcd")
o3d.visualization.draw_geometries([pcd])
downpcd = pcd.voxel_down_sample(voxel_size=0.01)
o3d.visualization.draw_geometries([downpcd])