BSP-Net: Generating Compact Meshes via Binary Space Partitioning 详解

markdown

motivation

Understand how to extract planes from images. This is the plane extraction of the latest picture. and its application.

Thesis sequence

BSP-Net: Generating Compact Meshes via Binary Space Partitioning

Purpose

The researchers proposed BSP-Net. The network learns an implicit field: given the coordinates of n points and a shape feature vector as input, the network outputs some value that indicates whether the points are inside or outside the shape. What does the shape feature vector refer to? In the paper is an expression generation network that can generate polygon-shaped meshes. It is different from the traditional three-dimensional grid.
As follows:

The generated network has a sharp structure.
The principle of the paper's work is: through BSP (Binary Space Partitioning), the spatial hierarchy is subdivided into multiple convex sets. This network can learn a set of convex to represent 3D shape. It is divided into two steps:
1: Generate a set of convexes through a set of planes
2: Generate a 3D shape through a set of convexes
Among them, it is easy to obtain a polygon network because of the convex. So the overall steps are the above two.

paper contribution

It is the first to generate polygonal meshes directly from the network. And these meshes can be well parameterized (high quality mesh). At the same time, its expression is novel. Using the plane generated by the image and its corresponding point cloud as the input feature is very novel.

Why can a polygon grid be generated

From the perspective of the steps of the method, the plane is generated from the image. The papers I have followed before all have this function, so I won’t talk about it anymore. The most important thing is how to generate a set of convex through the plane. Then generate the shape from the convex. The principles of these two steps are cleared up, in fact, it is ok, and the follow-up will introduce their principles and feasibility in two steps.
Its overall architecture is as follows:

it clearly represents two steps.

Planes to Convex

This is the mapping from Plane to Convex.

As can be seen from the picture, it groups different planes together.
Subdividing the architecture of the plane-to-convex network in the paper, it can be seen that it is the operation of each vertex, and the expression of this vertex is obtained by summing its relationship with each plane. The formula is expressed as follows:
where $D=x{P}^T$ is the vertex signed distances, where ($P^T$ ) is a vector composed of plane parameters (a, b, c, d), which is the multiplication of the vertex and the normal vector of each plane. Then passthe ppThe layer of$p\; planes.$get as$D_i$. From these planes map to ccOne of$the\; c\; convexes.$Indicates that the point cloud flows to a certain convex. This unit represents the convex flow towards a maximum value. Perform the same operation on each vertex of the point cloud to obtain a mapping from the point cloud convex to multiple convexes. The meaning is to express whether it belongs to convex through the plane to the vertices in the point cloud. For each point cloud, the network unit it passes through is as follows:
where each vertex of the first layer represents a plane, which maps to each convex. In my opinion, which convex do the vertices belong to?

The loss function from plane to convex is designed as:

why go to max, from$The\; p$ planes are mapped to a plane, and the max operation is used,so why can it be mapped to the correct convex.

Because max is not easy to find gradient, so the sum operation is used. But the obtained result is the same.

Convex to Shape

Because after multiple Convexes are generated, it needs to merge the Convexes to generate each Shape to achieve a poly shape. Its network architecture is as follows:

the combination of multiple Convex is called a shape. Operated by pooling. Its operation unit is as follows:

Its operation is as follows to save a certain convex.

This is the operation of groups convexes. Why is the operation of using min to get the correct shape? The meaning of the paper is the possibility of obtaining non-convex.
By understanding the above two operations, you can know why the polygonal grid can be obtained correctly.

The summary is as follows:
1) The expression of the point cloud is the information obtained by multiplying the point cloud and each plane, and its expression. This representation is very memory intensive. Is it possible to multiply the point cloud with only part of the plane?

2) The expression of the plane is ax+by+cy+d=0, can it be changed to (θ,ϕ,r), these three parameters represent instead of four.

3) This paper generates a mesh from the point cloud, using the constraints of the point cloud and the expression of the point cloud. My idea is that it is possible to use the plane information in the image, and first fill in the full plane of the point cloud (without planes, such as glass, etc.), how to complement it? To discuss. In accordance with the constraints of the network. Use the distance from the point cloud to each plane.

4) The above operations are merging and other operations that are not friendly to the network, and so on.