Article directory
overview
This paper proposes a new method for automatically reconstructing floor plans of indoor environments from raw sensor data. In contrast to existing methods that generate floor plans in the form of planar graphs by detecting corner points and connecting them, this paper adopts a strategy that decomposes the space into polygonal partitions and selects those belonging to the walls through energy minimization the edge of the structure. Through an efficient spatial partitioning data structure instead of the traditional fine corner detection task, our method provides high robustness to imperfect data.
Title: Floorplan Generation from 3D Point Clouds: a space partitioning approach
Authors: Hao Fang, Florent Lafarge, Cihui Pan, Hui Huang
Paper link:
https://www.sciencedirect.com/science/article/pii/S0924271621000538
Overview
The algorithm described in this paper is designed to take as input point clouds of real-world indoor scenes and associated pixel-level room instance label mappings, which are typically obtained by state-of-the-art instance semantic segmentation techniques. In order to better describe the geometry of the indoor scene, our algorithm first registers the input point cloud so that its upward direction is aligned with the z-axis in world coordinates, and uses standard methods to convert the point cloud into a dense triangle mesh. . This transformation is more robust to loss and occlusion in point cloud data describing indoor scenes.
The algorithm is divided into three steps: shape detection, plane projection and Markov random field problem solving. First, a set of local geometric primitives, such as vertical planes, are detected via traditional shape detection methods. Filter and regularize the extracted planes to obtain a more regular plane configuration. Second, all remaining walls are projected onto the XY plane and the 2D space is divided into a set of slices, edges, and vertices. Then, using a restricted global energy minimization formulation, the boundary shape of the indoor scene is recovered by selecting a subset of 2D aligned edges. This step divides the entire scene into internal and external spaces. Finally, by solving a Markov random field problem to assign a label configuration to each face within the boundary, adjacent slices with the same label are grouped together and treated as a single room.
This algorithm can effectively extract the geometric information of indoor scenes, convert it into the form of planar graphics, and assign labels to each room. This information can be used in areas such as indoor navigation, environment modeling, and virtual reality applications.
3.1 Primitive detection
The most representative structure in interior scenes is the wall plane. They separate interior spaces from exterior spaces and separate adjacent rooms. Therefore, the first stage of our algorithm aims to extract representative wall surfaces from a dense triangle mesh.
Planar extraction and filtering. First, a region growing algorithm is used to detect all planes, and each plane retains the corresponding inner triangle slices and vertices. Then, the floor and ceiling planes are extracted, whose normals are quasi-parallel to the z-axis and close to the 3D bounding box. The plane p whose normal vector is not orthogonal to the z-axis is eliminated, and the remaining plane is considered a vertical plane. However, given that indoor scenes tend to have complex layouts, there may be some noise planes that are not real parts of the walls, such as vertical parts of furniture. In order to avoid the negative impact of these noisy vertical planes on subsequent operations, this paper filters out planes that meet any of the following conditions: (a) the number of inner triangular faces is less than 2000; (b) the average distance from the vertices of the inner layer to the plane is greater than 0.15m ;©The minimum distance from the inner line point to the floor and roof is greater than 0.5m; (d) The area of the inner section is less than 0.5m^2. All remaining planes so far can be considered wall components.
Plane regularization. Prior knowledge of the indoor scene structure should also be taken into account. In most cases, the wall plane is perpendicular to the floor and ceiling planes. So reposition all wall planes with a new normal perpendicular to the floor, so that the quasi-orthogonal planes are aligned with each other. Finally, coplanar wall surfaces are merged if the wall planes are parallel, or the distance between them is less than 0.3m, as shown in Figure 1c.
Two-dimensional space division. Since a floor plan can be viewed as a plane figure in which each room is a closed loop, this paper discretizes the two-dimensional space into basic geometric elements, namely vertices, edges, and tangents. In order to solve this problem, this paper projects the wall plane onto the XY plane and uses a dynamic data structure to divide the two-dimensional space (as shown in Figure 1d). Compared with traditional permutation techniques, this data structure can significantly reduce the solution space for subsequent steps.
3.2 Boundary extraction
The goal of this step is to extract the boundary shape of the indoor scene that best represents the outline of the indoor space. Note that the boundary shape should also conform to the manifold assumption, where each vertex is connected to only two adjacent edges. Given the edge set E generated in the previous step, we achieve this by selecting a subset of these edges using a constrained integer programming formula.
As shown in Figure 3, noise edges that overlap with their associated walls are heavily penalized. The picture is the parameter that controls the weights of these two items.
Complexity term. In order to control the complexity of the final boundary shape, this article introduces a complexity term, which is defined as:
v is the set of vertices calculated in the previous step. This indicator returns true if (i) the two incident edges of vertex vi are active, and (ii) the two edges come from different walls. This tends to return compact boundary shape polygons with a small number of corner vertices. Figure 4 illustrates how the parameter λ affects the trade-off between data fidelity and output complexity.
Hard constraints. Since the output boundary shape must satisfy the manifold characteristics, this paper must constrain each vertex of the boundary shape to contain two adjacent edges. This is achieved through the following hard constraints:
Optimization. Under the constraints given by Equation 6, this paper finds the optimal activation state configuration picture by minimizing the energy formula in Equation 1. This constrained integer programming problem is solved by the SCIP algorithm. The edges are then activated to form the output boundary shape (see the green edges in Figure 1e).
3.3 Room segmentation
The boundary shape recovered in the previous step divides the two-dimensional space into inner and outer domains. This section only focuses on the interior domain and divides it into different rooms. Given a collection of facets within a bounding shape, we assign a room instance tag. In particular, this paper transforms this problem into a Markov random field method via the standard form of energy:
Unary items. The purpose of the image is to encourage assigning each face a label consistent with the input room instance label mapping:
Paired items. The image is designed with two purposes: (i) to return a low-complexity floor plan, and (ii) to separate rooms by wall planes within bounding shapes. To do this, we penalize edges whose adjacent slices retain distinct room instance labels and do not overlap with their associated wall planes:
Room division. The MRF model proposed in this article is solved through standard graph cut optimization techniques. This paper merges all adjacent faces into one large polygonal face, each face represents a room, as shown in Figure 1f. Finally simplify the planar graph by removing vertices connected to two collinear edges. Note that there may be some slices labeled with non-background room instances that do not have any pixels. In this case, you can choose to merge the faces into adjacent rooms with the longest common side.
experiment
Implementation details. Given an input point cloud, compute the axial bounding box of all points projected onto the XY plane. The 2D bounding box is then extended by 0.5m and discretized into a density map. If a point falls inside, each pixel value is equal to 1 (see Figure 1c). Because one of the most important goals of our method is to recover floor plans with detailed structures, a fine resolution of 1 cm was chosen, in which case some small but important wall structures can also be recovered.
Evaluation indicators:
• Room metrics. According to the definition of Floorsp, a predicted room image is a true positive if and only if (i) the image does not overlap with any other predicted room, and (ii) there is a true room with an IOU greater than 0.5.
• Geometric parameters. Since most human-annotated ground-truth floor plans do not exactly agree with the actual wall locations (see column 3 of Figure 5), the geometric metric between the predicted model and the ground-truth does not perfectly reflect the geometric accuracy of the proposed algorithm. Therefore, this article converts the 2D floor plan into a 3D CAD model and calculates the root mean square and chamfer distances between the 3D model and the input points.
Comparison of RGBD scenes. The algorithm in this paper is compared with the popular Douglas-Peucker algorithm, the object vectorization algorithm ASIP, and the current state-of-the-art floor plan generation method FloorSP. Figure 5 illustrates a qualitative comparison of various methods in the hard case, especially in non-Manhattan world scenes. ASIP and Douglas-Peucker output an isolated set of facets, caused by the disconnection of each region in the room instance labeled map. In contrast, FloorSP is able to fill this gap through the room shortest path optimization strategy. Our method also returns a 2D plan view by naturally recovering the boundary shape and dividing the internal domain into different polygonal faces. Furthermore, since our method and ASIP capture the exact location of walls in the scene, the reconstructed floor plan fits the input data better than FloorSP and Douglas-Peucker. Finally, due to our boundary shape extraction mechanism, our floor plan retains more structural details than other methods.
Figure 6 quantitatively illustrates the geometric accuracy of each method. Since small details are retained, the algorithm in this paper gives the smallest chamfer distance between the input point and the output model. In addition, Table 1 shows the average evaluation metrics, and Figure 7 shows the geometric metric distribution of all methods in 100 RGBD scenes. Our method provides the best scores on room metrics. This progress mainly comes from our two-step reconstruction method, which combines the distribution of points, the position of wall planes and the room instance label map. In contrast, other methods are less robust to imperfections contained in room instance labeling maps. In addition, for geometric indicators, this method also achieves the minimum root mean square and minimum chamfering errors. These scores can be explained by the robustness of the two-step optimization strategy, which encourages good alignment of the floor plan with small wall components.
LiDAR scene comparison.
In order to evaluate the robustness of each method300 on different sensor sources, this paper also collected 88 production-level indoor scenes scanned by lidar. Qualitatively, Figure 8 shows the floor plan reconstructed from LIDAR points by each method. Similar conclusions can be drawn regarding the quality of the output floor plans returned by all methods, as shown in Figure 5. Our algorithm still outperforms other methods in terms of geometric accuracy.
Quantitatively, Figure 9 illustrates the geometric measures between the 3D model generated by each method and the input points. Our algorithm outperforms other methods because the 3D plane figure is reconstructed from the detected walls, which successfully recovers some small but important structural details in the scene. Table 2 and Figure 10 give the average metric scores and their distributions of the four methods in the lidar dataset respectively. For RGBD data, the method in this paper obtained the best evaluation score.
Summarize
This paper proposes an automatic plane reconstruction algorithm based on indoor scene point clouds, starting from detecting walls and dividing the space into geometric elements. The boundary shape is then restored via a constrained integer programming formulation. Partitioning the interior domain into rooms using a solving multi-class labeling mechanism.
There are several aspects that need improvement in future work:
First, the splitting operator proposed in ASIP can potentially solve the problem of missing walls.
Secondly, this paper hopes to utilize the normal vector and visibility information of points to reduce the impact of inaccuracies in the input room instance label map.
Finally, our method can also be generalized to extract the shape of free-form scenes by detecting high-order geometric primitives.