【点云识别】Point-Voxel CNN for Efficient 3D Deep Learning (NIPS 2019)

Point-Voxel CNN for Efficient 3D Deep Learning

本文介绍一篇NIPS 2019里面关于点云识别的文章。
论文
 代码

1. 问题

目前的点云处理框架主要分为两大类

Voxel-based
Point-based
PointNet的问世，极大地促进了Point-based类方法的发展。
因为voxel类方法的缺陷很难解决，内存需求太大。
64x64x64, batch_size=16的3D-UNet 需要10 GB GPU memory 。
本文又找到了Point-based类方法的缺点，见下图。因为点云是不规则存储的，所以随机的内存访问导致效率很低。

所以本文为了解决上述问题，使用点来表示点云，但是在voxel中完成卷积。这样就避免了voxel的巨大内存需求，又解决了对不规则数据格式卷积时的时间花费。

2. 思想

在这里插入图片描述
思想很清晰，也没有对网络的架构做出巨大的调整。

        features, coords = inputs
        voxel_features, voxel_coords = self.voxelization(features, coords)
        voxel_features = self.voxel_layers(voxel_features)
        voxel_features = F.trilinear_devoxelize(voxel_features, voxel_coords, self.resolution, self.training)
        fused_features = voxel_features + self.point_features(features)

3. 算法

本文没有理论或者架构上的创新，更偏工程一点。
使用了三线插值的方法。

from torch.autograd import Function

from modules.functional.backend import _backend

__all__ = ['nearest_neighbor_interpolate']


class NeighborInterpolation(Function):
    @staticmethod
    def forward(ctx, points_coords, centers_coords, centers_features):
        """
        :param ctx:
        :param points_coords: coordinates of points, FloatTensor[B, 3, N]
        :param centers_coords: coordinates of centers, FloatTensor[B, 3, M]
        :param centers_features: features of centers, FloatTensor[B, C, M]
        :return:
            points_features: features of points, FloatTensor[B, C, N]
        """
        centers_coords = centers_coords.contiguous()
        points_coords = points_coords.contiguous()
        centers_features = centers_features.contiguous()
        points_features, indices, weights = _backend.three_nearest_neighbors_interpolate_forward(
            points_coords, centers_coords, centers_features
        )
        ctx.save_for_backward(indices, weights)
        ctx.num_centers = centers_coords.size(-1)
        return points_features

    @staticmethod
    def backward(ctx, grad_output):
        indices, weights = ctx.saved_tensors
        grad_centers_features = _backend.three_nearest_neighbors_interpolate_backward(
            grad_output.contiguous(), indices, weights, ctx.num_centers
        )
        return None, None, grad_centers_features


nearest_neighbor_interpolate = NeighborInterpolation.apply