Introduction

composition

1.PointNet classification network classification network

part segmentation network

data set

1.point clouds sampled from 3D shapes
2.ShapeNetPart dataset.

structure

Which is divided into the following three parts:

data processing
model building
Select

data processing

The point cloud into a form usable program, embodied in provider.pymainly comprising downloading data, preprocessing (shuffle-> rotate-> jitter), the format conversion (hdf5-> txt)

shuffle

def shuffle_data(data, labels):
    """ Shuffle data and labels.
        Input:
          data: B,N,... numpy array
          label: B,... numpy array
        Return:
          shuffled data, label and shuffle indices
    """
    idx = np.arange(len(labels))#返回一个列表
    # print('idx=',idx)#idx= [   0    1    2 ... 2045 2046 2047]
    np.random.shuffle(idx)#把idx进行shuffle
    # print('idx=', idx)
    return data[idx, ...], labels[idx], idx

rotate rotation process

def rotate_point_cloud(batch_data):
    # print('batch data shape=',batch_data.shape)#(32, 1024, 3)
    rotated_data = np.zeros(batch_data.shape, dtype=np.float32)
    for k in range(batch_data.shape[0]):
        rotation_angle = np.random.uniform() * 2 * np.pi#生成一个随机数
        cosval = np.cos(rotation_angle)
        sinval = np.sin(rotation_angle)
        rotation_matrix = np.array([[cosval, 0, sinval],
                                    [0, 1, 0],
                                    [-sinval, 0, COSVAL]]) 
        shape_pc = batch_data [K, ...] 
        rotated_data [K, ...] = np.dot (shape_pc.reshape ((-. 1,. 3 )), rotation_matrix)
         # let the shape_pc shape becomes (? 3), because the rotation matrix (3, 3) 
    return rotated_data

jitter Jitter processing

DEF jitter_point_cloud (batch_data, Sigma = 0.01, Clip = 0.05 ): 
    B, N, C = batch_data.shape
     Assert (Clip> 0) 
    jittered_data = np.clip (Sigma np.random.randn * (B, N, C), Clip * -1, Clip) # to limit the scope of the array (-1 * Clip, Clip) 
    jittered_data + = batch_data
     return jittered_data

model building

Feature transform net

with tf.variable_scope('transform_net1') as sc:#T-net
    transform = input_transform_net(point_cloud, is_training, bn_decay, K=3)
print('point cloud=',point_cloud)#(32, 1024, 3)
# print('input transform=',transform)#(32, 3, 3)
point_cloud_transformed = tf.matmul(point_cloud, transform)
# print('point_cloud_transformed=',point_cloud_transformed)#(32, 1024, 3)

MLP (64128.1024)

net = tf_util.conv2d(net_transformed, 64, [1,1],
                         padding='VALID', stride=[1,1],
                         bn=True, is_training=is_training,
                         scope='conv3', bn_decay=bn_decay)
print('net3=',net)#(32, 1024, 1, 64)
net = tf_util.conv2d(net, 128, [1,1],
                         padding='VALID', stride=[1,1],
                         bn=True, is_training=is_training,
                         scope='conv4', bn_decay=bn_decay)
print('net4=',net)#(32, 1024, 1, 128)
net = tf_util.conv2d(net, 1024, [1,1],
                         padding='VALID', stride=[1,1],
                         bn=True, is_training=is_training,
                         scope='conv5', bn_decay=bn_decay)
print('net5=',net)#(32, 1024, 1, 1024)

Category vote

Implementation

batch_pred_sum.shape=(?,40) The possibility of data # 40 every class

pred_val.shape=(?,) Possibility # each data belongs to the largest class

= np.argmax pred_val (batch_pred_sum,. 1 )
  # Returns the index of the maximum value along the shaft axis, i.e., the maximum predicted value idx obtain the kind of the (label)

Assess

Output (predicted label, real label)

</dump/pred_label.txt>

4, 4    
0, 0
2, 2
8, 8
14, 23
...

<shape_names.txt>

airplane
bathtub
bed
bench
bookshelf
bottle
bowl
car
chair
cone
cup

Save picture prediction is wrong, and visualization

</dump/xxxx_pred_name.jpg>
Name = prediction error of a few pictures + real + label prediction label

Examples /dump/1028_label_bed_pred_sofa.jpg

Three point cloud images, which are currently look after the point cloud data three different angles of rotation

save code

  for i in range(start_idx, end_idx):
        l = current_label[i]
        total_seen_class[l] += 1
        total_correct_class[l] += (pred_val[i-start_idx] == l)
        fout.write('%d, %d\n' % (pred_val[i-start_idx], l))
        # print('!!!!!!!!!!','%d, %d\n' % (pred_val[i-start_idx], l))
        if pred_val[i-start_idx] != l and FLAGS.visu: # ERROR CASE, DUMP!如果预测错了
            img_filename = '%d_label_%s_pred_%s.jpg' % (error_cnt, SHAPE_NAMES[l],
                                                   SHAPE_NAMES [pred_val [i - start_idx]])
             # The first several prediction error of + real picture prediction label + label 
            img_filename = os.path.join (dump_dir, img_filename) 
            output_img = pc_util.point_cloud_three_views (np.squeeze (current_data [i,: ,:])) 
            scipy.misc.imsave (img_filename, output_img) 
            error_cnt +. 1 =

Painting point cloud image code

draw_point_cloud()
Input:
points: Nx3 numpy array
Output:
gray image

Recording loss, the accuracy of prediction

/dump/log_evaluate.txt

eval mean loss: 1.816358
eval accuracy: 0.501216
eval avg class acc: 0.421297
  airplane: 0.980
   bathtub: 0.440
       bed: 0.940
     bench: 0.450
     ...

pointNet Code