Deep learning nova: Figure convolution neural network GCN

Author: Jin Song

introduction

Deep learning has always been to several classical model dominates, such as CNN, RNN, etc., regardless of their CV or another field of NLP have achieved excellent results, and that is how the GCN run out? Because we found a lot of problems CNN, RNN can not be solved or ineffective - the data graph structure.

Pictures or language, belong to the European space data, so the only dimension of the concept, the characteristics of the data structure of Euclidean space is very regular. But in real life, in fact, there are many, many irregular data structures, typical is the view of the structure, also known as topology, such as social networking, chemical molecular structure, knowledge maps, and so on; even the language, in fact, its interior is complex tree structure, but also a view of the structure; and like the picture, doing target recognition, we really just concerned about some of the key points on the two-dimensional picture, the structure of dots is a diagram of these.

The general structure diagram is very irregular, it can be considered infinite-dimensional A data, so it is no translation invariance . Surrounding structures may each node is unique, this data structure, let conventional CNN, RNN instant failure. So many scholars from the last century began to study how to deal with this type of the data. Here emerged many ways, e.g. GNN, DeepWalk, node2vec etc., GCN is only one, where they talk GCN, other available discuss later.

GCN, Figure convolution neural network, in fact, with effect as of CNN, is a feature extractor, but its object is the map data. GCN exquisitely designed a method of extracting features from the map data, so that we can use these features to perform data of FIG node classification (node classification), FIG classification (graph classification), the edge prediction (Link Prediction) , further Incidentally obtained can be embedded graph showing (graph embedding that) , visible widely used. So now people are brain-hole wide open, to allow GCN to shine in various fields.
This article will use the most simple classification task GCN do on Boxing Club social networks, so that no contact with the children's shoes quickly understand.

0.5 Description of the problem

First, a brief set of data.

Zachary's Karate Club is a description of the network of social relations of the University karate club members, presented by Wayne W. Zachary in the paper "An Information Flow Model for Conflict and Fission in Small Groups", it is a common example of a social network. This includes 34 members of the karate club, a conflict between the administrator and instructor Mr. Hi John A lead this club into two, half of its members around Mr. Hi set up a new club, the other half are either members found the new coach, either to give up karate. Thus, the corresponding social network, node is also divided into two groups, one belonging to Mr. Hi (Instructor), belonging to another group John A (Administrator), where the node 0 represents Mr. Hi, John node 33 on behalf of A.

We can use networkx direct access Zachary's Karate Club data, this time administrator John A is called Officer. The task is to predict which side will be added to each node (0or33). Visualization of the social network are as follows:

1. Create a graph

First, create a network of about boxing club

import dgl
import numpy as np

def build_karate_club_graph():
    # All 78 edges are stored in two numpy arrays. One for source endpoints
    # while the other for destination endpoints.
    src = np.array([1, 2, 2, 3, 3, 3, 4, 5, 6, 6, 6, 7, 7, 7, 7, 8, 8, 9, 10, 10,
        10, 11, 12, 12, 13, 13, 13, 13, 16, 16, 17, 17, 19, 19, 21, 21,
        25, 25, 27, 27, 27, 28, 29, 29, 30, 30, 31, 31, 31, 31, 32, 32,
        32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33,
        33, 33, 33, 33, 33, 33, 33, 33, 33, 33])
    dst = np.array([0, 0, 1, 0, 1, 2, 0, 0, 0, 4, 5, 0, 1, 2, 3, 0, 2, 2, 0, 4,
        5, 0, 0, 3, 0, 1, 2, 3, 5, 6, 0, 1, 0, 1, 0, 1, 23, 24, 2, 23,
        24, 2, 23, 26, 1, 8, 0, 24, 25, 28, 2, 8, 14, 15, 18, 20, 22, 23,
        29, 30, 31, 8, 9, 13, 14, 15, 18, 19, 20, 22, 23, 26, 27, 28, 29, 30,
        31, 32])
    # Edges are directional in DGL; Make them bi-directional.
    u = np.concatenate([src, dst])
    v = np.concatenate([dst, src])
    # Construct a DGLGraph
    return dgl.DGLGraph((u, v))

Print out the nodes and edges of a newly defined Graph

G = build_karate_club_graph()
print('We have %d nodes.' % G.number_of_nodes())
print('We have %d edges.' % G.number_of_edges())

Visualization with networkx new graph

import networkx as nx
# Since the actual graph is undirected, we convert it for visualization
# purpose.
nx_G = G.to_networkx().to_undirected()
# Kamada-Kawaii layout usually looks pretty for arbitrary graphs
pos = nx.kamada_kawai_layout(nx_G)
nx.draw(nx_G, pos, with_labels=True, node_color=[[.7, .7, .7]])

2. characteristics imparted to the edges and nodes

Feature map neural network will make joint training of nodes and edges.

In this example, since the features no node dimension is obtained by embedding method is characterized by one-hot 5

import torch
import torch.nn as nn
import torch.nn.functional as F

embed = nn.Embedding(34, 5)  # 34 nodes with embedding dim equal to 5
G.ndata['feat'] = embed.weight

Print out to verify the feature node

# print out node 2's input feature
print(G.ndata['feat'][2])

# print out node 10 and 11's input features
print(G.ndata['feat'][[10, 11]])

3. Define a convolutional network of FIG.

FIG simple definition of a convolutional neural network frame.

At $ $ L layers, each node $v_i^l$ With a knot vector $h_i^l$ Representation;
The purpose of each layer is polymeric GCN each node $v_i^{l}$ The neighbors who $u_i$ Vector used to generate the next layer represents $v_i^{l+1}$ And then a non-linear activation function.

The above steps can be seen as a whole message-passing paradigm: each node to receive information of neighboring nodes to update their own node represents. A graphic example of this is:

DGL libraries provide an implementation GCN layer

from dgl.nn.pytorch import GraphConv

Defined GCN GCN model contains two layers

class GCN(nn.Module):
    def __init__(self, in_feats, hidden_size, num_classes):
        super(GCN, self).__init__()
        self.conv1 = GraphConv(in_feats, hidden_size)
        self.conv2 = GraphConv(hidden_size, num_classes)

    def forward(self, g, inputs):
        h = self.conv1(g, inputs)
        h = torch.relu(h)
        h = self.conv2(g, h)
        return h

# The first layer transforms input features of size of 5 to a hidden size of 5.
# The second layer transforms the hidden layer and produces output features of
# size 2, corresponding to the two groups of the karate club.
net = GCN(5, 5, 2)

4. Prepare & initialization data

Using one-hot vector initialization node. Because it is a semi-supervised setting, only instructor (Node 0) and the president of the club (node 33) is assigned a label, to achieve the following:

inputs = embed.weight
labeled_nodes = torch.tensor([0, 33])  # only the instructor and the president nodes are labeled
labels = torch.tensor([0, 1])  # their labels are different

5. Training & visual display

And step training model as PyTorch

Create the optimizer,
Enter the input data,
Calculation of loss,
Using back-propagation optimization model

import itertools

optimizer = torch.optim.Adam(itertools.chain(net.parameters(), embed.parameters()), lr=0.01)
all_logits = []
for epoch in range(50):
    logits = net(G, inputs)
    # we save the logits for visualization later
    all_logits.append(logits.detach())
    logp = F.log_softmax(logits, 1)
    # we only compute loss for labeled nodes
    loss = F.nll_loss(logp[labeled_nodes], labels)

    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    print('Epoch %d | Loss: %.4f' % (epoch, loss.item()))

This is a very simple small example, not even divide validation and test sets. Therefore, because the model final output of two-dimensional vector for each node, we can easily in 2D space visualization of this process out, the following code shows the dynamic state of the training process from beginning to end all nodes are linear separable process.

import matplotlib.animation as animation
import matplotlib.pyplot as plt

def draw(i):
    cls1color = '#00FFFF'
    cls2color = '#FF00FF'
    pos = {}
    colors = []
    for v in range(34):
        pos[v] = all_logits[i][v].numpy()
        cls = pos[v].argmax()
        colors.append(cls1color if cls else cls2color)
    ax.cla()
    ax.axis('off')
    ax.set_title('Epoch: %d' % i)
    nx.draw_networkx(nx_G.to_undirected(), pos, node_color=colors,
            with_labels=True, node_size=300, ax=ax)

fig = plt.figure(dpi=150)
fig.clf()
ax = fig.subplots()
draw(0)  # draw the prediction of the first epoch
plt.close()

The following shows the dynamic process model After a period of training can accurately predict node belongs to which group.

ani = animation.FuncAnimation(fig, draw, frames=len(all_logits), interval=200)

Project combat Links: https://momodel.cn/workspace/5e8b3a29142d1d72944d121f/app

references:

About ##
Mo (URL: https: //momodel.cn) is a support Python artificial intelligence online modeling platform that can help you quickly develop, training and deployment model.

Recent Mo are ongoing related to machine learning introductory courses and thesis sharing activities, the public are welcome to look at our numbers for the latest information!

MoModel

Published 36 original articles · won praise 4 · views 10000 +

Private letter concerns