PyTorch builds a graph convolutional neural network (GCN) to complete the classification and prediction of papers (with source code and data set)

If you need data sets and source code, please like, follow and leave a message in the comment area~~~

1. Dataset Introduction

We will use the Cora dataset.

The data set has a total of 2708 sample points, each sample point is a scientific paper, all sample points are divided into 7 categories , the categories are 1) case-based; 2) genetic algorithm; 3) neural network; 4) Probabilistic methods; 5) Reinforcement learning; 6) Rule learning; 7) Theory

Each paper is represented by a 1433-dimensional word vector, so each sample point has 1433 features . Each element of the word vector corresponds to a word, and this element has only two values ​​of 0 or 1. Take 0 to indicate that the word corresponding to the element is not in the paper, and take 1 to indicate that it is in the paper. All words are derived from a dictionary with 1433 words.

Each paper has cited at least one other paper, or been cited by other papers, that is, there is a connection between the sample points, and no sample point is completely unconnected with other sample points. If the sample points are regarded as points in the graph, then this is a connected graph, and there are no isolated points.

There are two main files of the dataset: cora.cites , cora.content . Among them, cora.content contains the specific information of 2708 samples, each line represents a paper sample, and the format is

<paper id> <1433-dimensional features composed of 01> <paper category (label)>

In general, if papers are regarded as nodes of a "graph", the citation relationship is an edge of the "graph", and the paper node information and citation relationship together constitute graph data. In this experiment, we will use this information to predict the category of the paper and complete the classification task of the paper category.

2. Introduction to Graph Neural Network and Graph Convolutional Neural Network

 Graph Neural Networks (Graph Neural Networks, GNN), as a new artificial intelligence learning model, can regard practical problems as the connection and message propagation problems between nodes in the graph data, model the dependencies between nodes, and tap the traditional Latent information of non-Euclidean spatial data that cannot be analyzed by neural networks. In natural language processing, computer vision, biochemistry and other fields, graph neural networks are widely used and play an important role.

Graph Convolutional Networks (GCN) is currently the mainstream graph neural network branch, and classification tasks are common tasks in machine learning. We will use the GCN algorithm to complete the classification task, further understand the working principle of the graph neural network, the construction and implementation process of the GCN, and how to apply the GCN to the classification task.

3. Operation effect

As shown in the figure below, as the number of training increases, the loss rate decreases and the accuracy increases, converging around 200 times.

 

 

 

 4. Part of the source code

The main test class code is as follows

from __future__ import division
from __future__ import print_function
import os
os.environ["KMP_DUPLICATE_LIB_OK"]="TRUE"
import time
import argparse
import numpy as np
from torch.utils.data import  DataLoader
import torch
import torch.nn.functional as F
import torch.optim as optim

from utils import load_data, accuracy
from models import GCN
import matplotlib.pyplot as plt

# Training settings
parser = argparse.ArgumentParser()
parser.add_argument('--no-cuda', action='store_true', default=False,
                    help='Disables CUDA training.')
parser.add_argument('--fastmode', action='store_true', default=False,
                    help='Validate during training pass.')
parser.add_argument('--seed', type=int, default=42, help='Random seed.')
parser.add_argument('--epochs', type=int, default=300,
                    help='Number of epochs to train.')
parser.add_argument('--lr', type=float, default=0.01,
                    help='Initial learning rate.')
parser.add_argument('--weight_decay', type=float, default=5e-4,
                    help='Weight decay (L2 loss on parameters).')
parser.add_argument('--hidden', type=int, default=16,
                    help='Number of hidden units.')
parser.add_argument('--dropout', type=float, default=0.5,
                    help='Dropout rate (1 - keep probability).')

args = parser.parse_args()
args.cuda = not args.no_cuda and torch.cuda.is_available()

.manual_seed(args.seed)

# Load data
adj, features, labels, idx_train, idx_val, idx_test = load_data()

# Model and optimizer
model = GCN(nfeat=features.shape[1],
            nhid=args.hidden,
            nclass=labels.max().item() + 1,
            dropout=args.dropout)
optimizer = optim.Adam(model.parameters(),
                       lr=args.lr, weight_decay=args.weight_decay)

if args.cuda:
    model.cuda()
    features = features.cuda()
    adj = adj.cuda()
    labels = labels.cuda()
    idx_train = idx_train.cuda()
    idx_val = idx_val.cuda()
    idx_test = idx_test.cuda()
Loss_list = []


accval=[]

def train(epoch):
       t=time.time()
       model.train()
       optimizer.zero_grad()
       output=model(features,adj)
       loss_train=F.nll_loss(output[idx_train],labels[idx_train])
       acc_train=accuracy(output[idx_train],labels[idx_train])
       loss_train.backward()
       optimizer.step()

       if not args.fastmode:
           model.eval()
           output=model(features,adj)
       loss_val=F.nll_loss(output[idx_val],labels[idx_val])
       acc_val=accuracy(output[idx_val],labels[idx_val])
       print('Epoch:{:04d}'.format(epoch+1),
       'loss_train:{:.4f}'.format(loss_train.item()),
       'acc_train:{:.4f}'.format(acc_train.item()),
       'loss_val:{:.4f}'.format(loss_val.item()),
       'acc_val:{:.4f}'.format(acc_val.item()),
        'time:{:.4f}s'.format(time.time()-t))
       Loss_list.append(loss_train.item())
       Accuracy_list.append(acc_train.item())
       lossval.append(loss_val.item())
       accval.append(acc_val.item())











def test():
    model.eval()
    output = model(features, adj)
    loss_test = F.nll_loss(output[idx_test], labels[idx_test])
    acc_test = accuracy(output[idx_test], labels[idx_test])
    print("Test set results:",
          "loss= {:.4f}".format(loss_test.item()),
          "accuracy= {:.4f}".format(acc_test.item()))
    acc=acc_test.detach().numpy()
    loss=loss_test.detach().numpy()

    print(type(loss_test))
    print(type(acc_test))


    # 定义两个数组


# Train model
t_total = time.time()

for epoch in range(args.epochs):
    train(epoch)





print("Optimization Finished!")
printal time elapsed: {:.4f}s".format(time.time() - t_total))
'''
plt.plot([i for i in range(len(Loss_list))],Loss_list)
pplot([i for i in range(len(Accuracy_list))],Accuracy_list)
'''
plt.plot([i for i in range(len(lossval))],lossval)
plot([i for i in range(len(accval))],accval)
print(type(Loss_list))
print(type(Accuracy_list))
#plt.plot([i for i in range(len(Accuracy_list),Accuracy_list)])
plt.show()
# Testing

test()

The model class is as follows

import torch.nn as nn
import torch.nn.functional as F
from layers import GraphConvolution


class GCN(nn.Module):
    def __init__(self, nfeat, nhid, nclass, dropout):
        super(GCN, self).__init__()

        self.gc1 = GraphConvolution(nfeat, nhid)
    on(nhid, nclass)
        self.dropout = dropout

    def forward(self, x, adj):
        x=F.relu(self.gc1(x,adj))
        x=F.dropout(x,self.dropout,training=self.training)
        x=self.gc2(x,adj)
        return F.log_softmax(x,dim=1)

The layer class is as follows

import math

import torch

from torch.nn.parameter import Parameter
from torch.nn.modules.module import Module


class GraphConvolution(Module):
    """
    Simple GCN layer, similar to https://arxiv.org/abs/1609.02907
    """

    def __init__(self, in_features, out_features, bias=True):
        super(GraphConvolution, self).__init__()
        self.in_features=in_features
        self.out_features=out_features
        self.weight=Parameter(torch.FloatTensor(in_features,out_features))
        if bias:
            self.bias=Parameter(torch.FloatTensor(out_features))
        else:
            self.register_parameter('bias',None)
        self.reset_parameters()


    def reset_parameters(self):
        stdv = 1. / math.sqrt(self.weight.size(1))
        self.weight.data.uniform_(-stdv, stdv)
        if self.bias is not None:
            self.bias.data.uniform_(-stdv, stdv)

    def forward(self, input, adj):
         support=torch.mm(input,self.weight)
         output=torch.spmm(adj,support)
         if self.bias is not None:
             return output+self.bias
         else:
             return output

    def __repr__(self):
        return self.__class__.__name__ + ' (' \
               + str(self.in_features) + ' -> ' \
               + str(self.out_features) + ')'

The util class is as follows

import numpy as np
import scipy.sparse as sp
import torch


def encode_onehot(labels):
    classes = set(labels)
    classes_dict = {c: np.identity(len(classes))[i, :] for i, c in
                    enumerate(classes)}
    labels_onehot = np.array(list(map(classes_dict.get, labels)),
                             dtype=np.int32)
    return labels_onehot


def load_data(path="data/cora/", dataset="cora"):
    """Load citation network dataset (cora only for now)"""
    print('Loading {} dataset...'.format(dataset))

    idx_features_labels = np.genfromtxt("{}{}.content".format(path, dataset),
                                        dtype=np.dtype(str))
    features = sp.csr_matrix(idx_features_labels[:, 1:-1], dtype=np.float32)
    labels = encode_onehot(idx_features_labels[:, -1])

    # build graph
    idx = np.array(idx_features_labels[:, 0], dtype=np.int32)
    idx_map = {j: i for i, j in enumerate(idx)}
    edges_unordered = np.genfromtxt("{}{}.cites".format(path, dataset),
                                    dtype=np.int32)
    edges = np.array(list(map(idx_map.get, edges_unordered.flatten())),
                     dtype=np.int32).reshape(edges_unordered.shape)
    adj = sp.coo_matrix((np.ones(edges.shape[0]), (edges[:, 0], edges[:, 1])),
                        shape=(labels.shape[0], labels.shape[0]),
                        dtype=np.float32)

    # build symmetric adjacency matrix
    adj = adj + adj.T.multiply(adj.T > adj) - adj.multiply(adj.T > adj)

    features = normalize(features)
    adj = normalize(adj + sp.eye(adj.shape[0]))

    idx_train = range(140)
    idx_val = range(200, 500)
    idx_test = range(500, 1500)

    features = torch.FloatTensor(np.array(features.todense()))
    labels = torch.LongTensor(np.where(labels)[1])
    adj = sparse_mx_to_torch_sparse_tensor(adj)

    idx_train = torch.LongTensor(idx_train)
    idx_val = torch.LongTensor(idx_val)
    idx_test = torch.LongTensor(idx_test)

    return adj, features, labels, idx_train, idx_val, idx_test


def normalize(mx):
    """Row-normalize sparse matrix"""
    rowsum = np.array(mx.sum(1))
    r_inv = np.power(rowsum, -1).flatten()
    r_inv[np.isinf(r_inv)] = 0.
    r_mat_inv = sp.diags(r_inv)
    mx = r_mat_inv.dot(mx)
    return mx




de_to_torch_sparse_tensor(sparse_mx):
    """Convert a scipy sparse matrix to a torch sparse tensor."""
    sparse_mx = sparse_mx.tocoo().astype(np.float32)
    indices = torch.from_numpy(
        np.vstack((sparse_mx.row, sparse_mx.col)).astype(np.int64))
    values = torch.from_numpy(sparse_mx.data)
    shape = torch.Size(sparse_mx.shape)
    return torch.sparse.FloatTensor(indices, values, shape)

It's not easy to create and find it helpful, please like, follow and collect~~~