GCN（二）GCN模型介绍

上一节介绍了处理cora数据集，以及返回的结果：

• features：论文的属性特征，维度$2708\times 1433$，并且做了归一化，即每一篇论文属性值的和为1.
• labels：每一篇论文对应的分类编号：0-6
• adj：邻接矩阵，维度$2708\times 2708$
• idx_train：0-139
• idx_val：200-499
• idx_test：500-1499

这一节介绍GCN的模型。

GCN 模型

model:

import torch.nn as nn
import torch.nn.functional as F
from pygcn.layers import GraphConvolution


class GCN(nn.Module):
    def __init__(self, nfeat, nhid, nclass, dropout):
        super(GCN, self).__init__()

        self.gc1 = GraphConvolution(nfeat, nhid)  # 构建第一层 GCN
        self.gc2 = GraphConvolution(nhid, nclass)  # 构建第二层 GCN
        self.dropout = dropout

    def forward(self, x, adj):
        x = F.relu(self.gc1(x, adj))
        x = F.dropout(x, self.dropout, training=self.training)
        x = self.gc2(x, adj)
        return F.log_softmax(x, dim=1)

layers:

import math

import torch

from torch.nn.parameter import Parameter
from torch.nn.modules.module import Module


class GraphConvolution(Module):
    """
    Simple GCN layer, similar to https://arxiv.org/abs/1609.02907
    """

    def __init__(self, in_features, out_features, bias=True):
        super(GraphConvolution, self).__init__()
        self.in_features = in_features
        self.out_features = out_features
        self.weight = Parameter(torch.FloatTensor(in_features, out_features))  # input_features, out_features
        if bias:
            self.bias = Parameter(torch.FloatTensor(out_features))
        else:
            self.register_parameter('bias', None)
        self.reset_parameters()

    def reset_parameters(self):
        stdv = 1. / math.sqrt(self.weight.size(1))
        self.weight.data.uniform_(-stdv, stdv)  # 随机化参数
        if self.bias is not None:
            self.bias.data.uniform_(-stdv, stdv)

    def forward(self, input, adj):
        support = torch.mm(input, self.weight)  # GraphConvolution forward。input*weight
        output = torch.spmm(adj, support)  # 稀疏矩阵的相乘，和mm一样的效果
        if self.bias is not None:
            return output + self.bias
        else:
            return output

    def __repr__(self):
        return self.__class__.__name__ + ' (' \
               + str(self.in_features) + ' -> ' \
               + str(self.out_features) + ')'

初始化模型

调用模型：

model = GCN(nfeat=features.shape[1],
            nhid=args.hidden,
            nclass=labels.max().item() + 1,
            dropout=args.dropout)

具体参数：

model = GCN(nfeat=1433,
            nhid=16,
            nclass=7,
            dropout=0.5)

初始化模型两层GCN:

self.gc1 = GraphConvolution(nfeat=1433, nhid=16)  # 构建第一层 GCN
self.gc2 = GraphConvolution(nhid=16, nclass=7)  # 构建第二层 GCN
self.dropout = 0.5

初始化具体layer:
第一层：gc1

def __init__(self, in_features, out_features, bias=True):
    super(GraphConvolution, self).__init__()
    self.in_features = 1433
    self.out_features = 16
    self.weight = Parameter(torch.FloatTensor(1433, 16))  # input_features, out_features
	self.bias = Parameter(torch.FloatTensor(16))
    self.reset_parameters() # 初始化w和b

参数$w$的维度：$W_{1433\times 16}$
参数$b$的维度：$b_{1\times 16}$
第二层：gc2

def __init__(self, in_features, out_features, bias=True):
    super(GraphConvolution, self).__init__()
    self.in_features = 16
    self.out_features = 7
    self.weight = Parameter(torch.FloatTensor(16, 7))  # input_features, out_features
	self.bias = Parameter(torch.FloatTensor(7))
    self.reset_parameters() # 初始化w和b

参数$w$的维度：$W_{1433\times 16}$
参数$b$的维度： $b_{1\times 7}$

forward执行模型

1. 首先执行model:

def forward(self, x, adj):
    x = F.relu(self.gc1(x, adj))
    x = F.dropout(x, self.dropout, training=self.training)
    x = self.gc2(x, adj)
    return F.log_softmax(x, dim=1)

2. 执行self.gc1(x, adj)，x表示输入特征，维度$2708\times 1433$，adj表示邻接矩阵，维度$2708\times 2708$

3. 执行GCN layer gc1层，

        support = torch.mm(input, self.weight)  # GraphConvolution forward。input*weight
        output = torch.spmm(adj, support)

计算output，$outpu{t}_{2708\times 16}=ad{j}_{2708\times 2708}\times inpu{t}_{2708\times 1433}\times W_{1433\times 16}$，然后返回$output=outpu{t}_{2708\times 16}+bia{s}_{1\times 16}$

output[0]=
tensor([ 0.0201, -0.0242,  0.0608,  0.0272,  0.0133,  0.0085,  0.0084, -0.0265,
         0.0149, -0.0100,  0.0077,  0.0029,  0.0145, -0.0181, -0.0021, -0.0183],
       grad_fn=<SelectBackward>)
self.bias=
Parameter containing:
tensor([-0.2232, -0.0295, -0.1387,  0.2170, -0.1749, -0.1551,  0.1056, -0.1860,
        -0.0666, -0.1327,  0.0212,  0.1587,  0.2496, -0.0154, -0.1683,  0.0151],
       requires_grad=True)
(output + self.bias)[0]=
tensor([-0.2030, -0.0537, -0.0779,  0.2442, -0.1616, -0.1466,  0.1140, -0.2125,
        -0.0516, -0.1427,  0.0289,  0.1615,  0.2641, -0.0336, -0.1704, -0.0032],
       grad_fn=<SelectBackward>)

4. 使用$Relu$激活函数，

x = F.relu(self.gc1(x, adj))
x[0]=
tensor([0.0000, 0.0000, 0.0000, 0.2442, 0.0000, 0.0000, 0.1140, 0.0000, 0.0000,
       0.0000, 0.0289, 0.1615, 0.2641, 0.0000, 0.0000, 0.0000],
      grad_fn=<SelectBackward>)

5. 在training阶段，使用$dropout$， 执行$x=\frac{x}{1-0.5}$，并以0.5的概率去除:

x = F.dropout(x, self.dropout, training=self.training)
x[0]=
tensor([0.0000, 0.0000, 0.0000, 0.4884, 0.0000, 0.0000, 0.2280, 0.0000, 0.0000,
        0.0000, 0.0000, 0.3230, 0.5282, 0.0000, 0.0000, 0.0000],
       grad_fn=<SelectBackward>)

6. 执行第二层 gc2

        support = torch.mm(input, self.weight)  # GraphConvolution forward。input*weight
        output = torch.spmm(adj, support)

计算output，$outpu{t}_{2708\times 7}=ad{j}_{2708\times 2708}\times inpu{t}_{2708\times 16}\times W_{16\times 7}$，然后返回$output=outpu{t}_{2708\times 7}+bia{s}_{1\times 7}$

output[0]=
tensor([-0.1928,  0.1723,  0.1689, -0.0516,  0.0387, -0.0276, -0.1027],
       grad_fn=<SelectBackward>)

7. 将返回结果x，直接吐给$F.log\_softmax(x,dim=1)$，$dim=1$表示对7维度进行log_softmax

x[0]=
tensor([-2.1474, -1.7823, -1.7856, -2.0062, -1.9158, -1.9822, -2.0573],
       grad_fn=<SelectBackward>)

8. 将output与label进行计算loss 与 acc_train
   loss=tensor(1.9186, grad_fn=<NllLossBackward>) acc_train=tensor(0.1357, dtype=torch.float64)
9. 最后进行反向传播，更新梯度W和b
10. 完成一次train的过程