[Tutorial] Convert Python to C language and compile to generate binary files

Please indicate the source for reprinting: Senior Xiaofeng’s Big Bang Theory [xfxuezhang.cn]

This tutorial takes the DGL version of GCN as an example, and the others are similar.

1. Install cython and gcc:

sudo apt install cython  gcc -y

2. Install DGL and PyTorch:

pip3 install torch torchvision torchaudio
pip install dgl -f https://data.dgl.ai/wheels/cu117/repo.html
pip install dglgo -f https://data.dgl.ai/wheels-test/repo.html

3. Write gcn.py. Pay attention to adding #cython: language_level=3 , otherwise python2 will be used by default:

# cython: language_level=3

import torch
import torch.nn as nn
import torch.nn.functional as F
import dgl
from dgl.data import CoraGraphDataset
from dgl.nn import GraphConv

# 定义 GCN 模型
class GCN(nn.Module):
    def __init__(self, in_feats, h_feats, num_classes):
        super(GCN, self).__init__()
        self.conv1 = GraphConv(in_feats, h_feats)
        self.conv2 = GraphConv(h_feats, num_classes)

    def forward(self, g, in_feat):
        h = self.conv1(g, in_feat)
        h = F.relu(h)
        h = self.conv2(g, h)
        return h

if __name__ == "__main__":
    # 加载数据集
    dataset = CoraGraphDataset()
    g = dataset[0]

    # 创建模型实例
    model = GCN(g.ndata['feat'].shape[1], 16, dataset.num_classes)

    # 定义损失函数和优化器
    optimizer = torch.optim.Adam(model.parameters(), lr=0.01)
    criterion = nn.CrossEntropyLoss()

    # 训练模型
    for epoch in range(200):
        logits = model(g, g.ndata['feat'])
        loss = criterion(logits[g.ndata['train_mask']], g.ndata['label'][g.ndata['train_mask']])
        
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

        if epoch % 10 == 0:
            print(f'Epoch {epoch}, Loss: {loss.item()}')

    # 测试模型
    model.eval()
    with torch.no_grad():
        logits = model(g, g.ndata['feat'])
        _, predicted = torch.max(logits[g.ndata['test_mask']], 1)
        correct = (predicted == g.ndata['label'][g.ndata['test_mask']]).sum().item()
        acc = correct / len(predicted)
        print(f'Accuracy: {acc:.4f}')

4. Use cython to convert Python to C language, and a gcn.c file will be generated. Note that you need to add --embed :

cython gcn.py --embed

5. Then use the C compiler to compile the gcn.c file, and a gcn.o file will be generated:

gcc -c gcn.c `python3-config --includes` `python3-config --ldflags` -o gcn.o

6. Link to generate an executable file. At this time, a gcn executable file will be generated. Note that after -L, change it to your path:

gcc gcn.o -L/home/sxf/anaconda3/envs/dgl/lib  -lpython3.9 -o gcn

7. Run the binary executable file:

./gcn

8. If an error is reported: error while loading shared libraries: libpython3.9.so.1.0: cannot open shared object file: No such file or directory. Just include the path to the so file and re-execute step 7. Note that this is changed to your path later:

export LD_LIBRARY_PATH=/home/sxf/anaconda3/envs/dgl/lib/:$LD_LIBRARY_PATH 

9. Final effect:

Note: If you have multiple customized py files to import, then the customized py files need to be converted into so library files to be called by the main file. And if there is only one py file, there will be no such problem.