cgo practice 2: Access to C-packaged dynamic libraries in Go projects

Using a Gocalling Clibrary, tends to be more common than using a Ccalling dynamic library ( previous post ). The reason why the public library is encapsulated by the underlying similar language and called by the upper application layer language is either the bottleneck of performance; or the maintenance cost of public services is reduced, especially in a large company, there are multiple languages. The team that uses a language to encapsulate the core functions, and then invoked by the application layer language, often this development mode has lower maintenance costs.GoC

The application layer and core services are often two separate projects in the actual process, and are maintained by different teams. So the following process is completely standing in this actual scenario to think about Cthe common library of language design, how to cgointegrate it, and then Gouse it in the project. The main steps are as follows:

1. contract interface, encapsulate interface
2. Generate dynamic library
3. Access to use

1. Agreed interface, encapsulated interface

As both parties of the project, the first step must be to agree on the interface function. Here we assume that a public service similar to processing message queue data is designed. The core of this public service is to complete the initial connection ( Initfunction) of the message queue, pull data from the consumption queue ( Startfunction), and stop the service ( Stopfunction), so we put These three methods are exposed to the Golanguage and are called by Goorigin .

// 暴露给Go使用的接口
void Init(char *host, char *topic, uintptr_t callback);
int Start();
void Stop();

// cgo约定的接口，C -> Go 回传数据使用
extern void sendmsg_callback(uintptr_t callback, char *content);
复制代码

Since the processing of message queue data is a real-time polling process, once the data is read, we need to send the data back to the Goupper-layer application, so a callback function ( ) is also agreed here sendmsg_callback.

After agreeing on the function interface, you can go your own way. For the development of the underlying public services, it is only necessary to implement the detailed functions of three functions Init, , , and . The function needs to be implemented, so use the keyword tag in the declaration.StartStopsendmsg_callback()Goextern

1.1 封装C公共服务

这一步来说，其实比较独立，按照需求实现逻辑即可。所以对于项目头文件结构以及代码逻辑其实都没要求，只是要求最终导出为.so动态库即可，当然最低要求肯定需要包含上面四个接口函数。

# 项目目录
libkafka/
├── libkafka.h
└── libkafka.c
复制代码

这里定义一个简单的项目工程目录(如上)，libkafka.h头文件是我们自己来设计的，只要包含上述的四个函数即可。甚至你都不需要头文件。直接写在.c文件也是没问题。其实记住一点，就是开发这个公共并不需要纠结最终这个函数怎么集成到Go调用方，大家只需要按照接口约定来实现就行。因为很多同学刚开始使用cgo时，总是在纠结到底如何把自己已经封装好的库集成到Go项目中去，其实不用太关心，这些都是由些Go的同学来完成。

libkafka.h声明

#ifndef LIB_KAFKA
#define LIB_KAFKA

#include <stdint.h>

void Init(char *host, char *topic, uintptr_t callback);
int Start();
void Stop();
extern void sendmsg_callback(uintptr_t callback, char *content);

#endif
复制代码

libkafka.c代码实现

//libkafka.c
#include <strings.h>
#include <stdlib.h>
#include <pthread.h>
#include <unistd.h>
#include <stdio.h>
#include <string.h>
#include "libkafka.h"

typedef struct {
    char *host;
    char *topic;
    uintptr_t callback;
} kafka_t;

typedef struct {
    uintptr_t callback;
} poll_param_t;

kafka_t *kafka;
pthread_t pid;
poll_param_t param;

int cancel;

void *poll(void *args) {
    pthread_detach(pthread_self());

    // ... 模拟读取队列数据

    while (!cancel) {
        sleep(5); // 模拟数据, 每隔5秒处理一批数据
        // ... 返回数据给 Go回调函数
        char * msg = malloc(sizeof(char) * 6);
        strcpy(msg, "hello");
        printf(" trigger callback: %s, call func: %ld \n", msg, kafka->callback);
        sendmsg_callback(kafka->callback, msg);
    }

    return NULL;
}

void Init(char *host, char *topic, uintptr_t callback) {
    kafka = (kafka_t *) malloc(sizeof(kafka_t));
    kafka->host = host;
    kafka->topic = topic;
    kafka->callback = callback;

    printf("Init: host=%s, topic=%s, callback=%ld \n", host, topic, callback);
}

int Start() {
    printf("Start func \n");
    pthread_create(&pid, NULL, &poll, NULL);

    return 0;
}

void Stop() {
    if (kafka != NULL) { // 释放堆内存的Go分配的字符串内容
        free(kafka->host);
        free(kafka->topic);
        free(kafka);
    }
    cancel = 1;

    // 回收线程
    pthread_join(pid, NULL);

    printf("stop finished! \n");
}
复制代码

这里我们并没有真正实现这个服务逻辑，而是大致模拟这个服务功能，因为目标是主要是想了解和cgo开发整理流程是怎样的。上述代码大致实现内容是：

1. Init 函数负责连接队列函数，Go来调用，会把请求消息队列的IP、需要操作的topic、以及有数据时的回调函数ID传递过来了。
2. Start 函数负责来运行这个服务，这里我们开启一个子线程，模拟从消息队列不间断的拉取数据，如果有数据，我们就调用sendmsg_callback，回传给Go语言。
3. Stop 函数负责来处理停服时的一些回收工作。尤其是，Go语言带过来的字符串，指针类型数据，一定要free掉！！！

2. 导出动态库

生成动态库，这里我们还是使用gcc，这里我们生成一个名叫：libkafka.so的动态库：

gcc -lpthread -fPIC -shared libkafka.c -o libkafka.so
复制代码

3. 接入使用

有了libkafka.so这个动态库，准备工作算是做好了。剩下的便是接入方需要做的事情。

作为应用层Go语言来说，需要做点工作，假设我们有个项目需要使用这个公共服务，我们只需要引入这个.so文件调用里面约定的接口方法即可。我们需要编写一些cgo的代码：

package main

/*
#include <stdint.h>
#include <stdlib.h>

#cgo LDFLAGS: -L ./libs -lkafka

extern void Init(char *host, char *topic, uintptr_t callback);
extern int Start();
extern void Stop();
*/
import "C"

复制代码

首先来说，如果你没有单独写一个.h头文件，那么就需要把上面约定的函数接口定义在cgo注释代码中，约定的函数接口需要使用extern关键字修饰。而且如果使用了C相关的数据类型，相关的include头文件是必不可少。

而且这里我们把 libkafka.so 放在了当前项目的 libs/ 下，所以又声明了编译链接包查找路径为-L ./libs -lkafka，当然你也可以直接放在系统的目录下，省去了这一步。

其外，第二步我们也说了，还有一个sendmsg_callback()函数是约定的回调函数，是Go暴露的接口，所以这个需要Go语言来定义和实现：

//export sendmsg_callback
func sendmsg_callback(callback C.uintptr_t, content *C.char) {
   callFunc := cgo.Handle(callback).Value().(func(content string))
   callFunc(C.GoString(content))
   C.free(unsafe.Pointer(content))
}

func GoCallback(content string) {
   fmt.Println("Go Callback String", content)
}
复制代码

在 sendmsg_callback 我们直接通过当前回调函数索引，查找在Init时传递的Go类型（这里是个函数），然后再执行这个函数，这里会再触发执行GoCallback()这个函数。其实在实际中，如果你和C之前回调仅仅是一对一的，那么其实没必要使用GoCallback函数，主体逻辑直接可以在sendmsg_callback完成。这里主要是为了演示服务多场景下接入不同回调才这么做的。

最后，就是直接使用的main函数入口库代码了：

func main() {
   callback := cgo.NewHandle(GoCallback)
   C.Init(C.CString("127.0.0.1:9093"), C.CString("queue_test"), C.uintptr_t(callback))
   C.Start()

   time.Sleep(time.Second * 10)
   println("准备停止")
   C.Stop()
}
复制代码

cgo.NewHandleFor registration, it is a callback object (here is a function, you can also design it as other data types) to pass to C, Cthis will be returned when appropriate id, and then the idcorresponding data type will be found through this, in fact, this is also the official wiki introduction callback The main idea of ​​the function.

C.InitThe call is directly initialized, C.Startresponsible for starting tasks, and C.Stoppreparing for recycling tasks. Finally compile and run:

# build
go build -o bin/app main.go

# 运行
LD_LIBRARY_PATH=./libs ./bin/app 
复制代码

Unsurprisingly, every 5 seconds, the sendmsg_callbackfunction will receive Cthe string data returned by the core library Stop()until it is displayed and closed. This is the entire usage process.

Notice

In fact, after mastering the entire cgodevelopment process, the rest is not complicated. But it still needs to be emphasized that the memory destruction problem of pointer type data between the two languages, such as strings, arrays, and even more complex types, must pay attention to the memory release problem. GoWhat is passed to Cis C.CStringthe space opened up in the heap memory, no matter which party it is, the memory must be released in the end, just like in Cthe function implemented by the above Stop(), we freedropped all the passed string data.

Exactly, here is also a question for readers. If the underlying library returns a string pointer Csimilar to the above when calling back , do we need to manually release this memory?sendmsg_callback* C.char