从零开始搭建 gRPC 服务 - Golang 篇（一）

gRPC：一个高性能、开源的通用 RPC 框架，基于标准的 HTTP/2 进行传输，默认采用 Protocol Buffers 序列化结构化数据。本文将介绍如何从零搭建一个 Golang 的 gRPC 服务。

准备工作

本文所述的搭建环境基于滴滴云提供的 CentOS 7.2 标准镜像

安装 Golang

下载最新版本的 Golang 安装包

gRPC 依赖于 1.6 以上版本，如果对其他版本有诉求可以在 https://golang.org/dl/ 选择下载

$ wget https://dl.google.com/go/go1.11.2.linux-amd64.tar.gz

解压安装包

$ tar zxvf go1.11.2.linux-amd64.tar.gz

配置环境变量

$ mkdir /home/dc2-user/gopath
$ sudo vim /etc/profile.d/go.sh

export GOROOT=/home/dc2-user/go
export GOPATH=/home/dc2-user/gopath
export PATH=$GOROOT/bin:$GOPATH/bin:$PATH

$ source /etc/profile.d/go.sh

检查安装结果

$ go version && go env

出现以下信息则表明安装成功

go version go1.11.2 linux/amd64
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/dc2-user/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/dc2-user/gopath"
GOPROXY=""
GORACE=""
GOROOT="/home/dc2-user/go"
GOTMPDIR=""
GOTOOLDIR="/home/dc2-user/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build425133327=/tmp/go-build -gno-record-gcc-switches"

安装 Protocol Buffers

下载最新版本的 Protobuf 安装包

$ wget https://github.com/protocolbuffers/protobuf/releases/download/v3.6.1/protobuf-all-3.6.1.tar.gz

解压安装包

$ tar zxvf protobuf-all-3.6.1.tar.gz

安装 Protobuf

$ cd protobuf-3.6.1/
$ ./configure && make && sudo make install

安装 Protobuf Golang 插件

$ go get -u -v github.com/golang/protobuf/protoc-gen-go

检查安装结果

$ protoc --version && which protoc-gen-go

libprotoc 3.6.1
~/gopath/bin/protoc-gen-go

安装 gRPC

网络环境允许的同学安装 gRPC 非常方便，直接执行以下命令即可安装完成：

$ go get -u -v google.golang.org/grpc

Fetching https://google.golang.org/grpc?go-get=1
https fetch failed: Get https://google.golang.org/grpc?go-get=1: dial tcp 216.239.37.1:443: i/o timeout
package google.golang.org/grpc: unrecognized import path "google.golang.org/grpc" (https fetch: Get https://google.golang.org/grpc?go-get=1: dial tcp 216.239.37.1:443: i/o timeout)

如果出现以上问题，则可以按照下面的方式进行安装：

在 GOPATH 下创建 google.golang.org 目录

$ mkdir -p $GOPATH/src/google.golang.org/
$ cd $GOPATH/src/google.golang.org/

下载 gRPC 最新代码并解压

$ wget https://github.com/grpc/grpc-go/archive/master.tar.gz
$ tar zxvf master.tar.gz
$ mv grpc-go-master/ grpc

安装 gRPC

$ go install google.golang.org/grpc

grpc/internal/transport/controlbuf.go:27:2: cannot find package "golang.org/x/net/http2" in any of:
	/home/dc2-user/go/src/golang.org/x/net/http2 (from $GOROOT)
	/home/dc2-user/gopath/src/golang.org/x/net/http2 (from $GOPATH)
grpc/internal/transport/controlbuf.go:28:2: cannot find package "golang.org/x/net/http2/hpack" in any of:
	/home/dc2-user/go/src/golang.org/x/net/http2/hpack (from $GOROOT)
	/home/dc2-user/gopath/src/golang.org/x/net/http2/hpack (from $GOPATH)
grpc/server.go:36:2: cannot find package "golang.org/x/net/trace" in any of:
	/home/dc2-user/go/src/golang.org/x/net/trace (from $GOROOT)
	/home/dc2-user/gopath/src/golang.org/x/net/trace (from $GOPATH)
grpc/internal/channelz/types_linux.go:26:2: cannot find package "golang.org/x/sys/unix" in any of:
	/home/dc2-user/go/src/golang.org/x/sys/unix (from $GOROOT)
	/home/dc2-user/gopath/src/golang.org/x/sys/unix (from $GOPATH)
grpc/status/status.go:37:2: cannot find package "google.golang.org/genproto/googleapis/rpc/status" in any of:
	/home/dc2-user/go/src/google.golang.org/genproto/googleapis/rpc/status (from $GOROOT)
	/home/dc2-user/gopath/src/google.golang.org/genproto/googleapis/rpc/status (from $GOPATH)

如果在安装过程中出现以上错误，表明 gRPC 依赖的库缺失，则需按照错误提示逐步补全安装其依赖库

安装 golang.org/x/*

golang.org/x/ 在 github.com/golang/ 下均有 mirror，利用这个我们可以以 GitHub 为跳板来安装相应依赖。执行以下脚本则可以自动完成 golang.org/x/* 的安装

#!/bin/bash
MODULES="crypto net oauth2 sys text tools"
for module in ${MODULES}
do
    wget https://github.com/golang/${module}/archive/master.tar.gz -O ${GOPATH}/src/golang.org/x/${module}.tar.gz
    cd ${GOPATH}/src/golang.org/x && tar zxvf ${module}.tar.gz && mv ${module}-master/ ${module}
done

安装 google.golang.org/genproto

google.golang.org/genproto 在GitHub 上的 mirror 地址为：github.com/google/go-genproto

$ wget https://github.com/google/go-genproto/archive/master.tar.gz -O ${GOPATH}/src/google.golang.org/genproto.tar.gz
$ cd ${GOPATH}/src/google.golang.org && tar zxvf genproto.tar.gz && mv go-genproto-master genproto

依赖安装完毕后再次执行 go install google.golang.org/grpc 即可完成 gRPC 的安装，自此环境相关的准备工作完成。

构建一个简单的 gRPC 服务

以该 helloworld.proto 文件为例

扫描二维码关注公众号，回复： 4488741 查看本文章

syntax = "proto3";

option go_package = "github.com/grpc/example/helloworld";

package helloworld;

// The greeting service definition.
service Greeter {
  // Sends a greeting
  rpc SayHello (HelloRequest) returns (HelloReply) {}
}

// The request message containing the user's name.
message HelloRequest {
  string name = 1;
}

// The response message containing the greetings
message HelloReply {
  string message = 1;
}

编译 .proto 文件

$ protoc helloworld.proto --go_out=output
$ tree .
.
├── helloworld.proto
└── output
    └── github.com
        └── grpc
            └── example
                └── helloworld
                    └── helloworld.pb.go
5 directories, 2 files

$ head -n 15 output/github.com/grpc/example/helloworld/helloworld.pb.go

// Code generated by protoc-gen-go. DO NOT EDIT.
// source: helloworld.proto

package helloworld

import (
	fmt "fmt"
	proto "github.com/golang/protobuf/proto"
	math "math"
)

// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = fmt.Errorf
var _ = math.Inf

这样编译出来结果只是将 .proto 文件中描述的结构与方法翻译成了 Golang 而已，如果需要将其以 gRPC 的方式提供服务的话，需需要在编译时指定插件

$ protoc helloworld.proto --go_out=plugins=grpc:output
$ tree .
.
├── helloworld.proto
└── output
    └── github.com
        └── grpc
            └── example
                └── helloworld
                    └── helloworld.pb.go

5 directories, 2 files

$ head output/github.com/grpc/example/helloworld/helloworld.pb.go -n 15
// Code generated by protoc-gen-go. DO NOT EDIT.
// source: helloworld.proto

package helloworld

import (
	context "context"
	fmt "fmt"
	proto "github.com/golang/protobuf/proto"
	grpc "google.golang.org/grpc"
	math "math"
)

// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal

编写 client.go 与 server.go

在当前目录下创建一个 client.go，一个 server.go 文件，目录结构如下：

$ tree .
.
├── client.go
├── helloworld.proto
├── output
│   └── github.com
│       └── grpc
│           └── example
│               └── helloworld
│                   └── helloworld.pb.go
└── server.go

5 directories, 4 files

编写 client.go

package main

import (
	"context"
	"log"
	"os"
	"time"

	"google.golang.org/grpc"
	pb "./output/github.com/grpc/example/helloworld"
)

const (
	address     = "localhost:50051"
	defaultName = "world"
)

func main() {
	// Set up a connection to the server.
	conn, err := grpc.Dial(address, grpc.WithInsecure())
	if err != nil {
		log.Fatalf("did not connect: %v", err)
	}
	defer conn.Close()
	c := pb.NewGreeterClient(conn)

	// Contact the server and print out its response.
	name := defaultName
	if len(os.Args) > 1 {
		name = os.Args[1]
	}
	ctx, cancel := context.WithTimeout(context.Background(), time.Second)
	defer cancel()
	r, err := c.SayHello(ctx, &pb.HelloRequest{Name: name})
	if err != nil {
		log.Fatalf("could not greet: %v", err)
	}
	log.Printf("Greeting: %s", r.Message)
}

编写 server.go

package main

import (
	"context"
	"log"
	"net"

	"google.golang.org/grpc"
	pb "./output/github.com/grpc/example/helloworld"
	"google.golang.org/grpc/reflection"
)

const (
	port = ":50051"
)

// server is used to implement helloworld.GreeterServer.
type server struct{}

// SayHello implements helloworld.GreeterServer
func (s *server) SayHello(ctx context.Context, in *pb.HelloRequest) (*pb.HelloReply, error) {
	return &pb.HelloReply{Message: "Hello " + in.Name}, nil
}

func main() {
	lis, err := net.Listen("tcp", port)
	if err != nil {
		log.Fatalf("failed to listen: %v", err)
	}
	s := grpc.NewServer()
	pb.RegisterGreeterServer(s, &server{})
	// Register reflection service on gRPC server.
	reflection.Register(s)
	if err := s.Serve(lis); err != nil {
		log.Fatalf("failed to serve: %v", err)
	}
}

运行 gRPC 服务

打开两个会话窗口，在其中之一执行：

$ go run server.go

在另一个会话窗口运行：

 $ go run client.go gRPC
 2018/12/09 18:05:22 Greeting: Hello gRPC

自此一个简单的 gRPC 服务就搭建起来了。

构建一个安全的 gRPC 服务

同样是上面的 .proto 文件，我们想要为其增加鉴权与加密传输的能力

增加 TLS

生成服务端公私钥

$ openssl genrsa -out server.key 2048
$ openssl req -x509 -key server.key -out server.pem

目录结构为：

$ tree .
.
├── client.go
├── helloworld.proto
├── output
│   └── github.com
│       └── grpc
│           └── example
│               └── helloworld
│                   └── helloworld.pb.go
├── server.go
├── server.key
└── server.pem

5 directories, 6 files

改写 server.go

package main

import (
        "context"
        "crypto/tls"
        "log"
        "net"

        "google.golang.org/grpc"
        "google.golang.org/grpc/credentials"
        pb "./output/github.com/grpc/example/helloworld"
        "google.golang.org/grpc/reflection"
)

const (
        port = ":50051"
)

// server is used to implement helloworld.GreeterServer.
type server struct{}

// SayHello implements helloworld.GreeterServer
func (s *server) SayHello(ctx context.Context, in *pb.HelloRequest) (*pb.HelloReply, error) {
        return &pb.HelloReply{Message: "Hello " + in.Name}, nil
}

func main() {
        cert, err := tls.LoadX509KeyPair("./server.pem", "./server.key")
        if err != nil {
                log.Fatalf("failed to load key pair: %s", err)
        }
        lis, err := net.Listen("tcp", port)
        if err != nil {
                log.Fatalf("failed to listen: %v", err)
        }
        opts := []grpc.ServerOption{
                // Enable TLS for all incoming connections.
                grpc.Creds(credentials.NewServerTLSFromCert(&cert)),
        }
        s := grpc.NewServer(opts...)
        pb.RegisterGreeterServer(s, &server{})
        // Register reflection service on gRPC server.
        reflection.Register(s)
        if err := s.Serve(lis); err != nil {
                log.Fatalf("failed to serve: %v", err)
        }
}

改写 client.go

package main

import (
        "context"
        "crypto/tls"
        "log"
        "os"
        "time"

        "google.golang.org/grpc"
        "google.golang.org/grpc/credentials"
        pb "./output/github.com/grpc/example/helloworld"
)

const (
        address     = "localhost:50051"
        defaultName = "world"
)

func main() {
        opts := []grpc.DialOption{
                // credentials.
                grpc.WithTransportCredentials(
                        credentials.NewTLS(&tls.Config{InsecureSkipVerify: true}),
                ),
        }
        // Set up a connection to the server.
        conn, err := grpc.Dial(address, opts...)
        if err != nil {
                log.Fatalf("did not connect: %v", err)
        }
        defer conn.Close()
        c := pb.NewGreeterClient(conn)

        // Contact the server and print out its response.
        name := defaultName
        if len(os.Args) > 1 {
                name = os.Args[1]
        }
        ctx, cancel := context.WithTimeout(context.Background(), time.Second)
        defer cancel()
        r, err := c.SayHello(ctx, &pb.HelloRequest{Name: name})
        if err != nil {
                log.Fatalf("could not greet: %v", err)
        }
        log.Printf("Greeting: %s", r.Message)
}

运行 gRPC 服务

打开两个会话窗口，在其中之一执行：

$ go run server.go

在另一个会话窗口运行：

$ go run client.go tls_gRPC
2018/12/09 21:19:07 Greeting: Hello tls_gRPC

增加 OAuth2 鉴权

改写 server.go

package main

import (
        "context"
        "crypto/tls"
        "log"
        "net"
        "strings"

        "google.golang.org/grpc"
        "google.golang.org/grpc/codes"
        "google.golang.org/grpc/credentials"
        pb "./output/github.com/grpc/example/helloworld"
        "google.golang.org/grpc/reflection"
        "google.golang.org/grpc/metadata"
        "google.golang.org/grpc/status"
)

const (
        port = ":50051"
)

var (
        errMissingMetadata = status.Errorf(codes.InvalidArgument, "missing metadata")
        errInvalidToken    = status.Errorf(codes.Unauthenticated, "invalid token")
)

// server is used to implement helloworld.GreeterServer.
type server struct{}

// SayHello implements helloworld.GreeterServer
func (s *server) SayHello(ctx context.Context, in *pb.HelloRequest) (*pb.HelloReply, error) {
        return &pb.HelloReply{Message: "Hello " + in.Name}, nil
}

func main() {
        cert, err := tls.LoadX509KeyPair("./server.pem", "./server.key")
        if err != nil {
                log.Fatalf("failed to load key pair: %s", err)
        }
        lis, err := net.Listen("tcp", port)
        if err != nil {
                log.Fatalf("failed to listen: %v", err)
        }
        opts := []grpc.ServerOption{
                // The following grpc.ServerOption adds an interceptor for all unary
                // RPCs. To configure an interceptor for streaming RPCs, see:
                // https://godoc.org/google.golang.org/grpc#StreamInterceptor
                grpc.UnaryInterceptor(ensureValidToken),
                // Enable TLS for all incoming connections.
                grpc.Creds(credentials.NewServerTLSFromCert(&cert)),
        }
        s := grpc.NewServer(opts...)
        pb.RegisterGreeterServer(s, &server{})
        // Register reflection service on gRPC server.
        reflection.Register(s)
        if err := s.Serve(lis); err != nil {
                log.Fatalf("failed to serve: %v", err)
        }
}

// valid validates the authorization.
func valid(authorization []string) bool {
        if len(authorization) < 1 {
                return false
        }
        token := strings.TrimPrefix(authorization[0], "Bearer ")
        // Perform the token validation here. For the sake of this example, the code
        // here forgoes any of the usual OAuth2 token validation and instead checks
        // for a token matching an arbitrary string.
        if token != "some-secret-token" {
                return false
        }
        return true
}

// ensureValidToken ensures a valid token exists within a request's metadata. If
// the token is missing or invalid, the interceptor blocks execution of the
// handler and returns an error. Otherwise, the interceptor invokes the unary
// handler.
func ensureValidToken(ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (interface{}, error) {
        md, ok := metadata.FromIncomingContext(ctx)
        if !ok {
                return nil, errMissingMetadata
        }
        // The keys within metadata.MD are normalized to lowercase.
        // See: https://godoc.org/google.golang.org/grpc/metadata#New
        if !valid(md["authorization"]) {
                return nil, errInvalidToken
        }
        // Continue execution of handler after ensuring a valid token.
        return handler(ctx, req)
}

改写 client.go

package main

import (
        "context"
        "crypto/tls"
        "log"
        "os"
        "time"

        "golang.org/x/oauth2"
        "google.golang.org/grpc"
        "google.golang.org/grpc/credentials"
        "google.golang.org/grpc/credentials/oauth"
        pb "./output/github.com/grpc/example/helloworld"
)

const (
        address     = "localhost:50051"
        defaultName = "world"
)

func main() {
        perRPC := oauth.NewOauthAccess(fetchToken())
        opts := []grpc.DialOption{
                // In addition to the following grpc.DialOption, callers may also use
                // the grpc.CallOption grpc.PerRPCCredentials with the RPC invocation
                // itself.
                // See: https://godoc.org/google.golang.org/grpc#PerRPCCredentials
                grpc.WithPerRPCCredentials(perRPC),
                // oauth.NewOauthAccess requires the configuration of transport
                // credentials.
                grpc.WithTransportCredentials(
                        credentials.NewTLS(&tls.Config{InsecureSkipVerify: true}),
                ),
        }
        // Set up a connection to the server.
        conn, err := grpc.Dial(address, opts...)
        if err != nil {
                log.Fatalf("did not connect: %v", err)
        }
        defer conn.Close()
        c := pb.NewGreeterClient(conn)

        // Contact the server and print out its response.
        name := defaultName
        if len(os.Args) > 1 {
                name = os.Args[1]
        }
        ctx, cancel := context.WithTimeout(context.Background(), time.Second)
        defer cancel()
        r, err := c.SayHello(ctx, &pb.HelloRequest{Name: name})
        if err != nil {
                log.Fatalf("could not greet: %v", err)
        }
        log.Printf("Greeting: %s", r.Message)
}

// fetchToken simulates a token lookup and omits the details of proper token
// acquisition. For examples of how to acquire an OAuth2 token, see:
// https://godoc.org/golang.org/x/oauth2
func fetchToken() *oauth2.Token {
        return &oauth2.Token{
                AccessToken: "some-secret-token",
        }
}

由于 golang.org/x/oauth2 中依赖了 cloud.google.com/go/compute/metadata，其在 GitHub 上的 mirror 地址为 github.com/googleapis/google-cloud-go，于是我们需要按照之前的方式安装该依赖。

$ mkdir -p ${GOPATH}/src/cloud.google.com/
$ wget https://github.com/googleapis/google-cloud-go/archive/master.tar.gz -O ${GOPATH}/src/cloud.google.com/go.tar.gz
$ cd ${GOPATH}/src/cloud.google.com/ && tar zxvf go.tar.gz && mv google-cloud-go-master go

运行 gRPC 服务

打开两个会话窗口，在其中之一执行：

$ go run server.go

在另一个会话窗口运行：

$ go run client.go oauth2_tls_gRPC
2018/12/09 21:27:56 Greeting: Hello oauth2_tls_gRPC

自此一个安全的 gRPC 服务就搭建起来了。