Golang1.5 to Golang1.12 package management: golang vendor to go mod

1 Introduction

Golang It is a static high-level language that has been around for ten years now. It was officially launched in 2009, and then in the most recent one or two years, from 2017 to 2018, it suddenly went up and burst into flames, thanks to the container. O & M / Live / Short Video / Blockchain ...

Golang The syntax is simple, and simplicity is complexity. The core of software construction is to simplify complex things and handle the complexity.

As one gopher, we must know his package management, so that we can rationalize the code structure and do a good job in project management. ( gopher: Gopher)

2. Before GOPATH / Golang 1.5

GolangThe package management so that the population has the disease, start it with GOPATHto rely on library management, especially simple and crude.

If the environment variable:

export GOROOT=/home/love/go
export GOPATH=/home/love/code
export GOBIN=$GOROOT/bin

The above GOROOTrefers to Golang编译器以及其工具链，基础源码库the directory is located, GOPATHit is the code of the user-defined location.

The GOPATHfollowing structure:

├── src
    └── github.com
        └── hunterhug
            └── rabbit
                └── a
                    └── a.go
                └── main.go
├── bin
├── pkg 

The development package we wrote has simple and easy-to-understand paths. For example, my package is called github/hunterhug/rabbit.

We enter the rabbitdirectory, main.gothe code:

package main
import "github/hunterhug/rabbit/a"

func main(){
  ...
}

Then go build, then, when looking for packages, from'll GOPATH srcstart looking for below, such as rabbitthe package under main.gorelied github/hunterhug/rabbit/a, then it first from srcbelow by stitching down the path to find, and then find, the last generation, and the package name github/hunterhug/rabbitas the one called rabitbinary.

If we are go install, then this will be saved in the binary GOBINcase (if not present GOBIN, will be saved in GOPATH bincase). If we want to compile the package is missing, it go get -vwill download the source code dependencies to GOPATH srcthe next, and then GOPATH pkggenerates a static library of the package directory (with the next will not need to build from source, and count cache).

But when our package cannot be found:

love@love:~/code/src/github.com/hunterhug/fafacms$ go build
core/server/server.go:4:2: cannot find package "github.com/gin-contrib/cors" in any of:
        /home/love/go/src/github.com/gin-contrib/cors (from $GOROOT)
        /home/love/code/src/github.com/gin-contrib/cors (from $GOPATH)

We discovered that, in fact, is to go GOROOTdown to find the package, the package can not be found, go GOPATHfind, I moved to shed tears! For example, our GOPATHnext build a fmtpackage:

package fmt

func PPrintln() {
    print("i am diy fmt")
}

But we want to reference this library and main.gouse:

package main
import fmt
func main(){
  fmt.PPrintln()
}

I can't find it, 2333! Therefore, GOPATHunder the package it would be best not to GOROOTthe standard library under the same name!

You look at GOROOTthe structure:

├── src
    └── time
    └── fmt
├── bin
├── pkg 

This is not our GOPATHlike it, yes, now Golang编译器is a self-compiled, is used Golangto write Golang编译器, its compilers and intermediate products, basic libraries, and to maintain GOPATHa hair as seamless.

However, there are versions for different dependent packages. What if the version changes? This requires manual management.

3. Vendor / Vendor1.5 以后

Manage their own library version, I think is unlikely, after all, Javathere maven, Pythonthere pip, PHPthere compose, NodeJsthere npm.

So from Golang1.5the launch began vendorfolder mechanism ( vendor: vendors / hawkers).

From Golang1.6officially open this function.

For example, our package called awesomeProjectin GOPATHunder construction:

├── src
    └── awesomeProject
        └── vendor
            └── fmt
                └── fmt.go
        └── main.go
├── pkg 

Among them main.go:

package main

import "fmt"

func main() {
    fmt.PPrintln()
}

We enter the awesomeProjectdirectory, and go build, even successfully.

It will not be like not erupted in the above vendorreferenced directly when GOROOTthe standard package, we can finally use a standard package and a package of the same name, and that is on the main.gosame directory vendorbelow!

Now, we importare at the same level package will first vendorlook at, and then can not find under the previous method.

If we mainchange the reference to a non-existent package b:

package main

import (
    "b"
)

func main() {
    b.P()
}

Then go buildTip:

main.go:4:2: cannot find package "b" in any of:
        /home/love/code/src/awesomeProject/vendor/b (vendor tree)
        /home/love/go/src/b (from $GOROOT)
        /home/love/code/src/b (from $GOPATH)

If at this time we re a little wayward in GOPATH srcthe establishment of an empty under vendora folder, you are prompted:

main.go:4:2: cannot find package "b" in any of:
        /home/love/code/src/awesomeProject/vendor/b (vendor tree)
        /home/love/code/src/vendor/b
        /home/love/go/src/b (from $GOROOT)
        /home/love/code/src/b (from $GOPATH)

Well, we found that the loading method is:

包同目录下的vendor
GOPATH src 下的vendor
GOROOT src
GOPATH src

If GOROOTand GOPATHbuilt at vendorwhat will happen? We don't stop hurting, 233. .

Well, the question now is vendorhow the bubbling, if I main.goquoted vendor/b, while binside the package refers to a cpackage. At this point vendor/bwill be how to find the library?

├── src
    └── awesomeProject
        └── vendor
            └── b
                └── b.go
        └── main.go
├── pkg 

Now vendor/b/b.gothe contents:

package b

import "c"

func P() {
    print(" i am vendor b\n")
    c.P()
}

We enter the awesomeProjectproject go build, appears:

vendor/b/b.go:3:8: cannot find package "c" in any of:
    /home/love/code/src/awesomeProject/vendor/c (vendor tree)
    /home/love/code/src/vendor/c
    /home/love/go/src/c (from $GOROOT)
    /home/love/code/src/c (from $GOPATH)

Now the loading process is:

包同目录的包（即b包同目录看看有没有c包）
GOPATH src 下的vendor
GOROOT src
GOPATH src

At this point we are vendor/bat an empty construction vendor:

├── src
    └── awesomeProject
        └── vendor
            └── b
                └── vendor
                └── b.go
        └── main.go
├── pkg 

Into the awesomeProjectproject and then go buildthere will be:

vendor/b/b.go:3:8: cannot find package "c" in any of:
    /home/love/code/src/awesomeProject/vendor/b/vendor/c (vendor tree)
    /home/love/code/src/awesomeProject/vendor/c
    /home/love/code/src/vendor/c
    /home/love/go/src/c (from $GOROOT)
    /home/love/code/src/c (from $GOPATH)

If we meet the above cpackage, empathy in cturnkey empty vendor:

├── src
    └── awesomeProject
        └── vendor
            └── b
                └── vendor
                    └── c
                        └── vendor
                        └── c.go
                └── b.go
        └── main.go
├── pkg 

But cthe package c.goreferences nonexistent dpackage:

package c

import "d"

func P() {
    d.P()
}

Into the awesomeProjectproject and then go buildthere will be:

vendor/b/vendor/c/c.go:3:8: cannot find package "d" in any of:
    /home/love/code/src/awesomeProject/vendor/b/vendor/c/vendor/d (vendor tree)
    /home/love/code/src/awesomeProject/vendor/b/vendor/d
    /home/love/code/src/awesomeProject/vendor/d
    /home/love/code/src/vendor/d
    /home/love/go/src/d (from $GOROOT)
    /home/love/code/src/d (from $GOPATH)

Discovery, find the package vendoris bubbling up, a package references another package, take a look the same catalog vendorthere under this package, if not all the way back on the floor vendorto see if there is not, then to the next level
vendor, until GOPATH src/vendor.

So now the loading process is:

包同目录下的vendor
包目录向上的最近的一个vendor
...
GOPATH src 下的vendor
GOROOT src
GOPATH src

Summary: vendorBubble up! ! ! !

In this case, we can rely on the package are placed vendorunder, and then submitted to the warehouse, so you can save time pull the package, and relatively free, you can think how to change, you can put a person has been deleted githubpackage in the vendorcase. In this way, it is still manual and cannot manage dependent versions.

So many third-party, such as glide, godep, govendortools emerged, the use of these tools rely package must have a complete gitversion, then all will depend on the version written in a configuration file.

For example godep:

go get -v github.com/tools/godep

Execute under package

godep save 

Generated Godeps/Godep.jsonrecord version-dependent, and collected in the current packet vendorunder.

3. After go mod / Go1.11

Golang 1.11Began, there have been experimentally defined can not GOPATHfunction, and there is official go modsupport. Golang 1.12This feature is also formalized.

Now with Golang1.12were:

go mod init
go: modules disabled inside GOPATH/src by GO111MODULE=auto; see 'go help modules'

Wherein GO111MODULE=autoa switch on or off the module support, which has three possible values: off/ on/ autodefault values auto.

1. GO111MODULE=off, No module support, same as before.
2. GO111MODULE=onThe module supports, ignored GOPATHand vendorfolders, based only go.moddownload dependence.
3. GO111MODULE=autoThe project at GOPATH srcthe outside and the root directory has go.mod a file open module supports.

When using the module, GOPATH is meaningless, but it will still depend download store in GOPATH/src/modthe, it will go installresult in GOPATH/bin(if GOBINit does not exist)

We move the project out GOPATH, then:

go mod init

appear:

go: cannot determine module path for source directory /home/love/awesomeProject (outside GOPATH, no import comments)

Now main.goread:

package main // import "github.com/hunterhug/hello"

import (
    "b"
)

func main() {
    b.P()
}

Will generate go.mod:

module github.com/hunterhug/hello

go 1.12

At this time we:

go build
build github.com/hunterhug/hello: cannot load b: cannot find module providing package b

This time can not find vendor, and we come together with parameters:

go build -mod=vendor
build github.com/hunterhug/hello: cannot load c: open /home/love/awesomeProject/vendor/c: no such file or directory

Tears of tears shed, vendorwhat about bubbling? The original enabled go.mod, vendorunder the package bcan not be found b/vendorunder the package c, only to find one, 2,333,333, this is good or bad?

In general, vendorhere are vendorunscientific, godepand other tools will depend straighten out, to ensure that there is only one vendor.

Then go.modlead to vendorimpacts that can not bubble generated, is not large, shed moved to tears.

Now let's use it correctly go mod, under normal circumstances:

省略N步

Here, and we are very sorry to say goodbye, and now go modhas just come out, probably will be another update, you can search Google or other articles in this area, or:

go help modules

This part may be written in detail after a while.

The current production environment with go modnot very realistic, I recommend first the definition GOPATHand vendorusage.

4. Use Docker to compile Golang in multiple stages

Installation environment too hard, my God, every time I loaded environment, we can use the following method So easyto switch at any time Golangversion.

If your Golangproject depends stored in vendorthe lower, then we can use the multi-stage constructed and packaged into containers image, Dockefileas follows:

FROM golang:1.12-alpine AS go-build

WORKDIR /go/src/github.com/hunterhug/fafacms

COPY core /go/src/github.com/hunterhug/fafacms/core
COPY vendor /go/src/github.com/hunterhug/fafacms/vendor
COPY main.go /go/src/github.com/hunterhug/fafacms/main.go

RUN go build -ldflags "-s -w" -o fafacms main.go

FROM alpine:3.9 AS prod

WORKDIR /root/

COPY --from=go-build /go/src/github.com/hunterhug/fafacms/fafacms /bin/fafacms
RUN chmod 777 /bin/fafacms
CMD /bin/fafacms $RUN_OPTS

Where github.com/hunterhug/fafacmsis your project. Use golang:1.12-alpineto compile binary, and then into a binary base image: alpine:3.9This image is particularly small.

Compile:

sudo docker build -t hunterhug/fafacms:latest .

We have an extra mirror hunterhug/fafacms:latest , and it is very small, only a few M.

run:

sudo docker run -d  --net=host  --env RUN_OPTS="-config=/root/fafacms/config.json" hunterhug/fafacms

However, if we use cgo, then please Dockerfileread:

FROM golang:1.12 AS go-build

WORKDIR /go/src/github.com/hunterhug/fafacms

COPY core /go/src/github.com/hunterhug/fafacms/core
COPY vendor /go/src/github.com/hunterhug/fafacms/vendor
COPY main.go /go/src/github.com/hunterhug/fafacms/main.go

RUN go build -ldflags "-s -w" -o fafacms main.go

FROM bitnami/minideb-extras-base:stretch-r165 AS prod

WORKDIR /root/

COPY --from=go-build /go/src/github.com/hunterhug/fafacms/fafacms /bin/fafacms
RUN chmod 777 /bin/fafacms
CMD /bin/fafacms $RUN_OPTS

5. Summary

Managing dependencies, how to compile code into binary, is a process, and there are many details. The above is my experience, thanks for reading.