Go Basics 11-Understand the package import of Go language

The Go language uses packages as the basic unit to organize source code. It can be said that a Go program is built by linking several packages together. Although this is not an innovation compared to languages ​​such as Java and Python, it is much more "advanced" than the header file inclusion mechanism of the ancestor C language.

The fast compilation speed is an outstanding manifestation of this "advancement", even if every compilation starts from scratch. The Go language's construction model with packages as the basic building unit makes dependency analysis very simple, avoiding the huge overhead of analyzing dependencies through header files in C language.

The reasons why Go compiles quickly are reflected in the following three aspects.

● Go requires that each source file explicitly lists all dependent package imports at the beginning, so that the Go compiler does not have to read and process the entire file to determine the list of dependent packages.

● Go requires that there cannot be cyclic dependencies between packages, so the dependencies of a package form a directed acyclic graph. Due to the lack of loops, packages can be compiled individually or in parallel.

● The target file (file_name.o or package_name.a) corresponding to the compiled Go package not only records the export symbol information of the package itself, but also records the export symbol information of the packages it depends on.

In this way, when the Go compiler compiles a certain package P, it only needs to read one target file for each package that P depends on (such as importing package Q) (for example: the target file compiled into Q package already contains Q The export information of the package's dependent package) without reading the information in other files.
The definition and use of packages in Go language are very simple. Declare the package to which the Go source file belongs through the package keyword:

// xx.go
package a
...
上述源码表示文件xx.go是包a的一部分。
使用import关键字导入依赖的标准库包或第三方包：
package main
	import (
	"fmt" // 标准库包导入
	"a/b/c" // 第三方包导入
)
func main() {
    
    
c.Func1()
fmt.Println("Hello, Go!")
}

When many Gophers see the above code, they will take it for granted that the "c" and "fmt" after import and the c and fmt in c.Func1() and fmt.Println() are the same syntax element: the package name. But after studying the Go language in depth, everyone will find that this is not the case.

For example, when using the official client package provided by the real-time distributed messaging framework nsq, our package import is written like this:

import "github.com/nsqio/go-nsq"

But when using the exported functions provided by this package, we are not using go-nsq.xx but

nsq.xxx：
q, _ := nsq.NewConsumer("write_test", "ch", config)

Many Gophers have some doubts when learning to import Go packages: What does the last segment in the path after the import represent? Is it a package name or a path?

In this article, I will work with you to deeply explore and understand the package import of Go language.

Go program building process

Let’s first take a brief look at the construction process of Go programs as a precursor to understanding the import of Go packages. Like mainstream statically compiled languages, the construction of Go programs is simply composed of two stages: compile and link.

A non-main package will generate a .a file after compilation, which can be understood as the target file of the Go package.
The target file is actually processed through the pack tool ( $GOROOT/pkg/tool/darwi{n}_{a}md64/pack)isformed\; afterpackaging.ofiles.By\; default,the\; .afileis\; generated\; in\; the\; temporary\; directory\; during\; the\; compilation\; process.\; You\; cannot\; see\; the\; .a\; file\; unless\; you\; usego$ in s t a ll to install it under GOPATH/$pkg$$($$before$$Go$$1.11$$)$$.$If you are building an executable program, the .a file will be used from the linking stage of building the executable program.

The source code files of the standard library package are $GOROOT/src\; ,\; andthecorresponding.afile\; is\; stored$ under GOROOT/pkg/
darwin_amd64 (taking macOS as an example; if it is a Linux system, it is linux_amd64):

// Go 1.16
$tree -FL 1 $GOROOT/pkg/darwin_amd64
├── archive/
├── bufio.a
├── bytes.a
├── compress/
├── container/
├── context.a
├── crypto/
├── crypto.a
...

Readers who are "seeking a deeper explanation" may ask this question: When building a Go program, will the compiler recompile the source files of the dependent packages or directly link the .a files of the packages? Let’s give you the answer through an experiment.

Go 1.10 version introduced the build cache . In order to avoid the complexity brought by the build cache to the experimental process and analysis, we used Go 1.9.7 version to conduct this experiment.
The directory structure of our experimental environment is as follows:

chapter3-demo1
├── cmd/
│ └── app1/
│ └── main.go
└── pkg/
└── pkg1/
└── pkg1.go

For demonstration purposes only, the source codes of pkg1.go and main.go are very simple:

// cmd/app1/main.go
package main
import (
"github.com/bigwhite/effective-go-book/chapter3-demo1/pkg/pkg1"
)
func main() {
    
    
pkg1.Func1()
}
// pkg/pkg1/pkg1.go
package pkg1
import "fmt"
func Func1() {
    
    
fmt.Println("pkg1.Func1 invoked")
}

Enter chapter3-demo1 and execute the following command:

$go install github.com/bigwhite/effective-go-book/chapter3-demo1/pkg/pkg1

After that, we can

$GOPATH/pkg/darwin_amd64/github.com/bigwhite/effective-
go-book/chapter3-demo1/pkg下看到pkg1包对应的目标文件pkg1.a：
$ls $GOPATH/pkg/darwin_amd64/github.com/bigwhite/effective-go-book/chapter3-demo1/pkg
pkg1.a

We continue to compile the executable program app1 under the chapter3-demo1 path:

$go build github.com/bigwhite/effective-go-book/chapter3-demo1/cmd/app1

After executing the above command, we will see an executable file app1 under chapter3-demo1, execute the file:

$ls
app1* cmd/ pkg/
$./app1
pkg1.Func1 invoked

This is in line with our expectations, but now we still cannot know whether the source code of the pkg1 package or the target file pkg1.a was used to compile app1, because currently their output is consistent.

Modify the code of pkg1.go:

// pkg/pkg1/pkg1.go
package pkg1
import "fmt"
func Func1() {
    
    
fmt.Println("pkg1.Func1 invoked - Again")
}

Recompile and execute app1, and get the following results:

$go build github.com/bigwhite/effective-go-book/chapter3-demo1/cmd/app1
$./app1
pkg1.Func1 invoked - Again

Is it a path name or a package name?

Through the previous experiments, we learned that the compiler must use the source code of the package that the compilation unit (a package) depends on during the compilation process.

In order for the compiler to find the source code files of the dependent packages, it needs to know the source code paths of the dependent packages
. This path consists of two parts: the base search path and the package import path .

The basic search path is a global setting, and its rules are described below.
1) The source code base search path of all packages (whether standard library packages or third-party packages) includes $GOROOT/src.2)\; Based\; on\; the\; above\; basic\; search\; paths,\; different\; versions\; ofGocontaindifferent\; other\; basic\; search\; paths.●\; BeforeGo\; version1.11,\; the\; basic\; search\; path\; for\; the\; source\; code\; of\; the\; package\; also\; includedGOPATH$ /src.
● Go 1.11～Go 1.12 version, the basic search path of package source code has three modes:
● In classic gopath mode (GO111MODULE=off):$GOPATH/src●module\_-In\; awaremode\; (GO111MODULE=on):$ GOPATH/pkg/mod.
● In auto mode (GO111MODULE=auto): Under$Underthe\; GOPATH/src\; path,it\; is\; the\; same\; asthe\; gopathmode;\; outside\; theGOP\; A\; T\; H\; /\; src\; path\; and\; including\; go.mod,\; it\; isthe$ same as the module-aware mode.
● In Go version 1.13, there are two modes for the basic search path of package source code:
● In classic gopath mode (GO111MODULE=off):$GOPATH/src●module\_-In\; awaremode\; (GO111MODULE=on/auto):$ GOPATH/pkg/mod.
● Future Go versions will only have module-aware mode, which means that the source code of the package will only be searched in the module cache directory.
The second part of the search path is the package import path located at the head of each package source file. Combining the basic search path with
the package import path, the Go compiler can determine the set of source code paths of all dependent packages of a package. This
set constitutes the source code search path space of the Go compiler. Consider the following example:

// p1.go
package p1
import (
"fmt"
"time"
"github.com/bigwhite/effective-go-book"
"golang.org/x/text"
"a/b/c"
"./e/f/g"
)
...

Package name conflict problem

The same package name may exist in different projects and different warehouses. The same source code file
is likely to have a package with the same name in the source code search path space formed by its package import path. For example: we have another chapter3-demo2,
which also has a package named pkg1, and the import path is github.com/bigwhite/effective-go-book/chapter3-demo2/pkg/pkg1
.
What will happen if cmd/app3 imports the pkg1 packages of chapter3-demo1 and chapter3-demo2 at the same time ?

// cmd/app3
package main
import (
"github.com/bigwhite/effective-go-book/chapter3-demo1/pkg/pkg1"
"github.com/bigwhite/effective-go-book/chapter3-demo2/pkg/pkg1"
)
func main() {
    
    
pkg1.Func1()
}

Compile cmd/app3:

$go build github.com/bigwhite/effective-go-book/chapter3-demo1/cmd/app3
# github.com/bigwhite/effective-go-book/chapter3-demo1/cmd/app3
./main.go:5:2: pkg1 redeclared as imported package name
previous declaration at ./main.go:4:2

We see that the problem of package name conflict does occur. How to solve this problem? Or use
the method of explicitly specifying the package name for the package in the package import path:

package main
import (
pkg1 "github.com/bigwhite/effective-go-book/chapter3-demo1/pkg/pkg1"
mypkg1 "github.com/bigwhite/effective-go-book/chapter3-demo2/pkg/pkg1"
)
func main() {
    
    
pkg1.Func1()
mypkg1.Func1()
}

The pkg1 above refers to the package under chapter3-demo1/pkg/pkg1, and mypkg1 refers to the package under chapter3-demo1/pkg/pkg1. At this point, the package name conflict problem is easily solved.

We have further understood the package import of Go language through experiments. Gopher should keep the following conclusions in mind:

● The Go compiler must use the source code of the package on which the compilation unit (a package) depends during the compilation process;

● The part after import in the package import statement at the head of the Go source code file is a path, and the last segment of the path is the directory name, not the package name;

● The package source code search path of the Go compiler consists of the basic search path and the package import path. When the two are combined, the compiler can determine the set of source code paths of all dependent packages of a package. This set constitutes the Go compiler source code search path space;

● The problem of package name conflicts between dependent packages of the same source code file in the same source code search path space can be solved by explicitly specifying the package name.