[go] getting started with go modules

Go modules yes Go The project of official propaganda in the language depends on the solution ,Go modules（ Formerly known as vgo） On Go1.11 Official release , stay Go1.14 You are ready to , And can be used in production （ready for production） 了 ,Go Officials also encourage all users to migrate from other dependency management tools to Go modules.

and Go1.14, It has been officially released recently ,Go  The official himself “ shout ” You use it ：

So in today's article , I will bring you Go modules Of “ The ultimate entry point ”, Welcome to discuss .

Go modules yes Go Formal official project management tools in the language ,Go modules（ Formerly known as vgo） On Go1.11 Official release , stay Go1.14 You are ready to , And can be used in production （ready for production） 了 , Encourage all users to migrate from other dependency management tools to Go modules.

What is? Go Modules

Go modules yes Go Language dependent solutions , Published on Go1.11, Grow up in Go1.12, Rich in Go1.13, Officially on Go1.14 Recommended for use in production .

Go moudles Currently integrated in Go In the tool chain , Once installed Go, It's natural to use Go moudles 了 , and Go modules The emergence of the Go1.11 Some of the most common controversial issues ：

1. Go Language has long been a problem of dependency management .
2. “ Eliminate ” The existing GOPATH The usage mode of .
3. Unify other dependency management tools in the community （ Provides migration capabilities ）.

GOPATH Those little bits of

We mentioned Go modules One of the problems solved by is “ Eliminate ” fall GOPATH, however GOPATH And what is it , Why is it Go1.11 Use before GOPATH, and Go1.11 After that, we will gradually recommend the use of Go modules, No longer recommend GOPATH What about the model of ？

GOPATH What is it?

Let's take a look at the first question ,GOPATH What is it? , We can enter the following command to view ：

$ go env
GOPATH="/Users/eddycjy/go"
...

We type in go env After the command line, you can see GOPATH The result of the variable , Let's go to the directory to check , as follows ：

go
├── bin
├── pkg
└── src
    ├── github.com
    ├── golang.org
    ├── google.golang.org
    ├── gopkg.in
    ....

GOPATH There are three subdirectories in the directory , Namely ：

• bin： Store the compiled binary file .
• pkg： Store precompiled target files , To speed up the subsequent compilation of the program .
• src： Store all .go File or source code . Writing Go Applications , Package and library time , In general, I will use $GOPATH/src/github.com/foo/bar The path to store .

So in use GOPATH In mode , We need to store the application code in a fixed $GOPATH/src Under the table of contents , And if it does go get To pull external dependencies will be automatically downloaded and installed to $GOPATH Under the table of contents .

Why abandon GOPATH Pattern

stay GOPATH Of  $GOPATH/src  Proceed under  .go  Storage of files or source code , We can call it GOPATH The pattern of , This mode , Looks like there's no problem , So why should we discard it , See the following reasons ：

• GOPATH There is no concept of version control in the mode , Have a fatal flaw , At least it will cause the following problems ：

  • In execution go get When , You can't convey any version of the message of expectation , That is to say, you can't know which version you are updating , You can't pull the specific version you want by specifying .
  • Running Go When it comes to applications , You can't guarantee that others are the same version of the third-party library you expect to rely on , That is to say, in the management of project dependency Library , You can't guarantee that everyone's dependent versions are consistent .
  • You can't handle v1、v2、v3 And so on different versions of the citation problem , because GOPATH The import path in the mode is the same , All are github.com/foo/bar.
• Go Language official from Go1.11 Start pushing Go modules（ The predecessor vgo）,Go1.13 It is no longer recommended to use GOPATH The usage mode of ,Go modules It's also becoming more stable , Therefore, there is no need to continue using the new project GOPATH Pattern .

stay GOPATH The product of the model

Go1 stay 2012 year 03 month 28 Promulgated by the , and Go1.11 Is in 2018 year 08 month 25 It was officially released today （ Data sources ：Github Tag）, In this gap time , did not Go modules This one thing , In the early days, it might be better to say , Because just released , There are not many people to use , So there's no obvious exposure , But later Go More and more people are using the language , What to do with that ？

At this time, a large number of dependency solutions are emerging in the community , flowers , It's hard to choose , This includes what we know as vendor Directory mode , And was once thought to be “ Official announcement ” Of dep This kind of dependency management tools .

What is it for? dep It's not becoming official Xuan , It's because of Russ Cox And Go Other members of the team are constantly discussing , Find out dep Some of the details of seem more and more unsuitable Go, So the authorities took another step proposal The way to advance , The result of the scheme is to release vgo（Go modules The forerunner of , Know that you can , There is no need to know more about ）, And it turns out to be what we see now Go modules, Also in the Go1.11 Officially entered Go The tool chain .

So it's not so much “ stay GOPATH The product of the model ”, Rather, history provides important lessons for the present , So it's happening Go modules.

Go Modules Basic use

I have a preliminary understanding of Go modules The past and the present , We officially entered Go modules Use , First, we'll create a Go modules Project （ In principle, the created directory should not be placed in GOPATH In ）.

The order provided

stay Go modules in , We can use the following command to operate ：

command	effect
go mod init	Generate go.mod file
go mod download	download go.mod All dependencies specified in the file
go mod tidy	Sort out existing dependencies
go mod graph	Look at the existing dependency structure
go mod edit	edit go.mod file
go mod vendor	All dependencies of the export project depend on vendor Catalog
go mod verify	Verify that a module has been tampered with
go mod why	See why you need to rely on a module

Environment variables provided

stay Go modules There are the following common environment variables , We can go through  go env  Command to check , as follows ：

$ go env
GO111MODULE="auto"
GOPROXY="https://proxy.golang.org,direct"
GONOPROXY=""
GOSUMDB="sum.golang.org"
GONOSUMDB=""
GOPRIVATE=""
...

GO111MODULE

Go Language provides GO111MODULE This environment variable is used as Go modules The switch of , It allows the following parameters to be set ：

• auto： As long as the project contains go.mod File enabled Go modules, Currently in Go1.11 to Go1.14 Is still the default value .
• on： Enable Go modules, Recommended settings , It will be the default in future versions .
• off： Ban Go modules, Setting... Is not recommended .

GO111MODULE Small history of

You may notice GO111MODULE This name compares “ strange ”, In fact, Go There are often such periodic variables in language , GO111MODULE The name stands for Go Language in 1.11 Version added , in the light of Module The variable of .

It's like Go1.5 At version time , Also released a system environment variable GO15VENDOREXPERIMENT, It's used to turn on vendor Directory Support , At that time, the default value is not on , Just as experimental. It follows in Go1.6 Also change the default value to on in version , And finally as official,GO15VENDOREXPERIMENT System variables are out of the stage of history .

But the future GO111MODULE This system environment variable also faces this problem , It will also be adjusted to the default value of on（ Once in the Go1.13 Think of it as on, And it's merged PR, But in the end, for various reasons, it changed back to auto）, Then take it. GO111MODULE Get rid of your support , We guess it will be in Go2 take GO111MODULE Get rid of it , Because if you remove GO111MODULE Support for , There will be compatibility issues .

GOPROXY

This environment variable is mainly used to set Go Module agent （Go module proxy）, Its function is to make Go In the subsequent pull module version can break away from the traditional VCS The way , Directly through the mirror site to quickly pull .

GOPROXY The default value of is ：https://proxy.golang.org,direct, There is a very serious problem , Namely  proxy.golang.org  It's not accessible at home , So this will directly jam your first step , So you have to turn on Go modules When the , At the same time set up domestic Go Module agent , Execute the following command ：

$ go env -w GOPROXY=https://goproxy.cn,direct

GOPROXY The value of is a comma in English “,” Segmented Go Module agent list , Allow setting up multiple module agents , Suppose you don't want to use , It can also be set to “off” , This will prohibit Go Use any... In subsequent operations Go Module agent .

direct What is it?

And in the value just set , We can see that there is “direct” identification , What does it do ？

actually “direct” Is a special indicator , Used to indicate Go Go back to the source address of the module version to retrieve （ such as GitHub etc. ）, Scenario as follows ： When the last... In the list of values Go Module agent returns 404 or 410 When it's wrong ,Go Automatically try the next... In the list , meet “direct” Time goes back to the source , That is, go back to the source address to grab , And meet EOF To terminate and throw something similar “invalid version: unknown revision...” Error of .

GOSUMDB

Its value is a Go checksum database, Used to pull the module version （ Whether it's from the origin or through Go module proxy Pull ） Ensure that the pulled module version data has not been tampered , If you find inconsistencies , That is, there may be tampering , Will immediately suspend .

GOSUMDB The default value is ：sum.golang.org, It's not accessible at home , however GOSUMDB Can be Go Module agent （ See ：Proxying a Checksum Database）.

So we can set up GOPROXY To solve , And the module agent we set up earlier  goproxy.cn  Can support agents  sum.golang.org, So the problem is to set GOPROXY after , You don't need to worry too much about .

In addition, if yes GOSUMDB The value of has a custom requirement , It supports the following formats ：

• Format 1：<SUMDB_NAME>+<PUBLIC_KEY>.
• Format 2：<SUMDB_NAME>+<PUBLIC_KEY> <SUMDB_URL>.

It can also be set to “off”, That is, prohibition Go Verify the module version in subsequent operations .

GONOPROXY/GONOSUMDB/GOPRIVATE

These three environment variables are used in the current project, which depends on the private module , For example, private ownership of your company git Warehouse , Or is it github Private library in , All belong to private modules , It's all set up , Otherwise, it will fail to pull .

More specifically , Is to rely on the GOPROXY designated Go Module agent or by GOSUMDB Appoint Go checksum database Can not access the module when the scene .

In general, it is recommended to set it directly GOPRIVATE, Its value will be taken as GONOPROXY and GONOSUMDB The default value of , So the best position is to use it directly GOPRIVATE.

And their values are all comma in English “,” Divided module path prefix , That is, you can set multiple , for example ：

$ go env -w GOPRIVATE="git.example.com,github.com/eddycjy/mquote"

After setting , The prefix for git.xxx.com and github.com/eddycjy/mquote All of the modules of will be considered as private modules .

If you don't want to reset every time , We can also use wildcards , for example ：

$ go env -w GOPRIVATE="*.example.com"

If you set it like this , All module paths are example.com Subdomain （ for example ：git.example.com） Will not pass through Go module proxy and Go checksum database, It should be noted that example.com In itself .

Turn on Go Modules

at present Go modules It's not on by default , therefore Go Language provides GO111MODULE This environment variable is used as Go modules The switch of , It allows the following parameters to be set ：

• auto： As long as the project contains go.mod File enabled Go modules, Currently in Go1.11 to Go1.14 Is still the default value .
• on： Enable Go modules, Recommended settings , It will be the default in future versions .
• off： Ban Go modules, Setting... Is not recommended .

If you're not sure what your current value is , It can be executed go env command , View results ：

$ go env
GO111MODULE="off"
...

If necessary GO111MODULE The value of , Recommend to pass go env Command to set ：

 $ go env -w GO111MODULE=on

But it should be noted that if the corresponding system environment variable has a value （ Settings have been made ）, The following warning message will appear ：warning: go env -w GO111MODULE=... does not override conflicting OS environment variable.

Or you can set system environment variables directly （ Write the corresponding .bash_profile Documents can also be ） To achieve this goal ：

$ export GO111MODULE=on

Initialize project

At the completion of Go modules After opening , We need to create a sample project to demonstrate , Execute the following command ：

$ mkdir -p $HOME/eddycjy/module-repo 
$ cd $HOME/eddycjy/module-repo

Then proceed Go modules The initialization , as follows ：

$ go mod init github.com/eddycjy/module-repo
go: creating new go.mod: module github.com/eddycjy/module-repo

In execution  go mod init  On command , We specified the module import path as  github.com/eddycjy/module-repo. Next we create... In the root directory of the project main.go file , as follows ：

package main

import (
    "fmt"
    "github.com/eddycjy/mquote"
)

func main() {
    fmt.Println(mquote.GetHello())
}

Then execute... At the root of the project  go get github.com/eddycjy/mquote  command , as follows ：

$ go get github.com/eddycjy/mquote 
go: finding github.com/eddycjy/mquote latest
go: downloading github.com/eddycjy/mquote v0.0.0-20200220041913-e066a990ce6f
go: extracting github.com/eddycjy/mquote v0.0.0-20200220041913-e066a990ce6f

see go.mod file

When initializing a project , Will generate a go.mod file , It's enabled Go modules The most important identification required for the project , It's also GO111MODULE The value is auto When the identification mark of , It describes the current project （ That is, the current module ） Meta information of , Each line begins with a verb .

After we just initialized and simply pulled , Let's look at... Again go.mod file , The basic contents are as follows ：

module github.com/eddycjy/module-repo

go 1.13

require (
    github.com/eddycjy/mquote v0.0.0-20200220041913-e066a990ce6f
)

For further explanation , We use the following analogy ：

module github.com/eddycjy/module-repo

go 1.13

require (
    example.com/apple v0.1.2
    example.com/banana v1.2.3
    example.com/banana/v2 v2.3.4
    example.com/pear // indirect
    example.com/strawberry // incompatible
)

exclude example.com/banana v1.2.4
replace example.com/apple v0.1.2 => example.com/fried v0.1.0 
replace example.com/banana => example.com/fish

• module： The module path used to define the current project .
• go： The... Used to identify the current module Go Language version , Value is the version when the module is initialized , So far, it's just a sign .
• require： Used to set a specific module version .
• exclude： Used to exclude a specific module version from use .
• replace： Used to replace one module version with another .

Besides, you will find  example.com/pear  There will be a indirect identification ,indirect The identity indicates that the module is indirectly dependent , That is, in the current application import In the sentence , No explicit references to this module were found , It's possible that you'll do it manually first  go get  It's pulled down , It's also possible that the modules you depend on depend on , There are several situations .

see go.sum file

After the first pull module dependency , There will be one more go.sum file , It details all module versions that the current project directly or indirectly depends on , And write down the version of those modules SHA-256 Hash value for Go In the future operation, ensure that the module versions that the project depends on will not be tampered with .

github.com/eddycjy/mquote v0.0.1 h1:4QHXKo7J8a6J/k8UA6CiHhswJQs0sm2foAQQUq8GFHM=
github.com/eddycjy/mquote v0.0.1/go.mod h1:ZtlkDs7Mriynl7wsDQ4cU23okEtVYqHwl7F1eDh4qPg=
github.com/eddycjy/mquote/module/tour v0.0.1 h1:cc+pgV0LnR8Fhou0zNHughT7IbSnLvfUZ+X3fvshrv8=
github.com/eddycjy/mquote/module/tour v0.0.1/go.mod h1:8uL1FOiQJZ4/1hzqQ5mv4Sm7nJcwYu41F3nZmkiWx5I=
...

We can see that there are two possible paths for a module ：

github.com/eddycjy/mquote v0.0.1 h1:4QHXKo7J8a6J/k8UA6CiHhswJQs0sm2foAQQUq8GFHM=
github.com/eddycjy/mquote v0.0.1/go.mod h1:ZtlkDs7Mriynl7wsDQ4cU23okEtVYqHwl7F1eDh4qPg=

h1 hash yes Go modules Will target module version of zip After opening the package , For all in package files in turn hash, And then put their hash Results according to the fixed format and algorithm hash value .

and h1 hash and go.mod hash both , Or exist at the same time , Or there is only go.mod hash. Under what circumstances does it not exist h1 hash Well , Is that when Go I think it will be omitted when I can't use a module version h1 hash, There will be no h1 hash, There is only go.mod hash The situation of .

View global cache

What we have just succeeded in will  github.com/eddycjy/mquote  Module pulled down , Its pull results are cached in  $GOPATH/pkg/mod and  $GOPATH/pkg/sumdb  Under the table of contents , And in the mod The directory will use  github.com/foo/bar  The format for storage , as follows ：

mod
├── cache
├── github.com
├── golang.org
├── google.golang.org
├── gopkg.in
...

It should be noted that only one copy of data of the same module version is cached , All other modules are shared . If you want to clean up all cached module version data , It can be executed  go clean -modcache  command .

Go Modules Under the go get Behavior

When pulling project dependencies , You will find that the pulling process is divided into three major steps , Namely finding（ Find out ）、downloading（ download ） as well as extracting（ extract ）, And the pull information is divided into three sections ：

It should be noted that , The version of commit Time is based on UTC Time zone shall prevail , Not the local time zone , At the same time, we will find that we  go get  The version pulled by the command is v0.0.0, This is because we are directly executing  go get -u  Acquired , No version information specified , from Go modules Choose by yourself according to internal rules .

go get The act of pulling

We just used  go get  Orders pull new dependencies , that  go get  What functions are provided , The commonly used pull commands are as follows ：

command	effect
go get	Pull to rely on , There will be a specific pull （ to update ）, It doesn't update the other modules it depends on .
go get -u	Update existing dependencies , Will force all other modules it depends on to be updated , Excluding itself .
go get -u -t ./...	Update all direct and indirect dependent module versions , Including the unit tests used in .

So I want to choose how the specific version should be implemented , as follows ：

command	effect
go get golang.org/x/[email protected]	Pull the latest version , If exist tag, Then priority should be given to .
go get golang.org/x/[email protected]	Pull master The latest in the branch commit.
go get golang.org/x/[email protected]	Pull tag by v0.3.2 Of commit.
go get golang.org/x/[email protected]	Pull hash by 342b231 Of commit, It will eventually be converted to v0.3.2.

go get Version selection for

Let's review what we pulled  go get github.com/eddycjy/mquote, As a result,  v0.0.0-20200220041913-e066a990ce6f, Against the above  go get  In terms of behavior , You may have some doubts , That is in  go get  Without specifying any version , What's the version selection rule , That's why  go get  What you pull is  v0.0.0, When will it pull the normal version number tags Well . In fact, there are two situations that need to be distinguished , as follows ：

1. The modules you pull have been released tags：

   • If there is only a single module , Then take the one with the largest major version number tag.
   • If there are multiple modules , Then calculate the corresponding module path , Take the one with the largest major version number tag（ Sub module of tag The module path of will have a prefix requirement ）

2. The module you pulled has not been released tags：

   • Default to the latest time of the main branch commit Of commithash.

Not released tags

So why do you pull  v0.0.0  Well , Because  github.com/eddycjy/mquote  Nothing has been released tag, as follows ：

So it defaults to the latest time of the main branch commit Of commit Time and commithash, That is to say  20200220041913-e066a990ce6f, In the second case .

There's a release tags

There are releases in the project tags Under the circumstances , There are many patterns , That is, there are only one module and multiple modules , We show it in multiple modules , Because the case of multiple modules already includes the use of a single module , Here's the picture ：

In this project , We played two in all tag, Namely ：v0.0.1 and module/tour/v0.0.1. You may be surprised at this time , Why fight  module/tour/v0.0.1  such “ strange ” Of tag, What's the purpose of this ？

It's actually Go modules Multiple modules under the same project tag The way of expression , Its main directory structure is ：

mquote
├── go.mod
├── module
│   └── tour
│       ├── go.mod
│       └── tour.go
└── quote.go

You can see in the  mquote  The root directory of this project has a go.mod file , And in the  module/tour  There is also a go.mod file , The corresponding relationship between module import and version information is as follows ：

tag	Module import path	meaning
v0.0.1	github.com/eddycjy/mquote	mquote Project v 0.0.1 edition
module/tour/v0.01	github.com/eddycjy/mquote/module/tour	mquote Sub modules under the project module/tour Of v0.0.1 edition

Import main modules and sub modules

Combined with the above , Pull the main module , Still carry out the following orders as usual ：

$ go get github.com/eddycjy/[email protected]
go: finding github.com/eddycjy/mquote v0.0.1
go: downloading github.com/eddycjy/mquote v0.0.1
go: extracting github.com/eddycjy/mquote v0.0.1

If you want to pull submodules , Execute the following command ：

$ go get github.com/eddycjy/mquote/module/[email protected]
go: finding github.com/eddycjy/mquote/module v0.0.1
go: finding github.com/eddycjy/mquote/module/tour v0.0.1
go: downloading github.com/eddycjy/mquote/module/tour v0.0.1
go: extracting github.com/eddycjy/mquote/module/tour v0.0.1

We will compare the pull of the main module and the sub module , You will find that the pull of the sub module will take one more step , It will find out first  github.com/eddycjy/mquote/module, Continue to calculate , Finally pull to  module/tour.

Go Modules Import path description of

Different versions of the import path

In the previous module pull and reference , You will find that our module import path is  github.com/eddycjy/mquote  and  github.com/eddycjy/mquote/module/tour, There doesn't seem to be anything special .

It's not , actually Go modules The main version number is v0 and v1 The version number is omitted , And the main version number is v2 And the above need to specify the main version number , Otherwise there will be conflict , Its tag The approximate correspondence with the module import path is as follows ：

tag	Module import path
v0.0.0	github.com/eddycjy/mquote
v1.0.0	github.com/eddycjy/mquote
v2.0.0	github.com/eddycjy/mquote/v2
v3.0.0	github.com/eddycjy/mquote/v3

simply , The main version number is v0 and v1 when , There is no need to include major version information in the module import path , And in the v1 After the version , That is to say v2 rise , You must add the major version number... At the end of the module's import path , You need to adjust the reference to the following format ：

import (
    "github.com/eddycjy/mquote/v2/example"
)

In addition, the main version number is ignored v0 and v1 It's mandatory （ Not an option ）, So each package has only one clear and standardized import path .

Why ignore v0 and v1 Major version number of

1. Ignore... In import path v1 The reason for the version is ： Consider that many developers create once they arrive v1 Version will never change the package , This is what the government encourages , Don't think all of these developers are unintentionally releasing v2 Version should be forced to have a clear v1 Version suffix , This will lead to v1 The version becomes “ The noise ” And it doesn't make sense .
2. Ignored... In import path v0 The reason for the version is ： According to the semantic version specification ,v0 There is no compatibility guarantee at all for these versions of . Need an explicit v0 Version identification doesn't help much to ensure compatibility .

Go Modules Semantic version control

We are constantly in Go Modules Version number is mentioned in the use of , In fact, it is called “ Semantic version ”, Suppose our version number is v1.2.3, as follows ：

Its version format is “ The major version number . Sub version number . Revision number ”, The increment rule of version number is as follows ：

1. The major version number ： When you do incompatible API modify .
2. Sub version number ： When you do a downward compatible feature add .
3. Revision number ： When you do a downward compatibility problem fix .

Suppose you are the first version number or special case , You can append version information to “ The major version number . Sub version number . Revision number ” Behind , As an extension , as follows ：

So far we have introduced Go modules Two types of version number supported , After we release the new version, type tag When , Need to follow , Otherwise, version numbers that do not follow the semantic version rules cannot be pulled .

Go Modules The minimum version selection of

Now we have a module , It's also released tag, But a module often depends on many other modules , And different modules are likely to depend on different versions of the same module , Here's the picture （ come from Russ Cox）：

In the above dependence , modular A Depends on the module B And modules C, And modules B Depends on the module D, modular C Depends on the module D and F, modular D And it depends on modules E, And different versions of the same module also depend on different versions of the corresponding module . So at this point Go modules How to choose the version , Which version is chosen ？

We according to the proposal Can be learned that ,Go modules The list of dependent versions of each module will be sorted out , You end up with a build list , Here's the picture （ come from Russ Cox）：

We see rough list and final list, The difference between the two is the repeated reference module D（v1.3、v1.4）, The final list uses modules D Of v1.4 edition , Main cause ：

1. Semantic version control ： Because the module D Of v1.3 and v1.4 Version change , All belong to the change of minor version number , And under the constraint of semantic version ,v1.4 Must be downward compatible v1.3 edition , Therefore, it is considered that there is no destructive change , It's compatible .
2. Specification of module import path ： The main version number is different , The import path of the module is different , So if there's an incompatibility , Its major version number will change , The import path of the module will naturally change , So it doesn't conflict with the basis of the first point .

go.sum Do you want to submit the documents

Theoretically go.mod and go.sum All documents should be submitted to your Git In the warehouse to .

Suppose we don't upload go.sum file , Will cause everyone to execute Go modules Relevant command , And a new go.sum, That is to say, it will pull back to the upstream , It may have been tampered with when pulling again , There will be a lot of security risks , Lost with benchmark version （ The first person to submit , Expected version ） Verification content of , therefore go.sum Documents are required to be submitted .