Welcome To Golang By Example

Packages and Modules in Go (Golang) – Part 2

admin

This is the  chapter 5 of the golang comprehensive tutorial series. Refer to this link for other chapters of the series – Golang Comprehensive Tutorial Series

Next Tutorial – Variables
Previous Tutorial – Packages and Modules – Part 1

Now let’s check out the current tutorial. Below is the table of contents for current tutorial.

Overview

In the last tutorial we learn about package in detail and overview of modules.

In this tutorial we will focus on modules

Types of Modules

We learn that module is a directory containing nested go packages. So essentially module can be treated as a package only that contains nested packages. We have seen in the package tutorial can a package can be either an executable package or utility package (non-executable). Similar to package, modules can be of two types.

To create a executable for a module  (Only for module with main package)

Package vs Module

As per module definition, it is a directory containing a collection of nested and related go packages go.mod at its root.  The go.mod file defines the

Modules provides

Also in addition to go.mod file go also keeps a go.sum file which contains the cryptographic hash of bits of all project’s dependent modules. This to make validate that your project’s dependent modules are not changed.

The behaviour of packages inside a module is same as earlier. So whatever applied for a package also applies now. There is no change in that. However a collection of packages can be called as module when there is a requirement to version them separately. Also when it is common piece of code and you want to share that code across multiple projects. 

Add a dependency to your project

Let’s explore some ways of adding dependency to your project

Before looking at each of the ways, again let’s create a module first

go mod init sample.com/learn

Directly adding it to the go.mod file

We can  add direct dependency to the go.mod file too. Let’s do that

Add below dependency to the go.mod file

require github.com/pborman/uuid v1.2.1

With this dependency go.mod file will look like below

module sample.com/learn

go 1.14

require github.com/pborman/uuid v1.2.1

Now we need to download the newly added dependency as well. Fo that we can use the below command

go mod download

This command will download the github.com/pborman/uuid module as well all its dependencies. Also, it will update the go.sum file with the checksum and version of all direct and indirect dependencies. go build as well as go install also will download the dependencies and also build the binary. go run will also download and run the binary as well. go mod download command is used when you want to pre-download the dependencies without build or running it.

Do a go get

Simply doing a go get will also the add the dependency in the go.mod file. Remove the uuid dependency we added above from go.mod file and clean up go.sum file. Now run below command

export GO111MODULE=on
go get github.com/pborman/uuid

Now check the contents of go.mod file. Do a cat go.mod

module sample.com/learn

go 1.14

require github.com/pborman/uuid v1.2.1 //indirect

The dependency will be marked as //indirect as it is not being used in any of the source files. Once you do a go build after using this in the source files, the //indirect will be removed automatically by go. Also it will update the go.sum file with the checksum and version of all direct and indirect dependencies.

Add the dependency to your source code and do a go mod tidy

Basically go mod tidy command makes sure that your go.mod files reflects the dependencies that you have actually used in your project. When we run go mod tidy command then it will do two things

Let’s see an example. Create a module with an import path as “sample.com/learn“

go mod init sample.com/learn

Let’s create a file named uuid.go in the same directory with below contents

uuid.go

package main

import (
	"fmt"
	"strings"

	"github.com/pborman/uuid"
)

func main() {
	uuidWithHyphen := uuid.NewRandom()
	uuid := strings.Replace(uuidWithHyphen.String(), "-", "", -1)
	fmt.Println(uuid)
}

Notice that we have imported the dependency in the uuid.go as well

"github.com/pborman/uuid"

Let’s run the below command

go mod tidy

This command will download all the dependencies that are required in your source files and update go.mod file with that dependency. After running this command let’s now let’s again examine the contents of go.mod file. Do a cat go.mod

module sample.com/learn

go 1.14

require github.com/pborman/uuid v1.2.1

Adding a vendor directory

If you want to vendor your dependencies,  then below command can be used to achieve the same

go mod vendor

It will create a vendor directory inside your project directory. You can also check in the vendor directory to your VCS (Version Control System). This becomes useful in sense that none of the dependency needs to be downloaded at run time as it is already present in the vendor folder checked into VCS

Module Import Path

We have already seen that module import path is the prefix path that is used to import all packages within that module

There can be three cases that decide what import path name can be used with modules.

The module is a utility module and you plan to publish your module

If you plan to publish your module then the module name should match the URL of the repo which host that module. Go tries to download dependencies from the VCS using the same import path of the module.

The module is a utility module and you don’t plan to publish your module

This is the case when you only mean to use the utility module locally only. In this case the import path can be anything.

The module is a executable module

In this case also module import path can be anything. The module import path can be a non-url even if you plan to commit your module into VCS as it will not be used by any other module

However it is a good practice to use meaningful import path while creating module

Importing package within same module

Any package within the same module can be imported using the import path of module + directory containing that package. To illustrate lets create a module

go mod init sample.com/learn

main.go

package main

import (
	"fmt"
	"sample.com/learn/math"
)

func main() {
	fmt.Println(math.Add(1, 2))
}

math/math.go

package math

func Add(a, b int) int {
    return a + b
}

See how we have imported the math package in the main.go file

"sample.comlearn/math"

Here the import path is import path of module which is sample.com/learn +  directory containing the package which is math. Hence “sample.com/learn/math” . Packages in nested directory can also be imported in the same way. The way it works is that since the prefix is the module import path, hence go will know that you are trying to import from the same module. So it will directly refer it instead of downloading it.

Importing package from different module locally

There are cases when we want to import a module which is present locally. Let’s understand how we can import such a module. But first, we have to create a module that can be used by others and then import it into the other module. For that let’s create two modules

school module will be calling code of the sample.com/math module

Let’s first create the sample.com/math module which will be used by school module

go mod init sample.com/math
package math

func Add(a, b int) int {
	return a + b
}

Now let’s create the school module

go mod init school
module school

go 1.14

replace sample.com/math => ../math
package main

import (
	"fmt"
	"sample.com/math"
)

func main() {
	fmt.Println(math.Add(2, 4))
}

Now do a go run

go run school.go

It is able to call the Add function of the sample.com/math  module and correctly gives the output as 6.

Also it will update the go.mod with version information of the sample.com/math module

module school

go 1.14

replace sample.com/math => ../math

require sample.com/math v0.0.0-00010101000000-000000000000

Selecting the version of library

To understand how does GO’s approach while selecting the version of the library of which two versions are specified in the go.mod file, we have to first understand Semantic Versioning

Semantic Versioning is comprised of three parts separated by dots. Below is the format for versioning.

v{major_version}.{minor_version}.{patch_version}

where

Now there can be two cases

Let’s see what approach does go follows in the above two cases

Differ in minor or patch version

Go follows the minimum version policy approach while selecting the version of the library of which two versions are specified in the go.mod file which differ only in their minor or patch version.

For example in case you are using the two versions of same library which are

1.2.0

and

1.3.0

then go will choose 1.3.0 as it is the latest version.

Differ in major version

Go treats the major version as a different module itself. Now, what does that means? This essentially means that the import path will have a major version as its suffix. Let’s take the example of any go library with VCS as github.com/sample. Let’s latest semantic version is

v8.2.3

Then the go.mod file will like below

module github.com/sample/v8

go 1.13

..

It has major version in its import path. So any library which is using this sample library have to import it like

import "github.com/sample/v8"

If in future v9 version is released than it has to be imported in the application like

import "github.com/sample/v9"

Also the library will change its go.mod file to reflect the v9 major version

module github.com/sample/v9

What it essentially allows is to use different major version of the same library to be used within same go application.  We can also give meaningful names when different major version of the same library is imported in the same application. For eg

import sample_v8 "github.com/sample/v8"
import sample_v9 "github.com/sample/v9"

This is also known as Semantic Import Versioning

Also note that

Also for the same reason when you update a specific module using

go get -u

then it will only upgrade to the latest minor version or patch version whichever applicable. For example let’s say the current version used by an application is

v1.1.3

Also let’s say we have below versions available

v1.2.0
v2.1.0

Then when we run

go get

then it will update to

v1.2.0

The reason is because go get will only update the minor or patch version but never the major version as go treats major version of a module as a different module entirely.

To upgrade the major version, specify that  upgraded dependency explicitly  in the go.mod file or do a go get of that version.

Also couple of points to note about upgrading module

go get -u=patch 
go get dependency@version
go get @commit_number
go get ./...

go mod command

Below are some of the options for the go mod command.

go help mod edit

go help mod editFor eg below are some editing flags available

  1. -fmt flag will format the go.mod file. It will not make any other change
  2. -module flag can be used to set the module’s import path
go mod why sample.com/math

then below will be the output

# sample.com/math
school
sample.com/math

The output illustrates that the sample.com/math package is at one distance in the graph from main module which is school here.

Direct vs Indirect Dependencies in go.mod file

A direct dependency is the dependency which the module directly imports . An indirect dependency is the dependency which are imported by module’s direct dependencies. Also, any dependency that is mentioned in the go.mod file but not imported in any of the source files of the module is also treated as an indirect dependency.

go.mod file only records the direct dependency.However it may record an indirect dependency in below cases

go.sum will record the checksum of direct and indirect dependencies.

Example of Indirect Dependencies in go.mod file

Let’s understand it with an example. For that let’s first create a module

git mod init sample.com/learn

Let’s add colly lib version v1.2.0 as a dependency in the go.mod file. colly version v1.2.0 doesn’t have a go.mod file.

module sample.com/learn

go 1.14

require github.com/gocolly/colly v1.2.0

Now create a file learn.go

package main

import (
	"github.com/gocolly/colly"
)

func main() {
	_ = colly.NewCollector()
}

Now do a go build. Since colly version v1.2.0 doesn’t have a go.mod file , all dependencies required by colly will be added to the go.mod file with //indirect as suffix. Do a go build. Now check the go.mod file. You will see below contents of the file

module learn

go 1.14

require (
	github.com/PuerkitoBio/goquery v1.6.0 // indirect
	github.com/antchfx/htmlquery v1.2.3 // indirect
	github.com/antchfx/xmlquery v1.3.3 // indirect
	github.com/gobwas/glob v0.2.3 // indirect
	github.com/gocolly/colly v1.2.0
	github.com/kennygrant/sanitize v1.2.4 // indirect
	github.com/saintfish/chardet v0.0.0-20120816061221-3af4cd4741ca // indirect
	github.com/temoto/robotstxt v1.1.1 // indirect
	golang.org/x/net v0.0.0-20201027133719-8eef5233e2a1 // indirect
	google.golang.org/appengine v1.6.7 // indirect
)

All other dependencies are suffixed by //indirect. Also check that all direct and indirect dependencies will be recorded in the go.sum file.

Conclusion

This is all about packages and modules in golang. Hope you have liked this article. Please share feedback/mistakes/improvements in comments

Next Tutorial – Variables
Previous Tutorial – Packages and Modules – Part 1