Dockerize Go Applications (original) (raw)

📥 This article is long. Download a PDF to read it later.

Introduction

While most Go applications compile to a single binary, web applications also ship with templates, assets and configuration files; these can get out of sync and cause faulty deployments.

Docker lets us create a self-contained image with everything our application needs to work. In this tutorial, you will learn how to deploy a Go web application with Docker, and how Docker can help improve your development workflow and deployment process.

Goals

By the end of this article, you will:

Prerequisites

For this tutorial, you will need:

You can find all the code for this tutorial in the golang-mathapp repository.

Understanding Docker

Docker helps you create a single deployable unit for your application. This unit, also known as a container, has everything the application needs to work. This includes the code (or binary), the runtime, the system tools and libraries.

Packing all the requirements into a single unit ensures an identical environment for the application, wherever it is deployed. It also helps to maintain identical development and production setups.

Containers also eliminate a whole class of issues caused by files being out of sync or due to subtle differences in the production environments.

Advantages over Virtual Machines

Containers offer similar resource allocation and isolation benefits as virtual machines. However, the similarity ends there.

A virtual machine needs its own guest operating system while a container shares the kernel of the host operating system. This means that containers are much lighter and need fewer resources. A virtual machine is, in essence, an operating system within an operating system. Containers, on the other hand, are just like any other application in the system. Basically, containers need fewer resources (memory, disk space, etc.) than virtual machines, and have much faster start-up times than virtual machines.

Benefits of Docker During Development

Some of the benefits of using Docker in development include:

Why Use Docker with a Go Web Application?

Most Go applications are simple binaries. This begs the question—why use Docker with a Go application? Some of the reasons to use Docker with Go include:

Creating a Simple Go Web Application

We’ll create a simple web application in Go for demonstration in this article. This application, which we’ll call MathApp, will:

Visiting /sum/3/6 will show a page with the result of adding 3 and 6. Likewise, visiting /product/3/6 will show a page with the product of 3 and 6.

In this article, we used the Beego framework. Note that you can use any framework (or none at all) for your application.

Final Directory Structure

Upon completion, the directory structure of MathApp will look like:

MathApp
    ├── Dockerfile
    ├── Dockerfile.production
    └── src
        ├── conf
        │   └── app.conf    
        ├── go.mod    
        ├── go.src
        ├── main.go
        ├── main_test.go    
        ├── vendor
        └── views
            ├── invalid-route.html
            └── result.html

The main application file is main.go, located at the src directory. This file contains all the functionality of the app. Some of the functionality from main.go is tested using main_test.go.

The views folder contains the view files invalid-route.html and result.html. The configuration file app.conf is placed in the conf folder. Beego uses this file to customize the application.

Create the GitHub Repository

We’ll use Go mod, the official module manager, to handle Go modules in a portable way without having to worry about GOPATH.

We’ll start by creating a GitHub repository and cloning it to your machine.

Use the repository name to initialize the project:

$ mkdir src
$ cd src
$ export GOFLAGS=-mod=vendor
$ export GO111MODULE=on
$ go mod init github.com/YOUR_GITHUB_USER/YOUR_REPOSITORY_NAME 
# (example: go mod init github.com/tomfern/go-web-docker)

From now on, we can use these commands:

$ go mod download
$ go mod vendor
$ go mod verify

To download the required dependencies in the vendor/ folder (instead of downloading them in the GOROOT, this will come in handy later).

Application File Contents

Before continuing, let’s create the file structure:

$ mkdir conf views

The main application file (main.go) contains all the application logic. The contents of this file are as follows:

// main.go

package main

import (
    "strconv"
    
    "github.com/astaxie/beego"
)


func main() {
    /* This would match routes like the following:
       /sum/3/5
       /product/6/23
       ...
    */
    beego.Router("/:operation/:num1:int/:num2:int", &mainController{})
    beego.Run()
}

type mainController struct {
    beego.Controller
}


func (c *mainController) Get() {

    //Obtain the values of the route parameters defined in the route above    
    operation := c.Ctx.Input.Param(":operation")
    num1, _ := strconv.Atoi(c.Ctx.Input.Param(":num1"))
    num2, _ := strconv.Atoi(c.Ctx.Input.Param(":num2"))

    //Set the values for use in the template
    c.Data["operation"] = operation
    c.Data["num1"] = num1
    c.Data["num2"] = num2
    c.TplName = "result.html"

    // Perform the calculation depending on the 'operation' route parameter
    switch operation {
    case "sum":
        c.Data["result"] = add(num1, num2)
    case "product":
        c.Data["result"] = multiply(num1, num2)
    default:
        c.TplName = "invalid-route.html"
    }
}

func add(n1, n2 int) int {
    return n1 + n2
}

func multiply(n1, n2 int) int {
    return n1 * n2
}

In your application, this might be split across several files. However, for the purpose of this tutorial, I like to have everything in one place.

Test File Contents

The main.go file has some functions which need to be tested. The tests for these functions can be found in main_test.go. The contents of this file are as follows:

// main_test.go

package main

import "testing"

func TestSum(t *testing.T) {
    if add(2, 5) != 7 {
        t.Fail()
    }
    if add(2, 100) != 102 {
        t.Fail()
    }
    if add(222, 100) != 322 {
        t.Fail()
    }
}

func TestProduct(t *testing.T) {
    if multiply(2, 5) != 10 {
        t.Fail()
    }
    if multiply(2, 100) != 200 {
        t.Fail()
    }
    if multiply(222, 3) != 666 {
        t.Fail()
    }
}

Testing your application is particularly useful if you want to do Continuous Deployment. If you have adequate testing in place, then you can make stress-free deployments anytime, any day of the week.

View Files Contents

The view files are HTML templates; these are used by the application to display the response to a request. The content of views/result.html is as follows:

<!-- views/result.html -->
<!doctype html>
<html>
    <head>
        <title>MathApp - {{.operation}}</title>
    </head>
    <body>
        The {{.operation}} of {{.num1}} and {{.num2}} is {{.result}}
    </body>
</html>

The content of views/invalid-route.html is as follows:

<!-- invalid-route.html -->
<!doctype html>
<html>
    <head>
        <title>MathApp</title>
        <meta name="viewport" content="width=device-width, initial-scale=1">
        <meta charset="UTF-8">
    </head>

    <body>
        Invalid operation
    </body>
</html>

Configuration File Contents

The conf/app.conf file is read by Beego to configure the application. Its content is as follows:

appname = mathapp
runmode = "dev"
httpport = 8010

In this file:

Finally, if you haven’t yet done so, install the Go modules with:

$ go mod download
$ go mod vendor
$ go mod verify

Using Docker During Development

This section will explain the benefits of using Docker during development, and walk you through the steps required to use Docker in development.

Configuring Docker for Development

We’ll use a Dockerfile to configure Docker for development. The setup should satisfy the following requirements for the development environment:

Step 1 – Creating the Dockerfile

Go back to the top level of your project:

$ cd ..

The following Dockerfile should satisfy the above requirements.

FROM golang:1.18-bullseye

RUN go install github.com/beego/bee/v2@latest

ENV GO111MODULE=on
ENV GOFLAGS=-mod=vendor

ENV APP_HOME /go/src/mathapp
RUN mkdir -p "$APP_HOME"

WORKDIR "$APP_HOME"
EXPOSE 8010
CMD ["bee", "run"]

The first line:

FROM golang:1.18-bullseye

References the official image for Go as the base image. This image comes with Go 1.18 pre-installed.

The second line:

RUN go install github.com/beego/bee/v2@latest

Installs the bee tool globally (Docker commands run as root by default), which will be used to live-reload our code during development.

Next, we configure the environment variables for Go modules:

ENV GO111MODULE=on
ENV GOFLAGS=-mod=vendor

The next lines:

ENV APP_HOME /go/src/mathapp
RUN mkdir -p "$APP_HOME"
WORKDIR "$APP_HOME"

Creates a folder for the code and makes it active.

The next to last line tells Docker that port 8010 is of interest.

EXPOSE 8010

The final line:

CMD ["bee", "run"]

Uses the bee command to start our application.

Step 2 – Building the Image

Once the Docker file is created, run the following command to create the image:

$ docker build -t mathapp-development .

Executing the above command will create an image named mathapp:

This command can be used by everyone working on this application. This will ensure that an identical development environment is used across the team.

To see the list of images on your system, run the following command:

$ docker images

Note that the exact names and number of images might vary. However, you should see at least the golang and mathapp images in the list:

REPOSITORY               TAG            IMAGE ID            CREATED                 SIZE
golang                   1.18           25c4671a1478        2 weeks ago             809MB
mathapp-development      latest         8ae092824585        60 seconds ago          838MB

Step 3 – Running the Container

Once you have mathapp, you can start a container with:

$ docker run -it --rm -p 8010:8010 -v $PWD/src:/go/src/mathapp mathapp-development

Let’s break down the above command to see what it does.

Executing the above command starts the Docker container. This container exposes your application on port 8010. It also rebuilds your application automatically whenever you make a change. You should see the following output in your console:

______
| ___ \
| |_/ /  ___   ___
| ___ \ / _ \ / _ \
| |_/ /|  __/|  __/
\____/  \___| \___| v2.0.2
2022/05/10 13:39:29 INFO     ▶ 0003 Using 'mathapp' as 'appname'
2022/05/10 13:39:29 INFO     ▶ 0004 Initializing watcher...
2020/03/17 14:43:24.912 [I] [asm_amd64.s:1373]  http server Running on http://:8010

To check the setup, visit http://localhost:8010/sum/4/5 in your browser. You should see something similar to the following:

Note: This assumes that you’re working on your local machine.

To try the live-reload feature, make a modification in any of the source files. For instance, edit src/main.go, replace this line:

c.Data["operation"] = operation

To something like this:

c.Data["operation"] = "real " + operation

Bee should pick up the change, even inside the container, and reload the application seamlessly:

______
| ___ \
| |_/ /  ___   ___
| ___ \ / _ \ / _ \
| |_/ /|  __/|  __/
\____/  \___| \___| v2.0.2
2022/05/10 13:39:29 INFO     ▶ 0003 Using 'mathapp' as 'appname'
2022/05/10 13:39:29 INFO     ▶ 0004 Initializing watcher...
2022/05/10 13:39:29 INFO.   [asm_amd64.s:1373]  http server Running on http://:8010

Now reload the page on the browser to see the modified message:

Using Docker in Production

This section will explain how to deploy a Go application in a Docker container. We will use Semaphore to do the following:

Creating a Dockerfile for Production

We’ll write a new Dockerfile to create a complete, self-contained image; without external dependencies.

Enter the following contents in a new file called Dockerfile.production:

# Dockerfile.production

FROM registry.semaphore.io/golang:1.18 as builder

ENV APP_HOME /go/src/mathapp

WORKDIR "$APP_HOME"
COPY src/ .

RUN go mod download
RUN go mod verify
RUN go build -o mathapp

FROM registry.semaphore.io/golang:1.18

ENV APP_HOME /go/src/mathapp
RUN mkdir -p "$APP_HOME"
WORKDIR "$APP_HOME"

COPY src/conf/ conf/
COPY src/views/ views/
COPY --from=builder "$APP_HOME"/mathapp $APP_HOME

EXPOSE 8010
CMD ["./mathapp"]

Let’s take a detailed look at what each of these commands does. The first command:

FROM registry.semaphore.io/golang:1.18 as builder

Tells us this is a multi-stage build; it defines an intermediate image that will only have one job: compile the Go binary.

You might have noticed that we’re not pulling the image from Docker Hub, the default image registry. Instead, we’re using the Semaphore Docker Registry, which is more convenient, faster, and pulls don’t count against your Docker Hub rate limits.

The following commands:

ENV APP_HOME /go/src/mathapp

WORKDIR "$APP_HOME"
COPY src/ .

Creates the application folder for the app and copies the source code.

The last commands in the intermediate image download the modules and build the executable:

RUN go mod download
RUN go mod verify
RUN go build -o mathapp

Next comes the final and definitive container, where we will run the services.

FROM registry.semaphore.io/golang:1.18

We use the COPY command to copy files into the image, the --from argument let us copy the generated binary from the builder stage.

COPY src/conf/ conf/
COPY src/views/ views/
COPY --from=builder <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>A</mi><mi>P</mi><msub><mi>P</mi><mi>H</mi></msub><mi>O</mi><mi>M</mi><mi>E</mi><mi mathvariant="normal">/</mi><mi>m</mi><mi>a</mi><mi>t</mi><mi>h</mi><mi>a</mi><mi>p</mi><mi>p</mi></mrow><annotation encoding="application/x-tex">APP_HOME/mathapp </annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathnormal">A</span><span class="mord mathnormal" style="margin-right:0.13889em;">P</span><span class="mord"><span class="mord mathnormal" style="margin-right:0.13889em;">P</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.08125em;">H</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord mathnormal" style="margin-right:0.05764em;">OME</span><span class="mord">/</span><span class="mord mathnormal">ma</span><span class="mord mathnormal">t</span><span class="mord mathnormal">ha</span><span class="mord mathnormal">pp</span></span></span></span>APP_HOME

We finalize by exposing the port and starting the binary:

EXPOSE 8010
CMD ["./mathapp"]

To build the deployment image:

$ docker build -t mathapp-production -f Dockerfile.production .

You can run it with:

$ docker run -it -p 8010:8010 mathapp-production

Notice that we don’t need to map any directories, as all the source files are included in the container.

Continuous Integration with Semaphore

Docker is a great solution to package and deploy Go applications. The only downside is the additional steps required to build and test the image. This hurdle is easily is best dealt with Continuous Integration and Continuous Delivery (CI/CD).

A Continuous Integration (CI) platform can test our code on every iteration, on every push and every merge. Developers adopting CI no longer have to fear of merging branches, nor be anxious about release day. In fact, CI lets developers merge all the time and make safe releases any day of the week. A good CI setup will run a series of comprehensive tests, like the ones we prepared so far, to weed out any bugs.

Once the code is ready, we can extend our CI setup with Continuous Delivery (CD). CD can prepare and build the Docker images, leaving them ready to deploy at any time.

Push the Code to GitHub

Let’s push our modifications to GitHub:

# Dependency directories (remove the comment below to include it)
vendor/

# Build artifact
src/mathapp
$ git add Dockerfile*
$ git add src
$ git add .gitignore
$ git commit -m "initial commit"
$ git push origin master

Adding the Repository to Semaphore

We can add CI to our project for free in just a few minutes:

Create new project

Grab your repository

create a Dockerize Go workflow

You’ll get the Workflow Editor. Here’s an overview of how it works:

Semaphore workflow editor

Coming back to our setup. The started workflow expects the code at the project’s root, but our code is inside the src directory so we need to make a small modification:

sem-version go 1.18
export GO111MODULE=on
export GOPATH=~/go
export PATH=/home/semaphore/go/bin:$PATH
checkout
cd src
go get ./...
go test ./...
go build -v .

Go test commands

Switching GO version

If all goes well, after a few seconds the job should be completed without errors.

Enhancing the CI Pipeline

In this section, we’ll modify the pipeline so that:

To get started, click on the Edit Workflow button, then:

  1. Click on the block. We’ll completely replace its contents.
  2. Change the name of the block and the job to “Install”.
  3. Type the following content in the Job command box:
sem-version go 1.18
export GO111MODULE=on
export GOPATH=~/go
export PATH=/home/semaphore/go/bin:$PATH
checkout
cd src
cache restore vendor-$SEMAPHORE_GIT_BRANCH-$(checksum go.mod),vendor-$SEMAPHORE_GIT_BRANCH,vendor-master
go mod vendor
cache store vendor-$SEMAPHORE_GIT_BRANCH-$(checksum go.mod),vendor-$SEMAPHORE_GIT_BRANCH,vendor-master vendor

Build code before building a Dockerize Go pipeline

I think this is a good opportunity to learn about the Semaphore toolbox of built-in commands:

Let’s go back to our pipeline:

  1. Use the + Add Block dotted line button to create a new block.
  2. Call the block and the job “Test”.
  3. Open the Environment Variables section and create the GO111MODULE and GOFLAGS variables like we did on the previous block.
  4. Open the Prologue section, which executed before each job in the block, and type the following commands:
sem-version go 1.18
export GO111MODULE=on
export GOPATH=~/go
export PATH=/home/semaphore/go/bin:$PATH
checkout
cd src
cache restore vendor-$SEMAPHORE_GIT_BRANCH-$(checksum go.mod),vendor-$SEMAPHORE_GIT_BRANCH,vendor-master
  1. Type the following command in the job:
go test ./...

Test code before building a Dockerize Go pipeline

  1. Click on Run the Workflow and Start to try the updated pipeline.

Building the Docker Image

So far all we did enters in the Continuous Integration category, the natural next stage is to pack the application in a Docker container.

We’ll create a new delivery pipeline to:

First, we have to tell Semaphore how to connect to Docker Hub:

  1. On the account menu, click on Settings:

Settings menu

  1. Click on Secrets and then Create New Secret.
  2. Create two variables for your Docker Hub username and password:
    • DOCKER_USENAME = YOUR DOCKER USERNAME
    • DOCKER_PASSWORD = YOU DOCKER PASSWORD

Docker Hub secret to Dockerize Go

  1. Click on Save.

Going back to the pipeline:

  1. Click on Edit Workflow.
  2. Use the +Add First Promotion button to create a new linked pipeline:

Add a promotion

  1. Change the name of the pipeline to “Dockerize”
  2. Check Enable automatic promotion. You can set conditions to trigger the pipeline here:

Dockerize Go promotion

  1. Click +Add Block. We’ll call the new block “Build”
  2. Open the Secrets section and check the dockerhub box. This will import the variables we created earlier into the jobs in the block:

New block

  1. Type the following commands in the job:
checkout
echo "$DOCKER_PASSWORD" | docker login  --username "$DOCKER_USERNAME" --password-stdin
docker pull $DOCKER_USERNAME/mathapp-production:latest
docker build -f Dockerfile.production --cache-from <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>D</mi><mi>O</mi><mi>C</mi><mi>K</mi><mi>E</mi><msub><mi>R</mi><mi>U</mi></msub><mi>S</mi><mi>E</mi><mi>R</mi><mi>N</mi><mi>A</mi><mi>M</mi><mi>E</mi><mi mathvariant="normal">/</mi><mi>m</mi><mi>a</mi><mi>t</mi><mi>h</mi><mi>a</mi><mi>p</mi><mi>p</mi><mo>−</mo><mi>p</mi><mi>r</mi><mi>o</mi><mi>d</mi><mi>u</mi><mi>c</mi><mi>t</mi><mi>i</mi><mi>o</mi><mi>n</mi><mo>:</mo><mi>l</mi><mi>a</mi><mi>t</mi><mi>e</mi><mi>s</mi><mi>t</mi><mo>−</mo><mi>t</mi></mrow><annotation encoding="application/x-tex">DOCKER_USERNAME/mathapp-production:latest -t </annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathnormal" style="margin-right:0.02778em;">D</span><span class="mord mathnormal" style="margin-right:0.07153em;">OC</span><span class="mord mathnormal" style="margin-right:0.07153em;">K</span><span class="mord mathnormal" style="margin-right:0.05764em;">E</span><span class="mord"><span class="mord mathnormal" style="margin-right:0.00773em;">R</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0077em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.10903em;">U</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord mathnormal" style="margin-right:0.10903em;">SERN</span><span class="mord mathnormal">A</span><span class="mord mathnormal" style="margin-right:0.05764em;">ME</span><span class="mord">/</span><span class="mord mathnormal">ma</span><span class="mord mathnormal">t</span><span class="mord mathnormal">ha</span><span class="mord mathnormal">pp</span><span class="mspace" style="margin-right:0.2222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right:0.2222em;"></span></span><span class="base"><span class="strut" style="height:0.8889em;vertical-align:-0.1944em;"></span><span class="mord mathnormal">p</span><span class="mord mathnormal">ro</span><span class="mord mathnormal">d</span><span class="mord mathnormal">u</span><span class="mord mathnormal">c</span><span class="mord mathnormal">t</span><span class="mord mathnormal">i</span><span class="mord mathnormal">o</span><span class="mord mathnormal">n</span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">:</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:0.7778em;vertical-align:-0.0833em;"></span><span class="mord mathnormal" style="margin-right:0.01968em;">l</span><span class="mord mathnormal">a</span><span class="mord mathnormal">t</span><span class="mord mathnormal">es</span><span class="mord mathnormal">t</span><span class="mspace" style="margin-right:0.2222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right:0.2222em;"></span></span><span class="base"><span class="strut" style="height:0.6151em;"></span><span class="mord mathnormal">t</span></span></span></span>DOCKER_USERNAME/mathapp-production:latest .
docker push $DOCKER_USERNAME/mathapp-production:latest

Dockerize Go block

  1. Click on Run the Workflow and Start.
  2. Once the first two blocks are done, click on the Promote button:

Dockerize Go promotion

Wait a few seconds until the Dockerize pipeline is done:

Dockerize Go application pipeline

Check your Docker Hub repositories, you should find the new image, ready to use:

Dockerize Go application image

Finally, pull and test the new image in your machine:

$ docker pull YOUR_DOCKERHUB_USERNAME/mathapp-production
$ docker run -it -p 8010:8010 YOUR_DOCKERHUB_USERNAME/mathapp-production

What’s Next

Docker opens up the possibilities for deployments:

Heroku:

Kubernetes:

Conclusion

In this tutorial, we learned how to create a Docker container for a Go application and prepare a Docker container using Semaphore.

You should now be ready to use Docker to simplify the deployment of your next Go application. If you have any questions, feel free to post them in the comments below.

P.S. Want to continuously deliver your applications made with Docker? Check out Semaphore’s Docker support.

Read next:

📥 This article is long. Download a PDF to read it later.