Networking with overlay networks (original) (raw)

This series of tutorials deals with networking for swarm services. For networking with standalone containers, seeNetworking with standalone containers. If you need to learn more about Docker networking in general, see theoverview.

This page includes the following tutorials. You can run each of them on Linux, Windows, or a Mac, but for the last one, you need a second Docker host running elsewhere.

• Use the default overlay network demonstrates how to use the default overlay network that Docker sets up for you automatically when you initialize or join a swarm. This network is not the best choice for production systems.
• Use user-defined overlay networks shows how to create and use your own custom overlay networks, to connect services. This is recommended for services running in production.
• Use an overlay network for standalone containersshows how to communicate between standalone containers on different Docker daemons using an overlay network.

These require you to have at least a single-node swarm, which means that you have started Docker and run docker swarm init on the host. You can run the examples on a multi-node swarm as well.

In this example, you start an alpine service and examine the characteristics of the network from the point of view of the individual service containers.

This tutorial does not go into operation-system-specific details about how overlay networks are implemented, but focuses on how the overlay functions from the point of view of a service.

Prerequisites

This tutorial requires three physical or virtual Docker hosts which can all communicate with one another. This tutorial assumes that the three hosts are running on the same network with no firewall involved.

These hosts will be referred to as manager, worker-1, and worker-2. Themanager host will function as both a manager and a worker, which means it can both run service tasks and manage the swarm. worker-1 and worker-2 will function as workers only,

If you don't have three hosts handy, an easy solution is to set up three Ubuntu hosts on a cloud provider such as Amazon EC2, all on the same network with all communications allowed to all hosts on that network (using a mechanism such as EC2 security groups), and then to follow theinstallation instructions for Docker Engine - Community on Ubuntu.

Walkthrough

Create the swarm

At the end of this procedure, all three Docker hosts will be joined to the swarm and will be connected together using an overlay network called ingress.

1. On manager. initialize the swarm. If the host only has one network interface, the --advertise-addr flag is optional.
   Make a note of the text that is printed, as this contains the token that you will use to join worker-1 and worker-2 to the swarm. It is a good idea to store the token in a password manager.
2. On worker-1, join the swarm. If the host only has one network interface, the --advertise-addr flag is optional.
3. On worker-2, join the swarm. If the host only has one network interface, the --advertise-addr flag is optional.
4. On manager, list all the nodes. This command can only be done from a manager.
   You can also use the --filter flag to filter by role:
5. List the Docker networks on manager, worker-1, and worker-2 and notice that each of them now has an overlay network called ingress and a bridge network called docker_gwbridge. Only the listing for manager is shown here:

The docker_gwbridge connects the ingress network to the Docker host's network interface so that traffic can flow to and from swarm managers and workers. If you create swarm services and do not specify a network, they are connected to the ingress network. It is recommended that you use separate overlay networks for each application or group of applications which will work together. In the next procedure, you will create two overlay networks and connect a service to each of them.

Create the services

1. On manager, create a new overlay network called nginx-net:
   You don't need to create the overlay network on the other nodes, because it will be automatically created when one of those nodes starts running a service task which requires it.
2. On manager, create a 5-replica Nginx service connected to nginx-net. The service will publish port 80 to the outside world. All of the service task containers can communicate with each other without opening any ports.

   Note

   Services can only be created on a manager.
   The default publish mode of ingress, which is used when you do not specify a mode for the --publish flag, means that if you browse to port 80 on manager, worker-1, or worker-2, you will be connected to port 80 on one of the 5 service tasks, even if no tasks are currently running on the node you browse to. If you want to publish the port usinghost mode, you can add mode=host to the --publish output. However, you should also use --mode global instead of --replicas=5 in this case, since only one service task can bind a given port on a given node.

3. Run docker service ls to monitor the progress of service bring-up, which may take a few seconds.
4. Inspect the nginx-net network on manager, worker-1, and worker-2. Remember that you did not need to create it manually on worker-1 andworker-2 because Docker created it for you. The output will be long, but notice the Containers and Peers sections. Containers lists all service tasks (or standalone containers) connected to the overlay network from that host.
5. From manager, inspect the service using docker service inspect my-nginxand notice the information about the ports and endpoints used by the service.
6. Create a new network nginx-net-2, then update the service to use this network instead of nginx-net:
7. Run docker service ls to verify that the service has been updated and all tasks have been redeployed. Run docker network inspect nginx-net to verify that no containers are connected to it. Run the same command fornginx-net-2 and notice that all the service task containers are connected to it.

   Note

   Even though overlay networks are automatically created on swarm worker nodes as needed, they are not automatically removed.

8. Clean up the service and the networks. From manager, run the following commands. The manager will direct the workers to remove the networks automatically.

Prerequisites

This tutorial assumes the swarm is already set up and you are on a manager.

Walkthrough

1. Create the user-defined overlay network.
2. Start a service using the overlay network and publishing port 80 to port 8080 on the Docker host.
3. Run docker network inspect my-overlay and verify that the my-nginxservice task is connected to it, by looking at the Containers section.
4. Remove the service and the network.

This example demonstrates DNS container discovery -- specifically, how to communicate between standalone containers on different Docker daemons using an overlay network. Steps are:

• On host1, initialize the node as a swarm (manager).
• On host2, join the node to the swarm (worker).
• On host1, create an attachable overlay network (test-net).
• On host1, run an interactivealpine container (alpine1) on test-net.
• On host2, run an interactive, and detached,alpine container (alpine2) on test-net.
• On host1, from within a session of alpine1, ping alpine2.

Prerequisites

For this test, you need two different Docker hosts that can communicate with each other. Each host must have the following ports open between the two Docker hosts:

• TCP port 2377
• TCP and UDP port 7946
• UDP port 4789

One easy way to set this up is to have two VMs (either local or on a cloud provider like AWS), each with Docker installed and running. If you're using AWS or a similar cloud computing platform, the easiest configuration is to use a security group that opens all incoming ports between the two hosts and the SSH port from your client's IP address.

This example refers to the two nodes in our swarm as host1 and host2. This example also uses Linux hosts, but the same commands work on Windows.

Walk-through

1. Set up the swarm.
   a. On host1, initialize a swarm (and if prompted, use --advertise-addrto specify the IP address for the interface that communicates with other hosts in the swarm, for instance, the private IP address on AWS):
   b. On host2, join the swarm as instructed above:
   If the node fails to join the swarm, the docker swarm join command times out. To resolve, run docker swarm leave --force on host2, verify your network and firewall settings, and try again.
2. On host1, create an attachable overlay network called test-net:

   Notice the returned NETWORK ID -- you will see it again when you connect to it from host2.

3. On host1, start an interactive (-it) container (alpine1) that connects to test-net:
4. On host2, list the available networks -- notice that test-net does not yet exist:
5. On host2, start a detached (-d) and interactive (-it) container (alpine2) that connects to test-net:

   Note

   Automatic DNS container discovery only works with unique container names.

6. On host2, verify that test-net was created (and has the same NETWORK ID as test-net on host1):
7. On host1, ping alpine2 within the interactive terminal of alpine1:
   The two containers communicate with the overlay network connecting the two hosts. If you run another alpine container on host2 that is not detached, you can ping alpine1 from host2 (and here we add theremove option for automatic container cleanup):
8. On host1, close the alpine1 session (which also stops the container):
9. Clean up your containers and networks:
   You must stop and remove the containers on each host independently because Docker daemons operate independently and these are standalone containers. You only have to remove the network on host1 because when you stopalpine2 on host2, test-net disappears.
   a. On host2, stop alpine2, check that test-net was removed, then remove alpine2:
   a. On host1, remove alpine1 and test-net:

• Host networking tutorial
• Standalone networking tutorial
• Macvlan networking tutorial