Docker Deep Drive 2020 [2]


9. Deploying apps with Docker Compose

How to deploy multi-container applications using Docker Compose

Methods for Deploying container apps:

  • Docker Compose:
    • Deploys and manages multi-container applications on Docker nodes running in single-engine mode
  • Docker Stacks:
    • Deploys and manage multi-container apps on Docker nodes running in swarm mode

Micro-services : Modern cloud-native apps are made of multiple smaller services that interact to form a useful app. We call this pattern “micro-services”

Example (each of below is one service):

  • Web front-end
  • Ordering
  • Catalog
  • Backend database
  • Logging
  • Authentication
  • Authorization

Docker Compose lets you describe an entire app in a single declarative configuration file, and deploy it with a single command

❯ docker-compose --version
docker-compose version 1.27.4, build 40524192

YAML is a subset of JSON.

❯ ls -l
total 40
-rw-r--r--  1 azat  admin  109 Jan 16 16:20 Dockerfile
-rw-r--r--  1 azat  admin  187 Jan 16 16:20
-rw-r--r--  1 azat  admin  599 Jan 16 16:20
-rw-r--r--  1 azat  admin  367 Jan 16 16:20 docker-compose.yml
-rw-r--r--  1 azat  admin   11 Jan 16 16:20 requirements.txt

Docker Compose file:

  • version:
    • mandatory
    • Version of the compose file format (basically the API) — should normally use the latest version
  • services:
    • Different application microservers (container apps)
  • networks:
    • bridge
    • overlay
      • atatchable : true
  • volumes:
  • secrets
  • configs
version: "3.5"
    build: .
    command: python
      - target: 5000
        published: 5000
      - counter-net
      - type: volume
        source: counter-vol
        target: /code
    image: "redis:alpine"


  • build: . : build a new image in current directory
  • command: python — main app how to run. This can be in the docker file
  • ports:
    • -target: port in the container
    • published : public port (in the host)
  • network : network to attach
  • volumes: Tells Docker to mount the ‘counter-vol’ volume (source: , is in the host) to /code (target, is in the container)
    • source : location in the host
    • target : location in the container

Note: Technically speaking, we don’t need the command: python option. This is because the application’s Dockerfile already defines python as the default app for the image. However, we’re showing it here so you know how it works. You can also use Compose to override CMD instructions set in Dockerfiles.

Deploying app with compose

File: docker-compose.yaml

docker-compose up &

— >

  • Build or pulls required images
  • Creates required networks
  • Creates required volumes
  • Starts all required containers
docker-compose up -d 

— > -d flag to bring the app up in the background

docker-compose -f prod-equus-bass.yml up -d

— >

  • -f specify the file name of the compose yaml (when it’s default to docker-compose.yaml , do not need to use -f)
  • -d run in the background
docker image ls
❯ docker container ls
CONTAINER ID   IMAGE                COMMAND                  CREATED          STATUS          PORTS                    NAMES
d349875bf598   counter-app_web-fe   "python"          37 minutes ago   Up 37 minutes>5000/tcp   counter-app_web-fe_1
0cabc4ff675c   redis:alpine         "docker-entrypoint.s…"   37 minutes ago   Up 37 minutes   6379/tcp                 counter-app_redis_1
❯ docker network ls
NETWORK ID     NAME                      DRIVER    SCOPE
2a182a6b0bcd   bridge                    bridge    local
1c23527ac7cc   counter-app_counter-net   bridge    local
3cb8900f1c15   host                      host      local
0e1926106769   minikube                  bridge    local
0d92c2941210   none                      null      local
❯ docker volume ls
local     counter-app_counter-vol
local     f39a5f8b44cb95ab2d5f672f37be83be466f306759e2987de9a46b1f9ddc93b6
local     minikube


Managing an app with Compose

❯ docker-compose down
Stopping counter-app_web-fe_1 ...
Stopping counter-app_redis_1  ...
redis_1   | 1:signal-handler (1610795088) Received SIGTERM scheduling shutdown...
redis_1   | 1:M 16 Jan 2021 11:04:48.445 # User requested shutdown...
redis_1   | 1:M 16 Jan 2021 11:04:48.445 * Saving the final RDB snapshot before exiting.
Stopping counter-app_web-fe_1 ... done
counter-app_redis_1 exited with code 0
counter-app_web-fe_1 exited with code 0
Removing counter-app_web-fe_1 ... done
Removing counter-app_redis_1  ... done
Removing network counter-app_counter-net
[1]  + 41666 done       docker-compose up

It’s important to note that the counter-vol volume was not deleted

❯ docker-compose ps
Name   Command   State   Ports
❯ docker-compose up -d
Creating network "counter-app_counter-net" with the default driver
Creating counter-app_web-fe_1 ... done
Creating counter-app_redis_1  ... done
❯ docker-compose top
UID   PID    PPID   C   STIME   TTY     TIME         CMD
999   5015   4961   0   11:14   ?     00:00:00   redis-server

UID    PID    PPID   C   STIME   TTY     TIME                    CMD
root   5028   4985   0   11:14   ?     00:00:00   python
root   5149   5028   0   11:14   ?     00:00:00   /usr/local/bin/python /code/
❯ docker-compose ps
        Name                      Command               State           Ports
counter-app_redis_1 redis ...   Up      6379/tcp
counter-app_web-fe_1   python                    Up>5000/tcp
❯ docker-compose restart
Restarting counter-app_web-fe_1 ... done
Restarting counter-app_redis_1  ... done
❯ docker-compose stop
Stopping counter-app_web-fe_1 ... done
Stopping counter-app_redis_1  ... done
❯ docker container ls -a
CONTAINER ID   IMAGE                COMMAND                  CREATED              STATUS                      PORTS     NAMES
d9afb0a2f90f   counter-app_web-fe   "python"          About a minute ago   Exited (0) 20 seconds ago             counter-app_web-fe_1
62907319e651   redis:alpine         "docker-entrypoint.s…"   About a minute ago   Exited (0) 20 seconds ago             counter-app_redis_1
❯ docker-compose down
Removing counter-app_web-fe_1 ... done
Removing counter-app_redis_1  ... done
Removing network counter-app_counter-net
❯ docker container ls -a

Compose builds networks and volumes before deploying services.


Deploying apps with compose - The commands

  • docker compose up : expects:
    • docker-compose.yml or docker-compose.yaml
  • docker-compose stop
    • Stop container
    • DO NOT delete container
  • docker-compose rm
    • Delete a stopped container
  • docker-compose restart
  • docker-compose ps
  • docker-compose top
  • docker-compose down:
    • Stop containers
    • Delete containers
  • NOTE : compose reserves volumes

10. Docker Swarm

Docker Swarm:

  • Enterprise-grade secure cluster of Docker hosts
  • Engine for orchestrating microservices

— > :

  • Define apps in declarative manifest files
  • Perform rolling updates
  • Perform rollbacks
  • Scaling

Docker Swarm: vs Kubernets

  • Easier to configure
  • Easier to deploy
  • Excellent for small-to-medium businesses and application developments


Swarm: consists of

  • Docker Nodes:
    • Docker installed
    • Can communicate over network

Atomic unit of scheduling on a swarm is the service:

  • Service as an enhanced container

Swarm cluster:

  • Manager node
    • Control plane of the cluster
  • Worker nodes
    • Accept work from manager node

Example :


Ports should be open for Docker Swarn:

  • 2377/tcp secure client-to-swarm communication
  • 7946/tcp and ump: for control plane gossip
  • 4789/udp for VXLAN-based overlay networks (Virtual Extensible LAN)

Initializing a swarm (the process of building a swarm ) :

  • Initialize the first manager node
  • Join additional manager nodes
  • Join worker nodes
  • done

Docker nodes that are not part of a swarm are said to be in single-engine mode. Once they’re added to a swarm they’re automatically switched into swarm mode.

inet netmask 0xffffff80 broadcast
❯ docker node ls
nhcbt33c1v8bz9mxk4zedqx6z *   azt              Ready     Active         Leader           20.10.2
jzbm7bqavqcndy88zium23gfj     docker-desktop   Ready     Active                          20.10.2
p1swhwajjcpzo0vcaah4b22z0     rpi48            Ready     Active                          19.03.13
sx9r6mvdc4mh1ak5rnmtki1hf     ux32             Ready     Active                          20.10.0+dfsg2s
❯ docker network create -d overlay uber-net
❯ docker network ls
NETWORK ID     NAME              DRIVER    SCOPE
1eb8b8f691a9   bridge            bridge    local
d1a503020060   docker_gwbridge   bridge    local
d04af8a178f6   host              host      local
ywf8d7w1rmh8   ingress           overlay   swarm
77b174752c1e   none              null      local
04dqq32xybay   uber-net          overlay   swarm


To update :

❯ docker service update --image nigelpoulton/tu-demo:v2 --update-parallelism 2 --update-delay 20s uber-svc

Docker Swarm - The commands

  • docker swarm init

  • docker swarm join-token
  • docker node ls
  • docker service create
  • docker service ls
  • docker service ps <service>
  • docker service inspect
  • docker service scale
  • docker service update
  • docker service logs
  • docker service rm

11. Docker Networking

networks are at the center of everything — no network, no app!


CNM defines three major building blocks:

  • Sandboxes
    • Is an isolated network stack
    • Includes:
      • Ethernet interfaces
      • ports
      • Routing tables
      • DNS configs
  • Endpoints:
    • Virtual network interfaces : like veth
    • Responsible for making connection
    • Job : endpoint to connect a sandbox to a network
  • Networks:
    • Software implementation of a switch (802d bridge)



Container A has a single interface (endpoint) and is connected to Network A. Container B has two interfaces (endpoints) and is connected to Network A and Network B. The ==two containers will be able to communicate== because they are both connected to Network A. However, ==the two endpoints in Container B cannot communicate== with each other without the assistance of a layer 3 router.

End points behave like regular network adapters, meaning they can only be connected to a single network.


The CNM is the design doc, and libnetwork is the canonical implementation.

Actual relationship :


Docker native drovers (local drivers):

  • Linux:
    • bridge:
      • Single host bridge networks:
        • Single host: only exists on single docker host and can only connect containers that are on the same host
        • Bridge: it’s an implementation of an 802.1d bridge (layer 2 switch).
    • overlay
    • macvlan
  • Windows:
    • nat
    • overlay
    • transparent
    • l2bridge

Single host bridge networks


❯ docker network inspect
"docker network inspect" requires at least 1 argument.
See 'docker network inspect --help'.

Usage:  docker network inspect [OPTIONS] NETWORK [NETWORK...]

Display detailed information on one or more networks
❯ docker network inspect bridge
        "Name": "bridge",
        "Id": "1eb8b8f691a9e896c151ed36b31227f9be635fff5ca9343d108b28122ab58375",
        "Created": "2021-01-19T09:48:55.392836442+06:00",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                    "Subnet": "",
                    "Gateway": ""
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        "ConfigOnly": false,
        "Containers": {},
        "Options": {
            "": "true",
            "": "true",
            "": "true",
            "": "",
            "": "docker0",
            "": "1500"
        "Labels": {}
❯ ip link show docker0
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default
    link/ether 02:42:8f:36:4f:8c brd ff:ff:ff:ff:ff:ff
❯ docker network inspect bridge | grep
            "": "docker0",





Port mappings let you map a container to a port on the Docker host. Any traffic hitting the Docker host on the configured port will be directed to the container.


❯ docker container run -d --name web --publish 5000:80 nginx
Unable to find image 'nginx:latest' locally
latest: Pulling from library/nginx
a076a628af6f: Already exists
0732ab25fa22: Pull complete
d7f36f6fe38f: Pull complete
f72584a26f32: Pull complete
7125e4df9063: Pull complete
Digest: sha256:10b8cc432d56da8b61b070f4c7d2543a9ed17c2b23010b43af434fd40e2ca4aa
Status: Downloaded newer image for nginx:latest


❯ docker port web
80/tcp ->

Multi-host overlay Networks

Overlay networks are multi-host. They allow a single network to span multiple hosts so that containers on different hosts can communicate directly.

Connecting to existing networks

the containerized parts need a way to communicate with the non-containerized parts still running on existing physical networks and VLANs.

The built-in MACVLAN driver (transparent on Windows) was created with this in mind. It makes containers first- class citizens on the existing physical networks by giving each one its own MAC address and IP addresses





Service discovery

Service discovery allows all containers and Swarm services to locate each other by name. The only requirement is that they be on the same network.


Every Swarm service and standalone container started with the –name flag will register its name and IP with the Docker DNS service.

However, service discovery is network-scoped.

Ingress load balancing

Swarm supports two publishing modes:

  • Ingress mode (default)
  • Host mode


Behind the scenes, ingress mode uses a layer 4 routing mesh called the Service Mesh or the Swarm Mode Service Mesh.



Docker Networking - The commands

  • docker netwoerk ls
  • docker network create
  • docker network inspect
  • docker network prune — delete all unused networks on a docker host
  • docker network rm

12. Docker overlay networking


This is because swarm mode is a pre- requisite for overlay networks.


We can ping containers on overlay using container name or the ip.

First and foremost, Docker overlay networking uses VXLAN tunnels to create virtual Layer 2 overlay networks. So, before we go any further, let’s do a quick VXLAN primer.



To accomplish this, a new sandbox (network namespace) was created on each host. As mentioned in the previous chapter, a sandbox is like a container, but instead of running an application, it runs an isolated network stack —one that’s sandboxed from the network stack of the host itself.

A virtual switch (a.k.a. virtual bridge) called Br0 is created inside the sandbox.



For this example, we’ll call the container on node1 “C1” and the container on node2 “C2”. And let’s assume C1 wants to ping C2 like we did in the practical example earlier in the chapter.


Arp table — Mac address table

![Layer 2 vs Layer 3 Switch: Which One Do You Need? FS Community](

Switch (L2/L3) Vs Router: Comparison and Differences in TCP/IP Networks

Layer 2 vs Layer 3 Switch: Which One Do You Need?  FS Community

Layer 2 vs Layer 3 Switch: What's the Difference?

Layer 2 3 And 4 Switching  Black Box

Docker overlay Networking — The commands

  • docker network create
  • ` docker network ls`
  • docker network inspect
  • docker network rm

13. Volumes and persistant data

Data :

  • Persistent
  • Non-persistent




❯ docker volume inspect myvol
        "CreatedAt": "2021-01-22T14:03:50+06:00",
        "Driver": "local",
        "Labels": {},
        "Mountpoint": "/var/lib/docker/volumes/myvol/_data",
        "Name": "myvol",
        "Options": {},
        "Scope": "local"

If you specify an existing volume, Docker will use the existing volume • If you specify a volume that doesn’t exist, Docker will create it for you

❯ docker volume ls
local     bizvo

Sharing storage across cluster nodes


Volumes and persistent data — the commands

  • ` docker volume create`
  • ` docker volume ls`
  • ` docker volume inspect`
  • ` docker volume prune`
  • ` docker volume rm`
  • ` docker plugin install`
  • ` docker plugin ls`

14. Deploying apps with Docker Stacks


5 Services, 3 networks, 4 secrets, and 3 port mappings


  • version
  • services
  • secrets
  • networks

One difference between Docker Stacks and Docker Compose is that stacks do not support builds. This means all images have to be built prior to deploying the stack.


15. Security in Docker


Security on Docker:

  • Linux security technologies:
    • Namespaces
    • Control Groups
    • Capabilities
    • Mandatory Access Control
  • Docker platform security technologies:
    • Swarm mode
    • Image scanning
    • Docker content trust
    • Docker secrets


Docker on linux currently utilizes the following kernel namespaces :

  • Process ID (pid) : Docker uses the pid namespace to provide isolated process trees for each container
  • Network (net): Docker uses the net namespace to provide each container its own isolated network stack
  • Filesystem/mount (mnt) ; Every container gets its own unique isolated root (/) filesystem.
  • Inter-process Communication (ipc) : Docker uses the ipc namespace for shared memory access within a container. It also isolates the container from shared memory outside of the container.
  • User (user): Docker lets you use user namespaces to map users inside of a container to different users on the Linux host.
  • UTS (uts): Docker uses the uts namespace to provide each container with its own hostname.

Docker containers are an organized collection of namespaces.

every container has its own pid, net, mnt, ipc, uts, and potentially user namespace.


If namespaces are about isolation, control groups (cgroups) are about setting limits

Think of containers as similar to rooms in a hotel. While each room might appear isolated, every room shares a common set of infrastructure resources — things like water supply, electricity supply, shared swimming pool, shared gym, shared breakfast bar etc. Cgroups let us set limits so that (sticking with the hotel analogy) no single container can use all of the water or eat everything at the breakfast bar.

In the real world, not the hotel analogy, containers are isolated from each other but all share a common set of OS resources — things like CPU, RAM, network bandwidth, and disk I/O. Cgroups let us set limits on each of these so a single container cannot consume everything and cause a denial of service (DoS) attack.







16. What next