9. Deploying apps with Docker Compose
How to deploy multi-container applications using Docker Compose
Methods for Deploying container apps:
- Docker Compose:
- Deploys and manages multi-container applications on Docker nodes running in single-engine mode
- Docker Stacks:
- Deploys and manage multi-container apps on Docker nodes running in swarm mode
Micro-services : Modern cloud-native apps are made of multiple smaller services that interact to form a useful app. We call this pattern “micro-services”
Example (each of below is one service):
- Web front-end
- Ordering
- Catalog
- Backend database
- Logging
- Authentication
- Authorization
Docker Compose lets you describe an entire app in a single declarative configuration file, and deploy it with a single command
❯ docker-compose --version
docker-compose version 1.27.4, build 40524192
YAML is a subset of JSON.
❯ pwd
/Users/azat/Data/Containers/counter-app
❯ ls -l
total 40
-rw-r--r-- 1 azat admin 109 Jan 16 16:20 Dockerfile
-rw-r--r-- 1 azat admin 187 Jan 16 16:20 README.md
-rw-r--r-- 1 azat admin 599 Jan 16 16:20 app.py
-rw-r--r-- 1 azat admin 367 Jan 16 16:20 docker-compose.yml
-rw-r--r-- 1 azat admin 11 Jan 16 16:20 requirements.txt
Docker Compose file:
version:- mandatory
- Version of the compose file format (basically the API) — should normally use the latest version
services:- Different application microservers (container apps)
networks:- bridge
- overlay
- atatchable : true
volumes:- …
secretsconfigs
version: "3.5"
services:
web-fe:
build: .
command: python app.py
ports:
- target: 5000
published: 5000
networks:
- counter-net
volumes:
- type: volume
source: counter-vol
target: /code
redis:
image: "redis:alpine"
networks:
counter-net:
networks:
counter-net:
volumes:
counter-vol:
build: .: build a new image in current directorycommand: python app.py— main app how to run. This can be in the docker fileports:-target: port in the containerpublished: public port (in the host)
network: network to attachvolumes: Tells Docker to mount the ‘counter-vol’ volume (source:, is in the host) to/code(target, is in the container)source: location in the hosttarget: location in the container
Note: Technically speaking, we don’t need the command: python app.py option. This is because the application’s Dockerfile already defines python app.py as the default app for the image. However, we’re showing it here so you know how it works. You can also use Compose to override CMD instructions set in Dockerfiles.
Deploying app with compose
File: docker-compose.yaml
docker-compose up &
— >
- Build or pulls required images
- Creates required networks
- Creates required volumes
- Starts all required containers
docker-compose up -d
— > -d flag to bring the app up in the background
docker-compose -f prod-equus-bass.yml up -d
— >
-fspecify the file name of the compose yaml (when it’s default to docker-compose.yaml , do not need to use -f)-drun in the background
docker image ls
❯ docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d349875bf598 counter-app_web-fe "python app.py" 37 minutes ago Up 37 minutes 0.0.0.0:5000->5000/tcp counter-app_web-fe_1
0cabc4ff675c redis:alpine "docker-entrypoint.s…" 37 minutes ago Up 37 minutes 6379/tcp counter-app_redis_1
❯ docker network ls
NETWORK ID NAME DRIVER SCOPE
2a182a6b0bcd bridge bridge local
1c23527ac7cc counter-app_counter-net bridge local
3cb8900f1c15 host host local
0e1926106769 minikube bridge local
0d92c2941210 none null local
❯ docker volume ls
DRIVER VOLUME NAME
local counter-app_counter-vol
local f39a5f8b44cb95ab2d5f672f37be83be466f306759e2987de9a46b1f9ddc93b6
local minikube
Managing an app with Compose
❯ docker-compose down
Stopping counter-app_web-fe_1 ...
Stopping counter-app_redis_1 ...
redis_1 | 1:signal-handler (1610795088) Received SIGTERM scheduling shutdown...
redis_1 | 1:M 16 Jan 2021 11:04:48.445 # User requested shutdown...
redis_1 | 1:M 16 Jan 2021 11:04:48.445 * Saving the final RDB snapshot before exiting.
Stopping counter-app_web-fe_1 ... done
counter-app_redis_1 exited with code 0
counter-app_web-fe_1 exited with code 0
Removing counter-app_web-fe_1 ... done
Removing counter-app_redis_1 ... done
Removing network counter-app_counter-net
[1] + 41666 done docker-compose up
It’s important to note that the counter-vol volume was not deleted
❯ docker-compose ps
Name Command State Ports
------------------------------
❯ docker-compose up -d
Creating network "counter-app_counter-net" with the default driver
Creating counter-app_web-fe_1 ... done
Creating counter-app_redis_1 ... done
❯ docker-compose top
counter-app_redis_1
UID PID PPID C STIME TTY TIME CMD
-------------------------------------------------------------
999 5015 4961 0 11:14 ? 00:00:00 redis-server
counter-app_web-fe_1
UID PID PPID C STIME TTY TIME CMD
------------------------------------------------------------------------------------
root 5028 4985 0 11:14 ? 00:00:00 python app.py
root 5149 5028 0 11:14 ? 00:00:00 /usr/local/bin/python /code/app.py
❯ docker-compose ps
Name Command State Ports
--------------------------------------------------------------------------------------
counter-app_redis_1 docker-entrypoint.sh redis ... Up 6379/tcp
counter-app_web-fe_1 python app.py Up 0.0.0.0:5000->5000/tcp
❯ docker-compose restart
Restarting counter-app_web-fe_1 ... done
Restarting counter-app_redis_1 ... done
❯ docker-compose stop
Stopping counter-app_web-fe_1 ... done
Stopping counter-app_redis_1 ... done
❯ docker container ls -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d9afb0a2f90f counter-app_web-fe "python app.py" About a minute ago Exited (0) 20 seconds ago counter-app_web-fe_1
62907319e651 redis:alpine "docker-entrypoint.s…" About a minute ago Exited (0) 20 seconds ago counter-app_redis_1
❯ docker-compose down
Removing counter-app_web-fe_1 ... done
Removing counter-app_redis_1 ... done
Removing network counter-app_counter-net
❯ docker container ls -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
Compose builds networks and volumes before deploying services.
Deploying apps with compose - The commands
docker compose up: expects:docker-compose.ymlordocker-compose.yaml
docker-compose stop- Stop container
- DO NOT delete container
docker-compose rm- Delete a stopped container
docker-compose restartdocker-compose psdocker-compose topdocker-compose down:- Stop containers
- Delete containers
- NOTE : compose reserves volumes
10. Docker Swarm
Docker Swarm:
- Enterprise-grade secure cluster of Docker hosts
- Engine for orchestrating microservices
— > :
- Define apps in declarative manifest files
- Perform rolling updates
- Perform rollbacks
- Scaling
Docker Swarm: vs Kubernets
- Easier to configure
- Easier to deploy
- Excellent for small-to-medium businesses and application developments
Swarm: consists of
- Docker Nodes:
- Docker installed
- Can communicate over network
Atomic unit of scheduling on a swarm is the service:
- Service as an enhanced container
Swarm cluster:
- Manager node
- Control plane of the cluster
- Worker nodes
- Accept work from manager node
Example :
Ports should be open for Docker Swarn:
- 2377/tcp secure client-to-swarm communication
- 7946/tcp and ump: for control plane gossip
- 4789/udp for VXLAN-based overlay networks (Virtual Extensible LAN)
Initializing a swarm (the process of building a swarm ) :
- Initialize the first manager node
- Join additional manager nodes
- Join worker nodes
- done
Docker nodes that are not part of a swarm are said to be in single-engine mode. Once they’re added to a swarm they’re automatically switched into swarm mode.
inet 10.22.22.10 netmask 0xffffff80 broadcast 10.22.22.127
❯ docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
nhcbt33c1v8bz9mxk4zedqx6z * azt Ready Active Leader 20.10.2
jzbm7bqavqcndy88zium23gfj docker-desktop Ready Active 20.10.2
p1swhwajjcpzo0vcaah4b22z0 rpi48 Ready Active 19.03.13
sx9r6mvdc4mh1ak5rnmtki1hf ux32 Ready Active 20.10.0+dfsg2s
❯ docker network create -d overlay uber-net
04dqq32xybayqd5j6wzu4mxsz
❯ docker network ls
NETWORK ID NAME DRIVER SCOPE
1eb8b8f691a9 bridge bridge local
d1a503020060 docker_gwbridge bridge local
d04af8a178f6 host host local
ywf8d7w1rmh8 ingress overlay swarm
77b174752c1e none null local
04dqq32xybay uber-net overlay swarm
To update :
❯ docker service update --image nigelpoulton/tu-demo:v2 --update-parallelism 2 --update-delay 20s uber-svc
uber-svc
Docker Swarm - The commands
-
docker swarm init docker swarm join-tokendocker node lsdocker service createdocker service lsdocker service ps <service>docker service inspectdocker service scaledocker service updatedocker service logsdocker service rm
11. Docker Networking
networks are at the center of everything — no network, no app!
CNM defines three major building blocks:
- Sandboxes
- Is an isolated network stack
- Includes:
- Ethernet interfaces
- ports
- Routing tables
- DNS configs
- Endpoints:
- Virtual network interfaces : like
veth - Responsible for making connection
- Job : endpoint to connect a sandbox to a network
- Virtual network interfaces : like
- Networks:
- Software implementation of a switch (802d bridge)
Container A has a single interface (endpoint) and is connected to Network A. Container B has two interfaces (endpoints) and is connected to Network A and Network B. The ==two containers will be able to communicate== because they are both connected to Network A. However, ==the two endpoints in Container B cannot communicate== with each other without the assistance of a layer 3 router.
End points behave like regular network adapters, meaning they can only be connected to a single network.
The CNM is the design doc, and libnetwork is the canonical implementation.
Actual relationship :
Docker native drovers (local drivers):
- Linux:
bridge:- Single host bridge networks:
- Single host: only exists on single docker host and can only connect containers that are on the same host
- Bridge: it’s an implementation of an 802.1d bridge (layer 2 switch).
- Single host bridge networks:
overlaymacvlan
- Windows:
natoverlaytransparentl2bridge
Single host bridge networks
❯ docker network inspect
"docker network inspect" requires at least 1 argument.
See 'docker network inspect --help'.
Usage: docker network inspect [OPTIONS] NETWORK [NETWORK...]
Display detailed information on one or more networks
❯ docker network inspect bridge
[
{
"Name": "bridge",
"Id": "1eb8b8f691a9e896c151ed36b31227f9be635fff5ca9343d108b28122ab58375",
"Created": "2021-01-19T09:48:55.392836442+06:00",
"Scope": "local",
"Driver": "bridge",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "172.17.0.0/16",
"Gateway": "172.17.0.1"
}
]
},
"Internal": false,
"Attachable": false,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {},
"Options": {
"com.docker.network.bridge.default_bridge": "true",
"com.docker.network.bridge.enable_icc": "true",
"com.docker.network.bridge.enable_ip_masquerade": "true",
"com.docker.network.bridge.host_binding_ipv4": "0.0.0.0",
"com.docker.network.bridge.name": "docker0",
"com.docker.network.driver.mtu": "1500"
},
"Labels": {}
}
]
❯ ip link show docker0
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default
link/ether 02:42:8f:36:4f:8c brd ff:ff:ff:ff:ff:ff
❯ docker network inspect bridge | grep bridge.name
"com.docker.network.bridge.name": "docker0",
Port mappings let you map a container to a port on the Docker host. Any traffic hitting the Docker host on the configured port will be directed to the container.
❯ docker container run -d --name web --publish 5000:80 nginx
Unable to find image 'nginx:latest' locally
latest: Pulling from library/nginx
a076a628af6f: Already exists
0732ab25fa22: Pull complete
d7f36f6fe38f: Pull complete
f72584a26f32: Pull complete
7125e4df9063: Pull complete
Digest: sha256:10b8cc432d56da8b61b070f4c7d2543a9ed17c2b23010b43af434fd40e2ca4aa
Status: Downloaded newer image for nginx:latest
67f03cb72e5692dc859b7ca566784067f20724804e4e41cccba59f3344d62dfe
❯ docker port web
80/tcp -> 0.0.0.0:5000
Multi-host overlay Networks
Overlay networks are multi-host. They allow a single network to span multiple hosts so that containers on different hosts can communicate directly.
Connecting to existing networks
the containerized parts need a way to communicate with the non-containerized parts still running on existing physical networks and VLANs.
The built-in MACVLAN driver (transparent on Windows) was created with this in mind. It makes containers first- class citizens on the existing physical networks by giving each one its own MAC address and IP addresses
Service discovery
Service discovery allows all containers and Swarm services to locate each other by name. The only requirement is that they be on the same network.
Every Swarm service and standalone container started with the –name flag will register its name and IP with the Docker DNS service.
However, service discovery is network-scoped.
Ingress load balancing
Swarm supports two publishing modes:
- Ingress mode (default)
- Host mode
Behind the scenes, ingress mode uses a layer 4 routing mesh called the Service Mesh or the Swarm Mode Service Mesh.
Docker Networking - The commands
docker netwoerk lsdocker network createdocker network inspectdocker network prune— delete all unused networks on a docker hostdocker network rm
12. Docker overlay networking
This is because swarm mode is a pre- requisite for overlay networks.
We can ping containers on overlay using container name or the ip.
First and foremost, Docker overlay networking uses VXLAN tunnels to create virtual Layer 2 overlay networks. So, before we go any further, let’s do a quick VXLAN primer.
To accomplish this, a new sandbox (network namespace) was created on each host. As mentioned in the previous chapter, a sandbox is like a container, but instead of running an application, it runs an isolated network stack —one that’s sandboxed from the network stack of the host itself.
A virtual switch (a.k.a. virtual bridge) called Br0 is created inside the sandbox.
For this example, we’ll call the container on node1 “C1” and the container on node2 “C2”. And let’s assume C1 wants to ping C2 like we did in the practical example earlier in the chapter.
Arp table — Mac address table
|  |
Docker overlay Networking — The commands
docker network create- ` docker network ls`
docker network inspectdocker network rm
13. Volumes and persistant data
Data :
- Persistent
- Non-persistent
❯ docker volume inspect myvol
[
{
"CreatedAt": "2021-01-22T14:03:50+06:00",
"Driver": "local",
"Labels": {},
"Mountpoint": "/var/lib/docker/volumes/myvol/_data",
"Name": "myvol",
"Options": {},
"Scope": "local"
}
]
If you specify an existing volume, Docker will use the existing volume • If you specify a volume that doesn’t exist, Docker will create it for you
❯ docker volume ls
DRIVER VOLUME NAME
local bizvo
Sharing storage across cluster nodes
Volumes and persistent data — the commands
- ` docker volume create`
- ` docker volume ls`
- ` docker volume inspect`
- ` docker volume prune`
- ` docker volume rm`
- ` docker plugin install`
- ` docker plugin ls`
14. Deploying apps with Docker Stacks
5 Services, 3 networks, 4 secrets, and 3 port mappings
stack.yaml:
- version
- services
- secrets
- networks
One difference between Docker Stacks and Docker Compose is that stacks do not support builds. This means all images have to be built prior to deploying the stack.
15. Security in Docker
Security on Docker:
- Linux security technologies:
- Namespaces
- Control Groups
- Capabilities
- Mandatory Access Control
- Docker platform security technologies:
- Swarm mode
- Image scanning
- Docker content trust
- Docker secrets
Docker on linux currently utilizes the following kernel namespaces :
- Process ID (pid) : Docker uses the pid namespace to provide isolated process trees for each container
- Network (net): Docker uses the net namespace to provide each container its own isolated network stack
- Filesystem/mount (mnt) ; Every container gets its own unique isolated root (/) filesystem.
- Inter-process Communication (ipc) : Docker uses the ipc namespace for shared memory access within a container. It also isolates the container from shared memory outside of the container.
- User (user): Docker lets you use user namespaces to map users inside of a container to different users on the Linux host.
- UTS (uts): Docker uses the uts namespace to provide each container with its own hostname.
Docker containers are an organized collection of namespaces.
every container has its own pid, net, mnt, ipc, uts, and potentially user namespace.
If namespaces are about isolation, control groups (cgroups) are about setting limits
Think of containers as similar to rooms in a hotel. While each room might appear isolated, every room shares a common set of infrastructure resources — things like water supply, electricity supply, shared swimming pool, shared gym, shared breakfast bar etc. Cgroups let us set limits so that (sticking with the hotel analogy) no single container can use all of the water or eat everything at the breakfast bar.
In the real world, not the hotel analogy, containers are isolated from each other but all share a common set of OS resources — things like CPU, RAM, network bandwidth, and disk I/O. Cgroups let us set limits on each of these so a single container cannot consume everything and cause a denial of service (DoS) attack.
16. What next
