Docker : And Virtualization Became Efficient… (Part 3)

They went further

Four months ago we have been introducing Docker, explained why we think it is going to radically change the traditional approach to Virtualization, and quickly shown you how to build and start a simple application using Docker. A great deal of water has flown under the bridges since then, bringing amazing news from the Docker team: the acquisition of SocketPlane and Kitematica, a $95M successful series-D raising, millions of new users, and thousands of new images stored in the Docker Hub. Long story short, if there was still a need to prove that Docker is becoming a major actor of the Virtualization market, this is no longer the case.

Among all these good news, Docker has released three key features in their Beta versions: Machine, Swarm, and Compose. With these new tools all combined together, Docker aims to address three issues that any complex infrastructure architecture has to face: deployment, resiliency, and scalability. The idea is simple: we have already seen that using Docker, you can easily build an isolated stack – a container – with all the components needed, deploy your service, and run it. Fair enough. But what if the physical server crashes ? What if a lot more users than expected are accessing it at the same time ? Or simply if for any reason, the connection to your service is lost ? The obvious answer is: “Let’s deploy it on several servers, in several datacenters!”. Sure. And you will also have to add a load-balancing service as an entry point to your overall architecture. And every time you will be willing to add a new server to the pool, you will have to declare it to the load-balancer. And if you want to deploy a new version of you application, you will have to do it on all these servers… Anyway: what a nightmare.

Of course, you can find a lot of powerful tools on the market to help you automate all these steps. You will learn about continuous integration, clustering, automatic fail-overs, etc. But most of the time, it takes a lot of time to set-up, and costs a lot of money to maintain. Unfortunately, most of the time, if you want your project to be successful, you won’t rely on a huge budget and unlimited resources, but on efficiency. And this is exactly where Docker is positioning the three new Orchestration tools mentioned above.

Since these features are designed to bring efficiency to your project, they are supposed to be simple. Then, it shouldn’t be a big deal to resume their purposes in one sentence:

  • Docker Machine = “I want to be able to install the Docker engine anywhere in a single command line”
  • Docker Swarm = “I want to run my application on several servers, in several datacenters”
  • Docker Compose = “I want the deployment of the different services composing my application to be straight-forward”

Hands-on now: in the previous article, we have designed a Web server Docker image, which is available in the Docker Hub: maestrano/nex-s-phpmysql. Let’s say that we want to deploy this service locally on a virtual machine, but also remotely on an AWS instance.

Preparing the cluster

First step, I use both Machine and Swarm to install the Docker engine on my two servers, and to link these hosts in a single cluster:

// generate the swarm token
$ docker run swarm create
> bed6e3d60450bed4c70f3b30b57168cf

// create the swarm master on a local virtual machine
$ docker-machine create -d virtualbox --swarm --swarm-master --swarm-discovery token://bed6e3d60450bed4c70f3b30b57168cf master

// create a swarm node on a remote aws instance
$ docker-machine create -d amazonec2 --amazonec2-access-key=MY_ACCESS_KEY --amazonec2-secret-key=MY_SECRET_KEY --amazonec2-vpc-id=MY_VPC_ID --swarm --swarm-master --swarm-discovery token://bed6e3d60450bed4c70f3b30b57168cf node1

And that’s it, Docker Machine has prepared two instances, one local and the other one remote. Docker Swarm has linked them as two nodes of a same cluster (the local virtual machine being the cluster’s master).

I can display these instances states by using:

$ docker-machine ls
NAME   ACTIVE     DRIVER     STATE   URL                       SWARM
master            virtualbox Running tcp:// master (master)
node1  *          amazonec2  Running tcp://123.456.78.901:2376 master

And check the cluster state by calling the master engine with the docker info command:

$ docker -H tcp:// info
> Containers: 3
Strategy: spread
Filters: affinity, health, constraint, port, dependency
Nodes: 2
 └ Containers: 2
 └ Reserved CPUs: 0 / 8
 └ Reserved Memory: 0 B / 1.023 GiB
 node1: 123.456.78.901:2376
 └ Containers: 1
 └ Reserved CPUs: 0 / 8
 └ Reserved Memory: 0 B / 1.023 GiB

Starting the containers

Docker Swarm can be considered as the most important tool of the Orchestration suite proposed by Docker. It is the one that will allow me to efficiently deploy my containers on the two hosts I’ve just prepared.

Let’s assume that I would like to deploy my container, which is a Web server stack, 4 times so that the load will be equally shared between 4 different “workers”.

Then, I just start my container 4 times on the Swarm master, and Swarm will take care of pulling the images and running the containers on both the two hosts:

$ docker -H tcp:// run -d maestrano/nex-s-phpmysql nginx
$ docker -H tcp:// run -d maestrano/nex-s-phpmysql nginx
$ docker -H tcp:// run -d maestrano/nex-s-phpmysql nginx
$ docker -H tcp:// run -d maestrano/nex-s-phpmysql nginx

And now, if I check that all the containers have been started properly:

CONTAINER ID  IMAGE                           COMMAND              CREATED         STATUS        PORTS                                   NAMES
3e0ecc9d3977  maestrano/nex-s-phpmysql:latest "nginx"              2 minutes ago   Up 2 minutes  8080/tcp                                master/evil_curie
7edfe8950262  maestrano/nex-s-phpmysql:latest "nginx"              2 minutes ago   Up 2 minutes  8081/tcp                                node1/insane_wilson
08a4939e4361  maestrano/nex-s-phpmysql:latest "nginx"              2 minutes ago   Up 2 minutes  8082/tcp                                master/reverent_darwin
baefc2a22158  maestrano/nex-s-phpmysql:latest "nginx"              2 minutes ago   Up 2 minutes  8083/tcp                                node1/reverent_darwin
ab578cfed270  swarm:latest                    "/swarm join --addr  15 minutes ago  Up 15 minutes 2375/tcp                                node1/swarm-agent
ab578cfed270  swarm:latest                    "/swarm join --addr  17 minutes ago  Up 17 minutes 2375/tcp                                master/swarm-agent 
862030e612f8  swarm:latest                    "/swarm manage --tls 17 minutes ago  Up 17 minutes 2375/tcp,>3376/tcp master/swarm-agent-master

Here stands the magic of Swarm: as promised, 2 containers have been started on the local virtual machine (master), and the two others on the remote one (node1).

So now, if one of the hosts crashes…

…It will automatically restart my containers on the other one and re-route the users to them, right ?

Well, no. Or more exactly, not yet. By the time this article is being written, Docker Swarm is in its beta version, and the high-availability / automatic fail-over features are still in development.

But what we have to consider is that this is definitely the path that Docker is following (according to this post on their blog).

In a very close future, and considering the work that has already been done, we can reasonably expect that using Docker, building a high-available geo-cluster with 4 running instances of your application, will cost you 15 minutes and 7 command lines (see above…).

What about Docker Compose ?

Compose is a bit like the icing on the cake. Again, it is one more tool that will help you deploy your app easier, especially if your application is made of several services.

As an example, if I want to start a WordPress application, based on the image maestrano/nex-a-wordpress, you can see on the Docker Hub page of this image that it requires another service to be linked to. Indeed, a WordPress application requires a database, which will be running in a separate container. I wouldn’t say that it is starting to become complicated, but it results in typing two commands:

docker run -d \
 --name wordpress-db \
 -v /home/cesar/poc/dockerization/storage/db/data:/var/lib/mysql \
 -e MYSQL_ROOT_PASSWORD=maestradmin \
docker run -p 49207:80 -d --privileged=true\
 --name wordpress-app \
 -v /home/cesar/poc/dockerization/storage:/storage \
 --link=wordpress-db:mysql \

Docker Compose aims to reduce it in a single command line. First, you will have to define a docker-compose.yml file in which you will specify all the rules to be taken in account when starting your application:

  image: maestrano/nex-a-wordpress
  build: .
    - /home/cesar/poc/dockerization/storage:/storage
    - wordpress-db
    - "49207:80"
  privileged: true
 image: mysql:5.5
 build: .
   - /home/cesar/poc/dockerization/storage/db/data:/var/lib/mysql
   - MYSQL_ROOT_PASSWORD=maestradmin

Finally, a simple “docker-compose up” will replace the two “docker run” specified above. It can look like a “nice-to-have” feature, but just imagine how handy it will be when you will be willing to deploy complex applications based on 6, 7 different services ? Imagine how powerful Compose is if you use it combined with Swarm and Machine ?

Conclusion: we can’t wait for the rest !

Even if these latest features need some more work to be production-ready, what we can analyze so far is that:

  • Docker has positioned itself on a market that urgently needed some innovation: Virtualization.
  • They have been able to answer a great deal of issues by proposing some tools that are extremely easy to use, efficient, and at no cost at all. By doing so, they managed to get the approval of the developers community (+200M downloads, +100,000 images in the Hub…) AND enough confidence from the investors to secure the project in the long term.
  • Staying on this track, they are now determined to focus their efforts on what can be considered the “heart” of infrastructure architecture: Orchestration. And from what I’ve seen by testing these three tools in their beta version, I can’t wait to see what it will look like in the end.

So, thanks Docker, and carry on !

<< Part 2 : Why is Docker a revolution?

César Tonnoir