Docker Datacenter for AWS

Cloud Platforms, Containers & Kubernetes,

In this blog, I will cover Docker datacenter usage with AWS. This blog is part of my Docker for AWS series and uses the sample voting application for illustration.

Docker Datacenter is Docker’s enterprise grade CaaS(Container as a service) solution where they have integrated their open source software with some proprietary software and support to make it into a commercial product. Docker Datacenter can be deployed on-premise or in cloud providers like AWS. Docker Datacenter is available free for 30 day trial period.

Docker Datacenter Architecture

Following picture shows the core components of Docker Datacenter:

Continue reading Docker Datacenter for AWS →

Docker Cloud for AWS

Cloud Platforms, Containers & Kubernetes, ,

In this blog, I will cover Docker cloud usage with AWS. This blog is part of my Docker for AWS series and uses the sample voting application for illustration.

Docker cloud is a hosted service from Docker to manage Containers. Docker cloud is free to try for 1 private repository and node and is chargeable after that. Docker cloud was originally an acquisition from Tutum. Docker cloud can be used to manage infrastructure nodes in the cloud or nodes in the local data center. For basics on Docker cloud/Tutum, please refer to my earlier blog here. Since Docker cloud was an acquisition, it does not use some of the Docker ecosystem software. Following are some important differences:

Continue reading Docker Cloud for AWS →

Docker for AWS beta

Cloud Platforms, Containers & Kubernetes,

In this blog, I will cover “Docker for AWS” beta service launched by Docker. This blog is part of my Docker for AWS series and uses the sample voting application for illustration.

As part of Docker 1.12 announcement, Docker released AWS Docker integration as beta software. With this software, Docker is trying to  simplify AWS integration by better integrating Docker with AWS services like load balancer, security groups, logs etc. Docker launched similar integration service with Microsoft Azure as well. Docker 1.12 RC4 is available as part of this integration, so Swarm mode feature can be used.
Following are some features that Docker has added as part of this integration with AWS:

Continue reading Docker for AWS beta →

Docker-machine for AWS

Containers & Kubernetes

In this blog, I will use docker-machine to deploy and manage Containers in AWS cloud. This blog is part of my Docker for AWS series and uses the sample voting application for illustration.

Docker-machine has EC2 driver for creating a Docker node out of AWS. Docker node in this context means a AWS VM instance with Docker pre-installed. With default options, docker-machine picks up T2-micro EC2 node with Ubuntu 15.10 OS and installs latest Docker engine on the AWS instance. docker-machine also takes care of configuring appropriate certificates which allows us to access the AWS VM securely.

If we have to do the steps manually, we first need to create EC2 instance with appropriate OS, install Docker, manage ssh keys etc. docker-machine takes care of automating this whole process.

Following are the summary of steps used here:

Continue reading Docker-machine for AWS →

Service Discovery and Load balancing Internals in Docker 1.12

Containers & Kubernetes, Distributed Systems, , ,

Docker 1.12 release has revamped its support for Service Discovery and Load balancing. Prior to 1.12 release, support for Service discovery and Load balancing was pretty primitive in Docker. In this blog, I have covered the internals of Service Discovery and Load balancing in Docker release 1.12. I will cover DNS based load balancing, VIP based load balancing and Routing mesh.

Technology used

Docker service discovery and load balancing uses iptables and ipvs features of Linux kernel. iptables is a packet filtering technology available in Linux kernel. iptables can be used to classify, modify and take decisions based on the packet content. ipvs is a transport level load balancer available in the Linux kernel.

Sample application

Following is the sample application used in this blog:

Continue reading Service Discovery and Load balancing Internals in Docker 1.12 →

Comparing Swarm, Swarmkit and Swarm Mode

Containers & Kubernetes, ,

One of the big features in Docker 1.12 release is Swarm mode. Docker had Swarm available for Container orchestration from 1.6 release. Docker released Swarmkit as an opensource project for orchestrating distributed systems few weeks before Docker 1.12(RC) release. I had some confusion between these three projects. In this blog, I have tried to put my perspective on the similarities and differences between these three software components. I have also created a sample application and deployed it using the three approaches which makes it easier to compare.

Docker Swarm mode is fundamentally different from Swarm and it is confusing to use the same name Swarm. It would have been good if Docker could have renamed this to something different. Another point adding to the confusion is that native Swarm functionality will continue to be supported in Docker 1.12 release, this is done to preserve backward compatibility. In this blog, I have used the term “Swarm” to refer to traditional Swarm functionality, “SwarmNext” to refer to new Swarm mode added in 1.12, “Swarmkit” to refer to the plumbing open source orchestration project.

Swarm, SwarmNext and Swarmkit

Following table compares Swarm and SwarmNext:

Continue reading Comparing Swarm, Swarmkit and Swarm Mode →

Mesos DC/OS Hands-On

Containers & Kubernetes, Distributed Systems, ,

In this blog, I will cover some of the hands-on stuff that I tried with Opensource DC/OS. I created DC/OS cluster using Vagrant and deployed multi-instance nginx webserver using Marathon. For Mesos FAQ, please refer to my previous blog.

I followed the instructions here to create DC/OS Vagrant cluster.

Pre-requisites

I tried DC/OS Vagrant cluster in my Windows machine. Virtualbox and Vagrant needs to be installed before-hand.

Setting up cluster

Following are the instructions that I used to setup the cluster:

Continue reading Mesos DC/OS Hands-On →

Mesos and Mesosphere – FAQ

Containers & Kubernetes, Distributed Systems, , , ,

The most popular Container Orchestration solutions available in the market are Kubernetes, Swarm and Mesos. I have used Kubernetes and Swarm, but never got a chance to use Mesos or DC/OS. There were a bunch of questions I had about Mesos and DC/OS and I never got the time to explore that. Recently, I saw the announcement about Mesosphere opensourcing DC/OS and I found this as a perfect opportunity for me to try out Opensource DC/OS. In this blog, I have captured the answers to questions I had regarding Mesos and DC/OS. In the next blog, I will cover some hands-on that I did with Opensource DC/OS.

What is the relationship between Apache Mesos, Opensource DC/OS and Enterprise DC/OS?

Apache Mesos is the Opensource distributed orchestrator for Container as well as non-Container workloads. Both Opensource DC/OS and Enterprise DC/OS are built on top of Apache Mesos. Opensource DC/OS adds  Service discovery, Universe package for different frameworks, CLI and GUI support for management and Volume support for persistent storage. Enterprise DC/OS adds enterprise features around security, performance, networking, compliance, monitoring and multi-team support that the Opensource DC/OS project does not include. Complete list of differences between Opensource and Enterprise DC/OS are captured here.

What does Mesosphere do and how it is related to Apache Mesos?

Mesosphere company has products that are built on top of Apache Mesos. Lot of folks working in Mesosphere contribute to both Apache Mesos and Opensource DC/OS. Mesosphere has the following products currently:

  • DC/OS Enterprise – Orchestration solution
  • Velocity- CI, CD solution
  • Infinity – Big data solution

Why DC/OS is called OS?

Sometimes folks get confused thinking Mesos being a Container optimized OS like CoreOS, Atomic. Mesos is not a Container optimized OS. Similar to the way Desktop OS provides resource management in a single host, DC/OS provides cluster management across entire cluster. Mesos master(including first level scheduler) and agent are perceived as kernel components and user space components include frameworks, user space applications, dns and load balancers. The kernel provides primitives for the frameworks.

What are Mesos frameworks and why they are needed?

Continue reading Mesos and Mesosphere – FAQ →

Looking inside Container Images

Containers & Kubernetes, Networking & SDN, , , , ,

This blog is a continuation of my previous blog on Container standards. In this blog, we will look inside a Container image to understand the filesystem and manifest files that describes the Container. We will cover Container images in Docker, APPC and OCI formats. As mentioned in previous blog, these Container images will converge into OCI format in the long run.

I have picked two Containers for this blog: “nginx”which is a standard webserver and “smakam/hellocounter” which is a Python web application.

Docker format:

To see Container content in Docker format, do the following:

docker save nginx > nginx.tar
tar -xvf nginx.tar

Following files are present:

  • manifest.json – Describes filesystem layers and name of json file that has the Container properties.
  • <id>.json – Container properties
  • <layer directory> – Each “layerid” directory contains json file describing layer property and filesystem associated with that layer. Docker stores Container images as layers to optimize storage space by reusing layers across images.

Following are some important Container properties that we can see in the JSON file:

Continue reading Looking inside Container Images →