This blog is a continuation of my previous blog on Docker Experimental Networking. In this blog, I will walk through the example mentioned in this link where experimental Docker is integrated with Compose and Swarm. I have made some modifications here and I will cover this here.

I will create 2 applications in this example using Docker Compose.

1. Counter container connecting to redis container running on 2 different hosts.
2. WordPress container connecting to mysql container running on 2 different hosts.

I have used AWS instead of Digitalocean. First step is to create Consul machine and start Consul server.

docker-machine create --driver=amazonec2 --amazonec2-access-key=xxx --amazonec2-secret-key=xxx --amazonec2-vpc-id=vpc-5f77c23a --amazonec2-region=us-west-2 --engine-install-url "https://experimental.docker.com" consul

docker $(docker-machine config consul) run -d \
    -p "8500:8500" \
    -h "consul" \
    progrium/consul -server -bootstrap

Next, create 2 machines connecting both the nodes to Consul:

Continue reading Docker Experimental Networking – 3 →

Networking support in Docker was primitive till now. Single host connectivity was through Linux bridge and there was no native mechanism to connect Containers across hosts. With Pipework, we could do a hacky approach to connect Containers across hosts. Companies like Socketplane, Weave have been working to address this Networking gap. I have written multiple blogs before on Docker Networking and they can be referred here. Socketplane was recently acquired by Docker and they provide the native batteries-included Docker Networking solution and solutions like from Weave will be available as a Docker Networking plugin. With Docker experimental release, we can connect Containers across hosts using Docker native solution as well as use Networking plugins to connect Containers across hosts. In this blog, I will cover some basics of the solution and will walk-through some of the hands-on stuff that I tried with the experimental Docker release.

Docker Networking blocks:

At high level, the diagram above describes the flow for Docker Networking.

• Docker runtime was integrated previously with Networking and there was no way to separate the 2. Libnetwork is the new Networking library that provides the Networking functionality and is seperated from Core Docker. Docker 1.7 release has already included the libnetwork and is backward compatible from enduser perspective.
• Drivers implement the APIs provided by libnetwork. Docker is leaning towards plugin approach for major functionalities like Networking, Storage, Orchestration where Docker provides a native solution which can be substituted with technologies with other vendors as long as they implement the APIs provided by the common library. In this case, Bridge and Overlay are the Native Docker networking drivers and remote drivers can be implemented by third-party. There are already many remote drivers available like Weave.

Docker Container Networking model: Continue reading Docker Experimental Networking – 1 →

I recently tried out AWS ECS(EC2 Container service). In this blog, I will provide overview of EC2 container service, some hands-on stuff I tried and my experiences with ECS. AWS ECS is available as a preview currently with only CLI interface. I assume general availability with web interface will be available soon. Amazon does not charge separately for the ECS service, instances used for containers will be charged appropriately. I have a AWS free-tier account and I was able to try the ECS with the free-tier account.

Pre-requisites:

• Familiarity with Containers and Docker. You can refer to my blog on Docker series.
• Familiarity with AWS services. You can refer to my previous blogs on AWS overview and how to access AWS services.
• For the hands-on stuff, you need to have atleast a free-tier account with AWS. AWS cli needs to be installed since ECS is available only with a CLI interface currently.

Overview of ECS:

I had written a blog earlier on Docker Orchestration. ECS is a Docker Container Orchestration service that provides the following functionalities.

• Allows to deploy Containers across a cluster of hosts using either AWS scheduler or integrate with third party schedulers like Mesos.
• Allows management of an application that spans multiple containers rather than just managing containers.
• Provides high availability for Containers by monitoring Container health.

Following are components of ECS:

• Cluster – Logical entity that contains multiple EC2 instances. Instances within the container can be of different types.
• Container instances – Each instance is an EC2 instance with Docker and Container agent pre-installed and is available as AWS AMI image. Container agent registers itself to the cluster and it monitors health of the Containers in the host. AWS is open-sourcing the Container agent project and its available in github.
• Tasks – Tasks contains the application definition, resources needed and it can span multiple containers. Task definitions are specified in json format.
• ECS Master – This manages everything. ECS master is responsible for scheduling the containers and also for talking to the container agent to get the health of container instances. When ECS master schedules the containers, it takes into account resources needed for the container and the resources available in the container instance.

ECS hands-on:

Most of the steps below are as mentioned in AWS ECS developer guide.

Continue reading AWS ECS(EC2 Container service) →