Following link captures the slides on Docker 1.9 feature overview that I presented at Docker Meetup, Bangalore on December 5, 2015. You can find more details on the meetup here and here.
Docker Multihost Networking demo
Tag Archives: Containers & Kubernetes
Docker Meetup Bangalore -Networking presentation
Docker Experimental Networking – 3
This blog is a continuation of my previous blog on Docker Experimental Networking. In this blog, I will walk through the example mentioned in this link where experimental Docker is integrated with Compose and Swarm. I have made some modifications here and I will cover this here.
I will create 2 applications in this example using Docker Compose.
- Counter container connecting to redis container running on 2 different hosts.
- WordPress container connecting to mysql container running on 2 different hosts.
I have used AWS instead of Digitalocean. First step is to create Consul machine and start Consul server.
docker-machine create --driver=amazonec2 --amazonec2-access-key=xxx --amazonec2-secret-key=xxx --amazonec2-vpc-id=vpc-5f77c23a --amazonec2-region=us-west-2 --engine-install-url "https://experimental.docker.com" consul
docker $(docker-machine config consul) run -d \
-p "8500:8500" \
-h "consul" \
progrium/consul -server -bootstrap
Next, create 2 machines connecting both the nodes to Consul:
Docker Experimental Networking – 2
This blog is a continuation of my previous blog on Docker Experimental Networking. In this blog, I will cover an example of Docker container connectivity using bridge and overlay driver.
Following is a sample usecase that I tried:
- Container C1 in host H1 will be able to talk to Container C3 in Host H2 using service endpoints S1 in C1 and S4 in C3 through Network N2(Overlay driver).
- On Host H1, Container C1 has 2 services and C2 has 1 service in the Network N1(bridge driver) and they will be able to talk to each other using the Network N1.
- On Host H2, Container C3 has 2 services and C4 has 1 service in the Network N3(bridge driver) and they will be able to talk to each other using the Network N3.
- Containers will be able to discover services in the same network.
I came across the following limitations/bugs:
Docker Experimental Networking – 1
Networking support in Docker was primitive till now. Single host connectivity was through Linux bridge and there was no native mechanism to connect Containers across hosts. With Pipework, we could do a hacky approach to connect Containers across hosts. Companies like Socketplane, Weave have been working to address this Networking gap. I have written multiple blogs before on Docker Networking and they can be referred here. Socketplane was recently acquired by Docker and they provide the native batteries-included Docker Networking solution and solutions like from Weave will be available as a Docker Networking plugin. With Docker experimental release, we can connect Containers across hosts using Docker native solution as well as use Networking plugins to connect Containers across hosts. In this blog, I will cover some basics of the solution and will walk-through some of the hands-on stuff that I tried with the experimental Docker release.
Docker Networking blocks:
At high level, the diagram above describes the flow for Docker Networking.
- Docker runtime was integrated previously with Networking and there was no way to separate the 2. Libnetwork is the new Networking library that provides the Networking functionality and is seperated from Core Docker. Docker 1.7 release has already included the libnetwork and is backward compatible from enduser perspective.
- Drivers implement the APIs provided by libnetwork. Docker is leaning towards plugin approach for major functionalities like Networking, Storage, Orchestration where Docker provides a native solution which can be substituted with technologies with other vendors as long as they implement the APIs provided by the common library. In this case, Bridge and Overlay are the Native Docker networking drivers and remote drivers can be implemented by third-party. There are already many remote drivers available like Weave.
Docker Container Networking model: Continue reading Docker Experimental Networking – 1
Flocker – Docker Data volumes(Updates)
This blog is an update from my previous blog on Flocker. I had faced a bunch of issues while trying out Flocker. Based on the cases raised by me, Flocker team has been kind enough to help me to sort out some of these issues. In this blog, I have updated the cases where I found a workable solution. There are couple of more issues pending. I will update this blog as I make more progress.
Approach 3(Experimental Flocker install with AWS):
Procedure for this tutorial is covered here and case details are covered in this link. The first issue I faced was that the volume creation got stuck. This was because of an operator error on my part w.r.to setting permission for the AWS user to create EBS.
Following is my sample cluster.yml file.
Flocker – Docker Data volumes
In this blog, I will cover Flocker from ClusterHQ which is used for Container data/volume management. I will cover some Flocker basics and some hands-on stuff that I tried with Flocker.
Docker Volumes:
Docker volumes are used to store persistent data that is outside the scope of the Container Union file system. 1 common example is database. Data volumes are attached to the container so that Containers can update the volume and these volumes can be reattached to other containers or shared across Containers. Data volumes are not deleted when Containers are deleted, Data volumes needs to be deleted manually.
Following is a simple example to create container with mount volume. Here, /webapp is mounted as a data volume.
docker run -d -P --name web -v /webapp training/webapp python app.py
When Containers are deleted, volumes don’t get deleted. To delete volumes when Containers are deleted, we can use:
docker rm -v
Volumes are stored in host under /var/lib/docker/volumes.
Flocker basics:
Containers – Format, Runtime and Platform
1 of the big announcements in Dockercon 2015 was the Open Container project(OCP). OCP is an Opensource project under Linux foundation to define a common Container format. Container format, runtime and platform mean different things. There are many Container formats, runtime and multiple acronyms surrounding it. In this blog, I have tried to capture my understanding around these. I have not discussed about traditional Linux containers in this blog. This is how I see the relationship between Container formats, Container runtime and Container platforms.
Openstack and Docker – Part 2
This is a continuation of my previous blog on Openstack and Docker. In this blog, I will cover Openstack Docker heat plugin and Magnum.
Following are some of the items that Nova Docker driver cannot do currently:
- Passing environment variables
- Linking containers
- Specifying volumes
- Orchestrating and scheduling the containers
Heat docker plugin solves problems 1-3 and partially solves problem 4. Following is the architecture diagram I found in Openstack Docker wiki for heat.
- Nova is not involved here. Openstack heat uses Docker plugin to talk to Docker agent on the host.
- The host here is the VM spawned. The VM can either be spawned by Nova or Heat can spawn this using Nova driver.
- Glance is not involved here as the container images are stored in Docker registry.
- The Heat approach allows us to specify environment variables, link containers, specify volumes as well as orchestrate the host on which the Docker runs.
Using Heat plugin:
Openstack and Docker – Part 1
In this blog, I will cover the different ways in which Openstack can create and manage Docker Containers. The 3 predominant approaches are using Nova Docker driver, Heat Docker plugin and Magnum. Magnum is pretty new and is under development. Openstack is opensource cloud orchestration software and Docker is opensource container management software. For this blog, I am assuming users are already familiar with Openstack and Docker. There are lot of resources for learning Openstack and Docker available in the web, my blogs related to these topics can be found here and here.
Nova Docker Driver:
Nova typically manages VMs. In this approach, Nova driver is extended to spawn Docker Containers. Following is the architecture diagram mentioned in the Nova Docker wiki.
- To spawn containers Nova compute driver is pointed to Docker driver.
- Nova Docker Virt driver talks to Docker agent using http api calls.
- Docker images are stored in the Docker registry and images are exported to glance from Docker registry which Nova uses to create Containers.
Sreenivas Makam's Blog 