Container Standards

Containers & Kubernetes, Networking & SDN, , , , , , ,

In this blog, I will cover some of the standardization effort that is happening in the Containers area. I will cover some history, current status and also mention how the future looks like. In the next blog, we will look inside ACI and OCI Container images.

Container Standards

Lot of developments in Container area are done as Open source projects. That still does not automatically mean that these projects will become standards. Following are the areas where Container standardization is important:

  • Container image format – Describes how an application is packaged into a Container. The application can be an executable from any programming language. As you would know, Containers packages an application along with all its application dependencies.
  • Container runtime – Describes the environment(namespaces, cgroups etc) necessary to run the Container and the APIs that Container runtime should support.
  • Image signing – Describes how to create Container image digest and to sign these so that Container images can be trusted.
  • Image discovery – Describes alternate approaches to discover Container images other than using registry.
  • Container Networking – This is a pretty complex area and it describes ways to network Containers in same host and across hosts. There are different implementations based on the use-case.

Having common Container standards would allow things like this:

Continue reading Container Standards →

Docker macvlan and ipvlan network plugins

Containers & Kubernetes, , ,

This is a continuation of my previous blog on macvlan and ipvlan Linux network drivers. Docker has added support for macvlan and ipvlan drivers and its currently in experimental mode as of Docker release 1.11.

Example used in this blog

In this example, we will use Docker macvlan and ipvlan network plugins for Container communication across hosts. To illustrate macvlan and ipvlan concepts and usage, I have created the following example.

vlan4

Following are details of the setup:

Continue reading Docker macvlan and ipvlan network plugins →

Macvlan and ipvlan in CoreOS

Containers & Kubernetes, DevOps & Automation, , , , ,

This is a continuation of my previous blog on macvlan and ipvlan Linux network drivers. In this blog, I will cover usage of macvlan and ipvlan network plugins with CoreOS Rkt Container runtime and CNI(Container network interface).

Rkt and CNI

Rkt is another Container runtime similar to Docker. CNI is Container networking standard proposed by CoreOS and few other companies. CNI exposes standard APIs that network plugins needs to implement. CNI supports plugins like ptp, bridge, macvlan, ipvlan and flannel. IPAM can be managed by a second level plugin that CNI plugin calls.

Pre-requisites

We can either use multi-node CoreOS cluster or a single node CoreOS for the macvlan example used in this blog. I have created three CoreOS cluster using Vagrant. Following is the cloud-config user-data that I used.

macvlan and ipvlan config

Following is the relevant section of Cloud-config for macvlan:

- path: "/etc/rkt/net.d/20-lannet.conf"
    permissions: "0644"
    owner: "root"
    content: |
      {
        "name": "lannet",
        "type": "macvlan",
        "master": "eth0",
        "ipam": {
          "type": "host-local",
          "subnet": "20.1.1.0/24"
        }
      }

In the above cloud-config, we specify the properties of macvlan plugin that includes the parent interface over which macvlan will reside. We use IPAM type as “host-local” here, this means IP address will be assigned from within the range “20.1.1.0/24” as specified in the configuration. The macvlan type defaults to “bridge”.

Following is the relevant section of cloud-config for ipvlan:

Continue reading Macvlan and ipvlan in CoreOS →

Macvlan and IPvlan basics

Containers & Kubernetes, , ,

Macvlan and ipvlan are Linux network drivers that exposes underlay or host interfaces directly to VMs or Containers running in the host. In this blog, I will cover basics of macvlan and ipvlan, compare macvlan and ipvlan to Linux bridge and sub-interfaces and also show how to create these interfaces in Linux system. In the next set of blogs, I will cover how macvlan and ipvlan interfaces are used in Docker and CoreOS.

VM and Container networking

When running a baremetal server, host networking can be straightforward with few ethernet interfaces and a default gateway providing external connectivity. When we run multiple VMs in a host, it is needed to provide connectivity between VMs within the host and across hosts. On an average, the number of VMs in a  single host does not exceed 15-20. When running Containers in a host, the number of Containers in a single host can easily exceed 100. It is needed to have sophisticated mechanism to interconnect Containers. Broadly, there are two ways for Containers or VMs to communicate to each other. In Underlay network approach, VMs or Containers are directly exposed to host network. Bridge, macvlan and ipvlan network drivers are examples of this approach. In Overlay network approach, there is an additional level of encapsulation like VXLAN, NVGRE between the Container/VM network and the underlay network.

Linux Bridge

Linux Bridge acts like a regular hardware switch with learning and also supports protocols like STP for loop prevention. In linux bridge implementation, VMs or Containers will connect to bridge and bridge will connect to outside world. For external connectivity, we would need to use NAT. The following picture shows 2 Containers connected to a Linux bridge with ethx interface providing external connectivity.

Continue reading Macvlan and IPvlan basics →

Experimental Docker with Docker machine

Containers & Kubernetes, Storage & Virtualization, , ,

Docker Experimental channel is used to release experimental Docker features so that Docker users can try the new features and provide feedback.  It is nice to use the experimental Docker in a test environment rather than upgrading Docker in the main development machine. The preferred approach is to use docker-machine and create a VM with experimental Docker. In this blog, I will describe the approach that I use to create docker-machine with experimental Docker VM. For basics of Docker machine, please refer to my blog on Docker machine.

Following are the steps needed to build the experimental boot2docker ISO and copy it to the docker-machine default location:

git clone https://github.com/boot2docker/boot2docker.git
cd boot2docker
docker build -t my-boot2docker-img -f Dockerfile.experimental .
docker run --rm my-boot2docker-img > boot2docker.iso
mv boot2docker.iso ~/.docker/machine/cache/boot2docker.iso

We need to specify Docker experimental release location in Dockerfile.experimental. In this case, it is https://experimental.docker.com/builds/Linux/x86_64/docker-latest.

Following command will start a Docker machine in Virtualbox with experimental Docker:

docker-machine create -d virtualbox exp

Following is the experimental Docker version running in my host:

$ docker --version
Docker version 1.11.0-dev, build 6c2f438, experimental

Installing custom software in boot2Docker image:
I had a recent usecase where I needed boot2docker to have ipvsadm installed. Package manager is not available in boot2docker. Other than installing ipvsadm, I had to copy few libraries. Following is my boot2docker.experimental file that I used for this usecase:

FROM boot2docker/boot2docker
MAINTAINER Sreenivas Makam ""

#DESCRIPTION use the latest experimental build of Docker

RUN apt-get update && apt-get install -y ipvsadm
RUN cp /sbin/ipvsadm $ROOTFS/sbin/
RUN cp /lib/x86_64-linux-gnu/libnl-genl-3.so.200 /rootfs/lib/libnl-genl-3.so.200
RUN cp /lib/x86_64-linux-gnu/libnl-3.so.200 /rootfs/lib/libnl-3.so.200
RUN cp /lib/x86_64-linux-gnu/libpopt.so.0 /rootfs/lib/libpopt.so.0

#get the latest experimental docker
RUN cd $ROOTFS/usr/local/bin && curl -fL -O https://experimental.docker.com/builds/Linux/x86_64/docker-1.12.0-rc4.tgz && tar -xvzf docker-1.12.0-rc4.tgz --strip-components=1 && chmod +x $ROOTFS/usr/local/bin/docker* && rm docker-1.12.0-rc4.tgz

RUN echo "" >> $ROOTFS/etc/motd
RUN echo "  WARNING: this is an experimental.docker.com build, not a release." >> $ROOTFS/etc/motd
RUN echo "" >> $ROOTFS/etc/motd

RUN /make_iso.sh
CMD ["cat", "boot2docker.iso"]

Issue faced:
I was not able to use custom Docker image with docker-machine version 0.8.0-rc1. I could not find an option to prevent docker-machine from downloading latest Docker image. I have opened an issue here. The only workaround I found was to copy boot2docker image to ~/.docker/machine/cache/ , remove internet connection and then create docker-machine host.

References:

Contiv Networking policy Hands-on

Containers & Kubernetes, DevOps & Automation, Networking & SDN, , , ,

Contiv is an Open source project driven primarily by Cisco for policy based networking, storage and cluster management for containerized applications. In this blog, I will cover some of the hands-on stuff that I tried with Contiv Networking. I used the sample examples provided in Contiv documentation as starting point. For Contiv networking basics, you can refer to my previous blog here.

Contiv environment

I followed the “Contiv getting started” guide to setup a two node Contiv cluster with Vagrant. I started the cluster in Packet baremetal cloud. Contiv netmaster runs in one of the nodes, Contiv netplugin is installed in both the nodes.

git clone https://github.com/contiv/netplugin
cd netplugin; make demo

Following command shows the 2 node Vagrant cluster:

root@contiv:~/netplugin# vagrant status
Current machine states:

netplugin-node1           running (virtualbox)
netplugin-node2           running (virtualbox)

Following are the business details of the sample application that I have used in this blog:

Continue reading Contiv Networking policy Hands-on →

Contiv – Policy based networking for Containers

Cloud Platforms, Containers & Kubernetes, Networking & SDN, , , ,

Contiv is an Open source project driven primarily by Cisco for policy based networking, storage and cluster management for containerized applications. In this blog, I will focus on how Contiv does policy based Container networking. In the next blog, I will cover some hands-on stuff that I tried with Contiv.

Container Policy

Policies have become critical to control the business logic in a Cloud environment. There are 2 ways to describe policy. In imperative model, policy is defined in terms of how the end goal is achieved. For example, we specify the filters and actions with Openflow protocol that achieves end goal of packet handling and this is an example of imperative model. In declarative model, policy is defined in terms of the end goal and it gives flexibility to the end-system to implement the policy in different ways. Congress and Opflex are examples of declarative policy model. With declarative model, it is possible to specify the policy in terms of business logic without specifying implementation detail. For example, the business logic can say that web container should not talk to database container. The implementation of this business logic can be achieved by having an iptables rule or by having a hardware tcam rule to block specific ports. In a cloud computing world, policies can be defined for compute, storage and networking. Both Containers and VM needs policies to implement business logic. Following are examples of some policies that can be applied to applications deployed in Cloud using either VMs or Containers:

  • Authorization policy – Specifies tenants and their privileges.
  • Resource usage policy – Specifies resource constraints for tenants, containers and VMs.
  • Application access policy – Specifies containers that can communicate to each other and containers that are exposed to outside world.

Contiv Networking

Contiv Networking project provides policy based networking for Docker Containers. Following are some details on Contiv Networking:

Continue reading Contiv – Policy based networking for Containers →

Openstack Deployment using Containers

Cloud Platforms, Containers & Kubernetes, DevOps & Automation, , , , , ,

I recently saw the Openstack self-healing demo from CoreOS team using Tectonic(Stackanetes project) and I kind of felt that the boundary between Containers and VMs are blurring. In this blog, I discuss the usecase of deploying Openstack using Containers.

We typically think of Openstack as a VM Orchestration tool. Openstack is composed of numerous services and deploying Openstack as one monolithic blob is pretty complex and difficult to maintain. The demo described showed how Containers simplify Openstack deployment. This is a great example of using Microservices architecture to simplify infrastructure deployment.

Following diagram shows the Openstack deployment model using Containers. The diagram below shows how Openstack service containers deploys user VM. The user VMs deployed using Openstack can run Containers as well..

vm_container1.PNG

Following are some notes on the architecture:

  • Openstack services like Nova, Heat, Horizon are containerized using Openstack Kolla project as Docker Containers. Some Openstack services like Nova is composed of multiple Containers.
  • Infrastructure components like Ceph, Openvswitch, Mongodb are also deployed as Containers.
  • For Container deployment, Openstack natively uses Ansible. Kubernetes can also be used for Orchestration.
  • Using Containers for Openstack service containers gives all the build, ship and deploy advantages of Containers.
  • Using orchestration solution like Kubernetes gives all the resiliency and deployment advantages for Openstack services.

This work also shows how Containers and VMs can work closely with each other for lot of use-cases. There are other Openstack projects like Magnum and Kuryr where there is an intersection between Containers and VMs. Magnum project deals with Container orchestration using Openstack and Kuryr project deals with doing Container networking using Openstack Neutron.

References:

Opensource Meetup Presentation

Containers & Kubernetes, DevOps & Automation, Distributed Systems, , , , , ,

I did a presentation on CoreOS and Service Discovery in Opensource Meetup group last week. Following are related slides and demo recording.

CoreOS Overview and Current Status

Slides:

CoreOS HA Demo recording:

Scripts used are available here.

Service Discovery using etcd, Consul and Kubernetes

Slides:

Consul Service Discovery Demo:

Following are the commands to start Consul Container, Registrator Container and 3 Container services.

docker run -d -p 8500:8500 -p 192.168.0.1:53:8600/udp -p 8400:8400 gliderlabs/consul-server -node myconsul -bootstrap
docker run -d -v /var/run/docker.sock:/tmp/docker.sock --net=host gliderlabs/registrator -internal consul://localhost:8500

docker run -d -p :80 -e "SERVICE_80_NAME=http" -e "SERVICE_80_ID=http1" -e "SERVICE_80_CHECK_HTTP=true" -e "SERVICE_80_CHECK_HTTP=/" --name=nginx1 nginx
docker run -d -p :80 -e "SERVICE_80_NAME=http" -e "SERVICE_80_ID=http2" -e "SERVICE_80_CHECK_HTTP=true" -e "SERVICE_80_CHECK_HTTP=/" --name=nginx2 nginx
docker run -ti smakam/myubuntu:v3 bash