Opencompute Networking – Facebook Contributions

Networking & SDN, ,

Earlier, I had written a blog on Opencompute networking project. There were few recent contributions from Facebook to Opencompute. In this blog, I wanted to capture major Opencompute Networking contributions from Facebook. I see these contributions as pretty significant in the context of White box switches.

Wedge hardware design for TOR:

The goal here was to develop modular switches where any hardware component of the switch can be replaced independently. Servers in the TOR use the Group Hug architecture where standard connector is used to connect the servers to the motherboard. This can allow a easier change of processor.

Continue reading Opencompute Networking – Facebook Contributions →

AWS ECS(EC2 Container service)

Cloud Platforms, Containers & Kubernetes, ,

I recently tried out AWS ECS(EC2 Container service). In this blog, I will provide overview of EC2 container service, some hands-on stuff I tried and my experiences with ECS. AWS ECS is available as a preview currently with only CLI interface. I assume general availability with web interface will be available soon. Amazon does not charge separately for the ECS service, instances used for containers will be charged appropriately. I have a AWS free-tier account and I was able to try the ECS with the free-tier account.

Pre-requisites:

  • Familiarity with Containers and Docker. You can refer to my blog on Docker series.
  • Familiarity with AWS services. You can refer to my previous blogs on AWS overview and how to access AWS services.
  • For the hands-on stuff, you need to have atleast a free-tier account with AWS. AWS cli needs to be installed since ECS is available only with a CLI interface currently.

Overview of ECS:

I had written a blog earlier on Docker Orchestration. ECS is a Docker Container Orchestration service that provides the following functionalities.

  • Allows to deploy Containers across a cluster of hosts using either AWS scheduler or integrate with third party schedulers like Mesos.
  • Allows management of an application that spans multiple containers rather than just managing containers.
  • Provides high availability for Containers by monitoring Container health.

Following are components of ECS:

  • Cluster – Logical entity that contains multiple EC2 instances. Instances within the container can be of different types.
  • Container instances – Each instance is an EC2 instance with Docker and Container agent pre-installed and is available as AWS AMI image. Container agent registers itself to the cluster and it monitors health of the Containers in the host. AWS is open-sourcing the Container agent project and its available in github.
  • Tasks – Tasks contains the application definition, resources needed and it can span multiple containers. Task definitions are specified in json format.
  • ECS Master – This manages everything. ECS master is responsible for scheduling the containers and also for talking to the container agent to get the health of container instances. When ECS master schedules the containers, it takes into account resources needed for the container and the resources available in the container instance.

ECS hands-on:

Most of the steps below are as mentioned in AWS ECS developer guide.

Continue reading AWS ECS(EC2 Container service) →

Openstack Juno – Management interfaces

Cloud Platforms, DevOps & Automation, Programming, , , ,

This blog is part of my series on Openstack Juno. In this blog, I will cover different management interfaces to Openstack. Following are the different management interfaces available.

  • Horizon web interface
  • CLI interface to each service. CLI interface is provided by Python script. Internally, the script calls the REST interface.
  • REST interface. This is accessible either through Curl or a POSTMAN kind of client.
  • Programmatic interface using Python SDK.

Horizon interface:

On the host where stacking is done, webserver runs on port 80 and all Openstack services can be configured using this interface. Login to Horizon can be done with either tenant userid or admin userid. Based on the userid, privileges are granted.

CLI interface:

CLI interface is provided for each service. Nova services are accessible through “nova” client, Swift services are accessible with “swift” client and so on. Following example lists running VMs. Continue reading Openstack Juno – Management interfaces →

Openstack Juno – Group based policy

Cloud Platforms, , ,

This blog is part of my series on Openstack Juno. In this blog, I will cover Group based policy in Openstack Juno. For more information on Group based policy(GBP), please refer to my earlier blogs on GBP basics and ODL integration.

Group based policy(GBP) support in Juno:

Preliminary support for GBP is available in Juno, more functionality will come in Kilo release. We need to pull a GBP label based devstack to get the GBP functionality. I followed the instructions in Openstack GBP wiki to try out GBP functionality with devstack. The exercise walks through a dummy application stack with 2 client groups and 1 web server group and having some policies between the 3 group application to control the interactions.

Following are some advantages that I realized as part of trying the exercise:

Continue reading Openstack Juno – Group based policy →

Openstack Juno – Multihost Networking

Cloud Platforms, , ,

This blog is part of my series on Openstack Juno.

In this blog, I will cover adding a compute host to the Openstack Devstack cluster and connect the 2 hosts using either vxlan or gre encapsulation.
Lets first do the stacking on the controller host using the steps mentioned in my previous blog.
Lets look at the hypervisor list. As expected, we see only 1 host.

$ nova hypervisor-list
+----+-------------------------+
| ID | Hypervisor hostname     |
+----+-------------------------+
| 1  | sreeubuntu14-VirtualBox |
+----+-------------------------+

Lets look at the networks created by default:

Continue reading Openstack Juno – Multihost Networking →

Openstack Juno services – Swift, Glance, Heat, Ceilometer

Cloud Platforms, , , , , ,

This blog is part of my series on Openstack Juno. In this blog, I will cover the usage of Openstack services Swift, Glance, Heat, Ceilometer.

Swift:

Swift is used for Object based storage. Its similar to AWS S3 service.
First, create a container to store objects:

$ swift post mycont

Upload a file to the container

$ swift upload mycont local.conf

List container and objects.

$ swift list
mycont
$ swift list mycont
local.conf

List status of created container.

Continue reading Openstack Juno services – Swift, Glance, Heat, Ceilometer →

Openstack Juno services – Nova, Cinder

Cloud Platforms, , , ,

This blog is part of my series on Openstack Juno. In this blog, I will cover the usage of Openstack services Nova, Cinder.

I found this blog on Openstack services good in giving a highlevel overview of services and comparing individual Openstack services with Amazon AWS services.

Nova basics:

Nova is the Openstack compute service.
Following command shows the flavors available by default. Flavors are different configurations(CPU, memory, hard disk) for the VMs that can be created from Nova.

Continue reading Openstack Juno services – Nova, Cinder →

Openstack Juno Install using Devstack

Cloud Platforms, ,

Earlier, I had written 2 blogs on Openstack devstack Icehouse installation and networking. The feedback was positive on these blogs. This prompted me to do a series on different aspects of Openstack Juno. Openstack Juno is the 10th and latest release of Openstack that got released in October 2014. In the next series of blogs, I will cover the following aspects of Openstack Juno.

  • Openstack Juno devstack installation and debugging.
  • Accessing Openstack services through CLI, API and SDK.
  • Overview of different Openstack services along with a demo of their usage.
  • Multi-node devstack installation.

In this blog, I will cover the steps to install Openstack Juno in Devstack.

My environment:

  • Windows 7 with Virtualbox 4.3.20
  • 24 GB RAM (Devstack install per VM requires a minimum of 4GB)

Installation steps:

Continue reading Openstack Juno Install using Devstack →

Containers – Docker, LXC and Rkt

Containers & Kubernetes, , ,

This blog is part of my ongoing series on Docker containers. In my previous blog, I covered LXC. When I tried out LXC, I realized that there are lots of similarities between Docker and LXC. Also, I saw a recent announcement about Rkt which is another Container runtime technology. In this blog, I have tried to answer multiple questions that I had about these technologies based on reading through the reference materials mentioned below. This is a pretty controversial topic as folks have strong opinions about these technologies, I have tried to keep it as neutral as possible.

How is Container management different from Container technologies?

I found this diagram from Docker blog very helpful in answering the above question.

docker3

Linux kernel has support for Container technologies like namespaces, cgroups etc. Docker, LXC and Rocket use the technologies available in Linux kernel to manage the lifecycle of the Container. Container management involves Container creation, deletion and modification, image format and the tools around it. Before Docker version 0.9, Docker was using LXC to interact with Linux kernel. From Docker version 0.9, Docker directly interacts with Linux kernel using libcontainer interface that they developed.

How is Docker different from LXC?

Continue reading Containers – Docker, LXC and Rkt →