Tag Archives: Docker

Hashicorp Atlas workflow with Vagrant, Packer and Terraform

Cloud Platforms, Containers & Kubernetes, DevOps & Automation, Storage & Virtualization, , , , , , , , ,

I have used and loved Vagrant for a long time and I recently used Consul and I was very impressed by both these Devops tools. Recently, I saw some of the videos of Hashiconf and I learnt that Hashicorp has an ecosystem of tools addressing Devops needs and that these tools can be chained together to create complete application delivery platform from development to production. Atlas is Hashicorp’s product that combines its open source tools into a platform and it has a commercial version as well. In this blog, I will cover a development to production workflow for a LAMP application stack using Atlas, Vagrant, Packer and Terraform.

Overview of Vagrant, Packer, Terraform and Atlas

Vagrant

Vagrant provides a repeatable VM development environment. Vagrant integrates well with major hypervisors like Virtualbox, VMWare, HyperV. “Vagrantfile” describes the VM settings as well as initial bootstrap provisioning that needs to be done on the VM. Vagrant also integrates well with other provisioning tools like Chef, Ruby and Ansible to describe the provisioning. Simply by doing “vagrant up”, the complete VM environment is exactly reproduced. The typical problems like “it does not work for me even though its working in your machine” goes away.

Packer

Packer is a tool to create machine images for providers like Virtualbox, VMWare, AWS, Google cloud. Packer configuration is described as a JSON file and images for multiple providers can be created in parallel. The typical workflow is for developer to create development environment in Vagrant and once it becomes stable, the production image can be built from Packer. Since the provisioning part is baked into the image, the deployment of production images becomes much faster. Following link describes how Vagrant and Packer fits well together.

Continue reading Hashicorp Atlas workflow with Vagrant, Packer and Terraform →

Trying out Docker in Windows

Containers & Kubernetes, ,

Few folks recently asked me how to try out Docker in Windows machine to get familiar with Docker and Containers. This is not the same as running Docker engine in Windows, there is some active work going on to achieve this. The easiest way to try out Docker is to use Docker Toolbox. I covered Toolbox in one of my earlier blogs. In this blog, I will provide the steps to create a simple environment to play with Docker engine, Swarm, Compose and Networking in Windows.

Pre-requisites:

  • Install Docker Toolbox from here.
  • Docker Toolbox 1.9 version has the following components(Docker client, Docker machine, Virtualbox, Kitematic, Compose, Git). If you already have Virtualbox and Git bash for Windows, you can skip the installation of these 2.
  • In this blog, I will use Docker CLI, so Kitematic is not needed.
  • I have executed commands below from git bash shell.

Following are the versions of Docker components that gets installed with Toolbox 1.9:

Continue reading Trying out Docker in Windows →

Preview launch of my book “Mastering CoreOS”

Containers & Kubernetes, DevOps & Automation, , , ,

Last 6 months, I have been blogging very little since I was busy writing a book on CoreOS. The book is available for pre-ordering now from the publisher website, Amazon US and Amazon India. Discounts are available till Feb 29th from publisher and discount code is available here. The tentative publishing date is early end of Feb/early March.

Why I wrote the book other than the fact that I can make some money out of it?
I started blogging around later part of 2013. The first open source project that I tried was Opendaylight. It was very exciting to create virtual networks using Mininet and manage it with Openflow and Opendaylight. In early 2014, I focused on cloud technologies like AWS, Google Cloud and Openstack and learnt how Public and Private clouds revolutionized how we consume resources. Around August 2014, I started spending time on Devops and learnt how tools like Ansible eased the management pain not just in server infrastructure but also in networking infrastructure. From December 2014, I started spending time on Docker and CoreOS. The first Docker version I used was 1.3.1(Now its 1.9) and first CoreOS version I used was 500.x(Now its 933.x). Even though Microservices and Containers were present before, Docker made Containers easier to use which in turn made Microservices popular. CoreOS pretty much invented the Cluster OS or Container-optimized OS category and provided OS and tools that ease Container deployment. By writing this book, I felt that I could connect the dots between SDN, Cloud, Devops technologies that I focussed on earlier with Docker and CoreOS.

What are some questions the book tries to answer?
What is a Container-Optimized OS?
Why CoreOS and Containers are needed?
How to deploy microservices and distributed applications?
How to setup, maintain and update CoreOS cluster?
What role does key CoreOS services etcd, fleet, systemd play?
How does Docker and CoreOS manage networking and storage?
What role does standards play in Containers?
Why there are multiple Container runtimes like Docker and Rkt?
How does Kubernetes and Swarm orchestrate Container deployment?
How is Openstack, AWS ECS, Google Container engine, Tectonic related to CoreOS and Docker?
How to debug Containers and CoreOS?
What are the production considerations when deploying microservices?

What did I learn?
That there is a lot more to learn..
Open source is very powerful and it aids innovation and collaboration
Trying hands-on is more important than reading through software manuals
Can learn a lot from Conference Video recordings/slides, blogs
Need lot more patience for writing a book when compared to writing a blog
It is good to do something different part-time from daily office job to keep life interesting

If you read the book or read few chapters of the book, please feel free to provide your feedback..

Thanks to CoreOS, Docker and Container community for the amazing technologies that they have developed. Big thanks to Open source community for making software easily accessible to everyone.

Gopaddle Meetup Bangalore – CI, CD Presentation

Containers & Kubernetes, DevOps & Automation, , ,

Following link captures the slides on CI, CD with Docker, Jenkins and Tutum that I presented at GoPaddle meetup, Bangalore on January 23, 2015. You can find more details on the meetup here. In this presentation, I covered the following:

  • Overview of Continuous Integration(CI), Continuous deployment(CD)
  • Tutum Overview
  • Jenkins with Docker Integration
  • CI, CD Use cases with Docker and Jenkins, Docker and Tutum

CI, CD with Docker, Jenkins and Tutum

Containers & Kubernetes, DevOps & Automation, , ,

In this blog, I will give an overview of Continuous Integration (CI) and Continuous Deployment (CD) and cover few CI, CD Use cases with Docker, Jenkins and Tutum. Docker provides Container runtime and tools around Containers to create a Container platform. Jenkins is a CI/CD application to build, test and deploy applications. Tutum is a SaaS Container platform that can be used to build, deploy and manage Docker Containers. I have covered overview of Tutum in previous blog. There are Use cases where these applications work well with each other and the Use cases in this blog will illustrate that.

CI/CD Overview

Traditional approach of releasing software has the following problems:

  • Software release cycles were spaced apart which caused new features taking longer time to reach the customers.
  • Majority of the process of getting software from development stage to production was manual.
  • Considering the different deployment scenarios, it was difficult to guarantee that software worked in all environments and configurations.

Containers have tried to mitigate some of the problems above. By using microservices and Container approach, it is guaranteed that the application will behave similar in development and production stages. Process automation and appropriate testing is still needed for Container based environment.

Continuous Integration refers to the process of making an executable or a Container image automatically after developer has done the UT and commit.

Continuous delivery refers to the process of taking the developer built image, setting up the staging environment for testing the image and deploying it successfully for production.

Following diagram show the different stages of CI, CD cycle.

cicd1

Following are some notes on the above diagram:

Continue reading CI, CD with Docker, Jenkins and Tutum →

Tutum Introduction

Containers & Kubernetes, ,

Tutum is a SaaS Container platform that can be used to build, deploy and manage Docker Containers. Docker acquired Tutum in October 2015. I have been playing with Tutum for the past few weeks and I will share some basics of Tutum and my experiences with Tutum in this blog.

Advantages

  • Very easy to get started and create Containers and multi-Container applications
  • Works with majority of cloud providers like AWS, GCE and our own node can also be used.
  • Hides orchestration, networking and storage complexity.
  • Provides GUI, CLI as well as api access
  • Gives own private repository within Tutum per user, other repositories can also be used
  • Logging, monitoring, Scaling and HA is built-in
  • Does rolling upgrade automatically
  • Suited for CI, CD Usecases

What can we do with Tutum?

  • Build Docker images from Dockerfiles
  • Have a private repository in Tutum for storing Docker images and also interact with already existing repositories
  • Deploy single container and multi-container application in cloud provider or in our own machine
  • Integrate with Github for automatic Container image build and deploy Containers automatically using rolling upgrade
  • Monitor, debug, manage and upgrade Containers and Services

Getting Started

  • Need Tutum account – Free at this point since Tutum is still in Beta
  • Cloud provider account or own node

Continue reading Tutum Introduction →

Docker Toolbox

Containers & Kubernetes,

Docker Toolbox simplifies the creation of Docker environment for Windows and Mac. This deprecates boot2docker. Following components are included in Docker Toolbox.

  • Docker Client
  • Docker Machine
  • Docker Compose (Mac only)
  • Docker Kitematic
  • VirtualBox

I recently tried out Docker Toolbox. I had few issues to get it working and after some hiccups, I was able to get it working with some help from mailers. In this blog, I will share my experiences.

Continue reading Docker Toolbox →

Docker Experimental Networking – 3

Cloud Platforms, Containers & Kubernetes, ,

This blog is a continuation of my previous blog on Docker Experimental Networking. In this blog, I will walk through the example mentioned in this link where experimental Docker is integrated with Compose and Swarm. I have made some modifications here and I will cover this here.

I will create 2 applications in this example using Docker Compose.

  1. Counter container connecting to redis container running on 2 different hosts.
  2. WordPress container connecting to mysql container running on 2 different hosts.

I have used AWS instead of Digitalocean. First step is to create Consul machine and start Consul server.

docker-machine create --driver=amazonec2 --amazonec2-access-key=xxx --amazonec2-secret-key=xxx --amazonec2-vpc-id=vpc-5f77c23a --amazonec2-region=us-west-2 --engine-install-url "https://experimental.docker.com" consul

docker $(docker-machine config consul) run -d \
    -p "8500:8500" \
    -h "consul" \
    progrium/consul -server -bootstrap

Next, create 2 machines connecting both the nodes to Consul:

Continue reading Docker Experimental Networking – 3 →