Tag Archives: Google cloud

VPC native GKE clusters – IP address management

Cloud Platforms, ,

This blog was written by me after a long gap of close to 7 months. Many reasons including busy work schedule, some health issues in the middle and a little bit of laziness contributed to this. I hope to be a more active blogger going forward.

In this blog series, I will cover the following topics:

The first blog in this series will talk about GKE default IP address management.

Following are the Kubernetes abstractions that needs IP addresses:

  • Node IP address – Assigned to individual nodes. The node ip address is assigned from the VPC subnet range.
  • Pod IP address – Assigned to individual pods. All containers within a single pod share same IP address.
  • Service IP address- Assigned to individual service

By default, “/14” address gets allocated for cluster IP range. Pod and service IP addresses comes out this pool. “/24” address that comes out of the cluster IP range gets assigned to each individual node and is used for pod IP allocation. “/20” address that comes out of the cluster IP range gets assigned for Kubernetes services. The user has a choice to select cluster IP range when creating the cluster.

To illustrate some of the above points, I have created a 3 node Kubernetes cluster with IP aliasing disabled. By default, VPC native clusters(ip aliasing enabled) is disabled and has to enabled manually. In the future GKE release, VPC native clusters will be the default mechanism.

Cluster output:

Continue reading VPC native GKE clusters – IP address management →

My Data engineer certification notes

Cloud Platforms

This blog is more of a quick reference notes rather than a real blog. I shared this with few Google internal folks and since they found it useful, I thought I will share these notes in the blog as well.

Please note that the Data engineer exam format changed a little from March 29, 2019. I took this exam in January 2019, so some of the notes below might not apply. The 2 big changes from before are that there are no case studies in the new format and certain new data and ML products like composer, Automl, kubeflow have been introduced.

There are many references on preparation steps for GCP Data engineer exam. Why am I writing 1 more? What I found reading through an individual’s experience is that I always get some new insight. I am hoping that this write up would help some new person preparing for the exam.

I cleared the exam last week of January 2019. I come from an infrastructure background, Data engineer was not my area of strength and this needed around a total of 8 week preparation time for me. The 8 weeks is not a dedicated time but few hours split over a 8 week period. I have been with Google cloud team for 1+ years and I found that the practical experience is important for the exam.

The exam is an objective choice with 50 questions to be done in 2 hours time frame. There are also questions asked regarding the 2 case studies. For my preparation, I used the data engineer modules of Coursera training and Linux academy. It will be good to review and breakdown the case studies beforehand so that we can avoid spending time reading the case study text in the exam. Qwiklabs and the labs associated with Coursera helps quite a bit. I have added the links in the references.

The certification tests both the theoretical knowledge as well as practical knowledge using the GCP products and technologies. My suggestion is to take the exam after getting some practical experience with GCP. Another thing that I noticed in the exam is for some questions, more than 1 answer would seem appropriate and it takes thorough analysis before answering. 1 tip would be to mark these questions for review and spend time on these after answering all other questions.

Continue reading My Data engineer certification notes →

NEXT 100 Webinar – Top 3 reasons why you should run your enterprise workloads on GKE

Cloud Platforms, Containers & Kubernetes, , , ,

I presented this webinar “Top 3 reasons why you should run your enterprise workloads on GKE” at NEXT100 CIO forum earlier this week. Businesses are increasingly moving to Containers and Kubernetes to simplify and speed up their application development and deployment. The slides and demo covers the top reasons why Google Kubernetes engine(GKE) is one of the best Container management platforms for enterprises to deploy their containerized workloads.

Following are the slides and recording:

Recording link

 

 

Container Conference Presentation

Cloud Platforms, Containers & Kubernetes, , ,

This week, I did a presentation in Container Conference, Bangalore. The conference was well conducted and it was attended by 400+ quality attendees. I enjoyed some of the sessions and also had fun talking to attendees. The topic I presented was “Deep dive into Kubernetes Networking”. Other than covering Kubernetes networking basics, I also touched on Network control policy, Istio service mesh, hybrid cloud and best practises.

Slides:

Recording:

Demo code and Instructions:

Github link

Recording of the Istio section of the demo: (the recording was not at conference)

As always, feedback is welcome.

I was out of blogging action for last 9 months as I was settling into my new Job at Google and I also had to take care of some personal stuff. Things are getting little clear now and I am hoping to start my blogging soon…

 

My new Journey – Cisco to Google cloud

Cloud Platforms, Networking & SDN,

After an amazing 10+ years in Cisco Systems, I have decided to move on. I have joined Google’s cloud division. I thought its a good time to reflect on my learnings in Cisco and what I am looking forward for the next few years.

Before joining Cisco, I worked in few startups in US. I joined Cisco after I moved back to India. I worked in different development engineering groups in Cisco spanning carrier ethernet, service provider and data center products. I played different roles including Software Engineer, Technical lead/architect, Engineering manager. Cisco is a great company and it has given me lot of good opportunities.

Continue reading My new Journey – Cisco to Google cloud →

Comparing Docker deployment options in public cloud

Cloud Platforms, Containers & Kubernetes, , , , ,

Few weeks back, I gave a presentation in Container conference, Bangalore comparing different solutions available to deploy Docker in the public cloud.

Slides are available here. I have also put the steps necessary along with short video for each of the options in the github page here.

Abstract of the talk:

Containers provide portability for applications across private and public clouds. Since there are many options to deploy Docker Containers in public cloud, customers get confused in the decision making process. I will compare Docker machine, Docker Cloud, Docker datacenter, Docker for AWS, Azure and Google cloud, AWS ECS, Google Container engine, Azure Container service. A sample multi-container application will be deployed using the different options. The deployment differences including technical internals for each option will be covered. At the end of the session, the user will be able to choose the right Docker deployment option for their use-case.

Note:

  • I have focused mainly on Docker centric options in the comparison.
  • There are few CaaS platforms like Tectonic, Rancher that I have not included since I did not get a chance to try them.
  • Since all the solutions are under active development, some of the gaps will get covered by the solutions in the future.

Microservices Infrastructure using Mantl

Cloud Platforms, Containers & Kubernetes, DevOps & Automation, Distributed Systems, Networking & SDN, , , , , , , , , , ,

Mantl is an Open source project from Cisco and it provides an integrated solution to deploy distributed Microservices. Any company deploying Microservices has to integrate different components before the solution becomes production ready. Mantl makes it easier by integrating the different components and providing the glue software that integrates the components. In this blog, I will cover the following:

  • Distributed Microservice infrastructure components and the need for Mantl.
  • Mantl Architecture.
  • Mantl installation using Vagrant
  • Mantl installation using AWS public cloud

Microservices infrastructure

Following are typical components in Container based Microservices infrastructure:

Continue reading Microservices Infrastructure using Mantl →

Baremetal cloud using Packet

Cloud Platforms, Containers & Kubernetes, Networking & SDN, Storage & Virtualization, , , , ,

Typical Opensource demo applications comes packaged as a Vagrant application which starts a bunch of VMs and does automatic provisioning. I have a Windows machine with Virtualbox and VMWare player installed. Since Virtualbox does not support nested virtualization with 64 bit VMs(More details can be found in my previous blogs on Virtualbox and VMWare player), I use VMWare player to try out demo applications that needs 64 bit VMs. The demo applications typically run on Linux, so running them on Windows with Virtualbox is ruled out. I was recently trying this Mantl project for deploying distributed microservices and I found that it was very slow to run in VMWare player with nested virtualization. I tried to run the application in AWS and I found that AWS does not support nested virtualization(More details can be found here). Then I tried out Google cloud. Even though Google cloud supports nested virtualization, hardware virtualization is disabled on the guest VMs and this prevents running 64 bit VMs inside Google cloud VMs. After I ran out of these options, I stumbled upon the possibility of using baremetal cloud. I used baremetal cloud from Packet and it worked great for my usecase mentioned above. Though this is not a typical use case, I was very happy with the performance and the possibilities that this provides. In this blog, I will share the use cases for baremetal cloud and my experiences with using Packet service.

Bare metal cloud Use case

Typical cloud providers like Amazon, Google, Digitalocean, Microsoft rent out VMs as part of their compute offering. These VMs run on top of a hypervisor. Though the user is guaranteed a specific performance, these VMs share the same resources with other VMs running on the same host machine. With bare metal cloud, the cloud provider hosts machines that the user can rent which is not shared with anyone. Cloud providers provide different configurations for bare metal and the user can choose based on their performance needs and the costing is based on the performance provided by the bare metal server. Following are some advantages that bare metal cloud provides:

Continue reading Baremetal cloud using Packet →

Hashicorp Atlas workflow with Vagrant, Packer and Terraform

Cloud Platforms, Containers & Kubernetes, DevOps & Automation, Storage & Virtualization, , , , , , , , ,

I have used and loved Vagrant for a long time and I recently used Consul and I was very impressed by both these Devops tools. Recently, I saw some of the videos of Hashiconf and I learnt that Hashicorp has an ecosystem of tools addressing Devops needs and that these tools can be chained together to create complete application delivery platform from development to production. Atlas is Hashicorp’s product that combines its open source tools into a platform and it has a commercial version as well. In this blog, I will cover a development to production workflow for a LAMP application stack using Atlas, Vagrant, Packer and Terraform.

Overview of Vagrant, Packer, Terraform and Atlas

Vagrant

Vagrant provides a repeatable VM development environment. Vagrant integrates well with major hypervisors like Virtualbox, VMWare, HyperV. “Vagrantfile” describes the VM settings as well as initial bootstrap provisioning that needs to be done on the VM. Vagrant also integrates well with other provisioning tools like Chef, Ruby and Ansible to describe the provisioning. Simply by doing “vagrant up”, the complete VM environment is exactly reproduced. The typical problems like “it does not work for me even though its working in your machine” goes away.

Packer

Packer is a tool to create machine images for providers like Virtualbox, VMWare, AWS, Google cloud. Packer configuration is described as a JSON file and images for multiple providers can be created in parallel. The typical workflow is for developer to create development environment in Vagrant and once it becomes stable, the production image can be built from Packer. Since the provisioning part is baked into the image, the deployment of production images becomes much faster. Following link describes how Vagrant and Packer fits well together.

Continue reading Hashicorp Atlas workflow with Vagrant, Packer and Terraform →

Kubernetes and Google container engine

Cloud Platforms, Containers & Kubernetes, , ,

In this blog, I will cover the Google container engine service that I tried out.

Pre-requisites:

  • Need Google cloud account.
  • Install Google cloud SDK.

Google container engine is not available in the normal gcloud SDK installation. To use container engine service, we need to update preview component.

$ gcloud components update preview

I followed the 2 examples mentioned in the container engine documentation.
WordPress application:
In this example, we create a cluster which has a single master and single worker node. We create a pod running WordPress container in the cluster and expose the WordPress service to external world. Since there is only 1 pod, we dont create a service.
Following are the commands used:

Continue reading Kubernetes and Google container engine →