Marriage made in heaven - Google + Project Magnum

On July 16, Google announced that it is becoming a sponsor to the OpenStack Foundation.  This announcement immediately generated a lots of excitement and this news is all over the social media.  Immediately lots of blogs and technical news outlets published articles on this announcement.

My immediate reaction was this:

as I express myself on Twitter.  

In this blog post, Product Manager of Google, Craig McLuckie stated that by becoming a sponsor to the OpenStack Foundation, Google is expected to bring valuable knowledge, expertise and deployment experience of container-oriented computing to OpenStack. The blog also stated that Google is seeing 2 emerging trends in the enterprise computing sector:

1. Moving toward the adoption of hybrid cloud (public and private)
2. Moving toward the adoption of container-based computing

Quote directly from the blog post:

"We (Google) will be working over the coming months with the (OpenStack) community to integrate Kubernetes, as well as complementary container technologies, to create a stronger hybrid cloud."

The excitement about Google sponsoring OpenStack Foundation is not just just another "big" company joining IBM to endorse the future of OpenStack but what Google can bring and to enhance what OpenStack can provide for its consumers. Google has been running containers for over 10 years. Google's various applications such as Gmail, web search or map services are running as containers. It is said on the internet that Google launch 2 billions containers in one week.

Why Linux Container?
Linux Container is not a new technology and its origin can trace back to FreeBSD Jails in year 2000. Docker Inc, is the company behind the open source Docker platform. The Docker platform is an orchestration or packaging tool that allows application and their dependencies to run on container technology. Docker makes it easy for developer and sys admin to deploy applications with container technology.

The advantage of Linux container is light weight and fast provision as compare to a virtual machine.  It is also portable

Linux containers is just perfect for DevOps as well as Micro-services architecture.

Linux Container is part of the equation along with DevOps and Micro-services to answer the modern day business requirement – agility. In other words the ability to react to the change of the business environment fast or faster than the competitor.

Both DevOps and Micro-services are big topics that will take up an entire blog post to talk about.

In a nutshell, DevOps is not a technology but a culture and a form of application deployment methodology.  It relies on the continuous feedback model from operation to development so that applications can be deployed in a faster manner.

image source: http://www.appdynamics.com/blog/wp-content/uploads/2013/06/DevOps-infinity-loop2.png

Micro-services is the new and popular way of deploying applications.  Application used to be one monolithic process and is not as agile.  Micro-services breaks the application into smaller processes running together.

 image source: http://www.thoughtworks.com/sites/default/files/assets/MicroservicesFowler.png

DevOps, Linux container and micro-services architecture fits right into each other because they share one common characteristic and that is to break one complex problem into smaller problems which can be resolved easier and faster.

Linux containers are gaining traction in the enterprise IT environment.  In my other blog post, I talked about VMware embracing Docker with Project Bonneville making Docker to run just like a virtual machine in vSphere so that Docker container can take advantage of the matured high availability, security and networking infrastructure that VMware had developed.  

Microsoft is also embracing Docker in its public cloud Azure.

We can see that container is an emerging technology integrating with various enterprise ready technologies independently providing features for consumer of the IT infrastructure or cloud.

What is Kubernetes?

Kubernetes is developed by Google to manage containers in a clustered environment.  Google makes Kubernetes open source and can be found on GitHub here.  

The beauty of open source project is that there is always description of the project.  The  description of Kubernetes on GitHub is:

"Kubernetes is an open source system for managing containerized applications across multiple hosts, providing basic mechanisms for deployment, maintenance, and scaling of applications."

An interesting point to note is that Kubernetes is still considered as pre-production beta status and heavy development is still being done. (Well GitHub said it is in pre-production beta but on July 21 at oscon, Kubernetes 1.0 is released.  Release 1.0 and above is usually production ready).

Kubernetes works off the following concepts:

image source: 13110153-conversion-gate01/95/devnexus-2015-kubernetes-container-engine-17-638.jpg?cb=1426244619

Red Hat also used Kubernetes on their PaaS offering - OpenShift.

I will write up a more detailed blog post on Kubernetes later.  If you cannot wait, take a look at here.

Project Magnum
Before the July 16 Google announcement, OpenStack already has an ongoing project called Magnum - Containers as a Service. 

Project Magnum allows containers be deployed in an OpenStack infrastructure as easy as a virtual machine but in the fashion that container should be deployed.  This opens up a lot more business use case on DevOps and Micro-services for OpenStack.

In Nova, Docker is considered as one of the supported hypervisor type.  The main difference for Magnum over Nova is that Magnum uses container orchestration tool to deploy containers at scale and in a rapid manner.  Deployment characteristic of a container is different than a virtual machine such that container come and go more frequently and containers usually are deployed in group as a cluster and sometimes across multiple hosts.  These containers in the cluster needs interact with each other when they operate together to offer the service of one application to the user as in the Micro-service architecture.

There is not much description about the Magnum on GitHub.  Better description can be found on OpenStack wiki page for Magnum The OpenStack Magnum wiki page even has a 5 minute video on how Magnum works.  This project on OpenStack wiki is described as:

"OpenStack API service making container orchestration engines such as Docker Swarm and Kubernetes available as first class resources in OpenStack"

The main idea is to take advantage of the existing Docker orchestration engine and make it work on an OpenStack infrastructure.

I will write another blog post to describe Magnum in more detail.  For now we will look at Magnum at a very high level.  As a preview, the architecture of Magnum looks like this:

 

image source: https://wiki.openstack.org/wiki/File:Magnum_architecture.png

With this diagram, we can see the general idea on how Magnum works.  Magnum also takes advantage of the OpenStack Heat project.

This slide from the OpenStack Vancouver summit presentation summarize what is on the OpenStack Magnum wiki page:

image source:http://image.slidesharecdn.com/tuckeropenstacksummitvancouverpossibilities-150525221145-lva1-app6891/95/openstack-in-an-ever-expanding-world-of-possibilities-vancouver-2015-summit-25-638.jpg?cb=1432592195

What is ahead of us?
I see that Project Magnum is the reaction to the modern day business requirement and now with Google being able to inject its valuable knowledge, expertise and deployment experiences to the project, Linux container deployment at scale in OpenStack will become mature in due time. It will also integrate better with the other OpenStack projects such as Swift, Keystone or Neutron that more business use cases can be supported.

Google has over 15 years of deploying application at large scale and will certainly able to bring its experience and expertise to the Magnum project.

The most important question is how fast and what features Project Magnum can produce in a timely manner meaning if the market is ready to embrace what is offered. Sometimes the technology is way more advanced than what the market is ready for. I see in the 2000, there is a start-up already trying to offer SDN but at that time the market is not ready for it and that start-up closed down due to lack of funding around 2003.


Reference:
- "Containers + Private Cloud. Google Sponsors OpenStack Foundation." Google Cloud Platform Blog. N.p., n.d. Web. 17 July 2015. 
- "GoogleCloudPlatform/kubernetes." GitHub. N.p., n.d. Web. 18 July 2015. 
- "Magnum." - OpenStack. N.p., n.d. Web. 18 July 2015.